Summary:
D1720014 line 183 in HHWheelTimer.h broke proxygen/facebook/httpclient clang build:

./folly/io/async/HHWheelTimer.h:183:17: error: declaration shadows a local variable [-Werror,-Wshadow]
Wrapper(F fn) : fn_(std::move(fn)) {}
^
./folly/io/async/HHWheelTimer.h:181:28: note: previous declaration is here
void scheduleTimeoutFn(F fn, std::chrono::milliseconds timeout) {

I renamed the `fn` in `Wrapper` to `f`.

Test Plan: pass

Reviewed By: seanc@fb.com

Subscribers: njormrod, folly-diffs@, subodh, seanc, kmdent, fma

FB internal diff: D1724835

Signature: t1:1724835:1418052896:d9a72230d4fd1a4a734a33142966fe796f4fd362

Summary:
I just found that gcc (4.8.2) failed to unroll the loop in
`pullAtMost()`, so it didn't replace `memcpy` with a simple load
for small `len`.

Test Plan:
fbconfig -r folly/io/test thrift/lib/cpp2/test && fbmake runtests_opt -j32

Ran unicorn-specific thrift deserialization benchmark from
D1724070, verified 50% improvement in `SearchRequest` deserialization
performance.

`thrift/lib/cpp2/test/ProtocolBench` results:

```
|---- before -----| |---- after  -----|
================================================================================================
thrift/lib/cpp2/test/ProtocolBench.cpp          relative  time/iter  iters/s  time/iter  iters/s
================================================================================================
BinaryProtocol_read_Empty                                   21.72ns   46.04M    17.58ns   56.89M
BinaryProtocol_read_SmallInt                                43.03ns   23.24M    23.64ns   42.30M
BinaryProtocol_read_BigInt                                  43.72ns   22.87M    22.03ns   45.38M
BinaryProtocol_read_SmallString                             88.57ns   11.29M    47.01ns   21.27M
BinaryProtocol_read_BigString                              365.76ns    2.73M   323.58ns    3.09M
BinaryProtocol_read_BigBinary                              207.78ns    4.81M   169.09ns    5.91M
BinaryProtocol_read_LargeBinary                            187.81ns    5.32M   172.09ns    5.81M
BinaryProtocol_read_Mixed                                  161.18ns    6.20M    68.41ns   14.62M
BinaryProtocol_read_SmallListInt                           177.32ns    5.64M    96.91ns   10.32M
BinaryProtocol_read_BigListInt                              77.03us   12.98K    15.88us   62.97K
BinaryProtocol_read_BigListMixed                             1.79ms   557.79   923.99us    1.08K
BinaryProtocol_read_LargeListMixed                         195.01ms     5.13   103.78ms     9.64
================================================================================================
```

Reviewed By: soren@fb.com

Subscribers: alandau, bmatheny, mshneer, trunkagent, njormrod, folly-diffs@

FB internal diff: D1724111

Tasks: 5770136

Signature: t1:1724111:1417977810:b7d643d0c819a0bbac77fa0048206153929e50a8

Summary:
Implicitly bouncing between size_t and int for identifiers
causes problems when building on 64-bit iOS. Fix this.

Test Plan: fbmake runtests

Reviewed By: meyering@fb.com, njormrod@fb.com

Subscribers: trunkagent, njormrod, folly-diffs@, fma, kmdent, shikong, pgriess

FB internal diff: D1722061

Signature: t1:1722061:1417819039:8e3938cf8d4d241551ed3dd3978c1b11f57398c5

Summary:
This reverts commit 7871df125fc1fbe5d8030a41e21b8241b25d4e62.

As explained in t5763478, I don't see good reason for a revert.
Also the errors under question shouldn't show up after D1720841.

Test Plan: revert-hammer

Reviewed By: stepan@fb.com

Subscribers: njormrod, folly-diffs@

FB internal diff: D1722928

Tasks: 5763478

Signature: t1:1722928:1417818663:1fe402d14c0cf389fb3611355db270c5f2c33738

Summary:
This was supposed to be an update to D1720014 but I screwed up with arc.

Test Plan: run the test by hand and see the output

Reviewed By: jsedgwick@fb.com

Subscribers: exa, njormrod, folly-diffs@

FB internal diff: D1721951

Signature: t1:1721951:1417801009:72a27ec2ca473a996785ff9cae48bd51a43d9045

Summary:
codemod to replace all `makeFuture().via` with just `via`.

Test Plan: contbuild

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, zeus-diffs@, targeting-diff-backend@, abirchall, fugalh, msk, exa, benj, njormrod, folly-diffs@

FB internal diff: D1715876

Signature: t1:1715876:1417712302:b882916d394f90caa23bd73fa68b8f786af649e3

Summary:
Better support and test chaining of `via` and `activate`/`deactivate`.

The real problem is that without the ref qualifier a multiple chain of lvalue references can end up with a destructed object being referenced after.

https://akrzemi1.wordpress.com/2014/06/02/ref-qualifiers/

Test Plan:
This is mostly new tests that would fail and now pass.
I think maybe the tests are a bit weak, it would be good to find a way to ensure we aren't going to see the access-after-free bugs in these tests.

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, fugalh, exa, njormrod, folly-diffs@

FB internal diff: D1714873

Tasks: 5489801

Signature: t1:1714873:1417628538:9e610c5ba5e0a22c19a11d53aa956be45d585058

Summary: dalek-exterminate

Test Plan:
Moved `Later` tests to `via` tests.
fbgs
contbuild

Reviewed By: jsedgwick@fb.com

Subscribers: fbcode-common-diffs@, adityab, lins, trunkagent, fugalh, exa, njormrod, folly-diffs@, hannesr

FB internal diff: D1714862

Tasks: 5409538

Signature: t1:1714862:1417621949:f63f49e1093a021170d2346e8e673db042d2bc56

Summary:
cpp11ify

I'm gonna use this for Wangle timeouts.

Test Plan:
new unit test
_bin/folly/io/async/test/HHWheelTimerTest

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, exa, njormrod, folly-diffs@

FB internal diff: D1720014

Tasks: 5002969

Signature: t1:1720014:1417732753:ddcad431b6c305c9ba27f6b9e3454ab158c13f4b

Summary: This reverts commit 6584412293cc5b9aad2004e99d550e3478df1e5d.

Test Plan: thisisrevert

Reviewed By: smarlow@fb.com

Subscribers: njormrod, folly-diffs@

FB internal diff: D1721739

Signature: t1:1721739:1417786147:01aaaf50b679da7942b2a30ad039eb0ac1031430

Summary:
SSLSessionCacheManager uses glfags but doesn't include it. This
breaks on some systems.

Test Plan: fbconfig -r folly && fbmake runtests

Reviewed By: robbert@fb.com

Subscribers: trunkagent, sdwilsh, fugalh, ssl-diffs@, njormrod, folly-diffs@

FB internal diff: D1716997

Signature: t1:1716997:1417720032:548c2ad8f54e41d42585c331324f2065733595c1

Summary: Schedule destroyInstances to be executed via atexit in registrationComplete. registrationComplete is called from main(), so this makes sure folly::Singleton's will be destoyed before all singletons constructed before main() (which should cover all third-party libraries' singletons).

Test Plan: unit test

Reviewed By: chip@fb.com

Subscribers: trunkagent, njormrod, folly-diffs@

FB internal diff: D1717963

Tasks: 5666654

Signature: t1:1717963:1417659478:c50c271d1786be75499565f6ab9c0a1a6f6f347d

Summary:
The getsockopt's last parameter (optlen) is a value-result parameter
and AsyncSocket::getSockOpt was passing a value as argument.

Test Plan: Unit tests

Reviewed By: alikhtarov@fb.com

Subscribers: trunkagent, njormrod, folly-diffs@

FB internal diff: D1717463

Tasks: 4867290

Signature: t1:1717463:1417664828:6c7a74ff31725121f892ce1adba2653e70728192

Summary:
... and employ it in EventBase's destructor to stop leaking memory from unexecuted NotificationQueue-variety runInEventBaseThread() callbacks

in addition to the attached task, this should also fix the root cause of the already-worked-around #5564342

Test Plan:
test no longer leaks.* on the other hand, IOThreadPoolExecutor::stop() no longer actually stops, it joins. I added comments to stop() in headers indicating that it should be treated as best-effort.

* hilariously, this actually isn't true - there's a new leak of a thread local inside the io pool because callbacks can outlive the pool now. i'll leave a full explanation for the upcoming patch.

Reviewed By: davejwatson@fb.com

Subscribers: alandau, bmatheny, mshneer, trunkagent, fugalh, njormrod, folly-diffs@

FB internal diff: D1682860

Tasks: 5336655

Signature: t1:1682860:1416347774:ac5f31fb72373992f425c93ac284a0cd27608db4

Summary:
s/`sendAsyncLater`/`sendAsyncVia`/ and change it to return a cold future with equivalent semantics. We have to call `via` twice in the implementation—once to gate the result and again to make sure the caller will get a future that executes in that executor's context also.

NB this is a slight semantic change - if the executor is, say, a threadpool then now it will go through the threadpool queue (and maybe execute in two different threads) whereas before the whole later chain would execute in the same thread. I don't *think* this is a problem, but something to think about.

Test Plan:
stuff we fbconfig'd builds
contbuild
unit tests
thinking really hard

Reviewed By: hannesr@fb.com

Subscribers: trunkagent, fbcode-common-diffs@, net-systems@, ldbrandy, hannesr, fugalh, zhuohuang, exa, watashi, smarlow, akr, bnitka, jcoens, darshan, njormrod, anfarmer, folly-diffs@

FB internal diff: D1644012

Tasks: 5409538

Signature: t1:1644012:1417625401:99b1b7df6de4cfcdd945eed7104d4c82e8c0b78f

Summary:
Benchmark.cpp code is dependent on initialization order, which leads
to benchmarks not running. The root cause is that on some linkers the code which
adds benchmarks to the benchmarks vector is run before the vector initialization.

The original bug report: https://github.com/facebook/folly/issues/18
See: http://www.parashift.com/c++-faq-lite/static-init-order.html

Test Plan: run benchmark_test and compare result with master output

Reviewed By: andrei.alexandrescu@fb.com, njormrod@fb.com

Subscribers: trunkagent, sdwilsh, folly-diffs@

FB internal diff: D1710588

Signature: t1:1710588:1417468175:fd2705a573cef5c4ff020b60c6aec3d83bcdbbab

Summary:
Description:
Last time a merge error broke stuff, fixing and readding

Test Plan:
Ran unit test

This reverts commit 1075ab9d6f62cad36a228bd3fc7203a75fdf5baf.

Reviewed By: davejwatson@fb.com

Subscribers: njormrod, folly-diffs@, wstefancik

FB internal diff: D1695920

Signature: t1:1695920:1416529609:3f726ac5df4e33f254075738ea6655a6fc01bb7f

Summary: folly::dynamic::insert is broken. When we try to insert an rvalue and the key already exists in dynamic, insert will replace the value with empty dynamic.

Test Plan: unit test attached

Reviewed By: stepan@fb.com

Subscribers: trunkagent, njormrod, folly-diffs@

FB internal diff: D1704995

Signature: t1:1704995:1417053631:8e0163693df721c54b016f8c27e81e7bf60194cb

Summary: This is not a super clean solution, given that I have just copied the code over form Varint.h. Another way would be to add a peek(IOBuf&, size_t) method to CursorBase and use that to peek deeper into the chain.

Test Plan: unit tests

Reviewed By: tulloch@fb.com

Subscribers: trunkagent, njormrod, dcapel, benr, folly-diffs@

FB internal diff: D1689872

Signature: t1:1689872:1416964989:b7f12a2686233f161401288ffcb8c51926b01fdf

Summary:
IOBufs now support larger than 4GiB chunks. One of the checks
was not removed. Tracing through the capacity, its use if all 64-bit
compatible.

Test Plan: moveToFbString a 4GiB+ IOBuf element.

Reviewed By: maxim@fb.com

Subscribers: net-systems@, sdwilsh, njormrod, folly-diffs@

FB internal diff: D1705316

Tasks: 5648445

Signature: t1:1705316:1416965241:d9d064bbde53e033d32bfe872b3445ee59005f5e

Summary:
If a single IOBuf has size exceeding 2^32, then our zlib
compression algorithm fails. Specifically, zlib z_stream.avail_in is
only 32 bytes (I think it's a long?
http://www.gzip.org/zlib/zlib_faq.html#faq32), and so a too-big IOBuf
will overflow the z_stream and cause data loss.

This diff breaks up large IOBufs into smaller chunks.

Test Plan:
fbconfig -r folly && fbmake runtests

Also compressed biggrep's configerator blob, which is how this bug was
caught. It now works. See the associated task.

Reviewed By: robbert@fb.com

Subscribers: trunkagent, sdwilsh, njormrod, folly-diffs@

FB internal diff: D1702925

Tasks: 5648445

Signature: t1:1702925:1416958232:459d498ff1db13e1a20766855e6f2f97da8cde8c

Summary:
Some platforms that SSLContext run on call exit() despite best practices.
This cause the static structures to be destructed and cause race conditions
and crashes.

This new's the static structures so that they won't be destructed on exit()

Test Plan: Unit tests

Reviewed By: seanc@fb.com

Subscribers: trunkagent, ssl-diffs@, seanc, njormrod, folly-diffs@

FB internal diff: D1702186

Tasks: 5572637

Signature: t1:1702186:1416941649:c5bdfd8cc604fc3618f51bdb8b910b5b2cf350ad

Summary: also kill duplicated test file

Test Plan: run all exp/wangle tests

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, njormrod, folly-diffs@

FB internal diff: D1698503

Signature: t1:1698503:1416612139:7139c6aa2ca79eabdaec6c33192194df984af348

Summary: bad merge with D1661169 caused a unittest to fail.  Update unittest

Test Plan: fbconfig -r folly/experimental/wangle; fbmake runtests

Reviewed By: jsedgwick@fb.com

Subscribers: doug, fugalh, njormrod, folly-diffs@

FB internal diff: D1698435

Signature: t1:1698435:1416611458:555b6b50baa1bc5850b10e19c9c94978f93f2a5a

Summary: As above. This paid off with a couple bugfixes.

Test Plan: run em all

Reviewed By: hans@fb.com

Subscribers: fugalh, njormrod, folly-diffs@

FB internal diff: D1695106

Signature: t1:1695106:1416522038:0c3345aadf954bf346d35b99877e7f8dfcf3ceff

Summary:
mostly copypasta from TAsyncTransportHandler

Test Plan: compiles

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, fugalh, njormrod, folly-diffs@

FB internal diff: D1690973

Signature: t1:1690973:1416529528:4feb187a68ad5405662b9b0efb160edd253a2977

Summary:
MLE: my first custom matcher

revealed some include issues, also fixed

Test Plan: guess

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, fugalh, njormrod, folly-diffs@

FB internal diff: D1686223

Signature: t1:1686223:1416334120:41bce88d56dd9ca107f3ddc3927c6be9419370cf

Summary:
Basically the same interface as before, but you must specify the read and write types for the ends of pipeline. Implementation is cleaner as well; there's fewer levels of indirection

This dynamic casts shit all over the place and is less typesafe then the previous iteration, but I think with some carefully placed static_asserts, could be just as safe (in the case where you don't do any modification, anyway)

Right now you can only add to the front or back of the pipeline but the way it's set up you could add any number of mutations, including ones that are triggered by handlers. But this should (might?) be enough for Tunnel, which was the motivation.

Test Plan: basic test compiles, thrift2 diff still works with a one line change

Reviewed By: hans@fb.com

Subscribers: trunkagent, fugalh, njormrod, folly-diffs@, bmatheny

FB internal diff: D1661169

Tasks: 5002299

Signature: t1:1661169:1416521727:1f126279796c0b09d1905b9f7dbc48a9e5540271

Summary: Now a subprocess can be reliably made a group leader -- good for job control.

Test Plan:
unit test, checked that the pgid test worked in bash, too (only OKAY_XXX is printed)

==> XXX.sh <==
#!/bin/sh
test $(cut -d ' ' -f 5 /proc/$$/stat) == $$ && echo OKAY_XXX
./YYY.sh

==> YYY.sh <==
#!/bin/sh
test $(cut -d ' ' -f 5 /proc/$$/stat) == $$ && echo OKAY_YYY
./ZZZ.sh

==> ZZZ.sh <==
#!/bin/sh
test $(cut -d ' ' -f 5 /proc/$$/stat) == $$ && echo OKAY_ZZZ

Reviewed By: simpkins@fb.com

Subscribers: net-systems@, njormrod, folly-diffs@, simpkins

FB internal diff: D1492526

Signature: t1:1492526:1416528620:3cf98b1c1e334a7d551b2c2f3e76b1c70f07ad7c

Summary: Diff that had @override-unit-failures broke the build

Test Plan: fbconfig -r folly && fbmake dbg && fbmake runtests

Reviewed By: robbert@fb.com

Subscribers: sdwilsh, njormrod, folly-diffs@

FB internal diff: D1694388

Tasks: 5662947

Signature: t1:1694388:1416510278:d50f4896814a34d73d55be292961f5ecd74b1bb4

Blame Revision: D1680735

Summary: someone made another event base test while the big set still lived in thrift. merge it now that the old tests have been moved.

Test Plan: fbconfig folly/io/async/test && fbmake runtests --extended-tests

Reviewed By: davejwatson@fb.com

Subscribers: njormrod, folly-diffs@

FB internal diff: D1690764

Signature: t1:1690764:1416413748:f3ae64d509432844bb2e1b689a75d66825269d15

Summary: Documentation brain-dump.  The sources already have pretty good doxygen-style comments, but could use a good high-level overview.

Test Plan: Reviewers get to edit

Reviewed By: jsedgwick@fb.com, afrind@fb.com, dcsommer@fb.com, alandau@fb.com

Subscribers: fugalh, subodh, kmdent, seanc, vloh, sjenkins, kparton, doug, net-systems@, njormrod, folly-diffs@, sgolemon, alikhtarov, andrii, tudorb

FB internal diff: D1690976

Tasks: 5648428

Signature: t1:1690976:1416419706:3894976c32904f040d518b9a4ce26ccb40d7ed80

Summary:
This one is a little tricky, but it explains what happened in t5388156. Basically what we observed in the stack traces is that the HHWheelTimer count_ is 0, but there is still a timeout scheduled on the AsyncTimeout.

How I can see this happening is the following:
- SASL times out
- We fall back to insecure mode in the callback and schedule a send of a real message to the server. Here we schedule another timeout on the same HHWheelTimer object. Since count_ is 0 at this point, we schedule a timeout in AsyncTimeout.
- Now the initial SASL timeout was late, so the HHWheelTimer is in a catchup state. This means that it's possible for the callback on the timeout that was just scheduled to fire on the same "tick".
- This callback invokes SR's callbacks which try to detach the event base from the channel. But it can't detach because we have something scheduled on the AsyncTimeout.
- We crash.

Test Plan:
- will wait for contbuild
- Something like this repros the problem: https://phabricator.fb.com/P17220063. Note I hacked the catchup part to get it to work. Now running loadgen with security = permitted causes the crash.

// hphp tests are stalled, have to do this...

Reviewed By: mshneer@fb.com

Subscribers: bmatheny, mshneer, sandeepkk, trunkagent, njormrod, folly-diffs@, alandau, mhorowitz, srenfro, hitesh, wstefancik

FB internal diff: D1680735

Tasks: 5388156

Signature: t1:1680735:1416252123:e76668860ccda9380a87996b4fa3de957e129404

Summary:
All pretty trivial:

https://github.com/facebook/folly/issues/99
https://github.com/facebook/folly/issues/98
https://github.com/facebook/folly/issues/45

Test Plan: Will watch jenkins fbthrift build

Reviewed By: dcsommer@fb.com

Subscribers: doug, njormrod, folly-diffs@

FB internal diff: D1669952

Signature: t1:1669952:1415643677:906234f0a89f38645b0072d3c88762d8fa2729dc

Summary:
ServerBootstrap a la netty.

This should be enough for some refactoring of thrift server and proxygen servers - but there are still lots of TODOs left to do

Test Plan:
Unittests included

Depends on D1638358

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, doug, fugalh, alandau, bmatheny, mshneer, jsedgwick, afrind, dcsommer

FB internal diff: D1649521

Tasks: 5488516

Signature: t1:1649521:1416256073:fc003fd471bdfd137160dd6d7befd933ee8addd2

Summary: Fix the TARGETS from the refactor

Test Plan: ran it locally

Reviewed By: davejwatson@fb.com

Subscribers: doug, njormrod, folly-diffs@

FB internal diff: D1689151

Signature: t1:1689151:1416353673:3d3575b10f963e9f558954b8f3eba8bf48cdaa74

Summary:
getLoad() is called in servicerouter loadbalancing when a load_counter is not specified.  I added a custom getLoad() for leaf and made it so that different metrics can be tested without code pushes.

Changes:
1) Overloaded PriorityThreadManager::getCodel() so you can specify a PRIORITY.  aggr->leaf thrift calls are marked as HIGH priority.
2) The value of FLAGS_load_metric determines what metric the custom getLoad() will use (codel, max requests, or fallback to ThriftServer::getLoad()).
3) Smc listener for FLAGS_load_metric.
4) Removed server->setIsUnevenLoad(false) since there isn't reason this need to be set, and activeRequests_ and getPendingCount() are not tracked when isUnevenLoad_ is false.

Test Plan: Run leaf and check that smc listener works.  Custom getLoad() won't be called until t5625774 is resolved.

Reviewed By: hitesh@fb.com

Subscribers: njormrod, folly-diffs@, trunkagent, varunk, fugalh, alandau, nkgupta, bmatheny, jaredg, xie, mshneer, yguo

FB internal diff: D1685215

Signature: t1:1685215:1416339297:0d1f8cb0cec27242d95f0067337d80149b0a720e

Summary: Clean up API per stepan's comments in D1641057: runBeforeLoop callbacks should be run before the if block, so runInLoop calls will work.

Test Plan: added unittest

Reviewed By: hans@fb.com

Subscribers: trunkagent, doug, alandau, bmatheny, njormrod, mshneer, folly-diffs@

FB internal diff: D1679773

Signature: t1:1679773:1415916488:f20dd98bf40dd4cb19a4b11875dc0c9fa3e7870e

Blame Revision: D1641057