Summary:
There was a race reading `callback_` in `maybeCallback` and setting `callback_` in `setCallback`. This diff reworks the state machine to make this unpossible. To avoid the explosion of states due to the cross-product of has-interrupt-handler/has-been-interrupted/etc. I introduce a separate lock for setting interrupt handler and interruption, since this is primarily orthogonal. Other attributes (active, for example) are still atomic variables, and while somewhat tied into the state machine logically (e.g. transitioning from Armed to Done only happens when active) they are mostly independent, keeping the state machine simple (and probably faster).

I think it may even be possible to do some things cheaper. In some states, we may not need to protect the writing of `callback_` and `result_`. But we'd need to enforce some ordering so I'm not going to try to tackle that. But that could be some speedup if we can do it cheaply.

Test Plan:
Builds and all existing tests pass.

Reviewed By: rockyliu4@fb.com

Subscribers: yfeldblum, stepan, trunkagent, exa, folly-diffs@, jsedgwick

FB internal diff: D1807854

Tasks: 6087856

Signature: t1:1807854:1422656713:25b62706cd7952b2dde06dab08074f8030db456b

Summary:
Add a description of the fault-after-unlock problem that can
occur when a synchronization object is used to guard its own
destruction.

Test Plan: comment change only

Reviewed By: alikhtarov@fb.com

Subscribers: trunkagent, folly-diffs@

FB internal diff: D1808446

Signature: t1:1808446:1422481173:83c0369e2277cd5b8cf6377fcd92b132019611a7

Summary: That is, for all inputs except r-values.

Test Plan:
Run tests
@override-test-failures

Reviewed By: ajaymenon@fb.com

Subscribers: trunkagent, folly-diffs@

FB internal diff: D1807789

Signature: t1:1807789:1422410689:442d50ab17f8e3b08e34f33318a4dc7f23b4c6cc

Summary:
This speeds up TLS handshake and might be a factor of liger perf regression.
The enabling is guarded by an #ifdef. The condition itself is defined in an
openssl patch.

Test Plan: folly unit tests pass. Tried on devices as well and from tcpdump data was sent before the new ticket was received, essentially speeded up the handshake process.

Reviewed By: subodh@fb.com

Subscribers: trunkagent, kmdent, seanc, benyluo, ssl-diffs@, ranjeeth, folly-diffs@

FB internal diff: D1806856

Tasks: 5284979

Signature: t1:1806856:1422494521:0a048ea9001da13b5d698b5a764d1e66dcbedc99

Test Plan: Unit tests

Reviewed By: ajaymenon@fb.com

Subscribers: trunkagent, folly-diffs@

FB internal diff: D1808023

Signature: t1:1808023:1422410709:8db2d73d5b4c0c2eab563643e5fa1557ebfd4730

Summary:
Included climits and limits.h due to non-conflicting changes,
clean this up.

Test Plan: compiled

Reviewed By: kelarini@fb.com

Subscribers: trunkagent, folly-diffs@

FB internal diff: D1810133

Signature: t1:1810133:1422483209:0826d2198769efed86607de6c5b28ce0bc4ac04b

Summary: should have been using fulfilTry anyways. and fulfilTry needs to take the Try by value so fix that too

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: fugalh, folly-diffs@, jsedgwick

FB internal diff: D1808923

Tasks: 6098987

Signature: t1:1808923:1422478981:d4b9727394339c996ebccb7841b94e0c7b2bffb4

Summary: Now CHAR_BIT will be found

Test Plan: ran multifeed and folly

Reviewed By: seanc@fb.com

Subscribers: seanc, folly-diffs@

FB internal diff: D1810027

Signature: t1:1810027:1422480946:4568c5190e318ff097d33b3d0434593b72f8205e

Summary: ^

Test Plan: fbmake

Reviewed By: woo@fb.com

Subscribers: folly-diffs@

FB internal diff: D1810061

Signature: t1:1810061:1422481560:089c9186834f2bf87ac6d10b71d53784299b7992

Blame Revision: D1806632

Summary:
We do a lot of copying of TransportInfo in proxygen, and in most cases
the SSL structs don't change after connection establishment. We could
cut down on memory usage by sharing these huge strings. This is
especially true with SPDY where all streams belonging to a session could
share these fields.

Facebook:

Test Plan: Unit tests, will canary

Reviewed By: afrind@fb.com

Subscribers: fugalh, bmatheny, ssl-diffs@, folly-diffs@, jsedgwick, woo

FB internal diff: D1807557

Tasks: 5343753

Signature: t1:1807557:1422472932:53038345fca620632097586fb9e410bca8fe748d

Summary: Normally I would use the folly::to<> function, but it would cause a circular dependency, so I just added a static cast

Test Plan: Ran on iOS and ran the folly tests

Reviewed By: seanc@fb.com

Subscribers: lucian, mpawlowski, marcelo, tudorb, aalexandre, seanc, folly-diffs@

FB internal diff: D1806632

Signature: t1:1806632:1422416646:b8104f18f90eb7457f2f358428f2bd800f8f1db5

Summary: as above, there were less of these than I expected, nice

Test Plan: unit

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick

FB internal diff: D1789332

Tasks: 5949939

Signature: t1:1789332:1421878033:d7c2979a77b51a5257b8bcd910ad9296ca1aa7e0

Summary:
std::to_string doesn't exist on Android so don't use it.

Facebook: Did a sync to fbandroid and confirmed liger compiles with this fix.

Test Plan: existing tests

Reviewed By: ranjeeth@fb.com

Subscribers: trunkagent, folly-diffs@, shikong, kmdent, fma, pgriess

FB internal diff: D1808037

Signature: t1:1808037:1422410556:d78e0633a1554254b1a1f25bef49a4550a1817c6

Summary:
See D1785572. Check out the new Future test and the commented portion of wait(Duration) for the fix

The test only fails a few times out of a hundred before the fix, but hasn't failed yet after

Test Plan: futures unit, wait for contbuild

Reviewed By: hans@fb.com

Subscribers: trunkagent, rushix, fbcode-common-diffs@, hero-diffs@, cold-storage-diffs@, adamsyta, zhuohuang, darshan, micha, folly-diffs@, lins, tingy, hannesr, jsedgwick

FB internal diff: D1803526

Tasks: 5940008, 6059995

Signature: t1:1803526:1422309486:3613c59a708ecac312d241723828763feb2a57aa

Summary: Reciprocal of readFile

Test Plan: fbmake runtests

Reviewed By: tudorb@fb.com

Subscribers: trunkagent, net-systems@, folly-diffs@

FB internal diff: D1807551

Signature: t1:1807551:1422410565:f5bda294deb788da9f3881c19bb20cfa1f588c09

Summary:
Fixing following the issues raised by nsuke on github
following his diff (https://github.com/facebook/folly/pull/112)

Test Plan: visual inspection, tested build on arch linux

Reviewed By: davejwatson@fb.com

Subscribers: folly-diffs@

FB internal diff: D1807378

Tasks: 5909189

Signature: t1:1807378:1422403093:ca024f396a27ecd9b304c57643f0107764a38ca1

Summary: no need to be mutable, since it only calls AsyncUDPSocket::getFD() which is a const method

Test Plan: just compiled -- simple change

Reviewed By: afrind@fb.com

Subscribers: trunkagent, doug, folly-diffs@

FB internal diff: D1802765

Signature: t1:1802765:1422307972:eab6ab4b16407df6e5034cef78c79f45cd9fa46f

Summary:
Make python scripts compatible woth both python 3 and python 2.
Added ignore rule for test/glog directory.

Facebook:
While verifying some issues reported on github i noticed that
folly does not build when python 3 is the default (at least on arch
linux). As it turns out it is fairly easy to make our python files
compatible with both python 2 and python 3.
I also added test/glog*/* to gitignore so that it is easier to work
on folly outside of facebook.

Test Plan:
unit tests in fb envirnoment and unit test on arch linux
(python 3.4.2, gcc 4.9.2)

Reviewed By: njormrod@fb.com

Subscribers: trunkagent, folly-diffs@

FB internal diff: D1801368

Signature: t1:1801368:1422294777:4e7b6b5634e8dccd849194e171c9a342bd1cb8b8

Summary: print the value which failed conversion to aid debugging.

Test Plan: run code which throws in to<> and contbuild

Reviewed By: tudorb@fb.com, andrei.alexandrescu@fb.com

Subscribers: trunkagent, folly-diffs@

FB internal diff: D1786294

Signature: t1:1786294:1421795912:00cc10923990e5aac0a15eedec09deb8e8d470d1

Summary: All is good now.

Test Plan: Run all the tests.

Reviewed By: hans@fb.com

Subscribers: jsedgwick, trunkagent, fugalh, folly-diffs@

FB internal diff: D1758272

Signature: t1:1758272:1421889405:a015d30783715e106a1e6667e971f7cf47c8392d

Summary: Not sure why we'd need that? This avoids a move, I think.

Test Plan:
Run all the tests. Also unbreaks this code, which used to work at some
point.
https://phabricator.fb.com/diffusion/FBCODE/browse/master/taoThriftService/TaoServiceHandler.cpp;a27bc2ef36cd671d65ae0fd2cc1fb1f823e68ae4$246

Reviewed By: hans@fb.com

Subscribers: trunkagent, folly-diffs@, jsedgwick

FB internal diff: D1800619

Signature: t1:1800619:1422053903:f9e65a0be3d820a9a9989b3cf5dfaf2a61e2e900

Test Plan: g-unittest

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@

FB internal diff: D1801665

Signature: t1:1801665:1422093104:eecf86ea8c1ab39dcb3d01442ed66ae45ef3bede

Summary: I'm in unfamiliar territory, so shout if I'm doing something dumb.  Perhaps it's a bad assumption that if the ipv4 port is free that the ipv6 port is also free?

Test Plan: g-unittest

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, ps, bmatheny, folly-diffs@

FB internal diff: D1795120

Signature: t1:1795120:1422034693:bd315023ab6cd9e9bda12161d05dd781dc401546

Summary: Now all connections are always scheduled with connectionManager's default timeout.  This diff enables us to specify the different timeouts for each managed connections.

Test Plan: prospect testing

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, fugalh, folly-diffs@, jsedgwick

FB internal diff: D1797193

Tasks: 5343753

Signature: t1:1797193:1422034092:ed67b96efe8af8f8d1355d3f86fb1289daafb178

Summary: Future service interfaces similar to finagle.  Service creators for client, filters

Test Plan: Unittests included - also sets up a simple pipeline to test a full stack client/server.

Reviewed By: hans@fb.com

Subscribers: jsedgwick, trunkagent, njormrod, folly-diffs@, doug, fugalh

FB internal diff: D1573086

Tasks: 5002456

Signature: t1:1573086:1421970698:328453c4a980bb6950fc9aeed6a2b6d9819c20db

Summary:
Subject:

This reverts commit d8eeb9a12f932b74d268c74ce7ce2610dc14b152.

Test Plan: Watch contbuild?

Reviewed By: njormrod@fb.com

Subscribers: trunkagent, doug, fbcode-common-diffs@, hero-diffs@, cold-storage-diffs@, adamsyta, zhuohuang, darshan, micha, folly-diffs@, lins, tingy, hannesr, jsedgwick

FB internal diff: D1797163

Tasks: 5940008

Signature: t1:1797163:1421959040:c9d528367fed2de2f7dafdac9416aa01bdb47df9

Summary:
This also adds a dependency from all folly::Singletons on RequestContext. Better fix would probaly be to switch RequestContext to be managed by folly::Singleton, however that would require us to switch all singletons which use it to be folly::Singletons too.

Facebook:
Making sure we initialize SR singletons when OBCClient singleton is created, so that SR is destoyed after OBCClient.

Test Plan:
fbconfig -r --sanitize=address --platform-all=gcc-4.8.1-glibc-2.17 dragon/
fbmake runtests

Reviewed By: meyering@fb.com, davejwatson@fb.com

Subscribers: trunkagent, folly-diffs@, meyering

FB internal diff: D1796847

Tasks: 5986816

Signature: t1:1796847:1421954531:e9ec177d5289a33c12fe0947426d89963c8e262c

Summary:
Histogram::getPercentileBucketIdx() and getPercentileEstimate() are
implemented using template methods of HistogramBuckets.

This updates Instantiations.cpp to also instantiate the versions of
these template methods that are required by Histogram<int64_t>.  Without
this, anyone using these methods was required to also include
Histogram-defs.h, or otherwise they may get link errors.  This makes it
so they no longer need to include Histogram-defs.h

The C++ syntax required is pretty horrendous.  We could potentially
avoid this by refactoring the code so that these methods are no longer
templates (and require that the Bucket class provide methods for this
purpose).  At the moment adding instantiations seems simpler, and
doesn't change any public APIs.

Test Plan:
Successfully linked code using
Histogram<int64_t>::getPercentileEstimate() without including
Histogram-defs.h

Reviewed By: ldbrandy@fb.com, meyering@fb.com

Subscribers: trunkagent, doug, net-systems@, exa, folly-diffs@

FB internal diff: D1786346

Signature: t1:1786346:1421429734:da17b7ea326c64a1e158fec3b87dc49c3db3d848

Summary:
This splits out some code from the existing getIov() method, to support
appending to an existing iovec array, rather than always returning a new
one.

I have a use case where I have multiple IOBuf chains to write at once,
preceded by some header data.  appendToIov() makes it easier to populate
a single iovec array with this data, and then perform a single writev()
call.

Test Plan: Updated the unit tests to exercise appendToIov()

Reviewed By: jasmeetbagga@fb.com

Subscribers: trunkagent, doug, exa, net-systems@, folly-diffs@

FB internal diff: D1783090

Tasks:

Signature: t1:1783090:1421450476:d5c89d6e1adbd2f2239057238e020b302df09f23

Blame Revision:

Summary:
Introduce ProgressableExecutor, which is an Executor that can be driven somehow. Examples include EventBase and ManualExecutor
Then introduce Future<T>::getVia(ProgressableExecutor*) and Future<T>::waitVia(ProgressableExecutor*) that drive the given executor until the Future is complete, with the usual semantics of get and wait respectively
This is a really common pattern in tests and you can see in the various changes to other projects lends sopme nice redness and cleanliness

Some notes:
1. I don't like ProgressableExecutor::makeProgress() that much. Too verbose. Maybe DrivableExecutor::drive()? Something else? Thoughts?
2. I still need to integrate this with some stuff in Zeus (SessionFuture) and Zookeeper (ZookeeperFuture) but I'm going to do that in a separate diff because it's going to be a little more intrusive
3. These APIs take a raw ptr so that they are consistent with via()
4. See inline note on ManualExecutor
5. See inline note in added unit tests

Test Plan: add unit for new API, wait for contbuild

Reviewed By: hans@fb.com

Subscribers: trunkagent, dresende, pzq, tdimson, fbcode-common-diffs@, targeting-diff-backend@, alandau, apollo-diffs@, bmatheny, everstore-dev@, zhuohuang, laser-diffs@, mshneer, folly-diffs@, hannesr, jsedgwick

FB internal diff: D1789122

Tasks: 5940008

Signature: t1:1789122:1421868315:6ea2fc2702be1dc283c24a46d345fb5da3935b32

Summary: as above. nice redness. couple of comments inline though

Test Plan: contbuild

Reviewed By: hans@fb.com

Subscribers: trunkagent, zeus-diffs@, cold-storage-diffs@, targeting-diff-backend@, apollo-diffs@, micha, folly-diffs@, jsedgwick

FB internal diff: D1787776

Tasks: 5940008

Signature: t1:1787776:1421776843:13772348538a0af27ceb5a363b818ece0cdfffc6

Summary:
see task. also adjust all callsites to get() or wait() as appropriate. this is a lot better, especially in various tests

Facebook:

cc @rushix, @wez as this was quite nice for docstore and novak, and you might want to be aware of new conventions

Test Plan: futures unit, wait for contbuild

Reviewed By: hans@fb.com

Subscribers: trunkagent, fbcode-common-diffs@, hero-diffs@, cold-storage-diffs@, adamsyta, zhuohuang, darshan, micha, folly-diffs@, lins, tingy, hannesr, jsedgwick, wez, rushix

FB internal diff: D1785572

Tasks: 5940008

Signature: t1:1785572:1421866794:a879de4d0bc14e96c434f623ed2a74361e25f28c

Summary:
Start of codec framework.  Copied the frame based codecs almost exactly from netty, but made to fit our pipeline management.

BytesToMessageCodec is slightly different: Netty preprocesses all the available data to a List<Message>, while this codec a) Only does one message at a time to avoid queueing issues, and b) doesn't template the message type, and just passes an IOBuf

I'm fighting the type system to get the pipelines to play nice to each other:  I'd rather template the message type, but it prevents stacking outgoing handlers on top of it, since you ahve to specify both the input/output type for each handler, even if you only care about the output type.  Suggestions to fix?   Netty gets around this by lots of dynamic casting to Object type, but we can't do that in C++ since we don't have a base object type

Test Plan: Includes lots of tests

Reviewed By: jsedgwick@fb.com

Subscribers: jsedgwick, doug, fugalh, folly-diffs@

FB internal diff: D1758189

Tasks: 5002361, 5002316

Signature: t1:1758189:1421170225:6bc8cc6a0bb461a965665bc88f7009033b215da9

Test Plan:
fbconfig folly/experimental/test:singletonvault_c_test && fbmake runtests
buck test folly/experimental/test:singletonvault_c_test

Reviewed By: andrii@fb.com

Subscribers: trunkagent, sdwilsh, folly-diffs@

FB internal diff: D1780054

Signature: t1:1780054:1421187023:b18c1d5575b2864b3d3df001a0024b91eb321c46

Summary:
IdleTime unit test has been failing randomly (#5996886). The
event base loop calculates an exponentially moving average of loop busy
time in each loop. In the test, the busy time is initialized to 5900 and
each loop runs more than 8000 us. In normal case decaying of the
previous loop busy time will take 6 loops to  move this average to >6000.
But if the test is running on a heavily loaded machine the busy+idle time
a loop is longer than it should be, causing the decaying happen faster.
This diff skips this test if this happens.

Test Plan:
It won't affect anything if running on my devserver. It only
helps on a heavily loaded contbuild host.

Reviewed By: alandau@fb.com

Subscribers: folly-diffs@

FB internal diff: D1788450

Tasks: 5996886

Signature: t1:1788450:1421460774:1ec575c50f881e10b5a0208717fe68164f0d0f57

Test Plan: checked this is the right function name

Reviewed By: ngbronson@fb.com

Subscribers: folly-diffs@

FB internal diff: D1788386

Signature: t1:1788386:1421453582:82d0c4685c1bd2d888f5974fd2a998ebbc27a9b5

Summary:
The big comment block at the top of MPMCQueue had a non-sense
sentence resulting from a merge conflict, and also didn't include
details about ordering or recommended patterns for handling consumer
shutdown.

Test Plan: comment change

Reviewed By: lesha@fb.com

Subscribers: folly-diffs@

FB internal diff: D1788230

Signature: t1:1788230:1421451643:7c768c2b127499e3f2b06ee241fba851b4ed2a29

Summary:
We want all socket options to be applied to the listen sockets too.

The old code here that only applied SO_KEEPALIVE is a bit superstitious.
Not all options will be inherited by accept()ed sockets, but that's
fine.  We set them again post-accept().

Applying all these options to the listening sockets will fix at least
one long standing bug, which is that QoS is not set correctly for the
first few packets in a flow.  This also will let us do the right thing
when we gain the ability to set TCP_MAXSEG in D1741753.

Test Plan: fbmake runtests & canary in prod

Reviewed By: afrind@fb.com, davejwatson@fb.com

Subscribers: jsedgwick, satadru, trunkagent, fugalh, exa, folly-diffs@, agartrell

FB internal diff: D1745182

Tasks: 2911597

Signature: t1:1745182:1418860157:045680a91b153482bcd4a014894fb28059955d06

Summary:
My initial change to avoid the -Wsign-compare error neglected
the fact that the difference of two unsigned types is also
unsigned. That broke some unicorn tests, as seen in D1776343,
which worked around the problem by changing the type of
bitOffset_ like this: s/unsigned int/size_t.

The real problem lay in the distance_to function, which computed
the signed difference of two bitOffset_ values.
To work properly with the narrower unsigned type,
we could either first widen each operand to ssize_t
before computing their parenthesized difference, or
(as Philip suggested and I prefer), just drop the
parentheses and let the signedness of the preceding
operands ensure that each successive intermediate
result is also signed.

Test Plan:
Choose one of the failing tests, and ensure it now passes:

fbconfig -r unicorn/diskindex4/test:doc_map_test
fbmake runtests

Also run all unicorn tests, for good measure (still waiting):
fbconfig -r unicorn && fbmake runtests

Reviewed By: philipp@fb.com, andrei.alexandrescu@fb.com

Subscribers: trunkagent, net-systems@, folly-diffs@

FB internal diff: D1776624

Tasks: 5941250

Signature: t1:1776624:1420872874:b1ea8a3f23f21269aa65cbe1647e86a8f622a710

Blame Revision: D1770613

Summary:
it's redundant and we shouldn't encourage it, so i'm changing all callsites
i changed docs as well

Test Plan: wait for contbuild

Reviewed By: hans@fb.com

Subscribers: trunkagent, fbcode-common-diffs@, hero-diffs@, cold-storage-diffs@, targeting-diff-backend@, adityab, everstore-dev@, zhuohuang, darshan, micha, folly-diffs@, wch, lins, tingy, jsedgwick

FB internal diff: D1784667

Tasks: 5936469

Signature: t1:1784667:1421364620:83169739320e5342d28a744e3689794f16108fea