Summary: A potential "real life" scenario that maybe can hit this bug is if we erase almost all elements and then iterate over the whole map.

Test Plan: Added a test for it.

Reviewed By: mwilliams@fb.com

Subscribers: folly-diffs@, lovro

FB internal diff: D1751455

Tasks: 5866368

Signature: t1:1751455:1419016611:b44c41d348f54397844460cb38002ad0d9704972

Summary:
D1746899 enforced the constraint (previously in a comment) on the
constructor `Range(const Range&, size_t, size_t)` that `Iter` is a
`const char*`. There is however no reason for this constraint.

This patch generalizes and simplifies the constructor, and since it
has the same semantics as `subpiece`, the latter is implemented in
terms of the constructor.

Test Plan:
fbconfig -r folly && fbmake runtests_opt

Reviewed By: soren@fb.com

Subscribers: trunkagent, folly-diffs@

FB internal diff: D1747958

Signature: t1:1747958:1418930360:fcd6beeda34e64ec8a34b9491a57674ae2265596

Summary:
This temporary class will be useful to cut dependencies on thrift
while (roughly) maintaining the old interface.

Fixup various test classes that depend on the interface.  Next diff will be to cut fbthrift dep in proxygen/mcrouter

Test Plan: unit tests, jenkins

Reviewed By: hans@fb.com

Subscribers: fbcode-common-diffs@, andrewcox, alandau, bmatheny, alikhtarov, mshneer, trunkagent, doug, njormrod, folly-diffs@, davejwatson

FB internal diff: D1696094

Signature: t1:1696094:1418762312:6028815cce2845ac5f8e754132a038fabf5fd957

Summary:
We've discussed these a bunch and here they are. I stole a bunch of Hannes' magic from https://www.facebook.com/groups/715931878455430/permalink/772854686096482/ to make this easier

Test Plan: added lots of unit tests

Reviewed By: hans@fb.com

Subscribers: fugalh, folly-diffs@, hannesr

FB internal diff: D1748418

Signature: t1:1748418:1418945606:e14c7d6a31245e222bc6a0d259d0e2b9ddd5a830

Summary:
Various TARGETS could definitely use some rearranging but I'd rather wait until we're cut over to the new repo

Test Plan: wait for contbuild

Reviewed By: davejwatson@fb.com

Subscribers: ptarjan, joelm, trunkagent, hphp-diffs@, ps, fbcode-common-diffs@, fugalh, alandau, bmatheny, everstore-dev@, mwa, jgehring, fuegen, mshneer, folly-diffs@, hannesr

FB internal diff: D1740858

Tasks: 5802833

Signature: t1:1740858:1418752569:4d7d9c5b955e4d9fab4b322cf08a3d285e3db7ce

Summary: autotools is fun.  rule name has to match

Test Plan: No contbuild for this, only a jenkins hourly build: https://ci-builds.fb.com/job/folly/

Reviewed By: noamler@fb.com

Subscribers: doug, net-systems@, folly-diffs@

FB internal diff: D1747221

Signature: t1:1747221:1418929497:a29320b5e21a657fe7e0221d2bea8b332626627b

Summary:
In preparation for a global IOThreadPoolExecutor - we can't override the workerFactory if we are using a global one.

Instead, add a new getEventBases() to IOTHreadPoolExecutro and just run code on each eventBase.  For shutdown, explicitly add a stop(), instead of doing the work after loop() as exited

Cpp2Worker required minor changes to be able to use EventBaseManager's eventbase vs. their own

Test Plan: fbconfig -r thrift/lib/cpp2; fbmake runtests

Reviewed By: jsedgwick@fb.com

Subscribers: trunkagent, andrewcox, doug, fugalh, alandau, bmatheny, mshneer, folly-diffs@

FB internal diff: D1734795

Signature: t1:1734795:1418759367:b45f419e5cbc482a64a5873d9e00e3bcb071db5a

Summary:
`Range<Iter>` has an implicit constructors from strings for any
`Iter`, however such constructors are invalid (compilation fails)
if `Iter` is not `[const] char *`.

This can be an issue for overload resolution: for example

struct IsString {
bool operator()(folly::Range<int*>) { return false; }
bool operator()(folly::StringPiece) { return true; }
};

IsString()(std::string());

fails to compile because the overload is ambiguous, even if the
conversion to `ByteRange` is invalid.

This patch disables all the invalid constructors from
`[const] char*`, `std::string`, and `fbstring`.

Test Plan:
fbconfig -r folly && fbmake runtests_opt

Reviewed By: philipp@fb.com

Subscribers: folly-diffs@

FB internal diff: D1746899

Signature: t1:1746899:1418868361:50784c4993df0bd96eeb62c09c659d5e53964d9b

Summary: see attached task. concurrency is hard.

Test Plan: stress ran the hell out of it, induced failures before, can't after

Reviewed By: hans@fb.com

Subscribers: fugalh, folly-diffs@

FB internal diff: D1746879

Tasks: 5589311

Signature: t1:1746879:1418860467:0ec3f51a398eee8a836532770682a4c58758ce93

Summary: Fix the path to detail/SpinLockImpl.h in Makefile.am

Test Plan: Will confirm the Jenkins builds pass for fbthrift and proxygen.

Reviewed By: davejwatson@fb.com

Subscribers: doug, net-systems@, exa, folly-diffs@

FB internal diff: D1745795

Signature: t1:1745795:1418841636:8a861275a62b018a8181541c7d70ba0890d7129f

fix bug with elf object left with file open upon error causing Entered fatal signal handler recursively later

Summary: fix bug with elf object left with file open upon error causing Entered fatal signal handler recursively later

Test Plan: run publisher at the limit

Reviewed By: tudorb@fb.com

Subscribers: trunkagent, chip, folly-diffs@

FB internal diff: D1743052

Tasks: 5752190

Signature: t1:1743052:1418780889:ad2ee0e6999cf417c2b9fd1ef6bc82d2086774a1

Summary:
This makes several improvements to the SpinLock code:

- Add a SpinLock implementation using pthread_spinlock_t.  On non-x86_64
platforms this is preferred over the pthread_mutex_t implementation
when available.

- For the pthread implementations, throw std::system_error on error,
rather than completely aborting the program using glog's CHECK()
macros.

- Update the pthread_mutex_t implementation to call
pthread_mutex_destroy() on destruction.

- Always unit test all implementations that can be compiled on the
current build platform, even though only a single implementation will
be selected as folly::SpinLock.  This way x86_64 builds will still
unit test the pthread-based implementations.

Test Plan: Ran the unit tests.

Reviewed By: seanc@fb.com

Subscribers: trunkagent, doug, net-systems@, exa, folly-diffs@

FB internal diff: D1735770

Signature: t1:1735770:1418445953:b238aa8fb835a8d55e6e98e20c4615ae1938b98f

Summary:
Remove the PortableSpinLock header file now that all call sites have
been migrated to the new SpinLock name.

Test Plan: Built and ran unit tests for folly, thrift, and proxygen.

Reviewed By: seanc@fb.com

Subscribers: trunkagent, doug, net-systems@, exa, folly-diffs@

FB internal diff: D1734685

Signature: t1:1734685:1418335405:e834f64d7282393e942a1de3638f4cf67410eaab

Summary:
Update call sites to use the new folly::SpinLock name, instead of
folly::io::PortableSpinLock.

Test Plan: Built and ran the folly unit tests.

Reviewed By: davejwatson@fb.com

Subscribers: doug, net-systems@, exa, folly-diffs@

FB internal diff: D1734647

Signature: t1:1734647:1418334925:1bbcffccd06907224de2a102f8d4bfbe1bd62fd1

Summary:
folly::io::PortableSpinLock seems generally useful outside of the io
code.  This moves it into the base folly namespace, and renames it from
PortableSpinLock to just SpinLock.

For most users, the main difference between MicroSpinLock and SpinLock
is that SpinLock provides a constructor that does the right thing, while
MicroSpinLock has to be explicitly initialized.

Test Plan:
Added some new unit tests, and tested both the MicroSpinLock and
pthread_spinlock_t implementations.  I didn't test the Mac OS version,
although that code remains unchanged.

Reviewed By: seanc@fb.com

Subscribers: trunkagent, doug, net-systems@, exa, folly-diffs@

FB internal diff: D1734632

Signature: t1:1734632:1418394033:99f6fbe73b485a1d631a2ef7d1b39ea6f49ecb0b

Summary: This adds API which makes folly::Singleton as performant as Meyers/static-object singletons.

Test Plan:
unit test + benchmark

============================================================================
folly/experimental/test/SingletonTest.cpp       relative  time/iter  iters/s
============================================================================
NormalSingleton                                            333.32ps    3.00G
MeyersSingleton                                  100.00%   333.33ps    3.00G
FollySingletonSlow                                 0.35%    94.36ns   10.60M
FollySingletonFast                                99.43%   335.24ps    2.98G
FollySingletonFastWeak                             0.62%    53.74ns   18.61M
============================================================================

Reviewed By: alikhtarov@fb.com

Subscribers: trunkagent, folly-diffs@

FB internal diff: D1741961

Signature: t1:1741961:1418765462:d9806f1bf5275bfbe2c4c53a41b735bda93753fe

Summary:
Add a bind api similar to binding by port across all interfaces.
However, this bind will only attempt to bind sockets to the supplied
IP addresses. I'd like to add this to aid in moving TURN to IPv6.

TURN will use two specified addresses, one for IPv4, and one for IPv6, to
handle incoming connections from clients. In order to avoid duplicating
workers, we'd like to take advantage of the multi-socket implementation
of AsyncServerSocket. However, we don't want to bind to all interfaces,
especially for port 443.

Test Plan:
Plan to test by adapting TurnTcpListener to use this new API. See that
1. using one address still works as expected,
2. using zero addesses will cause exception
3. using multiple addresses works as expected

Will write unit tests if needed

Reviewed By: davejwatson@fb.com

Subscribers: hannesr, trunkagent, net-systems@, folly-diffs@, naizhi

FB internal diff: D1729442

Tasks: 3633642

Signature: t1:1729442:1418752467:22a60da4ec9009ea0b7fe28a8a436a179e0449aa

Summary:
To handle the strange relationship between steady_clock and
system_clock (on some platforms these represent only one type, on some
platforms they are separate) Futex::futexWaitUntil converts the deadline
to a duration and back.  On Xcode 6 system_clock is measured in
microseconds and steady_clock in nanoseconds, resulting in a compilation
failure.  This diff fixes it.

Test Plan:
1. compile snippet manually using Xcode
2. new unit test that causes the same implicit conversion failure in a slightly different way

Reviewed By: mssarang@fb.com

Subscribers: trunkagent, folly-diffs@

FB internal diff: D1740903

Tasks: 5831196

Signature: t1:1740903:1418754770:32c999abf1dc87415ffebf45083a903abbded9f2

Summary: Last thing before moving experimental/wangle here. Once everything is in the same directory I'm probably going to consolidate experimental/wangle/concurrent with the executors in this directory into wangle/executors/. And probably rename some of these targets. For now, a dumb move is enough.

Test Plan: waiting for contbuild

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, fbcode-common-diffs@, chaoyc, search-fbcode-diffs@, lars, ruibalp, hero-diffs@, zeus-diffs@, vikas, danielg, mcduff, cold-storage-diffs@, unicorn-diffs@, ldbrandy, targeting-diff-backend@, netego-diffs@, fugalh, adamsyta, atlas2-eng@, alandau, bmatheny, adityab, everstore-dev@, zhuohuang, sweeney, mwa, jgehring, smarlow, akr, bnitka, jcoens, luk, zhguo, jying, apodsiadlo, alikhtarov, fuegen, dzhulgakov, mshneer, folly-diffs@, wch, tingy, hannesr

FB internal diff: D1740327

Tasks: 5802833

Signature: t1:1740327:1418752541:82d7486293b0a12938730ae66d480c120aded4dc

Summary: This removes one layer on locking on the fast path, when ptr to singleton object is read from SingletonEntry.

Test Plan:
unit test

Before:

============================================================================
folly/experimental/test/SingletonTest.cpp       relative  time/iter  iters/s
============================================================================
NormalSingleton                                            335.26ps    2.98G
MeyersSingleton                                   99.50%   336.96ps    2.97G
FollySingleton                                     0.28%   120.64ns    8.29M
============================================================================

After:

============================================================================
folly/experimental/test/SingletonTest.cpp       relative  time/iter  iters/s
============================================================================
NormalSingleton                                            336.76ps    2.97G
MeyersSingleton                                   99.91%   337.07ps    2.97G
FollySingleton                                     0.36%    92.69ns   10.79M
============================================================================

Reviewed By: alikhtarov@fb.com

Subscribers: trunkagent, folly-diffs@

FB internal diff: D1727604

Signature: t1:1727604:1418701171:1728b516191a8ec4439e981d78634370b4bcf7a1

Summary:
This is something we've talked about for a while. It's also an alternative to the mechanism in D1714645.
If we like it, I'll do something similar for a global cpu executor. That functionality should probably just be
baked into Executor itself instead of a separate subclass, which is why the IOExecutor stuff is in Executor.h/.cpp,
because it'll be pretty similar. The main exception is that for getCPUExecutor() you could return a default global
InlineExecutor instead of exploding as in getIOExecutor()

Test Plan: wangle unit, will start plumbing this into the services in #5003045 if we like it

Reviewed By: davejwatson@fb.com

Subscribers: hannesr, trunkagent, fugalh, alandau, mshneer, folly-diffs@, bmatheny

FB internal diff: D1727894

Tasks: 5002442

Signature: t1:1727894:1418344077:9e54088a6acb3f78e53011a32fd1dfe8b3214c1d

Summary:
this removes the dep EventBase has on wangle as we prepare to split off wangle
also changes namespace from folly::wangle to folly

Test Plan: just a couple of codemods so waiting for contbuild

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, fbcode-common-diffs@, chaoyc, search-fbcode-diffs@, zeus-diffs@, nli, cold-storage-diffs@, unicorn-diffs@, targeting-diff-backend@, hannesr, vighnesh, fugalh, alandau, bmatheny, adityab, zhuohuang, luk, darshan, gunjan, hdoshi, dzhulgakov, alihussains, panin, ves, mshneer, folly-diffs@, lins, nimishshah

FB internal diff: D1737376

Tasks: 5802833

Signature: t1:1737376:1418423430:82d219c34fcc50218c380a17435e5880e53db6bd

Summary:
It turned out at least one optimization we were doing for
stringPrintf (using the tail of the input buffer) was causing a
performance degradation in some cases -- if the string was already
pre-sized, our resize() call would end up memset'ing the tail.  In some
instances, this could cause significant performance penalties, such as
when multiple stringAppendf calls were made.

So, this diff removes the "optimization" around using the tail of the input
buffer and instead uses a standalone stack buffer.  If vsnprintf deems
that buffer too small, a heap buffer is instead used.

As there is no std::string method that resizes the string to fill the
underlying buffer without setting the values to a default, and as it is
not legal to write past the end of the data buffer (even if capacity()
says there is enough), let's just abandon that optimization.  It turns
out this doesn't have a major performance loss for most cases since,
with this diff, most small strings will fit on-stack and then hopefully
in the string's tail anyway.

Test Plan: runtests

Reviewed By: simpkins@fb.com

Subscribers: trunkagent, net-systems@, lins, anca, folly-diffs@

FB internal diff: D1733774

Tasks: 5735468

Signature: t1:1733774:1418323943:ec47007c9756aca6cf0466bce92722ac4c7e89b2

Summary: While folly has join methods that return a string (and thus don't need a string referenced passed), it does not support this idiom with iterators. This diff adds support for calls like folly::join("/", itr.begin(), itr.end() - 1).

Test Plan: Unit test

Reviewed By: njormrod@fb.com

Subscribers: philipp, trunkagent, njormrod, folly-diffs@, yfeldblum

FB internal diff: D1702892

Tasks: 5691439

Signature: t1:1702892:1418344206:9c1736f5d8e41be1df71a415e3803fe846b387b7

Summary:
Move AsyncUDPSocket to folly.

There is also one under realtime/voip/async that looks functionaly equivalent?  I think this one is only used in gangplank currently.

Test Plan: contbuild

Reviewed By: hans@fb.com

Subscribers: trunkagent, doug, alandau, bmatheny, njormrod, mshneer, folly-diffs@

FB internal diff: D1710675

Tasks: 5788116

Signature: t1:1710675:1417477000:9aebb466757554a5fa49d7c36cb504b4d8711b68

Summary:
Before optimizing this code, it is better to measure different
aspects of it to ensure improvements are worth it and don'tintroduce
other regressions.

This adds a new benchmark as well as parameterizes an old one.

Test Plan: run it

Reviewed By: lovro@fb.com

Subscribers: lins, anca, folly-diffs@

FB internal diff: D1733760

Tasks: 5735468

Signature: t1:1733760:1418313479:9f572d5a4cf014bd266d91d0f7a75407d1514f65

Summary:
sorry this was bothering me
also remove extra include

Test Plan: OK

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, folly-diffs@

FB internal diff: D1733723

Signature: t1:1733723:1418312030:088eb084f69e2bd7703724464b2f3e71abc2607a

Summary:
This is a squashed diff consisting of the following approved diffs

D1714007
D1714645
D1715686
D1719963
D1720725
D1721838
D1721856
D1721920
D1721956
D1724910
D1728289

- execute future_* handlers in TM thread
- call async thrift clients from any thread, not just EB thread
- store a request's EventBase in the RequestContext for easier client management
* this last one will change in favor of using a global IO executor but this should still land as an intermediate step. See D1727894

Test Plan: see component diff test plans. otherwise, contbuild.

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, ruibalp, targeting-diff-backend@, hannesr, alandau, prometheus-diffs@, jeremyfein, mshneer, folly-diffs@, bmatheny, tingy

FB internal diff: D1732289

Tasks: 5003045, 5645785

Signature: t1:1732289:1418253508:30b724a91717cf7fe21029e0a1eaf239db1650be

Summary:
Bug reported by Yan Lin (not a facebook employee) through @robbert.

@philipp and @jdelong: you are the only two remaining facebookers to have
made non-trivial changes to this code.

Description of bug: layer 0 is 1->4, and we're looking for 3. We pass
over 4, and see that 1 is less than 3 in the for loop. Now a race
condition: another thread inserts 2, so layer 0 is now 1->2->3. Then,
since ht==0, we return pred->skip(0), which is now 2 instead of 4.

Why this is bad: it really breaks the lower_bound function (lower_bound
calls findNode calls findNodeDownRight), since it returns an element
that is lesser.

This patch doesn't change the behavior of the code in the normal case;
it just recycles previously-computed values so that this race condition
doesn't crash and burn.

Patch based off of Yan Lin's in-email suggestion.

Test Plan:
fbconfig -r folly && fbmake runtests

Reviewed By: philipp@fb.com

Subscribers: sdwilsh, folly-diffs@, jdelong, robbert, philipp

FB internal diff: D1732434

Signature: t1:1732434:1418260198:8c707435825cfa2a1093b681e066f320193e98f2

Summary: Fix shadowed variable name

Test Plan: compile

Reviewed By: mmandal@fb.com

Subscribers: fugalh, mathieubaudet, folly-diffs@

FB internal diff: D1732033

Signature: t1:1732033:1418250864:d460fea376ac93c05b86977629b95dac60b8b916

Summary: Some contbuild machines aren't on 3.10 yet.

Test Plan:
fbconfig folly/experimental/wangle/bootstrap; fbmake runtests
tested on 3.2 and 3.10

Reviewed By: jsedgwick@fb.com

Subscribers: doug, fugalh, folly-diffs@

FB internal diff: D1732040

Tasks: 5800250

Signature: t1:1732040:1418250452:a3203ab5769494594f2bf0b50f1c62052ddeb24c

Summary:
Due to how Conv.h uses the less_than template inside
not-actually-static static ifs clang still ends up validating generated
code inside a FOLLY_RANGE_CHECK which is never executed due to the
templated types. This code however still generates -Wsign-compare issues
so suppress that in order to allow includers to use this flag.

A simple example will illustrate this:

uint64_t foo = 1;
int bar = folly::to<int>(foo);

Test Plan: fbmake runtests

Reviewed By: meyering@fb.com, njormrod@fb.com

Subscribers: trunkagent, folly-diffs@, bmatheny, ranjeeth, subodh, kmdent, fma, shikong, pgriess, jdelong

FB internal diff: D1731411

Signature: t1:1731411:1418243200:ed1f34a1485669c9cb18f9f6029aca70e498953c

Summary:
Remove member variable name shadowing

Facebook: This diff adds a default executor for async requests which do not specify one. This ensures we don't run request callbacks on the event threads (which in turn ensures we don't make blocking tcc calls from the event threads)

Test Plan: fbconfig tao/client; fbmake runtests

Reviewed By: hannesr@fb.com

Subscribers: hannesr, fugalh, zhuohuang, folly-diffs@, tao-eng@

FB internal diff: D1731078

Tasks: 5752037

Signature: t1:1731078:1418237801:5b9bbe74c288292d5e7c7e6a38e67be864627d89

Summary: Add mock singleton injection support to folly/experimental Singleton (t5653148).

Test Plan:
Added unit tests for MockSingleton, all unit tests passed.
Validated this works for my scenario.

Reviewed By: andrii@fb.com

Subscribers: trunkagent, njormrod, folly-diffs@

FB internal diff: D1690946

Tasks: 5653148

Signature: t1:1690946:1418080331:948d7051a5e5a2653dc393c123f188c56072c6db

Summary: Use acceptor pool instead of current thread for accepts.  Use reuse_port option to support multiple threads

Test Plan: Updated unittests

Reviewed By: hans@fb.com

Subscribers: doug, fugalh, njormrod, folly-diffs@

FB internal diff: D1710619

Tasks: 5788110

Signature: t1:1710619:1417477350:eee5063186e582ef74c4802b8149563af029b3de

Test Plan: fbconfig -r folly/io/test && fbmake runtests_opt -j32

Reviewed By: viswanath@fb.com

Subscribers: folly-diffs@

FB internal diff: D1728927

Tasks: 5788015

Signature: t1:1728927:1418161580:9ed66412e35c62486a56e19aa9bf119863b5f1a1

Blame Revision: D1724111

Summary:
Adds a reuse port option to AsyncServerSocket, so multiple sockets can bind to the same accept port.  Allows for multiple accept threads, so accepts can be greater, since there is no longer a single accept lock.

reuse port option is ifdefd, since this wouldn't build with some of the older kernels otherwise.

Postponed overnight

Test Plan: Builds.  Used in an upcoming diff.

Reviewed By: jsedgwick@fb.com

Subscribers: benj, trunkagent, doug, njormrod, folly-diffs@

FB internal diff: D1710600

Tasks: 5488516, 5788110

Signature: t1:1710600:1418066966:627e03857f9b5ff831f2922add08e90cc525c95c

Summary:
The code that uses TransportInfo assumes these byte counts are
compatible with size_t so make them so.

Test Plan: compiled

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, bmatheny, njormrod, folly-diffs@, benyluo, ranjeeth, fma, kmdent

FB internal diff: D1725026

Tasks: 5538843

Signature: t1:1725026:1418061933:b4072ca926e792716776ddc00d79db9376510859