Summary: Moar code sharing between proxygen/thrift.  Code must be moved to a common library: The only one we currently have is folly, but if someone wants to argue for a separate lib, that's cool too

Test Plan: fbconfig -r proxygen; fbmake dev

Reviewed By: dcsommer@fb.com

Subscribers: doug, fugalh, bmatheny, njormrod

FB internal diff: D1519919

Tasks: 5002343

Summary:
Spun off from https://phabricator.fb.com/D1534506 as this seemed different enough for a new diff

Similar to previous diff but attempts to reuse a common thread management process between cpu and io bound thread pools. Also sets the stage for other common functionality, e.g. stats, monitoring, timeouts, and so on

Here is some output from the queue benchmark in common/concurrent with both of these pools added (changes to BM not in this diff): https://phabricator.fb.com/P16308560

Test Plan: added a unit test, ran benchmark

Reviewed By: davejwatson@fb.com

Subscribers: fugalh, njormrod, bmatheny

FB internal diff: D1555443

Tasks: 5002392, 5002425

Summary:
Fix a few sign comparison issues in folly::IPAddress

@override-unit-failures

Test Plan: existing tests, compiled with clang

Reviewed By: meyering@fb.com, davejwatson@fb.com

Subscribers: njormrod, bmatheny, subodh, ranjeeth, pgriess

FB internal diff: D1559152

Summary:
Abstract the with_exception() logic into a static template
function which will work with a const or non-const object.

Test Plan: exception_wrapper_test

Reviewed By: vloh@fb.com

Subscribers: njormrod

FB internal diff: D1549175

Tasks: 5127267

Summary:
It would be great if we have IO stats tracking built within AsyncIO.
It would enable upper layer application to better track the number of I/O
that was submitted and completed

Test Plan:
$ fbmake runtests
Test Results Summary:
Passed: 1734
100% successful

Reviewed By: philipp@fb.com

Subscribers: njormrod, schen, stanislav

FB internal diff: D1557472

Tasks: 5008299

Summary:
It is difficult to guarantee every binary can call
registrationComplete; for now, support a mode where we don't enforce the
strict registration lifecycle.  In the fullness of time, we can make
strictness the default.

Test Plan: runtests

Reviewed By: lins@fb.com

Subscribers: lins, anca, njormrod

FB internal diff: D1551688

Summary:
std::exception_ptr expose methods to compare equality and it
will be great to do so for folly::exception_wrapper (it helps in gmock
testing).

Test Plan: added some unit tests to test this new functionality

Reviewed By: mhorowitz@fb.com

Subscribers: njormrod

FB internal diff: D1546597

Summary:
Step 1 of moving TAsyncSocket and friends to folly.

folly::IPAddress changes (can split these off if requested):
* isLoopback looks for a single address, as opposed to subnet
* For V6, isPrivate & isLoopback don't seem to check mapped addresses

TSocketAddress changes:
* Make union use folly::IPAddress instead of sockaddr struct.  external_ bool to tell if using AF_UNIX instead.
* manage the port and scope/zone info ourselves

@override-unit-failures

Overriding failures, can't get contbuild to finish all builds after several tries.  None of the failures look related

Test Plan:
fbconfig -r thrift
fbmake runtests
fbconfig folly/test:network_address_test
fbmake runtests

Reviewed By: simpkins@fb.com

Subscribers: snaptu-diffs@, trunkagent, hphp-diffs@, njormrod, fbcode-common-diffs@, doug, ruibalp, ps, alandau, bmatheny

FB internal diff: D1479365

Tasks: 3658665

Summary:
Clang complained about this.
* folly/gen/test/FileBenchmark.cpp (BENCHMARK): Initialize "rfd" to -1
and verify that it is set to something nonnegative in the first
BENCHMARK_SUSPEND block.

Test Plan:
Run this:
fbconfig -r --clang --with-project-version clang:dev folly/gen && fbmake runtests
and observe that compilation now succeeds. (and test results remain unchanged)

Reviewed By: tudorb@fb.com

Subscribers: njormrod

FB internal diff: D1551180

Tasks: 4090011

Summary: This may have just been forgotten when imported to folly

Test Plan: live. verified file name

Reviewed By: ranjeeth@fb.com

Subscribers: doug, njormrod

FB internal diff: D1548804

Summary:
We don't always have features.h
and our glibc doesn't have __GLIBC_PREREQ()

@override-unit-failures

Test Plan: Builds on MacOSX

Reviewed By: pt@fb.com

Subscribers: njormrod

FB internal diff: D1546679

Summary: Broken on other platforms, because demangle(typeid()) seems to return different results?  not entirely sure, but would like the builds to be green again

Test Plan: watch contbuild for 'fbthrift jenkins build' to pass

Reviewed By: mhorowitz@fb.com

Subscribers: doug, njormrod

FB internal diff: D1544919

Blame Revision: D1535681

Summary:
Move wheel timer to folly.  Needed for abstracting some code out of proxygen/thrift

The only code change was removing usage of thrift's Util::currentTime() in favor of std::chrono::steady_clock::now()

Test Plan: fbconfig thrift/lib/cpp/test:HHWheelTimerTest; fbmake runtests

Reviewed By: alandau@fb.com

Subscribers: doug, alandau, bmatheny, njormrod

FB internal diff: D1519909

Summary:
Convert from jemalloc's obsolete *allocm() to *allocx().

Strengthen goodMallocSize() to always return a jemalloc size class, so
that xallocx() success/failure detection is simple.

@override-unit-failures

Test Plan: Folly, unicorn, and HHVM tests.

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: trunkagent, hphp-diffs@, ps, chaoyc, search-fbcode-diffs@, unicorn-diffs@, ptc, njormrod

FB internal diff: D1535841

Tasks: 4996808

Summary:
templates and overloads are hard.  This uses SFINAE to get
the desired behavior, and includes a test.

This also removes a hack in ExceptionWrapper which isn't needed with this fix.

Test Plan: string_test, exception_wrapper_test

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: dreiss, dancol, njormrod

FB internal diff: D1535681

Blame Revision: D1517701

Summary: the bug is in InString state when we escape only '"' character. We should escape at least '\\' and '"'. This diffs simply escapes any character after '\\'.

Test Plan: folly unit tests, mcrouter unit tests

Reviewed By: stepan@fb.com

Subscribers: njormrod

FB internal diff: D1540836

Summary:
We considered it circular if we tried to create a singleton
while the singleton was being created.  In a single threaded world, this
is correct, but under concurrency, two threads can be in the singleton
creation codepath and become confused about the state of the singleton.

This change uses a condition variable to notify when creation completes
so that other threads can wait on the creation to complete.  Circular
creation is detected via thread id.

Test Plan:
runtests (new test case; failed without the fix, passes with
it)

Reviewed By: hans@fb.com

Subscribers: lins, anca, njormrod, rkroll

FB internal diff: D1534081

Summary:
Adding some basic functionality to `Later` to make it easier to chain
things.

Test Plan:
Added tests.

`fbconfig -r folly/wangle && fbmake runtests_{dbg,dbgo,opt}`
`fbconfig --clang -r folly/wangle && fbmake runtests_{dbg,dbgo,opt}`

Reviewed By: hans@fb.com

Subscribers: fugalh, njormrod

FB internal diff: D1527826

Tasks: 4993420

Summary:
The GCC shadow variable detector is quite zealous in warning about
shadowed members. Here, we have local variables that share the name
of member functions. Rename them so that -Wshadow doesn't complain.

Test Plan:
It didn't build, and now it does.

`fbconfig -r folly && fbmake runtests`

Reviewed By: njormrod@fb.com

Subscribers: fbcode-common-diffs@, njormrod

FB internal diff: D1534033

Summary:
There are times when you want multiple singletons of the same
underlying type -- for instance, a fast and slow request handling
singleton.  This diff allows for that with an optional name that
disambiguates multiple singletons of the same type.

In addition, we now also allow direct dereferencing of the
Singleton<Foo> object to get to the underlying singleton.  This is most
useful in cases where a singleton is used inside of the same cpp file it
is defined.

Finally, make get() faster by caching the underlying pointer rather than
accessing the shared pointer.  If you're using it, you already have a
race condition and hopefully your singleton lifecycle is sane and safe
from problems this may cause.

Test Plan: runtests

Reviewed By: hans@fb.com

Subscribers: njormrod, lins, anca

FB internal diff: D1485887

Test Plan:
fbconfig -r folly/wangle && fbmake runtests -j32

Reviewed By: lucian@fb.com

Subscribers: sdoroshenko, fugalh, njormrod

FB internal diff: D1525044

Tasks: 2680246

Blame Revision: D1358230

Summary:
Change the strategy here: provide a more generic interface
which can be used safely with any exception types, but operates more
efficiently for explicitly named types.  Update the callers to use the
new pattern, which mostly means get() and operator* are replaced with
other more generic methods.  (This diff was constructed by merging
D1490304, D1501644, and D1500861.)

Test Plan:
run tests.  Run something which generates an exception
message, and make sure that message isn't "std::exception"

Reviewed By: marccelani@fb.com

Subscribers: ruibalp, mcduff, marccelani, hitesh, mshneer, rtgw-diffs@, alandau, bmatheny, adityab, wormhole-diffs@, bwester, njormrod

FB internal diff: D1517701

Summary:
The previous message from CHECK_THROW was difficult to debug.
This improves the messaging to be more actionable.

Test Plan: runtests

Reviewed By: lins@fb.com

Subscribers: lins, anca, fugalh

FB internal diff: D1483915

Summary: based on code review feedback in D1516171

Test Plan: ran the test

Reviewed By: philipp@fb.com

Subscribers: njormrod

FB internal diff: D1516187

Summary:
Defining hash as a struct and a class was causing some junk to
get printed during fbmake.

Test Plan: `fbconfig -r proxygen; fbmake dbg`, ran unit tests, didn't see output

Reviewed By: njormrod@fb.com

Subscribers: doug, shilin, njormrod

FB internal diff: D1514793

Summary:
folly/test/ArenaTest.h assumes that goodMallocSize will account
for the block overhead, if the size is +1. However, when compiling
without jemalloc (as, I would imagine, most of our open-source clients
do), goodMallocSize(64 + 1) returns 65, which is less than 64 +
sizeof(Block), and so the tests fail.

I have added a constant to Arena.h which exposes the overhead taken by
the Block, and changed the tests to use that value instead of 1.
(Arena::getTotalSize relies on the totalAllocatedSize_ variable, which
is only non-trivially set in allocateSlow, where the size of the heap is
added to sizeof(Block)).

Test Plan: fbconfig --allocator=malloc folly/test:arena_test && fbmake runtests

Reviewed By: jon.coens@fb.com

Subscribers: sdwilsh, njormrod

FB internal diff: D1512231

Summary:
This allows projects that use -Wextra (sans -Wunused-parameter) to
compile folly.

Test Plan: compiled with gcc and clang

Reviewed By: meyering@fb.com

Subscribers: njormrod, kmdent, fma, benyluo, shikong, ranjeeth, subodh, pgriess

FB internal diff: D1509827

Summary:
A few of folly's includes do not start with folly/, and so were
missed by D1411225's include-quotation ##""## -> ##<>## transformation change.
There were also some folly includes in deeper folly directories and the
docs, which I also fixed.

The search ##egrep "# *include *\"" -R folly/*## now only includes two
lines in FBString.h, which are included for libgcc purposes.

The wangle/test/Thens.cpp file, which is auto-generated, was special.
I changed the generator .rb script, but when I ran it I overwrote some
manual changes to Thens.cpp. For simplicity, I have manually applied the
change to Thens.cpp as well.

Test Plan: fbconfig -r folly && fbmake runtests

Reviewed By: hannesr@fb.com

Subscribers: sdwilsh, fugalh

FB internal diff: D1511712

Summary:
The first (and sometimes 2nd) argument to __attribute__
should begin and end with "__". If it does not, then it
is at risk of conflicting with user cpp-defined symbols.
Don't do that.  This change mechanically corrects most
such violations in fbcode.  A companion change (see parent task)
adds a lint rule to help keep things clean going forwards.

Define this function to filter the list of file names to which
we will apply the transformations. First, ensure we modify only
C and C++ sources.  The second process excludes a few directories
that appear to contain third-party code:

filter() {
grep -E '\.(c|h|cc|cpp|hpp)$' | grep -vE \
'^external/|/unbound/|/gearmand_|(3rd|third.)party|/external/libevent'
}

then, run this command,

git grep -l __attribute__ | filter \
| xargs perl -pi -e 's/(__attribute__ *)\(\(( *[^_]\w+)/$1((__${2}__/g'

inducing changes like this, for each such keyword:

-} __attribute__((packed));
+} __attribute__((__packed__));

That got all but the __format__ archetype arguments like this:
__attribute__((__format__(printf.  That final "printf"
keyword should have the "__" prefix and suffix, too.

Run this command to transform those remaining uses:

git grep -l '__attribute__.*__format' | filter | xargs perl -pi -e \
's/(__attribute__ *\(\(__format__ *)\( *(printf|scanf|strftime|strfmon)/$1(__${2}__/g'

That command induces changes like this:

-static inline int __attribute__ ((__format__ (printf, 3, 4)))
+static inline int __attribute__ ((__format__ (__printf__, 3, 4)))

This exercise is useful to avoid spurious name-space conflicts,
especially when the unprotected keyword is used in a header file.

Test Plan:
I've ensured tao builds and passes its tests.
I will watch the arc-run tests.

Reviewed By: njormrod@fb.com

Subscribers: rounak, trunkagent, hphp-diffs@, nli, ps, fbcode-common-diffs@, mcdonald, chaoyc, bill, search-fbcode-diffs@, lars, net-systems@, davejwatson, varunk, ruibalp, hero-diffs@, andrewcox, sorg, dfechete, dhruba, sdoroshenko, mcduff, marccelani, hitesh, mshneer, cold-storage-diffs@, omry, jcoens, unicorn-diffs@, ldbrandy, sumeet, abirchall, fugalh, atlas2-eng@, dcapra, mpawlowski, alandau, nkgupta, shilin, bmatheny, everstore-dev@, zhuohuang, wormhole-diffs@, vnalla, msk, maoy, mwa, jgehring, adsatlasreporting@, mconnor, oujin, bwester, micha, tulloch, ptc, logdevice-diffs, alikhtarov, shikong, fuegen

FB internal diff: D1508983

Tasks: 4947824
@override-unit-failures

Summary: Also save the link-local scope in the V6 address.  See D1479365 for more details

Test Plan: fbconfig folly/test:network_address_test; fbmake runtests

Reviewed By: simpkins@fb.com

Subscribers: marccelani, doug, ps, bmatheny

FB internal diff: D1486435

Summary:
preallocate once and for all, so that we
are really malloc efficient in toAppendDelim

Test Plan: unit tests

Reviewed By: marcelo.juchem@fb.com

FB internal diff: D1496310

Tasks: 4886092

Summary:
The Optional test relied on std::string clearing its data when
it is the source of a move. This does not happen for in-situ fbstrings,
so the test breaks in the -fb platform. The solution: wrap the string in
a class that explicitly invalidates its data upon a move.

Test Plan:
fbconfig --platform-all=gcc-4.8.1-glibc-2.17-fb -r folly
fbmake runtests
fbconfig -r folly
fbmake runtests

Reviewed By: tudorb@fb.com

Subscribers: sdwilsh

FB internal diff: D1506840

Tasks: 4943996

Test Plan: Unit test

Reviewed By: andrei.alexandrescu@fb.com

Subscribers: shilin

FB internal diff: D1497250

Summary: It should remove one malloc as per the task.

Test Plan: `fbconfig folly/wangle && fbmake runtests` Also, `--clang`.

Reviewed By: davejwatson@fb.com

Subscribers: fugalh

FB internal diff: D1498331

Tasks: 4913277

Summary:
Common pattern: a function returns Optional<X>, indicating whether some X was
present or not; you want a default value if the X was not present. Requires
that X is copiable.

Test Plan: optional_test (test added)

Reviewed By: tjackson@fb.com

Subscribers: aalexandre, fjargsto, jhj, ntv, lesha, kma

FB internal diff: D1502397

Summary:
The test runner for folly seems to run from folly/test rather than the base fbcode directory. This hack ensures that we find the necessary test files in either case.

This fixes the breakage introduced by {D1493963}.

Test Plan: fbconfig -r folly && fbmake runtests

Reviewed By: davejwatson@fb.com

Subscribers: dipanshu

FB internal diff: D1502161

Summary:
Currently folly::Synchronized can't be used with non-default
constructible types because copy and move constructors require
creation of datum_ before it's assignment.
To fix this we can initialize them in initializer-list and
handle guard creation using helper constructor

Test Plan:
fbconfig -r folly && fbmake runtests
Fixes compilation error: Synchronized started working for non-default
constructible types in my project(rfe).

Reviewed By: yuri@fb.com

Subscribers: dhruvbird

FB internal diff: D1502110