fbshipit-source-id: f5fba149a4200b522bf90700f320685cbf0737f9

Summary:
[Folly] Pure C++ for TSAN annotation wrappers, v.s. using the preprocessor.

The implementation uses the preprocessor in order to support platforms without weak symbols, but the interface is now pure C++.

Reviewed By: nbronson

Differential Revision: D10291176

fbshipit-source-id: 3636652793b8d0da59c7c9b1095d4c1b0c506372

Summary:
This setting is already set by FollyConfigChecks.cmake, but was not
actually defined by folly-config.h

This setting is only checked by a few of the concurrency tests.

Reviewed By: igorsugak

Differential Revision: D10405959

fbshipit-source-id: 1602730e6adb7e05a82e65bdc5c367d105f59224

Summary:
When using clang with libstdc++ `-latomic` must be explicitly specified
as part of the link command in order to get some of the symbols required
for C++ std::atomic support.

The old autoconf-based build did have this check (added in github commit
60a5636b, D2519083).  However this check was missing from the
CMake-based build.

Reviewed By: igorsugak

Differential Revision: D10405960

fbshipit-source-id: 37291edc5a387e2e966ece79241aa00312194cbd

Summary: Folly zerocopy improvements

Reviewed By: djwatson

Differential Revision: D10324653

fbshipit-source-id: dacb2941e08eaf2e7588c04a01800e297a0643cf

Summary: Update the CMake files to support compiling with ASAN.

Reviewed By: igorsugak

Differential Revision: D10401270

fbshipit-source-id: 08c7fe3b661c17ba21fd8a8eb3b5cf716ebd3d35

Summary: then to thenValue to remove calls to deprecated functions.

Reviewed By: yfeldblum

Differential Revision: D10366311

fbshipit-source-id: 99c4cd8bcfa2d9db80c2d8eb22300d92609a36fc

Summary:
Add a new macro to track if folly was compiled with ASAN enabled.  This
is in addition to the existing FOLLY_SANITIZE_ADDRESS macro, which
reports if the current .cpp file is being compiled with ASAN enabled.

In particular, these macros differ in open source builds if folly was
compiled without ASAN enabled, but some of its headers are being
included from downstream libraries that are being built with ASAN
enabled.

Previously several headers would define structures differently based on
`FOLLY_SANITIZE_ADDRESS`, which meant that different .cpp files would see
different definitions of these structures.  `FOLLY_ASAN_ENABLED` is generally
the correct macro to use for situations like this.

This diff fixes F14Table.h to use FOLLY_ASAN_ENABLED, but several other
headers probably should also be updated in the future.

I did not change the behavior of FOLLY_SANITIZE_ADDRESS, since it appears to
be used by several projects outside of folly, and they expect it to report if
their code is being compiled with ASAN enabled.

At the moment the open source build does not support compiling with ASAN,
so FOLLY_ASAN_ENABLED is always disabled in open source builds.

Reviewed By: nbronson

Differential Revision: D10401271

fbshipit-source-id: 7e028737b42eecae93f06538dcf76dd64bd32e6a

Summary: FOLLY_HAVE_LIBJEMALLOC does not get defined, breaking builds in certain configurations (e.g., jemalloc present, Fedora 28 with cmake). FOLLY_USE_JEMALLOC does. Substitute former with latter,

Differential Revision: D10349332

fbshipit-source-id: e30245f8f19c6b2b7861cd88a4e1719a15c10fce

Summary: [Folly] Use explicit memory orders in `SerialExecutor`.

Reviewed By: andriigrynenko

Differential Revision: D10370862

fbshipit-source-id: 5cc5903d188b131f851a62690cd182de6ba47eb9

Summary:
Reduced contention with removing connection between queue_ and schedule_ access.
Replaced std::queue with unbounded multiple producer single consumer queue.

Reviewed By: yfeldblum

Differential Revision: D10345872

fbshipit-source-id: 3307d908d6e4826eab4d981447c8771021bfe580

Summary: Pends the freeing of the internal SSL until the socket is finally destroyed. This ensures that the async job can write out the result and call the socket's callback. This also always calls restartSSLAccept in order to let it handle errors and cleaning up of async jobs.

Reviewed By: knekritz

Differential Revision: D9599917

fbshipit-source-id: 8c4ce8b762fe59f08c2a40e76a0bebe59cd2929e

Summary: Allows for a runner to execute the accept function and return the result via a callback. If no runner is specified, it simply does the accept as usual.

Reviewed By: yfeldblum

Differential Revision: D9849138

fbshipit-source-id: ef43ccc8668bcf1fe7f75b0b6fdcdba7adc891da

Summary:
Fixes #947.

>libc++ 6+ (and Xcode 10's libc++, which seems to be based on it) adds std::launder, but does not define the __cpp_lib_launder feature-test macro. As a result, folly's lang/Launder.h concludes std::launder is missing and defines its own implementation. Since std::launder does actually exist, ADL causes 'ambiguous call' errors whenever a standard library type like std::pair is laundered within the folly namespace. (This breaks compilation of folly's futures, for example, and I believe it will cause compilation issues anywhere else F14Map is used, since it launders std::pair in its F14Policy.)
Pull Request resolved: https://github.com/facebook/folly/pull/948

Reviewed By: ot

Differential Revision: D10287027

Pulled By: yfeldblum

fbshipit-source-id: be3469fe6855d2af266e1b6a62968770e8cfd76f

Summary: [Folly] Opportunity to use `exchange` in `ScopeGuardImpl`.

Reviewed By: Orvid

Differential Revision: D10119838

fbshipit-source-id: 64a00f4f60d913f8af248ebf891f730d2c69fec8

Summary: then to thenValue to remove calls to deprecated functions.

Reviewed By: Orvid

Differential Revision: D10362133

fbshipit-source-id: e638d24e95c5c6c46d0366eabcd1ff9537421bc6

Summary:
Just require that MSVC is at least MSVC 2017

Closes: https://github.com/facebook/folly/issues/949

Reviewed By: yfeldblum

Differential Revision: D10355264

fbshipit-source-id: 52ac7489200285f06c5f4fb3d176f6427974b163

Summary:
Make StringPiece and MutableStringPiece implicitly convertible
to std::string_view.  This diff also adds FOLLY_HAS_STRING_VIEW to
Portability.h, which will be defined to be 0 or 1.

Reviewed By: vitaut

Differential Revision: D10187660

fbshipit-source-id: 90e6f3ac82e47eae423dab8f1de1d39e5ac997c4

Summary: [Folly] `exchange` in `DelayedDestruction` move-ctor.

Reviewed By: knekritz

Differential Revision: D10320776

fbshipit-source-id: b6b1f910b927098e61f6f6c8e414fdfcca843033

Summary: Clear the zerocopy maps in AsyncSocket::failAllWrites if the fd_ has been closed

Reviewed By: djwatson

Differential Revision: D10347608

fbshipit-source-id: a60d0450b13d4989d6849ca4cbe1057ecb6b92dd

Summary: Future::then to Future::thenValue to remove calls to deprecated functions.

Reviewed By: Orvid

Differential Revision: D10346695

fbshipit-source-id: 7f9641ef5b7f93ac4376a52724d42502eb1703b9

Summary: MSVC doesn't support inline assembly.

Reviewed By: yfeldblum, aary

Differential Revision: D10345334

fbshipit-source-id: 5b95d9f6bd8158fe426e01fa176ecfa732cf5bc3

Summary:
DistributedMutex is a small, exclusive-only mutex that distributes the
bookkeeping required for mutual exclusion in the stacks of threads that are
contending for it.  It tries to come at a lower space cost than std::mutex
while still trying to maintain the fairness benefits that come from using
std::mutex.  DistributedMutex provides the entire API included in
std::mutex, and more, with slight modifications.  DistributedMutex is the
same width as a single pointer (8 bytes on most platforms), where on the
other hand, std::mutex and pthread_mutex_t are both 40 bytes.  It is larger
than some of the other smaller locks, but the wide majority of cases using
the small locks are wasting the difference in alignment padding anyway

Benchmark results are good - at the time of writing in the common
uncontended case, it is 30% faster than some of the other small mutexes in
folly and as fast as std::mutex, which recently optimized its uncontended
path.  In the contended case, it is about 4-5x faster than some of the
smaller locks in folly, ~2x faster than std::mutex in clang and ~1.8x
faster in gcc.  DistributedMutex is also resistent to tail latency
pathalogies unlike many of the other small mutexes.  Which sleep for large
time quantums to reduce spin churn, this causes elevated latencies for
threads that enter the sleep cycle.  The tail latency of lock acquisition
on average up to 10x better with DistributedMutex

DistributedMutex reduces cache line contention by making each thread wait
on a thread local spinlock and futex.  This allows threads to keep working
only on their own cache lines without requiring cache coherence operations
when a mutex heavy contention.  This strategy does not require sequential
ordering on the centralized atomic storage for wakeup operations as each
thread assigned its own wait state

Non-timed mutex acquisitions are scheduled through intrusive LIFO
contention chains.  Each thread starts by spinning for a short quantum and
falls back to two phased sleeping.  Enqueue operations are lock free and
are piggybacked off mutex acquisition attempts.  The LIFO behavior of a
contention chain is good in the case where the mutex is held for a short
amount of time, as the head of the chain is likely to not have slept on
futex() after exhausting its spin quantum.  This allow us to avoid
unnecessary traversal and syscalls in the fast path with a higher
probability.  Even though the contention chains are LIFO, the mutex itself
does not adhere to that scheduling policy globally.  During contention,
threads that fail to lock the mutex form a LIFO chain on the central mutex
state, this chain is broken when a wakeup is scheduled, and future enqueue
operations form a new chain.  This makes the chains themselves LIFO, but
preserves global fairness through a constant factor which is limited to the
number of concurrent failed mutex acquisition attempts.  This binds the
last in first out behavior to the number of contending threads and helps
prevent starvation and latency outliers

This strategy of waking up wakers one by one in a queue does not scale well
when the number of threads goes past the number of cores.  At which point
preemption causes elevated lock acquisition latencies.  DistributedMutex
implements a hardware timestamp publishing heuristic to detect and adapt to
preemption.

DistributedMutex does not have the typical mutex API - it does not satisfy
the Lockable concept.  It requires the user to maintain ephemeral
bookkeeping and pass that bookkeeping around to unlock() calls.  The API
overhead, however, comes for free when you wrap this mutex for usage with
folly::Synchronized or std::unique_lock, which is the recommended usage
(std::lock_guard, in optimized mode, has no performance benefit over
std::unique_lock, so has been omitted).  A benefit of this API is that it
disallows incorrect usage where a thread unlocks a mutex that it does not
own, thinking a mutex is functionally identical to a binary semaphore,
which, unlike a mutex, is a suitable primitive for that usage

Timed locking through DistributedMutex is implemented through a centralized
algorithm - all waiters wait on the central mutex state, by setting and
resetting bits within the pointer-length word.  Since pointer length atomic
integers are incompatible with futex(FUTEX_WAIT) on most systems, a
non-standard implementation of futex() is used, where wait queues are
managed in user-space.  See p1135r0 and folly::ParkingLot

Reviewed By: djwatson

Differential Revision: D8949918

fbshipit-source-id: a772a70114e943ff68525c990da45e32ad1a5077

Summary:
[Folly] Reimplement `SCOPE_SUCCESS` and `SCOPE_FAIL` on top of `ScopeGuardImpl`.

Allows the underlying type to be safely move-constructed, in case the compiler does not do RVO copy elision in pre-C++17 builds.

Fixes #940.

Reviewed By: ericniebler

Differential Revision: D10120199

fbshipit-source-id: 889b867041338d6b76a0af5f04076b16e3120f63

Summary:
[Folly] Disable `-Wdeprecated-declarations` in cmake builds.

To hide deprecation spam in unit-tests for uses of deprecated declarations which cannot yet be removed.

Reviewed By: Orvid

Differential Revision: D10287167

fbshipit-source-id: 2dddd9f69ca1dcced61e4a2d6cd16bad54b2a2ad

Summary:
Remove the form of Future::then that could take a continuation with no parameters from elsewhere in Future.h as it is deprecated, and pending deletion.

In particular this required splitting the multi forms of the functions which still allow flexible typing at this point in time.

Reviewed By: yfeldblum

Differential Revision: D10206118

fbshipit-source-id: bb2ae241d3bfd5b89cae077e667cb39c4cc62797

Summary: Do not check zeroCopyEnabled_ in AsyncSocket::isZeroCopyMsg to avoid an IOBuf leak when zeroCopyEnabled_ changes from true to false

Reviewed By: yfeldblum

Differential Revision: D10285664

fbshipit-source-id: 5f13426f6140668d029a89786eb9a92466d4b1f4

Summary:
I was reading through about the various approaches to small-string
optimization: https://shaharmike.com/cpp/std-string/.
I noticed that Clang can store up to 22 bytes inline but FBString claims it
can do 23, so I read through the code to figure out where the extra byte came
from.

Specifically, I wasn't sure how it could have space to store the size as well
as ensure the buffer was null-terminated. After playing with the code for a bit
(this was further complicated because running the code under ASAN changes the
behavior, which I didn't realize before I started this exploration), I saw
how we don't store the size, but `maxSmallSize - size`, so that null-termination
works out even when `size==23`.

This updates the docs to hopefully save someone else this same exploration.

(Note: this ignores all push blocking failures!)

Reviewed By: ot

Differential Revision: D10258831

fbshipit-source-id: bfc0dd7ae55518af4173625bd719cfd4778180cc

Summary: [Folly] Fix cmake tests build after {D10244149}, which added `instructions::dispatch`.

Reviewed By: ot

Differential Revision: D10287193

fbshipit-source-id: f555229c8fcebccf61cb5577f5bf3edee9c46163

Summary:
[Folly] DRY template args in `Future::thenImplementation`.

Explicitly passing the template arguments which are also the types of non-template arguments is weird. Either limit the explicit template arguments to those which are not also the types of non-template arguments, or pass everything as non-template arguments.

Reviewed By: andriigrynenko

Differential Revision: D10247910

fbshipit-source-id: 64a56c5ea2072803781143b5247145ec2ee09ae0

Summary: as described in title

Reviewed By: Orvid

Differential Revision: D10279251

fbshipit-source-id: 0ab66033193026ccb0b6a06e57557a822ccb8a70

Summary:
- A couple of tests pragma push/pop to avoid `-Wself-move` warning in
  code that does a move-assignment with itself.
- Use `static_cast<T&&>` instead of `std::move` which will suppress
  the warning still without the need for pragma push/pop.
Pull Request resolved: https://github.com/facebook/folly/pull/946

Reviewed By: Orvid

Differential Revision: D10253159

Pulled By: yfeldblum

fbshipit-source-id: 9d75db95018e6115a195f246dbd93b0a5935d80d

Summary: [Folly] Enable `Fingerprint` test in CMake builds.

Reviewed By: Orvid

Differential Revision: D10081558

fbshipit-source-id: e4f753feed5b2698dd485cf32403c1c4d15d0c25

Summary:
TSAN complains about a lock inversion between folly/io/async/Request.cpp
lines 242 and 277.  The code as 242 was changed to use folly::acquireLocked() in
D9797715, but the code at 277 needs to be changed in the same way (except for
requesting a write lock on the child)

Reviewed By: kennyyu

Differential Revision: D10251470

fbshipit-source-id: c476a3664bfe83edbddbd402cf568eac885b9123

Summary:
These were failing in the watchman build

```
$ cd build && python fbcode_builder/shell_builder.py > ~/run.sh && bash ~/run.sh
Traceback (most recent call last):
  File "fbcode_builder/shell_builder.py", line 110, in <module>
    steps = make_steps(builder)
  File "/home/travis/build/facebook/watchman/build/fbcode_builder/utils.py", line 94, in <lambda>
    steps_for_spec(builder, config['fbcode_builder_spec'](builder))
  File "/home/travis/build/facebook/watchman/build/fbcode_builder/fbcode_builder.py", line 144, in build
    return [self.setup(), self.diagnostics()] + steps
  File "fbcode_builder/shell_builder.py", line 61, in setup
    ).format(ccache_dir=ccache_dir)
  File "/home/travis/build/facebook/watchman/build/fbcode_builder/shell_quoting.py", line 64, in format
    (k, shell_quote(v).do_not_use_raw_str) for k, v in kwargs.items()
KeyError: u'CC'
```

Reviewed By: snarkmaster

fbshipit-source-id: 614723e631a82f277739765a920731c872700c45

Summary: Add a simple function to dispatch the `instructions` types based on current CPU (overridable in tests).

Reviewed By: luciang

Differential Revision: D10244149

fbshipit-source-id: ac0c1ff0d14402fa77c939361398db6122dea728

Summary: For determining proper use of `strerror_r` based on available decl, move messy `#if` logic out of `String.cpp` and into buck config.

Reviewed By: yfeldblum

Differential Revision: D10243924

fbshipit-source-id: 6c10fc43562765c158d13ed74961277b633bb606

Summary:
- LLVM 7 emits a warning about self-assignment.
- Silence the warning by using a static_cast.
Pull Request resolved: https://github.com/facebook/folly/pull/945

Reviewed By: Orvid

Differential Revision: D10236656

Pulled By: yfeldblum

fbshipit-source-id: 43b46563357d89db9e0911b894f8fead0a14d660

Summary:
Apply fixes that work with new unified headers and remain compatible with old per-API+arch headers.

I do this to unclutter the diff that enables unified headers, so that it only contains things that are incompatible with old per-API+arch headers.

Reviewed By: yfeldblum

Differential Revision: D10225372

fbshipit-source-id: b92cb3ef45cf23f2233c39520236c4531dba715b

Summary:
Use `FOLLY_ATTR_WEAK` for the annotate functions.

(Note: this ignores all push blocking failures!)

Differential Revision: D10234953

fbshipit-source-id: 534cb1d2d7d6053bc000e4c7ae3c6df16fc7a958