Summary:
This diff adds helper methods for allocating
possibly-over-aligned types using C++ allocators.  It uses one of three
strategies:

* REBINDING - When there is a type that is not over-aligned with the
requested alignment, the allocator is rebound to that a type with the
correct alignment (this allows allocating 1 char with alignment 8,
for example).

* BYPASSING - When the requested alignment is larger than max_align_t
but the standard allocator is in use, the allocator is bypassed and
aligned_malloc/aligned_free are used directly to manage the allocation.

* OVER-ALLOCATING - When the type is truly over-aligned and the allocator
is custom, extra memory is requested and a delta is stored before the
returned memory to allow the original base location to be recovered.

Reviewed By: shixiao

Differential Revision: D8592767

fbshipit-source-id: 487b5ff7f7f6b1b29a3f2a80d07129afd50bda24

Summary: Increase thread cache capacity from 3 to 6 hazard pointers to accommodate 2 ConcurrentHashMap iterators.

Reviewed By: yfeldblum

Differential Revision: D8595800

fbshipit-source-id: 53275c3cf6fb11c976a4da987d79ec33b9e5e617

Summary:
To reduce unnecessary copying for user code handling ConstIterator-s.

Added move test and change UpdateStressTest to use few concurrent iterators.

Reviewed By: yfeldblum

Differential Revision: D8587056

fbshipit-source-id: d6b592a3aa40ca8a7599b258b45a1d339088c978

Summary: `folly::as_const` instead

Reviewed By: yfeldblum

Differential Revision: D8248219

fbshipit-source-id: 22853d86c93065ee584458f18b7cd0e8d528d2f0

Summary: Faster thread local iteration using the ThreadEntryNode

Reviewed By: djwatson

Differential Revision: D8551597

fbshipit-source-id: 9118852c0a823851a95b63fe807bfc2679112beb

Summary:
Change the folly/future/... tests to use mainly[*] the rvalue-qualified versions of get() and get(dur).

[*] Although the lvalue-qualified get is deprecated, it is not yet removed, so this diff leaves a test for the lvalue-qualified version. But the bulk of the tests are for the rvalue-qualified get.

Reviewed By: yfeldblum

Differential Revision: D8239792

fbshipit-source-id: c8a2e90cb5ab97a59ec97994c8c6a6ca27dab86e

Summary: [Folly] Fix forwarding/moving in Futures `reduce`.

Reviewed By: marshallcline

Differential Revision: D8601105

fbshipit-source-id: 96c1632821e9e5088a900ccecdaff9f8d108b905

Summary: We have to extend DeferredExecutor to be aware of nested DeferredExecutor, so that it can propagate various signals to them.

Reviewed By: LeeHowes

Differential Revision: D8600961

fbshipit-source-id: 2b9520598cad65bd80f0346e6f76e224903780a1

Summary:
[Folly] Let Futures `Barrier::ControlBlock` ctor fully construct.

Invoke the constructor when allocating the control block, rather than try to initialize each field afterward.

Differential Revision: D8605292

fbshipit-source-id: d51648a1267ed7e3e13d9a9e8d8e5d1c18e24a98

Summary:
[Folly] Be explicit about memory orders in Futures.

This clarifies the basic purpose of each atomic variable - namely, whether it protects control flow (relaxed) v.s. state (typically acq/rel).

Reviewed By: davidtgoldblatt

Differential Revision: D8472791

fbshipit-source-id: 350ec9b38d7c36cfb6d9ae9f1abf758b1f2e97ba

Summary: When converting to folly::Future we have to perform the void -> folly::Unit conversion.

Differential Revision: D8463177

fbshipit-source-id: 96312d0447dfc381a0ee586030f21b9cdbf4bee4

Summary: [Folly] Fix Range with clang + opt builds + ASAN, where functions marked with `__attribute__((__noinline__))` are getting inlined.

Reviewed By: nbronson

Differential Revision: D8597928

fbshipit-source-id: c963feea270b9e28e118d196950b04d8457b9c66

Summary: [Folly] Fix incorrect unaligned load in `Range`.

Reviewed By: santazhang

Differential Revision: D8597638

fbshipit-source-id: 2c8c0baa0c90fb7a7d5a24f84686d3993ae8455b

Summary:
When moving from chunk i to i-1 in an F14{Value,Node}{Map,Set}
iteration, this diff prefetches chunk i-2.  This code will never be run
for small maps and sets, since they fit in a single chunk, and it does
not affect the fast-path of intra-chunk operator++.

Reviewed By: shixiao

Differential Revision: D8395569

fbshipit-source-id: 60cace5997222fd7d1fa03ffcee93f439f2fcf03

Summary:
Folly uses level [-1, -3] for the default/fastest/best compression levels.
Introduce a `ZSTD_FAST` codec to use negative compression levels.

Reviewed By: chipturner

Differential Revision: D8529340

fbshipit-source-id: 7fb7ff65f08d4dcab009c2eb073920b53055ed9b

Summary: [Folly] One `Core` alias in `Promise` and `Future`. Having two aliases was odd.

Reviewed By: marshallcline

Differential Revision: D8472386

fbshipit-source-id: fb2241445e100927602faed9d89775534d06d10a

Summary:
[Folly] Refactor non-variadic `collect` variants.

No change in behavior - just moving code around.

Reviewed By: marshallcline

Differential Revision: D8440630

fbshipit-source-id: a6349c73c2109237f1be174c2260e9c9eb2d1c0e

Summary:
[Folly] Refactor variadic `collect` and `collectAll`.

No change in behavior - just moving code around. Principally, moving related code closer together, such as moving helper context types into the functions which use them.

Reviewed By: andriigrynenko

Differential Revision: D8416843

fbshipit-source-id: 3637c46c073191ae2dabc78cbaa145fe34cca220

Differential Revision:
D8304491

Original commit changeset: 9988e9012c31

fbshipit-source-id: cc88b31cfb65e3dbc0a2b8f7fbcc565a28a796f9

Summary:
Folly hasher for integers is avalanching on 32-bit platforms,
which is properly handled by the associated traits.  This diff fixes
the unit tests to match.

Reviewed By: yfeldblum

Differential Revision: D8584675

fbshipit-source-id: 5895b2ad8d82afdef2bf277819bb6e3fff56d99a

Summary: Optional thread finalizer can help do cleanup work before thread exit.

Reviewed By: yfeldblum

Differential Revision: D8576224

fbshipit-source-id: b9ecd1b790b1935616fbe71c870dad82ec379b4e

Summary:
On 64-bit platforms uint64_t and std::size_t are the same
size, but that is not the case on 32-bit platforms such as arm or x86.
This diff makes folly/io 32-bit clean, at least to the point that it
compiles and passes unit tests.  There is no change to behavior on
64-bit platforms.

Reviewed By: shixiao

Differential Revision: D8559702

fbshipit-source-id: c61a5d3d0e327f36cdf9a606a48a256943efc098

Reviewed By: Orvid

Differential Revision: D8577555

fbshipit-source-id: 9fa46102f9acf17ab4cc5412c136df17eb8982c4

Summary: This diff makes folly/test/memoryTest pass on 32-bit platforms.

Reviewed By: shixiao

Differential Revision: D8559725

fbshipit-source-id: 472c184a655c1330f71f9f594f8804bec23e4c60

Summary: Add ThreadLocal::accessAllThreads benchmark

Reviewed By: djwatson

Differential Revision: D8555358

fbshipit-source-id: 44da3830df96183158d3641fc4611c8e36364c00

Summary: Automated reformat of folly/io subdir in preparation for other changes

Reviewed By: yfeldblum

Differential Revision: D8559473

fbshipit-source-id: c94d9c05ee77e25b6a61ee7a47b472ccea1f18f3

Summary:
This fixes two issues related to CMAKE_REQUIRED_FLAGS

----

CMAKE_REQUIRED_FLAGS, unlike the other CMAKE_REQUIRED_* variables, is a
string, not a list. A list with more than one item in
CMAKE_REQUIRED_FLAGS results in a semicolon showing up in the middle of
the command line for at least check_symbol_exists and check_type_size
calls, prematurely terminating the compilation command and resulting in
potentially incorrect CMake test failures.

For example, CMake fails to determine the size of __int128 on macOS
10.13.3, with Apple Clang 9 (clang-900.0.39.2), even though __int128 is
supported for this platform, as the following C program

    int main() {
        __int128 i;
    }

compiles successfully with the command "clang test.c". CMake fails to
compile a program with __int128 for an entirely different reason:

    Building CXX object CMakeFiles/cmTC_d7e3f.dir/INT128_SIZE.cpp.o
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++  -DGFLAGS_IS_A_DLL=0 -isystem /usr/local/include -I/usr/local/opt/openssl@1.1/include  -fsanitize=address,undefined -std=gnu++14;-Werror=unknown-warning-option   -o CMakeFiles/cmTC_d7e3f.dir/INT128_SIZE.cpp.o -c /Users/awang/code/CRES/scc-analysis/build-debug/CMakeFiles/CheckTypeSize/INT128_SIZE.cpp
    clang: error: no input files
    /bin/sh: -Werror=unknown-warning-option: command not found
    make[1]: *** [CMakeFiles/cmTC_d7e3f.dir/INT128_SIZE.cpp.o] Error 127
    make: *** [cmTC_d7e3f/fast] Error 2

Using set(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} <new_flag>")
avoids this problem.

----

CMAKE_REQUIRED_FLAGS is given to both C and C++ compiler invocations.
The C++-specific flag -std=gnu++14 causes compilation to fail on Apple
LLVM 9.1.0 (clang-902.0.39.2) with errors along the lines of:

    Building C object CMakeFiles/cmTC_af532.dir/CheckSymbolExists.c.o
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -DGFLAGS_IS_A_DLL=0 -isystem /usr/local/include -I/usr/local/opt/openssl@1.1/include  -std=gnu++14   -o CMakeFiles/cmTC_af532.dir/CheckSymbolExists.c.o   -c /Users/awang/code/CRES/scc-analysis/build-debug/CMakeFiles/CMakeTmp/CheckSymbolExists.c
    error: invalid argument '-std=gnu++14' not allowed with 'C/ObjC'
    make[1]: *** [CMakeFiles/cmTC_af532.dir/CheckSymbolExists.c.o] Error 1
    make: *** [cmTC_af532/fast] Error 2

The affected functions are:

  - pthread_atfork
  - memrchr
  - preadv
  - pwritev
  - clock_gettime
  - cplus_demangle_v3_callback
  - malloc_usable_size

Of these, only pthread_atfork and clock_gettime were detected as being
absent when they were actually present.

The failure to correctly identify the presence of pthread_atfork, at
least, can cause Folly to fail to build on macOS because Folly requires
pthread_atfork for macOS but thinks it is absent due to the failed test.
Closes https://github.com/facebook/folly/pull/811

Reviewed By: simpkins

Differential Revision: D7502576

Pulled By: yfeldblum

fbshipit-source-id: a508e432e460cf5492e5c74da7c8b1190ec923a3

Summary: [Folly] `constexpr_find_first_set`, `constexpr_find_last_set`; with an application into `F14Table`.

Reviewed By: nbronson

Differential Revision: D8529579

fbshipit-source-id: 90d02215001abf1ffb3f04550eda22021263bbb1

Summary: [Folly] Fix narrowing conversion in `folly::popcount`.

Reviewed By: ot, nbronson

Differential Revision: D8555467

fbshipit-source-id: fdef83fc05e3e595087e7b8a3aba98400e5a954a

Summary:
Previously, `visitContiguousRanges` for fallback F14 sets did not build and
`visitAllocationClasses` was invoking the visitor with the wrong order.

This diff fixes these bugs and moves the tests so that it'll get run regardless
of whether vector intrinsics are available.

Reviewed By: nbronson

Differential Revision: D8542771

fbshipit-source-id: 3aee3ab9e3278110ec8349bd9c858d19df779eb1

Summary:
There should be no expectations on what hash is internally
used in dynamic, so we don't need to maintain compatibility with
`FBString`, which hasn't been used as string storage in `dynamic` for
quite a while.

Reviewed By: yfeldblum

Differential Revision: D8304491

fbshipit-source-id: 9988e9012c31b894746488494dd644054492def3

Summary:
GCC 8 introduces a new `class-memaccess` warning. The compiler rightly complains about `memcpy`-ing a non-POD datatype even though we know it's ok. Casting to `void*` before `memcpy` silences the compiler warning.
Closes https://github.com/facebook/folly/pull/869

Reviewed By: simpkins

Differential Revision: D8475831

Pulled By: danobi

fbshipit-source-id: c0f4eb9a653803cc912be1c6148f50ab71a8c6ab

Summary:
The tdigest_test is sometimes timing out on the Travis CI build,
perhaps because it runs the tests sequentially rather than in parallel
as under buck.  This diff reduces the size of the random tests.  At the
reduced size it was slightly exceeding the hard-coded expected accuracy
bounds, so the diff also loosens those.

This is a test-only change.

Reviewed By: yfeldblum

Differential Revision: D8548385

fbshipit-source-id: f78aee6aa22d59b297c2aef757da1dc9e0831052

Summary:
The folly logging library introduced a new shared library:
libfollylogging. folly::init recently started to depend on on
libfollylogging's initLoggingOrDie(). The problem is that libfolly is
never linked with libfollylogging, leading to the following error
upon application link time:

    /usr/local/lib/libfolly.so: undefined reference to
    folly::initLoggingOrDie(folly::Range<char const*>)

This patch forces libfollylogging to implicitly link with libfolly
through the removal of libfollylogging altogether.

Note that logging/InitWeak.cpp has been added to the source list.
This was previously omitted.
Closes https://github.com/facebook/folly/pull/872

Reviewed By: simpkins

Differential Revision: D8530795

Pulled By: danobi

fbshipit-source-id: 280fbc2170364621e9a682ae38cf812c23bde5e5

Summary:
[Folly] Better control over ASAN disabling in `Range` SSE42 functions.

Just disable ASAN for the specific loads which are at risk of reading memory slightly past the end of the given buffers.

Reviewed By: ot, nbronson

Differential Revision: D8473594

fbshipit-source-id: a93565137feae7500e1588eb239ff21f01530e25

Summary: [Folly] Smaller implementations for `findFirstSet`, `findLastSet`.

Reviewed By: nbronson

Differential Revision: D8471816

fbshipit-source-id: 699af98d404e6dd76df02e344d09c850abc1c32f

Summary: [Folly] Split BitIterator test and bench, improving the benchmark in some small ways, and fixing missing `std::iterator_traits` applications along the way.

Reviewed By: nbronson

Differential Revision: D8509129

fbshipit-source-id: 0fad0ce4d1c5abf76e33e04e98db36912e9cb651

Summary:
Before this diff the vector storage policy for F14 allocated the
chunk array and the value_type array via separate calls to the allocator.
After this diff they are performed via a single allocation using a byte
allocator rebound from the original value allocator.  This will halve
the number of calls to malloc and free from a majority of F14FastMap-s
and F14FastSet-s (and from all F14VectorMap and F14VectorSet).
The optimization is likely to be most important for small sets and maps.

In a microbenchmark that just creates and destroys 1-element
F14VectorMap-s, this diff was a 20% CPU win.

This diff is not pure win.  The unified allocation of
`(64+10*sizeof(value_type))*N` causes more internal fragmentation in
jemalloc for most value_type sizes than separate allocations of `64*N`
and `10*sizeof(value_type)*N`.  48-byte value_type is probably the most
affected in practice; this diff will increase the memory footprint
of vector-policy tables with 48-byte keys by 13% once internal memory
fragmentation is taken into account.

Taking into account the default jemalloc allocation classes,

    sizeof(value_type)  |  footprint change
    --------------------|--------------------
                   24   |   1.03797
                   32   |   1.
                   40   |   1.07462
                   48   |   1.1273
                   56   |   1.0185
                   64   |   1.06558
                   72   |   1.02742
                   80   |   1.02672
                   88   |   1.
                   96   |   1.
                  104   |   1.05961
                  112   |   1.05849
                  120   |   0.965071
                  128   |   1.10305
                  136   |   0.942188
                  144   |   0.942984
                  152   |   1.05624
                  160   |   1.05549
                  168   |   0.927757
                  176   |   1.0238
                  184   |   1.02353
                  192   |   1.02327
                  200   |   1.08255
                  208   |   1.08173
                  216   |   1.08093
                  224   |   1.08015
                  232   |   0.981946
                  240   |   0.982102
                  248   |   0.982255
                  256   |   1.12316

Reviewed By: yfeldblum

Differential Revision: D8485933

fbshipit-source-id: 1a7df390e11e71e1f56f23527aebec4806eb03d1

Summary:
Restrict F14 availability (and use the fallback version) on 32-bit platforms
until we audit the code, removing assumptions such as `size_t` is 64-bit and
finding a mixer that does not require `__int128`.

Reviewed By: nbronson

Differential Revision: D8525351

fbshipit-source-id: 54fce48fe94235c8525074216426cf254db26b4e

Summary: Fix broken build due to "missing braces around initializer" error

Reviewed By: nbronson

Differential Revision: D8541080

fbshipit-source-id: f73f973c575b6d3dad999a454d1306f7608122c7