Summary: Optional thread finalizer can help do cleanup work before thread exit.

Reviewed By: yfeldblum

Differential Revision: D8576224

fbshipit-source-id: b9ecd1b790b1935616fbe71c870dad82ec379b4e

Summary:
On 64-bit platforms uint64_t and std::size_t are the same
size, but that is not the case on 32-bit platforms such as arm or x86.
This diff makes folly/io 32-bit clean, at least to the point that it
compiles and passes unit tests.  There is no change to behavior on
64-bit platforms.

Reviewed By: shixiao

Differential Revision: D8559702

fbshipit-source-id: c61a5d3d0e327f36cdf9a606a48a256943efc098

Reviewed By: Orvid

Differential Revision: D8577555

fbshipit-source-id: 9fa46102f9acf17ab4cc5412c136df17eb8982c4

Summary: This diff makes folly/test/memoryTest pass on 32-bit platforms.

Reviewed By: shixiao

Differential Revision: D8559725

fbshipit-source-id: 472c184a655c1330f71f9f594f8804bec23e4c60

Summary: Add ThreadLocal::accessAllThreads benchmark

Reviewed By: djwatson

Differential Revision: D8555358

fbshipit-source-id: 44da3830df96183158d3641fc4611c8e36364c00

Summary: Automated reformat of folly/io subdir in preparation for other changes

Reviewed By: yfeldblum

Differential Revision: D8559473

fbshipit-source-id: c94d9c05ee77e25b6a61ee7a47b472ccea1f18f3

Summary:
This fixes two issues related to CMAKE_REQUIRED_FLAGS

----

CMAKE_REQUIRED_FLAGS, unlike the other CMAKE_REQUIRED_* variables, is a
string, not a list. A list with more than one item in
CMAKE_REQUIRED_FLAGS results in a semicolon showing up in the middle of
the command line for at least check_symbol_exists and check_type_size
calls, prematurely terminating the compilation command and resulting in
potentially incorrect CMake test failures.

For example, CMake fails to determine the size of __int128 on macOS
10.13.3, with Apple Clang 9 (clang-900.0.39.2), even though __int128 is
supported for this platform, as the following C program

    int main() {
        __int128 i;
    }

compiles successfully with the command "clang test.c". CMake fails to
compile a program with __int128 for an entirely different reason:

    Building CXX object CMakeFiles/cmTC_d7e3f.dir/INT128_SIZE.cpp.o
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++  -DGFLAGS_IS_A_DLL=0 -isystem /usr/local/include -I/usr/local/opt/openssl@1.1/include  -fsanitize=address,undefined -std=gnu++14;-Werror=unknown-warning-option   -o CMakeFiles/cmTC_d7e3f.dir/INT128_SIZE.cpp.o -c /Users/awang/code/CRES/scc-analysis/build-debug/CMakeFiles/CheckTypeSize/INT128_SIZE.cpp
    clang: error: no input files
    /bin/sh: -Werror=unknown-warning-option: command not found
    make[1]: *** [CMakeFiles/cmTC_d7e3f.dir/INT128_SIZE.cpp.o] Error 127
    make: *** [cmTC_d7e3f/fast] Error 2

Using set(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} <new_flag>")
avoids this problem.

----

CMAKE_REQUIRED_FLAGS is given to both C and C++ compiler invocations.
The C++-specific flag -std=gnu++14 causes compilation to fail on Apple
LLVM 9.1.0 (clang-902.0.39.2) with errors along the lines of:

    Building C object CMakeFiles/cmTC_af532.dir/CheckSymbolExists.c.o
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -DGFLAGS_IS_A_DLL=0 -isystem /usr/local/include -I/usr/local/opt/openssl@1.1/include  -std=gnu++14   -o CMakeFiles/cmTC_af532.dir/CheckSymbolExists.c.o   -c /Users/awang/code/CRES/scc-analysis/build-debug/CMakeFiles/CMakeTmp/CheckSymbolExists.c
    error: invalid argument '-std=gnu++14' not allowed with 'C/ObjC'
    make[1]: *** [CMakeFiles/cmTC_af532.dir/CheckSymbolExists.c.o] Error 1
    make: *** [cmTC_af532/fast] Error 2

The affected functions are:

  - pthread_atfork
  - memrchr
  - preadv
  - pwritev
  - clock_gettime
  - cplus_demangle_v3_callback
  - malloc_usable_size

Of these, only pthread_atfork and clock_gettime were detected as being
absent when they were actually present.

The failure to correctly identify the presence of pthread_atfork, at
least, can cause Folly to fail to build on macOS because Folly requires
pthread_atfork for macOS but thinks it is absent due to the failed test.
Closes https://github.com/facebook/folly/pull/811

Reviewed By: simpkins

Differential Revision: D7502576

Pulled By: yfeldblum

fbshipit-source-id: a508e432e460cf5492e5c74da7c8b1190ec923a3

Summary: [Folly] `constexpr_find_first_set`, `constexpr_find_last_set`; with an application into `F14Table`.

Reviewed By: nbronson

Differential Revision: D8529579

fbshipit-source-id: 90d02215001abf1ffb3f04550eda22021263bbb1

Summary: [Folly] Fix narrowing conversion in `folly::popcount`.

Reviewed By: ot, nbronson

Differential Revision: D8555467

fbshipit-source-id: fdef83fc05e3e595087e7b8a3aba98400e5a954a

Summary:
Previously, `visitContiguousRanges` for fallback F14 sets did not build and
`visitAllocationClasses` was invoking the visitor with the wrong order.

This diff fixes these bugs and moves the tests so that it'll get run regardless
of whether vector intrinsics are available.

Reviewed By: nbronson

Differential Revision: D8542771

fbshipit-source-id: 3aee3ab9e3278110ec8349bd9c858d19df779eb1

Summary:
There should be no expectations on what hash is internally
used in dynamic, so we don't need to maintain compatibility with
`FBString`, which hasn't been used as string storage in `dynamic` for
quite a while.

Reviewed By: yfeldblum

Differential Revision: D8304491

fbshipit-source-id: 9988e9012c31b894746488494dd644054492def3

Summary:
GCC 8 introduces a new `class-memaccess` warning. The compiler rightly complains about `memcpy`-ing a non-POD datatype even though we know it's ok. Casting to `void*` before `memcpy` silences the compiler warning.
Closes https://github.com/facebook/folly/pull/869

Reviewed By: simpkins

Differential Revision: D8475831

Pulled By: danobi

fbshipit-source-id: c0f4eb9a653803cc912be1c6148f50ab71a8c6ab

Summary:
The tdigest_test is sometimes timing out on the Travis CI build,
perhaps because it runs the tests sequentially rather than in parallel
as under buck.  This diff reduces the size of the random tests.  At the
reduced size it was slightly exceeding the hard-coded expected accuracy
bounds, so the diff also loosens those.

This is a test-only change.

Reviewed By: yfeldblum

Differential Revision: D8548385

fbshipit-source-id: f78aee6aa22d59b297c2aef757da1dc9e0831052

Summary:
The folly logging library introduced a new shared library:
libfollylogging. folly::init recently started to depend on on
libfollylogging's initLoggingOrDie(). The problem is that libfolly is
never linked with libfollylogging, leading to the following error
upon application link time:

    /usr/local/lib/libfolly.so: undefined reference to
    folly::initLoggingOrDie(folly::Range<char const*>)

This patch forces libfollylogging to implicitly link with libfolly
through the removal of libfollylogging altogether.

Note that logging/InitWeak.cpp has been added to the source list.
This was previously omitted.
Closes https://github.com/facebook/folly/pull/872

Reviewed By: simpkins

Differential Revision: D8530795

Pulled By: danobi

fbshipit-source-id: 280fbc2170364621e9a682ae38cf812c23bde5e5

Summary:
[Folly] Better control over ASAN disabling in `Range` SSE42 functions.

Just disable ASAN for the specific loads which are at risk of reading memory slightly past the end of the given buffers.

Reviewed By: ot, nbronson

Differential Revision: D8473594

fbshipit-source-id: a93565137feae7500e1588eb239ff21f01530e25

Summary: [Folly] Smaller implementations for `findFirstSet`, `findLastSet`.

Reviewed By: nbronson

Differential Revision: D8471816

fbshipit-source-id: 699af98d404e6dd76df02e344d09c850abc1c32f

Summary: [Folly] Split BitIterator test and bench, improving the benchmark in some small ways, and fixing missing `std::iterator_traits` applications along the way.

Reviewed By: nbronson

Differential Revision: D8509129

fbshipit-source-id: 0fad0ce4d1c5abf76e33e04e98db36912e9cb651

Summary:
Before this diff the vector storage policy for F14 allocated the
chunk array and the value_type array via separate calls to the allocator.
After this diff they are performed via a single allocation using a byte
allocator rebound from the original value allocator.  This will halve
the number of calls to malloc and free from a majority of F14FastMap-s
and F14FastSet-s (and from all F14VectorMap and F14VectorSet).
The optimization is likely to be most important for small sets and maps.

In a microbenchmark that just creates and destroys 1-element
F14VectorMap-s, this diff was a 20% CPU win.

This diff is not pure win.  The unified allocation of
`(64+10*sizeof(value_type))*N` causes more internal fragmentation in
jemalloc for most value_type sizes than separate allocations of `64*N`
and `10*sizeof(value_type)*N`.  48-byte value_type is probably the most
affected in practice; this diff will increase the memory footprint
of vector-policy tables with 48-byte keys by 13% once internal memory
fragmentation is taken into account.

Taking into account the default jemalloc allocation classes,

    sizeof(value_type)  |  footprint change
    --------------------|--------------------
                   24   |   1.03797
                   32   |   1.
                   40   |   1.07462
                   48   |   1.1273
                   56   |   1.0185
                   64   |   1.06558
                   72   |   1.02742
                   80   |   1.02672
                   88   |   1.
                   96   |   1.
                  104   |   1.05961
                  112   |   1.05849
                  120   |   0.965071
                  128   |   1.10305
                  136   |   0.942188
                  144   |   0.942984
                  152   |   1.05624
                  160   |   1.05549
                  168   |   0.927757
                  176   |   1.0238
                  184   |   1.02353
                  192   |   1.02327
                  200   |   1.08255
                  208   |   1.08173
                  216   |   1.08093
                  224   |   1.08015
                  232   |   0.981946
                  240   |   0.982102
                  248   |   0.982255
                  256   |   1.12316

Reviewed By: yfeldblum

Differential Revision: D8485933

fbshipit-source-id: 1a7df390e11e71e1f56f23527aebec4806eb03d1

Summary:
Restrict F14 availability (and use the fallback version) on 32-bit platforms
until we audit the code, removing assumptions such as `size_t` is 64-bit and
finding a mixer that does not require `__int128`.

Reviewed By: nbronson

Differential Revision: D8525351

fbshipit-source-id: 54fce48fe94235c8525074216426cf254db26b4e

Summary: Fix broken build due to "missing braces around initializer" error

Reviewed By: nbronson

Differential Revision: D8541080

fbshipit-source-id: f73f973c575b6d3dad999a454d1306f7608122c7

Summary:
If RTTI is disabled for the compilation unit, simply skip recording the type of
F14Policy in computeStats.

This diff also contains a few fixes for when folly is built without RTTI.

Reviewed By: nbronson

Differential Revision: D8523618

fbshipit-source-id: dfbfa93ec462c0ff72c06b9f65d61b081e361f41

Summary:
The headerfile was removed from the source tree, but is still
referenced, causing autotools based builds to fail.
Closes https://github.com/facebook/folly/pull/874

Reviewed By: nbronson

Differential Revision: D8531520

Pulled By: calebmarchent

fbshipit-source-id: 33844e57f4551a66dd13ab9ec03a441c7779c41e

Summary: Adds header to the test headers exported by folly

Reviewed By: yfeldblum

Differential Revision: D8524926

fbshipit-source-id: 06983789c3ee9e403211fb5335365663878c2590

Summary: Extend AsyncUDPServerSocket to support dynamically pausing/resuming reading from the underlying socket. This functionality already exists for AsyncServerSocket but not for UDP server sockets.

Reviewed By: djwatson

Differential Revision: D8358256

fbshipit-source-id: 7d5cf14b656ef9280747515b23cd0ae2899ec38b

Summary:
Sometimes get_executor gets called from threads where there is no asyncio evenloop.
In those cases return NULL.

The caller is responsible for this situation.

Reviewed By: yfeldblum

Differential Revision: D8492709

fbshipit-source-id: 325689ecd4541b9ba097cfe1cb4d6d61ad81cb43

Summary:
Using the alignment bits to store the index for a packed
iterator is one of the steps toward better support of 32-bit platforms
(and pointer authentication in future platforms, such as ARMv8.3-A).
Previously this was complicated by a desire to make construction of
a packed ItemIter as cheap as possible.  The ItemPtr has a variable
number of bits available due to alignment, which may not be sufficient
to encode the index.  This diff observes that we can piggyback on the
chunk alignment to efficiently deduce the remaining bits of the index.
If sizeof(Item) gives us only 2 bits of alignment, for example, then we
can deduce 2 bits of the index from the Item*.

Differential Revision: D8452991

fbshipit-source-id: 40dfd9957da3733090bb947646c553620cc7d33d

Summary: Cleanup checks if the current thread has retired objects during bulk reclaim. It does so by flushing the thread local set of retired objects and checking if domain's list of retired objects.

Reviewed By: djwatson

Differential Revision: D8503159

fbshipit-source-id: 59ddd29d8ae1429b62b5bc9511b84e4c9e7128f2

Summary: Add shutdown indicator to domain to be checked in retire to avoid accessing thread local private list of objects at shutdown time when the thread local structure may be already destroyed.

Reviewed By: djwatson

Differential Revision: D8501530

fbshipit-source-id: 16adef34f2fd265f35851c792bd9e62a2f19b81a

Summary: As title

Reviewed By: yfeldblum

Differential Revision: D8517773

fbshipit-source-id: 971178184e446c9de476cf43f65e04f49f05e0ad

Summary: facepaw

Reviewed By: yfeldblum

Differential Revision: D8465375

fbshipit-source-id: 9cbd00286963366e8e987b7fe865ec5d334e65b3

Reviewed By: shixiao

Differential Revision: D8516223

fbshipit-source-id: 0e259ce65183d779f066869091cdea837b7bd237

Summary:
Sometimes unit tests compare the JSON output of a
component. We want a simple way to compare them semantically and also
to ignore numerical errors for double values.

Reviewed By: yfeldblum

Differential Revision: D8505209

fbshipit-source-id: 04da89889f08a595e8296716f71b9a22156e3506

Summary: [Folly] Let `window` work with functions returning `SemiFuture`.

Reviewed By: LeeHowes

Differential Revision: D8505836

fbshipit-source-id: 51c14b131760528eb6c367298cb23c4deec30ab3

Summary: [Folly] Let `makeSemiFutureWith` accept functions returning `Future`, joining what would otherwise be the resulting `SemiFuture<Future<...>>` into `SemiFuture<...>`.

Reviewed By: LeeHowes

Differential Revision: D8505834

fbshipit-source-id: 10b626fae00b5002877295b59716066f089cb6a8

Summary:
rehash's max_size check was previously relying on the optimizer
to inline and then remove a loop whose value could be determined at
compile time.  It failed to do this even in some trivial cases, wasting
about 10 nanoseconds per rehash.  This code moves the max_size check lower
(more than 6 entries) and removes the loop entirely.

Reviewed By: ot

Differential Revision: D8499582

fbshipit-source-id: 8f5c288d3922b094da81ee9261254c729e8cb1b9

Summary: exception_wrapper requires the actual exception object to be passed in for any std::exception.

Reviewed By: memorybank

Differential Revision: D8501214

fbshipit-source-id: 034a4c7df9979711dcb3740ca5c4313944f29cb6

Summary:
The `glog/logging.h` header is a bit expensive here considering its size, and how frequently `Range.h` is used.

Replace the `DCHECK_${comparison}` macro usage with macros defined in this file.

Reviewed By: yfeldblum

Differential Revision: D8473303

fbshipit-source-id: cc2a7e9eabab2c1f83a377de5191f708f1c9f317

Summary: Adjust the growth factor based on the StaticMetaBase::head_::elementsCapacity

Reviewed By: yfeldblum

Differential Revision: D8409650

fbshipit-source-id: 189b839e19a1d3f2f1ac8e4b35a2f11a4d319ad7

Summary:
- Use the hazard pointer library under folly/synchronization.
  - NodeT derived from hazptr_obj_base_linked.
  - Bucket heads are hazptr_root-s with dtor that automatically unlinks nodes
  - Provides support for uncertain removal of NodeT-s both from NodeT and Buckets
  - As intended, hazptr_cleanup reclaims all unprotected removed objects even if they have not been determined to be retirable at the start of cleanup.

Fixes the following:
- The pattern of calling h.get_protected(node->next_), when hazptr_holder h is protecting node is unsafe because try_protect (inside get_protected) may fail and we end protecting neither node nor next. This diff uses a different hazptr_holder to protect the next node. A correct pattern is to call h2.get_protected(node->next_) then h.swap(h2); h2.reset();
- The pattern of calling h2.reset(node) then h1.reset() to continue protecting a node already protected by h1 is unsafe because h2's protection may be too late after node was retired. A correct pattern is h2.swap(h1).

Reviewed By: djwatson

Differential Revision: D7708325

fbshipit-source-id: 617dcfe19410071888abafa32f0e1ada9d123983

Summary: [Folly] `bool_constant` (C++17 backport) and `index_constant`.

Reviewed By: aary

Differential Revision: D8440131

fbshipit-source-id: 2bc4fae1e2d1f404d4060c209ee325beebe2a415