Summary: Pull Request resolved: https://github.com/facebook/folly/pull/1319

Reviewed By: yfeldblum, lnicco

Differential Revision: D19951699

Pulled By: afrind

fbshipit-source-id: b71d96020ca486139472512726183ae4d3555b28

Summary: Use sdallocx for IOBuf::releaseStorage

Reviewed By: marksantaniello

Differential Revision: D19925485

fbshipit-source-id: 142276fb64e570e10bfd0b6e22d8c83b204d1c2b

Summary:
Add FOLLY_CLANG_DISABLE_WARNING for Unistd.h

(Note: this ignores all push blocking failures!)

Reviewed By: yfeldblum

Differential Revision: D19652177

fbshipit-source-id: 1fcc3bb65c3b0d73870e599f9fc0463c6836cf9b

Summary:
The Buck build of RocksDB supports LZ4, so if an Eden instance's
object cache ends up compressed with LZ4 compression, the opensource
getdeps build cannot open it. Enable LZ4 in the open source getdeps
build of rocksdb.

Reviewed By: pkaush

Differential Revision: D19961878

fbshipit-source-id: 9a4f37e12bd2b40a0c5dcf2dd1684876886fae26

Reviewed By: yfeldblum

Differential Revision: D19953637

fbshipit-source-id: c2135fc24084b7bba0c9d6fc978c8b6c8f9943d4

Differential Revision:
D19848850

Original commit changeset: 21e61529d8b9

fbshipit-source-id: c490eb0024a4cda830389a359ee2631615345f2d

Summary: Previously if the loop simply failed to execute the test could pass, add some minor changes to ensure control flow is as required.

Reviewed By: andriigrynenko

Differential Revision: D19960864

fbshipit-source-id: 5a437d9e7d3b7c249cbf999fe7ea5eecfe668658

Summary: Missing includes which break modular builds.

Reviewed By: igorsugak, luciang

Differential Revision: D19738860

fbshipit-source-id: 97a0329f3e558309ca87f4dd7be8dd96e9ba09a8

Summary:
If the waited upon object throws an exception when blockingWait is called within a fiber context this can cause a SIGABT as it is not permissible to yield (due to the inner baton) in an exception context.

To fix this, instead capture the exception and permit destruction to occur outside of exception handling.

Reviewed By: andriigrynenko

Differential Revision: D19951124

fbshipit-source-id: cf1bf9a62557147863390132f551d08e20c02c2d

Summary: It was originally disabled to let test pass in ovrsource, they are now passing, enabling them back

Reviewed By: pranjalssh

Differential Revision: D19848850

fbshipit-source-id: 21e61529d8b9f519b4c8cbf4787c8e3f28b99bb1

Summary:
Add folly::sizedFree

(Note: this ignores all push blocking failures!)

Reviewed By: ot

Differential Revision: D19944311

fbshipit-source-id: b5b1a648a180662fe838237c3c3b153e42fe0617

Summary: In C++20, u8 prefixed strings are typed as `char8_t` not `char`, so we need to support that in `StringPiece`.

Reviewed By: yfeldblum

Differential Revision: D19822967

fbshipit-source-id: c3441ba718b40f0781766d4a80c3432b4514a35c

Reviewed By: yfeldblum

Differential Revision: D19768449

fbshipit-source-id: b5a259486403601cf60d4e7066640e1ed4601410

Summary:
Currently the best way to get an eager SemiFuture from an AsyncGenerator is
  folly::coro::co_invoke(
      [&]()
          -> folly::coro::Task<folly::coro::AsyncGenerator<T&&>::NextResult> {
        co_return co_await gen.next();
      })
      .scheduleOn(&ex)
      .start();
which is very verbose. This diff adds helpers to make this a simple function call.

Reviewed By: andriigrynenko

Differential Revision: D19835473

fbshipit-source-id: f6e7e72cad0e0596df375d2662c87d4412536c03

Summary: There are downstream oss builds showing build failure caused by introduction of metadata. We want to make them proceed to build so we won't get errors in CI.

Reviewed By: iahs

Differential Revision: D19900007

fbshipit-source-id: 4201448f7980b53e407fd2bc1c74ed4ffa8e18c1

Summary:
If a key type has a templated constructor that can implicitly
accept const_iterator, then a call to erase or eraseInto with a non-const
iterator results in an ambiguous overload. This diff adds an extra
overload of eraseInto to avoid the ambiguity, as well as adding unit
tests that cover the rare corner case.

Reviewed By: yfeldblum

Differential Revision: D19809069

fbshipit-source-id: 499849da2d6ad64070b13eb0a9ed86a3d49ad9b0

Summary:
It's useful to be able to move the key and value
out of a hash table as that entry is being erased. Since
std::unordered_map's extract API isn't implementable for hash tables
that perform bulk allocation, we expose similar functionality via
F14{Fast,Value,Vector,Node}Map::eraseInto.  eraseInto gives a key_type&&
and mapped_type&& to the specified callback, allowing it to move them
elsewhere before the underlying value_type is destroyed.

The keen observer will note that there is no way to move the key out
of a std::pair<key_type const,mapped_type> without relying on undefined
behavior. We already rely on this form of UB to avoid copying keys during
rehash, and perform it only when we have total control over the lifecycle
of the object and the memory backing the value_type.

eraseInto was previously implemented for F14 sets.

Reviewed By: shixiao

Differential Revision: D19777644

fbshipit-source-id: 9904224af2c6e51abeadae62cc5e66f7bf4b1ae7

Summary: This issue was only showing up in a single test, but include orders meant that stdio.h could end up being included in a way that caused it to not properly define the normal names it's supposed to, as they were getting disabled due to the `_CRT_INTERNAL_NON_STDC_NAMES` changes being made.

Reviewed By: yfeldblum

Differential Revision: D19866950

fbshipit-source-id: 9d6391f5c0e5a2ef44f0f8ba6dad064cfecf0a82

Summary: Newer versions of boost thread re-wrote how this works, so use the new way.

Reviewed By: yfeldblum

Differential Revision: D19824658

fbshipit-source-id: a3d66fd6ef781ac83c92f6313cf47a7920cfebda

Reviewed By: iahs

Differential Revision: D19873296

fbshipit-source-id: f92785feb33be09bdef4c047532b4bd3dd42972e

Summary:
g++ 8.2.1 on some platforms crashes on the gtests defined
in EventBaseTestLib.h that declare inner classes.  This diff
refactors the tests so that they can be successfully compiled.

Reviewed By: yfeldblum

Differential Revision: D19871529

fbshipit-source-id: 3e84c2983e90b7445c014c002bbac504e0366657

Summary: It is possible for blockingWait to crash if just after it leaves it's drive() method in the destructor, from another thread something releases the keep alive. If then the destructor continues it will clean up, but that other thread will then be manipulating freed data.

Reviewed By: andriigrynenko

Differential Revision: D19860616

fbshipit-source-id: 25df16b5e900f283bbc0a264233770b6d3667ba3

Summary:
Take the README.md from
https://github.com/facebookexperimental/mononoke/blob/7ead0e29e41aec0771531a4650b6170bc1ff6566/README.md
and apply it on Eden repo.

Re-add the Cargo.toml file that declares Cargo workspace.

Re-add fbcode_builder/getdeps manifest for Mononoke
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/13

Test Plan:
./build/fbcode_builder/getdeps.py build mononoke
  ./build/fbcode_builder/getdeps.py test mononoke

Reviewed By: ahornby

Differential Revision: D19833059

Pulled By: lukaspiatkowski

fbshipit-source-id: fb37e13306c0b9969a7c4e52b05e1a66a577022f

Summary: When connection is reset by server it's possible for a write to fail before the read callback is called. Make sure we still try to process all the data that was received before closing the socket.

Reviewed By: rhodo

Differential Revision: D19851139

fbshipit-source-id: af50b939ccd7f25384e48bcf29b4892a23c2ef83

Summary: platform_mapping library needs RE2 lib.

Reviewed By: shri-khare

Differential Revision: D19838529

fbshipit-source-id: 71a2c06437ce05a42396a2867ddadb029e36926e

Summary: [Folly] Remove trivial-copyability requirement from `LockFreeRingBuffer` since we can rely on the compiler to optimize copy operations over types which are trivially-copyable without our code having to specify `memcpy`.

Reviewed By: yfeldblum

Differential Revision: D19844612

fbshipit-source-id: 4770e46ac8c63fa68b68598fb5012cb58896f550

Summary: [Folly] Use SingletonThreadLocal for the current fiber manager, which does not depend on whether `__thread` or `thread_local` is available.

Reviewed By: andriigrynenko

Differential Revision: D19229511

fbshipit-source-id: a92016aca89205c2842dc4f71054b1b6306b14b9

Summary: [Folly] Prefer `invoke_result_t` in `DistributedMutex` over the underlying `decltype` and `declval` implementation.

Reviewed By: aary

Differential Revision: D19321643

fbshipit-source-id: 323a0fce49447d2f009ad3bc61e285de3f311ff1

Summary: [Folly] Use invocability helper variables in preference to types.

Reviewed By: aary

Differential Revision: D18748026

fbshipit-source-id: e7a76c1238360c55dd6d41c53f763e9df4069acb

Summary:
1. Add `"-Wshorten-64-to-32"` to `_extra_clang_flags`
2. Fix warnings generated from `buck build --keep-going wangle/...`

Reviewed By: chadaustin

Differential Revision: D19647345

fbshipit-source-id: 417840fe82ed3d5c9c2d989a0acf67e12758c1f9

Summary:
Folly does not compile with Windows SDK 1903. The issues is in folly/portability/Windows.h and surfaces when compiling files including it, e.g.: xplat/folly.portability/Stdlib.cpp
```
xplat\folly/portability/SysStat.h:47:5: error: conflicting types for 'mkdir'
int mkdir(const char* fn, int mode);
    ^
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\ucrt\direct.h:111:26: note: previous declaration is here
    _ACRTIMP int __cdecl mkdir(
                         ^
xplat\folly\portability\Stdlib.cpp:47:11: error: no matching function for call to 'mkdir'
    ret = mkdir(ptr, 0700);
          ^~~~~
C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\ucrt\direct.h:111:26: note: candidate function not viable:
      requires single argument '_Path', but 2 arguments were provided
    _ACRTIMP int __cdecl mkdir(
                         ^
2 errors generated.
```
Now internal Windows functions collide with folly portability ones defined in https://github.com/facebook/folly/blob/master/folly/portability/Unistd.h#L61-L93

C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\ucrt\direct.h defines a Windows ```mkdir``` function when
```
#if defined(_CRT_INTERNAL_NONSTDC_NAMES) && _CRT_INTERNAL_NONSTDC_NAMES
//...
    _Check_return_ _CRT_NONSTDC_DEPRECATE(_mkdir)
    _ACRTIMP int __cdecl mkdir(
        _In_z_ char const* _Path
        );

    _Check_return_ _CRT_NONSTDC_DEPRECATE(_rmdir)
    _ACRTIMP int __cdecl rmdir(
        _In_z_ char const* _Path
        );

#endif // _CRT_INTERNAL_NONSTDC_NAMES
```

When diffing the current Windows SDK 1903 version: C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\ucrt\corecrt.h
with a previous one, e.g.: 1809 C:\tools\toolchains\vs2017_15.9\WindowsSdk\Include\10.0.17763.0\ucrt\corecrt.h
we see a fundamental change:

## SDK 1903
```
#if ( defined _CRT_DECLARE_NONSTDC_NAMES && _CRT_DECLARE_NONSTDC_NAMES) || \
    (!defined _CRT_DECLARE_NONSTDC_NAMES && !__STDC__                 )
    #define _CRT_INTERNAL_NONSTDC_NAMES 1
#else
    #define _CRT_INTERNAL_NONSTDC_NAMES 0
#endif
```

## SDK 1809
```
#define _CRT_INTERNAL_NONSTDC_NAMES                                            \
    (                                                                          \
        ( defined _CRT_DECLARE_NONSTDC_NAMES && _CRT_DECLARE_NONSTDC_NAMES) || \
        (!defined _CRT_DECLARE_NONSTDC_NAMES && !__STDC__                 )    \
    )
```

Reviewed By: Orvid

Differential Revision: D19804183

fbshipit-source-id: cbbe32c611a3c011175145329c29de4a9597868c

Summary:
[Folly] Adjust allocator ctors.

* Add missing family ctor to `SysAllocator` to bring it into better alignment with generic C++ allocator expectations.
* Make other family ctors implicit.
* Ensure that the actual copy ctors take precedence.
* Hide copy assignment operators.

Reviewed By: marksantaniello

Differential Revision: D19817836

fbshipit-source-id: de90fdd7cae36ac178a89556e63a29f4014532a1

Summary: we've seen people using `apache::thrift::Stream` as a heavyweight way to produce this functionality. As we move to delete that we want to provide an alternative using `AsyncGenerator`.

Reviewed By: andriigrynenko

Differential Revision: D19774841

fbshipit-source-id: 5ebaa0afcb08f04d7fb899b34ac7dc0270186852

Summary: we've seen people using `apache::thrift::Stream` as a heavyweight way to produce this functionality. As we move to delete that we want to provide an alternative using `AsyncGenerator`.

Reviewed By: andriigrynenko

Differential Revision: D19726875

fbshipit-source-id: 047e1db4b429883a69af9819e0c6ad04022ef91f

Summary:
Range lists are contained in a separate object file section called .debug_ranges. A range list is
indicated by a DW_AT_ranges attribute whose value is represented as an offset from the
beginning of the .debug_ranges section to the beginning of the range list.
Each entry in a range list is either:
a range list entry,
a base address selection entry, or
an end of list entry.

Reviewed By: luciang

Differential Revision: D19658819

fbshipit-source-id: ed70915465f0ac6bb8ab6f7efe415a3d8c5b04ee

Summary:
Here is an example for class inline member function declared and defined in the same file:

```
Definition:
0x00004c0a:           DW_TAG_subprogram [55] *
                        DW_AT_linkage_name [DW_FORM_strp] ( .debug_str[0x00177124] = "_ZNK5folly10symbolizer4test24ClassWithInlineFunctions9inlineBarERNS0_10FrameArrayILm100EEE")
                        DW_AT_name [DW_FORM_strp] ( .debug_str[0x00089a73] = "inlineBar")
                        DW_AT_decl_file [DW_FORM_data1] (0x1a)
                        DW_AT_decl_line [DW_FORM_data1] (0xa5)
                        DW_AT_declaration [DW_FORM_flag_present]  (true)
                        DW_AT_external [DW_FORM_flag_present] (true)
                        DW_AT_accessibility [DW_FORM_data1] (0x01)
…

Declaration:   -- no DW_AT_decl_file
0x000164c1:   DW_TAG_subprogram [228] *
                DW_AT_specification [DW_FORM_ref4]  (cu + 0x48ea => {0x00004c0a})
                DW_AT_inline [DW_FORM_data1]  (0x01)
                DW_AT_object_pointer [DW_FORM_ref4] (cu + 0x161ac => {0x000164cc})

…

Inline function call
0x0001657b:     DW_TAG_inlined_subroutine [140] *
                  DW_AT_abstract_origin [DW_FORM_ref4]  (cu + 0x161a1 => {0x000164c1})
                  DW_AT_low_pc [DW_FORM_addr] (0x000000000040fa6e)
                  DW_AT_high_pc [DW_FORM_data4] (0x00000055)
                  DW_AT_call_file [DW_FORM_data1] (0x1a)
                  DW_AT_call_line [DW_FORM_data1] (0xe2)
```

Here is an example for class inline member function declared and defined in different files:

```
Definition:  DW_AT_decl_file: 0x01  --> SymbolizerTestUtils.h
0x00004c3d:           DW_TAG_subprogram [55] *
                        DW_AT_linkage_name [DW_FORM_strp] ( .debug_str[0x0017717f] = "_ZNK5folly10symbolizer4test22InlineFunctionsWrapper9inlineBarERNS0_10FrameArrayILm100EEE")
                        DW_AT_name [DW_FORM_strp] ( .debug_str[0x00089a73] = "inlineBar")
                        DW_AT_decl_file [DW_FORM_data1] (0x01)
                        DW_AT_decl_line [DW_FORM_data1] (0x30)
                        DW_AT_declaration [DW_FORM_flag_present]  (true)
                        DW_AT_external [DW_FORM_flag_present] (true)
                        DW_AT_accessibility [DW_FORM_data1] (0x01)

…

Declaration:   DW_AT_decl_file: 0xbf  --> SymbolizerTestUtils-inl.h
0x00017ee8:   DW_TAG_subprogram [231] *
                DW_AT_decl_file [DW_FORM_data1] (0xbf)
                DW_AT_decl_line [DW_FORM_data1] (0x21)
                DW_AT_specification [DW_FORM_ref4]  (cu + 0x491d => {0x00004c3d})
                DW_AT_inline [DW_FORM_data1]  (0x01)
                DW_AT_object_pointer [DW_FORM_ref4] (cu + 0x17bd5 => {0x00017ef5})

…

Inline function call
0x00017fa5:     DW_TAG_inlined_subroutine [158] *
                  DW_AT_abstract_origin [DW_FORM_ref4]  (cu + 0x17bc8 => {0x00017ee8})
                  DW_AT_low_pc [DW_FORM_addr] (0x00000000004104ee)
                  DW_AT_high_pc [DW_FORM_data4] (0x00000055)
                  DW_AT_call_file [DW_FORM_data1] (0x1a)
                  DW_AT_call_line [DW_FORM_data2] (0x0101)
```

Reviewed By: luciang

Differential Revision: D19658560

fbshipit-source-id: 78c1e42d76c19f09e4632c40e0df18bfb49479f9

Summary: Linkage name (DW_AT_linkage_name) contains more information than the identifier name (DW_AT_name).

Reviewed By: luciang

Differential Revision: D19657767

fbshipit-source-id: 87b2293e6f723ceb8b5202db0f3c569c37c6c7ea

Summary:
Actually class inline functions exist in both .debug_info and .debug_types sections. In this diff, only the information in .debug_info is used.

Step 1. Find the DW_TAG_inlined_subroutine debug info entry (DIE) based on the given address.
Step 2. Find the DW_TAG_subprogram DIE based on the DW_AT_abstract_origin attr in inlined_subroutine, but this DIE is just a declaration.
Step 3. Find the actual definition DW_TAG_subprogram DIE based on the DW_AT_specification attr.

```
.debug_info

0x00003657:           DW_TAG_subprogram [56] *   ---- step 3
                        DW_AT_linkage_name [DW_FORM_strp] ( .debug_str[0x0002932d] = "_ZN5folly10symbolizer4test24ClassWithInlineFunctions9inlineBarERNS0_10FrameArrayILm100EEE")
                        DW_AT_name [DW_FORM_strp] ( .debug_str[0x00014064] = "inlineBar")
                        DW_AT_decl_file [DW_FORM_data1] (0x18)
                        DW_AT_decl_line [DW_FORM_data1] (0xa1)
                        DW_AT_declaration [DW_FORM_flag_present]  (true)
                        DW_AT_external [DW_FORM_flag_present] (true)
                        DW_AT_accessibility [DW_FORM_data1] (0x01)

…

0x0000cd74:   DW_TAG_subprogram [203] * .   ---- step 2
                DW_AT_specification [DW_FORM_ref4]  (cu + 0x358f => {0x00003657})
                DW_AT_inline [DW_FORM_data1]  (0x01)
                DW_AT_object_pointer [DW_FORM_ref4] (cu + 0xccb7 => {0x0000cd7f})

…
0x0000cda0:   DW_TAG_subprogram [181] *
                DW_AT_low_pc [DW_FORM_addr] (0x00000000002a4980)
                DW_AT_high_pc [DW_FORM_data4] (0x00001e22)
                DW_AT_frame_base [DW_FORM_exprloc]  (<0x1> 56 )
                DW_AT_object_pointer [DW_FORM_ref4] (cu + 0xccf0 => {0x0000cdb8})
                DW_AT_specification [DW_FORM_ref4]  (cu + 0x312f => {0x000031f7})

…

0x0000ce02:     DW_TAG_inlined_subroutine [157] *               ---- step 1
                  DW_AT_abstract_origin [DW_FORM_ref4]  (cu + 0xccac => {0x0000cd74})
                  DW_AT_low_pc [DW_FORM_addr] (0x00000000002a4f66)
                  DW_AT_high_pc [DW_FORM_data4] (0x000000c0)
                  DW_AT_call_file [DW_FORM_data1] (0x18)
                  DW_AT_call_line [DW_FORM_data1] (0xe6)

.debug_types

0x00020fdf:         DW_TAG_class_type [33] *
                      DW_AT_calling_convention [DW_FORM_data1]  (0x05)
                      DW_AT_name [DW_FORM_strp] ( .debug_str[0x00067094] = "ClassWithInlineFunctions")
                      DW_AT_byte_size [DW_FORM_data1] (0x08)
                      DW_AT_decl_file [DW_FORM_data1] (0x18)
                      DW_AT_decl_line [DW_FORM_data1] (0x9f)

…

0x00020ff5:           DW_TAG_subprogram [56] *
                        DW_AT_linkage_name [DW_FORM_strp] ( .debug_str[0x0002932d] = "_ZN5folly10symbolizer4test24ClassWithInlineFunctions9inlineBarERNS0_10FrameArrayILm100EEE")
                        DW_AT_name [DW_FORM_strp] ( .debug_str[0x00014064] = "inlineBar")
                        DW_AT_decl_file [DW_FORM_data1] (0x18)
                        DW_AT_decl_line [DW_FORM_data1] (0xa1)
                        DW_AT_declaration [DW_FORM_flag_present]  (true)
                        DW_AT_external [DW_FORM_flag_present] (true)
                        DW_AT_accessibility [DW_FORM_data1] (0x01)

```

Reviewed By: luciang

Differential Revision: D19644033

fbshipit-source-id: fa5d6dc46dbb1a417b40c1d06d9eaf9178661d21

Summary:
[Folly] Revise `SignalSafeElfCache` to use `mmap` for dynamic allocation and not to require static preallocation.

While, technically, `mmap` is not documented to be async-signal-safe, in practice it is so at least on Linux. Take advantage.

Prior to this change, the `SignalSafeElfCache` with all of its storage would have to be preallocated before setting the signal handler and must not be destroyed ever. Prior to this change, the preallocation would require at least `sizeof(Path) * capacity`, which defaults to ~2MB.

It is now possible to use stack-allocated SignalSafeElfCache in signal handlers.

This change adds a multi-thread-safe and async-signal-safe `mmap`-based allocator. Details as follows:
* All allocations are rounded up to the nearest power of two in size and alignment.
* Page or larger allocations are handled directly by `mmap` and free by `munmap` and are not tracked.
* Smaller allocations are handled by an `mmap`-backed refcounted arena. Arena sections are 16 pages in size; they are allocated on demand; and they are linked together to allow for `munmap` when the allocator refcounted arena is destroyed.

Reviewed By: davidtgoldblatt

Differential Revision: D18760282

fbshipit-source-id: 56a34abfe39a7108f790537afeda832fd39714d7

Summary: [Folly] Put the current fiber manager behind an accessor function v.s. having it as a static class member.

Reviewed By: andriigrynenko

Differential Revision: D19229507

fbshipit-source-id: 7cf4eab00ee281bc6104f994a71f5e48d9553852