Summary: We want to use observer API to correlate tasks across same Thrift request. `RequestContext::getRootId` is the thing for it.

Reviewed By: yfeldblum

Differential Revision: D20600485

fbshipit-source-id: 1a79bde0f6c5b3692b4fb3805c98f377b1b317c9

Summary: Use pre-existing cached local contexts, rather than creating our own.

Reviewed By: bimbashrestha

Differential Revision: D18890550

fbshipit-source-id: d53d0bdf35ebfcfa5dfb45da6b99a886022d5f8b

Summary:
Fix PollIoBackend compile error

(Note: this ignores all push blocking failures!)

Reviewed By: yfeldblum, kevin-vigor

Differential Revision: D20955224

fbshipit-source-id: a2c7a34d9129cb282d3652607e79dcee1ce1cc78

Summary:
This diff extracts the fbsource commit hash and the date of that
commit and maintains that in place of just the commit hash that
we were previously extracting.

This data is exported into the environment that we pass on to
builders so that it is available if they choose to use it.

In a follow on diff I'll use this to default a version number
in the watchman project.

Reviewed By: fanzeyi

Differential Revision: D20949666

fbshipit-source-id: dc12bffe5f0efc4297b15ba0140c4c67a23ab0fd

Summary:
Allow a T that derives from EnableMasterFromThis<T> to use masterLockFromThis() to get a non-owning shared_ptr to this and to use masterRefFromThis() to get a MasterPtrRef<> from this.

Adds MasterPtr::cleanup() that returns SemiFuture<Unit>. join() just does a blocking wait on the SemiFuture returned from cleanup().

Allows a T to provide a T::cleanup() method that will be composed into the MasterPtr::cleanup() work.

MasterPtr now uses SemiFuture<Unit> instead of Baton. This allows users of MasterPtr::cleanup() to compose cleanup work with other tasks.

Andrii suggested that the cleanup feature be extracted out of MasterPtr

Adds cleanup traits (that MasterPtr satisfies) and a Cleanup type that satisfies the cleanup traits and allows objects that are not heap-allocated to participate in structured concurrency by deriving from Cleanup.

Reviewed By: andriigrynenko

Differential Revision: D19584561

fbshipit-source-id: aa2d608effe613ec84b08f902a1c61561f3458bb

Summary: [Folly] Simplify a few cases of writing a loop to test each element of the loop in small-locks and spin-locks unit-tests.

Reviewed By: markisaa

Differential Revision: D20936029

fbshipit-source-id: fd8a922d8e7b0a00ccd524c9905c641fd97a32f9

Add coroutine-frame memory allocation hooks to enable identifying async coroutine allocations in traces

Summary:
Customises the coroutine-frame allocation for all async coroutine types to call through
the new `folly_coro_async_malloc` and `folly_coro_async_free` functions when heap-allocating
coroutine frames.

This should allow the identification of samples/traces that are calling into memory allocation
for allocating coroutine frames and should enable better quantification of the CPU cycles
and memory usage attributable to coroutine-frame allocations by filtering to samples that
contain these functions in their stack-traces.

Reviewed By: davidtgoldblatt

Differential Revision: D20929042

fbshipit-source-id: a94691377b92fab42736942ec8c7316c82a4205d

Summary: Folly OSS does not list range-v3 as a dependency. As a result, test/RangeTest.cpp fails to compile because it tries to include a range-v3 header. This change gates the inclusion of the header with `__has_include`, making the dependency optional.

Reviewed By: yfeldblum

Differential Revision: D20839733

fbshipit-source-id: 2d5e9c17427a0913f40f1ee80e868e27080a68f0

Summary: Improve IOBuf::clone and IOBuf::cloneOne() perf

Reviewed By: yfeldblum

Differential Revision: D20882203

fbshipit-source-id: ce37c2bf3c0fbdbe3c316856dc431a0108e4b9bf

Summary:
Fix folly::AsyncUDPSocket::gro_.value() check - GRO is enabled if the value is > 0

(Note: this ignores all push blocking failures!)

Reviewed By: kevin-vigor

Differential Revision: D20930544

fbshipit-source-id: 6672b65cafe3f9859f86ad34a6c3a2d65c2dc355

Summary: This prevents one acquire/release.

Reviewed By: lewissbaker

Differential Revision: D20924353

fbshipit-source-id: 8c5135bd2ebc03b736b1ea3dbb385e01218cffbc

Summary:
This is a follow up from D16427231; after testing the code in a similar situation on a mac (with `clang` and `libc++`) I realized there were still standards compliance issues.

In that diff, the guarantee we provide is:

```
 *   - insert() single key variants, emplace(), and emplace_hint() only provide
 *     the strong exception guarantee (unchanged when exception is thrown) when
 *     std::is_nothrow_move_constructible<value_type>::value is true.
```

The implementation is eventually backed by `std::vector`.

The guarantees provided by `std::vector` in the standard are slightly weaker (https://en.cppreference.com/w/cpp/container/vector/insert):

```
If an exception is thrown when inserting a single element at the end, and T is CopyInsertable or std::is_nothrow_move_constructible<T>::value is true, there are no effects (strong exception guarantee).
```

i.e. it only provides this safety guarantee if the element is inserted at the end, and that's exactly what the new test shows.

After discussion with nbronson we discovered that `libstdc++` and `libc++` are using different techniques to handle the corner case of `v.insert(0, v[1])` for `insert(iter, value_type const&)`. `libstdc++` makes the copy into a temporary, and then moves it in, so it does in fact provide a stronger guarantee than necessary. `libc++` checks the address of the reference it is given, and if it is in the moved range it adjusts it (!) and then uses the copy assignment operator. This means on copy assignment exception there will be a moved-from element at the desired index.

Section 26.2.1 para 11 says that `insert` or `emplace` will have no effects if an exception is thrown, unless otherwise specified. 26.3.11.5 is the `vector` modifiers, but that only mentions `insert`. That means we can fix this by merely switch our uses of `std::vector::insert` to `std::vector::emplace` inside `sorted_vector_map/set`. On `libstdc++` this should't have any effect, since both insert and emplace make a temporary copy for the non-end insertion case (in different but similar code paths). On `libc++` this will avoid the address-adjusting trick and use the temporary copy technique like in `libstdc++`.

Reviewed By: yfeldblum

Differential Revision: D20785383

fbshipit-source-id: 290c3c7c061dedf1660da9534f4b9cc338da6224

Summary: This should reduce refcounting

Reviewed By: lewissbaker

Differential Revision: D20911114

fbshipit-source-id: be941a5e0a095652c916e4704b0dd29874e0979d

Summary:
Migration from Future-returning executor-erasing collectX forms to
SemiFuture-returning forms, that are less risky in particular with coroutines.

Earlier changes added collectXSemiFuture and collectXUnsafe as a migration
path. We then migrated collectX callsites to collectXSemiFuture or
collectXUnsafe and switched the implementation of collectX to the SemiFuture
form.

This diff adds deprecation flag to collectXSemiFuture during the migration to
collectX and and removes from folly tests that are not specific to testing those
operations.

Reviewed By: yfeldblum

Differential Revision: D20841855

fbshipit-source-id: 30818711362c88c3296cfc976cd75fbd94e1c15c

Summary:
VS2019 changes the internal implementation of basic_string and vector.
This diff adds support for the new internals, and fixes a linkage
problem on VS2017.  It also adds the UninitializedMemoryHacksTest to
the getdeps-built tests.

For basic_string a single version handles both old and new.
basic_string::_Eos changed from public to private, so code that uses
the template specialization hack to invoke it when it is private will
work for the older public method as well.

vector does not have any suitable internal methods, so we need to adapt
to the new internal structure.

This diff fixes the issues addressed by
https://github.com/facebook/folly/pull/1345 , but uses an alternate
strategy to avoid reinterpret_cast.

Reviewed By: yfeldblum

Differential Revision: D20838799

fbshipit-source-id: eba7db2bd6feade1349d51be224c481a9156732b

Reviewed By: simpkins

Differential Revision: D20885314

fbshipit-source-id: 8c3a5ccbfd6630107b421b0d6953f17a93da2412

Summary:
Reduce the cost of copying by keeping a vector of cleared RequestData references instead of keeping a hash map of the state of all RequestData references.

Also, added FOLLY_ALWAYS_INLINE to functions called in copying of request contexts.

Reviewed By: yfeldblum

Differential Revision: D20863318

fbshipit-source-id: 29cc38ac6e8abde3acaffcf8fa4d1e7ccbd45e87

Summary: Use memcpy for copying maps with the same capacity instead of rehashing.

Reviewed By: yfeldblum

Differential Revision: D20863257

fbshipit-source-id: 11e52f9f2246440fb724a94c2611e44e253e462d

Summary:
When incrementally adding in MasterPtr to an existing code-base there are some shared_ptr<> (eg. returned from factories) that need to be stored in MasterPtr.

since unique_ptr converts to shared_ptr, just take and store a shared_ptr

Yes, this removes the guarantee that ~T() will run in join(). I am open to suggestions

Reviewed By: andriigrynenko

Differential Revision: D19583122

fbshipit-source-id: 3daf0436d56b9c396b5e82d397ebaf93ed1a2a2d

Summary:
when T derives from EnableSharedFromThis<T> and T is placed in a MasterPtr<T> then users of T have access to MasterPtr<T> functionality.

`EnableSharedFromThis<T>::masterLockFromThis()`
`EnableSharedFromThis<T>::masterRefFromThis()`

Reviewed By: andriigrynenko

Differential Revision: D19583025

fbshipit-source-id: ee12e9de30fd844b5be36c39d7e1ade830e27bbb

Summary:
The read of globalConfigData outside of sm.lock() races with any other writer that is modifying
globalConfigData inside of sm.lock(). This is a documentation only change.
Pull Request resolved: https://github.com/facebook/folly/pull/1348

Reviewed By: paulmckrcu

Differential Revision: D20864073

Pulled By: yfeldblum

fbshipit-source-id: 31b7ed469716acb35913df9a71585c2a59e50bea

Summary:
Add microbenchmark for ShallowCopyRequestContextScopeGuard that involves copying RequestContext-s that hold references to multiple RequestData objects.

Also, removed microbenchmark results for less common patterns, and updated results after switching to the hazard pointer-based implementation (D19145252).

Reviewed By: A5he

Differential Revision: D20862753

fbshipit-source-id: b4fc36a548761d4281c7c55ec97f731c7b4bc98d

Summary: [Folly] Revise references to `TAsyncSocket`.

Differential Revision: D20826538

fbshipit-source-id: 4448f19d1cc64b88dbccc1eda2be63355d15bf42

Summary:
This diff moves a struct definition out from the inside of
a closure, where it seems to have been causing a compilation issue
under VS2019.

Reviewed By: yfeldblum

Differential Revision: D20845042

fbshipit-source-id: 7b32c898490a8ba59f5e77f21e890097d9069a4f

Summary: Avoid calling ::io_uring_register_files_update again in case of failure

Reviewed By: danobi

Differential Revision: D20847977

fbshipit-source-id: 9dedecb4e815b3bb0bd41b0ba908a0c2496a67d5

Summary:
This diff fixes a couple small issues in some folly tests that
are exposed when compiling with VS2019.

Reviewed By: yfeldblum

Differential Revision: D20845004

fbshipit-source-id: 44a3e140f2bb4334c4eb81ae80297da4975a13ae

Summary: Call addTimerFd only after a callback to avoid issuing duplicate poll requests

Reviewed By: danobi

Differential Revision: D20843253

fbshipit-source-id: 9aa538dcc4cc63f1d1a3431d41366ce3fa3644cb

Summary: Handle spurious poll returning 0 when we have high load at process startup time

Reviewed By: danobi

Differential Revision: D20821506

fbshipit-source-id: 93ec91f0908f44e5210974e1cf3f30ac7aa38950

Summary:
gcc-9.3 complains about folly::Format:

```
array subscript -1 is outside array bounds of ‘char [69]’
```

But the warning is incorrect; my guess is that it is confused because
it doesn't realize that higher-level logic guarantees that integer
formatting will run out of digits before it runs out of buffer.

Changing some code from `array + x` to `&array[x]` works around it.

Reviewed By: pixelb

Differential Revision: D20694399

fbshipit-source-id: 1176858054cdec9e415de4829371647958955103

Summary:
In coverage collection mode a special module loader is prepended to
`sys.meta_path`. In very specific conditions this module loader can end up
returning a loader pointing to a _completely wrong module_. When importing
symbols from the wrong module errors occur.

The conditions to trigger the bug are:

- running in coverage collection mode, enabling the custom loader
- the test binary is a zip (e.g. par_style=fastzip)
- having a module name where the end part matches the name of a builtin Python
  module

When these conditions were met, the special loader would return the builtin
Python module instead of the expected module. E.g. when loading a module like
`myapp.somemod.platform` in a zip style binary.

The custom loader first calls `imp.find_module()` to find the module it wants
to return a wrapped loader for. This fails for modules included in the test
binary, because the builtin `imp` module can not load from zips. This was the
trigger leading to the call to the buggy code.

When the initial call to `imp.find_module()` failed, the custom loader would
try a second call, asking the internal loader to this time try any path on
`sys.path`. For most module names this call would also fail, making the custom
loader return `None`, after which Python tries other loaders on `sys.path`.
However, when the final part of the module that was asked to load matches the
name of a Python builtin module, then the second call to the `imp` module would
succeed, returning a loader for the builtin module. E.g. `platform` when asking
for `myapp.somemod.platform`.

This diff fixes the issue by removing the broken second call to the internal
loader. This will never have worked, we just have not triggered or noticed
triggering the wrong loading before.

Differential Revision: D20798119

fbshipit-source-id: dffb54e308106a81af21b63c5ee64c6ca2041920

Summary:
https://www.openssl.org/source/openssl-1.1.1b.tar.gz is gone

changelog:
https://www.openssl.org/news/openssl-1.1.1-notes.html

Reviewed By: udippant

Differential Revision: D20810020

fbshipit-source-id: 0ed385f49b2187ec149defd79feb86e2c8b492d2

Summary:
Change to SemiFuture returning collect, which checks for nullptr passed to via, breaks this test which produced a default nullptr.

Instead remove default and force passing of a valid executor.

Reviewed By: yfeldblum

Differential Revision: D20795552

fbshipit-source-id: 5411fea910d9f85eb5a4ba825a1cfd48a72b8a44

Summary: Add support for specifying which fatal signals to handle

Reviewed By: simpkins

Differential Revision: D20672281

fbshipit-source-id: 16696abcf551efcd932f5b56b336d6c5bc0d7a55

Summary: io_uring backend error handling

Reviewed By: kevin-vigor

Differential Revision: D20737416

fbshipit-source-id: 23eebc98326bcd07ee308e8f71f972c7374f6108

Summary:
I was testing vs2019 vs vs2017 and realized that
we weren't reconfiguring when the toolchain was changed;
this resolves that.

Reviewed By: genevievehelsel

Differential Revision: D20795118

fbshipit-source-id: db80f090367cacfcc6b53887b77cf949f9cef0e6

Summary:
We see flakey builds to conflicting writes to .pdb files; the error
message suggests enabling `/FS` which is already enabled.  Some serious
internet research reveals that we can put the debug info into `.obj` files
instead of having everyone contend with the same `.pdb` file.

```
C:\PROGRA~2\MIB055~1\2017\BUILDT~1\VC\Tools\MSVC\1416~1.270\bin\Hostx64\x64\cl.exe  /nologo /TP -DBOOST_CONFIG_SUPPRESS_OUTDATED_MESSAGE -DGFLAGS_IS_A_DLL=1 -DWIN32_LEAN_AND_MEAN -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_WARNINGS -D_ENABLE_EXTENDED_ALIGNED_STORAGE -D_SCL_SECURE_NO_WARNINGS -D_STL_EXTRA_DISABLED_WARNINGS="4774 4987" -IZ:\shipit\folly -I. -IZ:\installed\boost-6XBjAEHS6zoZVAY7uQHhSxYamF-vUNWngO1YaEIiiko\include\boost-1_69 -IZ:\installed\double-conversion-8uX8TcZDEz7gXyY70bMXqPUF2uLqH2OTWBffkLfU7FU\include -IZ:\installed\gflags-Md0yNU5drHUQ6MCdsB6iC0WS5YX-RFZk_eTHNKjkcPs\include -IZ:\installed\glog-_3NtzkcZDkkSFkgC70vwofqrnao05-mPxEa0E546a9g\include -IZ:\installed\libevent-TIUwVyvnVZZVHDsesjdIidM_iH4er-_6pEE0X4dzuL4\include -IZ:\installed\openssl-0G_glZ7y5BplM752sREIHKuam0GcoPqw7m6yjEGefyE\include -IZ:\installed\zlib-lxujVMVR4hhdh4-S0SCxkb55DAWh5ISCLWbfL7m7hwg\include -IZ:\installed\zstd-M8csiME1A2X7ZGECWeSGmWbqM1glHd6xoskhyPHgpaw\include -IZ:\installed\snappy-Z3WzzGXxgX1-6zSn4NdRg8Z3coQaz7Gob3gGoomN95E\include -IZ:\installed\fmt-W9-akKzyOlf4GoHcFsw1_rCxt3y0ghyjO3gb5Xve1Gc\include /DWIN32 /D_WINDOWS /W3 /GR  /MD /Zi /O2 /Ob1 /DNDEBUG   /EHs /GF /Zc:referenceBinding /Zc:rvalueCast /Zc:implicitNoexcept /Zc:strictStrings /Zc:threadSafeInit /Zc:throwingNew /permissive- /std:c++latest /bigobj /favor:blend /Zc:inline /Wall /MP /Gw /Gy /Qpar /Oi /Ot /wd4191 /wd4291 /wd4309 /wd4310 /wd4366 /wd4587 /wd4592 /wd4628 /wd4723 /wd4724 /wd4868 /wd4996 /wd4068 /wd4091 /wd4146 /wd4800 /wd4018 /wd4365 /wd4388 /wd4389 /wd4100 /wd4459 /wd4505 /wd4701 /wd4702 /wd4061 /wd4127 /wd4200 /wd4201 /wd4296 /wd4316 /wd4324 /wd4355 /wd4371 /wd4435 /wd4514 /wd4548 /wd4571 /wd4574 /wd4582 /wd4583 /wd4619 /wd4623 /wd4625 /wd4626 /wd4643 /wd4647 /wd4668 /wd4706 /wd4710 /wd4711 /wd4714 /wd4820 /wd5026 /wd5027 /wd5031 /wd5045 /we4099 /we4129 /we4566 /showIncludes /FoCMakeFiles\folly_base.dir\folly\Format.cpp.obj /FdCMakeFiles\folly_base.dir\ /FS -c Z:\shipit\folly\folly\Format.cpp

Z:\shipit\folly\folly\Format.cpp: fatal error C1041: cannot open program database 'Z:\build\folly\CMakeFiles\folly_base.dir\vc140.pdb'; if multiple CL.EXE write to the same .PDB file, please use /FS

ninja: build stopped: subcommand failed.
```

Reviewed By: yfeldblum

Differential Revision: D20752325

fbshipit-source-id: 4f20871d54cc0973b3d025734dd0886c202c839b

Summary:
This diff exposes the maximum number of internal stripes
supported by the AccessSpreader, increases that value from 128 to 256
for server platforms, and uses bulk string operations to speed up the
internal initialization. This change only affects the spreading results
on machines with more than 128 hardware threads.

(Note: this ignores all push blocking failures!)

Reviewed By: yfeldblum

Differential Revision: D20772443

fbshipit-source-id: 5c7142efb2bb9ba07cc423195074fc33aeea6dcb

Summary: This helps avoid TSAN false positives.

Reviewed By: yfeldblum

Differential Revision: D20787056

fbshipit-source-id: e62537235e3ad876e3ec5266f9c449d0d8340330

Summary: Detach by destruction is common, and for SemiFutures drops the continuation. We can instead make detach an explicit operation, parameterised optionally by an executor, and phase out implicit detach on destruction of SemiFuture.

Reviewed By: yfeldblum, andriigrynenko

Differential Revision: D20683652

fbshipit-source-id: 909d2ffcadbe71501edcddbf0847a9cb12aa3cce

Summary:
Migration from Future-returning executor-erasing collectX forms to SemiFuture-returning forms, that are less risky in particular with coroutines.

Earlier changes added collectXSemiFuture and collectXUnsafe as a migration path. This diff changes the behaviour of collectX to return SemiFuture.

Reviewed By: yfeldblum

Differential Revision: D20260537

fbshipit-source-id: 9e01536e4d9af68ff429f9fcd10aea4881d62041