Reviewed By: yfeldblum

Differential Revision: D22604684

fbshipit-source-id: 4dedb9c831d0a2137d16bbe0514e00a1c8f77f4a

fbshipit-source-id: 798decc90db4f13770e97cdce3c0df7d5421b2a3

fbshipit-source-id: 5113fe0c527595e4227ff827253b7414abbdf7ac

Summary: [Folly] `FOLLY_KEEP` for keeping function definitions when using function-sections even if the linker would remove them.

Reviewed By: alexshap

Differential Revision: D22575832

fbshipit-source-id: 08410b3c318e361b1f3efcbb8f168091faed86e8

Summary: For program running in different general user contexts, e.g. services, multithreading, it's very useful to extend log prefixes to contain the domain specific context strings in debugging. This diff adds one interface function to push one callback to the logger. The callbacks will be executed when the log message data is constructed, and the strings returned by callbacks will be appended to the tail of log entry prefixes.

Reviewed By: simpkins

Differential Revision: D21291284

fbshipit-source-id: e141bf1c907f6730ea741d0dc06975add93d489c

Summary:
`folly::coro::Materialize` uses a `union` of `ManualLifetime` objects to store either the result (value) or the error of an async operation. However, it accesses these fields without ever activating them by in-place constructing the `ManualLifetime` objects. Likewise, when changing the active field, the `ManualLifetime` objects must by destroyed. It doesn't matter that these operations are no-ops. The must be there to avoid UB, which can manifest at higher optimization levels.

I fix this by providing two functions: `activate` and `deactivate`, for use constructing and destructing values in `ManualLifetime` objects when those objects are fields in a `union`.

Reviewed By: yfeldblum, kirkshoop

Differential Revision: D22527895

fbshipit-source-id: 8cc1c3fae75672387d977dddeff61c82093abb9a

Summary:
This diff adds a minimal workflow for running integrations tests for Mononoke. Currently only one test is run and it fails.

This also splits the regular Mononoke CI into separate files for Linux and Mac to match the current style in Eden repo.
There are the "scopeguard::defer" fixes here that somehow escaped the CI tests.
Some tweaks have been made to "integration_runner_real.py" to make it runnable outside FB context.
Lastly the change from using "[[ -v ... ]" to "[[ -n "${...:-}" ]]; in "library.sh" was made because the former is not supported by the default Bash version preinstalled on modern MacOS.

Pull Request resolved: https://github.com/facebookexperimental/eden/pull/26

Reviewed By: krallin

Differential Revision: D22541344

Pulled By: lukaspiatkowski

fbshipit-source-id: 5023d147823166a8754be852c29b1e7b0e6d9f5f

Summary: remove pushmi from folly

Reviewed By: yfeldblum

Differential Revision: D22531090

fbshipit-source-id: 3e6561f220cc3fd227dec321112ec848c4f17c6f

Summary:
Add warning about using the default inline executor for asynchronous reclamation, which may lead to deadlock if there are unintended circular dependencies. The use of inline executor is likely due to not calling folly::enable_hazptr_thread_pool_executor at some earlier point (e.g., due to not calling folly::init()).

Also changed some function names for easy identification and fixed typos.

Reviewed By: yfeldblum

Differential Revision: D22536210

fbshipit-source-id: 89e74411a909ec97b03e8ff90f7039ab233c087b

Summary:
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/27

Pull Request resolved: https://github.com/facebookexperimental/rust-shed/pull/9

Original diffs: D22417488 (https://github.com/facebook/folly/commit/43d80f110b181dd7f72f0f935322a58f9cbb5147), D22528869 (https://github.com/facebook/folly/commit/3930a6ef4437f161c50ac27141cd397eda999fa7)

Reviewed By: markbt

Differential Revision: D22571972

fbshipit-source-id: c6f013565680a757b642dd79e647207fce3351ec

Summary: Fix building folly sysuio portability when targeting the iOS simulator, because the current Xcode 12 SDK does not contain `preadv` and `pwritev` for the iOS simulator on Catalina, even though it declares it's there, so we get link errors.

Reviewed By: yfeldblum

Differential Revision: D22569172

fbshipit-source-id: 870b47063cb37a24364d1de292aba1f966edfb7c

Summary:
This diff adds a constructor that takes an `AsyncSocket*`, so that lifecycle observers are informed via `move` when a socket is transformed.

If one `AsyncSocket` is constructed from another using the `AsyncSocket(AsyncSocket* oldAsyncSocket)` constructor, then `move(oldSocket, newSocket)` will be triggered for attached observers, allowing the observers to detach from the old socket and attach to the new socket. However, we currently transform between `AsyncSocket` by detaching the fd and event base, and passing them to the new constructor:

```
# from FizzAcceptorHandshakeHelper.cpp
auto evb = transport_->getEventBase();
auto fd = transport_->getUnderlyingTransport<folly::AsyncSocket>()
                ->detachNetworkSocket()
                .toFd();
transport_.reset();
sslSocket_ = createSSLSocket(sslContext_, evb, fd);
```

When this happens, any `AsyncSocket::LifecycleObserver` that were attached on accept to become separated from the fd/socket that they're attempting to follow. With this change, we can do the following instead (next diff):

```
 sslSocket_ = createSSLSocket(
      sslContext_, transport_->getUnderlyingTransport<folly::AsyncSocket>());
transport_.reset();
```

Because `createSSLSocket` uses the `AsyncSocket*` constructor, the `move` observer event will be triggered.

Differential Revision: D21614835

fbshipit-source-id: 6c995f879fe41935850247a28ff8af2b33349445

Summary:
Adds `AsyncTransport::LifecycleCallback`, an observer that is notified when a transport is closed and destroyed.

Currently only supported for `AsyncSocket` and derived types (e.g., `AsyncSSLSocket`).

`AsyncSocket::LifecycleCallback` is derived from `AsyncTransport::LifecycleCallback` and adds support for additional lifecycle events relevant to `AsyncSocket`.

Details:
- Can be used by instrumentation that ties its lifetime to that of the transport.
- Intentionally separate from all existing callbacks that may be added / used by application logic because it is designed to be used by instrumentation that is generic, and thus separate / unaware of application logic.
- Multiple observer can be registered, but a `folly::small_vector` is used to minimize alloc overhead in the common case of 0 - 2 being registered.
- The observer implementation is responsible for calling `removeLifecycleObserver` to remove itself if it is destroyed before the socket is destroyed.

Differential Revision: D21613750

fbshipit-source-id: 92bb5de30bc8bab56fa29e62800bf58e47486f1e

Summary:
We have traditionally used `WriteFlags::EOR` to communicate that ACK timestamping should occur. This is confusing and mostly for legacy reasons when we had a custom kernel patch that would send ACK timestamps if the EoR flag was set.

Creating proper flags for ACK and SCHED timestamps. I will integrate the logic that uses these flags into `AsyncSocket` in a subsequent diff.

Differential Revision: D22039799

fbshipit-source-id: 23f534365475bb67f91d86657919cce02439f0c8

Summary:
Socket timestamps (ACK / TX) and EoR tracking currently break for `AsyncSSLSocket` if SSL renegotiation occurs while a timestamped write / EoR write is in progress.

- If EoR tracking is enabled, the EoR flag and any timestamp flags are not included until `AsyncSSLSocket` writes a buffer containing the final byte to the socket. This is to avoid these flags from being set on a partial write of the passed in buffer.
- If a write occurs while an SSL renegotiation is in progress, OpenSSL will return `SSL_ERROR_WANT_WRITE`. When this happens, we need to call write again, passing back in the same buffer.
- The current logic for deciding whether to include the EoR and timestamping flags (`eorAwareSSLWrite`) adds the number of bytes pending to the value returned by `AsyncSSLSocket::getRawBytesWritten` to determine the minimum byte offset when the flags should be added.
  - However, when a write fails due to SSL renegotiation, `getRawBytesWritten` may include some of the bytes that were passed in the last call, despite how they have not actually been written to the transport yet. This is because `getRawBytesWritten` is calculated based on the BIO chain length.
  - As a result, the current logic for calculating the offset on which to add the flags overshoots -- it returns an offset that will never be written. This causes the flags to not be added, and timestamps to timeout.
- This results in one of two things:
  - Timestamp timeouts, where the timestamps are never received
  - If a subsequent write is timestamped, the timestamps from that write may be used instead. This will cause the timestamps to be inflated, and leads to higher TX / ACK times at upper percentiles.

Fix is as follows:
- Change the logic that determines whether the EoR is included in the buffer to no longer rely on `getRawBytesWritten`. In addition, simplify logic so that it is no longer a separate function and easier to make sense of.
- Even if EoR tracking is enabled, always write timestamp flags (TX, ACK, etc.) on every write. This reduces the amount of coordination required between different components. The socket error message handler will end up with more cases of timestamps being delivered for earlier bytes than the last body byte, but they already need to deal with that today due to partial writes.

I considered just outright removing support for EoR tracking (EoR was previously used for timestamping) but decided against this as there's still some value in setting the EoR flag for debugging; see notes in code.

Reviewed By: yfeldblum

Differential Revision: D21969420

fbshipit-source-id: db8e7e5fbd70d627f88f2c43199387f5112b5f9e

Summary:
MSVC upstream decided not to add these builtins in the end, so drop the guards as these are still needed.
Fixes: https://github.com/facebook/folly/issues/1412

Reviewed By: yfeldblum

Differential Revision: D22559891

fbshipit-source-id: 7652da1299d8be7fd64a24f9cffd11b721071d68

Summary: explicitly move exception_ptr arg to fix error

Reviewed By: Orvid

Differential Revision: D22528984

fbshipit-source-id: e72e6008e6a899663970c6dff43c8790d044dd69

Summary:
By using `list::splice`, we can avoid making a call to create and delete nodes. By doing so, only internal pointers are reassigned.

`list::splice` documentation: https://www.boost.org/doc/libs/1_67_0/doc/html/boost/intrusive/list.html#idp55504432-bb

Reviewed By: yfeldblum

Differential Revision: D22369091

fbshipit-source-id: efde0f520f81506b96272b320ae9d65ce1cabcb2

Summary:
When we build boost 1.69.0 with newer version of Xcode, it will fail with:

```
clang: error: unknown argument: '-fcoalesce-templates'
```

This commit fixes this build failure by telling boost's build system that we are building with clang on macOS.

Reviewed By: chadaustin

Differential Revision: D22417488

fbshipit-source-id: 0b3d22835abbba6d06812c656acb0311a60d8c67

Reviewed By: wez

Differential Revision: D22528869

fbshipit-source-id: 66c394b6fafcc45503b593f9f6b0605b5578ce56

Summary:
Fix the documentation errors in Folly when building with Xcode 12. Xcode supports validating doxygen comments by enabling the `-Wdocumentation` flag. In Clang 12, a new feature was added to warn when an inline Doxygen comment has no argument (https://reviews.llvm.org/rL367809).

`\a` and `\b` are special commands in doxygen and were being flagged as errors. I enclosed the example escape sequences within a code block to fix these errors. You can read about all the special doxygen commands here: https://www.doxygen.nl/manual/commands.html

Reviewed By: yfeldblum

Differential Revision: D22488410

fbshipit-source-id: bc4acf31b3df2a860202d0bd1ee356ce8f8fe49f

Summary:
Moveable types should have no-throw move assignment operators. `Executor::KeepAlive<>`'s move assignment operator is not marked `noexcept`. It calls the virtual `Executor::keepAliveRelease`, which is also not marked `noexcept`, despite the fact that virtually (haha) all overrides of that function do nothing more than decrement an atomic and free some resources.

This diff makes the following `noexcept`:
* `Executor::keepAliveAcquire`
* `Executor::keepAliveRelease`
* `KeepAlive::reset()`
* `KeepAlive::operator=(KeepAlive&&)`

 ---

Reviewed By: yfeldblum

Differential Revision: D22505764

fbshipit-source-id: 8e0a04e057c971673cf75da974c1abca2bdf87e8

Summary: If the caller wants a shared_ptr, UniquePtr will implicitly promote.

Reviewed By: yfeldblum

Differential Revision: D22046594

fbshipit-source-id: 2e7d527d3ca33dae974c62db909df605c532ea44

Summary:
Remove the Future-returning form of collectAny completely.

(Note: this ignores all push blocking failures!)

Reviewed By: yfeldblum

Differential Revision: D22345361

fbshipit-source-id: 180bb74c8f64052de372f5d982ad4e77cbff0119

Summary:
`std::function` has a footprint of 32 bytes and (almost) always allocates the interrupt handler. By using an intrusively reference-counted atomic pointer the footprint is just 8 bytes, and we save further 8 bytes by eliminating `interruptHandlerSet_` (it's a `bool`, but poorly aligned). This also allows to share the handler along the continuation chain, instead of copying for every core.

In addition, the `getInterruptHandler()`/`setInterruptHandlerNoLock()` API was replaced by a single `initializeInterruptHandlerFrom()`, so we don't need to expose the internal storage details anymore.

Reviewed By: yfeldblum

Differential Revision: D22474230

fbshipit-source-id: 059828de3b89c25684465baf8e94bc1b68dac0da

Reviewed By: mzlee

Differential Revision: D22495160

fbshipit-source-id: 3d6240906dd086ccac6668d907074ec7ca86ebce

Summary:
The build with Clang 10 on iOS broke with the following errors:

```
folly/detail/MemoryIdler.h:86:68: error: implicit conversion from 'uint64_t' (aka 'unsigned long long') to 'float' may lose precision [-Werror,-Wimplicit-int-float-conversion]
        static_cast<float>(std::numeric_limits<uint64_t>::max()) * h;
                                                                 ~ ^
```

```
folly/detail/MemoryIdler.h:87:38: error: implicit conversion from 'std::__1::chrono::duration<long long, std::__1::ratio<1, 1000000000> >::rep' (aka 'long long') to 'float' may lose precision [-Werror,-Wimplicit-int-float-conversion]
    auto tics = uint64_t(idleTimeout.count() * (1 + extraFrac));
                         ~~~~~~~~~~~~^~~~~~~ ~
```

This diff fixes the problem by making the existing implicit casts explicit.

Reviewed By: Orvid

Differential Revision: D22490364

fbshipit-source-id: 90bd116290de1d8906140d514f1d4880c3b3b085

Summary:
Mark the `await_resume()` method of the `final_suspend()` awaiter as `[[noreturn]]`.

This helps the compiler to dead-code eliminate this code, and in particular helps
the compiler to determine that code cannot continue execution after a
`co_yield co_error(ex)` expression (which calls `final_suspend()`).

This enables us to write code that has a `co_yield co_error()` statement as the
last line in a non-void `Task` coroutine without the compiler emitting a warning
about control running off the end of the coroutine.

Reviewed By: yfeldblum

Differential Revision: D22229020

fbshipit-source-id: b7a63b030cb42653198731d542ffa9bbf90daa83

Summary:
`boost::variant` is an expensive template, and `boost/variant.hpp` is an expensive header. In the one place it is used in futures/detail/Core.h (a commonly-included header), it can be trivially replaced with a `union`, so do so.

As a drive-by, I mark as `noexcept` those members of `KeepAliveOrDeferred` that can be done so unconditionally.

Reviewed By: yfeldblum, kirkshoop, ot

Differential Revision: D22460453

fbshipit-source-id: 5a01f873058273d1a20265507d87796450cc008b

Summary: Adds a strongly discouraged discard helper for cases where we know there is no deferred work.

Reviewed By: yfeldblum

Differential Revision: D22248947

fbshipit-source-id: 26265c33d9cd10848ab730f9c5f0356eb843025b

Summary:
Use sized-deallocation (`sdallocx`) if possible in `folly::SysAllocator` and `folly::Arena`.

`Arena` has always allocated two types of blocks:
1. Normal (fixed-sized): size is the "goodSize adjusted" `minBlockSize`
2. Large (variable-sized): when #1 is too small

Type #2 makes sized-deallocation tricky -- we need somewhere to remember the allocated sizes.

The old code used a single type `Block` and kept a single list. Here I change to have two types and two lists.  The `LargeBlock` has an additional `allocSize` data member.

This makes the Arena object itself 16B larger, but seems better than adding a 4B `allocSize` to each and every block, regardless of type.

Note that, prior to this change, it was possible to `merge()` arenas with different `minBlockSize`.  This is no longer possible.

Reviewed By: davidtgoldblatt

Differential Revision: D22466906

fbshipit-source-id: 719f357a0a1f6cfcda3208391837195c3859ab69

Summary:
Fixes include:
1. Passing "GETDEPS_BUILD_DIR" and "GETDEPS_INSTALL_DIR" env variable and using them in eden/scm/Makefile rather than assuming the source code is always in the same place regardless getdeps arguments (it isn't).
2. Added "fbthrift-source" and "fb303-source" to avoid unnecessary compilation (at least of fb303) and to put fbthrift and fb303 source code in an easy to locate place inside getdeps' "installed" folder.

Pull Request resolved: https://github.com/facebookexperimental/eden/pull/25

Test Plan: sandcastle, check oss-eden_scm-darwin-getdeps

Reviewed By: farnz

Differential Revision: D22431872

Pulled By: lukaspiatkowski

fbshipit-source-id: 8ccbb090713ec085a5dd56df509eb58ab6fb9e34

Summary:
The build with Clang 10 on iOS broke with the following error:

```
folly/test/ConvTest.cpp:321:45: error: expression does not compute the number of elements in this array; element type is 'const char *const', not 'const char *const [4]' [-Werror,-Wsizeof-array-div]
    FOR_EACH_RANGE (i, 0, sizeof(uStrings2) / sizeof(uStrings2)) {
                                 ~~~~~~~~~  ^
```

Looking at nearby uses of `FOR_EACH_RANGE`, it looks like the new compiler has caught a typo, which this diff fixes.

Reviewed By: yfeldblum

Differential Revision: D22470932

fbshipit-source-id: 3090ed9824af488fc429becf2c1084c7725daf5a

Summary:
On windows `<experimental/coroutine>` comes from the Microsoft SDK and uses the MSVC intrinsics. If you are compiling with clang this header exists but nothing from it will work because clang does not implement the MSVC intrinsics.

Further, the clang and MSVC intrinsics are ABI incompatible with each other to the extent that there is no way I can find to implement the MSVC intrinsics behavior as inline functions in a clang TU.

MS have indicated publicly that they will work with the LLVM project to make sure that clang and MSVC will be compatible for `<coroutine>`, but until then we have to say that folly does not have coroutines when clang is used on Windows.

(Note: this ignores all push blocking failures!)

Reviewed By: yfeldblum

Differential Revision: D22301662

fbshipit-source-id: 96b1c3909a3916fe300073af34429eeb5c08d1fd

Summary:
Attempt to set minimum version of an OpenSSL context through `SSL_CTX_set_min_proto_version` instead of disabling them explicitly through options, as recommended here: https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_min_proto_version.html

The test checks that the `SSLContext`'s `SSL_CTX` can keep a lower version regardless of the OpenSSL's configured minimum.

Reviewed By: yfeldblum

Differential Revision: D22338879

fbshipit-source-id: 356328c2ffba2a6a9d4243300a11fc823bc345d7

Summary:
D22371898 (https://github.com/facebook/folly/commit/4981497ad3333ab084e18b2a02a574cbf9438585) introduced a race by attempting to share the storage between `proxy_` and `callback_`: `setProxy()` and `setCallback()` may concurrently try to set both, and there's no good ordering we can use in `setCallback()` to fetch the proxy pointer before constructing the callback.

So revert to the logic pre-D22371898 (https://github.com/facebook/folly/commit/4981497ad3333ab084e18b2a02a574cbf9438585). This means increasing the `Core` footprint by a further 8 bytes, but D22474230 should recover that.

Differential Revision: D22474532

fbshipit-source-id: c63ca6ecd166fb71dcbeddbb2c04eb07494f99ea

Summary: Instead of hard coding the max deferred readers allowed to be 64, statically allocate large enough slots and pick max deferred readers allowed dynamically based on the platform running the service. Specifically, we set the `maxDeferredReaders = 2 * nextPowTwo(numCPU)`, which four times the number of physical cores, to allow faster reads. We are effectively giving each HW thread two slots.

Reviewed By: yfeldblum

Differential Revision: D22407478

fbshipit-source-id: 4001cf96dc502e00f00a27d57c63ba0028a52671

Summary: After D22371898 we can move the `CoreBase` implementation out of the header.

Reviewed By: luciang

Differential Revision: D22371900

fbshipit-source-id: bd4fe6df298c0ba02988b357cd7413eb8f7d8b67

Summary:
`Core<T>` instantiations share a large part of the implementation, but in the current shape the code is duplicated for each instantiation. This diff moves whatever possible to a common base class.

This diff only splits the typed and untyped parts, for ease of review. The untyped parts are moved to the cpp file in D22371900.

Reviewed By: andriigrynenko

Differential Revision: D22371898

fbshipit-source-id: db48c202d0bdcbefbebe150e7c7d9f35e06687d0

Summary:
Fix jemalloc folly weak symbol issue

(Note: this ignores all push blocking failures!)

Reviewed By: davidtgoldblatt

Differential Revision: D22420581

fbshipit-source-id: 136fa6a8c74d8fea3fc96ba1d205266f2f2fa02c