Summary:
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/67

With this change it will be possible to build dependencies of and run integration tests using getdeps.py.

This is the first goal of Q4 as per https://fb.quip.com/v8YzAYNSYgot: "Get Open Source version of integration tests running on Legocastle".

Before this diff:
The OSS integration tests run now on GitHub by:
- Building some test dependencies with getdeps.py
- Building some test dependencies with homebrew/apt-get
- Running tests via python script

The OSS integration tests were not running on Sandcastle.

After this diff:
The OSS integration tests run on Github by:
- Building and executing tests via getdeps.py (execution of tests happens by getdeps.py calling Make calling python script)

The OSS integration tests run on Sandcastle using the same getdeps.py setup as Github.

Reviewed By: krallin

Differential Revision: D24253268

fbshipit-source-id: cae249b72d076222673b8bbe4ec21866dcdbb253

Summary:
# What is the goal?

Address the problem in the post at https://fb.workplace.com/groups/code.indexing/permalink/1929080303917865/ with the comment(s) in Codex for EventBaseManager.

# What was not the way?

Jie Jua noticed that the current comment on "EventBaseManager() {}" in Codex was

```
encouraged. You should instead use the global singleton if possible.
```

# What is the new way?

Use documentation comments (doxygen) style `/** double starts */` instead of `//` ordinary comments. This should show the full comment which is:

```
XXX Constructing a EventBaseManager directly is DEPRECATED and not
encouraged. You should instead use the global singleton if possible.
```

Which really needs that first line.

Differential Revision: D24333522

fbshipit-source-id: 8e84e0a49426844d5a6f6438b0d76fb9886742fe

Summary: The bare name isn't defined on windows at all, so we have to use the new internal name instead.

Reviewed By: yfeldblum

Differential Revision: D24044971

fbshipit-source-id: 24b37903ed0f309910d9f5141240b951bf5fba15

Reviewed By: yfeldblum

Differential Revision: D23545773

fbshipit-source-id: e93f84d0658166e875710a4c0fb5dc165a8628d8

Summary: Migrate AsyncSSLSocketTest to new AsyncSSLSocket session API

Reviewed By: mingtaoy

Differential Revision: D24240249

fbshipit-source-id: ea9bd7669f377c738e44d3348400a2cfbd3c2580

Summary: Use the forkInstrumented call when running under TSAN.

Reviewed By: yfeldblum

Differential Revision: D23787959

fbshipit-source-id: 159f8354ef9746b8374d6da12b22e389c2bc71f4

Summary: [Folly] Fix `is_constexpr_default_constructible` under clang-10, which suffers a crash (https://bugs.llvm.org/show_bug.cgi?id=47620) and divergent behavior in certain constexpr evaluations.

Reviewed By: mpark, Mizuchi

Differential Revision: D24302661

fbshipit-source-id: b6a58b5b675f367bf2dc1d810edb9926cfb87a2e

Summary: TSAN fix for folly/io/test:shutdown_socket_set_test

Differential Revision: D24321155

fbshipit-source-id: 753a808c868da4c608c3ee95d16a3673d6f700b0

Summary: Reduce io_uring test mem requirements

Reviewed By: xiaofeidu008, danobi

Differential Revision: D24314788

fbshipit-source-id: 35ec01475fbc1cf851a81e33cf8c7fec8dd88ddb

Summary: Cache unwrapped observer instead of unwrapping it on every getObserver call.

Reviewed By: jmswen

Differential Revision: D24325934

fbshipit-source-id: d8a023e59c933222a5f43aaacbd9c955236739b3

Summary:
Remove comment about potential high contention under sustained frequent removal from CHM-s.

Recent changes to the hazard pointer library eliminated this problem:
- Asynchronous reclamation of tagged objects (including CHM objects) no longer needs to be done while holding the lock on the list of tagged objects.
- The list of tagged objects is now sharded and sharding can be easily increased if needed.

Reviewed By: yfeldblum

Differential Revision: D24318407

fbshipit-source-id: 3124e8d5651848a16084899a6f31d69cc235c702

Summary:
allow-large-files

black_any_style

Reviewed By: zertosh

Differential Revision: D24325133

fbshipit-source-id: b4afe80d1e8b2bc993f4b8e3822c02964df47462

Summary: Apparently it's flaky...

Differential Revision: D24179337

fbshipit-source-id: 75416adaf8dbd37fc9ddeaa0500ddfaeb80cffa9

Summary: [Folly] Cut linter suppression in `TryTest.cpp`.

Reviewed By: zertosh

Differential Revision: D24294536

fbshipit-source-id: 64b26ec4632ffdae8e246aa27a6c58c579e2db10

Summary: Move folly binary

Reviewed By: yfeldblum

Differential Revision: D24294709

fbshipit-source-id: e7cf0be6e45d3a132fc81143103f3b5cdedbb76f

Summary:
This is best effort. We cannot control how many tests are running
at the same time. And if the user is running other tests that consumes resource

Differential Revision: D24289056

fbshipit-source-id: 0810ba0989e84a5bce6d695dd6df0dfe2618fbf7

Summary:
Gmock's `Return(x)` is a valid action as long as `x` is implicitly convertible to the return type of the function.

The current implementation of `CoReturn(x)` does not support that, since `Task<T>` is not convertible to `Task<Q>` even if `Q` is convertible to `T`.

This diff changes the `CoReturn` implementatio to match the way Gmock does, allowing for the implicit conversion.

This is extremely valuable when using `Task<Expected<...>>`, as it allows to write `CoReturn(makeUnexpected(...))` instead of `CoReturn(Expected<...>(unexpected, ...))`.

It also make the translation from a mock being sync to async more straightforward: add `Task` in the mock return type and replace `Return` with `CoReturn`.

Reviewed By: lewissbaker

Differential Revision: D24046013

fbshipit-source-id: 9f166237a624600be51c00616f145ae4e1a45443

Summary:
This diff adds all third party dependencies that are required by getdeps to be able to build and runn Mononoke's integration tests.

Also add a stub Makefile with no-op steps that will be filled in next diff.

Reviewed By: ahornby

Differential Revision: D24251894

fbshipit-source-id: 67384ecfd0ced6762dddc3c6e61feb1240b1162d

Summary:
This specifically fixes a TSAN detected race where SemiFuture destructor was checking Core state to see if deferred executor has to be detached, while that state could become Empty if the callback is being proxied and both threads code be racing while accessing the executor.
hasCallback() checks if the callback was ever set (not if the callback is stil alive) and it's true for State::Empty (similar to State::Done).

Differential Revision: D24271346

fbshipit-source-id: 5f52f20783d36745d1200497e0300bc086b65213

Summary: Add support for UDP RX timestamps

Reviewed By: mjoras

Differential Revision: D24146914

fbshipit-source-id: 381839fc5402f13e428364c47c5752473acda6af

Summary: Disable CLANGTIDY checks for several places at the code where HOWTOEVEN checks are disabled.

Reviewed By: suo

Differential Revision: D24198276

fbshipit-source-id: c52cd10941252fcacfea30589660c57eb7099011

Summary:
Pull Request resolved: https://github.com/facebook/folly/pull/1472

Disable CLANGTIDY checks for several places at the code where HOWTOEVEN checks are disabled.

Reviewed By: suo

Differential Revision: D24197524

fbshipit-source-id: e99cfe1a7440a275edd63643ad819ca9e2a07788

Summary:
- Folly FiberManager makes the assumption that asan always has dlfcn.h
   available.
   - This assumption is not true on windows.
 - This diff would mean that windows asan would not use:
   - `__sanitizer_start_switch_fiber`
   - `__sanitizer_finish_switch_fiber`
   - `__asan_unpoison_memory_region`
 - The behavior of `GetProcAddress` on windows is sufficiently different to `dlsym` to make switching to it without significant testing unappealing.

(Note: this ignores all push blocking failures!)

Reviewed By: andriigrynenko

Differential Revision: D24167183

fbshipit-source-id: 2587a4000901b5e3a8857e9aa5816270d2be34dc

Summary:
ThreadCachedInts uses asymmetric barriers to heavily optimize the reader side. TSAN does not support (symmetric or asymmetric) barriers. In this particular case, this diff replaces the memory order for accessing increments and decrements with memory_order_seq_cst only when running under TSAN, and that, I think, is a valid alternative to the barriers because it ensures a total order between all atomic accesses to increments and decrements (so that, when we read all decrements before all increments, we make sure that any writes to increments are read as long as decrements are read, which seems to be the purpose of the barriers).

Introduced additional tests to RcuTest and to AtomicReadMostlyMainPtrTest. The problem with existing tests is that this only triggers TSAN when the counter was already 0 when calling synchronize(), not when synchronize() had to invoke futexWait, because futexWait is itself a synchronization point.

Reviewed By: davidtgoldblatt

Differential Revision: D24029131

fbshipit-source-id: 57c7bf32061868ccd5a4e20154c8c7db6e4eeef5

Summary: Adds a form of detach that is safe to use in recursive code. Detaches a task after a delay, and checks for availability of the global executor at that point to decouple shutdown from the delayed detach.

Reviewed By: andriigrynenko

Differential Revision: D24174495

fbshipit-source-id: e6b22485a817c18319c897fd51c3f8ba81d13041

Summary:
Changes:
- Make the tagged list sharded.
- Manage a common count for both tagged and untagged cohort objects.
- Combine timed asynchronous reclamation for both tagged and untagged cohort objects.
- Integrate checking the threshold for asynchronous reclamation with managing timed asynchronous reclamation.
- Combine the asynchronous reclamation of both tagged and untagged objects.

Reviewed By: davidtgoldblatt

Differential Revision: D24136818

fbshipit-source-id: 8df1b8eeef1df7f14b5d68ed5ad82d0459a381f5

Differential Revision:
D24046011 (https://github.com/facebook/folly/commit/99dbd7aae8a57a477a1b3db2466dfe8db7157142)

Original commit changeset: 91f52198242c

fbshipit-source-id: 2041d1f6017fa833ff7bb8ac5c9adc2d3fe80553

Reviewed By: zertosh

Differential Revision: D24188387

fbshipit-source-id: a540cf6db895dda48723492aa843bf32d66011f3

Summary: Disable CLANGTIDY checks for several places at the code.

Differential Revision: D24046011

fbshipit-source-id: 91f52198242ceaa87e923a43a3dbe6610efe4e49

Summary:
This test was written in a way that relied on an internal OpenSSL implementation
detail -- that OpenSSL would invoke the `handshakeVer` callback on the
root certificate.

OpenSSL commit https://github.com/openssl/openssl/commit/e2590c3a162eb118c36b09c2168164283aa099b4, which
is part of OpenSSL 1.1.1h, alters the control flow of the X509_verify routine
such that the handshakeVer callback is no longer called on self-signed
certificates in the trust store (aka CA certificates).

The purpose of this test was to ensure that a forced failed verification on the
end entity certificate would elicit a particular behavior. This diff adjusts
the implementation to match the original intention and removes the reliance
on implementation detail specifics.

Differential Revision: D24183002

fbshipit-source-id: abc8337f76d3529966d276cae2337ad136456199

Summary:
Defines new data-structures folly::AsyncStackFrame and folly::AsyncStackRoot
that can be used to build intrusive data-structures that represent
asynchronous call-chains.

These data-structures can be walked at either runtime from within the process
or by external tooling, such as debuggers or profilers, allowing them to capture
stack-traces that correctly attribute the calls to asynchonous callbacks to the
asynchronous caller that initiaded the operation rather than to the immediate
caller of the callback (typically an executor's event-loop).

These data-structures are initially intended to be used by folly::coro coroutines
to allow chains of these coroutines to be walked, but should also general enough
to be applied to other types of asynchronous operations, such as futures or even
asynchronous operations from other languages (eg. Rust).

Reviewed By: andriigrynenko

Differential Revision: D21118885

fbshipit-source-id: 0d6130b40fd04ef330800eae4feb93bb79f48105

Summary: It turns out that perf_buffer API is only available in libbpf 0.2.0, which hasn't been released yet. To unblock katran, I grabbed the latest libbpf revision and created a temporary manifest. To avoid relying on this "beta" version libbpf on other projects only katran uses it. I'll make sure to remove it in favor of the official libbpf 0.2.0 once it's released.

Reviewed By: anakryiko

Differential Revision: D24156655

fbshipit-source-id: 32f6e04079a862fbfe96fd5475678cfa4ae1b3db

Summary:
Adding support for de-structuring the Proxy object inside a for loop, this will let users write code that looks like the following while still supporting the existing interface.

```
for (const auto& [index, element] : folly::enumerate(collection)) {
    std::cout << index << ": " << element;
}
```

Also: update an existing unittest for c++17

Please let me know if there's anything I'm failing to consider/you disagree with in this change and I'll do my best to make it right.

Reviewed By: yfeldblum, ot

Differential Revision: D23948526

fbshipit-source-id: 6da83c9e9e0e911e33c6febc08ad2fd5b06498eb

Summary:
AsyncBase::submit() is racy: it calls submitOne() and then later calls op->start(). But
op can be completed between return from submit and call to start(). In this case a CHECK
will fail somewhere, either because we see op as COMPLETED in start(), or because we
see op as INITALIZED in complete().

Call start() to put ops into PENDING state *before* issuing them; add "unstart" method
to put them back into INITIALIZED state when submit fails.

In passing, fix bug: if submit submits fewer ops than expected, decrement pending count
properly.

Differential Revision: D24094495

fbshipit-source-id: 2508dbd7ba5d306ceba34d62471d957f6cb20ff5

Summary: Fix for a zerocopy AsyncSocket issue

Reviewed By: kevin-vigor

Differential Revision: D24083612

fbshipit-source-id: 607c3daac882c6515bc9d031a84cd3efb8736177

Summary: Time to update libbpf version (the latest release is now 0.1.1).

Reviewed By: udippant

Differential Revision: D24063680

fbshipit-source-id: 715ac74e9671f0f8ed5b8fe9174fe4070fc0f991

Summary: Add flags to set the io_uring mlock rlimit

Reviewed By: danobi, kevin-vigor

Differential Revision: D24063687

fbshipit-source-id: be7cdad10797c087a21370ce42811a9752e4f61a

Summary:
Evidently unlike folly, headers may be present in cmake even if they are excluded from
build.

Check that we have required headers before including AsyncIO.h or IoUring.h.

Differential Revision: D24117788

fbshipit-source-id: fec359c8298a21575a8714dd2abdbd9d4dc3e5c4

Reviewed By: jmswen

Differential Revision: D24102346

fbshipit-source-id: 9bb59bbce04285f93e9d6bb03e04841eaf10a882

Summary: Refactory `throw` -> `throw_exception`;

Reviewed By: yfeldblum

Differential Revision: D21773804

fbshipit-source-id: fd59f1579a143db385e05c5e3442fedee243d25f