Summary:
Pull Request resolved: https://github.com/facebook/folly/pull/1472

Disable CLANGTIDY checks for several places at the code where HOWTOEVEN checks are disabled.

Reviewed By: suo

Differential Revision: D24197524

fbshipit-source-id: e99cfe1a7440a275edd63643ad819ca9e2a07788

Summary:
- Folly FiberManager makes the assumption that asan always has dlfcn.h
   available.
   - This assumption is not true on windows.
 - This diff would mean that windows asan would not use:
   - `__sanitizer_start_switch_fiber`
   - `__sanitizer_finish_switch_fiber`
   - `__asan_unpoison_memory_region`
 - The behavior of `GetProcAddress` on windows is sufficiently different to `dlsym` to make switching to it without significant testing unappealing.

(Note: this ignores all push blocking failures!)

Reviewed By: andriigrynenko

Differential Revision: D24167183

fbshipit-source-id: 2587a4000901b5e3a8857e9aa5816270d2be34dc

Summary:
ThreadCachedInts uses asymmetric barriers to heavily optimize the reader side. TSAN does not support (symmetric or asymmetric) barriers. In this particular case, this diff replaces the memory order for accessing increments and decrements with memory_order_seq_cst only when running under TSAN, and that, I think, is a valid alternative to the barriers because it ensures a total order between all atomic accesses to increments and decrements (so that, when we read all decrements before all increments, we make sure that any writes to increments are read as long as decrements are read, which seems to be the purpose of the barriers).

Introduced additional tests to RcuTest and to AtomicReadMostlyMainPtrTest. The problem with existing tests is that this only triggers TSAN when the counter was already 0 when calling synchronize(), not when synchronize() had to invoke futexWait, because futexWait is itself a synchronization point.

Reviewed By: davidtgoldblatt

Differential Revision: D24029131

fbshipit-source-id: 57c7bf32061868ccd5a4e20154c8c7db6e4eeef5

Summary: Adds a form of detach that is safe to use in recursive code. Detaches a task after a delay, and checks for availability of the global executor at that point to decouple shutdown from the delayed detach.

Reviewed By: andriigrynenko

Differential Revision: D24174495

fbshipit-source-id: e6b22485a817c18319c897fd51c3f8ba81d13041

Summary:
Changes:
- Make the tagged list sharded.
- Manage a common count for both tagged and untagged cohort objects.
- Combine timed asynchronous reclamation for both tagged and untagged cohort objects.
- Integrate checking the threshold for asynchronous reclamation with managing timed asynchronous reclamation.
- Combine the asynchronous reclamation of both tagged and untagged objects.

Reviewed By: davidtgoldblatt

Differential Revision: D24136818

fbshipit-source-id: 8df1b8eeef1df7f14b5d68ed5ad82d0459a381f5

Differential Revision:
D24046011 (https://github.com/facebook/folly/commit/99dbd7aae8a57a477a1b3db2466dfe8db7157142)

Original commit changeset: 91f52198242c

fbshipit-source-id: 2041d1f6017fa833ff7bb8ac5c9adc2d3fe80553

Reviewed By: zertosh

Differential Revision: D24188387

fbshipit-source-id: a540cf6db895dda48723492aa843bf32d66011f3

Summary: Disable CLANGTIDY checks for several places at the code.

Differential Revision: D24046011

fbshipit-source-id: 91f52198242ceaa87e923a43a3dbe6610efe4e49

Summary:
This test was written in a way that relied on an internal OpenSSL implementation
detail -- that OpenSSL would invoke the `handshakeVer` callback on the
root certificate.

OpenSSL commit https://github.com/openssl/openssl/commit/e2590c3a162eb118c36b09c2168164283aa099b4, which
is part of OpenSSL 1.1.1h, alters the control flow of the X509_verify routine
such that the handshakeVer callback is no longer called on self-signed
certificates in the trust store (aka CA certificates).

The purpose of this test was to ensure that a forced failed verification on the
end entity certificate would elicit a particular behavior. This diff adjusts
the implementation to match the original intention and removes the reliance
on implementation detail specifics.

Differential Revision: D24183002

fbshipit-source-id: abc8337f76d3529966d276cae2337ad136456199

Summary:
Defines new data-structures folly::AsyncStackFrame and folly::AsyncStackRoot
that can be used to build intrusive data-structures that represent
asynchronous call-chains.

These data-structures can be walked at either runtime from within the process
or by external tooling, such as debuggers or profilers, allowing them to capture
stack-traces that correctly attribute the calls to asynchonous callbacks to the
asynchronous caller that initiaded the operation rather than to the immediate
caller of the callback (typically an executor's event-loop).

These data-structures are initially intended to be used by folly::coro coroutines
to allow chains of these coroutines to be walked, but should also general enough
to be applied to other types of asynchronous operations, such as futures or even
asynchronous operations from other languages (eg. Rust).

Reviewed By: andriigrynenko

Differential Revision: D21118885

fbshipit-source-id: 0d6130b40fd04ef330800eae4feb93bb79f48105

Summary: It turns out that perf_buffer API is only available in libbpf 0.2.0, which hasn't been released yet. To unblock katran, I grabbed the latest libbpf revision and created a temporary manifest. To avoid relying on this "beta" version libbpf on other projects only katran uses it. I'll make sure to remove it in favor of the official libbpf 0.2.0 once it's released.

Reviewed By: anakryiko

Differential Revision: D24156655

fbshipit-source-id: 32f6e04079a862fbfe96fd5475678cfa4ae1b3db

Summary:
Adding support for de-structuring the Proxy object inside a for loop, this will let users write code that looks like the following while still supporting the existing interface.

```
for (const auto& [index, element] : folly::enumerate(collection)) {
    std::cout << index << ": " << element;
}
```

Also: update an existing unittest for c++17

Please let me know if there's anything I'm failing to consider/you disagree with in this change and I'll do my best to make it right.

Reviewed By: yfeldblum, ot

Differential Revision: D23948526

fbshipit-source-id: 6da83c9e9e0e911e33c6febc08ad2fd5b06498eb

Summary:
AsyncBase::submit() is racy: it calls submitOne() and then later calls op->start(). But
op can be completed between return from submit and call to start(). In this case a CHECK
will fail somewhere, either because we see op as COMPLETED in start(), or because we
see op as INITALIZED in complete().

Call start() to put ops into PENDING state *before* issuing them; add "unstart" method
to put them back into INITIALIZED state when submit fails.

In passing, fix bug: if submit submits fewer ops than expected, decrement pending count
properly.

Differential Revision: D24094495

fbshipit-source-id: 2508dbd7ba5d306ceba34d62471d957f6cb20ff5

Summary: Fix for a zerocopy AsyncSocket issue

Reviewed By: kevin-vigor

Differential Revision: D24083612

fbshipit-source-id: 607c3daac882c6515bc9d031a84cd3efb8736177

Summary: Time to update libbpf version (the latest release is now 0.1.1).

Reviewed By: udippant

Differential Revision: D24063680

fbshipit-source-id: 715ac74e9671f0f8ed5b8fe9174fe4070fc0f991

Summary: Add flags to set the io_uring mlock rlimit

Reviewed By: danobi, kevin-vigor

Differential Revision: D24063687

fbshipit-source-id: be7cdad10797c087a21370ce42811a9752e4f61a

Summary:
Evidently unlike folly, headers may be present in cmake even if they are excluded from
build.

Check that we have required headers before including AsyncIO.h or IoUring.h.

Differential Revision: D24117788

fbshipit-source-id: fec359c8298a21575a8714dd2abdbd9d4dc3e5c4

Reviewed By: jmswen

Differential Revision: D24102346

fbshipit-source-id: 9bb59bbce04285f93e9d6bb03e04841eaf10a882

Summary: Refactory `throw` -> `throw_exception`;

Reviewed By: yfeldblum

Differential Revision: D21773804

fbshipit-source-id: fd59f1579a143db385e05c5e3442fedee243d25f

Summary: [Folly] Backport more of `is_detected`.

Reviewed By: Mizuchi

Differential Revision: D23984149

fbshipit-source-id: 1e1ed3b0cf3e54999ac56aae11e7d5f0288cae4e

Summary: small bug

Differential Revision: D24080766

fbshipit-source-id: bb03739014023aa98b55859d433f88149529e5e7

Summary: The only thing FOLLY_USE_SYMBOLIZER guards now is whether to install signal handlers, and we can do that on any unix.

Reviewed By: yfeldblum, luciang

Differential Revision: D23881199

fbshipit-source-id: 45631b59fe5df9656a227b68683f23d4ff65e223

Summary: Add API set (but not get) an OpenSSL session. This is intended to handle the situation where the user already has a TLS1.3 resumable session. For example, this currently occurs when the user obtains a session through wangle's SSLSessionCallbacks.

Reviewed By: mingtaoy

Differential Revision: D22285536

fbshipit-source-id: 7ad8be501909dbb80ab74624991415e74441ef6e

Summary: See title

Reviewed By: yfeldblum

Differential Revision: D22046592

fbshipit-source-id: 289828fbecb83e086d75156a8696541dd3f7f194

Summary: This doesn't do anything anymore and is going to be removed in D23993306, let's remove it here first.

Reviewed By: yns88

Differential Revision: D23993954

fbshipit-source-id: 4d7dd5f992e34be7a0da16ce7cf59810407649c4

Summary: hitting https://github.com/google/googletest/issues/1373

Reviewed By: mjoras

Differential Revision: D23785392

fbshipit-source-id: 114849ed966fb196bb8392bd71ee3c2972834279

Summary:
Move the reclamation of tagged objects outside the critical section on the tagged list lock (in asynchronous reclamation of the domain tagged list).

Tagged objects are subject to both synchronous and asynchronous reclamation. Each tagged object belongs to a cohort.

Currently, asynchronous reclamation of the domain list of tagged objects reclaims tagged objects while holding the lock on the list, so that synchronous reclamation of a cohort's objects does not miss such objects.

This diff adds a singly linked list of objects that are safe to reclaim to the cohort structure. Asynchronous reclamation pushes reclaimable tagged objects into the safe lists in their respective cohorts, instead of reclaiming such objects while holding the lock on the list of tagged objects. By pushing objects to their respective cohorts, synchronous reclamation can find not-yet-reclaimed objects in the cohort's own safe list.

Reviewed By: davidtgoldblatt

Differential Revision: D23973107

fbshipit-source-id: 2a9758cc8b03bb98a45b5579bab2964d9ae3d60c

Summary:
Hello,

It seems I did a mistake in PR https://github.com/facebook/folly/issues/1328

CMake find_package directive looks for Find<PackageName>.cmake.
There is a mistake in the folly-deps.cmake leading to:

CMake Error at CMake/folly-deps.cmake:248 (find_package):
  No "Findfmt.cmake" found in CMAKE_MODULE_PATH.

Pull Request resolved: https://github.com/facebook/folly/pull/1462

Reviewed By: yfeldblum

Differential Revision: D24000731

Pulled By: Orvid

fbshipit-source-id: 2239750228f134776b57c794d04f65f69a92ed3a

Summary:
Hello,

Adding data in those variables can have an impact upon feature detections.

For instance, using 'check_cxx_source_compiles' to detect libatomic can fail because the link command uses -rdynamic and the list of shared objects. But as libcrypto needs zlib which is not added in CMAKE_REQUIRED_LIBRARIES, the test will fail, as we can see here (from CMakeError.log, modified for readability):

```
$ g++  -DNDEBUG -O3 -flto -ffat-lto-objects -fuse-linker-plugin -std=gnu++20 -fno-working-directory -ggdb3 -Wno-error=maybe-uninitialized -Wno-error=shadow=compatible-local -Wno-error=array-bounds -Wno-error=stringop-overflow -DFOLLY_CPP_ATOMIC_BUILTIN  -std=gnu++20  -DNDEBUG -O3 -Wl,-flto -ffat-lto-objects -fuse-linker-plugin -fno-working-directory -ggdb3 -L /path/to/mylibs  -pthread -lgflags -lglog -levent -levent_core -ldouble-conversion   -rdynamic CMakeFiles/cmTC_4a1a2.dir/src.cxx.o  -o cmTC_4a1a2  /path/to/mylibs/libgflags.so  /path/to/mylibs/libssl.so  /path/to/mylibs/libcrypto.so  -ldl
ld: warning: libz.so.1, needed by /path/to/mylibs/libssl.so, not found (try using -rpath or -rpath-link)
ld: /path/to/mylibs/libcrypto.so: undefined reference to `deflateInit_'
ld: /path/to/mylibs/libcrypto.so: undefined reference to `deflate'
ld: /path/to/mylibs/libcrypto.so: undefined reference to `deflateEnd'
ld: /path/to/mylibs/libcrypto.so: undefined reference to `inflate'
ld: /path/to/mylibs/libcrypto.so: undefined reference to `inflateInit_'
ld: /path/to/mylibs/libcrypto.so: undefined reference to `inflateEnd'
ld: /path/to/mylibs/libcrypto.so: undefined reference to `zError'
```
I only give the example of the detection of libatomic but this error prevents the probing mechanism from detecting other features like the presence of ASN1_TIME_diff function or elf.h.

I propose to avoid setting things in those variables or, if it needs to be modified, clean it up after it was useful.

Pull Request resolved: https://github.com/facebook/folly/pull/1463

Reviewed By: yfeldblum

Differential Revision: D24000700

Pulled By: Orvid

fbshipit-source-id: ace1b5b5dee6362c4a30ab0a986984005e7dc569

Summary: This file contains a wrapper that is protected from `basename` fn name collision between `libiberty.h` and `libgen.h` (0fb6287719abb3cf52d7eda457efbe567985d1d4). I ran into this same issue again when trying to use `libiberty`'s `cplus_demangle_v3`, so let's add a similar wrapper for this function.

Reviewed By: fjricci

Differential Revision: D24008824

fbshipit-source-id: 868e90579f753f0c5b62aaeb27af2ef0a2006155

Summary:
Add SimpleAsyncIO class. This creates an instance of an AsyncIO
and manages a completion thread and a pool of ops, so user does
not have to worry about any of these details.

Differential Revision: D23945993

fbshipit-source-id: 90f9b83ca3447a19e3bc4aaf2a2f196ee4ea614d

Summary: If retries is 0, then `0 < 0` is false, meaning we will skip the `while` loop completely and just try to read retcode, which was never assigned.

Reviewed By: fanzeyi

Differential Revision: D23999523

fbshipit-source-id: fac4a1104eba3585fb52fc8d83163cb1a87b8fee

Summary: This diff adds a utility `withJitter` that creates an observer proxying updates from the input observer, with the difference that each update will be propagated with a random lag.

Reviewed By: andriigrynenko

Differential Revision: D23795834

fbshipit-source-id: cdaee7460c3c346e70f6c8c78f18467fe8cd4e47

Summary: Use separate b_, e_ range members

Reviewed By: yfeldblum

Differential Revision: D23983016

fbshipit-source-id: b17061d7c1022c4776a8a8cb76ad5d0b9da9ace2

Summary: [Folly] A few simplifying tweaks to SFINAE in `Function`.

Reviewed By: aary

Differential Revision: D23798698

fbshipit-source-id: e5a3ecfe755f6eb371c253cfb925a23d4f5d29e4

Summary:
In the sync from fbsource to ovrsource, we found that signal_set macro from libevent
has a conflict with boost::asio::signal_set function in boost_context.

e.g. failure https://www.internalfb.com/intern/sandcastle/job/18014398839678371/

This undef removes the build failures on the ovrsource side.

Reviewed By: yfeldblum

Differential Revision: D23941679

fbshipit-source-id: 25e208bdbc93f289d787ca742623fd6815863705

Summary:
Currently, Hardware.h assumes the `__rdtsc` intrinsic is always available when building with MSVC. However, this is not the case when building for Windows for ARM64.

This blocks successful compilation of https://github.com/facebook/rocksdb on Windows for ARM64, since that library embeds Folly and includes this specific header.

Instead, use a condition similar to the check in `asm_volatile_pause` in Asm.h. This results in successful compilation and a fallback to `steady_clock::now()` on Windows for ARM64 targets.

Pull Request resolved: https://github.com/facebook/folly/pull/1461

Reviewed By: yfeldblum

Differential Revision: D23952622

Pulled By: Orvid

fbshipit-source-id: 1f02d04cbbbde42c80ad56d8d7ccc4dd7fe86847

Summary: It seems after updating zstd to 1.4.5. `Dllexport` for zstd.dll was not being picked up correctly. Instead of having zstd being a runtime dependency let's try statically link it to avoid the DLL issue.

Reviewed By: vitaut

Differential Revision: D23970349

fbshipit-source-id: 3b14dddb64d410cb9546c416f27d73b7604b21ba

Summary:
Fix race condition in hazptr_obj_cohort deactivation

Race condition:
- Thread T1 is doing asynchronous reclamation of objects.
- Thread T2 starts deactivating cohort C.
- T1 reclaims an object that its destructor retires a tagged object A that belongs to cohort C.
- T1 checks that C is active.
- T2 sets C.active_ to false.
- T2 pops and reclaims the objects in C's list.
- T1 pushes A in C's list
- T2 expects to find C's list empty but it is not. Assertion failure.

With the fix.
- Thread T1 is doing asynchronous reclamation of objects.
- Thread T2 starts deactivating cohort C.
- T1 reclaims an object that its destructor retires a tagged object A that belongs to cohort C.
- T1 checks that C is active.
- T2 sets C.active_ to false.
- **T2 waits for T1 to release the lock on the domain tagged list**
- T1 pushes A in C's list
- **T1 releases the lock**
- **T2 cleans up C's objects in the domain tagged list**
- **No subsequent asynchronous reclamation of objects in the domain will involve objects that belong to C**
- T2 pops and reclaims the objects in C's list **including A**
- T2 finds C's list empty as expected.

Reviewed By: yfeldblum

Differential Revision: D23947490

fbshipit-source-id: 943664e61190da7b4ad1bb31ac47609bdea2314d

Summary: Simple typo that I spotted.

Differential Revision: D23963553

fbshipit-source-id: 018e81c4878f4cb27b6676e88278d47d38706ed4