Summary: Original commit changeset: 2d4ad67a4a01

Reviewed By: yfeldblum

Differential Revision: D23064000

fbshipit-source-id: 88d8d550772c2045f61d5a355cf33c9d8df7d046

Summary: Found because I saw the error message

Reviewed By: magedm

Differential Revision: D23019298

fbshipit-source-id: 6744e9e6d0c21cf76db756f68b3e948a044e6888

Summary: [Folly] Fix TSAN-reported race in `ShutdownSocketSet` test where the shutdown does not wait for the server.

Reviewed By: yfeldblum

Differential Revision: D23057822

fbshipit-source-id: 4b682e6e80cff503e85cb2b663f8399b69265a77

Summary:
The folly::fibers GDB extension, specifically the `info fibers` command,
depended on a specific format of the std::unique_ptr pretty printer
output. This pretty printer format changed between GCC 7 and GCC 9, with
the latter omitting the target address. However, the new pretty printer
does expose an iterator for the child target address that we can use
instead.

Differential Revision: D23063364

fbshipit-source-id: 6125edaba9abcfd0b7a9c0741a18854637c97804

Summary:
`std::allocator<void>::allocate` is not invocable as policy. Let folly allocators behave the same.

Switch existing sites to parameterizing the allocators over `char` instead, since `std::allocator<char>::allocate` is invocable.

Reviewed By: yfeldblum

Differential Revision: D22986960

fbshipit-source-id: ca1ff3cb1785029b6b6f59ddc4046b1b105d8bd6

Summary:
The hinted forms of insert_or_assign were forwarding with
std::move rather than std::forward, which has the potential to elevate
a non-const lvalue ref to an rvalue ref.

(Note: this ignores all push blocking failures!)

Reviewed By: yfeldblum

Differential Revision: D21475865

fbshipit-source-id: e91e759762cf6f045c843cd7de1402953e6afb61

Summary: Original commit changeset: 8d5e96feea4d

Reviewed By: yfeldblum

Differential Revision: D23046629

fbshipit-source-id: e24c4a0ba5f40a584646a602f618e881dbf0f344

Summary:
basic_fbstring unfortunately just ignores the allocator. Switching it to use std::basic_string.

This is necessary because ElfCache must be async-signal-safe, but basic_fbstring ignores the allocator and always uses standard malloc/free, which are not safe async-signal-safe.

Reviewed By: yfeldblum

Differential Revision: D23035304

fbshipit-source-id: 7b4e77a3764701366556a5fe5a352f952b052ed4

Summary:
These are all test setup issues:
 - EventBase needs to be destructed before objects that callbacks may refer to.
 - A socket should not be operated on by a thread that is different from the event base it is associated with (in this case, the socket was being closed on the RSA computation thread).

Reviewed By: yfeldblum

Differential Revision: D23043485

fbshipit-source-id: 733868317aef09e32e79169aedb84a988de3bb41

Summary: Increase coverage of blocking operation logging.

Reviewed By: yfeldblum

Differential Revision: D22933413

fbshipit-source-id: e820927ddafa0bb5f2d50dbb1dceeeb2e89133b3

Summary: A doc comment in folly/Poly.h makes reference to a non-existent entity "`Decay`" and is missing namespace qualifiers, making the example confusing and wrong.

Reviewed By: Alfus, yfeldblum

Differential Revision: D23038865

fbshipit-source-id: 50f515b31919552e11ae18ff645ef71812d77bfb

Summary: [Folly] Simplify `reentrant_allocator` hierarchy while preserving presence/absence rules of `allocate` and `dealllocate`. Apply same rules to `max_size` and `address`. Plus some assorted tweaks. Plus some comments.

Reviewed By: luciang

Differential Revision: D23016816

fbshipit-source-id: 601318ea112193afcffc93d6783dbd79168a164e

Summary: [Folly] Add `exception_wrapper::from_exception_ptr` taking by `&&`, permitting callers to move, since `std::exception_ptr` copy is expensive and `std::rethrow_exception` takes by value.

Reviewed By: ericniebler, markisaa

Differential Revision: D23016839

fbshipit-source-id: c07695fef0828a50426fe4e2b40fae9582c7ffd9

Summary:
This failed on mode/opt, because the `- sh_addr + sh_offset` was actually using the address and offset of the *symbol table section*, not the *data section* (where the kStringValue variable actually is). It still worked, because for mapped sections, the `offset - address` just happens to be the same for all sections, so it didn't matter what section we used; in debug mode, we use dynamic linking, so the symbol section is called .dynsym and is loaded in memory, but in opt mode, the symbol section is called .symtab and is *not* loaded, so sh_addr is zero, and that formula no longer works.

The goal there was really just to subtract the binary base address encoded in the ELF file. ElfFile already provides this via getBaseAddress(), and that's also how we can identify whether the loaded ELF (in this case, the same as the current executable) is PIE; so switched both the calculation and the if condition to use this result.

Reviewed By: yfeldblum

Differential Revision: D23020998

fbshipit-source-id: f7e66bb554c62aa13d7d2fc1017cb31542507be9

Summary: Avoid traversing the callbacks map twice when inserting a non-existing entry

Reviewed By: kevin-vigor

Differential Revision: D23030404

fbshipit-source-id: afbf7564f4039ffbd942f407429874fde795025d

Summary: Fix Singleton TSAN issue (mutex used after being destroyed)

Reviewed By: yfeldblum

Differential Revision: D22925120

fbshipit-source-id: e25c113cdef7076c93a189281f5b9d842f576dd3

Summary: Add definition to EventBase.

Differential Revision: D23004586

fbshipit-source-id: 828767b3d9a43c33bcf047b9b06c08bfba8fde68

Reviewed By: andriigrynenko

Differential Revision: D23008623

fbshipit-source-id: db3e5a3dfe8dbbf1c9583b8e4924da31c2f5c174

Summary:
The implementation of `std::basic_string` with the legacy ABI and with `_GLIBCXX_FULLY_DYNAMIC_STRING`, as appears by default on CentOS 7, incorrectly requires the provided allocator type to have a default constructor. The `Allocator` concept does not require this and many allocator types with runtime state shared between copies will not provide this.

Switch to `folly::fbstring` in `ElfCache` to sidestep this problem.

Fixes: https://github.com/facebook/folly/issues/1344.

Reviewed By: yfeldblum

Differential Revision: D22970418

fbshipit-source-id: 1e188d668991d25f53da5702b31f4a7a7ec7d7e8

Summary: (Note: this ignores all push blocking failures!)

Reviewed By: andriigrynenko

Differential Revision: D23005233

fbshipit-source-id: 4ea528505e06fec0af7841ccbd6d41eabe0de558

Summary:
A tracepoint semaphore address is written to the notes section as either the absolute address (if binary is compiled as non-PIE) or the relative address (if binary is compiled as PIE) of the semaphore. In the former case, the test passes; in the latter case, the test as it was could not possibly pass because the binary is mapped to some random base address.

This diff makes a guess: if the addresses are equal, then we're non-PIE; otherwise check if the address offset is valid - that offset is the address at which the binary is mapped, so it always begins with the ELF header, so we can check against that.

If someone knows how to easily figure out that random address or to detect whether the binary is PIE (thereby disabling the test) that'd be great too.

This fixes TSAN for the test, as TSAN binaries are built as PIE.

Reviewed By: yfeldblum

Differential Revision: D22752824

fbshipit-source-id: cd320f9fb8355a20f65e2385f0a1ab1c24ec0b81

Summary:
Previously `OpenSSLUtils::getCommonName` was allocating a 64 byte long string and copying the common name from the `X509` into it. We unexpectedly returned that string potentially containing trailing `\0`s.

This diff rewrites the function to allocate a big enough buffer, write to it, then return an appropriately trimmed new string from it.

Reviewed By: yfeldblum

Differential Revision: D22910431

fbshipit-source-id: 77b4fc3ccde4f7ddb0f62b884ecd7ff0c868c1ca

Reviewed By: yfeldblum, andriigrynenko

Differential Revision: D22965140

fbshipit-source-id: 8f9996ccc28aff621879774e901b0c75e8c62883

Summary: Make hazptr_obj_cohort active_ atomic to be data-race-free.

Reviewed By: yfeldblum

Differential Revision: D22976574

fbshipit-source-id: 5a09aa7cedc26c71b2160b2df14a7e0f7b55e2e7

Summary:
The deps for Fedora in the public README are a little out of date.  Add /
update some.

Reviewed By: yfeldblum

Differential Revision: D22951514

fbshipit-source-id: 645af3df1b71dcdab06408d73d99c34c482ea534

Summary:
When we do the context switch onto the fiber, the stack frame points back to
the main context stack. While the fiber is running, this allows backtraces to
continue back onto the main context. When the fiber is suspended, we execute
code on the main context's stack, the pointers to this stack are now garbage,
backtracing in gdb runs into problems, such as garbage frames, gdb
crashes, and occasional cycles.

The fix here is to write a 0 to the saved return address in the last frame, when a fiber suspends.
gdb detects this and stops the trace
  #38 folly::fibers::FiberManager::activateFiber(folly::fibers::Fiber*) (this=0x0, fiber=0x0) at folly/fibers/FiberManagerInternal-inl.h:82
  #39 folly::fibers::FiberManager::runReadyFiber(folly::fibers::Fiber*) (this=0x0, fiber=0x0) at folly/fibers/FiberManagerInternal-inl.h:127
  #40 0x0000000000000000 in  ()

I gated the fix to linux since I'm not sure about the ABI for other platforms.

Reviewed By: yfeldblum

Differential Revision: D22774189

fbshipit-source-id: 2d4ad67a4a01bebc4ff1f9c285f6e3e95d13432c

Summary:
- Move to the newer and forward moving libzstd-dev
https://packages.ubuntu.com/search?suite=default&section=all&arch=any&keywords=libzstd&searchon=names

Reviewed By: wez

Differential Revision: D22981784

fbshipit-source-id: 3647357aad60f2335ea494c35c174e9ffec61346

Summary: The find path reads a node that the insert path writes; they need to have acquire/release semantics due to the node being allocated on the fly.

Reviewed By: yfeldblum, magedm

Differential Revision: D22937394

fbshipit-source-id: c1497ed1764cd43e396bc520e9b716cdfe29e5ca

Summary:
Redirecting stdout means that ninja/cmake won't act as if it's invoked
interactively, ie: it will buffer the output, show every single files being
compiled (instead of just a line or progress), etc. This results in a fairly
janky UX when getdeps is used on the command line. By not redirecting stdout,
we get immediate feedback about the tests being run, and the files being
compiled.

Reviewed By: wez

Differential Revision: D22967815

fbshipit-source-id: 872ddbf421065686c384a3a876d0acb8832baf2e

Summary: [Folly] Prefer `std::condition_t` vs preprocessor in `AsyncSocket`.

Reviewed By: bschlinker, lnicco

Differential Revision: D22975614

fbshipit-source-id: 562d1b794ec3dbab538ad507b2ec5d0fe07c7757

Summary: [Folly] Let `exceptionStr` peek into unknown exception types when building with libstdc++ via `std::exception_ptr::__cxa_exception_type()`.

Reviewed By: ot, luciang

Differential Revision: D22521812

fbshipit-source-id: b75811ed4159df81e651c5df1386c406f6ac6d60

Summary: Generic code taking in a `Task<void|T>` cannot easily forward the result without throwing exceptions because the `return_value` overload taking a `Try` cannot be added to `Task<void>` which already has `return_void`. Instead we use `yield_value` as we did with `co_error` to make the desired generic code possible.

Reviewed By: lewissbaker

Differential Revision: D22934887

fbshipit-source-id: ec50de5832f9038d92749bdde2c1d43244288a3c

Summary: We know ahead of time the size of this list, so may as well pre-allocate space for it

Reviewed By: yfeldblum

Differential Revision: D22838412

fbshipit-source-id: 7d70495fe0910e65dfea49fe7baf675308ab9331

Summary: If the binary is compiled as PIE, such as in the case of TSAN, the kStringValue symbol's value, which is an address that in turn points to the "coconuts" character array, may be stored as 0 in the binary file, and rely on the dynamic linker to relocate at loading time. The corresponding relocation entry (in the .rela.dyn section) contains the relative offset instead. We do not seem to have implemented parsing of the relocation section, so this diff changes the test to be more lenient, and upon seeing a zero will just verify the address of the symbol against the live binary.

Reviewed By: yfeldblum

Differential Revision: D22782080

fbshipit-source-id: ffbfc7c23b84865fb29bbfb782feb896d687594b

Summary: Add the ability to set the event EventRecvmsgCallback

Reviewed By: danobi

Differential Revision: D22959687

fbshipit-source-id: 274cecf4ab4b8015d5e7e725c7e899f603cea7a0

Summary: [Folly] Add missing include in `folly/ExceptionString.cpp`.

Reviewed By: ot, luciang

Differential Revision: D22529764

fbshipit-source-id: 91af74023fd81b1882a19889fd92fda4e1bdfcc7

Summary:
Two issues:
 - Executor not properly shutdown before destruction
 - Shadow stack overflow due to TSAN not handling fibers correctly. For this we just reduce the number of tasks for now; the proper fix would be likely quite involved and it's not clear whether it's worth it at this point since I think it only affects diagnostic messages that TSAN prints.

Reviewed By: yfeldblum

Differential Revision: D22901657

fbshipit-source-id: 833454e6f54ab6fc7c53333157197345e8d887e2

Summary:
Environment variable WITH_ASAN can be set to enable building
 with ASAN options.

 Ran into issues with getdeps.py extensions P137131160.
 Once that is resolved will do away with the ENV approach

 Requires building folly lib also with ASAN to avoid linking issues

Reviewed By: shri-khare

Differential Revision: D22860508

fbshipit-source-id: 48dea49378319972894f69b11bac6fa63ee20db9

Summary: This spin lock is not required, and guards quite a complex region that could easily fallback to the slow path

Reviewed By: andriigrynenko

Differential Revision: D22839268

fbshipit-source-id: 045a3562a2d4358b4c95e155a4b9958b2b7f67d4

Summary:
Two ordering issues:

setTreeNode cannot store the node as relaxed, because the node is newly allocated before storing, and if another thread gets the node pointer value, relaxed ordering cannot guarantee that the node has been fully initialized before it.

There are two ways this node can be read, one via optimisticReadValue, which loads with memory_order_acquired, which would now correctly synchronize with the release; the other is readValue, which I think was intended to be used only when the lock corresponding to the node is held, but there is one case on line 781 in forceInsert where we fail to acquire the lock but it looks like we want to retry unless we're out of capacity (on a best effort basis) and therefore we call readValue to check the capacity (on a best effort basis), so it looks as if memory_order_relaxed is okay, but it is not, because the node is a pointer that may not be fully initialized with relaxed ordering. So I'm changing that to memory_order_acquire as well.

In general it's not really valid to use relaxed ordering if the value is a pointer and the pointee is allocated on the fly.

Reviewed By: yfeldblum

Differential Revision: D22936002

fbshipit-source-id: 1c8a000b6033e4a81df5f9a5e850cfe350dbeb15