Summary:
Without this, `folly::gen`'s other `byLine()` implementations only
support a single read pass (usually through the whole input file) without
support for different formats through the file.

Reviewed By: yfeldblum

Differential Revision: D9540396

fbshipit-source-id: 6539851a70aed9a88d3c61986a459f60dfb7be5b

Summary:
Update readFull() and related functions to warn if their return value is
ignored.  Callers should pretty much always need to examine the return value,
as they need this to determine how much valid data was read.

Reviewed By: yfeldblum, meyering

Differential Revision: D9526760

fbshipit-source-id: c090044256bfe4bf9907a24ec6090eeae04296b3

Summary:
Files opened by `readFile()` should not be propagated to children processes
if another thread happens to spawn a subprocess while `readFile()` is running.

Reviewed By: yfeldblum, meyering

Differential Revision: D9553354

fbshipit-source-id: 2a6028b3a920bb1e76cd0b07f82e6af57d01c2a4

Summary:
A `CustomLogFormatter` can be enabled by setting the `formatter` option to `custom`. This formatter reads an option named `log_format` which specifies the format of the logs. The format string is similar to what `folly::vformat(format,...)` expects. Check out the following example format:

  {L}{m:02d}{D:02d} {H:02d}:{M:02d}:{S:02d}.{USECS:06d} {THREAD:5d} {FILE}:{LINE}]

The format above prints logs in the same format as GLOGs.

Another option `colored` takes in one of the following values: `never`, `auto`, and `always`. When enabled, the output will be colored in the following way:

{F135233848}

The config used for the image above is:
  FOLLY_INIT_LOGGING_CONFIG(".=DBG; default:formatter=custom,log_format={L} {FILE}::{FUN}():{LINE}],colored=true");

As you can see, we have also included the function name using `{FUN}`. This can be particularly helpful when debugging errors.

Reviewed By: simpkins

Differential Revision: D8974950

fbshipit-source-id: 87d624f53dd67c0630fb222093c46fe79ff44ee1

Summary:
F14NodeMap is a safe drop-in replacement for std::unordered_map. It is smaller
than std::unordered_map (32 bytes instead of 56 bytes) and has been shown
to be faster in many production use cases. This diff replaces the hash table
inside folly::dynamic. The effect is that sizeof(folly::dynamic) will drop from 64
bytes to 40 bytes, and use cases that use folly::dynamic as an object may also
get a CPU win.

Reviewed By: nbronson

Differential Revision: D9518110

fbshipit-source-id: 72daba13577c560bbb88c4d68a457afedf1af38e

Summary:
Adds a new InlineTask coroutine type that is executor unaware.

*Execution Contexts*
The InlineTask's coroutine will start execution on the execution context of the coroutine that awaits it. When the InlineTask coroutine awaits something internally, it will resume execution on whatever execution context the operation it awaited on completes on. The coroutine awaiting the InlineTask will be resumed on whatever execution context the operation completed on.

It has been put in the `folly::coro::detail` namespace for now to discourage usage outside of the operators exposed from `folly::coro` as it can be easily misused to accidentally run logic on the wrong execution context.

This type is based around the same design as the `std::experimental::task<T>` that was proposed in [P1056R0](http://wg21.link/P1056R0).

Reviewed By: andriigrynenko

Differential Revision: D9388154

fbshipit-source-id: bc94a9bf07459410ef423b4cfd3b57d13688f744

Summary:
Add a generic function for performing a blocking-wait on any awaitable object.

This obsoletes the use of the `.wait()` and `.get()` methods on  `folly::coro::Future<T>` and so these methods have now been removed.

Reviewed By: andriigrynenko

Differential Revision: D9384837

fbshipit-source-id: bcff5b348c024a3fc9337c67d996c447e0723fcc

Summary:
* Contract for Future::hasValue() had a typo - fixed that.
* Contract for Future::hasException() was inconsistent with hasValue() - made it consistent.

Differential Revision: D9527398

fbshipit-source-id: 93e1ebd768fa38476949818c0902e06a7dab5209

Summary:
Save python import cost in folly/python get_fiber_manager.

In my testing, using call_once here is safe even in multi-threaded and forked environment.
Though I wouldn't be surprised if a corner case is found in the future, as I couldn't find a definite answer about the safety of memoization Python C-API import calls.

Reviewed By: fried

Differential Revision: D9542909

fbshipit-source-id: c62a08ebf67bd4eb82f1bf4b7657c4a3a6daf4db

Summary:
The current futex API required a reference to a futex object in order to
invoke `futexWake()`, although this is not buggy by itself, it is technically
UB and nothing is stopping ASAN from catching this sort of use-after-free and
reporting it as an error.  Especially when the futex is represented as a
pointer, requiring a dereference to reach a member function

The bug can come up when you call `futexWake()` on a futex that has been
destroyed, for example

```
auto&& futex_ptr = std::atomic<Futex<>*>{nullptr};
auto&& thread = std::thread{[&]() {
  auto&& futex = Futex<>{0};
  futex_ptr.store(&futex);

  while (futex.load(std::memory_order_relaxed) != 1) {
    futex.futexWait(0);
  }
}};

while (!futex_ptr.load()) {}
futex_ptr.load()->store(1);
futex_ptr.load()->futexWake(1);
thread.join();
```

Here immediately after the `store(1)`, our thread could have loaded the value,
seen that it had changed, and never went to sleep.  Or it could have seen the
value as 0, went to sleep and immediately returned when it saw that the value
in the futex word was not what was expected.

In the scenario described above calling `futexWake()` is done on a "dangling"
pointer.  To avoid this, we just never dereference the pointer, and pass the
pointer to the futex syscall, where it will do the right things

A side benefit to the refactor is that adding specializations is very easy.
And we don't have to mess with member function specializations anymore, which
are inherently hard to work with (eg. cannot partially specialize)

The ADL extension points (currently implemented for `Futex<std::atomic>`,
`Futex<DeterministicAtomic>` and `Futex<EmulatedFutexAtomic>`) are

```
int futexWakeImpl(FutexType* futex, int count, uint32_t wakeMask);

FutexResult futexWaitImpl(
    FutexType* futex,
    uint32_t expected,
    std::chrono::system_clock::time_point const* absSystemTime,
    std::chrono::steady_clock::time_point const* absSteadyTime,
    uint32_t waitMask);
```

Reviewed By: yfeldblum

Differential Revision: D9376527

fbshipit-source-id: bb2b54e511fdf1da992c630a9bc7dc37f76da641

Summary:
Add the BZIP2 stream codec.

The `FlushOp::FLUSH` does not guarantee that the decompressor can read all the input processed so far, due to a bug in the bzip2 library. This is likely not important, since `FLUSH` is not a common operation, especially with bzip2.

Reviewed By: yfeldblum

Differential Revision: D9484325

fbshipit-source-id: 40770b6f301a16d86c4de8c2b0875f931f00cba2

Summary:
F14 library relies on the dependent targets to set the neon-related compiler
flags consistently. As it turns out, this is difficult for non-aarch64
platforms where neon availability is not guaranteed.

Hence, for safety (so that we don't cause SIGILL on older generation Android
devices), we restrict the neon version of F14 to aarch64.

Reviewed By: Maratyszcza, nbronson

Differential Revision: D9519198

fbshipit-source-id: cd0c92b17403bce9156a915ad96020f565ca3d6a

Summary: Initialize `std::atomic<std::thread::id> SingletonHolder::creating_thread_`.

Reviewed By: yfeldblum

Differential Revision: D9525334

fbshipit-source-id: 93831c40bed21d1f3356bd2181e78ba08b7b116f

Add folly::tryEmplaceWith() helper for safely initialising a Try<T> with result of calling a function

Summary:
Adds a new folly::tryEmplaceWith() function that is the equivalent of makeTryWith() but for in-place initialisation of an existing Try<T> object.

Using `tryEmplaceWith(t, func)` rather than `t = makeTryWith(func)` can potentially be more exception safe since the latter could potentially still throw if `Try<T>::operator=()` throws (ie. if T's move-constructor can throw).

Updated FiberManager's internals to make use of this to fix a potential exception-safety issue that could have caused a memory leak if the Try<T> move-constructor of the result of fiber threw an exception.

Reviewed By: yfeldblum

Differential Revision: D9511067

fbshipit-source-id: 827cc88ea1d4fcd505c8d8a9886b87b26db90544

Summary: Fix folly/io/async/test:async_test - AsyncSSLSocketTest.ConnectTFOFallbackTimeout SIGSEGV

Reviewed By: djwatson

Differential Revision: D9520104

fbshipit-source-id: c1133bcb0617ca7f8d5f8a9ee46080c7c7f5b105

Summary:
Add some methods to `folly::Try<T>` to allow the user to in-place construct a value or exception_wrapper inside the Try object at some point after the Try object was constructed.

Previously, you had to construct a new temporary `Try<T>` object from the value and move-assign the value.

The difference between `t.emplace(args...)` and `tryEmplace(t, args...)` is that the latter catches any exceptions thrown by the constructor and stores the exception in the Try object whereas the former lets the exception propagate out to the caller and leaves the Try object in an empty state.

The `Try<void>` implementation now only conditionally constructs the contained exception_wrapper only if hasValue_ is false. This was needed to be able to have the `.emplaceException()` method correctly in-place construct the `exception_wrapper` object.

Reviewed By: yfeldblum

Differential Revision: D9352439

fbshipit-source-id: f34a4479acb2f93aed11f9626b3152511386bfea

Summary: Init the std::atomic since in the default constructor is "no initialization takes place"

Reviewed By: yfeldblum, djwatson

Differential Revision: D9515004

fbshipit-source-id: cdc73c490f77496335e77732b0c7c43c180a54b6

Reviewed By: lewissbaker

Differential Revision: D9354535

fbshipit-source-id: 49c228e773f21a2fcfa4701eec4cb4dc76c2f990

Summary:
This is part of "the great r-valuification of folly::Future":
* This is something we should do for safety in general.
* Several of folly::Future's methods are lvalue-qualified even though they act as though they are rvalue-qualified, that is, they provide a postcondition that says, in effect, callers should act as though the method invalidated its `this` object (regardless of whether that invalidation was actual or logical).
* This violates the C++ principle to "Express ideas directly in code" (see Core Guidelines), and generally makes it more confusing for callers as well as hiding the actual semantics from tools (linters, compilers, etc.).
* This dichotomy and confusion has manifested itself by some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* The goal of rvalueification is to make sure methods that are logically rvalue-qualified are actually rvalue-qualified, which forces callsites to acknowledge that rvalueification, e.g., `std::move(f).onTimeout(...)` instead of `f.onTimeout(...)`. This syntactic change in the callsites forces callers to acknowledge the method's rvalue semantics.

Reviewed By: LeeHowes

Differential Revision: D9441979

fbshipit-source-id: a2f3416bdf481d32d1b94cde1bb713e1afe8ab67

Summary:
If the value copy/move constructor threw an exception during copy/move assignment then the Try<T> object would not be left in the empty state.

This could lead to double-deletion of resources, or referencing uninitialised memory.

Reviewed By: yfeldblum

Differential Revision: D9425118

fbshipit-source-id: 6639a0903258ecb031c26264f418fa8794519be3

Summary:
This is part of "the great r-valuification of folly::Future":
* This is something we should do for safety in general.
* Several of folly::Future's methods are lvalue-qualified even though they act as though they are rvalue-qualified, that is, they provide a postcondition that says, in effect, callers should act as though the method invalidated its `this` object (regardless of whether that invalidation was actual or logical).
* This violates the C++ principle to "Express ideas directly in code" (see Core Guidelines), and generally makes it more confusing for callers as well as hiding the actual semantics from tools (linters, compilers, etc.).
* This dichotomy and confusion has manifested itself by some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* The goal of rvalueification is to make sure methods that are logically rvalue-qualified are actually rvalue-qualified, which forces callsites to acknowledge that rvalueification, e.g., `std::move(f).within(...)` instead of `f.within(...)`. This syntactic change in the callsites forces callers to acknowledge the method's rvalue semantics.

Reviewed By: LeeHowes

Differential Revision: D9442319

fbshipit-source-id: c751a5e295620bff22c8bdaed57d417bcc0973d1

Summary:
Makes it invalid to call SemiFuture::defer with a lambda that takes no arguments.

Fix to callsites in folly tests that checked this behaviour.

Reviewed By: yfeldblum

Differential Revision: D9511831

fbshipit-source-id: 11e600cea78887c653e3d0d0e1cd2091c4eb191e

Summary:
This is part of "the great r-valuification of folly::Future":
* This is something we should do for safety in general.
* Several of folly::Future's methods are lvalue-qualified even though they act as though they are rvalue-qualified, that is, they provide a postcondition that says, in effect, callers should act as though the method invalidated its `this` object (regardless of whether that invalidation was actual or logical).
* This violates the C++ principle to "Express ideas directly in code" (see Core Guidelines), and generally makes it more confusing for callers as well as hiding the actual semantics from tools (linters, compilers, etc.).
* This dichotomy and confusion has manifested itself by some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* The goal of rvalueification is to make sure methods that are logically rvalue-qualified are actually rvalue-qualified, which forces callsites to acknowledge that rvalueification, e.g., `std::move(f).onError(...)` instead of `f.onError(...)`. This syntactic change in the callsites forces callers to acknowledge the method's rvalue semantics.

Reviewed By: yfeldblum

Differential Revision: D9441402

fbshipit-source-id: 1c39ca7b33bb406d2fd3a995f487693a2d013747

Summary:
F14 previously had a space optimization that relied on the fact
that tuple<A,B,C> was std::is_empty if each of A, B, and C was empty
(and non-final), but this is not the case on libc++.  This diff removes
the use of std::tuple and manually packages the hash table functors to
enable the Empty Baseclass Optimization.

Reviewed By: yfeldblum

Differential Revision: D9507243

fbshipit-source-id: 3ee8aa2aa444d91de56210662c86e555406d8eca

Summary: We may want to disable exceptions, so lets make this work

Reviewed By: yfeldblum

Differential Revision: D9381726

fbshipit-source-id: 6e6d022740728a40a5b710ec32309da99f6d0bd0

Summary:
This doesn't need to be in Memory.h and is one of the reasons Memory.h fails to compile with -fno-exceptions, so split it out

I include EnableSharedFromThis.h because I'm not sure how else to ensure backward compatibility any other way (and I have the macro so we can include Memory.h without including EnableSharedFromThis.h)

Reviewed By: yfeldblum

Differential Revision: D9385709

fbshipit-source-id: bf2c3a757ae4eefba69e6191309d64b347becf12

Summary:
This is part of "the great r-valuification of folly::Future":
* This is something we should do for safety in general.
* Several of folly::Future's methods are lvalue-qualified even though they act as though they are rvalue-qualified, that is, they provide a postcondition that says, in effect, callers should act as though the method invalidated its `this` object (regardless of whether that invalidation was actual or logical).
* This violates the C++ principle to "Express ideas directly in code" (see Core Guidelines), and generally makes it more confusing for callers as well as hiding the actual semantics from tools (linters, compilers, etc.).
* This dichotomy and confusion has manifested itself by some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* The goal of rvalueification is to make sure methods that are logically rvalue-qualified are actually rvalue-qualified, which forces callsites to acknowledge that rvalueification, e.g., `std::move(f).within(...)` instead of `f.within(...)`. This syntactic change in the callsites forces callers to acknowledge the method's rvalue semantics.

Codemod changes:

* expr.within(...) ==> std::move(expr).within(...)  // if expr is not already an xvalue
* expr->within(...) ==> std::move(*expr).within(...)

Note: operator precedence of that last step is safe - no need to parenthesize `expr`. Reason: `->` binds more tightly than unary `*`.

Reviewed By: yfeldblum

Differential Revision: D9511943

fbshipit-source-id: 0cf90e3198453dd997194c2c7f36cc1fa74906f5

Summary:
[Folly] `invoke_noreturn_cold`, a helper function for invoking another function with arguments in a `[[noreturn]]` and `[[gnu::cold]]` context.

Potentially useful for making the bytecode at the call-site small for icache size or always-inlined fast-path size concerns, when constructing the exception object to be thrown is more complex than passing pre-existing values by reference, such as when computing a message is required.

Reviewed By: marshallcline

Differential Revision: D9509658

fbshipit-source-id: b8272629876f9e7e81ae0f7a7f06404484d297f2

Summary:
This is part of "the great r-valuification of folly::Future":
* This is something we should do for safety in general.
* Several of folly::Future's methods are lvalue-qualified even though they act as though they are rvalue-qualified, that is, they provide a postcondition that says, in effect, callers should act as though the method invalidated its `this` object (regardless of whether that invalidation was actual or logical).
* This violates the C++ principle to "Express ideas directly in code" (see Core Guidelines), and generally makes it more confusing for callers as well as hiding the actual semantics from tools (linters, compilers, etc.).
* This dichotomy and confusion has manifested itself by some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* The goal of rvalueification is to make sure methods that are logically rvalue-qualified are actually rvalue-qualified, which forces callsites to acknowledge that rvalueification, e.g., `std::move(f).semi()` instead of `f.semi()`. This syntactic change in the callsites forces callers to acknowledge the method's rvalue semantics.

Reviewed By: LeeHowes

Differential Revision: D9442704

fbshipit-source-id: 50b1912e33ac23d1a693682fbcd55cd637cdb149

Summary: [Folly] Cut use of boost headers in `sorted_vector_types.h`.

Reviewed By: elsteveogrande

Differential Revision: D9508079

fbshipit-source-id: 46950a07d1fbc0e316bd6a8cb9766f5d0aeaa74b

Summary: [Folly] Fix `folly/hash/detail/Crc32CombineDetail.cpp` with `-Werror=attributes`.

Reviewed By: tehnerd

Differential Revision: D9507179

fbshipit-source-id: 67ce235a5e1555973a276de2b4009acab2329ddc

Summary:
[Folly] Compute the crc32c and crc32 powers tables at compile time v.s. hardcoding the tables.

We translate `gf_multiply_sw` to C++11 `constexpr` form for those cases which still require gcc49.

Reviewed By: djwatson

Differential Revision: D9495241

fbshipit-source-id: 39aa8b53f7bd8a6dec1ba49013636c57972ef93b

Summary:
Modify the implementations of thenValue and thenError to be in terms of thenImplementation rather than then.

This is a preliminary transformation before splitting then and deprecating parts of it in turn.

Reviewed By: marshallcline

Differential Revision: D9494305

fbshipit-source-id: 38cd165866f98514ee506dc01f625b7df1db211b

Summary:
This is part of "the great r-valuification of folly::Future":

* This is something we should do for safety in general.
* Several of folly::Future's methods are lvalue-qualified even though they act as though they are rvalue-qualified, that is, they provide a postcondition that says, in effect, callers should act as though the method invalidated its `this` object (regardless of whether that invalidation was actual or logical).
* This violates the C++ principle to "Express ideas directly in code" (see Core Guidelines), and generally makes it more confusing for callers as well as hiding the actual semantics from tools (linters, compilers, etc.).
* This dichotomy and confusion has manifested itself by some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* The goal of rvalueification is to make sure methods that are logically rvalue-qualified are actually rvalue-qualified, which forces callsites to acknowledge that rvalueification, e.g., `std::move(f).thenMulti(...)` instead of `f.thenMulti(...)`. This syntactic change in the callsites forces callers to acknowledge the method's rvalue semantics.

Reviewed By: LeeHowes

Differential Revision: D9437627

fbshipit-source-id: f62dc095ce57904057a97c83c958de5227a112a8

Differential Revision: D9499697

fbshipit-source-id: b04e9e0d1bf4abf1b5ff497cefbc26da0176eaa2

Summary: In some cases, it may be necessary to unicode-escape regular ASCII characters in JSON serialization (example: for JSON sent to browsers that may be interpreted as HTML, "<" should be escaped). Allow additional escape characters to be specified via a bitmap in the serializer options.

Reviewed By: yfeldblum, luciang

Differential Revision: D8980189

fbshipit-source-id: 000c5279ab0f37a3ee4b2eb38f20afa49dcc5a27

Summary:
Problem:
- There are several tests enabled which rely on using
  shared_ptr_internals found in
  folly/concurrency/detail/AtomicSharedPtr-detail.h and
  the implementation only works for libstdc++ and not with other vendors
  such as libc++ and libcpp.
- This results in a hard error when compiling with Clang with libc++.

Solution:
- Add a CMake configure check to detect whether one is using libstdc++.
  FOLLY_USE_LIBSTDCPP macro is set to 1 if using libstdc++, 0 otherwise.
- Use FOLLY_USE_LIBSTDCPP to conditionally run the tests relying on
  shared_ptr_internals only when using libstdc++.

Note:
- A longer-term solution would be to implement the similar
  shared_ptr_internals with the libc++ types and provide a fallback
  locking approach for libcpp (MSVC standard library implementation).

Pull Request resolved: https://github.com/facebook/folly/pull/900

Reviewed By: djwatson

Differential Revision: D9312236

Pulled By: yfeldblum

fbshipit-source-id: 657eddc1e530b00bd5f5b10e7155bc3b6aac3726

Summary: [Folly] Cut unnecessary uses of `Super::` in `F14Policy.h`.

Reviewed By: shixiao

Differential Revision: D9478746

fbshipit-source-id: ea5b57a66e265ea459a66ff3756a2a54326e73f2

Summary:
This is part of "the great r-valuification of folly::Future":
* This is something we should do for safety in general.
* Several of folly::Future's methods are lvalue-qualified even though they act as though they are rvalue-qualified, that is, they provide a postcondition that says, in effect, callers should act as though the method invalidated its `this` object (regardless of whether that invalidation was actual or logical).
* This violates the C++ principle to "Express ideas directly in code" (see Core Guidelines), and generally makes it more confusing for callers as well as hiding the actual semantics from tools (linters, compilers, etc.).
* This dichotomy and confusion has manifested itself by some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* The goal of rvalueification is to make sure methods that are logically rvalue-qualified are actually rvalue-qualified, which forces callsites to acknowledge that rvalueification, e.g., `std::move(f).thenMulti(...)` instead of `f.thenMulti(...)`. This syntactic change in the callsites forces callers to acknowledge the method's rvalue semantics.

Reviewed By: LeeHowes

Differential Revision: D9437767

fbshipit-source-id: f40d0660a50ccfec1040e42c8028efde956a4b48