Summary: [Folly] Fix a regression in per-task allocations in `SerialExecutor`.

Reviewed By: andriigrynenko

Differential Revision: D15589007

fbshipit-source-id: 4461798435833cb317b41cfea932762eb7ac8662

Summary: [Folly] Cut outdated comment in `F14Table`, referring to the return value of a function which returns `void`.

Reviewed By: nbronson

Differential Revision: D15586980

fbshipit-source-id: b75f7b989c39b52e589ce8051663085201939433

Summary: [Folly] Move `SerialExecutor` test-specific helper types into the tests which need them.

Reviewed By: andriigrynenko

Differential Revision: D15587721

fbshipit-source-id: 603e5381b2e00b4317f7e9836c6102238158c5d0

Summary: Pull Request resolved: https://github.com/facebook/folly/pull/1142

Reviewed By: shixiao

Differential Revision: D15574540

Pulled By: yfeldblum

fbshipit-source-id: 2ce91f7510f0b278677c15b3a9f1f80b43ff8998

Summary:
SerialExecutor can not rely on parent to save RequestContext for him as it is not guaranteed that an enqueued function will be executed from the parent callback that was scheduled at the same time.
This diff captures RequestContext in a lambda and sets it correctly

Reviewed By: shixiao

Differential Revision: D15381223

fbshipit-source-id: 9b3ab1f96e57eb5e697efb52105d80c2899a03e7

Summary: Lock elision requires non-mutating functions that check if lock acquisition could succeed. This diff adds such functions to folly::SharedMutex, including eligible_for_lock_elision(), eligible_for_lock_upgrade_elision(), and eligible_for_lock_shared_elision(). The diff also adds assertions to validate these functions' correctness under single-threaded executions.

Reviewed By: nbronson

Differential Revision: D15545398

fbshipit-source-id: 0037d473c9dd360f7143ea4c4c9092fb9bb7f5f9

Summary: Acquiring a lock into an unused temporary is generally a bug. Anyone wanting to do this can suppress their compiler's warning by explicitly `static_cast`ing the lock holder to void.

Reviewed By: vitaut

Differential Revision: D15504563

fbshipit-source-id: 1033f6a1d4044b9698934b73eefebfe83602a2a5

Summary:
[Folly] Use a M:N caches-to-lifetimes in `SingletonThreadLocal`.

Sometimes the compiler will not merge cache and lifetime thread-local variables in expected ways, so be resilient. More details in #1135 and https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90562.

At the same time, move the tracking into the wrapper. The lifetime was an intrusively-linked-list node which was clever and which was allocation-free but which increased the TLS requirement per instantiation of `SingletonThreadLocal`. Moving the tracking into the wrapper decreases the TLS requirement, although the tracking now requires heap allocations.

Fixes #1135.
Closes #1138.

Reviewed By: andriigrynenko

Differential Revision: D15508024

fbshipit-source-id: a344e8dd52bd52724ad85b0fc88d6a2a349952ac

Summary: The cacheline padding in folly/test/SharedMutexTest is not actually functional since the compiler is free to rearrange and optimize away local variables. This diff fixes the cacheline padding by aligning the false-sharing critical values.

Reviewed By: nbronson

Differential Revision: D15523241

fbshipit-source-id: c51b1e4cd78ab33f75fa93faf0a0346ec602f97c

Summary:
[Folly] Move executor to local var immediately in `doCallback`, and always move it into the callback.

(Note: this ignores all push blocking failures!)

Reviewed By: andrewcox

Differential Revision: D15532854

fbshipit-source-id: 971a6a64ad3c47c9f5e9f75907a568f9d59b1000

Summary:
- Replace some calls sites from `NDEBUG` to `kIsDebug`.
- Not every call site is changed; there are a few useful cases of
  `NDEBUG` still. `xlog` macros are one of them. Another is where we
  conditionally define a variable in a local scope and then use it in an
  `assert`.
Pull Request resolved: https://github.com/facebook/folly/pull/1128

Reviewed By: aary

Differential Revision: D15490848

Pulled By: yfeldblum

fbshipit-source-id: 4f2d1689f90baa25d3ba9e99a2a388d9cfc43f18

Summary: [Folly] Cut extra `return` statement in `Core<T>::doCallback`.

Reviewed By: Orvid

Differential Revision: D15527963

fbshipit-source-id: 9396f9ac2b246676b33d13c87e24dd42198a11ec

Summary:
[Folly] Template fibers baton timed-wait over the deadline types, rather than over the timeout types and rather than fixing a single timeout type.

Templating it over the deadline types permits specifying the clock and permits passing a single deadline through unchanged through multiple time-delayed layers of code without skew, and templating it rather than fixing it over a single timeout type permits caller-specified fine-grained or coarse-grained timings.

Currently, the `EventBase` timers do not parameterize over the deadline types, so the fiber version of timed-wait must for now convert to a fixed timeout type.

Reviewed By: andriigrynenko

Differential Revision: D15448177

fbshipit-source-id: aa3fdbffdcb83cbfd1571780ba431a1004beb347

Summary: it seems since D15286181, we updated docker os_image to ubuntu18 and gcc7. this diff changes the README.md to reflect the change.

Reviewed By: yfeldblum

Differential Revision: D15513580

fbshipit-source-id: 0a3518c21c912903921946850bd08c948f09c295

Summary:
[Folly] Disable TLS optimization in PIC mode for `SingletonThreadLocal`.

In PIC mode, TLS offsets must be translated to addresses at runtime via calls to `__tls_get_addr`. Since there is not much of a small inline path or a fast inline fast path anymore anyway in PIC mode, might as well skip the bulky slow caching.

Also, some versions of gcc do not properly support this code in shared libraries. Details in #1135 and https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90562.

Fixes #1135.

Reviewed By: andriigrynenko

Differential Revision: D15468086

fbshipit-source-id: 038e13f900fe56205c1333a620ea3bfa468c47ad

Summary:
Avoid redefinition of mmsghdr with newer Android NDK unified headers.

Unified headers unconditionally define `struct mmsghdr`, but the `sendmmsg` and `recvmmsg` system calls are conditional on API level. API 21 is the first level that supports them.

Reviewed By: yfeldblum

Differential Revision: D15344307

fbshipit-source-id: 6f03197cc346511ce7a02f1468c35313140a8fd7

Summary:
The `TIMESTAMP_TX` flag can be used to signal a request TX / NIC timestamping. This flag needs to be passed through to the application via the `SendMsgParamsCallback::getAncillaryData` callback so that the application can populate a socket control message with the ancillary data required to signal the timestamping request.

`folly::AsyncSSLSocket` has extra logic for tracking the end of record (EOR) byte, but this logic currently only passes through `WriteFlags::EOR`. This diff adds support for passing through any flag specified as an EOR-related write flag when the EOR byte is written. As part of this change, the relevant unit tests are extended / cleaned up (some of them still reference MSG_EOR, which is out of date).

Reviewed By: yfeldblum

Differential Revision: D15465432

fbshipit-source-id: 2ab5619607959dd829427a695aefd95a33b4abce

Summary: Need to use `readAll` to ensure that we've read in all bytes that were written (up to EOF). Current approach sometimes returns after only a partial read.

Reviewed By: yfeldblum

Differential Revision: D15456484

fbshipit-source-id: f36a596a3557d4d3714bfaa3c33c726d81ccb5a4

Summary:
This should enable test pilot to skip broken/flakey tests.

The `--tag-new-tests` flag is only appropriate for code that has been
landed on master and is used by the FB infra to classify new tests
appropriately.

For continuous builds we use a test collection with different parameters
from the normal developer facing flow so that the infra can re-assess
their status and enable/disable high-signal/noisy tests.

Depends: D15495344

Reviewed By: Ben0mega

Differential Revision: D15500955

fbshipit-source-id: f3b7976cec6a5cf70f5d128b38bde11620b26918

Summary:
A few changes:

- Pessimize DistributedMutex::try_lock() to do a load on the mutex before
  attempting a lock bts.  This allows us to be effectively parallel in the case
  where acquiring the mutex is not a hard requirement for the application.
  Whenever perf-sensitive try_lock()s come up, they tend to fit this model -
  where acquiring the mutex allows one thread to make progress, but missing the
  lock doesn't lead to incorrectness.  It mostly does not hurt performance, so
  seems like a good thing to have for all cases.
- More unit tests that test progress in the presence of many locks
- Also update comments to be slightly more clear, in particular add a small
  comparison with std::atomic in the first few sentences, and
- Rerun benchmarks on both broadwell and skylake

Reviewed By: djwatson

Differential Revision: D15378164

fbshipit-source-id: 124db4052dadae102a5c04ad9b9f3bcbefe74458

Summary: Pull Request resolved: https://github.com/facebook/folly/pull/1127

Reviewed By: reanimus

Differential Revision: D15329439

Pulled By: yfeldblum

fbshipit-source-id: 6c639ef367be6adede2e6a1c487a8d89a146779d

Summary:
getdeps.py's find_existing_win32_subst_for_path function tries to reuse an existing build directory alias. (On Windows, the build directory is aliased to a drive letter to shorten paths.) If this function does not find and existing build directory alias, getdeps.py invalidates many of its caches.

On my Windows machine, find_existing_win32_subst_for_path always fails, so all of my builds are super slow. This happens because find_existing_win32_subst_for_path is given a path with a lower-case drive letter ("c:\users\..."), but the subst command returns paths with an upper-case drive letter ("C:\users\...").

When comparing paths, use ntpath.normpath. This makes the comparison case-insensitive for drive letters. (It also makes the comparison case-insensitive for other path components.)

On Linux and macOS, this diff should not change behavior.

Reviewed By: wez

Differential Revision: D15466096

fbshipit-source-id: 1669aa0a6eaeb6012154f3a9e24eba3f835262c6

Summary:
[Folly] Fix returning reference to temporary in futures test.

```
folly/futures/test/FutureTest.cpp: In instantiation of 'typename std::enable_if<(! isTry), R>::type folly::Try<NoThrowTestResult>::get() [with bool isTry = false; R = NoThrowTestResult&&; typename std::enable_if<(! isTry), R>::type = NoThrowTestResult&&]':
folly/futures/Future-inl.h:95:30:   required from 'auto folly::futures::detail::wrapInvoke(folly::Try<T>&&, F&&) [with T = NoThrowTestResult; F = Future_NoThrow_Test::TestBody()::<lambda(NoThrowTestResult&&)>]'
folly/futures/Future-inl.h:1209:39:   required from 'folly::Future<typename folly::futures::detail::valueCallableResult<T, F>::value_type> folly::Future<T>::thenValue(F&&) && [with F = Future_NoThrow_Test::TestBody()::<lambda(NoThrowTestResult&&)>; T = NoThrowTestResult; typename folly::futures::detail::valueCallableResult<T, F>::value_type = NoThrowTestResult]'
folly/futures/test/FutureTest.cpp:1408:14:   required from here
folly/futures/test/FutureTest.cpp:1392:33: warning: returning reference to temporary [-Wreturn-local-addr]
 1392 |     return std::forward<R>(value());
```

Fixes #1130.

Reviewed By: LeeHowes

Differential Revision: D15457369

fbshipit-source-id: 045ae3321f957546a6218ee5e3118753e4c8ceb2

Summary:
This should be ok to do. Main thing I'm not sure about is if it's ok
to change these DCHECKs to CHECKs. We could probably also pass state into
InlineFunctionRunner::run() to keep these as DCHECKS here and CHECK in our
implementation.

Reviewed By: andriigrynenko

Differential Revision: D15463243

fbshipit-source-id: c1de7198e8bf48d8af0e03361bed493642e6c173

Summary: Add the ability to add multiple IOBuf free callbacks

Reviewed By: yfeldblum

Differential Revision: D15423850

fbshipit-source-id: ad0352b345835b0472ec090b5ff07e30e14532d8

Reviewed By: yfeldblum

Differential Revision: D15452123

fbshipit-source-id: 4ceec36f007e10284e8ac0482018099c7cea94d9

Summary:
In the folly/io/async tests, allow an extra 20ms of tolerance when performing
timeout checks on non-Linux platforms.

On Linux we can fairly reliably determine the amount of time that the test
spent waiting to be scheduled on the CPU.  This allows the timeout tests to be
relatively accurate even on overloaded systems.

On non-Linux platforms we cannot determine the time spent waiting to be
scheduled, and the `getSchedTimeWaiting()` implementation always returns 0.
This causes the tests to fail pretty frequently on Mac OS.  Adding an extra
20ms of tolerance should hopefully help reduce the test failure rate on Mac.

Reviewed By: yfeldblum

Differential Revision: D15464672

fbshipit-source-id: 2c3096df6a15dcdc68bf14c947736e4731929c0f

Summary:
Since D3640289 the EventBase destructor has invoked currently scheduled
LoopCallbacks.

Some of the tests in `EventBaseTest` installed loop callbacks and checked that
they were never run during the normal loop invocations.  Prior to D3640289
these callbacks never ran.  However, post D3640289 these functions do run, and
they attempt to update local stack variables that have already gone out of
scope.

This rearranges these functions to ensure the variables in question outlive
the EventBase object.

Reviewed By: yfeldblum

Differential Revision: D15462319

fbshipit-source-id: 8f490ddbb978fe45711eb7cbda4502dd6a0fb36b

Summary:
A few of the EventBase timeout tests fail sometimes on the Travis CI
infrastructure, reporting that timeouts fired sooner than expected.
e.g.: https://api.travis-ci.org/v3/job/535980430/log.txt

I believe these failures are due to the fact that the tests schedule the
events/timeouts before capturing the start time point that they compare
against.  This updates the tests to instead capture the start time point
before scheduling events.

Reviewed By: yfeldblum, chadaustin

Differential Revision: D15462320

fbshipit-source-id: dfc321d899768bc70ddaeaa0ef88b42ddd0b70ca

Summary:
This test was accidentally updating `timestamp1` twice rather than updating
`timestamp0`.

The existing check for `timestamp0` only succeeded because `timestamp0` was
never actually updated.  With the current EventBase behavior, even though we
asked the function to be scheduled immediately, it actually ends up waiting
for a full HHWheelTimer tick interval (10ms by default) before it runs.
Update the check to take this into account.

Reviewed By: yfeldblum

Differential Revision: D15462318

fbshipit-source-id: e1851c9c4f2e082ad73debf85343d99bbf244b23

Summary:
Update the `XCHECK_EQ()` macros to use `folly::logging::objectToString()` to
format the message on failure.

This ensures that `XCHECK_EQ()` will successfully compile and work with any
arguments.  Previously it would fail to compile if called with arguments that
did not work with `folly::to<std::string>()`

Reviewed By: chadaustin

Differential Revision: D14545301

fbshipit-source-id: 442657a1770f30cbf01462ad62d32301e60490e5

Summary:
If `toAppend(std::string*, const Object&)` is defined for a particular object
type, make `objectToString()` simply return the exact output of `toAppend()`.

Previously the code returned `(type_name: toAppendResult)`

The new logic is based on the assumption that if `toAppend()` is implemented
for an object, it should emit all relevant information about the object that
may be needed by a developer.

The old behavior of always including type information was potentially helpful
in cases where `objectToString()` was called only in exceptional cases.
(e.g., after an exception thrown by `folly::format()`).  However it made
`objectToString()` too verbose in other non-exceptional situations where we
want to format objects (e.g., in `XCHECK_EQ()`).  There doesn't really seem to
be a strong need to keep the type information in the `folly::format()`
exception case, so this just drops it rather than adding two separate versions
of this function.

Reviewed By: chadaustin

Differential Revision: D14545303

fbshipit-source-id: 29e2f12d65fd22112486ed11f989b63875980088

Summary:
Update the `logging::objectToString()` and `appendToString)` functions when
converting objects that do not have a `toAppend(std::string*, Object)`
implementation.

Previously the code simply emitted `<no_string_conversion>` after the object
type name.  Now the code emits a hexdump of the object.  The emitted hexdump
size is based compile-type visible type of the object.  For subclass instances
the correct dynamic type name will be emitted (if RTTI is supported), but the
hexdump will only include the portion of object known from the compile-time
type.

Reviewed By: yfeldblum, chadaustin

Differential Revision: D14545305

fbshipit-source-id: d1e3df001b7f7c40c7bcb0888e76bf775512e5d3

Summary:
In LogStreamProcessor.h we previously had some `fallbackFormat()` functions
that attempted to convert arbitrary objects to strings in case an exception
was thrown when formatting them with `folly::format()`.

These functions basically attempt to perform best-effort conversion that
ideally should never fail (barring failure to append to the output string).
These functions attempt to use `toAppend()` for the object, similar to how
`folly::to<std::string>()` works.  However, if no `toAppend()` function is
available, or if it throws an exception, they fall back to returning a generic
string.

This diff moves these functions to a separate header file to make it easier to
re-use them in other parts of the code.  In particular I plan to update the
`XCHECK_EQ()` and related macros to use these functions to format its
arguments.

Reviewed By: chadaustin

Differential Revision: D14545304

fbshipit-source-id: 053a504340ba4513abc9bec186a31ffaa03a943a

Reviewed By: lewissbaker

Differential Revision: D15363390

fbshipit-source-id: 98c30928773992c5bb7f040e6f5afb8ee6fef713

Summary: [Folly] Cut unnecessary template param default of `futexWaitUntil`; the param is deduced from the arguments.

Reviewed By: aary

Differential Revision: D15448750

fbshipit-source-id: ec81fc10efbf68ad328a1fb1b7d2fdb2e93551cd

Summary: Helper function to wrap a nested dispatch. Returns a function object that launches the passed function onto the executor and returns a future representing the return value of the function.

Reviewed By: andriigrynenko

Differential Revision: D15300575

fbshipit-source-id: eca4b4139c6405f216a9f1cbc87a8ec2ff437780

Summary:
[Folly] Remove explicit `throw`, `try`, and `catch` uses from `IOBuf`, preferring scope guards and `throw_exception`.

(Note: this ignores all push blocking failures!)

Differential Revision: D15442544

fbshipit-source-id: 2f182256ff12f5aad6166a7b40ed018c1b32e923

Summary:
[Folly] Mark some more `IOBuf` members as `noexcept` and remove an explicit `abort` which is there to simulate `noexcept`.

(Note: this ignores all push blocking failures!)

Differential Revision: D15442545

fbshipit-source-id: adaef4d1f6c2e551e9c19b151e3f08c6ba11f717

Summary:
- GCC 9.1 warns when moving a local object in a return statement
  prevents copy elision.
- Remove the explicit `std::move` in those return statements.
Pull Request resolved: https://github.com/facebook/folly/pull/1131

Reviewed By: LeeHowes

Differential Revision: D15391413

Pulled By: yfeldblum

fbshipit-source-id: af29ea0f6f65957645fa2debce2f5303906a2507