Summary:
As part of trying to get our internal usage of folly off a fork, we have various compiler warnings set ridiculously high.

This gets rid of a couple of warnings that we hit in folly code.
Pull Request resolved: https://github.com/facebook/folly/pull/1145

Reviewed By: LeeHowes, Orvid

Differential Revision: D15586209

Pulled By: yfeldblum

fbshipit-source-id: dc3ebffbddd0b82f2c2ee719a95b7fcfe551e074

Summary: swap the order of getLocalAddress() and getPeerAddress(), so we are more likely to log the local address after exception happens.

Reviewed By: knekritz

Differential Revision: D15559814

fbshipit-source-id: 50b7a964d6c3add2e4ae1da9fde8e9e4e67e5715

Summary:
Grabbing the mutex of the newly created context is unecessary cost.
folly::Synchronized's constructor support this optimization, so it's an easy change.

Reviewed By: yfeldblum

Differential Revision: D15600165

fbshipit-source-id: e92570a7ac7ac1e908d7c797bed6de22f940d387

Summary:
The shallow copy code path currently iterates and copies all values manually. Instead of benefiting from the underlying data collection's implementation of self-copy.
At the very least, it's simpler looking code.

Inheriting unique_ptr is usually discouraged, but in this specific (internal impl detail) use case, I can't foresee any risk.

Reviewed By: yfeldblum

Differential Revision: D15599260

fbshipit-source-id: 73edea27a27e8fbc8b057cf3e974ee3d83a6e760

Summary: Pull Request resolved: https://github.com/facebook/folly/pull/1141

Reviewed By: LeeHowes

Differential Revision: D15560029

Pulled By: yfeldblum

fbshipit-source-id: 5f0d5ee8d09be834d1216d4c39244e0855d2dc0f

Summary:
- Old versions of GCC (specifically, GCC 4.7) warn about uninitialized memory
  in `Expected.h`
- Newer versions of GCC do not complain, so remove the pragma push/pop which
  was disabling the warning.
Pull Request resolved: https://github.com/facebook/folly/pull/1143

Reviewed By: ericniebler

Differential Revision: D15586279

Pulled By: yfeldblum

fbshipit-source-id: 7b01934c7601ecf035f1e05f2ed3e5f799cf25b9

Summary:
This can be used to check if ChaCha20 is available for use.
Signed-off-by: Rosen Penev <rosenp@gmail.com>

related: https://github.com/facebookincubator/fizz/pull/27
Pull Request resolved: https://github.com/facebook/folly/pull/1146

Reviewed By: knekritz

Differential Revision: D15583706

Pulled By: yfeldblum

fbshipit-source-id: 1195d623098018992811cd7d6cef72e2366ace19

Summary: [Folly] Fix a regression in per-task allocations in `SerialExecutor`.

Reviewed By: andriigrynenko

Differential Revision: D15589007

fbshipit-source-id: 4461798435833cb317b41cfea932762eb7ac8662

Summary: [Folly] Cut outdated comment in `F14Table`, referring to the return value of a function which returns `void`.

Reviewed By: nbronson

Differential Revision: D15586980

fbshipit-source-id: b75f7b989c39b52e589ce8051663085201939433

Summary: [Folly] Move `SerialExecutor` test-specific helper types into the tests which need them.

Reviewed By: andriigrynenko

Differential Revision: D15587721

fbshipit-source-id: 603e5381b2e00b4317f7e9836c6102238158c5d0

Summary: Pull Request resolved: https://github.com/facebook/folly/pull/1142

Reviewed By: shixiao

Differential Revision: D15574540

Pulled By: yfeldblum

fbshipit-source-id: 2ce91f7510f0b278677c15b3a9f1f80b43ff8998

Summary:
SerialExecutor can not rely on parent to save RequestContext for him as it is not guaranteed that an enqueued function will be executed from the parent callback that was scheduled at the same time.
This diff captures RequestContext in a lambda and sets it correctly

Reviewed By: shixiao

Differential Revision: D15381223

fbshipit-source-id: 9b3ab1f96e57eb5e697efb52105d80c2899a03e7

Summary: Lock elision requires non-mutating functions that check if lock acquisition could succeed. This diff adds such functions to folly::SharedMutex, including eligible_for_lock_elision(), eligible_for_lock_upgrade_elision(), and eligible_for_lock_shared_elision(). The diff also adds assertions to validate these functions' correctness under single-threaded executions.

Reviewed By: nbronson

Differential Revision: D15545398

fbshipit-source-id: 0037d473c9dd360f7143ea4c4c9092fb9bb7f5f9

Summary: Acquiring a lock into an unused temporary is generally a bug. Anyone wanting to do this can suppress their compiler's warning by explicitly `static_cast`ing the lock holder to void.

Reviewed By: vitaut

Differential Revision: D15504563

fbshipit-source-id: 1033f6a1d4044b9698934b73eefebfe83602a2a5

Summary:
[Folly] Use a M:N caches-to-lifetimes in `SingletonThreadLocal`.

Sometimes the compiler will not merge cache and lifetime thread-local variables in expected ways, so be resilient. More details in #1135 and https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90562.

At the same time, move the tracking into the wrapper. The lifetime was an intrusively-linked-list node which was clever and which was allocation-free but which increased the TLS requirement per instantiation of `SingletonThreadLocal`. Moving the tracking into the wrapper decreases the TLS requirement, although the tracking now requires heap allocations.

Fixes #1135.
Closes #1138.

Reviewed By: andriigrynenko

Differential Revision: D15508024

fbshipit-source-id: a344e8dd52bd52724ad85b0fc88d6a2a349952ac

Summary: The cacheline padding in folly/test/SharedMutexTest is not actually functional since the compiler is free to rearrange and optimize away local variables. This diff fixes the cacheline padding by aligning the false-sharing critical values.

Reviewed By: nbronson

Differential Revision: D15523241

fbshipit-source-id: c51b1e4cd78ab33f75fa93faf0a0346ec602f97c

Summary:
[Folly] Move executor to local var immediately in `doCallback`, and always move it into the callback.

(Note: this ignores all push blocking failures!)

Reviewed By: andrewcox

Differential Revision: D15532854

fbshipit-source-id: 971a6a64ad3c47c9f5e9f75907a568f9d59b1000

Summary:
- Replace some calls sites from `NDEBUG` to `kIsDebug`.
- Not every call site is changed; there are a few useful cases of
  `NDEBUG` still. `xlog` macros are one of them. Another is where we
  conditionally define a variable in a local scope and then use it in an
  `assert`.
Pull Request resolved: https://github.com/facebook/folly/pull/1128

Reviewed By: aary

Differential Revision: D15490848

Pulled By: yfeldblum

fbshipit-source-id: 4f2d1689f90baa25d3ba9e99a2a388d9cfc43f18

Summary: [Folly] Cut extra `return` statement in `Core<T>::doCallback`.

Reviewed By: Orvid

Differential Revision: D15527963

fbshipit-source-id: 9396f9ac2b246676b33d13c87e24dd42198a11ec

Summary:
[Folly] Template fibers baton timed-wait over the deadline types, rather than over the timeout types and rather than fixing a single timeout type.

Templating it over the deadline types permits specifying the clock and permits passing a single deadline through unchanged through multiple time-delayed layers of code without skew, and templating it rather than fixing it over a single timeout type permits caller-specified fine-grained or coarse-grained timings.

Currently, the `EventBase` timers do not parameterize over the deadline types, so the fiber version of timed-wait must for now convert to a fixed timeout type.

Reviewed By: andriigrynenko

Differential Revision: D15448177

fbshipit-source-id: aa3fdbffdcb83cbfd1571780ba431a1004beb347

Summary: it seems since D15286181, we updated docker os_image to ubuntu18 and gcc7. this diff changes the README.md to reflect the change.

Reviewed By: yfeldblum

Differential Revision: D15513580

fbshipit-source-id: 0a3518c21c912903921946850bd08c948f09c295

Summary:
[Folly] Disable TLS optimization in PIC mode for `SingletonThreadLocal`.

In PIC mode, TLS offsets must be translated to addresses at runtime via calls to `__tls_get_addr`. Since there is not much of a small inline path or a fast inline fast path anymore anyway in PIC mode, might as well skip the bulky slow caching.

Also, some versions of gcc do not properly support this code in shared libraries. Details in #1135 and https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90562.

Fixes #1135.

Reviewed By: andriigrynenko

Differential Revision: D15468086

fbshipit-source-id: 038e13f900fe56205c1333a620ea3bfa468c47ad

Summary:
Avoid redefinition of mmsghdr with newer Android NDK unified headers.

Unified headers unconditionally define `struct mmsghdr`, but the `sendmmsg` and `recvmmsg` system calls are conditional on API level. API 21 is the first level that supports them.

Reviewed By: yfeldblum

Differential Revision: D15344307

fbshipit-source-id: 6f03197cc346511ce7a02f1468c35313140a8fd7

Summary:
The `TIMESTAMP_TX` flag can be used to signal a request TX / NIC timestamping. This flag needs to be passed through to the application via the `SendMsgParamsCallback::getAncillaryData` callback so that the application can populate a socket control message with the ancillary data required to signal the timestamping request.

`folly::AsyncSSLSocket` has extra logic for tracking the end of record (EOR) byte, but this logic currently only passes through `WriteFlags::EOR`. This diff adds support for passing through any flag specified as an EOR-related write flag when the EOR byte is written. As part of this change, the relevant unit tests are extended / cleaned up (some of them still reference MSG_EOR, which is out of date).

Reviewed By: yfeldblum

Differential Revision: D15465432

fbshipit-source-id: 2ab5619607959dd829427a695aefd95a33b4abce

Summary: Need to use `readAll` to ensure that we've read in all bytes that were written (up to EOF). Current approach sometimes returns after only a partial read.

Reviewed By: yfeldblum

Differential Revision: D15456484

fbshipit-source-id: f36a596a3557d4d3714bfaa3c33c726d81ccb5a4

Summary:
This should enable test pilot to skip broken/flakey tests.

The `--tag-new-tests` flag is only appropriate for code that has been
landed on master and is used by the FB infra to classify new tests
appropriately.

For continuous builds we use a test collection with different parameters
from the normal developer facing flow so that the infra can re-assess
their status and enable/disable high-signal/noisy tests.

Depends: D15495344

Reviewed By: Ben0mega

Differential Revision: D15500955

fbshipit-source-id: f3b7976cec6a5cf70f5d128b38bde11620b26918

Summary:
A few changes:

- Pessimize DistributedMutex::try_lock() to do a load on the mutex before
  attempting a lock bts.  This allows us to be effectively parallel in the case
  where acquiring the mutex is not a hard requirement for the application.
  Whenever perf-sensitive try_lock()s come up, they tend to fit this model -
  where acquiring the mutex allows one thread to make progress, but missing the
  lock doesn't lead to incorrectness.  It mostly does not hurt performance, so
  seems like a good thing to have for all cases.
- More unit tests that test progress in the presence of many locks
- Also update comments to be slightly more clear, in particular add a small
  comparison with std::atomic in the first few sentences, and
- Rerun benchmarks on both broadwell and skylake

Reviewed By: djwatson

Differential Revision: D15378164

fbshipit-source-id: 124db4052dadae102a5c04ad9b9f3bcbefe74458

Summary: Pull Request resolved: https://github.com/facebook/folly/pull/1127

Reviewed By: reanimus

Differential Revision: D15329439

Pulled By: yfeldblum

fbshipit-source-id: 6c639ef367be6adede2e6a1c487a8d89a146779d

Summary:
getdeps.py's find_existing_win32_subst_for_path function tries to reuse an existing build directory alias. (On Windows, the build directory is aliased to a drive letter to shorten paths.) If this function does not find and existing build directory alias, getdeps.py invalidates many of its caches.

On my Windows machine, find_existing_win32_subst_for_path always fails, so all of my builds are super slow. This happens because find_existing_win32_subst_for_path is given a path with a lower-case drive letter ("c:\users\..."), but the subst command returns paths with an upper-case drive letter ("C:\users\...").

When comparing paths, use ntpath.normpath. This makes the comparison case-insensitive for drive letters. (It also makes the comparison case-insensitive for other path components.)

On Linux and macOS, this diff should not change behavior.

Reviewed By: wez

Differential Revision: D15466096

fbshipit-source-id: 1669aa0a6eaeb6012154f3a9e24eba3f835262c6

Summary:
[Folly] Fix returning reference to temporary in futures test.

```
folly/futures/test/FutureTest.cpp: In instantiation of 'typename std::enable_if<(! isTry), R>::type folly::Try<NoThrowTestResult>::get() [with bool isTry = false; R = NoThrowTestResult&&; typename std::enable_if<(! isTry), R>::type = NoThrowTestResult&&]':
folly/futures/Future-inl.h:95:30:   required from 'auto folly::futures::detail::wrapInvoke(folly::Try<T>&&, F&&) [with T = NoThrowTestResult; F = Future_NoThrow_Test::TestBody()::<lambda(NoThrowTestResult&&)>]'
folly/futures/Future-inl.h:1209:39:   required from 'folly::Future<typename folly::futures::detail::valueCallableResult<T, F>::value_type> folly::Future<T>::thenValue(F&&) && [with F = Future_NoThrow_Test::TestBody()::<lambda(NoThrowTestResult&&)>; T = NoThrowTestResult; typename folly::futures::detail::valueCallableResult<T, F>::value_type = NoThrowTestResult]'
folly/futures/test/FutureTest.cpp:1408:14:   required from here
folly/futures/test/FutureTest.cpp:1392:33: warning: returning reference to temporary [-Wreturn-local-addr]
 1392 |     return std::forward<R>(value());
```

Fixes #1130.

Reviewed By: LeeHowes

Differential Revision: D15457369

fbshipit-source-id: 045ae3321f957546a6218ee5e3118753e4c8ceb2

Summary:
This should be ok to do. Main thing I'm not sure about is if it's ok
to change these DCHECKs to CHECKs. We could probably also pass state into
InlineFunctionRunner::run() to keep these as DCHECKS here and CHECK in our
implementation.

Reviewed By: andriigrynenko

Differential Revision: D15463243

fbshipit-source-id: c1de7198e8bf48d8af0e03361bed493642e6c173

Summary: Add the ability to add multiple IOBuf free callbacks

Reviewed By: yfeldblum

Differential Revision: D15423850

fbshipit-source-id: ad0352b345835b0472ec090b5ff07e30e14532d8

Reviewed By: yfeldblum

Differential Revision: D15452123

fbshipit-source-id: 4ceec36f007e10284e8ac0482018099c7cea94d9

Summary:
In the folly/io/async tests, allow an extra 20ms of tolerance when performing
timeout checks on non-Linux platforms.

On Linux we can fairly reliably determine the amount of time that the test
spent waiting to be scheduled on the CPU.  This allows the timeout tests to be
relatively accurate even on overloaded systems.

On non-Linux platforms we cannot determine the time spent waiting to be
scheduled, and the `getSchedTimeWaiting()` implementation always returns 0.
This causes the tests to fail pretty frequently on Mac OS.  Adding an extra
20ms of tolerance should hopefully help reduce the test failure rate on Mac.

Reviewed By: yfeldblum

Differential Revision: D15464672

fbshipit-source-id: 2c3096df6a15dcdc68bf14c947736e4731929c0f

Summary:
Since D3640289 the EventBase destructor has invoked currently scheduled
LoopCallbacks.

Some of the tests in `EventBaseTest` installed loop callbacks and checked that
they were never run during the normal loop invocations.  Prior to D3640289
these callbacks never ran.  However, post D3640289 these functions do run, and
they attempt to update local stack variables that have already gone out of
scope.

This rearranges these functions to ensure the variables in question outlive
the EventBase object.

Reviewed By: yfeldblum

Differential Revision: D15462319

fbshipit-source-id: 8f490ddbb978fe45711eb7cbda4502dd6a0fb36b

Summary:
A few of the EventBase timeout tests fail sometimes on the Travis CI
infrastructure, reporting that timeouts fired sooner than expected.
e.g.: https://api.travis-ci.org/v3/job/535980430/log.txt

I believe these failures are due to the fact that the tests schedule the
events/timeouts before capturing the start time point that they compare
against.  This updates the tests to instead capture the start time point
before scheduling events.

Reviewed By: yfeldblum, chadaustin

Differential Revision: D15462320

fbshipit-source-id: dfc321d899768bc70ddaeaa0ef88b42ddd0b70ca

Summary:
This test was accidentally updating `timestamp1` twice rather than updating
`timestamp0`.

The existing check for `timestamp0` only succeeded because `timestamp0` was
never actually updated.  With the current EventBase behavior, even though we
asked the function to be scheduled immediately, it actually ends up waiting
for a full HHWheelTimer tick interval (10ms by default) before it runs.
Update the check to take this into account.

Reviewed By: yfeldblum

Differential Revision: D15462318

fbshipit-source-id: e1851c9c4f2e082ad73debf85343d99bbf244b23

Summary:
Update the `XCHECK_EQ()` macros to use `folly::logging::objectToString()` to
format the message on failure.

This ensures that `XCHECK_EQ()` will successfully compile and work with any
arguments.  Previously it would fail to compile if called with arguments that
did not work with `folly::to<std::string>()`

Reviewed By: chadaustin

Differential Revision: D14545301

fbshipit-source-id: 442657a1770f30cbf01462ad62d32301e60490e5

Summary:
If `toAppend(std::string*, const Object&)` is defined for a particular object
type, make `objectToString()` simply return the exact output of `toAppend()`.

Previously the code returned `(type_name: toAppendResult)`

The new logic is based on the assumption that if `toAppend()` is implemented
for an object, it should emit all relevant information about the object that
may be needed by a developer.

The old behavior of always including type information was potentially helpful
in cases where `objectToString()` was called only in exceptional cases.
(e.g., after an exception thrown by `folly::format()`).  However it made
`objectToString()` too verbose in other non-exceptional situations where we
want to format objects (e.g., in `XCHECK_EQ()`).  There doesn't really seem to
be a strong need to keep the type information in the `folly::format()`
exception case, so this just drops it rather than adding two separate versions
of this function.

Reviewed By: chadaustin

Differential Revision: D14545303

fbshipit-source-id: 29e2f12d65fd22112486ed11f989b63875980088

Summary:
Update the `logging::objectToString()` and `appendToString)` functions when
converting objects that do not have a `toAppend(std::string*, Object)`
implementation.

Previously the code simply emitted `<no_string_conversion>` after the object
type name.  Now the code emits a hexdump of the object.  The emitted hexdump
size is based compile-type visible type of the object.  For subclass instances
the correct dynamic type name will be emitted (if RTTI is supported), but the
hexdump will only include the portion of object known from the compile-time
type.

Reviewed By: yfeldblum, chadaustin

Differential Revision: D14545305

fbshipit-source-id: d1e3df001b7f7c40c7bcb0888e76bf775512e5d3