Summary:
[Folly] Move executor to local var immediately in `doCallback`, and always move it into the callback.

(Note: this ignores all push blocking failures!)

Reviewed By: andrewcox

Differential Revision: D15532854

fbshipit-source-id: 971a6a64ad3c47c9f5e9f75907a568f9d59b1000

Summary:
- Replace some calls sites from `NDEBUG` to `kIsDebug`.
- Not every call site is changed; there are a few useful cases of
  `NDEBUG` still. `xlog` macros are one of them. Another is where we
  conditionally define a variable in a local scope and then use it in an
  `assert`.
Pull Request resolved: https://github.com/facebook/folly/pull/1128

Reviewed By: aary

Differential Revision: D15490848

Pulled By: yfeldblum

fbshipit-source-id: 4f2d1689f90baa25d3ba9e99a2a388d9cfc43f18

Summary: [Folly] Cut extra `return` statement in `Core<T>::doCallback`.

Reviewed By: Orvid

Differential Revision: D15527963

fbshipit-source-id: 9396f9ac2b246676b33d13c87e24dd42198a11ec

Summary:
[Folly] Template fibers baton timed-wait over the deadline types, rather than over the timeout types and rather than fixing a single timeout type.

Templating it over the deadline types permits specifying the clock and permits passing a single deadline through unchanged through multiple time-delayed layers of code without skew, and templating it rather than fixing it over a single timeout type permits caller-specified fine-grained or coarse-grained timings.

Currently, the `EventBase` timers do not parameterize over the deadline types, so the fiber version of timed-wait must for now convert to a fixed timeout type.

Reviewed By: andriigrynenko

Differential Revision: D15448177

fbshipit-source-id: aa3fdbffdcb83cbfd1571780ba431a1004beb347

Summary: it seems since D15286181, we updated docker os_image to ubuntu18 and gcc7. this diff changes the README.md to reflect the change.

Reviewed By: yfeldblum

Differential Revision: D15513580

fbshipit-source-id: 0a3518c21c912903921946850bd08c948f09c295

Summary:
[Folly] Disable TLS optimization in PIC mode for `SingletonThreadLocal`.

In PIC mode, TLS offsets must be translated to addresses at runtime via calls to `__tls_get_addr`. Since there is not much of a small inline path or a fast inline fast path anymore anyway in PIC mode, might as well skip the bulky slow caching.

Also, some versions of gcc do not properly support this code in shared libraries. Details in #1135 and https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90562.

Fixes #1135.

Reviewed By: andriigrynenko

Differential Revision: D15468086

fbshipit-source-id: 038e13f900fe56205c1333a620ea3bfa468c47ad

Summary:
Avoid redefinition of mmsghdr with newer Android NDK unified headers.

Unified headers unconditionally define `struct mmsghdr`, but the `sendmmsg` and `recvmmsg` system calls are conditional on API level. API 21 is the first level that supports them.

Reviewed By: yfeldblum

Differential Revision: D15344307

fbshipit-source-id: 6f03197cc346511ce7a02f1468c35313140a8fd7

Summary:
The `TIMESTAMP_TX` flag can be used to signal a request TX / NIC timestamping. This flag needs to be passed through to the application via the `SendMsgParamsCallback::getAncillaryData` callback so that the application can populate a socket control message with the ancillary data required to signal the timestamping request.

`folly::AsyncSSLSocket` has extra logic for tracking the end of record (EOR) byte, but this logic currently only passes through `WriteFlags::EOR`. This diff adds support for passing through any flag specified as an EOR-related write flag when the EOR byte is written. As part of this change, the relevant unit tests are extended / cleaned up (some of them still reference MSG_EOR, which is out of date).

Reviewed By: yfeldblum

Differential Revision: D15465432

fbshipit-source-id: 2ab5619607959dd829427a695aefd95a33b4abce

Summary: Need to use `readAll` to ensure that we've read in all bytes that were written (up to EOF). Current approach sometimes returns after only a partial read.

Reviewed By: yfeldblum

Differential Revision: D15456484

fbshipit-source-id: f36a596a3557d4d3714bfaa3c33c726d81ccb5a4

Summary:
This should enable test pilot to skip broken/flakey tests.

The `--tag-new-tests` flag is only appropriate for code that has been
landed on master and is used by the FB infra to classify new tests
appropriately.

For continuous builds we use a test collection with different parameters
from the normal developer facing flow so that the infra can re-assess
their status and enable/disable high-signal/noisy tests.

Depends: D15495344

Reviewed By: Ben0mega

Differential Revision: D15500955

fbshipit-source-id: f3b7976cec6a5cf70f5d128b38bde11620b26918

Summary:
A few changes:

- Pessimize DistributedMutex::try_lock() to do a load on the mutex before
  attempting a lock bts.  This allows us to be effectively parallel in the case
  where acquiring the mutex is not a hard requirement for the application.
  Whenever perf-sensitive try_lock()s come up, they tend to fit this model -
  where acquiring the mutex allows one thread to make progress, but missing the
  lock doesn't lead to incorrectness.  It mostly does not hurt performance, so
  seems like a good thing to have for all cases.
- More unit tests that test progress in the presence of many locks
- Also update comments to be slightly more clear, in particular add a small
  comparison with std::atomic in the first few sentences, and
- Rerun benchmarks on both broadwell and skylake

Reviewed By: djwatson

Differential Revision: D15378164

fbshipit-source-id: 124db4052dadae102a5c04ad9b9f3bcbefe74458

Summary: Pull Request resolved: https://github.com/facebook/folly/pull/1127

Reviewed By: reanimus

Differential Revision: D15329439

Pulled By: yfeldblum

fbshipit-source-id: 6c639ef367be6adede2e6a1c487a8d89a146779d

Summary:
getdeps.py's find_existing_win32_subst_for_path function tries to reuse an existing build directory alias. (On Windows, the build directory is aliased to a drive letter to shorten paths.) If this function does not find and existing build directory alias, getdeps.py invalidates many of its caches.

On my Windows machine, find_existing_win32_subst_for_path always fails, so all of my builds are super slow. This happens because find_existing_win32_subst_for_path is given a path with a lower-case drive letter ("c:\users\..."), but the subst command returns paths with an upper-case drive letter ("C:\users\...").

When comparing paths, use ntpath.normpath. This makes the comparison case-insensitive for drive letters. (It also makes the comparison case-insensitive for other path components.)

On Linux and macOS, this diff should not change behavior.

Reviewed By: wez

Differential Revision: D15466096

fbshipit-source-id: 1669aa0a6eaeb6012154f3a9e24eba3f835262c6

Summary:
[Folly] Fix returning reference to temporary in futures test.

```
folly/futures/test/FutureTest.cpp: In instantiation of 'typename std::enable_if<(! isTry), R>::type folly::Try<NoThrowTestResult>::get() [with bool isTry = false; R = NoThrowTestResult&&; typename std::enable_if<(! isTry), R>::type = NoThrowTestResult&&]':
folly/futures/Future-inl.h:95:30:   required from 'auto folly::futures::detail::wrapInvoke(folly::Try<T>&&, F&&) [with T = NoThrowTestResult; F = Future_NoThrow_Test::TestBody()::<lambda(NoThrowTestResult&&)>]'
folly/futures/Future-inl.h:1209:39:   required from 'folly::Future<typename folly::futures::detail::valueCallableResult<T, F>::value_type> folly::Future<T>::thenValue(F&&) && [with F = Future_NoThrow_Test::TestBody()::<lambda(NoThrowTestResult&&)>; T = NoThrowTestResult; typename folly::futures::detail::valueCallableResult<T, F>::value_type = NoThrowTestResult]'
folly/futures/test/FutureTest.cpp:1408:14:   required from here
folly/futures/test/FutureTest.cpp:1392:33: warning: returning reference to temporary [-Wreturn-local-addr]
 1392 |     return std::forward<R>(value());
```

Fixes #1130.

Reviewed By: LeeHowes

Differential Revision: D15457369

fbshipit-source-id: 045ae3321f957546a6218ee5e3118753e4c8ceb2

Summary:
This should be ok to do. Main thing I'm not sure about is if it's ok
to change these DCHECKs to CHECKs. We could probably also pass state into
InlineFunctionRunner::run() to keep these as DCHECKS here and CHECK in our
implementation.

Reviewed By: andriigrynenko

Differential Revision: D15463243

fbshipit-source-id: c1de7198e8bf48d8af0e03361bed493642e6c173

Summary: Add the ability to add multiple IOBuf free callbacks

Reviewed By: yfeldblum

Differential Revision: D15423850

fbshipit-source-id: ad0352b345835b0472ec090b5ff07e30e14532d8

Reviewed By: yfeldblum

Differential Revision: D15452123

fbshipit-source-id: 4ceec36f007e10284e8ac0482018099c7cea94d9

Summary:
In the folly/io/async tests, allow an extra 20ms of tolerance when performing
timeout checks on non-Linux platforms.

On Linux we can fairly reliably determine the amount of time that the test
spent waiting to be scheduled on the CPU.  This allows the timeout tests to be
relatively accurate even on overloaded systems.

On non-Linux platforms we cannot determine the time spent waiting to be
scheduled, and the `getSchedTimeWaiting()` implementation always returns 0.
This causes the tests to fail pretty frequently on Mac OS.  Adding an extra
20ms of tolerance should hopefully help reduce the test failure rate on Mac.

Reviewed By: yfeldblum

Differential Revision: D15464672

fbshipit-source-id: 2c3096df6a15dcdc68bf14c947736e4731929c0f

Summary:
Since D3640289 the EventBase destructor has invoked currently scheduled
LoopCallbacks.

Some of the tests in `EventBaseTest` installed loop callbacks and checked that
they were never run during the normal loop invocations.  Prior to D3640289
these callbacks never ran.  However, post D3640289 these functions do run, and
they attempt to update local stack variables that have already gone out of
scope.

This rearranges these functions to ensure the variables in question outlive
the EventBase object.

Reviewed By: yfeldblum

Differential Revision: D15462319

fbshipit-source-id: 8f490ddbb978fe45711eb7cbda4502dd6a0fb36b

Summary:
A few of the EventBase timeout tests fail sometimes on the Travis CI
infrastructure, reporting that timeouts fired sooner than expected.
e.g.: https://api.travis-ci.org/v3/job/535980430/log.txt

I believe these failures are due to the fact that the tests schedule the
events/timeouts before capturing the start time point that they compare
against.  This updates the tests to instead capture the start time point
before scheduling events.

Reviewed By: yfeldblum, chadaustin

Differential Revision: D15462320

fbshipit-source-id: dfc321d899768bc70ddaeaa0ef88b42ddd0b70ca

Summary:
This test was accidentally updating `timestamp1` twice rather than updating
`timestamp0`.

The existing check for `timestamp0` only succeeded because `timestamp0` was
never actually updated.  With the current EventBase behavior, even though we
asked the function to be scheduled immediately, it actually ends up waiting
for a full HHWheelTimer tick interval (10ms by default) before it runs.
Update the check to take this into account.

Reviewed By: yfeldblum

Differential Revision: D15462318

fbshipit-source-id: e1851c9c4f2e082ad73debf85343d99bbf244b23

Summary:
Update the `XCHECK_EQ()` macros to use `folly::logging::objectToString()` to
format the message on failure.

This ensures that `XCHECK_EQ()` will successfully compile and work with any
arguments.  Previously it would fail to compile if called with arguments that
did not work with `folly::to<std::string>()`

Reviewed By: chadaustin

Differential Revision: D14545301

fbshipit-source-id: 442657a1770f30cbf01462ad62d32301e60490e5

Summary:
If `toAppend(std::string*, const Object&)` is defined for a particular object
type, make `objectToString()` simply return the exact output of `toAppend()`.

Previously the code returned `(type_name: toAppendResult)`

The new logic is based on the assumption that if `toAppend()` is implemented
for an object, it should emit all relevant information about the object that
may be needed by a developer.

The old behavior of always including type information was potentially helpful
in cases where `objectToString()` was called only in exceptional cases.
(e.g., after an exception thrown by `folly::format()`).  However it made
`objectToString()` too verbose in other non-exceptional situations where we
want to format objects (e.g., in `XCHECK_EQ()`).  There doesn't really seem to
be a strong need to keep the type information in the `folly::format()`
exception case, so this just drops it rather than adding two separate versions
of this function.

Reviewed By: chadaustin

Differential Revision: D14545303

fbshipit-source-id: 29e2f12d65fd22112486ed11f989b63875980088

Summary:
Update the `logging::objectToString()` and `appendToString)` functions when
converting objects that do not have a `toAppend(std::string*, Object)`
implementation.

Previously the code simply emitted `<no_string_conversion>` after the object
type name.  Now the code emits a hexdump of the object.  The emitted hexdump
size is based compile-type visible type of the object.  For subclass instances
the correct dynamic type name will be emitted (if RTTI is supported), but the
hexdump will only include the portion of object known from the compile-time
type.

Reviewed By: yfeldblum, chadaustin

Differential Revision: D14545305

fbshipit-source-id: d1e3df001b7f7c40c7bcb0888e76bf775512e5d3

Summary:
In LogStreamProcessor.h we previously had some `fallbackFormat()` functions
that attempted to convert arbitrary objects to strings in case an exception
was thrown when formatting them with `folly::format()`.

These functions basically attempt to perform best-effort conversion that
ideally should never fail (barring failure to append to the output string).
These functions attempt to use `toAppend()` for the object, similar to how
`folly::to<std::string>()` works.  However, if no `toAppend()` function is
available, or if it throws an exception, they fall back to returning a generic
string.

This diff moves these functions to a separate header file to make it easier to
re-use them in other parts of the code.  In particular I plan to update the
`XCHECK_EQ()` and related macros to use these functions to format its
arguments.

Reviewed By: chadaustin

Differential Revision: D14545304

fbshipit-source-id: 053a504340ba4513abc9bec186a31ffaa03a943a

Reviewed By: lewissbaker

Differential Revision: D15363390

fbshipit-source-id: 98c30928773992c5bb7f040e6f5afb8ee6fef713

Summary: [Folly] Cut unnecessary template param default of `futexWaitUntil`; the param is deduced from the arguments.

Reviewed By: aary

Differential Revision: D15448750

fbshipit-source-id: ec81fc10efbf68ad328a1fb1b7d2fdb2e93551cd

Summary: Helper function to wrap a nested dispatch. Returns a function object that launches the passed function onto the executor and returns a future representing the return value of the function.

Reviewed By: andriigrynenko

Differential Revision: D15300575

fbshipit-source-id: eca4b4139c6405f216a9f1cbc87a8ec2ff437780

Summary:
[Folly] Remove explicit `throw`, `try`, and `catch` uses from `IOBuf`, preferring scope guards and `throw_exception`.

(Note: this ignores all push blocking failures!)

Differential Revision: D15442544

fbshipit-source-id: 2f182256ff12f5aad6166a7b40ed018c1b32e923

Summary:
[Folly] Mark some more `IOBuf` members as `noexcept` and remove an explicit `abort` which is there to simulate `noexcept`.

(Note: this ignores all push blocking failures!)

Differential Revision: D15442545

fbshipit-source-id: adaef4d1f6c2e551e9c19b151e3f08c6ba11f717

Summary:
- GCC 9.1 warns when moving a local object in a return statement
  prevents copy elision.
- Remove the explicit `std::move` in those return statements.
Pull Request resolved: https://github.com/facebook/folly/pull/1131

Reviewed By: LeeHowes

Differential Revision: D15391413

Pulled By: yfeldblum

fbshipit-source-id: af29ea0f6f65957645fa2debce2f5303906a2507

Summary:
[Folly] Cut `folly/LockTraitsBoost.h`. Also, discourage specializing `LockTraits`.

For ODR sanity, the better mechanism is to wrap mutex types with non-standard interfaces in mutex wrapper types with standard interfaces.

Reviewed By: Orvid

Differential Revision: D15426235

fbshipit-source-id: f54aee8c47951f6214f5b23fce7f8f6c29c8b5c2

Summary: Add lizmq and sigar in prepare for fbzmq oss build migration.

Reviewed By: wez

Differential Revision: D15387435

fbshipit-source-id: 1ec2057972bf801a1598af0a1ae87f958394d812

Summary:
with the reshuffling around getdeps.py, and changes in folly/thrift/wangle,
our internal package stopped building.  This diff brings it back to life.  It is
a little bit interwined:

* Introduce a `--facebook-internal` switch to getdeps that causes `fb` in the
  build context to be set to `on` rather than `off`.  This allows specifying
  options in the manifest that are FB specific.
* Introduce cmake options to control the WATCHMAN_STATE_DIR and whether cmake
  should attempt to create WATCHMAN_STATE_DIR when it is installed.
* For FB specific builds on macOS, set the state dir to match our existing location
  and disable creation of WATCHMAN_STATE_DIR because that will ultimately be
  handling during package installation and we don't have privs for that on
  the machine generating the package.
* Adjust Facebook CI scripts to use getdeps for building and testing watchman

Reviewed By: simpkins

Differential Revision: D15337970

fbshipit-source-id: 8ab3c1f810decf5f0b16337dc1cbe6dc73d26426

Summary: Add a comment that the Dynamic version of MPMCQueue is deprecated and that UnboundedQueue should be used instead.

Reviewed By: yfeldblum

Differential Revision: D15392160

fbshipit-source-id: 5958b7602d3062878dd3e78907cb7091532ad4b8

Summary: Guard pages are cheap (VM only + mapping state). More than one is occasionally useful (e.g. imagine a 10k object on stack that's not used, subsequent function calls will create a frame that jumps over a single 4K guard page and we get hard to debug memory corruption instead of a clean segfault).

Reviewed By: andriigrynenko

Differential Revision: D15367640

fbshipit-source-id: 8f1d42a71926653929dd847eeba172c680f98589

Summary:
The schedule type is used to differentiate between a build run during
code review (and thus runs code that isn't yet in the master branch),
from continuous or other types of run that operate on landed code.

This doesn't change any behavior yet; this diff just adds
the plumbing to pass down an optional arbitrary schedule type string
from the CI system.

In the future, we'll use the schedule type to influence the behavior
of running tests.

Reviewed By: strager

Differential Revision: D15300120

fbshipit-source-id: 3b46afef2ff171b3fa095763dd5006a54ea328b8

Summary:
F14TestUtil's SwapTrackingAlloc's metadata accesses were
triggering a -Wclass-memaccess warning in gcc 9.1 despite being safe. This
diff changes the accesses to use explicit casting to avoid the warning.

Reviewed By: yfeldblum

Differential Revision: D15393963

fbshipit-source-id: 088811354f00d623a58678c3f41a0e8effdb5ce3

Reviewed By: udippant

Differential Revision: D15368849

fbshipit-source-id: 5e101fc38417b807a531fa456b38234d6dd08b7a

Summary: [Folly] Namespace the helper macro in `folly/ssl/OpenSSLPtrTypes.h`.

Reviewed By: Orvid

Differential Revision: D15370971

fbshipit-source-id: aa907f60fee5ef58c5484838a3bde0d68e65b249