Summary: Pull Request resolved: https://github.com/facebook/folly/pull/1110

Reviewed By: Orvid

Differential Revision: D21110331

Pulled By: yfeldblum

fbshipit-source-id: 57522232efb7268bc290e4f22057f531a380fb81

Summary:
SemiFuture::within creates two valid SemiFutures and promptly overwrites them, both discarding valid SemiFutures and wasting the core allocation. It later drops the updated versions without directly consuming them, instead stealing the deferred executors.

This update initializes the two SemiFutures as empty, and safely terminates the chain with trivial callbacks so the SemiFutures can be treated as consumed.

Reviewed By: yfeldblum

Differential Revision: D21769500

fbshipit-source-id: eab13a78b17ecfc937b42c8d5e01e1292158d9ed

Differential Revision: D21757385

fbshipit-source-id: 28af042285d343944452e1c2cf94794a121c0e06

Summary:
`value` could trigger "unused parameter" warning in NDEBUG builds. The fix is straightforward, to use the new-ish `[[maybe_unused]]` attribute.

The alternatives are to use `FOLLY_MAYBE_UNUSED` or even manual `(void)value;`. But since this is coroutines library, it should be reasonably safe to assume the compiler supports the cleaner and more expressive native C++17 option.

Reviewed By: lewissbaker

Differential Revision: D21784931

fbshipit-source-id: 895bf95d65df6533d768b65c44a898a73e0a636f

Summary:
There were two problems with the clock_gettime fallback logic on
macOS. One is that it used undefined macros if AvailabilityMacros.h
wasn't included. Another is that it compiled the fallback when the
minimum iPhone OS version (as when building on macOS) was not set, and
vice versa.

Fix that.

Differential Revision: D21757362

fbshipit-source-id: 99fd17260cb25ed07c39ba79466e835f08a1fc7c

Summary:
It may be useful to see if a threadlocal has been created on a thread before accessing it.

One way this could be useful: the data is only relevant to a few threads but will be accessed by many. This would allow you to save the memory on the non-relevant threads.

Differential Revision: D21731260

fbshipit-source-id: 0b1835b454f9b0c457f722ab38f84f89aca383fb

Summary: Add detailed comment to SSL_CTX_set_session_cache_mode call in SSLContext, since any changes can result in breaking functionality or increased memory usage.

Reviewed By: mingtaoy

Differential Revision: D21769914

fbshipit-source-id: 61a172f24cda870d6c13a24d279509e2e2804538

Make collectAll, collectAllRange, collectAllWindowed consistently rethrow exception from first failure

Summary:
The void-returning variants of collectAllRange() and collectAllWindowed() were
previously failing with the error from the first task to fail whereas the other
variants of collectAll() were failing with an arbitrary error (typically first
by argument order) in the case of multiple errors.

This was problematic because after the first task failed it requests cancellation
of the other tasks, which then often will complete with an 'OperationCancelled'
error. This mean that, unless the original error occurred on the first failed
task in argument order, the error that caused the cancellation would be discarded
and instead, the collectAll() operation would result in an 'OperationCancelled'
error.

This change updates those algorithms to now consistently fail with the error
from the first task to fail (in time) as this error will typically be the most
useful in determining the root cause of the failure.

Also it now discards any errors that occur in child tasks if they fail after
cancellation was requested on the parent task. In this case the collectAll()
will complete with an 'OperationCancelled' error.

Also made a few drive-by changes to avoid rethrowing exceptions where possible
by using `co_yield co_error(e)`.

Reviewed By: yfeldblum

Differential Revision: D21681812

fbshipit-source-id: dfe15d0672dcbdeb4e156962cdffe767ddf91a0b

Summary:
Adds the ability to request cancellation of the folly::coro::UnboundedQueue's
dequeue() operation by injecting a CancellationToken using co_withCancellation().

Most of the work for supporting this was implemented in the change to
Semaphore::co_wait() to support cancellation.

This diff just adds some tests for cancellation of UnboundedQueue::dequeue()
and provides a more efficient code-path for the cancellation-case that does
not involve throwing an exception.

Reviewed By: yfeldblum

Differential Revision: D21555325

fbshipit-source-id: 7fbae8aa426b40bbea4d0d6dec85df4bad813733

Summary:
The `Semaphore::co_await()` method now response to requests for cancellation
communicated through the awaiting context's CancellationToken.

This allows interrupting a wait-operation if the result is no longer required,
in which case the co_wait() task completes with the OperationCancelled exception.

Reviewed By: yfeldblum

Differential Revision: D21655946

fbshipit-source-id: 8c7f862580ddebd2b4d3eb3cc3ab5e01ad5ee289

Summary:
Now use an intrusive list structure to store the list of waiters waiting
for the semaphore to be signalled.

This is primarily motivated by the need to support efficient cancellation,
which requires the ability to efficiently remove a specific element from
the waiter list without having to scan the list like we would for the
existing `std::deque`-based list storage.

Another beneift of this change is that it now eliminates memory allocations
that were previously required by the std::deque container.

The `Semaphore::future_wait()` implementation should now also have one fewer
allocations as the allocation of the Baton and the Baton::Waiter are now fused.

Reviewed By: yfeldblum

Differential Revision: D21692257

fbshipit-source-id: 9beff176b250ae3f37df9e2138f93b15d2607353

Summary: These only had a 1 month expiration after D21271202.

Reviewed By: xybu, mingtaoy

Differential Revision: D21765601

fbshipit-source-id: 18b9f400e96b2ba39df650db9e6627f1cf917250

Summary:
If inline functions are enabled, actual frames used can be larger than the
address count.

Reviewed By: luciang

Differential Revision: D21371112

fbshipit-source-id: c1e602b62b6af427f11701bf25894a99f554ec47

Summary:
Baton is the primitive that allows fibers to block. The goal of this diff is to provide an `Async` interface to blocking calls that operate on Batons.

Overloading `await` directly was one option, however, there are 3 blocking baton APIs (wait, try_wait_for and try_wait_until). Morover these APIs can accept arguments (something async::await doesn't do so far)

So, this diff adds a bunch of wrappers that return Async<> results that must be 'awaited' on. This keeps static-analysis happy, and performs the onFiber check.

Reviewed By: yfeldblum

Differential Revision: D21667887

fbshipit-source-id: 774c19d223ebb0932c043e4c09635c5e85cc8241

Summary:
Fix race condition between releaseRefClearDelete and releaseRefDeleteOnly by fixing handling in releaseRefClearDelete of the case where the clear count is 1 and the delete count is > 1.

The correct behavior of releaseRefClearDelete when delete count > 1 is to release the clear count first and then release the delete count, instead of releasing the two counts in one step.

The following sequence of events demonstrates the problem in the absence of the fix:
- clear count == 1 and delete count == 2
- Thread 1 calls releaseRefClearDeleteCount
- Thread 2 calls releaseRefDeleteOnly
- Thread 1 atomically decrements the counts to 0 and 1
- Thread 2 atomically decrements the delete count to 0
- Thread 2 deletes the RequestData object
- Thread 1 call to onClear accesses deleted object

Reviewed By: yfeldblum, davidtgoldblatt

Differential Revision: D21705266

fbshipit-source-id: 8ec2151c4e98968beaef7e9ccec0a786ca5ae267

Summary: This diff factors out some of the logic needed in the stacked diff D21628842. We want `RocketClientChannel::sendRequestResponse` to work without a fiber if the caller is not already on a fiber, which means we cannot only use `Baton::TimeoutHandler` for the request's timeout callback.

Reviewed By: iahs

Differential Revision: D21628871

fbshipit-source-id: 2e739f205fcb0a255108fa88f94107378a1bab49

Summary:
In order to use the Python 3 streaming thrift client on platforms that
don't support coroutines, conditionally define anything that depends
on coroutines.

Reviewed By: yfeldblum

Differential Revision: D21692900

fbshipit-source-id: 9f26d598097181ae43d1ff377a22b1c2f789321e

Summary: Add missing includes of `folly/synchronization/Baton.h`.

Differential Revision: D21716396

fbshipit-source-id: 8c8fb818f3083b4fe04fd1c62923d7ba698dcedd

Summary: Add AsyncUDPSocket::maxReadsPerEvent_ support

Reviewed By: yfeldblum

Differential Revision: D21509991

fbshipit-source-id: 5c4f939c1cfdda8711864df3b5f5c58439f8d7bf

Summary: [Folly] Compare `exception_wrapper::class_name()` to `demangle(typeid(...))` in tests v.s. comparing it to string literals since the actual result of `...::class_name()` depends on the platform and whether `libiberty-dev` is available.

Differential Revision: D21711569

fbshipit-source-id: b2289d89923d6775c7e806eff9c3732ad37a0218

Summary: [Folly] Prefer `EndsWith` matcher in `exception_wrapper` test so that, in case of failure, the non-matching input will be shown.

Differential Revision: D21708674

fbshipit-source-id: 022f7d64a030417b80b8796ae3d78534295509fc

Summary: [Folly] Comments in `folly/synchronization/Lock.h`.

Differential Revision: D21689050

fbshipit-source-id: 65ffa2cf9bf3afe8740ae0c4a0d529c706397dae

Summary: [Folly] Extract utf-16 surrogate-pair helpers from `folly/json.cpp` into `folly/Unicode.h`.

Reviewed By: luciang

Differential Revision: D21634400

fbshipit-source-id: 64d15e79fe19cce5f5c9c38837d4663209b5f888

Summary: Add tracing to SemiFuture destruction that is potentially work-dropping.

Reviewed By: yfeldblum

Differential Revision: D21671142

fbshipit-source-id: 8e3e733d9a0845b00fe7ccf465e9c817a10fee59

Summary:
Now that we've deployed the new eden build with globfiles
support, we can enable the use of eden prefetch in the getdeps
build when running inside an EdenFS mount on Windows.

Reviewed By: fanzeyi

Differential Revision: D21692689

fbshipit-source-id: b42e778901976cf0385ec31056c227b2049162dc

Summary:
`folly::expected_detail::Promise<Value, Error>` won't build with exceptions
turned off (e.g. by passing `-fno-exceptions` on Clang's command line) because
its `unhandled_exception()` says `throw;`.  This diff replaces the `throw;`
with a call to `folly::rethrow_current_exception();`.

Reviewed By: yfeldblum

Differential Revision: D21682605

fbshipit-source-id: 294752590d54b0671395982ccdeb0386a742c01e

Summary: [Folly] Use `RequestToken` in `RequestEventBase` to speed up lookup.

Reviewed By: zhxchen17

Differential Revision: D21529465

fbshipit-source-id: eaaef7b48be2c5a8d76fb5f4a5293f7db7e68bed

Summary: [Folly] Fix technical typo in checking hw checksum support, but which makes no practical difference.

Differential Revision: D21689112

fbshipit-source-id: 8ad71d20a561f4bbf5c413ce94e7bec227d9d940

Reviewed By: yfeldblum

Differential Revision: D21666740

fbshipit-source-id: 96975e777df0745f343c5b722f1d8c05bb8ac32e

Summary: [Folly] Tweak `BlockingContext` in various ways.

Differential Revision: D21657789

fbshipit-source-id: 4b03b3b43dddc0e9011a7594001994f2594a1a5b

Summary:
[Folly] Lock utilities: `upgrade_lock`, `transition_to_shared_lock`, `transition_to_upgrade_lock`, `transition_to_unique_lock`, `try_transition_to_unique_lock`.

Note: `std::shared_mutex` does not have an upgrade state or any atomic transitions; there is no `std::upgrade_lock`; and `std::shared_lock` does not have a constructor from `std::unique_lock`.

Reviewed By: simpkins, nbronson

Differential Revision: D15532568

fbshipit-source-id: d32f387b26b57539857922c3090dc75ffcaaa538

Summary:
Migration from Future-returning executor-erasing collectX forms to
SemiFuture-returning forms, that are less risky in particular with coroutines.

Earlier changes added collectXSemiFuture and collectXUnsafe as a migration
path. We then migrated collectX callsites to collectXSemiFuture or
collectXUnsafe and switched the implementation of collectX to the SemiFuture
form.

This diff removes collectXSemiFuture completely.

Reviewed By: yfeldblum, mpark

Differential Revision: D21631886

fbshipit-source-id: c7b296d51f30c8b51ab0c7d06b1ce62c8cdd28d5

Summary:
`folly::detail::OptionalPromise<Value>` won't build with exceptions turned off
(e.g. by passing `-fno-exceptions` on Clang's command line) because its
`unhandled_exception()` says `throw;`.  This diff replaces the `throw;` with a
call to `folly::rethrow_current_exception();`.

Reviewed By: yfeldblum

Differential Revision: D21655976

fbshipit-source-id: 5f431372f2494a09f98e3b8c36a94e8475efff29

Summary:
This fails to build on win32 platform with:
```
xplat\folly\portability\pthread.cpp(142): error C2664: 'HANDLE CreateThread(LPSECURITY_ATTRIBUTES,SIZE_T,LPTHREAD_START_ROUTINE,LPVOID,DWORD,LPDWORD)': cannot convert argument 3 from 'DWORD (__cdecl *)(void *)' to 'LPTHREAD_START_ROUTINE'
```

This seems to fix it.

Reviewed By: Orvid

Differential Revision: D21623386

fbshipit-source-id: 1b18a74bd71ffc75f34a92074e07f3d69742c07d

Summary: [Folly] Small simplification of `AsyncSocket` read loop.

Differential Revision: D21514325

fbshipit-source-id: 0af27205b4aa2807a800430e8a2b3dea36356970

Summary: Add io_uring_submit_and_wait call

Reviewed By: yfeldblum

Differential Revision: D21653382

fbshipit-source-id: 945aa877a30b3c6431089a113ece5999d1217eb4

Reviewed By: yfeldblum, luciang

Differential Revision: D21656746

fbshipit-source-id: 5e19d72ccd524926c698b8814cc6c8f2c6133565

Summary: fsync the temp file

Reviewed By: yfeldblum, kevin-vigor

Differential Revision: D21668049

fbshipit-source-id: 85825f81dfbe10a452f7f946133bb3d8993850f8

Summary: [Folly] `rethrow_current_exception`, a `-fno-exceptions` compatibility wrapper for `throw;`.

Reviewed By: ispeters

Differential Revision: D21656269

fbshipit-source-id: 8f37fe71c6255162b8f05268cae04ac5927013d7

Summary:
gcc-6.5 on OS X <= 10.6 complains about the `\r` when this code path
is enabled. I don't think we officially support `__i386__`, but I want to
backport the patch from zstd. https://github.com/facebook/zstd/pull/1418

(Note: this ignores all push blocking failures!)

Reviewed By: yfeldblum, Orvid

Differential Revision: D13104652

fbshipit-source-id: 737a68253c7d90c3a1cc026069b048551cb5ced9