Summary: This demonstrates a deadlock that's possible when using InlineTask.

Reviewed By: lbrandy

Differential Revision: D9883432

fbshipit-source-id: 45bb14181f6cfec41851315c0369232a30ba13d9

Summary:
It seems like memdup may fail in certain builds with nullptr, 0 and cause OpenSSL to put a malloc failure on the stack.  See https://github.com/openssl/openssl/blob/master/ssl/ssl_lib.c#L2752

This diff clears the error stack when we do this.

Reviewed By: knekritz

Differential Revision: D10037676

fbshipit-source-id: ea7b136cafde818564e8ee2401f0cef30f52a45d

Summary: Because DeferredExecutor now acts as a simple proxy to user executor, we can actually support multi-hop SemiFutures.

Reviewed By: LeeHowes

Differential Revision: D9928793

fbshipit-source-id: c7e030bac3582b2931bc9639c766a4398ce23a76

Summary:
This diff adds support for doing heterogenous lookups
to the dynamic::get_ptr(dynamic) function.

Reviewed By: yfeldblum

Differential Revision: D10007208

fbshipit-source-id: 2453f3fe8ab36dfd9e5627aa75cf8eb7aee51699

Summary: Move the wait() and timed_wait() functionality out of DeferredExecutor. This is now possible because folly::Futures are using Executor::KeepAlive to manage executors' life-time.

Reviewed By: LeeHowes

Differential Revision: D9925417

fbshipit-source-id: e9818417048f44ee66648439e781060fdabca6b9

Summary:
MSVC (14.12.25827, VS 2017 15.5) fails to compile with internal error when using `/std:c++14` parameter:
```
Future-inl.h(1484): fatal error C1001: An internal error has occurred in the compiler.
(compiler file 'msc1.cpp', line 1507)
 To work around this problem, try simplifying or changing the program near the locations listed above.
```

Reviewed By: yfeldblum, Orvid

Differential Revision: D10018915

fbshipit-source-id: ba886265b68684aacc2bf471c79ba6c39e611c8e

Summary:
If `Source` isn't copyable, parallel generator cannot be composed
further due to implicitly deleted move ctor.

Example:

```
byLines(File(...))
| parallel(map(...))
| sum;
```

Reviewed By: ddrcoder, ot

Differential Revision: D10004477

fbshipit-source-id: 51ce23e834413bd53547a683f8ed89a1811f74a2

Summary:
Some of the error messages had the incorrect expected types.
Fix those occurences

Reviewed By: shixiao

Differential Revision: D10003775

fbshipit-source-id: ba8e3b7d14ee2c5442dfd24b9fbf7775d601f6f0

Summary:
Deprecate the legacy forms of Future::then to assist migration to thenValue and thenTry.

This is part of a longer process to improve type correctness for folly::Future. By separating the forms of then we no longer have ambiguity when passing a polymorphic lambda as a continuation and can more easily tweak the implementation to modernise it.

Reviewed By: yfeldblum, WillerZ

Differential Revision: D9974379

fbshipit-source-id: 26f986329cb5d2d33d366b8f9e7b2664656642ff

Summary:
Add implementation for std::hash<MacAddress> so that it can be used in
maps.

Reviewed By: yfeldblum

Differential Revision: D10004544

fbshipit-source-id: 46cb177577123b1248cd8ff679f0fbe286596819

Summary: To allow deprecation, remove use cases of Future::then inside FutureSplitter.h

Reviewed By: yfeldblum

Differential Revision: D9995264

fbshipit-source-id: d44c63c45a08ca675b9bff33fabcbb2836ea2a64

Summary: Went a step too far removing then from inside Future-inl.h. Map needs to be generic over value/try. Just remove that change for now.

Reviewed By: yfeldblum

Differential Revision: D10002565

fbshipit-source-id: c37f744d47f9dd123114b6b62c1c832519cb4f8e

Summary: [Folly] A faster `empty()` shortcut in `IOBufCompare`.

Reviewed By: spalamarchuk

Differential Revision: D9981606

fbshipit-source-id: 4a9762b83fe2f26b1387c162a1e23c201088977f

Summary: [Folly] Let `IOBufCompare` have a self-compare shortcut.

Reviewed By: vitaut

Differential Revision: D9981523

fbshipit-source-id: e984cffc2816a0f4f06bf418b16d4d3eb055f6ef

Summary: To allow deprecation, remove use cases of Future::then inside Future.h/Future-inl.h.

Reviewed By: yfeldblum

Differential Revision: D9984465

fbshipit-source-id: 48e8a5b2985c86fd5384e43e6df282da298b1d99

Reviewed By: zertosh

Differential Revision: D9993527

fbshipit-source-id: 52c4c0bfe6b5c73c652dd350f599fb846a003a4f

Summary: (Note: this ignores all push blocking failures!)

Reviewed By: spalamarchuk

Differential Revision: D9994424

fbshipit-source-id: 2055d3b09ce2e1d783624a9902463efade4714d4

Summary:
This is known at compile time, but we still pay the cost since
SettingCore has to look it up through its own reference.
Hint it directly at the callsite instead.

Differential Revision: D9983460

fbshipit-source-id: 1c6ca5e3cbd9e859ba88d641cdca59f0d4d65dfb

Summary: Requires a new header to break the circular dependency

Differential Revision: D9983196

fbshipit-source-id: f15ab3c6128d30906ac25384df92813aa9d76c9a

Summary:
- Change the get() API to return T by value if T is small
- Internally store small T in a global atomic to avoid thread local lookups

Differential Revision: D9982433

fbshipit-source-id: c42d06a75b9ff48307a90f77d0bb72bfa6375d6c

Summary: Makes further changes cleaner

Differential Revision: D9982335

fbshipit-source-id: 19268c8e3fc1b6aee589a8d3d38b0348db8c5a43

Summary: inline the small, fast path only

Differential Revision: D9982097

fbshipit-source-id: f1bcbab2b444d372e178aa548175a5c27be972b6

Summary:
With this change the setting lookup is now inlined at each callsite as:
```
if (UNLIKELY(global_setting_ptr == 0)) {
  global_setting_ptr = &initialize_setting(); // OK to race
}
// use *global_setting_ptr
```

Differential Revision: D9981958

fbshipit-source-id: a04b296d8345d1bb37dc06ce72277e00e745a94a

Summary:
The folly::coro::Task<T> and folly::coro::Future<T> types would previously return an lvalue-reference to the result when you co_await them.

This means that for move-only types that code like `auto x = co_await someTask;` would fail to compile. Instead you would need to write `auto x = std::move(co_await someTask);`.

Awaiting a Task<T> or Future<T> now returns type T instead of T&.

As part of this change we now only allow co_awaiting an rvalue folly::coro::Task<T>, folly::coro::Future<T> as well as folly::coro::Future<T> to indicate that the operation is destructive and a one-time operation.

Reviewed By: andriigrynenko

Differential Revision: D9731525

fbshipit-source-id: bee9e633b57b203a0d048cf3eb0e2fc48b899481

Summary: It was possible for interruptHandler() to be called concurrently with the ThreadWheelTimekeeper shutdown resulting in use-after-free when trying to schedule work on the EventBase.

Reviewed By: yfeldblum

Differential Revision: D9948555

fbshipit-source-id: 4eff3cce0488778559dffc47bfc6eb59a65bb203

Summary: The && overload here caused me to call the wrong function and delay an error as it changed the type of the Future completely. There seems to be little benefit to making this r-value qualified, and is likely a side effect of consistency with the forwarding reference in the non-Try overload.

Reviewed By: yfeldblum

Differential Revision: D9979805

fbshipit-source-id: e8121d5bc2e5fe02d38d6ea9cb83adbe98010df2

Summary: The tests that would have caught this aren't currently in the CMake file, but this is needed before trying to use the intrinsics here.

Reviewed By: yfeldblum

Differential Revision: D9952025

fbshipit-source-id: 40c7034ef7e0877ccb95525fa09ab46a3526af0b

Summary: FOLLY_TLS is not guaranteed to be defined, so add a check

Differential Revision: D9952230

fbshipit-source-id: 1f85d1ec39514b5b3c355339a74d067ecd85b267

Summary: [Folly] Lift some crc magic constants in `folly/hash/detail/Crc32CombineDetail.cpp`.

Differential Revision: D9934404

fbshipit-source-id: e2508a22e300e22f2265383a0a111df53d2738c5

Summary: Fix opt mode error: unused parameter 'data'

Reviewed By: aary

Differential Revision: D9970478

fbshipit-source-id: 8d78e5f12613776486e33e8c685e400cd88f9d03

Summary: This change will allow simplifying DeferredExecutor and implementing both wait() and timed wait() using custom executors.

Reviewed By: LeeHowes

Differential Revision: D9890644

fbshipit-source-id: ad8472213e7022ccc23b4a9c48f8b9c4b9c70dfa

Summary: This will make it easy to use coro primitives from fibers code.

Reviewed By: lewissbaker

Differential Revision: D9932012

fbshipit-source-id: cd0524ccfee7bee9405f29ddaa3533084b18b38b

Summary: [Folly] Fix move-of-const in `ThreadWheelTimekeeper`.

Reviewed By: capickett

Differential Revision: D9950653

fbshipit-source-id: 76d45cdac1026c41de73489936c4afdac3f57fd7

Summary: - Adding hasValue() function

Reviewed By: yfeldblum

Differential Revision: D9926823

fbshipit-source-id: 2316eb67cd6a906850ad0dec913988f116f840d6

Summary: [Folly] Just use `constexpr` keyword in `folly/lang/Bits.h` - since they are now template functions, `constexpr` is okay even when all instantiations would be non-`constexpr`.

Reviewed By: Orvid

Differential Revision: D9934124

fbshipit-source-id: cdf4fc87262a058af6f63e28bb2c390e44f2cad0

Summary:
`hash_range(c.begin(), c.end()` combines hashes of individual elements of a
container in an ordered manner. This diff provides the equivalent for
unordered containers.

Unlike `hash_range`, `commutative_hash_combine_range` defaults to `folly::Hash`
as its hasher; it mixes the individual hash if the `hasher` is not deemed
avalanching.

It uses a commutative accumulator described in this paper:
https://www.preprints.org/manuscript/201710.0192/v1/download
In the experiments in the paper, the symmetric polynomial yielded a better
spread of hash values and lower collision rates than `+` or `xor`.

Reviewed By: yfeldblum

Differential Revision: D9688687

fbshipit-source-id: c812b25975a53a868d98f78645146cb8bdbb5c32

Summary:
`IOThreadPoolExecutor.pickThread()` has some sketchy behavior where, if the
threadpool has no threads, it returns `*thisThread_` unconditionally. My read
is that this can happen only in two cases.

First, `getEventBase` could be called from outside the threadpool while the
threadpool has no threads (either because it was stopped or because somebody
set it to 0 threads.) In this case, `*thisThread_` will be nullptr and
`getEventBase` will try to immediately dereference it, (hopefully) crashing.

Second, `getEventBase` could be called from inside the threadpool, but from a
thread which is shutting down. This is also somewhat dodgy, because by the time
we get to this case (`me && !contains(threadList_.get(), me)`), stopThreads
will have already called terminateLoopSoon on the evb. While it would still be
possible to use the evb safely (in the sense that one could write a program
which does not hit undefined behavior), I think it'd be strongly dependent on
the way IOTPE and EventBase are currently implemented and therefore probably a
bad idea.

I don't have the bandwidth to really dig into this now and figure out what the
correct behavior is, or how to fix people who might be relying on the broken
one. But I figure I can at least leave a comment behind that this is something
maybe worth looking into.

Reviewed By: meyering

Differential Revision: D9754314

fbshipit-source-id: 6851d49e634707cacc803a95763eda023967b932

Summary: Because reasons :(

Reviewed By: yfeldblum

Differential Revision: D9923447

fbshipit-source-id: 7a3a9ab86764dc5c7f5b70f2f728aec8b67bdf1f

Summary:
This diff adds spurious rehash at places where the caller should
not be assuming reference (unless guaranteed) or iterator stability,
allowing ASAN to detect these problems.  Under ASAN it also junk-fills
deleted value_type instances that can't be immediately deallocated.

Reviewed By: yfeldblum

Differential Revision: D9826273

fbshipit-source-id: 2d543473b17ffd7debb52082a3e72842121ece1a

Summary:
An allocator that uses a jemalloc arena backed by 2MB huge pages.

This uses the jemalloc extent hooks to let jemalloc hand out memory backed by huge pages.
The huge pages are allocated when init(int nr_pages) is called, and does not grow (although extending it to grow in the future should be fairly straightforward).

madvise is used to mark the pages as huge. It does not guarantee success, and the only way to tell is by looking at /proc/<pid>/smaps and the total huge pages in /proc/<pid>/status.

The HugePageAllocator can be used as template parameter for stl and folly collections.

Reviewed By: yfeldblum

Differential Revision: D9297627

fbshipit-source-id: 734ca1b19972a8b5d7a12ee60cba1ffa3da50dc2