Summary:
I want to access the list of deps without the install
instructions attached later in this stack of diffs in `shell_builder.py`.

Refs: https://github.com/facebook/watchman/pull/639

Reviewed By: snarkmaster

Differential Revision: D9552419

fbshipit-source-id: 333cfa0c33bc1aa623de7ac3546a8f1785869a4f

Summary:
This is a bit of a tricky issue; here's the bug in python 2.7
that prevents using the function-like syntax:
https://bugs.python.org/issue21591

The nature of this bug is that scoped exec only works with nested functions
when using the `exec` statement in 2.7.   That is a syntax error in python 3,
which only allows using the `exec` function.

To make the function syntax work in 2.7 we therefore need to avoid using
lambdas or nested functions and have to resort to using a global variable
to act as the "closure".

Refs: https://github.com/facebook/watchman/pull/639

Reviewed By: snarkmaster

Differential Revision: D9552413

fbshipit-source-id: a2cb9d325e18fb6fb06b9e284a84f0a0c80cfe7e

Summary: Per T33416893, D9441402 caused a regression in FB4A. Goal: reenable lvalue-qual Future::onError(). Make it deprecated to minimize the chances of anyone creating new callsites.

Reviewed By: yfeldblum

Differential Revision: D9597490

fbshipit-source-id: 9c6a34c37c8f576e2497164bd440cdbc3565920d

Summary:
[Folly] Use `static_cast` v.s. C cast in `folly/Expected.h`.

And don't have empty namespaces - at least have a comment.

Reviewed By: igorsugak

Differential Revision: D9596449

fbshipit-source-id: 6b5b712d7d46eba4a83b7f62c0628a8bcda1f722

Summary: [Folly] Formatting of `FOR_EACH` etc uses.

Reviewed By: Orvid

Differential Revision: D9595505

fbshipit-source-id: bdb506f8904de2577c42ada6755a2af4f3efb095

Summary:
In `folly::symbolizer::ElfFile`, we `mmap` the file and access the file content via `mmap`-ed pointer directly. This is nice and efficient, but causes the whole program crashes on `SIGBUS` is the underlying frile is broken.

This Diff changes the first read of the file, for ELF magic number, to use `read` syscall and check return result, instead of accessing the `mmap`-ed pointer directly. This way APIs like `ElfFile::openNoThrow` would return error instead of crashing if the underlying file is broken.

It still doesn't solve the problem if the file becomes broken between we read the ELF magic number and we read file content with the `mmap`-ed file pointer later. But this is the best effort to check for the case with minimal performance cost.

Reviewed By: yfeldblum

Differential Revision: D9474328

fbshipit-source-id: 969e0323fcb4011e6a080d914452c9215d2fd13c

disable on android, fix getAllocatedMemorySize test to account for implementation-defined bucket_count for default containers

Summary:
This serves as a mitigation until we look further into the android crashes we
discovered in android instrumentation tests.

The bucket_count of the default constructed `unordered_map` is
implementation-defined. This diff takes that into account in the
`getAllocatedMemorySize` test.

`unordered_map|set` impl in Android NDK does not match the assumptions made in
`visitAllocationClasses`. Temporarily disable the tests with that method.

Reviewed By: mzlee

Differential Revision: D9578626

fbshipit-source-id: 355940b026f00f783bf65c2f1e10725d5aa920d3

Summary:
I noticed this while testing some changes; the comment
doesn't seem to match up to the `raise` statement, and attempting
to throw a list fails in python.

Reviewed By: strager

Differential Revision: D9566018

fbshipit-source-id: 563e43dfd30867993cc2a0f816db3b835a52e6df

Summary:
When running on macOS there is no /etc/issue.  When running
with `set -e` enabled, we don't want this to break the build.

Neither of these is possible today, but will be later in this diff stack.

While I'm in here, also print out the cmake version.

Refs: https://github.com/facebook/watchman/pull/639

Reviewed By: snarkmaster

Differential Revision: D9552416

fbshipit-source-id: 432b5b70678908de9f79787d37c3615a7f8649d3

Summary: Support for building Folly with autoconf is going away; this isn't going to keep working, so remove it.

Reviewed By: yfeldblum, philipjameson

Differential Revision: D9569019

fbshipit-source-id: 1fe99453c741ff9fc8632bbcfdd69df6b1f86d01

Summary: This means that Windows builds no longer require pthreads in order to run tests. (previously we were relying on the pthreads implementation we were using to provide semaphore.h for us)

Reviewed By: yfeldblum

Differential Revision: D9568125

fbshipit-source-id: 712f542c9a29627f72b41e1def2b79d591a33317

Summary: It's not a perfect mapping, but it's good enough.

Reviewed By: yfeldblum

Differential Revision: D9569405

fbshipit-source-id: 7559fb94f8a57717b5a594be0c05a99ac88c6c6e

Summary: The `(get|set)rlimit` stub implementations don't support setting memory limits, and I have no intention of implementing it, so disable it, as there are no meaningful memory limits on Windows to begin with.

Reviewed By: yfeldblum

Differential Revision: D9569424

fbshipit-source-id: 738d73f90eb33957ca4a80cb9b3c568034ef141e

Summary:
gcc8 fails compilation with:
    error: taking address of temporary array
      return {+A{Cs..., Char(0)}, sizeof...(Cs)};
So leverage c++14 to base manipulations on a local array.

Reviewed By: yfeldblum, ericniebler, Orvid

Differential Revision: D9310662

fbshipit-source-id: 0ba375ac0a4cca913e437290bc277c0c6bb8fdc8

Summary: These need to be escaped (they also don't work on Windows)

Reviewed By: yfeldblum

Differential Revision: D9566079

fbshipit-source-id: a202673c7cea62f3534ceff9cb81ee84aca65049

Summary: Platforms such as MSVC.

Reviewed By: yfeldblum

Differential Revision: D9569093

fbshipit-source-id: 3d836d5f660b610eed47a6692bc0ce33396fbfa5

Summary: as title

Reviewed By: Orvid

Differential Revision: D9568603

fbshipit-source-id: b5ce1ba8163af87f3ec6369407e3f8546e80afb1

Reviewed By: JunqiWang

Differential Revision: D9564136

fbshipit-source-id: 3eeb42e1629fdadfee758c3b9c92e167e126c946

Summary: This should work, probably.

Reviewed By: yfeldblum

Differential Revision: D9566267

fbshipit-source-id: 052295702a46edd85471c75ca82937dc3a508577

Summary: Because Bits.h feels the need to exercise all variants of popcount.

Reviewed By: yfeldblum

Differential Revision: D9565298

fbshipit-source-id: 307b3e4a70216c76d63104853e91e36a340ff1ab

Summary: MSVC wasn't treating this as an initializer list construction, so make it do so.

Reviewed By: yfeldblum

Differential Revision: D9565470

fbshipit-source-id: 15465b5c87a427920a7cad3f5204bb661ba230c2

Summary: Because, rather impressively, it wasn't being included transitively by anything here when compiling on Windows.

Reviewed By: yfeldblum

Differential Revision: D9565417

fbshipit-source-id: b83fa0b121d83cbeb7ff0bd79dec506b6549385e

Summary: MSVC defines timeval in a weird place, so include the portability header.

Reviewed By: yfeldblum

Differential Revision: D9564726

fbshipit-source-id: f9e43225a889bfb0395178ef809aaa81fb9ccc36

Summary: One of these warnings is informational, the other we will continue to blissfully ignore until either includes become cheap, or the eventual heat death of the universe, whichever comes first.

Reviewed By: yfeldblum

Differential Revision: D9565514

fbshipit-source-id: e5a8d8c868c122d4b06635b061540c8f329ec93d

Summary: Because the latter was removed in C++17, and MSVC no longer supports it.

Reviewed By: yfeldblum

Differential Revision: D9565195

fbshipit-source-id: 7603ac5280ccf540abec296c7c76a43142581eb3

Summary: Because if you tried to capture them, they wouldn't be usable as template arguments.

Reviewed By: yfeldblum

Differential Revision: D9565384

fbshipit-source-id: 9c4579848be1c5c48251c17b0f55443aade75eba

Summary: Because the former is standard and the latter doesn't exist on Windows.

Reviewed By: yfeldblum

Differential Revision: D9565344

fbshipit-source-id: c29c0623c87a20d4c16a29fec1a7311907c1c1cf

Summary: Not all platforms have pthread.

Reviewed By: yfeldblum

Differential Revision: D9565232

fbshipit-source-id: fced220115315f976dbea8c037869cb6aef461fd

Summary: Because forking is not a thing on Windows.

Reviewed By: yfeldblum

Differential Revision: D9565259

fbshipit-source-id: 140103d0df3b6d2d70a8e7d4eac96682e3430143

Summary:
Without this, `folly::gen`'s other `byLine()` implementations only
support a single read pass (usually through the whole input file) without
support for different formats through the file.

Reviewed By: yfeldblum

Differential Revision: D9540396

fbshipit-source-id: 6539851a70aed9a88d3c61986a459f60dfb7be5b

Summary:
Update readFull() and related functions to warn if their return value is
ignored.  Callers should pretty much always need to examine the return value,
as they need this to determine how much valid data was read.

Reviewed By: yfeldblum, meyering

Differential Revision: D9526760

fbshipit-source-id: c090044256bfe4bf9907a24ec6090eeae04296b3

Summary:
Files opened by `readFile()` should not be propagated to children processes
if another thread happens to spawn a subprocess while `readFile()` is running.

Reviewed By: yfeldblum, meyering

Differential Revision: D9553354

fbshipit-source-id: 2a6028b3a920bb1e76cd0b07f82e6af57d01c2a4

Summary:
A `CustomLogFormatter` can be enabled by setting the `formatter` option to `custom`. This formatter reads an option named `log_format` which specifies the format of the logs. The format string is similar to what `folly::vformat(format,...)` expects. Check out the following example format:

  {L}{m:02d}{D:02d} {H:02d}:{M:02d}:{S:02d}.{USECS:06d} {THREAD:5d} {FILE}:{LINE}]

The format above prints logs in the same format as GLOGs.

Another option `colored` takes in one of the following values: `never`, `auto`, and `always`. When enabled, the output will be colored in the following way:

{F135233848}

The config used for the image above is:
  FOLLY_INIT_LOGGING_CONFIG(".=DBG; default:formatter=custom,log_format={L} {FILE}::{FUN}():{LINE}],colored=true");

As you can see, we have also included the function name using `{FUN}`. This can be particularly helpful when debugging errors.

Reviewed By: simpkins

Differential Revision: D8974950

fbshipit-source-id: 87d624f53dd67c0630fb222093c46fe79ff44ee1

Summary:
F14NodeMap is a safe drop-in replacement for std::unordered_map. It is smaller
than std::unordered_map (32 bytes instead of 56 bytes) and has been shown
to be faster in many production use cases. This diff replaces the hash table
inside folly::dynamic. The effect is that sizeof(folly::dynamic) will drop from 64
bytes to 40 bytes, and use cases that use folly::dynamic as an object may also
get a CPU win.

Reviewed By: nbronson

Differential Revision: D9518110

fbshipit-source-id: 72daba13577c560bbb88c4d68a457afedf1af38e

Summary:
Adds a new InlineTask coroutine type that is executor unaware.

*Execution Contexts*
The InlineTask's coroutine will start execution on the execution context of the coroutine that awaits it. When the InlineTask coroutine awaits something internally, it will resume execution on whatever execution context the operation it awaited on completes on. The coroutine awaiting the InlineTask will be resumed on whatever execution context the operation completed on.

It has been put in the `folly::coro::detail` namespace for now to discourage usage outside of the operators exposed from `folly::coro` as it can be easily misused to accidentally run logic on the wrong execution context.

This type is based around the same design as the `std::experimental::task<T>` that was proposed in [P1056R0](http://wg21.link/P1056R0).

Reviewed By: andriigrynenko

Differential Revision: D9388154

fbshipit-source-id: bc94a9bf07459410ef423b4cfd3b57d13688f744

Summary:
Add a generic function for performing a blocking-wait on any awaitable object.

This obsoletes the use of the `.wait()` and `.get()` methods on  `folly::coro::Future<T>` and so these methods have now been removed.

Reviewed By: andriigrynenko

Differential Revision: D9384837

fbshipit-source-id: bcff5b348c024a3fc9337c67d996c447e0723fcc

Summary:
* Contract for Future::hasValue() had a typo - fixed that.
* Contract for Future::hasException() was inconsistent with hasValue() - made it consistent.

Differential Revision: D9527398

fbshipit-source-id: 93e1ebd768fa38476949818c0902e06a7dab5209

Summary:
Save python import cost in folly/python get_fiber_manager.

In my testing, using call_once here is safe even in multi-threaded and forked environment.
Though I wouldn't be surprised if a corner case is found in the future, as I couldn't find a definite answer about the safety of memoization Python C-API import calls.

Reviewed By: fried

Differential Revision: D9542909

fbshipit-source-id: c62a08ebf67bd4eb82f1bf4b7657c4a3a6daf4db

Summary:
The current futex API required a reference to a futex object in order to
invoke `futexWake()`, although this is not buggy by itself, it is technically
UB and nothing is stopping ASAN from catching this sort of use-after-free and
reporting it as an error.  Especially when the futex is represented as a
pointer, requiring a dereference to reach a member function

The bug can come up when you call `futexWake()` on a futex that has been
destroyed, for example

```
auto&& futex_ptr = std::atomic<Futex<>*>{nullptr};
auto&& thread = std::thread{[&]() {
  auto&& futex = Futex<>{0};
  futex_ptr.store(&futex);

  while (futex.load(std::memory_order_relaxed) != 1) {
    futex.futexWait(0);
  }
}};

while (!futex_ptr.load()) {}
futex_ptr.load()->store(1);
futex_ptr.load()->futexWake(1);
thread.join();
```

Here immediately after the `store(1)`, our thread could have loaded the value,
seen that it had changed, and never went to sleep.  Or it could have seen the
value as 0, went to sleep and immediately returned when it saw that the value
in the futex word was not what was expected.

In the scenario described above calling `futexWake()` is done on a "dangling"
pointer.  To avoid this, we just never dereference the pointer, and pass the
pointer to the futex syscall, where it will do the right things

A side benefit to the refactor is that adding specializations is very easy.
And we don't have to mess with member function specializations anymore, which
are inherently hard to work with (eg. cannot partially specialize)

The ADL extension points (currently implemented for `Futex<std::atomic>`,
`Futex<DeterministicAtomic>` and `Futex<EmulatedFutexAtomic>`) are

```
int futexWakeImpl(FutexType* futex, int count, uint32_t wakeMask);

FutexResult futexWaitImpl(
    FutexType* futex,
    uint32_t expected,
    std::chrono::system_clock::time_point const* absSystemTime,
    std::chrono::steady_clock::time_point const* absSteadyTime,
    uint32_t waitMask);
```

Reviewed By: yfeldblum

Differential Revision: D9376527

fbshipit-source-id: bb2b54e511fdf1da992c630a9bc7dc37f76da641

Summary:
Add the BZIP2 stream codec.

The `FlushOp::FLUSH` does not guarantee that the decompressor can read all the input processed so far, due to a bug in the bzip2 library. This is likely not important, since `FLUSH` is not a common operation, especially with bzip2.

Reviewed By: yfeldblum

Differential Revision: D9484325

fbshipit-source-id: 40770b6f301a16d86c4de8c2b0875f931f00cba2