Summary:
This diff expose the user counters when using json verbose mode.

(Note: this ignores all push blocking failures!)

Reviewed By: Orvid

Differential Revision: D15147243

fbshipit-source-id: ebd13710d683163bdc83274bf227a2eb8d6053b3

Summary: [Folly] Rename `co_schedule` to `co_reschedule_on_current_executor`, which is longer but which more clearly identifies the behavior.

Reviewed By: lewissbaker

Differential Revision: D15215844

fbshipit-source-id: aada82e8985224608415ee5458ac8138ce13f8d0

Summary:
- Replace `fbstring_core` private copy assignment special member function
  with an explicit `=delete` equivalent.
Pull Request resolved: https://github.com/facebook/folly/pull/1123

Reviewed By: Orvid

Differential Revision: D15216597

Pulled By: yfeldblum

fbshipit-source-id: 319109f739d972e9efc73a7dbc2dd6ae401b18fc

Summary:
- Since folly requires GCC 5+ compiler, remove several macros which were
  guarding against when we supported GCC 4.9.
Pull Request resolved: https://github.com/facebook/folly/pull/1096

Reviewed By: LeeHowes

Differential Revision: D14681043

Pulled By: LeeHowes

fbshipit-source-id: fcc2cd25e00c512f7154a1568003e7c2f0df9c54

Summary: constrain transform submit on the validity of the underlying submit

Reviewed By: ericniebler

Differential Revision: D15104546

fbshipit-source-id: 7eea68074e4f9a4e51e77af55abf8c1b10e936af

Summary:
- The current TimedRwMutex behavior is biased heavily towards readers. This can lead to writer starvation under heavy read load which can result in writers unable to acquire the lock for several minutes
- The current change adds a writer priority mode to TimedRWMutex which can be specified at creation time. The default behavior will still be reader priority and since the logic is handled through templates it shouldn't introduce any side effects for the reader priority behavior

Reviewed By: matheweis

Differential Revision: D15204620

fbshipit-source-id: e940868aa9cec8f4a0a6b59c695ed3beca11c11a

Summary: Try and answer some common recurring questions

Differential Revision: D15081359

fbshipit-source-id: 1bb0fde2194b9b8304d556f2a4b690ac4aa1e414

Summary: Enforce SingletonThreadLocal uniqueness without FOLLY_TLS

Reviewed By: yfeldblum

Differential Revision: D15182815

fbshipit-source-id: 5f2daba43a84092c5fdc43a4e94465446e828b5b

Summary: The `co_invoke()` helper function now supports callables that return `AsyncGenerator`.

Reviewed By: rhodo

Differential Revision: D15054793

fbshipit-source-id: d7edc7db7d2fa44115f2c25872a6849b1e07bd4c

Summary:
Adds some functions for concurrently awaiting a collection of tasks but with an upper bound on the number of tasks to process concurrently.

- `folly::coro::collectAllWindowed(range, maxConcurrency)`
- `folly::coro::collectAllTryWindowed(range, maxConcurrency)`

Reviewed By: andriigrynenko

Differential Revision: D15014011

fbshipit-source-id: 8f62dbae47e0548eacec8dc59239668040d3a771

Summary: SingletonThreadLocal benchmark

Reviewed By: yfeldblum

Differential Revision: D15116981

fbshipit-source-id: cf44104542614ee9b79a88a14182c2ddbaf7a34c

Summary:
`UnboundedQueue::try_dequeue` supports non default-constructible types but `UnboundedQueue::dequeue` does not.
This diff fixes it and cleans up functions made obsolete.

Reviewed By: shixiao

Differential Revision: D15199743

fbshipit-source-id: b2041c5cbffbbcfe153c218c467cfe04e5e5a544

Summary:
- As of C++17, `std::string` contains a non-const `data()` member function.
  This is useful when working with C APIs that have `char*` output parameters.
- Implements P0272 for `FBString`:
  http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0272r1.html

Example of workaround without this:

```
  fbstring s{};
  some_api(&s.front());
```

- The expression `&s.front()` above causes undefined behavior since the string
  is empty.

- Add non-const public member function `data()` to `fbstring` which
  delegates to `fbstring_core`.
Pull Request resolved: https://github.com/facebook/folly/pull/1116

Reviewed By: Orvid

Differential Revision: D15213884

Pulled By: yfeldblum

fbshipit-source-id: f533542db0a7e34c09a622e5257d7c1e66ab75e2

Summary:
[Folly] `small_vector<T>::get_allocator()` for compatibility with other container types.

While `small_vector` is not actually parameterizable over the allocator types, compatibility with generic interfaces such as the presorted-ctor interfaces of `sorted_vector_set` and `sorted_vector_map` require `get_allocator`.

Reviewed By: vitaut

Differential Revision: D15194509

fbshipit-source-id: 3a7842d5640f8e95a09c4cd8e2a9ab6ffc4fe9ac

Summary: [Folly] container_type member alias for `sorted_vector_set` and `sorted_vector_map`, following the protocol from `std::queue`, `std::priority_queue`, and `std::stack` container adaptors.

Reviewed By: vitaut

Differential Revision: D15194108

fbshipit-source-id: 91332478c66779690f415d598b99274b8ee52df4

Summary: [Folly] `kGlibcxxVer`, from `_GLIBCXX_RELEASE` if available.

Differential Revision: D15209042

fbshipit-source-id: 077e94db623f33ebc3ec2cb93a5d7774d384a863

Summary: [Folly] Always `#include <bits/c++config.h>` if possible from `folly/portability/Config.h`. This is the libstdc++ (glibcxx) version/config/features header, so include it alongside the corresponding glibc version/config/features header.

Differential Revision: D15209040

fbshipit-source-id: 6f624283f58523d74317f3d998aa99c1c50ee2b3

Summary:
Split validateRecord to allow validating header and data separately. This is
useful when the header is used while storing records on storage devices.
Validating header alone before validating the record helps avoiding unnecessary
device reads. It also helps avoiding unnecessary memory allocations while trying to read invalid dataLength bytes present in an invalid header.

Reviewed By: yfeldblum

Differential Revision: D15184763

fbshipit-source-id: e1f008df82cf2d040322b8d62135ea2b5e0dcc0e

Summary: [Folly] Use `noexcept` in `IOBuf` v.s. reimplementing it via explicit `catch` and `abort`, with the intended side-effect of adding support for `-fno-exceptions`.

Reviewed By: simpkins

Differential Revision: D15208427

fbshipit-source-id: bcf9d7b0f1a556e43d996c84b1df43ff891da974

Differential Revision:
D15073229

Original commit changeset: 417a1f4e42d0

fbshipit-source-id: 689b010627482e9a05f3dd34e89baae485875025

Summary:
This is useful especially on Windows where the up-to-date
checks for the dependencies take a long time.

The idea is that you might run this to start:

```
$ getdeps.py build eden
```

and then while in the edit/debug/build iteration cycle:

```
$ getdeps.py build --no-deps eden
```

Reviewed By: pkaush

Differential Revision: D15200352

fbshipit-source-id: 086f2f49db967ef4d1914a69fa80067104d79136

Summary:
Previously, we were using autoconf to build sqlite,
but that isn't available on Windows.  Instead, here's a builder
that generates a little cmake configuration for building and
installing sqlite.

Using cmake for this means that we can test the same builder
on all platforms that need to pull in sqlite.

Reviewed By: simpkins

Differential Revision: D15179387

fbshipit-source-id: fccf969182565982bd5be55545a2d2625aa99124

Summary:
When running in FB infra, prefer to download from our local LFS
server rather than going out to the internet.

Fall back to a normal internet download if the LFS get fails for some reason.

Upload to LFS after successfully verifying the hash for the downloaded archive.

Add a subcommand that performs a fetch for all possible platforms so that it
is easier to ensure that the lfs-pointers file is up to date.

Reviewed By: simpkins

Differential Revision: D14978660

fbshipit-source-id: 240fc32fc7003d1e06c88b80d85054dae36e2f31

Summary:
In order to pull in the treemanifest and other libraries
from our mercurial repo, add a manifest file for it, and then
adjust the logic in our cmake module to look for it.

The fb-mercurial manifest just copies the source tree to the
installation dir.  In the future, we could teach it to invoke
the build for real.

Reviewed By: simpkins

Differential Revision: D14969806

fbshipit-source-id: cb270c5003a1c134eeea92c7481a84938f1c5957

Summary:
This prints out the installation or source prefix for a given project.
This is useful in eg: packaging scripts to figure out where they can
find the built artifacts.

Reviewed By: simpkins

Differential Revision: D14967378

fbshipit-source-id: 7e1b5de2ca7219af24cfb07b4b42de22aa410469

Summary:
If `testpilot` is available, generate a buck compatible json file describing the available test binaries and feed that to testpilot to have it run the tests.

A later (yet to be written) diff will be able to pass appropriate flags down to testpilot in continuous runs and that will allow testpilot to auto-disable and file tasks for tests in the opensource builds.

Reviewed By: simpkins

Differential Revision: D14766856

fbshipit-source-id: 4e144ff18f6788cf5e830d29788eabd2dbbae46a

Summary:
The loader makes it possible to monkey patch the functions
that are responsible for loading manifests.  It is intended to be use
in tests that are run in bucks sandboxed environment and that don't
have direct access to the manifest files on disk.

Reviewed By: simpkins

Differential Revision: D14781326

fbshipit-source-id: 18f69f8ce5768dc605b1a9388a80b7b7b9ffe0f4

Summary:
previously, a relatively lame hash was computed to use
for the build directory based on some hash of the source directory.
That was good enough to get things off the ground, but in the
interest of being able to cache the build outputs and safely
invalidate them we need a slightly more rigorous implementation.

This diff computes a hash based on the manifest contents and
relevant environmental factors.

The hash is used to name the build directory which will ultimately
be cached eg: using the travis/appveyor cache directory configuration
and some other means for the FB internal CI.

The hash needs to be sufficient that we change the hash when
the manifests change.  We can tolerate a false positive change
in hash (it just means that a build will take longer), but
cannot tolerate a false negative (which would result in an
incorrect build).

Reviewed By: simpkins

Differential Revision: D14710332

fbshipit-source-id: ebc2e74eafc6f3305d4412a82195bc9fb9dfa615

Summary:
This command schedules a facebook specific sandcastle job for the current
commit in your repo for each of the platforms we have support for in
sandcastle.

You can use `--dry-run` to have it print out the job specs.

To support this, I've moved around some of the support utilities
to make it easier to import them.

Reviewed By: simpkins

Differential Revision: D14710330

fbshipit-source-id: fb1e2a2ce78e52894291159514977da97028b37f

Summary:
Adds a `test` subcommand that runs the tests for project.
We're mostly interested in the 1st party facebook projects for this.

The `sandcastle` flow will run the `test` subcommand just for the "leaf" project--the one named on the command line.

Reviewed By: simpkins

Differential Revision: D14710331

fbshipit-source-id: 7d04a46cfd723894d61018de2f230140b52285ac

Summary:
This cleans up the scratch dir, removing everything except
for downloaded tarballs.

Reviewed By: simpkins

Differential Revision: D14781328

fbshipit-source-id: 9304e44a268cf7996c5e572a2eca219aefbf4b46

Summary:
This fixes a TODO; in our CI environment we want to use the
real shipit, so we'll use this flag to make that happen.

Reviewed By: simpkins

Differential Revision: D14695576

fbshipit-source-id: 64ee72c210e2472d295dcbd39c86549273b68452

Summary:
This makes it possible to disable eden (and thus thrift) support.

I've defaulted this to off so that we don't break the existing
watchman CI.

Reviewed By: simpkins, strager

Differential Revision: D14726767

fbshipit-source-id: 0f4d0597d901a91850f1ba6d71609c059c064c22

Summary:
adds the command that is used to drive a build.

The dependencies of the named project are computed using
the same mechanism behind the `list-deps` subcommand.
If a manifest specifies that it uses the `cmake` builder then
a dependency on `cmake` is synthesized so that we can download
a recent version of cmake ahead of time.

The build command uses the fetcher to update the src and then
executes the builder.

Reviewed By: simpkins

Differential Revision: D14691011

fbshipit-source-id: 31ae59614651ef021a9505e89c13b5717b440071

Summary:
this could do with a better name; the NopBuilder doesn't actually
build anything, but instead copies some files to the destination location.

This is used together with eg: cmake to install pre-built binaries downloaded
from a tarball.

Reviewed By: simpkins

Differential Revision: D14691015

fbshipit-source-id: a938e977aa4ec5a664bdb8085ff708319a204594

Summary:
the boost builder knows how to perform the non-standard
configure and build for boost.

Ideally we'd just build this statically and be happy but there are
some nuances I've observed while building on different platforms:

* One of our projects (thrift or wangle) explicitly requests static
  boost linkage for reasons unspecified
* on darwin the install_name is broken when building dynamic libs

For the sake of expediency in getting getdeps up and running, the
solution for the moment is to build static on posix systems and
build both static and shared on windows systems.

Reviewed By: simpkins

Differential Revision: D14691009

fbshipit-source-id: 634770a6f53c3ada42d1877cc6c3dacc6eed7d18

Summary:
the openssl builder knows how to perform the non-standard
configuration and build steps to build openssl.

On Linux systems the manifests for our projects don't mention
openssl, causing them to pick up the system openssl.

On Mac, apple don't ship openssl headers so we need to build our own.
On Windows there is no standard openssl installation so we also need
to build our own.

As a result, this builder only works on windows and mac.

Reviewed By: simpkins

Differential Revision: D14691010

fbshipit-source-id: 9f8979f9eaeb5209c290cf4f43c97c0cb43d13a2

Summary:
this builder is used to bootstrap the ninja build tool.

On Windows and mac the manifest for ninja is set to download a pre-built executable.
While pre-built executables are available for linux they aren't portable enough
for our purposes so we need to be able to build it for ourselves.

Reviewed By: simpkins

Differential Revision: D14690992

fbshipit-source-id: b60fd02ad04f58dc7c2931280341791270609737

Summary:
the cmake builder knows how to use cmake to configure a build
for (preferably) and out-of-src build.  The `cmake.defines` section of
the manifest is used to pass `-Dkey=value` options to the cmake configure
command line.

We prefer to use `ninja` to execute the build so that we can use more
cores than 1 on Windows and just for consistency across platforms
with mac and linux.

Reviewed By: simpkins

Differential Revision: D14690998

fbshipit-source-id: 8102e8b4a47da515ca001772788ed0e5f2645ad7

Summary:
the autoconf builder performs an out-of-source build using
the autoconf suite to configure the build rules.

The `autoconf.args` section from the manifest is passed to the `./configure`
command line.

If an `autogen.sh` script is present then it will be used to regenerate
a missing `configure` script, otherwise we'll try `autoreconf`.

Reviewed By: simpkins

Differential Revision: D14691002

fbshipit-source-id: ab8cceafb833dab513d5a50c65f4c895a4f40047