Summary: The oss jobs are causing problems to the developers now, lets disable them temporarily.

Reviewed By: StanislavGlebik

Differential Revision: D21400898

fbshipit-source-id: f7a3567056633d9eef98a8d05a37cd029c9e506c

Summary:
While compiling EdenFS on Windows, for every single file being compiled,
cl.exe complains that 2 incompatible /std option are being passed in on the
command line, a c++latest, and a c++17. The former is added by this file,
while the later is added by cmake via CMAKE_CXX_STANDARD.

I'm not exactly sure why the default is set to c++latest, but it's guaranteed
to always force cl.exe to generate a warning, as it simply cannot be set via
CMAKE_CXX_STANDARD.

A better fix to this would be to set CMAKE_CXX_STANDARD to 17, but since
folly still supports cmake 3.0, that didn't know about c++17, this can't
be done.

Reviewed By: yfeldblum

Differential Revision: D21392360

fbshipit-source-id: fc0e29ed89adf0a2c0b0193e7bd5f033463ed376

Summary: Add support for detecting x86_64 vectorized AES (VAES) and vectorized carry-less multiplication (VPCLMULQDQ) CPU flags.

Reviewed By: yfeldblum

Differential Revision: D21271564

fbshipit-source-id: 08be20c07d6dfb562b8a0c1e86b77a9a416445fb

Summary:
When the commit hash changed in fbsource, we would correctly decide
that we'd need to rebuild first-party projects but we would incorrectly skip
running the fetcher.update method.  This would mean that we'd not perform the
shipit run and that our shipit tree would diverge from the source tree.

This commit resolves this by performing the fetcher.update but ignoring
the source update status in this case.

Reviewed By: xavierd

Differential Revision: D21364131

fbshipit-source-id: b4001e549c7d3f27aa4a21b19893c9bb7c0f6d1f

Summary:
We didn't do a great job of recognizing that we'd need to
build a project when one of its dependencies had changed: we relied
chiefly on the dependency hash for this and could fail to handle
changes in individual source files.

This commit helps to improve this situation by checking to see if
any installed files in the dependencies of a manifest are newer than
the most recent built time of a given manifest.  If so, we'll trigger
a build.  We try to be reasonably smart about deciding when to trigger
a cmake reconfigure if it looks like cmake files in the deps have
been changed.

Reviewed By: xavierd

Differential Revision: D21364132

fbshipit-source-id: 7534496e10d1f532aa9cf865900ace84a8785327

Summary:
Only run cmake reconfigure for .cmake, .cmake.in and CMakeLists.txt
files changes.

Previously we would reconfigure for any change to a file with a path that
matched `cmake` which could result in false positives in cases where
you may be iterating on .py or .c files in shared cmake directories.

This also reclassifies non-cmake files under fbcode_builder/CMake as source
files so that we run cmake for those; previously they would cause a
reconfigure and build, now they just cause a build.

Reviewed By: xavierd

Differential Revision: D21364133

fbshipit-source-id: a1231f657d6c6056b269656c677d3449d8715cf6

Summary:
Our linter really wants to include formatting changes unrelated
to my diff stack.

This is a formatting only change to avoid clouding my diffs; no functional
effect.

Reviewed By: xavierd

Differential Revision: D21364519

fbshipit-source-id: 7670dd4154e788f593f256aabdfdeef6d17aeec4

Summary:
Previously getdeps would remove the entire top-level `CMakeFiles` directory
from the build output when it wanted to invalidate the CMake cache.  This
directory is used to keep all of the compiled object files for any libraries
or executables defined in the top-level CMakeLists.txt file.  Blowing away
this directory forces all of these sources to be re-compiled, even if this was
not necessary.  This is particularly problematic for folly, which compiles all
of its source files via rules in the top-level CMakeLists.txt target file.

I did have the code still blow away the CMake error and output logs in this
directory: in the past I have seen situations where CMake would not update
these files on new CMake runs if they already existed.

Reviewed By: wez

Differential Revision: D21360668

fbshipit-source-id: 6fcd1a8e371d756114fbab60d8636be8cd5f8978

Summary:
Update the manifest file for folly to indicate a dependency on lz4.
folly does not require lz4 be available, but it will use it if it is found at
configure time.

getdeps is unfortunately not strict about providing projects only with the
dependencies that they require at build time.  This causes it to sometimes
make lz4 available to folly (if you are also building another project that
requires lz4), and sometimes not.  This ends up causing changes in
folly-config.h depending on which projects you are building, forcing all of
the folly sources to be recompiled.

In the future we perhaps should update getdeps to consistently only pass in
include directories for dependencies actually listed in the manifest file.
However, specifying that folly depends on lz4 also works to mitigate this
particular issue for now, and it is also generally desirable to build folly
with lz4 support.

Reviewed By: wez

Differential Revision: D21359995

fbshipit-source-id: aaf61671b7750d6c47e3613c732d220b3311b5ba

Summary: [Folly] `c_array`, a container for returning C arrays from `constexpr` functions. May be used until C++17 may be assumed, since C++17 specifies `constexpr` member functions of `std::array` which make `std::array` suitable for the purpose and a better choice since it  can house zero-sized arrays.

Reviewed By: markisaa

Differential Revision: D21382680

fbshipit-source-id: 56a6528bb05988c1f1c8916f9267575783dc6276

Summary: fix D21371986's sadness (SEGV  for `folly::make_exception<TException>().class_name()`)

Reviewed By: yfeldblum

Differential Revision: D21373766

fbshipit-source-id: 0dd951fd9393e73c5c38670167312c3d2e619fb5

Summary:
Sorry about the churn (& time spent on this), but this backs out the change in D20671620.

Despite best efforts it seems I've missed something and the issue runs deeper than TSAN observing thread exit.

My change has possibly only altered the timing of events, and has exposed another related failure during testpilot runs that didn't occur before, so I think it's best to return to baseline.

Reviewed By: yfeldblum

Differential Revision: D21310561

fbshipit-source-id: 369865f185415123ea8fddf6007115b87ea8fe82

Summary:
In the initial stages of the windows port we had
problems building rocksdb on windows, so we disabled it.

These days we're able to build it and detect it--we even
require it in the cmake code, but hadn't gotten around
to telling the rest of the code that we can use it.

This commit re-enables it in the build but leaves sqlite
as the default engine until we're able to perform some
benchmarking.

Rocksdb itself has some build issues on Windows; it doesn't
use cmake to locate dependencies, so even though we built
snappy it doesn't know how to find it without modifying the
source:
https://github.com/facebook/rocksdb/blob/master/thirdparty.inc#L4

For that reason, we disable the use of Snappy in the Windows build.
However, in the version of rocksdb that we were using, it would
default to trying to use Snappy even though it wasn't compiled in
and throw an exception.

I've upgraded to a newer version of rocksdb that will simply not
use compression if no compression was enabled at build time.

Given that we mostly store relatively small objects, I'm assuming
that the lack of compression is fine for now.

Reviewed By: xavierd

Differential Revision: D21319896

fbshipit-source-id: 2a2d06d4bd5382706e9220f9b4a2de99dc18311d

Summary:
Add support for io_uring based async read/recvmsg

(Note: this ignores all push blocking failures!)

Reviewed By: kevin-vigor

Differential Revision: D21208891

fbshipit-source-id: 21b3b5e0f201e982ecb50a284d03ab53413ec3e3

Summary:
Upgrade to the 1.10.0 release.  This includes some new features like the
`GTEST_SKIP()` macro.

Reviewed By: genevievehelsel

Differential Revision: D21309360

fbshipit-source-id: 163db628fc99aaa786aeb207f35c7d6295cb5e25

Summary:
Update the generated `run_cmake.py` script to tell ctest to print the test
output on failure.  Also pass in a `-j` flag to run tests in parallel by
default.

These flags are already passed in by default when running `getdeps.py test`;
this simply updates this developer utility script to do the same.

Reviewed By: wez

Differential Revision: D21307806

fbshipit-source-id: 42045b0f9362494042c79bc946a1004ff8ad98b6

Summary:
Use template parameter packing in the constructor to make the implicit constructor usable for the uniform intialization of container types.

We want the return syntax to be unchanged to the extent possible when annotating functions.

Reviewed By: A5he

Differential Revision: D21290083

fbshipit-source-id: 140bbaf987cdfd673046e92dd263b5180c8e9daa

Summary:
Add DecoratedAsyncTransportWrapper zerocopy support

(Note: this ignores all push blocking failures!)

Reviewed By: mingtaoy

Differential Revision: D21317312

fbshipit-source-id: 9a33402bbea5ab975231d812febe48ac2517ebe0

Summary: [Folly] `MacAddress` updater and factory suite for parsing string and binary representations and throwing or returning unexpected on parse failures.

Differential Revision: D21273907

fbshipit-source-id: 664cd1980cb12f3352eecf0e25671dca04c4a8a0

Summary: otherwise the one in Mock doesn't work

Reviewed By: yfeldblum

Differential Revision: D21264567

fbshipit-source-id: c73e6bac1f0293edb67355ee88b6c2ec1b9684fd

Summary:
On Linux the debug info sections in projects downstream from folly and
thrift are huge due to a lot of C++ template usage.

We build in RelWithDebInfo mode as that is most useful when debugging
locally and because the build times are long, but most casual consumers
of the binaries via GH actions really don't care about debug info,
and this should save ~180MB of size in (for example) the watchman
executables.

Pull Request resolved: https://github.com/facebook/watchman/pull/809
Refs: https://github.com/facebook/watchman/issues/804

Test Plan:
Review the artifact size on the linux build in this PR (see
https://github.com/facebook/watchman/actions/runs/91688952) and confirm that it
is a bit smaller than 180MB; now under 3 MB.

Reviewed By: simpkins

Differential Revision: D21318302

Pulled By: wez

fbshipit-source-id: f78bc5e735dd78771e0604abae64c0a23cf9158d

Summary:
The mismatched project output settings were causing
the dyndeps fixup to fail to find any objects and thus none of
the artifacts are populated for posix platforms.

Pull Request resolved: https://github.com/facebook/watchman/pull/808

Test Plan:
The PR published artifacts to:

* https://github.com/facebook/watchman/actions/runs/91617994
* https://github.com/facebook/watchman/actions/runs/91617996
* https://github.com/facebook/watchman/actions/runs/91617997

Reviewed By: simpkins

Differential Revision: D21315090

Pulled By: wez

fbshipit-source-id: 60461809f55e73119f7206e3f392d5b237722f85

Summary:
Rather than have a single main.yml file containing all off the different
builds, split that up so that we have one file per build environment
(linux, mac, windows).

This has a couple of advantages:

* It is quicker to see the status of just one of the platforms
* Artifact collection for one platform is not blocked pending completion
  of the builds for all systems
* It's a little easier to understand what is happening for a single platform

To support having multiple files I've changed the output-file option to
be an output-dir.

I've included the rm of main.yml in this commit.  Once this gets
imported back to the FB internal system I'll amend in an update to
the helper script that updates all of our opensource builds and run
and amend that.
Pull Request resolved: https://github.com/facebook/folly/pull/1360

Test Plan:
the GH action status on this PR should show three different
actions running, one for each platform.

I updated and ran
`fbcode/opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh`
to regenerate all the actions files for FB.

Reviewed By: yfeldblum

Differential Revision: D21310991

Pulled By: wez

fbshipit-source-id: 604ef652c8f746781a4b410c6b996cdee4524e0d

Summary:
This commit resolves an issue with our zipapp executables
on Windows that meant that the only reliable way to start them was
to use the fully path to the executable.

The root cause is that the __wargv array is produced by parsing the
process command line into an array, and if you ran `watchman-wait -h`
__wargv[0] would have `watchman-wait` rather than the fully qualified
path to the executable that the zipapp plumbing requires.

The fix is to ask the system for the fully qualified path and ensure that
that gets set as both argv[0] AND argv[1].

Reviewed By: xavierd

Differential Revision: D21190350

fbshipit-source-id: eeb95084592d30a028a93b2b03877f8cc6c72729

Summary:
The environment changed since I tested  D20740410 and now it appears
that we'll need to re-export a versioned variable in order for cmake
to detect boost on the GH actions hosts.
Pull Request resolved: https://github.com/facebook/folly/pull/1359

Test Plan:
the GH actions status of this diff.

The workflow was updated via:

```
python3 build/fbcode_builder/getdeps.py generate-github-actions folly --output-file .github/workflows/main.yml
```

Reviewed By: yfeldblum

Differential Revision: D21307640

Pulled By: yfeldblum

fbshipit-source-id: 1555cbcade822775379cd9054be37fdbc17b4d93

Summary:
From the outset, we wanted to be sure that getdeps was able
to source and build the dependencies so that we knew that we'd have
a repeatable build.  This came at the cost of build times: having
to build boost on each CI run is a bit of a chore.

This commit adds three new elements to the manifest files:

* `rpms` - a list of RPM names that are all required to be present
  in order to consider the dependency satisfied
* `debs` - like `rpms` above, but scoped to debian package names
* `preinstalled.env` - a list of environment variables that if they
  are all set and non-empty will satisfy the dependency.

A new `--allow-system-packages` option to getdeps enables the new
logic that looks for system packages; it is off by default, but
enabled in the generated GitHub Actions workflows.

A new `install-system-deps` subcommand is provided that will attempt
to install the system packages needed to satisfy the build.  This
typically needs to be run via sudo and is thus broken out separately
from the main getdeps build flow.

I made a pass over the manifest files and added package names that
satisfy the build on ubuntu-18 and fedora-31.

shri-khare: I renamed the `Python3.7.6` manifest to just `python` as
part of this change; the version of python that it pulls in through
the normal build is the same and I believe that an equal or newer
version of python3 is available in the GH actions builder.

The `preinstalled.env` is used only by the boost manifest: it references
the name of an environment variable that is set by the github
windows hosts and that points to a pre-built and pre-installed
copy of boost.  Since there is no package manager that we can
easily query for this sort of thing, probing from the environment
seems like a reasonable and fast way to check for this.  We
may need to evolve this over time to become more feature rich,
but this seems like a good starting point.

This commit has the potential to save 20 minutes of build time
from each public CI build just due to the boost dependency alone!

Refs: https://github.com/facebook/watchman/pull/797

Reviewed By: yfeldblum

Differential Revision: D20740410

fbshipit-source-id: 6c38019449c54465127656c3d18a6ff1f30adaea

Summary:
The MSVC compiler was complaining that an implicit conversion from intmax_t
to long was being performed. Let's use static_cast to silence it.

Reviewed By: simpkins

Differential Revision: D21267752

fbshipit-source-id: e466f107ca8e861c12e0abd1f8fa7448f22adfd9

Summary:
The MSVC compiler complains about a potential lossy conversion happening,
which feels mostly right due to Iterators being potentially far apart.

Reviewed By: yfeldblum

Differential Revision: D21280438

fbshipit-source-id: 8fcb18d865688aa1fae869a41256ad3dc3ad2a97

Summary: [Folly] `identity`, an initialized function object matching `Identity`, which is renamed `identity_fn`.

Reviewed By: markisaa, lewissbaker

Differential Revision: D21265889

fbshipit-source-id: 5cc9f2e624da9a4acd0f65985ff10a4dd306efdc

Summary:
The test cert "test-cert.pem" is signed with a SHA1 based signature algorithm, and newer distros such as
CentOS 8 will fail loading this certificate due to its default security policy.

This diff regenerates the test certificates, following the old certificates
names as closely as possible. I included the script to generate these in case
they need to be changed in the future.

Reviewed By: AjanthanAsogamoorthy

Differential Revision: D21271202

fbshipit-source-id: de4227ae81dfffaf00f8d355e83936db6eb5fe31

Summary:
Folly links explicitly links against boost statically on Windows.  However,
many downstream projects that use folly independently performed their own
discovery of boost and normally used the default option of linking against it
dynamically.  This resulted in boost being listed twice on the link line on
Windows for some downstream projects: both its static libraries and dynamic
libraries would be present.  This ends up causing linking failures due to
duplicate definitions.

This updates folly's installed CMake file to correctly call
`find_dependency(Boost)` so that downstream projects don't need to perform
their own independent discovery.  This ensures that all downstream projects
use a consistent method of linking against Boost.  I updated many downstream
projects to remove their explicit separate discovery, and rely only on this
behavior from folly.

I also added a configuration setting to allow explicitly selecting whether to
link against boost statically at folly configure time.

Reviewed By: wez

Differential Revision: D21232164

fbshipit-source-id: 9ecc3ce988add48905252297e979403c42e7e148

Summary:
## Why is this diff needed?
This diff presents an annotation framework designed to address some of the issues we've had working with fibers in NodeAPI:
- It's difficult to identify functions doing I/O. As a result, we've had I/Os happening sequentially in for loops.
- It's difficult to tell if a function is running on fibers. This is especially hard in large code bases such as ours

## What does this diff do?
The `Async<T>` type can be used as the return type of a function. This accomplishes a couple of things:
- It indicates that this function should run on fibers and that a fiber can suspend in this function (or in one of it's children)
- It forces calling functions to `await` the result of this function. When coupled with a good lint rule, this enforces annotations right to the top of the stack.

Reviewed By: A5he

Differential Revision: D21159049

fbshipit-source-id: ee922093b140b22d8e7a7587b87aa0e783055b6c

Reviewed By: wez

Differential Revision: D21238976

fbshipit-source-id: 13a6def8d6c8d25057409956cc883589b205cd04

Summary:
Declaring this method noexcept was incorrect and could unnecessarily lead to
program termination if one of the calls threw an exception.

Reviewed By: yfeldblum

Differential Revision: D21187354

fbshipit-source-id: 109a33d52659cdf42e5ced9e18be41ae4c3daf85

Summary:
I noticed that copytree was taking forever and realized
that it wasn't issuing a prefetch call so I started looking in here;
this commit teaches getdeps how to recognize and EdenFS repo on
Windows but skips calling prefetch on Windows.

Currently the prefetch implementation triggers some very slow
processing in mercurial that is slower to start than just
enumerating the files in the opensource build.

It turned out that my original problem was just that my credentials
had expired and we weren't surfacing that error on Windows yet.

Reviewed By: simpkins

Differential Revision: D20755905

fbshipit-source-id: 8d3695cdd1f04199d1d409895482b8c706285d5f

Summary:
By and large, trying to test folly::coro with gcc-10's (very green) coroutines implementation was a failure, but it did turn up a couple of issues:

* Friends of friends are not friends with gcc
* -Wshadow picks up a few more instances of shadowing
* `std::exchange` needs `#include <utility>`.

Reviewed By: lewissbaker

Differential Revision: D21190145

fbshipit-source-id: d3f3148baa26bb8a2c0c1954fc930fd8124672e1

Summary: The configerator structs are used in many top level functions in Mononoke and are required in order to build all the code on github

Reviewed By: ahornby

Differential Revision: D21130546

fbshipit-source-id: 7f17d92173f5ecf7c3406ae4202359a0db8df84a

Summary:
We're recommending collectAny as a replacement for
collectAllSemiFutures :(

Reviewed By: LeeHowes

Differential Revision: D21197163

fbshipit-source-id: 0055e0feb5cc74a767370bc1dc9fe8e4d8dffcfc

Summary:
Add support for variable number of addrs so we can send data to multiple destinations.

(Note: this ignores all push blocking failures!)

Reviewed By: kevin-vigor

Differential Revision: D21032857

fbshipit-source-id: f73b98d44f5d7d92f3692dfddb9b1c76ebcc51c5

Summary:
Disable TSAN instrumentation across StaticMetaBase::onThreadExit() to avoid a shortcoming in TSAN on Linux that produces sprurious failures.

This was recently worked around by ignoring TSAN failures in ld-linux's _dl_deallocate_tsd() (D20597534), but that turns out to be insufficient, because other TSAN failures can occur on the TLS area after the dynamic linker has called free() on it.  This diff backs out the change from D20597534, and works around the problem in a more direct & complete way.

Reviewed By: yfeldblum

Differential Revision: D20671620

fbshipit-source-id: 05a74e709b620391f18467c7a5e4d14dcf9f4a1e