Summary: Instead of hard coding the max deferred readers allowed to be 64, statically allocate large enough slots and pick max deferred readers allowed dynamically based on the platform running the service. Specifically, we set the `maxDeferredReaders = 2 * nextPowTwo(numCPU)`, which four times the number of physical cores, to allow faster reads. We are effectively giving each HW thread two slots.

Reviewed By: yfeldblum

Differential Revision: D22407478

fbshipit-source-id: 4001cf96dc502e00f00a27d57c63ba0028a52671

Summary: After D22371898 we can move the `CoreBase` implementation out of the header.

Reviewed By: luciang

Differential Revision: D22371900

fbshipit-source-id: bd4fe6df298c0ba02988b357cd7413eb8f7d8b67

Summary:
`Core<T>` instantiations share a large part of the implementation, but in the current shape the code is duplicated for each instantiation. This diff moves whatever possible to a common base class.

This diff only splits the typed and untyped parts, for ease of review. The untyped parts are moved to the cpp file in D22371900.

Reviewed By: andriigrynenko

Differential Revision: D22371898

fbshipit-source-id: db48c202d0bdcbefbebe150e7c7d9f35e06687d0

Summary:
Fix jemalloc folly weak symbol issue

(Note: this ignores all push blocking failures!)

Reviewed By: davidtgoldblatt

Differential Revision: D22420581

fbshipit-source-id: 136fa6a8c74d8fea3fc96ba1d205266f2f2fa02c

Summary:
We provide an optional callback that callers can use to save parameter names
where desired.

Differential Revision: D22399275

fbshipit-source-id: 7f9440a46cbcc4085d48a2387ca3d2db51f2fe7e

Differential Revision:
D22189916 (https://github.com/facebook/folly/commit/0d2bdbf0e0618bd9366d3c35345e091ee24f0fae)

Original commit changeset: e6fba48eaae0

fbshipit-source-id: 4ea03c56fda84240e178b2b4baf83a57d3afa344

Summary:
Use sized-deallocation (`sdallocx`) if possible in `folly::SysAllocator` and `folly::Arena`.

`Arena` has always allocated two types of blocks:
1. Normal (fixed-sized): size is the "goodSize adjusted" `minBlockSize`
2. Large (variable-sized): when #1 is too small

Type #2 makes sized-deallocation tricky -- we need somewhere to remember the allocated sizes.

The old code used a single type `Block` and kept a single list. Here I change to have two types and two lists.  The `LargeBlock` has an additional `allocSize` data member.

This makes the Arena object itself 16B larger, but seems better than adding a 4B `allocSize` to each and every block, regardless of type.

Note that, prior to this change, it was possible to `merge()` arenas with different `minBlockSize`.  This is no longer possible.

Reviewed By: yfeldblum

Differential Revision: D22189916

fbshipit-source-id: e6fba48eaae0b5cc8456b856b02d2cfc71c03834

Summary: Replace loop with clear_and_dispose()

Reviewed By: yfeldblum

Differential Revision: D22450280

fbshipit-source-id: cad040abd3f40e0fbb1f2ab27d06bf5174760c2c

Summary: IoUringBackend free mempool rework

Reviewed By: kevin-vigor

Differential Revision: D22424753

fbshipit-source-id: 5f8482702ec17561058ea841493e8a5a57b642a6

Summary: Fix issue where `preadv` and `pwritev` are now included in iOS 14 and macOS 11 SDKs, so we need to do runtime checks for those platforms.

Reviewed By: Orvid

Differential Revision: D22275221

fbshipit-source-id: 12e45c55a0fcf10f540f2d4cb9a3c392d7e182cb

Summary: `Future.h` is widely included; `KeepAliveOrDeferred` and `DeferredExecutor` are not templates, we can move all the definitions to a cpp file.

Reviewed By: yfeldblum, luciang

Differential Revision: D22371899

fbshipit-source-id: 23a77b2e0560d82f6c3006597eef85d11527cb13

Summary: processSubmit rework

Reviewed By: mokomull

Differential Revision: D22405528

fbshipit-source-id: 99db524e62264d80556e6ac771f6d504cb31bd37

Summary:
The GitHub actions have been failing because `rsocket-cpp` has been deleted.

https://github.com/facebook/watchman/runs/843777632

This commit is generated with `./opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh`

Reviewed By: wez

Differential Revision: D22417304

fbshipit-source-id: e64a8d011c753bf8ac90ed2c9c90036baa8cf950

Summary: Implement await as a CPO - Allow customization for user defined types outside of folly.

Reviewed By: yfeldblum

Differential Revision: D22268765

fbshipit-source-id: 4badba9274b7206afc339e9d94f6ca991dc5674e

Summary:
This diff introduces a new macro function to resolve whether the current OpenSSL version available is greater than or equal to the given (major, minor, fix) version tuple.

We used the rules defined in the [OPENSSL_VERSION_NUMBER man page](https://www.openssl.org/docs/man1.1.1/man3/OPENSSL_VERSION_NUMBER.html), ie. nibbles for each of
```
 MNNFFPPS: major minor fix patch status
```

We choose to ignore `patch` and `status` for simplicity and because we don't expect to build against such versions.

The existing `FOLLY_OPENSSL_IS_110` variable already checks for greater than or equal to version number 1.1.0, so I didn't feel the need to modify it (nor the other variables). We can do some clean up in a future diff.

Reviewed By: yfeldblum, mingtaoy

Differential Revision: D22346692

fbshipit-source-id: 156ee69ecd619de12319d7d63239f28c323820a4

Summary:
move template instantiations for folly::Unit of several types into explicit instantiations in cpp files.

`folly::Future<folly::Unit>`
`folly::SemiFuture<folly::Unit>`
`folly::Promise<folly::Unit>`
`folly::SharedPromise<folly::Unit>`
`folly::futures::detail::FutureBase<folly::Unit>`
`folly::futures::detail::Core<folly::Unit>`
`folly::Try<folly::Unit>`

Reviewed By: yfeldblum, ericniebler

Differential Revision: D22092858

fbshipit-source-id: b8faa1c8aedfc193c17e2c3bc2e5b0f429ae6b0f

Summary: This commit adds a flag `--retry` to getdeps and teach it to run retry failed test. This allows us to still pass the tests when there are some flaky tests presents.

Reviewed By: wez

Differential Revision: D22291063

fbshipit-source-id: 572af48a52ceb4a9abbf530cc0154ded0120c0de

Summary:
`folly::coro::blockingWait(...)` returns `void` for `Task<void>`.

This diff implements a template specialization on taskWait for `Task<void>`

Differential Revision: D22265598

fbshipit-source-id: ff13d2b18b8706a6e8257ef706e407b315daf98a

Summary:
The variadic implementation of collectAll uses the futures APIs, and is likely to be slow.

This diff adds a fiber only version similar to fibers::foreach

Reviewed By: yfeldblum

Differential Revision: D22212232

fbshipit-source-id: f71765c5411d2c6ca4388fe373baafad65f1b01c

Summary:
After some experimenting, it is a little awkward if we want to specify a relative path based on the executable location. We'd need to add a bunch of path calculations to make it right, and I don't think the added complexity is really worth the effort.

As a result, let's just remove the use of relative path, and if we ever want to ship a copy of Python distribution, we can place it under the same directory as the binary.

Reviewed By: chadaustin

Differential Revision: D22394180

fbshipit-source-id: 86d27f6d16a03fe08826b5e5eafcef2a1c77997f

Reviewed By: yfeldblum

Differential Revision: D22393040

fbshipit-source-id: 920a7dfbf1d4334aecb475add96d3799c89aa211

Summary: Added for symmetry with `IOBuf::cloneAsValue()`

Reviewed By: luciang

Differential Revision: D21737695

fbshipit-source-id: 40f5695565ffc7d7c389c9953bffc3ab85508ed1

Summary:
In order to do what the title says, this diff does:
1. Add the `eden/oss/.../third-party/rust/.../Cargo.toml` files. As mentioned in the previous diff, those are required by GitHub so that the third party dependencies that are local in fbsource are properly defined with a "git" dependency in order for Cargo to "link" crates properly.
2. Changes to `eden/scm/Makefile` to add build/install commands for getdeps to invoke. Those command knowing that they are called from withing getdeps context they link the dependencies brought by getdeps into their proper places that match their folder layout in fbsource. Those Makefile commands also pass a GETDEPS_BUILD env to the setup.py invocations so that it knows it is being called withing a getdeps build.
3. Changes to `eden/scm/setup.py` that add "thriftasset" that makes use of the getdeps.py provided "thrift" binary to build .py files out of thrift files.
4. Changes to `distutils_rust` to use the vendored crates dir provided by getdeps.
5. Changes to `getdeps/builder.py` and `getdeps/manifest.py` that enable more fine-grained configuratior of how Makefile builds are invoked.
6. Changes to `getdeps/buildopts.py` and `getdeps/manifest.py` to disable overriding PATH and pkgconfig env, so that "eden/scm" builds in getdeps using system libraries rather than getdeps-provided ones (NOTE: I've tried to use getdeps provided libraries, but the trickiest bit was that Rust links with Python, which is currently not providable by getdeps, so if you try to build everything the system provided Python libraries will collide with getdeps provided ones)
7. Added `opensource/fbcode_builder/manifests/eden_scm` for the getdeps build.

Reviewed By: quark-zju

Differential Revision: D22336485

fbshipit-source-id: 244d10c9e06ee83de61e97e62a1f2a2184d2312f

Differential Revision:
D22211304 (https://github.com/facebook/folly/commit/c9c5564232aaffbc0f1d3807b21bb2cd60ec4d2d)

Original commit changeset: 0dbe904c9fd8

fbshipit-source-id: 49cf24dab4f91fcebf9dc5a2db66d23683fcaa37

Differential Revision:
D22304614 (https://github.com/facebook/folly/commit/cf31609549e6085b1d9be523e67cd62f05287d5e)

Original commit changeset: bf846730c594

fbshipit-source-id: c1982958588fb0bf1194c84de93d6ed1336ccdb5

Summary: Add io_uring support for fsync/fdatasync

Reviewed By: kevin-vigor

Differential Revision: D22344375

fbshipit-source-id: c6e1de2d778568c4abbefa0a732a3cf8cffc7c97

Summary:
In EdenFS's latest Windows package. We are seeing DLL import errors coming from `asyncio` as it requires a system native module `_overlapped.pyd`.

The underlying cause is because when we build EdenFS CLI on Sandcastle, we are linking with Python 3.6.2. The Python36.dll shipped with the EdenFS package is also coming from that version.

However, on Windows laptop. We have Python 3.6.3. Since we are not shipping the Python system libraries with us. It uses the libraries installed in the system, and it attempts to import the `_overlapped.pyd` located at `C:\Pythone36\DLLs\`. This version is compiled against Python 3.6.3, which is incompatible with the Python36.dll we are using.

----

To resolve this, we need either ship an embedded copy of Python along with EdenFS, or teach EdenFS to use the Python distribution installed in the system. This commit tweaks the executable we prepend to the archive created with zipapp to locate `Python3.dll` dynamically. This allows us to use the Python installed in the system so we can avoid the version mismatch issue.

With this setup, we can also be shipping an embedded Python version along with EdenFS, and the Python loader can look for that path. This is demonstrated with the relative DLL loading `..\python`.

In theory, we can have a package structure like this:

```
.
├── python
│   ├── ....
│   └── python3.dll
└── bin
    ├── ...
    ├── edenfsctl.exe
    └── edenfs.exe
```

Reviewed By: xavierd

Differential Revision: D22325210

fbshipit-source-id: 96a3f9503e7865a5f9d95710ff13f019afcf04f1

Summary: fbthrift no longer depends on rsocket-cpp

Reviewed By: simpkins

Differential Revision: D22275231

fbshipit-source-id: c1f217f1ce97591b8ebca002bf8ae3af701be641

Summary: [Folly] Small benchmark for `coarse_steady_clock`.

Differential Revision: D22320499

fbshipit-source-id: dbd93e886b9e984d7469288258eefaa4a3fafb10

Summary:
The upper bitvector size (in bits) can exceed the domain of `SizeType` even if the list otherwise respects the contract.

Also improve some of the tests.

Reviewed By: yfeldblum, philippv

Differential Revision: D22304614

fbshipit-source-id: bf846730c59451457d3e2d3cf14d90e3df012e0b

Summary: Complete support of the FiberManager::addTask*

Reviewed By: pranavtbhat

Differential Revision: D22305146

fbshipit-source-id: 94e4a02f754b306d8b5fe4994832a7e2dc2cf4a5

Summary: Simplify API understanding for code reader

Reviewed By: pranavtbhat

Differential Revision: D22272546

fbshipit-source-id: 341625ed9079f3bc4aee09851eadc02fd05b2434

Summary: `clang-cl` and some other invocations of `clang` on windows define `_MSC_VER`. However `clang` does not implement `#pragma optimize` so we need to use the other code-path here.

Reviewed By: yfeldblum

Differential Revision: D22301661

fbshipit-source-id: e51656497a5fc109bf04f7daac82f824ba8c5c8e

Summary: Filtering is one of the basic stream operations and it would be convenient to have it for AsyncGenerators

Reviewed By: yfeldblum

Differential Revision: D22136080

fbshipit-source-id: ca6a233a8c1aeddb63899aeaba8ca4c1b6dfa535

Summary:
# Note: This is a resubmission after reverting this change.
The failing `DCHECK` has been fixed with an explicit cast to `size_t` to accommodate long lists.

# Summary

This diff introduces the following changes to the EliasFanoReader implementation.

- `EliasFanoReader` no longer requires knowledge of the last value in the list at construction time. This removes the need to access the last byte of the encoded list.
- Operations in `UpperBitsReader` are now responsible for ensuring validity.
- Removes constraint that the last set bit in upperBits must be in the last byte of the encoded list.

In addition, the diff also extends the unit-tests for Elias Fano coding to add arbitrary extensions to the upperBound at construction to ensure that additional 0-blocks at the end of the list do not affect the behavior of the reader.

Reviewed By: ot, luciang

Differential Revision: D22211304

fbshipit-source-id: 0dbe904c9fd8cd9568a480355e5e6a4525922966

Summary: Only accept Async input now

Reviewed By: yfeldblum

Differential Revision: D22268635

fbshipit-source-id: cd974af7a7a3adc25b3c90029efe4c6080c3739a

Summary: When zero copy is enabled on AsyncSSLSocket every IOBuf written via writeChain is accumulated in idZeroCopyBufInfoMap_ and never freed.

Differential Revision: D22272411

fbshipit-source-id: 576d16ddb4e5c7c19209b77f7346949f10fbf32d

Summary:
This provides a set of three functions under folly::fibers::async, to schedule then wait for work on a remote fiber manager.
The functions cover those 3 (legitimate) needs:

- Blocking current thread: `T executeOnRemoteFiberAndWait`
- Blocking current fiber: `Async<T> executeOnRemoteFiber`
- Awaitable (barebone version for now): `Future<T> addFiberRemoteFuture`

See code docblock for more details

Reviewed By: pranavtbhat

Differential Revision: D22252598

fbshipit-source-id: add9c23f5bdacb233e47f30b2639d376027167e3

Summary: More io_uring file read/write tests

Reviewed By: kevin-vigor

Differential Revision: D22236012

fbshipit-source-id: 37abd398d07af2bb41cfe3b3b6dfeb40410554d5

Summary:
Koray reported issue with OpenNSA while using build VM and observed that we
don't hit the issue on devserver thanks to downloading the lfs cached file.

The root cause:

Broadcom added License notice and uploaded new 6.5.19 with same name.
This changed the hash though - causing fbcode builder to complain.
Fix it by adjusting the hash.

See the first line in the recursive diff of before vs. after: P134467317

(the broken symlinks is an unrelated existing issue, which we have reported to
Broadcom).

Reviewed By: bkoray

Differential Revision: D22280971

fbshipit-source-id: 781079df426f83901509225156cf77a3966d3301