Summary:
This is an interim solution that unblocks failing FBOSS OSS builds.

Broadcom's OpenNSA is available on github and contains large precompiled
libraries. For large files (> 100Mb), github uses git-lfs.

git clones to OpenNSA started failing yesterday with:

Error downloading object: lib/x86-64/libopennsa.a (7f21d94): Smudge error:
Error downloading lib/x86-64/libopennsa.a
(7f21d941ac32ccae38adb05b3386be739bdccd502a878364d386e7a5cb172f35): batch
response: This repository is over its data quota. Account responsible for LFS
bandwidth should purchase more data packs to restore access.

Github documents (refer:
https://help.github.com/en/github/managing-large-files/about-storage-and-bandwidth-usage)
“When you download a file tracked with Git LFS, the total file size is counted
against the repository owner’s bandwidth limit”. It appears we are hitting that
limit.

Thus, github is not a good choice here. Even if we create a 'github release
tarball', the tarball continues to carry lfs files and thus downloading those
counts towards the quota (which is limited).

Broadcom paid some amount to bump up the quota, but it is matter of time before
we hit it esp given that these builds run on-diff.

A better solution is to download a tarball (that contains real files, not lfs),
and then allow Facebook lfs to cache it.

While github does not support that, it seems bitbucket does.

Thus, we forked OpenNSA from github into a bitbucket repo and this patch
modifies the manifest to download a tarball from bitbucket that will be cached.

Broadcom is working on a cleaner solution but this will keep our FBOSS OSS build
clean in the interim.

Reviewed By: wez

Differential Revision: D19437386

fbshipit-source-id: b1213a186a0af57b9fd71b30e8899e80affd4cc0

Summary: Add a call in folly init to folly::start_hazptr_thread_pool_executor() to make the default hazptr domain use a CPUThreadPoolExecutor by default.

Reviewed By: davidtgoldblatt

Differential Revision: D19107125

fbshipit-source-id: 1cebe0864de14b54349f723061c4c2b5905b105b

Summary:
This allows the caller to, for example, pass their own ManualExecutor or EventBase
and have the task scheduled onto that executor and have the current thread drive
that executor until it completes.

Reviewed By: yfeldblum

Differential Revision: D19205803

fbshipit-source-id: 6042f68decf8d7d9bb1d37977e5804b0504daa41

Summary: add the socket to the callback parameter

Reviewed By: yangchi

Differential Revision: D19192549

fbshipit-source-id: 068bdb560273ab3203a1b7eaefa5b01c4e4e9968

Summary: Add support for registered fds

Reviewed By: kevin-vigor

Differential Revision: D19313387

fbshipit-source-id: 19527958b3986992cdafbad2c4c4902c887a9e9f

Summary:
Broadcom has formally replaced OpenNSL with newer OpenNSA:
https://github.com/Broadcom-Network-Switching-Software/OpenNSA/blob/master/README.md

With D16401189, FBOSS no longer depends on OpenNSL, but uses the new OpenNSA
instead. Remove this manifest.

Reviewed By: wez

Differential Revision: D19413708

fbshipit-source-id: f3662f1101d59a5e0e59cb7238797d652564ff90

Summary:
[Folly] Use `getrandom` instead of `/dev/urandom` when available.

The C library function `getrandom` is available on linux >= 3.17 and glibc >= 2.25.

Reviewed By: ot, simpkins, luciang

Differential Revision: D19211519

fbshipit-source-id: 07e7e856fc81d1f04efca4a6cfa0b76ada08f443

Summary:
When reading a `std::exception_ptr` out of a moved-from `folly::exception_wrapper`, we end up reading a `std::exception_ptr` field of a deleted `ExceptionPtr` object. This was causing MSAN much consternation.

The fix is to test for emptiness in `to_exception_ptr` and return a default-constructed `std::exception_ptr`.

This commit also adds extra test cases, since `to_exception_ptr` does something slightly different depending on whether the `exception_wrapper` object is `const` or not.

Reviewed By: yfeldblum

Differential Revision: D19403528

fbshipit-source-id: fde74a5c4b2ddb2025703432c1ac5a19d26b9426

Summary:
I have been encountering

```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==937752==ERROR: AddressSanitizer: SEGV on unknown address 0x24a12000e4f18 (pc 0x7f3902a6e696 bp 0x7f37bc5e4380 sp 0x7f37bc5e42a0 T107)
==937752==The signal is caused by a READ memory access.
SCARINESS: 20 (wild-addr-read)

    #0 0x7f3902a6e695 in folly::ThreadLocalPtr<folly::SingletonThreadLocal<folly::hazptr_tc<std::atomic>, void, folly::detail::DefaultMake<folly::hazptr_tc<std::atomic> >, void>::Wrapper, void, void>::get() const folly/ThreadLocal.h:160
    #1 0x7f3902a6d600 in get folly/ThreadLocal.h:69
    #2 0x7f3902a6d600 in folly::ThreadLocal<folly::SingletonThreadLocal<folly::hazptr_tc<std::atomic>, void, folly::detail::DefaultMake<folly::hazptr_tc<std::atomic> >, void>::Wrapper, void, void>::operator*() const folly/ThreadLocal.h:78
    #3 0x7f3902a6ca1b in folly::SingletonThreadLocal<folly::hazptr_tc<std::atomic>, void, folly::detail::DefaultMake<folly::hazptr_tc<std::atomic> >, void>::getWrapper() folly/SingletonThreadLocal.h:149
    #4 0x7f3902a6cbe5 in folly::SingletonThreadLocal<folly::hazptr_tc<std::atomic>, void, folly::detail::DefaultMake<folly::hazptr_tc<std::atomic> >, void>::LocalLifetime::~LocalLifetime() folly/SingletonThreadLocal.h:121
    #5 0x7f38f04d9fd5 in (anonymous namespace)::run(void*) /data/users/kirkshoop/gcc/trunk/libstdc++-v3/libsupc++/atexit_thread.cc:75:16
    #6 0x7f38ef9ee551 in __nptl_deallocate_tsd.part.8 /home/engshare/third-party2/glibc/2.26/src/glibc-2.26/nptl/pthread_create.c:301:8
    #7 0x7f38ef9ef7c8 in __nptl_deallocate_tsd /home/engshare/third-party2/glibc/2.26/src/glibc-2.26/sysdeps/nptl/futex-internal.h:200:3
    #8 0x7f38ef9ef7c8 in start_thread /home/engshare/third-party2/glibc/2.26/src/glibc-2.26/nptl/pthread_create.c:475:3
```

I tried using a unique Tag for the SingletonThreadLocal for debugging purposes and this seemed to fix the issue.

Reviewed By: magedm

Differential Revision: D19356179

fbshipit-source-id: a18c6b02f0a40e0509753075614eeff71d859bf7

Summary:
With the release of OpenNSA (https://github.com/Broadcom-Network-Switching-Software/OpenNSA), we can stop using opennsal altogether!

This diff was created using following steps:

cd fboss/agent
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/opennsl_spl_//g' {} \;
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/opennsl/bcm/g' {} \;
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/OPENNSL/BCM/g' {} \;
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/OpenNSL/Bcm/g' {} \; # for symbols like snmpBcmTransmittedPkts2048to4095Octets

Furthermore:

- In all TARGETS under fboss/agent replace wrapped_opennsl_symbols with wrapped_bcm_symbols.
- hg mv agent/facebook/test/OpenNSLInterface.h agent/facebook/test/BcmInterface.h
- BcmEgress.cpp remove operator== overload for opennsl API altogether, and remove operator== BCM API version from inside unnamed namespace, and put it in fboss namespace.
- BcmCinter.h and BcmCinter.cpp now have some duplicate definitions as opennsl_ got renamed to bcm_. Delete one of the definitions "as appropriate".
- Similarly, remove duplicate definitions in FakeSdk.h and FakeSdk.cpp
- Similarly, remove duplicate definitions in SdkTracer.cpp (remove definitions calling CALL_WRAPPERS_RV_NO_CINTER). And lastly, remove CALL_WRAPPERS_RV_NO_CINTER itself.
- arc lint
- Remove OpennslCompatTests.cpp
- Fix ./agent/hw/bcm/tests/facebook/BcmEgressTest.cpp to have #include
  "fboss/agent/facebook/test/MockOpenNSL.h" instead of MockBcm.h

TODO:
- This patch retains the directory structure (facebook/ subdirectory), which will be fixed in subsequent patches.
- Remove opennsl references from TARGETS file, additional cleanup etc.
- Remove all oss directories...

Differential Revision: D16401189

fbshipit-source-id: 607b6c0f97ffcacf3707f6b7a4c9454cc0b24476

Summary:
Broadcom provides this library and they decided to rename it from OpenBCM to
OpenNSA. Thus, rename corresponding fbcode_builder code.

Reviewed By: wez

Differential Revision: D19396687

fbshipit-source-id: 8233dbf4de9342b5a0e54ae275d6c73d43abe6d0

Summary: [Folly] Cut unused `AsyncSocketException::SASL_HANDSHAKE_TIMEOUT`.

Reviewed By: vitaut

Differential Revision: D19343864

fbshipit-source-id: ed4374bc2b376ff71e592323ed8c37290700e870

Summary: Add recvmmsg

Differential Revision: D18927890

fbshipit-source-id: de500dd1c266f21fbfc39c2972df787da50c7a80

Summary:
UndefinedBehaviorSanitizer: implicit-integer-truncation buck-out/dev/gen/folly/synchronization/atomic_util#header-mode-symlink-tree-with-header-map,headers/folly/synchronization/AtomicUtil-inl.h:115:28
```

Here both `mask` and the first argument in `std::atomic<..>::fetch_and` have type  `unsigned char`. Before bitwise not operation `mask` variable undergoes [integral promotion](https://en.cppreference.com/w/cpp/language/implicit_conversion#Integral_promotion) resulting in truncation error.

Since here this is intentional add a `static_cast` to be explicit about it.

Reviewed By: yfeldblum

Differential Revision: D19320340

fbshipit-source-id: 41c987c19b77fa21ee9aa83cf06dfc7cddba91c8

Summary:
Found with readability-simplify-boolean-expr
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Pull Request resolved: https://github.com/facebook/folly/pull/1301

Reviewed By: yfeldblum

Differential Revision: D19302630

Pulled By: Orvid

fbshipit-source-id: 9833e4ab3fe93592fa8f868f6ed2d56a263949fe

Summary: Enable `INSTALL_PYTHON_LIB=ON` for internal builds of Eden on Windows.

Reviewed By: pkaush

Differential Revision: D19060630

fbshipit-source-id: 40d4f8784958fbeaad4f0d332c55fa0cb810cb5e

Summary:
Add `FOLLY_NO_UNIQUE_ADDRESS ` to expose C++20's `[[no_unique_address]]` where available.

(Note: this ignores all push blocking failures!)

Reviewed By: pixelb

Differential Revision: D19375089

fbshipit-source-id: 9a83ae791002e02df5b93b61f20209e5f467a959

Reviewed By: yfeldblum

Differential Revision: D19359860

fbshipit-source-id: b084d8cd603238e76d61de1eed3f233cf5b7fb27

Reviewed By: yfeldblum

Differential Revision: D19359673

fbshipit-source-id: 610ae77e205038267e0e331a04486eef0329c165

Summary:
For long-lived IOBufs, it can be useful to know whether their memory footprint
can be usefully shrunk. This diff exposes a method that lets users see the
smallest capacity an IOBuf could have while wrapping a particularly sized
object.

Without this, they have to do hacky math.

Reviewed By: yfeldblum

Differential Revision: D18440359

fbshipit-source-id: 1324700b58f0b2762cb3347b907ca5e04ebecb99

Summary: This constructor would never be called since the class is abstract (the method `add` from `Executor` is not implemented). This causes a warning which fails builds because of `-Werror`.

Reviewed By: yfeldblum

Differential Revision: D19345492

fbshipit-source-id: 20f185e0903a03acbde2e55e5a575e20deb5ad2a

Summary: This variable shadows the field `options_`, so rename it to get rid of the warning `constructor parameter 'options_' shadows the field 'options_' of 'FiberManager'` (which is turned into an error because of `-Werror`). This happens on clang 8.

Reviewed By: yfeldblum

Differential Revision: D19331755

fbshipit-source-id: 3615cf9b6c7516775f1dc198a08bc784a52c0fb4

Summary: Add non persistent events overflow tests

Reviewed By: danobi, kevin-vigor

Differential Revision: D19349804

fbshipit-source-id: 360d1f41d074b3d1af5f163213dadfc35f28c353

Reviewed By: yfeldblum

Differential Revision: D19357723

fbshipit-source-id: d099419271f74b013a276079723250ce842e2b24

Summary: [Folly] Fix evaluation-order assumption in futures varidic collect - argument evaluation order is unspecified so it must not be relied on when side-effects must occur in a specific order.

Reviewed By: lewissbaker

Differential Revision: D19347873

fbshipit-source-id: db4326d995a241619f96ff2bacdf5a5fa2332091

Summary:
There have been weird edge cases we haven't tracked down yet that saw python segfaulting at shutdown, caused by drive() run from the destructor of AsyncioExecutor after Python had run Py_Finalize shutting down the runtime.

This fix uses new checks from Python >=3.7 to check if Python is finalizing

Reviewed By: yfeldblum, nanshu

Differential Revision: D19321677

fbshipit-source-id: 79a6f95df383a86d80e064eb55a8a81e29d50ace

Summary:
Pull Request resolved: https://github.com/facebook/folly/pull/1305

The current documentation for Folly::rcu_domain is lacking, so this commit adds a comment header.  It also updates the Rcu.h header comment

Reviewed By: vitaut

Differential Revision: D18604342

fbshipit-source-id: 165b43b450b90d6dbcf62b0cb4b0231fe38b7372

Reviewed By: yfeldblum

Differential Revision: D19337438

fbshipit-source-id: 361322c02824c078a92b9cdd36ffacd0869580b0

Summary: Make the behavior of reclamation using an executor be the same as without an executor, in particular reclamation of the tagged list is skipped if it is found to be locked.

Reviewed By: davidtgoldblatt

Differential Revision: D19325821

fbshipit-source-id: 2eae06b320717aa1122d7cc7afdd4c58791fbfa8

Reviewed By: yfeldblum

Differential Revision: D19328769

fbshipit-source-id: 5d929eaa3bf4e2d0a63b8f1d95a613fee68525b4

Summary: Fix mem leak when running in overflow mode

Reviewed By: danobi

Differential Revision: D19333236

fbshipit-source-id: f8a93113d095cb227808c9c8e5704112c7f5c4fb

Summary: `folly::cold_detail::ColdClass` was marking things (like `folly::Unexpected`) cold, but at the cost of inhibiting the inliner from doing its job. This is leading to bad codegen, which offsets any small wins we mind get for the `cold` attribute.

Reviewed By: yfeldblum

Differential Revision: D19324159

fbshipit-source-id: 7ed431b6c9d6e963c3bf438c707fa6cf6a38bf9d

Summary:
Don't error out if we can't read the ELF/MACH-O header; just treat
it is not an object.

Reviewed By: chadaustin, simpkins

Differential Revision: D19253434

fbshipit-source-id: c5ecc7f0bc7a20e2611b7e2ff754355155f095da

Reviewed By: yfeldblum

Differential Revision: D19312260

fbshipit-source-id: 9751f70ac83d801cbc8708741e608f73cad8a464

Summary:
We haven't found them super useful, so lets delete them and replace them with USDTs.

The problems with the counters are:
* We only have the aggregated results. We can't look at a particular process, stack trace, etc.
* The speed reported isn't accurate.

Reviewed By: yfeldblum

Differential Revision: D18319361

fbshipit-source-id: 778213fb63eae90cc7900b075e054c021ecf561e

Add IP_BIND_ADDRESS_NO_PORT AsyncSocket support. Add the ability to set the socket options pre and post bind()

Summary:
Add IP_BIND_ADDRESS_NO_PORT AsyncSocket support. Add the ability to set the socket options pre and post bind()

(Note: this ignores all push blocking failures!)

Reviewed By: danobi

Differential Revision: D19315183

fbshipit-source-id: a8dc330c1f585766a00ba7724dcd9f7a6e64bbfb

Summary: AsyncSSLSocket::connect() creates a new instance of AsyncSSLSocketConnector referred with a raw pointer and passes it to AsyncSocket::connect(). AsyncSSLSocket assumed a callback irrespective of connection success or failure so that this raw pointer will be properly disposed off.  However currently, if a connect attempt is cancelled the parent class may end up clearing the connect callback without invoking a connection error, thereby causing a leak. Since the cancelConnect() function also cancels the callback, the raw pointer should be cleaned up during cancellation.

Reviewed By: knekritz

Differential Revision: D18365027

fbshipit-source-id: 2cc171fc1a026b771dcb0bdd7c45b7ee450b97f7

Summary: [Folly] `folly::atomic_ref`, a partial backport of `std::atomic_ref` from C++20.

Reviewed By: davidtgoldblatt

Differential Revision: D18413170

fbshipit-source-id: dddaabd69d2119ee695c75b214ca1c0204edfe6b

Summary: [Folly] Fix some outdated comments referring to `TAsyncTransport`.

Reviewed By: rhodo

Differential Revision: D19287080

fbshipit-source-id: 7ed234f4e5c9e17d1a3fd645f851129f4ac86b8f

Summary:
wasi-libc already defines mmsghdr struct https://github.com/CraneStation/wasi-libc/blob/d4db3fa2120b27e01aac8f5f1b749dbbe1344880/libc-top-half/musl/include/sys/socket.h#L33

In folly we currently check for MSG_WAITFORONE to avoid redefinition, but MSG_WAITFORONE and bunch of other things are excluded from wasi-libc. So no to get to deep into rabbit hole just extending exclusion for wasm32.

Reviewed By: yfeldblum

Differential Revision: D19269397

fbshipit-source-id: 1bdd8344a2bec8a0b5be48bfedc94cc0d9a8635a