1. 27 Jun, 2021 40 commits
    • Francesco Zoffoli's avatar
      Support CO_ASSERT_THAT · bb47922f
      Francesco Zoffoli authored
      Summary:
      `ASSERT_THAT` is defined in gmock, this adds the equivalent for coroutine code to GmockHelper.
      
      The implementation depends on the inclusion of GtestHelper, but to avoid forcing anyone that includes GmockHelper to also include GtestHelper I didn't add it in the file.
      
      Would it be preferable to include the needed header?
      
      Reviewed By: yfeldblum
      
      Differential Revision: D29067561
      
      fbshipit-source-id: 26aa6021efe55aa03dd7cf064563a732e47e39a1
      bb47922f
    • Brandon Schlinker's avatar
      TcpInfo, an abstraction layer to capture and access TCP state · 68a78d99
      Brandon Schlinker authored
      Summary:
      An cross-platform abstraction layer for capturing current TCP and congestion control state.
      
      Fetches information from four different resources:
      - `TCP_INFO` (state of TCP)
      - `TCP_CONGESTION` (name of congestion control algorithm)
      - `TCP_CC_INFO` (details for a given congestion control algorithm)
      - `SIOCOUTQ`/`SIOCINQ` (socket buffers)
      
      `TcpInfo` is designed to solve two problems:
      
      **(1) `TcpInfo` unblocks use of the latest `tcp_info` struct and related structs.**
      
      As of 2020, the `tcp_info` struct shipped with glibc (sysdeps/gnu/netinet/tcp.h) has not been updated since 2007 due to compatibility concerns; see commit titled "Update netinet/tcp.h from Linux 4.18" in glibc repository. This creates scenarios where fields that have long been available in the kernel ABI cannot be accessed.
      
      Even if glibc does eventually update the `tcp_info` shipped, we don't want to be limited to their update cycle. `TcpInfo` solves this in two ways:
         - First, `TcpInfoTypes.h` contains a copy of the latest `tcp_info` struct for Linux, and `TcpInfo` always uses this struct for lookups; this decouples `TcpInfo` from glibc's / the platform's `tcp_info`.
         - Second, `TcpInfo` determines which fields in the struct are populated (and thus valid) based on the number of bytes the kernel ABI copies into the struct during the corresponding getsockopt operation. When a field is accessed through `getFieldAsOptUInt64` or through an accessor, `TcpInfo` returns an empty optional if the field is unavailable at run-time.
      
      In this manner, `TcpInfo` enables the latest struct to always be used while ensuring that programs can determine at runtime which fields are available for use --- there's no risk of a program assuming that a field is valid when it in fact was never initialized/set by the ABI.
      
      **(2) `TcpInfo` abstracts platform differences while still keeping details available.**
      
      The `tcp_info` structure varies significantly between Apple and Linux. `TcpInfo` exposes a subset of `tcp_info` and other fields through accessors that hide these differences, and reduce potential errors (e.g., Apple stores srtt in milliseconds, Linux stores in microseconds, `TcpInfo::srtt` does the conversions needed to always return in microseconds). When a field is unavailable on a platform, the accessor returns an empty optional.
      
      In parallel, the underlying structures remain accessible and can be safely accessed through the appropriate `getFieldAsOptUInt64(...)`. This enables platform-specific code to have full access to the underlying structure while also benefiting from `TcpInfo`'s knowledge of whether a given field was populated by the ABI at run-time.
      
      Support for FreeBSD will be added in a subsequent diff.
      
      Differential Revision: D22134355
      
      fbshipit-source-id: accae8762aa88c187cc473b8121df901c6ffb456
      68a78d99
    • JTJL's avatar
      Remove semicolons at the end of macros after `do {} while (0)` (#1605) · 16ac56e4
      JTJL authored
      Summary:
      The semicolons at the end of macros after `do {} while (0)` is useless and may cause potential compile errors in the future.
      
      Pull Request resolved: https://github.com/facebook/folly/pull/1605
      
      Reviewed By: Mizuchi
      
      Differential Revision: D29109549
      
      Pulled By: yfeldblum
      
      fbshipit-source-id: 0c585b2db059bc5f53a31671b044a2b86a707359
      16ac56e4
    • Prabhakaran Ganesan's avatar
      Set TOS for AsyncServer listener socket · a1056c1d
      Prabhakaran Ganesan authored
      Summary: Added set/get APIs to configure TOS for listener sockets. The setListenerTos() sets the TOS for the server sockets and all accepted connections are expected to inherit the same. These APIs would be used by higher layers (like thrift server) to set the TOS on the server socket.
      
      Reviewed By: jmswen
      
      Differential Revision: D28651968
      
      fbshipit-source-id: 30f251970269155adbf5e88e1079096dbeceb216
      a1056c1d
    • Francesco Zoffoli's avatar
      Support move only objects in `collectAny` · d92bb4bb
      Francesco Zoffoli authored
      Summary: `collectAny` does not compile when used with `Task`s that return move only objects
      
      Reviewed By: yfeldblum
      
      Differential Revision: D29137632
      
      fbshipit-source-id: d8fd4f46d4c014c7492dcd2fb7fe84921db8aad0
      d92bb4bb
    • Misha Shneerson's avatar
      fix race between EventBase and EventBaseLocal dtors · 033fa8af
      Misha Shneerson authored
      Summary:
      EventBase keeps a registry of EventBaseLocal instances.EventBaseLocal keeps a registry of EventBase instances.
      At destruction time, both are trying to remove themselves from the other's registry, and it is possible that dtors are racing each other.
      
      There are two changes to address the race:
      1. remove virtual method in EventBaseLocal because calling through vptr makes TSAN unhappy - the underlying vtbl is being mutated during destruction.
      2.  Since deregistration involves acquiring two locks, a lock inversion must be avoided. This is achieved by retrying if inner lock acquisition has failed.
      
      Reviewed By: andriigrynenko
      
      Differential Revision: D29120201
      
      fbshipit-source-id: 93c93c8d7cc7126e3432ac06562d55a838008e4a
      033fa8af
    • Ter Chrng Ng's avatar
      Add opt outs to shipit · 76c832bd
      Ter Chrng Ng authored
      Summary: As title
      
      Reviewed By: mzlee
      
      Differential Revision: D29140913
      
      fbshipit-source-id: 6a90756f1c340faaf9e857d743ccbeb1dc991b2f
      76c832bd
    • Mihnea Olteanu's avatar
      Stub out sockets for EMSCRIPTEN · 02d4e327
      Mihnea Olteanu authored
      Summary: Stub out sockets when building under EMSCRIPTEN (aka WASM compiler) like was done in D26579892 (https://github.com/facebook/folly/commit/c76b89b60652af52ee163795d526f2f10a114b20) for XROS.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D28107594
      
      fbshipit-source-id: 8a0d3033793a857cce587c5349934bc6f2a4bec5
      02d4e327
    • Bennett Magy's avatar
      Add CO_TEST_P · c30526f7
      Bennett Magy authored
      Summary:
      Copied TEST_P def from https://github.com/google/googletest/blob/master/googletest/include/gtest/gtest-param-test.h
      
      Implemented `TestBody()` as `blockingWait(co_TestBody())`. User is responsible for delivering impl of `co_TestBody()`.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D29124282
      
      fbshipit-source-id: ca8e9b874903b84ab529e7eefa6a2b7f72793b9b
      c30526f7
    • Genevieve Helsel's avatar
      add option not to prefer /usr/bin python on mac · a7b4818a
      Genevieve Helsel authored
      Reviewed By: chadaustin
      
      Differential Revision: D29084022
      
      fbshipit-source-id: 0605c1bfdd86ab94f4aa6893737b296ab4cdd913
      a7b4818a
    • Francesco Zoffoli's avatar
      Implement coro::collectAny · b8f35551
      Francesco Zoffoli authored
      Summary:
      `collectAll` allows to `co_await`s multiple tasks using structured concurrency.
      
      Unfortunately `future::collectAny` does not follow the structured concurrency pattern, and detaches the uncompleted operations.
      This can result in memory errors (the coroutines access data that has already been freed).
      
      This diff introduces `coro::collectAny`, which given a number of awaitables it returns the result of the first awaitable to finish, in addition to its index, cancels the remaining operations **and waits for them to complete**.
      
      The implementation uses `collectAll` as a building block.
      The return signature mirrors the one from `future::collectAny`.
      
      Reviewed By: yfeldblum, rptynan
      
      Differential Revision: D28945040
      
      fbshipit-source-id: 402be03e004d373cbc74821ae8282b1aaf621b2d
      b8f35551
    • Emanuele Altieri's avatar
      The Latch synchronization class · dc7ba0b5
      Emanuele Altieri authored
      Summary:
      Similar to std::latch (C++20) but with timed waits:
      https://en.cppreference.com/w/cpp/thread/latch
      
      The latch class is a downward counter which can be used to synchronize
      threads. The value of the counter is initialized on creation. Threads may
      block on the latch until the counter is decremented to zero. There is no
      possibility to increase or reset the counter, which makes the latch a
      single-use barrier.
      
      Example:
      
        const int N = 32;
        folly::Latch latch(N);
        std::vector<std::thread> threads;
        for (int i = 0; i < N; i++) {
          threads.emplace_back([&] {
            do_some_work();
            latch.count_down();
          });
        }
        latch.wait();
      
      A latch can be used to easily wait for mocked async methods in tests:
      
        ACTION_P(DecrementLatchImpl, latch) {
          latch.count_down();
        }
        constexpr auto DecrementLatch = DecrementLatchImpl<folly::Latch&>;
      
        class MockableObject {
         public:
          MOCK_METHOD(void, someAsyncEvent, ());
        };
      
        TEST(TestSuite, TestFeature) {
          MockableObject mockObjA;
          MockableObject mockObjB;
      
          folly::Latch latch(5);
      
          EXPECT_CALL(mockObjA, someAsyncEvent())
              .Times(2)
              .WillRepeatedly(DecrementLatch(latch)); // called 2 times
      
          EXPECT_CALL(mockObjB, someAsyncEvent())
              .Times(3)
              .WillRepeatedly(DecrementLatch(latch)); // called 3 times
      
          // trigger async events
          // ...
      
          EXPECT_TRUE(latch.try_wait_for(std::chrono::seconds(60)));
        }
      
      Reviewed By: yfeldblum
      
      Differential Revision: D28951720
      
      fbshipit-source-id: 6a9e20ad925a38d1cdb0134eedad826771bef3e0
      dc7ba0b5
    • Yedidya Feldblum's avatar
      complete the transition away from LockTraits · ddcb93e0
      Yedidya Feldblum authored
      Summary: `Synchronized` no longer needs a full lock-traits facility. Absorb the few things it needs and cut the rest.
      
      Reviewed By: simpkins
      
      Differential Revision: D28774648
      
      fbshipit-source-id: 0679a3192a8eb17444628d12704cdc34fe5911b3
      ddcb93e0
    • Yedidya Feldblum's avatar
      cut legacy LockedPtr::getUniqueLock · b65ef9f8
      Yedidya Feldblum authored
      Summary: Now that `LockedPtr::as_lock` is always available regardless of mutex type and regardless of lock category, `getUniqueLock` is no longer needed.
      
      Differential Revision: D28987941
      
      fbshipit-source-id: a6894cffb30d280ec8325c14784592b2d4381f4c
      b65ef9f8
    • Yedidya Feldblum's avatar
      migrate from LockedPtr::getUniqueLock · 07ab2e2b
      Yedidya Feldblum authored
      Summary: The new name is `LockedPtr::as_lock`.
      
      Reviewed By: aary
      
      Differential Revision: D28987868
      
      fbshipit-source-id: 8abd6a69a1b9c884adf137f06c24fe0df9ddd089
      07ab2e2b
    • Roman Koshelev's avatar
      Correcting and adding a coarse_ * clock (#1580) · 78e483e0
      Roman Koshelev authored
      Summary: Pull Request resolved: https://github.com/facebook/folly/pull/1580
      
      Reviewed By: luciang
      
      Differential Revision: D28627136
      
      Pulled By: yfeldblum
      
      fbshipit-source-id: 1362506502ad3282f53512999d1c79822f2ce6e8
      78e483e0
    • Yedidya Feldblum's avatar
      suppress lint-time diagnostics in OpenSSLThreadding.cpp · 7a06e2f4
      Yedidya Feldblum authored
      Differential Revision: D29089239
      
      fbshipit-source-id: 83cbe9d74d8f7f648e18b8ce1e3e13ca8cb33006
      7a06e2f4
    • Yedidya Feldblum's avatar
      revise Synchronized LockedPtr to use lock types · b805d853
      Yedidya Feldblum authored
      Summary:
      Use `std::unique_lock`, `std::shared_lock`, and `folly::upgrade_lock`. There are two reasons:
      
      * Makes generic the use of `std::unique_lock` with `std::mutex`, which is currently special-cased.
      * Permits specializations of `std::unique_lock` and the other lock types to be found automatically.
      
      In particular, this permits the use of `Synchronized<T, DistributedMutex>`, which is only proxy-lockable and not lockable.
      
      Reviewed By: simpkins
      
      Differential Revision: D28705607
      
      fbshipit-source-id: 48daa2910ce16ee4fde6f5ea629a41d9768f3c87
      b805d853
    • Yedidya Feldblum's avatar
      cut legacy friends of SharedMutex · 424e569f
      Yedidya Feldblum authored
      Summary: They were used as extension points at one time, but no longer.
      
      Reviewed By: Alfus
      
      Differential Revision: D28987212
      
      fbshipit-source-id: e9d59e5cf9641323657314b088eef516ce068112
      424e569f
    • Aaryaman Sagar's avatar
      Fix ParkingLot memory ordering bug · ff7ab9db
      Aaryaman Sagar authored
      Summary:
      ```
      auto x = std::atomic<std::uint64_t>{0};
      auto y = std::atomic<std::uint64_t>{0};
      
      // thread 1
      x.store(1, std::memory_order_release);
      auto one = y.load(std::memory_order_seq_cst);
      
      // thread 2
      y.fetch_add(1, std::memory_order_seq_cst);
      auto two = x.load(std::memory_order_seq_cst);
      ```
      Here it is possible for both `one` and `two` to end up with the value `0`. The
      code in ParkingLot assumed that this would not be possible; and the counter used
      to track the number of waiters could get reordered with respect to loads around
      it.  This diff adds a seq_cst fence to ensure unparking threads always sequence
      their stores before parking _before_ the counter load globally.
      
      Reviewed By: yfeldblum, ot
      
      Differential Revision: D28972810
      
      fbshipit-source-id: 06eb6a2e6df6b00bf07ac8454a79257a5276e154
      ff7ab9db
    • Yedidya Feldblum's avatar
      check functions for DistributedMutex · d418b5ee
      Yedidya Feldblum authored
      Summary: To observe how the compiler generates corresponding code.
      
      Reviewed By: Alfus
      
      Differential Revision: D28984027
      
      fbshipit-source-id: d1c86197931aad257eb922cec9810c71ecdfc20a
      d418b5ee
    • Yedidya Feldblum's avatar
      use lowest() in numeric traits · 348568d4
      Yedidya Feldblum authored
      Summary:
      The numeric functions `less_than` and `greater_than` are intended for integer use but, technically, they are not constrained to integer use. If they are used with floating-point types, `min()` does not do the expected thing so use use `lowest()`.
      
      Fixes: https://github.com/facebook/folly/issues/1604.
      
      Reviewed By: iahs
      
      Differential Revision: D29069620
      
      fbshipit-source-id: 369bd59338b889cb1ec0f56d232a3775500573d0
      348568d4
    • Jiawen Geng's avatar
      make deleted function public for AsyncUDPSocket.h (#1603) · 5c8255f7
      Jiawen Geng authored
      Summary:
      Ref: https://abseil.io/tips/143
      
      Pull Request resolved: https://github.com/facebook/folly/pull/1603
      
      Differential Revision: D29034248
      
      Pulled By: yfeldblum
      
      fbshipit-source-id: 87ae1970eab3f067d71a480fc7a95b18e2041c6d
      5c8255f7
    • Lee Howes's avatar
      Add back default SerialExecutor defaulted to getGlobalCPUExecutor. · 5d231c67
      Lee Howes authored
      Summary: Use the global immutable executor by default for SerialExecutor.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D28925750
      
      fbshipit-source-id: 91f75cfb3a4880098d933fe1f148d5c3b2e896e7
      5d231c67
    • Maged Michael's avatar
      HazptrDomain: Remove unused data members and function · b66627fc
      Maged Michael authored
      Summary:
      Remove unused HazptrDomain data members unprotected_ and children_ and the function reclaim_unprotected_safe.
      
      The data members and function were used before the change in October 2020 that eliminated the nesting of synchronous reclamation within asynchronous reclamation.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D29017460
      
      fbshipit-source-id: 645a61aedc801cb3eb14a4c3a085fea8b8422f1e
      b66627fc
    • Lee Howes's avatar
      Remove use of default SerialExecutor · 99893da2
      Lee Howes authored
      Summary: Remove all uses of the default SerialExecutor by changing callsites to be explicit, and removing the default parameter. This will allow us to change the default to a safer option in a subsequent diff.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D28842180
      
      fbshipit-source-id: 93027dcf8b19c44380534dabd731651780dac90e
      99893da2
    • Dan Melnic's avatar
      io_uring SendmsgRecvmsg test fixes · 61943560
      Dan Melnic authored
      Summary: io_uring SendmsgRecvmsg test fixes
      
      Reviewed By: danobi
      
      Differential Revision: D29003163
      
      fbshipit-source-id: c1139a67d7b687d0eab21be7c6329f593dbe6ea9
      61943560
    • Yedidya Feldblum's avatar
      keep_sink_nx · 3d4885b9
      Yedidya Feldblum authored
      Summary: A function like keep_sink but marked `noexcept`. Useful for shrinking check functions with duplicated cleanups.
      
      Reviewed By: aary
      
      Differential Revision: D28984340
      
      fbshipit-source-id: f1099d43db25492db0e633f8cc8a5d44bcf157bc
      3d4885b9
    • Dead Code Bot's avatar
      Remove dead includes in folly/executors · 12ccdf77
      Dead Code Bot authored
      Reviewed By: yfeldblum
      
      Differential Revision: D28995271
      
      fbshipit-source-id: bd64f4f98742fbad801f3db15a9cb8f5f71fe4b1
      12ccdf77
    • Chad Austin's avatar
      fix tpx command line when filtering is enabled · a6321e32
      Chad Austin authored
      Summary:
      The testpilot command line getdeps generated was not suitable for tpx
      when filtering. tpx would consider the passed test filter an
      environment variable.
      
      Reorder a few things to at least make filtering work.
      
      Reviewed By: fanzeyi
      
      Differential Revision: D28976061
      
      fbshipit-source-id: 21c45b3a4a59711a2a5da36a1bd11b7b4871ec5d
      a6321e32
    • Kudo Chien's avatar
      Fix openNoInt building error after android ndk r21 (#1593) · 73123851
      Kudo Chien authored
      Summary:
      Android NDK bionic with FORTIFY will override original `open()` definition and making folly `wrapNoInt` template failed to deduct.
      The issue may happen only after NDK r21 because [this commit landed after r21](https://android.googlesource.com/platform/bionic/+/9349b9e51b41d12fd054b925802b626ca2db0afb%5E%21/#F0)
      
      References:
      https://github.com/android/ndk/issues/1328
      https://github.com/llvm/llvm-project/commit/0a0e411204a2baa520fd73a8d69b664f98b428ba
      
      Pull Request resolved: https://github.com/facebook/folly/pull/1593
      
      Test Plan:
      Tested running `objdump -dr` on the object file generated in both `mode/opt`
      and `mode/opt-gcc` build modes and confirmed the generated code was
      identical.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D28953120
      
      Pulled By: simpkins
      
      fbshipit-source-id: 225583a5a011e8456592a0bcfcd669fe966ea6af
      73123851
    • Cameron Pickett's avatar
      Build in automatic cancellation support to AsyncScope · ed173626
      Cameron Pickett authored
      Summary: Introduces a CancellableAsyncScope type that automatically adds a CancellationToken to every task added() on the scope.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D28438228
      
      fbshipit-source-id: 2416725360c16f8d95dd5c35997dd624278d0980
      ed173626
    • Yedidya Feldblum's avatar
      add hasher<string_view> · e47bc9b4
      Yedidya Feldblum authored
      Summary: Add a specialization of `folly::hasher<std::string_view>` parallel to the specialilzation of `folly::hasher<std::string>`.
      
      Reviewed By: luciang
      
      Differential Revision: D28921343
      
      fbshipit-source-id: 77213d9ff66cec57b6d36b214c41e479e1d9455d
      e47bc9b4
    • Emanuele Altieri's avatar
      FB_LOG_ONCE() · b3427e71
      Emanuele Altieri authored
      Summary: We need a thread-safe alternative to `LOG_FIRST_N(severity, n)` -- specifically when n = 1, the most common case.
      
      Reviewed By: yfeldblum, ot, luciang
      
      Differential Revision: D28884240
      
      fbshipit-source-id: 41c937bf37dba036b71f1c62fd70445e79a3924a
      b3427e71
    • Andrew Huang's avatar
      Add TLS 1.3 to SSLContext::SSLVersion enum · db723b64
      Andrew Huang authored
      Summary: Since we're moving towards enabling TLS 1.3 by default in folly, add this version to the SSLVersion enum.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D28908538
      
      fbshipit-source-id: 19d9ee248fff4682eec59c6f3d699fc6c9fa1917
      db723b64
    • Yedidya Feldblum's avatar
      extract some SharedMutex params to a policy · 3d61464f
      Yedidya Feldblum authored
      Summary: There are many possible knobs that may be tweaked but long chains of positional params are awkward. Extract a policy param to house many of these chained params. Facilitates adding more params in the future as well.
      
      Reviewed By: ot, luciang
      
      Differential Revision: D28824188
      
      fbshipit-source-id: f56ec15242b148890eced06a3dd101d7cfaaefb1
      3d61464f
    • Yedidya Feldblum's avatar
      simplify catch clauses constructing exception_wrapper · 704df442
      Yedidya Feldblum authored
      Summary:
      Now that `exception_wrapper` can access the exception object directly from the `std::exception_ptr`, it no longer needs to be passed the exception object separately.
      
      This eliminates duplicative `catch (exception const&)` clauses which primarily construct `exception_wrapper` instances, reducing build artifact size.
      
      Reviewed By: akrieger
      
      Differential Revision: D27888763
      
      fbshipit-source-id: 5f60a07083b3c8d818f0eafd5b17afcfca239ab8
      704df442
    • Yedidya Feldblum's avatar
      let try_and_catch work with -fno-exceptions · fdc8edc2
      Yedidya Feldblum authored
      Summary: The helper `catch_exception` may be used to wrap the `try {} catch (...) {}` syntax.
      
      Reviewed By: Orvid, luciang
      
      Differential Revision: D28276166
      
      fbshipit-source-id: be903215251d8ea3321bd6d3b994b19359056f8c
      fdc8edc2
    • Yedidya Feldblum's avatar
      let proxy-lockable unlock pass the state by const& · 168d50be
      Yedidya Feldblum authored
      Summary: Avoid questions of what may the mutex unlock method do to the passed state: nothing.
      
      Reviewed By: aary
      
      Differential Revision: D28895552
      
      fbshipit-source-id: 8b73e76711bb553a344d958c80dcf14d56e74ffd
      168d50be
    • Robert Quitt's avatar
      Clean up old googletest conditional compilations · 1bc5dd9d
      Robert Quitt authored
      Summary:
      Remove all the conditional compilations that were used for the googletest upgrade. We're set on Googletest 1.10.x now, so no need to keep the old code lying around.
      
      TODO in the future, is to deprecate some of the other fb-only patches, and to switch over fbcode to use fbsource/third-party
      
      Reviewed By: yfeldblum, igorsugak
      
      Differential Revision: D28860854
      
      fbshipit-source-id: 8ffbaf860fca8897f6691218bc0154723ec954d4
      1bc5dd9d