1. 28 Oct, 2021 1 commit
    • Dmytro Stechenko's avatar
      Make Badge constructors noexcept · fe92e62e
      Dmytro Stechenko authored
      Summary:
      Make `folly::badge` and `folly::any_badge` noexcept constructible.
      Cannot make constructors explicitly defaulted due to classes becoming aggregate types.
      Aggregate type construction takes precedence over value constructors so we can construct them anywhere:
      ```
      class A{
          A() = default;
      };
      
      int main() {
          A a = {};
      }
      ```
      Cannot mark them `explicit` either, since we prefer to have conversions auto-magical where possible.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31983342
      
      fbshipit-source-id: ddeffc30e6332882a490afea5bf023522f96c18e
      fe92e62e
  2. 27 Oct, 2021 4 commits
    • Dmytro Stechenko's avatar
      Remove std::variant dependency from Badges · 8318a862
      Dmytro Stechenko authored
      Summary:
      Fixed some `folly::badge` docs to make them more precise.
      Renamed `folly::badges` to `folly::any_badge`.
      Made `folly::any_badge` an empty class that we can only lift into from a singular `folly::badge` that is a part of the badges allowed.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31905323
      
      fbshipit-source-id: f48394dda2c2c6b46169087c4231443552e056e4
      8318a862
    • Dan Melnic's avatar
      Fix C++20 compile issues · 80961096
      Dan Melnic authored
      Summary: Fix C++20 compile issues
      
      Reviewed By: ispeters
      
      Differential Revision: D31943747
      
      fbshipit-source-id: afafc8f2117c558401aaeb1a265b81fccaccf2a4
      80961096
    • Dan Melnic's avatar
      C++20 compile fixes · 84824a23
      Dan Melnic authored
      Summary: C++20 compile fixes
      
      Reviewed By: ispeters
      
      Differential Revision: D31943173
      
      fbshipit-source-id: 7b3a7e4dfe73dd9e258b96d8733d0eda78aeafa9
      84824a23
    • Giuseppe Ottaviano's avatar
      Prevent IOBufQueue::append() from packing into a shared tail · ba1a4c1b
      Giuseppe Ottaviano authored
      Summary:
      If a shared `IOBuf` is appended to `IOBufQueue` its tail is not considered writable, but we may write into it in the packing logic.
      
      Fix that, and share the implementation between the pointer and value versions.
      
      Reviewed By: luciang
      
      Differential Revision: D31890708
      
      fbshipit-source-id: 2b590cbcab7028af9421d29babb760869833bc65
      ba1a4c1b
  3. 26 Oct, 2021 4 commits
    • Keivaun Waugh's avatar
      Call undamped max latency callback on busy time · cd49bf89
      Keivaun Waugh authored
      Summary:
      D30916498 (https://github.com/facebook/folly/commit/a2792966998262d7aa2e64b5390b56ddf8ff12f4) added functionality to call a max latency callback using the undamped loop time (instead of the exponentially smoothed loop time). However, that diff used the total loop time to decide when to call the callback instead of the busy time. This means that if the event base is idle on an epoll, meaning it is waiting for work to do, then we'll end up calling the max latency callback. This doesn't make a lot of sense since what we really care about is the event base taking longer than expected to do one iteration of work. From my reading of the damped loop time callback, it looks like it's using the busy time, so this brings the undamped version in line with the damped version.
      
      Proxygen is currently the only consumer of the undamped max latency time: https://fburl.com/code/caheer2t, so this shouldn't change anyone else's counters.
      
      NOTE: the busy time doesn't seem to include the time for the [before loop callbacks](https://fburl.com/code/zsntr8lv) which I think is a mistake. I can address this in a following diff if others agree that this is a mistake.
      
      Reviewed By: jalopezsilva
      
      Differential Revision: D31905279
      
      fbshipit-source-id: 928add6243822a66ee7fa9cc0cbd93a0078ea0e9
      cd49bf89
    • Maged Michael's avatar
      hazard pointers: Optimize allocation of hazard pointers from domain · 0e92d3c2
      Maged Michael authored
      Summary:
      Optimize the management of available hazard pointers in the domain that are not in thread caches, by adding a linked list of available hazard pointers with a lock bit packed with the pointer to the head of the list. Pop operations are done in two atomic steps: (1) set lock bit, (2) pop hazard pointers and clear lock bit. Push operations are done in one atomic step, but can proceed only when the lock bit is clear.
      
      Microbenchmark results (in a process with 10K+ hazard pointers) show reduction in the latency of constructing destroying two 9-hazard-pointer-arrays that involve one TC hit and one TC miss/overflow from hundreds of microseconds to tens of nanoseconds.
      
      Before:
      ```
      1/1000 TC hit + miss & overflow                   550 ns    502 ns    468 ns
      ```
      
      After:
      ```
      TC hit + miss & overflow                           49 ns     48 ns     48 ns
      ```
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31102053
      
      fbshipit-source-id: ae923bf3b05676e7572cca35179531e947846300
      0e92d3c2
    • Maged Michael's avatar
      hazard pointers: Add microbenchmark of thread cache misses · 4a7ac66b
      Maged Michael authored
      Summary:
      Add a microbenchmark for thread cache misses.
      
      Add member function evict to hazptr_tc.
      
      Add free function hazptr_tc_evict for use in testing and benchmarking only.
      
      Remove the warning for thread cache overflow.
      
      Add member function delete_hazard_pointers to hazptr_domain to be used only for testing and benchmarking.
      
      ## Microbenchmark Results
      
      Results for TC hits and misses under an extreme case of 10000 in-use hazard pointers:
      ```
      10x construct/destruct hazptr_array<9>            105 ns     97 ns     95 ns
      1/1000 TC hit + miss & overflow                   550 ns    502 ns    468 ns
      ```
      
      The latency for constructing/destroying a 9-hazard-pointer-array from TC hits is about 10 ns.
      
      The latency for constructing/destroying two 9-hazard-pointer-arrays from one TC hit and one TC miss/overflow is about 500 us (i.e., 500,000 ns).
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31102027
      
      fbshipit-source-id: dee7333d77ebde25dcf18c6af72cc04e99788e43
      4a7ac66b
    • Giuseppe Ottaviano's avatar
      Use granular locking in SmartExceptionTracer · 88afc741
      Giuseppe Ottaviano authored
      Summary:
      `throwCallback()` holds a read lock on the global map while unwinding the stack trace, blocking all other threads from deleting (and thus throwing) exceptions, since the deleter needs a wlock.
      
      Switch to per-`ExceptionMeta` locking (on top of the map's locks), so we can hold the global lock only for the time to update the map, and move all unwinding, allocation, deallocation, and callbacks outside of it.
      
      Reviewed By: luciang
      
      Differential Revision: D31905278
      
      fbshipit-source-id: f46228dd815bfde844165dd7aee92d6c023ff049
      88afc741
  4. 25 Oct, 2021 5 commits
    • Dan Melnic's avatar
      Fix C++20 compile · b28c0ba2
      Dan Melnic authored
      Summary: Fix C++20 compile
      
      Reviewed By: ispeters
      
      Differential Revision: D31898506
      
      fbshipit-source-id: b7cba220dfb78d13181ae04a298e409cc883eda8
      b28c0ba2
    • Dan Melnic's avatar
      Fix CPP20 compile issues · 8288a7ca
      Dan Melnic authored
      Summary: Fix CPP20 compile issues
      
      Reviewed By: ispeters
      
      Differential Revision: D31900132
      
      fbshipit-source-id: 3338b84cc6aff8f317aad8a93a3e1fba5fdc6fc0
      8288a7ca
    • Giuseppe Ottaviano's avatar
      Reuse the writable tail after IOBufQueue::append(IOBuf) · 0d412cdd
      Giuseppe Ottaviano authored
      Summary:
      After calling `IOBufQueue::append(IOBuf)` (both the `unique_ptr` and value overloads) the existing writable tail is lost, because the new tail buffer may not have a writable tail (in practice it's often shared or sized to fit).
      
      This is especially a problem for zero-copy Thrift serialization, for example of a struct like
      ```
      struct S {
        1: binary (cpp2.type = "folly::IOBuf") data;
      }
      ```
      Thrift allocates at least 16KiB for each new buffer, and writes a handful of bytes before and after the `IOBuf`, so the epilogue will take a whole new 16KiB allocation. The problem is amplified with consecutive binary fields, if they don't fit in 4KiB (`kMaxPackCopy`) and the existing writable tail, each will cost an extra 16KiB, which means a 4x memory amplification.
      
      This diff reuses the previous writable tail buffer by appending a new `IOBuf` to the chain that references it.
      
      Reviewed By: philippv
      
      Differential Revision: D31882377
      
      fbshipit-source-id: fe1d0c82f7df5528534d42b46da78f6597a9e519
      0d412cdd
    • Dan Melnic's avatar
      Fix C++20 build issues · 59e15e8f
      Dan Melnic authored
      Summary: Fix C++20 build issues
      
      Reviewed By: ispeters
      
      Differential Revision: D31848258
      
      fbshipit-source-id: 55c7029f371f5686e62817712f9900ba4a2b4d0d
      59e15e8f
    • Dmytro Stechenko's avatar
      Add Badge pattern to lang utilities · e2d30a85
      Dmytro Stechenko authored
      Summary:
      Nifty little trick to abstract over friend classes.
      See: https://awesomekling.github.io/Serenity-C++-patterns-The-Badge/.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31881896
      
      fbshipit-source-id: 651eaa10336909973478c23dac1de548adb51f6f
      e2d30a85
  5. 23 Oct, 2021 1 commit
    • Nicholas Ormrod's avatar
      Add ImmutableRequestData · a00c6e30
      Nicholas Ormrod authored
      Summary: A simple folly::RequestData implementation.
      
      Reviewed By: praihan
      
      Differential Revision: D31738010
      
      fbshipit-source-id: 3d7c3394fd3daf2dbb2942fe6cdac25207e04e32
      a00c6e30
  6. 22 Oct, 2021 6 commits
    • Nick Terrell's avatar
      Add F14Table::prefetch() to speculatively prefetch · e5722ec4
      Nick Terrell authored
      Summary:
      `F14Table::prehash()` will prefetch the first cache line of the chunk, which F14 is guaranteed to need to find the item. However, more cache lines may be needed. `F14Table::prefetch()` will speculatively prefetch the next 2 cache lines of the chunk, up to the chunk size.
      
      I chose to add this prefetching to a new function `prefetch()` instead of putting it in `prehash()` because these loads are speculative. They may end up polluting the L1 cache with useless cache lines, if the item was in the first cache line. So I think it makes most sense to be explicitly opt-in, for latency sensitive use cases that know they have cold maps.
      
      Lastly, I disable the prefetching in `findImpl()` when using a `F14HashToken`, assuming that when it matters the caller will use `prefetch()`.
      
      Reviewed By: shixiao
      
      Differential Revision: D31751971
      
      fbshipit-source-id: 62043702bb026143cf32c129ac8ba8246d4883bc
      e5722ec4
    • Dan Melnic's avatar
      Fix C++20 build issues · 29ea4cac
      Dan Melnic authored
      Summary: Fix C++20 build issues
      
      Reviewed By: ispeters
      
      Differential Revision: D31842893
      
      fbshipit-source-id: 64a4332939fd101b99ccba6f27c7df740a346844
      29ea4cac
    • Dan Melnic's avatar
      C++20 fixes · bdf5b490
      Dan Melnic authored
      Summary: C++20 fixes
      
      Reviewed By: ispeters, meyering
      
      Differential Revision: D31832469
      
      fbshipit-source-id: 367b9d68104ae8a6e58cc11445b0489cbe9bcf84
      bdf5b490
    • Ben Blackburne's avatar
      Revert D31823117: remove folly:file dependency from folly:random · ad3ec3b8
      Ben Blackburne authored
      Differential Revision:
      D31823117
      
      Original commit changeset: 115a59d69d4c
      
      fbshipit-source-id: 5245896f91b26977f6d8bab204a4e719c23c0900
      ad3ec3b8
    • TJ Yin's avatar
      remove folly:file dependency from folly:random · 95223cc0
      TJ Yin authored
      Reviewed By: vitaut
      
      Differential Revision: D31823117
      
      fbshipit-source-id: 115a59d69d4c91ac35d60764ab24de175e27207c
      95223cc0
    • James Donald's avatar
      Suppress MSVC spurious warning about not-enough-args BOOST macro · 0cb87475
      James Donald authored
      Summary:
      These boost macros have a longstanding warning problem, not fixed in VS 2019
      ```
      folly\overload.h(72): warning C4003: not enough arguments for function-like macro invocation 'BOOST_PP_TUPLE_TO_LIST_1'
      folly\overload.h(73): warning C4003: not enough arguments for function-like macro invocation 'BOOST_PP_TUPLE_TO_LIST_1'
      ```
      
      Reviewed By: kosievdmerwe
      
      Differential Revision: D25497039
      
      fbshipit-source-id: 47e6555681f043d6d7a80cc0c963d6e326e70242
      0cb87475
  7. 21 Oct, 2021 3 commits
    • Cristian Lumezanu's avatar
      Fix ConstructorCallback destructor consistency issue · cf3af766
      Cristian Lumezanu authored
      Summary:
      ConstructorCallback enables the addition of callback functions to be called when a class constructor is called (D27056139 (https://github.com/facebook/folly/commit/d31902b958fc21c7bf812b0ec45c796b4bf9073c)). The ConstructorCallback object holds a static array with all installed callbacks. When a class constructor is called, the ConstructorCallback constructor traverses the array and executes each installed callback. The array is destroyed when all instances of the class are destroyed.
      
      This diff fixes a potential concurrency issue when ConstructorCallbacks are destroyed: the callback array is destroyed while another thread is calling the class constructor and potentially looping through the callback array. We fix the issue by putting the callback array, the callback count, and the mutex into a struct and initializing a `createGlobal` object from the struct.
      
      **Facebook:**
      
      We describe the issue in more detail in T100230454.
      
      Reviewed By: bschlinker
      
      Differential Revision: D30910255
      
      fbshipit-source-id: d345900dbadbf2e009719b6f930ab1d8aa284647
      cf3af766
    • Callum Ryan's avatar
      Task cancellation from Python · 359722f3
      Callum Ryan authored
      Summary:
      Extending the current support for python futures <-> folly tasks interop to also support cancellation.
      
      By adding a new version of `bridgeCoroTask` that also accepts a `folly::CancellationToken` we can give python code the ability to cancel the folly task.
      
      Reviewed By: nanshu
      
      Differential Revision: D31765430
      
      fbshipit-source-id: 4277e94e296f56b998c9c583823a93ce5b9e7586
      359722f3
    • Dan Melnic's avatar
      Fix C++20 build · ea4721b1
      Dan Melnic authored
      Summary: Fix C++20 build
      
      Reviewed By: luciang
      
      Differential Revision: D31786941
      
      fbshipit-source-id: 153ffcee7b64f4c6dfafb786df90e909cee723b3
      ea4721b1
  8. 20 Oct, 2021 5 commits
    • Nikita Lutsenko's avatar
      folly | Add stub for ThreadId when building with emscripten. · 63f559ca
      Nikita Lutsenko authored
      Summary: No thread id available built-in in emscripten, even though it has partial pthread support. Stub it out.
      
      Reviewed By: boguscoder
      
      Differential Revision: D31754569
      
      fbshipit-source-id: 29113ba31cf6b65e7d5c52455fd99ee1e659c041
      63f559ca
    • Yedidya Feldblum's avatar
      move make_unique_lock to folly/synchronization/Lock.h · 5bef498a
      Yedidya Feldblum authored
      Summary: Since all the other lock utilities are there.
      
      Differential Revision: D31523986
      
      fbshipit-source-id: 22a071c50e51518b061980d7429d1a28755dbe40
      5bef498a
    • TJ Yin's avatar
      fix file_util in windows · 1e7f9f89
      TJ Yin authored
      Summary:
      For MSVC, `ssize_t` and `std::make_signed_t<std::size_t>` are different types (even though both are 32 bit signed integer, `std::is_same` is false: https://godbolt.org/z/n4Wfbh3nM). This creates type mismatch between header and cpp file.
      
      This diff changed to include `folly/portability/SysTypes.h` which defines `ssize_t` in a portable way.
      
      Reviewed By: ot
      
      Differential Revision: D31722673
      
      fbshipit-source-id: c8968d5e0b5ac12d8d6ab4c141f791cb73d80686
      1e7f9f89
    • Dan Melnic's avatar
      Fix -Werror,-Wunused-result · 5cfeab7b
      Dan Melnic authored
      Summary: Fix -Werror,-Wunused-result
      
      Reviewed By: ispeters
      
      Differential Revision: D31786026
      
      fbshipit-source-id: 51af8e21a02f9d6f476d0cb79eb7c6d38f545f20
      5cfeab7b
    • Dan Melnic's avatar
      Fix C++20 build · 0f877ebd
      Dan Melnic authored
      Summary: Fix C++20 build
      
      Reviewed By: ispeters
      
      Differential Revision: D31785835
      
      fbshipit-source-id: ecfa717787546048c422084785b51e0f7d8eaa26
      0f877ebd
  9. 19 Oct, 2021 5 commits
  10. 18 Oct, 2021 3 commits
    • Cristian Lumezanu's avatar
      Add connectAttempt and connectSuccess AsyncSocket::LifecycleObserver callbacks · ba3aedb3
      Cristian Lumezanu authored
      Summary:
      Made the following changes AsyncTransport::LifecycleObserver callback
      - added `connectAttempt()`, called for each observer when connect is invoked;
      - changed `connect()` to `connectSuccess()`, called for each observer when connect is successful;
      
      Reviewed By: bschlinker
      
      Differential Revision: D29962337
      
      fbshipit-source-id: f1d2c0e48b6d7e263466f74c97d701bb9df9b313
      ba3aedb3
    • Yedidya Feldblum's avatar
      fix dynamic::operator--(int) · 81c70338
      Yedidya Feldblum authored
      Summary: A typo in its name meant it was missing and that `dynamic::operator-(int)` was defined instead.
      
      Reviewed By: ot, luciang
      
      Differential Revision: D31726709
      
      fbshipit-source-id: 42696c958f674de806dd56c3d89ec07dd60db961
      81c70338
    • Giuseppe Ottaviano's avatar
      Fix retryingJitteredExponentialBackoffDur when jitter_param=0 · 51669993
      Giuseppe Ottaviano authored
      Summary:
      `retryingJitteredExponentialBackoffDur()` supports `jitter_param=0` (sanctioned in tests, and widely used to disable jitter based on a runtime parameter), but `std::normal_distribution` does not, so just special-case it.
      
      In practice, libstdc++ already returns 0.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31722687
      
      fbshipit-source-id: d81b1736379c596c4fc352b0067a4ad5788b0f8c
      51669993
  11. 17 Oct, 2021 3 commits
    • Yedidya Feldblum's avatar
      revise sanitizer extern wrappers · e83309fe
      Yedidya Feldblum authored
      Summary: Make them a bit shorter with a bit less repetition. Performance-wise, trades away two direct calls for one indirect but perfectly-predictable call. And continues to optimize away in non-sanitizer builds.
      
      Reviewed By: luciang
      
      Differential Revision: D31673074
      
      fbshipit-source-id: f6d8b66cf45a81788d1ec0c5b58241f4fc693001
      e83309fe
    • Yedidya Feldblum's avatar
      use relaxed_atomic in BufferedRandomDevice · 0ae0ccc8
      Yedidya Feldblum authored
      Differential Revision: D31661218
      
      fbshipit-source-id: 48535870ab2b2f86b0d4e9286365fe906110dc3e
      0ae0ccc8
    • Yedidya Feldblum's avatar
      when Baton::wait marks timeout, just use relaxed · e5d88948
      Yedidya Feldblum authored
      Summary: While the baton is used to guard associated memory, if the wait times out, the associated memory is not available. There is no corresponding load-acquire, so marking the timeout may use a store-relaxed rather than a store-release.
      
      Reviewed By: luciang
      
      Differential Revision: D31676286
      
      fbshipit-source-id: dd2474db1ba053337de7e84aa88e7556e630b1ce
      e5d88948