1. 04 Nov, 2021 4 commits
    • Cristian Lumezanu's avatar
      Initialize state_ early when creating AsyncSocket from connected fd_ · 75752063
      Cristian Lumezanu authored
      Summary: Move `state_` and `addr_` to the initializer list of the AsyncSocket constructor. This is necessary as there may be installed callbacks for AsyncSocket (e.g., using ConstructorCallback) that need these values early.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31761420
      
      fbshipit-source-id: 2790cff1413835927be8610d7117208ef39e1cd1
      75752063
    • Alex Hornby's avatar
      openssl: update system rpms · cab1d598
      Alex Hornby authored
      Summary: It was missing openssl-libs
      
      Reviewed By: mzr
      
      Differential Revision: D32139851
      
      fbshipit-source-id: a60febe3c9e916d1b1c65dfe88a364f5bcfb33b0
      cab1d598
    • Alex Hornby's avatar
      add centos stream support · 9b43e4c4
      Alex Hornby authored
      Summary:
      Distro name wasn't matching and thus no rpms being installed or used
      
      Also improved the error message for unknown systems
      
      Reviewed By: farnz
      
      Differential Revision: D32139850
      
      fbshipit-source-id: 438278094159f732f6b1de506490b89d2c9cd3f7
      9b43e4c4
    • slowy07's avatar
      fix: typo spelling grammar (#1638) · ca670559
      slowy07 authored
      Summary:
      ## description
      fix typo spelling grammar and replace to correct words
      
      Pull Request resolved: https://github.com/facebook/folly/pull/1638
      
      Reviewed By: yfeldblum
      
      Differential Revision: D30577141
      
      Pulled By: Orvid
      
      fbshipit-source-id: 20c3f49fafcb6ca01a9419f5d71a2414dc7ea635
      ca670559
  2. 03 Nov, 2021 3 commits
    • Ilya Albrecht's avatar
      Up to 25% performance improvements for Skylake based platforms · 75a102ed
      Ilya Albrecht authored
      Summary: Implemented 8 times unrolled linear folding to cover for _mm_clmulepi64_si128 latency on Skylake
      
      Reviewed By: terrelln
      
      Differential Revision: D30376340
      
      fbshipit-source-id: 7828639c135ba51048b60c621f5427c7ac1938b4
      75a102ed
    • Maged Michael's avatar
      hazard pointers: Fix hazptr_tc_evict · 6e255d2e
      Maged Michael authored
      Summary: Fix exit-time destructor warning for hazptr_tc_evict.
      
      Differential Revision: D32099949
      
      fbshipit-source-id: f9f12143f4d06fda1122c8a8d741d6381105253f
      6e255d2e
    • Jun Li (Core System)'s avatar
      Add BoundedQueue implementation to folly::coro · ed5e3082
      Jun Li (Core System) authored
      Summary: BoundedQueue can be useful in many places. This implementation uses MPMCQueue/ProduerConsumerQueue and folly semaphore to achieve bounded queue behavior.
      
      Reviewed By: iahs
      
      Differential Revision: D31782931
      
      fbshipit-source-id: 06aa12ee6e181641eb7b5acb0c6513b58e436d91
      ed5e3082
  3. 02 Nov, 2021 4 commits
    • Konstantin Tsoy's avatar
      Proper branch name for mvfst · ea734e9d
      Konstantin Tsoy authored
      Summary:
      Need to fix branch name for OSS mvfst so CI continues on.
      
      Ran opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh
      
      Reviewed By: lnicco
      
      Differential Revision: D32041656
      
      fbshipit-source-id: f72aab91a3084814a5d3fa8643ae5f405908c313
      ea734e9d
    • Jim Meyering's avatar
      folly/detail/MemoryIdler.cpp: fix platform010-exposed build errors · b89470b9
      Jim Meyering authored
      Summary:
      We should all be migrating to platform010, for the improved performance of its
      generated code and for its improved diagnostics/portability.
      
      cast PTHREAD_STACK_MIN to size_t, since with glibc-2.34, it now has type long
      
      This avoids the following errors:
      
        folly/detail/MemoryIdler.cpp:140:18: error: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'long' [-Werror,-Wsign-compare]
      
      Differential Revision: D32057242
      
      fbshipit-source-id: 1fae2aec5829a837e2aecf068f6c2c061c911e95
      b89470b9
    • Yicheng Wang's avatar
      Adding AcceptInfo into connectionAccepted of AsyncSocket callback · 8405118b
      Yicheng Wang authored
      Summary:
      AsyncServerSocket works like this:
      1) a socket is accepted from the fd
      2) the socket is pushed into the socket queue
      3) the connectionAccepted() callback is called with the socket is dequeued
      
      Things happening before connectionAccepted() are like black box to us currently. For example the time taken for a socket to sit in the socket queue is completely unknown. Adding this AcceptInfo into the callback could allow us to pass more information to downstream for instrumenting.
      
      This diff only pass timestamp of before the socket is being pushed into the queue.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31633582
      
      fbshipit-source-id: 66072955318388c2410b48313d3d54078edce478
      8405118b
    • Samuel Miller's avatar
      Add static helper to extract X509 from cert · 3e26416d
      Samuel Miller authored
      Summary:
      Since callsites wanting an X509 from an AsyncTransportCertificate will have to
      perform a dynamic cast to an OpenSSLTransportCertificiate, we might as well
      abstract that away into a static helper.
      
      This makes it potentially a bit easier to change implementation details in the
      future without having to update a bunch of existing callsites (and it hides the
      ugliness of the dynamic cast).
      
      Reviewed By: yfeldblum, mingtaoy
      
      Differential Revision: D31358948
      
      fbshipit-source-id: 9ed2f81db4cf746a83871961895a0c1f1fb5c297
      3e26416d
  4. 30 Oct, 2021 2 commits
  5. 29 Oct, 2021 3 commits
    • Giuseppe Ottaviano's avatar
      Account for allocated bytes in setEgressBufferBackpressureThreshold() · d2154674
      Giuseppe Ottaviano authored
      Summary:
      The intention of `setEgressBufferBackpressureThreshold()` is to bound memory usage, but it accounts for write size instead. The two can have very large discrepancy, because Thrift allocates large `IOBuf`s when serializing a message: the minimum allocation size is 16KiB, but in Thrift streaming the messages can be much smaller, which can lead to under-accounting by orders of magnitude.
      
      This diff changes the definition to account for allocated bytes.
      
      Differential Revision: D31927400
      
      fbshipit-source-id: 1172f91648b3b21d7d4076d51ec9f569b0e0ba22
      d2154674
    • Mohammad Husain's avatar
      Introduce conditional XLOG_EVERY_ macros · dc171587
      Mohammad Husain authored
      Differential Revision: D31976729
      
      fbshipit-source-id: 617cce3a62bad6efbfaea0e7ea6d0f120ae35dcf
      dc171587
    • Dan Melnic's avatar
      Fix typo · 5f371615
      Dan Melnic authored
      Summary: Fix typo
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31949755
      
      fbshipit-source-id: 31af70aa09a9b15a6bc9ed5567d0ab717956c886
      5f371615
  6. 28 Oct, 2021 5 commits
    • Shai Szulanski's avatar
      folly::coro::Promise · fefc6887
      Shai Szulanski authored
      Summary:
      A lightweight replacement for folly::Promise for use with coroutines.
      The main selling point is cancellation-awareness. It also has a smaller memory footprint (similar CPU cost) and can be `blockingWait`ed.
      
      Differential Revision: D31561034
      
      fbshipit-source-id: d803e02cfd7753ee1f532c8f696cf7fbf672938b
      fefc6887
    • Shai Szulanski's avatar
      Add mechanism to store data alongside CancellationState · 8cf4f930
      Shai Szulanski authored
      Summary: Enables implementing coro::Promise with a single allocation. Not intended for direct use by folly users.
      
      Differential Revision: D31561033
      
      fbshipit-source-id: ae9d350694c05710dcf7abd507f11795786da09d
      8cf4f930
    • Dan Melnic's avatar
      Switch AutoTimer looging to fmt::format · 3a7c07a8
      Dan Melnic authored
      Summary: Switch AutoTimer looging to fmt::format
      
      Reviewed By: mpark
      
      Differential Revision: D31980311
      
      fbshipit-source-id: 91e819a169a16540984e503f7562f9c2d36f75db
      3a7c07a8
    • Dan Melnic's avatar
      Fix C++20 compile issues · 99c72881
      Dan Melnic authored
      Summary: Fix C++20 compile issues
      
      Reviewed By: yfeldblum, mpark, ispeters
      
      Differential Revision: D31938070
      
      fbshipit-source-id: eed570b98685bfe5438a5d02f0417a40f495cb0f
      99c72881
    • Dmytro Stechenko's avatar
      Make Badge constructors noexcept · fe92e62e
      Dmytro Stechenko authored
      Summary:
      Make `folly::badge` and `folly::any_badge` noexcept constructible.
      Cannot make constructors explicitly defaulted due to classes becoming aggregate types.
      Aggregate type construction takes precedence over value constructors so we can construct them anywhere:
      ```
      class A{
          A() = default;
      };
      
      int main() {
          A a = {};
      }
      ```
      Cannot mark them `explicit` either, since we prefer to have conversions auto-magical where possible.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31983342
      
      fbshipit-source-id: ddeffc30e6332882a490afea5bf023522f96c18e
      fe92e62e
  7. 27 Oct, 2021 4 commits
    • Dmytro Stechenko's avatar
      Remove std::variant dependency from Badges · 8318a862
      Dmytro Stechenko authored
      Summary:
      Fixed some `folly::badge` docs to make them more precise.
      Renamed `folly::badges` to `folly::any_badge`.
      Made `folly::any_badge` an empty class that we can only lift into from a singular `folly::badge` that is a part of the badges allowed.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31905323
      
      fbshipit-source-id: f48394dda2c2c6b46169087c4231443552e056e4
      8318a862
    • Dan Melnic's avatar
      Fix C++20 compile issues · 80961096
      Dan Melnic authored
      Summary: Fix C++20 compile issues
      
      Reviewed By: ispeters
      
      Differential Revision: D31943747
      
      fbshipit-source-id: afafc8f2117c558401aaeb1a265b81fccaccf2a4
      80961096
    • Dan Melnic's avatar
      C++20 compile fixes · 84824a23
      Dan Melnic authored
      Summary: C++20 compile fixes
      
      Reviewed By: ispeters
      
      Differential Revision: D31943173
      
      fbshipit-source-id: 7b3a7e4dfe73dd9e258b96d8733d0eda78aeafa9
      84824a23
    • Giuseppe Ottaviano's avatar
      Prevent IOBufQueue::append() from packing into a shared tail · ba1a4c1b
      Giuseppe Ottaviano authored
      Summary:
      If a shared `IOBuf` is appended to `IOBufQueue` its tail is not considered writable, but we may write into it in the packing logic.
      
      Fix that, and share the implementation between the pointer and value versions.
      
      Reviewed By: luciang
      
      Differential Revision: D31890708
      
      fbshipit-source-id: 2b590cbcab7028af9421d29babb760869833bc65
      ba1a4c1b
  8. 26 Oct, 2021 4 commits
    • Keivaun Waugh's avatar
      Call undamped max latency callback on busy time · cd49bf89
      Keivaun Waugh authored
      Summary:
      D30916498 (https://github.com/facebook/folly/commit/a2792966998262d7aa2e64b5390b56ddf8ff12f4) added functionality to call a max latency callback using the undamped loop time (instead of the exponentially smoothed loop time). However, that diff used the total loop time to decide when to call the callback instead of the busy time. This means that if the event base is idle on an epoll, meaning it is waiting for work to do, then we'll end up calling the max latency callback. This doesn't make a lot of sense since what we really care about is the event base taking longer than expected to do one iteration of work. From my reading of the damped loop time callback, it looks like it's using the busy time, so this brings the undamped version in line with the damped version.
      
      Proxygen is currently the only consumer of the undamped max latency time: https://fburl.com/code/caheer2t, so this shouldn't change anyone else's counters.
      
      NOTE: the busy time doesn't seem to include the time for the [before loop callbacks](https://fburl.com/code/zsntr8lv) which I think is a mistake. I can address this in a following diff if others agree that this is a mistake.
      
      Reviewed By: jalopezsilva
      
      Differential Revision: D31905279
      
      fbshipit-source-id: 928add6243822a66ee7fa9cc0cbd93a0078ea0e9
      cd49bf89
    • Maged Michael's avatar
      hazard pointers: Optimize allocation of hazard pointers from domain · 0e92d3c2
      Maged Michael authored
      Summary:
      Optimize the management of available hazard pointers in the domain that are not in thread caches, by adding a linked list of available hazard pointers with a lock bit packed with the pointer to the head of the list. Pop operations are done in two atomic steps: (1) set lock bit, (2) pop hazard pointers and clear lock bit. Push operations are done in one atomic step, but can proceed only when the lock bit is clear.
      
      Microbenchmark results (in a process with 10K+ hazard pointers) show reduction in the latency of constructing destroying two 9-hazard-pointer-arrays that involve one TC hit and one TC miss/overflow from hundreds of microseconds to tens of nanoseconds.
      
      Before:
      ```
      1/1000 TC hit + miss & overflow                   550 ns    502 ns    468 ns
      ```
      
      After:
      ```
      TC hit + miss & overflow                           49 ns     48 ns     48 ns
      ```
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31102053
      
      fbshipit-source-id: ae923bf3b05676e7572cca35179531e947846300
      0e92d3c2
    • Maged Michael's avatar
      hazard pointers: Add microbenchmark of thread cache misses · 4a7ac66b
      Maged Michael authored
      Summary:
      Add a microbenchmark for thread cache misses.
      
      Add member function evict to hazptr_tc.
      
      Add free function hazptr_tc_evict for use in testing and benchmarking only.
      
      Remove the warning for thread cache overflow.
      
      Add member function delete_hazard_pointers to hazptr_domain to be used only for testing and benchmarking.
      
      ## Microbenchmark Results
      
      Results for TC hits and misses under an extreme case of 10000 in-use hazard pointers:
      ```
      10x construct/destruct hazptr_array<9>            105 ns     97 ns     95 ns
      1/1000 TC hit + miss & overflow                   550 ns    502 ns    468 ns
      ```
      
      The latency for constructing/destroying a 9-hazard-pointer-array from TC hits is about 10 ns.
      
      The latency for constructing/destroying two 9-hazard-pointer-arrays from one TC hit and one TC miss/overflow is about 500 us (i.e., 500,000 ns).
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31102027
      
      fbshipit-source-id: dee7333d77ebde25dcf18c6af72cc04e99788e43
      4a7ac66b
    • Giuseppe Ottaviano's avatar
      Use granular locking in SmartExceptionTracer · 88afc741
      Giuseppe Ottaviano authored
      Summary:
      `throwCallback()` holds a read lock on the global map while unwinding the stack trace, blocking all other threads from deleting (and thus throwing) exceptions, since the deleter needs a wlock.
      
      Switch to per-`ExceptionMeta` locking (on top of the map's locks), so we can hold the global lock only for the time to update the map, and move all unwinding, allocation, deallocation, and callbacks outside of it.
      
      Reviewed By: luciang
      
      Differential Revision: D31905278
      
      fbshipit-source-id: f46228dd815bfde844165dd7aee92d6c023ff049
      88afc741
  9. 25 Oct, 2021 5 commits
    • Dan Melnic's avatar
      Fix C++20 compile · b28c0ba2
      Dan Melnic authored
      Summary: Fix C++20 compile
      
      Reviewed By: ispeters
      
      Differential Revision: D31898506
      
      fbshipit-source-id: b7cba220dfb78d13181ae04a298e409cc883eda8
      b28c0ba2
    • Dan Melnic's avatar
      Fix CPP20 compile issues · 8288a7ca
      Dan Melnic authored
      Summary: Fix CPP20 compile issues
      
      Reviewed By: ispeters
      
      Differential Revision: D31900132
      
      fbshipit-source-id: 3338b84cc6aff8f317aad8a93a3e1fba5fdc6fc0
      8288a7ca
    • Giuseppe Ottaviano's avatar
      Reuse the writable tail after IOBufQueue::append(IOBuf) · 0d412cdd
      Giuseppe Ottaviano authored
      Summary:
      After calling `IOBufQueue::append(IOBuf)` (both the `unique_ptr` and value overloads) the existing writable tail is lost, because the new tail buffer may not have a writable tail (in practice it's often shared or sized to fit).
      
      This is especially a problem for zero-copy Thrift serialization, for example of a struct like
      ```
      struct S {
        1: binary (cpp2.type = "folly::IOBuf") data;
      }
      ```
      Thrift allocates at least 16KiB for each new buffer, and writes a handful of bytes before and after the `IOBuf`, so the epilogue will take a whole new 16KiB allocation. The problem is amplified with consecutive binary fields, if they don't fit in 4KiB (`kMaxPackCopy`) and the existing writable tail, each will cost an extra 16KiB, which means a 4x memory amplification.
      
      This diff reuses the previous writable tail buffer by appending a new `IOBuf` to the chain that references it.
      
      Reviewed By: philippv
      
      Differential Revision: D31882377
      
      fbshipit-source-id: fe1d0c82f7df5528534d42b46da78f6597a9e519
      0d412cdd
    • Dan Melnic's avatar
      Fix C++20 build issues · 59e15e8f
      Dan Melnic authored
      Summary: Fix C++20 build issues
      
      Reviewed By: ispeters
      
      Differential Revision: D31848258
      
      fbshipit-source-id: 55c7029f371f5686e62817712f9900ba4a2b4d0d
      59e15e8f
    • Dmytro Stechenko's avatar
      Add Badge pattern to lang utilities · e2d30a85
      Dmytro Stechenko authored
      Summary:
      Nifty little trick to abstract over friend classes.
      See: https://awesomekling.github.io/Serenity-C++-patterns-The-Badge/.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31881896
      
      fbshipit-source-id: 651eaa10336909973478c23dac1de548adb51f6f
      e2d30a85
  10. 23 Oct, 2021 1 commit
    • Nicholas Ormrod's avatar
      Add ImmutableRequestData · a00c6e30
      Nicholas Ormrod authored
      Summary: A simple folly::RequestData implementation.
      
      Reviewed By: praihan
      
      Differential Revision: D31738010
      
      fbshipit-source-id: 3d7c3394fd3daf2dbb2942fe6cdac25207e04e32
      a00c6e30
  11. 22 Oct, 2021 5 commits
    • Nick Terrell's avatar
      Add F14Table::prefetch() to speculatively prefetch · e5722ec4
      Nick Terrell authored
      Summary:
      `F14Table::prehash()` will prefetch the first cache line of the chunk, which F14 is guaranteed to need to find the item. However, more cache lines may be needed. `F14Table::prefetch()` will speculatively prefetch the next 2 cache lines of the chunk, up to the chunk size.
      
      I chose to add this prefetching to a new function `prefetch()` instead of putting it in `prehash()` because these loads are speculative. They may end up polluting the L1 cache with useless cache lines, if the item was in the first cache line. So I think it makes most sense to be explicitly opt-in, for latency sensitive use cases that know they have cold maps.
      
      Lastly, I disable the prefetching in `findImpl()` when using a `F14HashToken`, assuming that when it matters the caller will use `prefetch()`.
      
      Reviewed By: shixiao
      
      Differential Revision: D31751971
      
      fbshipit-source-id: 62043702bb026143cf32c129ac8ba8246d4883bc
      e5722ec4
    • Dan Melnic's avatar
      Fix C++20 build issues · 29ea4cac
      Dan Melnic authored
      Summary: Fix C++20 build issues
      
      Reviewed By: ispeters
      
      Differential Revision: D31842893
      
      fbshipit-source-id: 64a4332939fd101b99ccba6f27c7df740a346844
      29ea4cac
    • Dan Melnic's avatar
      C++20 fixes · bdf5b490
      Dan Melnic authored
      Summary: C++20 fixes
      
      Reviewed By: ispeters, meyering
      
      Differential Revision: D31832469
      
      fbshipit-source-id: 367b9d68104ae8a6e58cc11445b0489cbe9bcf84
      bdf5b490
    • Ben Blackburne's avatar
      Revert D31823117: remove folly:file dependency from folly:random · ad3ec3b8
      Ben Blackburne authored
      Differential Revision:
      D31823117
      
      Original commit changeset: 115a59d69d4c
      
      fbshipit-source-id: 5245896f91b26977f6d8bab204a4e719c23c0900
      ad3ec3b8
    • TJ Yin's avatar
      remove folly:file dependency from folly:random · 95223cc0
      TJ Yin authored
      Reviewed By: vitaut
      
      Differential Revision: D31823117
      
      fbshipit-source-id: 115a59d69d4c91ac35d60764ab24de175e27207c
      95223cc0