1. 07 Nov, 2021 1 commit
  2. 06 Nov, 2021 3 commits
    • Andrew Smith's avatar
      Immediately cancel any running transform callback on sender cancellation · a4cffed8
      Andrew Smith authored
      Summary: Currently, when we get a cancelled notification on the output sender for a transform, we schedule the cancellation of any running transform callback on an executor. CancellationSource is thread-safe, so we don't need to do this. This diff ensures that cancelling the output receiver will immediately trigger cancellation of any running transform callback, allowing the transform callback to deterministically know that it has been cancelled.
      
      Reviewed By: aary
      
      Differential Revision: D32191796
      
      fbshipit-source-id: aabd58be9b58b0f560b8127b336e626c3482a644
      a4cffed8
    • Andrew Smith's avatar
      Change FanoutChannel to synchronously process handle destruction. · 85724a41
      Andrew Smith authored
      Summary: This ensures that closing a FanoutChannel will synchronously cancel its input receiver. We don't need to run this code on the provided executor, because it is not running as part of a callback (and because we already have a lock).
      
      Reviewed By: aary
      
      Differential Revision: D32191795
      
      fbshipit-source-id: 9ec2a2303a12580e052de7af8651ebf5b773b8b6
      85724a41
    • Yedidya Feldblum's avatar
      DelayedDestructionUniquePtr · b0066f2b
      Yedidya Feldblum authored
      Summary: To ease writing and reading IO code.
      
      Reviewed By: iahs
      
      Differential Revision: D32220718
      
      fbshipit-source-id: db695eb33a60aae0db1b34991bd7d5ce33489f86
      b0066f2b
  3. 05 Nov, 2021 9 commits
    • Yedidya Feldblum's avatar
      a comment explaining the Baton::wait early-delivery c/x · cfbc23fa
      Yedidya Feldblum authored
      Summary: The memory orders seem backwards but are correct. Add a comment explaining the surprising situation.
      
      Reviewed By: ot
      
      Differential Revision: D31594601
      
      fbshipit-source-id: 61a3f5a5700cfe640e025ad5d5388a5dda1f97eb
      cfbc23fa
    • Akrama Baig Mirza's avatar
      Add ThreadIdCollector to IOThreadPoolDeadlockDetectorObserver · d4af6bb8
      Akrama Baig Mirza authored
      Summary:
      Add an implementation of `WorkerProvider` to `IOThreadPoolDeadlockDetectorObserver`.
      
      - On thread start, the observer creates a `TaskStallDetector` and provides it a `ThreadIdCollector` containing the TID of the thread that started
      - The thread will be blocked from exiting until any existing `WorkerProvider::KeepAlive`s are destroyed
      
      Differential Revision: D31963855
      
      fbshipit-source-id: cd749d76a433684a8280d95a4dbff441e32bfda9
      d4af6bb8
    • Alex Hornby's avatar
      remove fbthrift_socket-02 references · ddcb646b
      Alex Hornby authored
      Summary: Move the hg build to fbthrift_socket, it was referencing the removed fbthrift_socket-02 from some cargo files
      
      Reviewed By: mzr
      
      Differential Revision: D32159904
      
      fbshipit-source-id: 58db0fd53cf859d19a237c0a1d50370d405132b8
      ddcb646b
    • Alex Hornby's avatar
      update boost rpm package names for centos stream · 6b0244e2
      Alex Hornby authored
      Summary:
      rpm names needed updating as boost-contract wasn't found.
      
      The rpm names now include the boost version which should be handy when boost dependency is updated next
      
      Unfortunately the centos boost 1.69 rpms aren't found automatically by the centos cmake boost rules, so this also adds builder.py logic to point to it using  BOOST_INCLUDEDIR and BOOST_LIBRARYDIR
      
      Reviewed By: mzr
      
      Differential Revision: D32140834
      
      fbshipit-source-id: 3e2dd822613957ae4d7be5b73623ff581f11d02b
      6b0244e2
    • Alex Hornby's avatar
      add rpms for double-conversion · c20ad836
      Alex Hornby authored
      Summary: Add the system dependencies so getdeps can install them if not present
      
      Reviewed By: krallin
      
      Differential Revision: D32152976
      
      fbshipit-source-id: 29311f37126e1eac8dfbe2f46f6bc3516848d48b
      c20ad836
    • Giuseppe Ottaviano's avatar
      Codemod EliasFanoEncoderV2 -> EliasFanoEncoder · a0e9a217
      Giuseppe Ottaviano authored
      Summary: There hasn't been a V1 for a very long time.
      
      Reviewed By: philippv, luciang
      
      Differential Revision: D32170751
      
      fbshipit-source-id: f495a81657cf53f870ce93572d46d835c7fb8496
      a0e9a217
    • Giuseppe Ottaviano's avatar
      Optimize skipTo() if it is guaranteed to make progress · 7139fa7a
      Giuseppe Ottaviano authored
      Summary: Add an option to specify that `skipTo()` cannot be called on the current value, so we can save a comparison.
      
      Reviewed By: luciang
      
      Differential Revision: D32161927
      
      fbshipit-source-id: 6ebfe7711a5ee3bcdf40cd8a1097da777f78c277
      7139fa7a
    • Giuseppe Ottaviano's avatar
      Do not read the end of the list on EliasFanoReader construction · 928aa5de
      Giuseppe Ottaviano authored
      Summary:
      Move size and upper bound accounting to `UpperBitsReader`, so we don't need to read the last value on construction, which can be expensive when opening a large number of small lists.
      
      This also makes the maximum `ValueType` representable, and lifts the requirement that the provided upper bound at construction time must be equal to the last element in the list, allowing multiple lists to share the encoding of the upper bound.
      
      Based on initial work by swaroopnm.
      
      Reviewed By: philippv
      
      Differential Revision: D32130075
      
      fbshipit-source-id: e98a053b46b10c435ac0d402ff94a56fcfe095ad
      928aa5de
    • Giuseppe Ottaviano's avatar
      Improve test coverage of EliasFanoCoding · 446ac5fa
      Giuseppe Ottaviano authored
      Summary:
      Add tests where the list size is close or at the maximum supported size, by fixing `EliasFanoReader` to support `uint16_t` as value and size type.
      
      Types smaller than `int` require special support because any arithmetic operation promotes them to `int`, so operations that rely on unsigned overflow break. Work around this by introducing a helper for addition that preserves the type.
      
      Reviewed By: philippv, luciang
      
      Differential Revision: D32130076
      
      fbshipit-source-id: 763a8e4cc2b023405c173316d2415ca8c3ec7148
      446ac5fa
  4. 04 Nov, 2021 4 commits
    • Cristian Lumezanu's avatar
      Initialize state_ early when creating AsyncSocket from connected fd_ · 75752063
      Cristian Lumezanu authored
      Summary: Move `state_` and `addr_` to the initializer list of the AsyncSocket constructor. This is necessary as there may be installed callbacks for AsyncSocket (e.g., using ConstructorCallback) that need these values early.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31761420
      
      fbshipit-source-id: 2790cff1413835927be8610d7117208ef39e1cd1
      75752063
    • Alex Hornby's avatar
      openssl: update system rpms · cab1d598
      Alex Hornby authored
      Summary: It was missing openssl-libs
      
      Reviewed By: mzr
      
      Differential Revision: D32139851
      
      fbshipit-source-id: a60febe3c9e916d1b1c65dfe88a364f5bcfb33b0
      cab1d598
    • Alex Hornby's avatar
      add centos stream support · 9b43e4c4
      Alex Hornby authored
      Summary:
      Distro name wasn't matching and thus no rpms being installed or used
      
      Also improved the error message for unknown systems
      
      Reviewed By: farnz
      
      Differential Revision: D32139850
      
      fbshipit-source-id: 438278094159f732f6b1de506490b89d2c9cd3f7
      9b43e4c4
    • slowy07's avatar
      fix: typo spelling grammar (#1638) · ca670559
      slowy07 authored
      Summary:
      ## description
      fix typo spelling grammar and replace to correct words
      
      Pull Request resolved: https://github.com/facebook/folly/pull/1638
      
      Reviewed By: yfeldblum
      
      Differential Revision: D30577141
      
      Pulled By: Orvid
      
      fbshipit-source-id: 20c3f49fafcb6ca01a9419f5d71a2414dc7ea635
      ca670559
  5. 03 Nov, 2021 3 commits
    • Ilya Albrecht's avatar
      Up to 25% performance improvements for Skylake based platforms · 75a102ed
      Ilya Albrecht authored
      Summary: Implemented 8 times unrolled linear folding to cover for _mm_clmulepi64_si128 latency on Skylake
      
      Reviewed By: terrelln
      
      Differential Revision: D30376340
      
      fbshipit-source-id: 7828639c135ba51048b60c621f5427c7ac1938b4
      75a102ed
    • Maged Michael's avatar
      hazard pointers: Fix hazptr_tc_evict · 6e255d2e
      Maged Michael authored
      Summary: Fix exit-time destructor warning for hazptr_tc_evict.
      
      Differential Revision: D32099949
      
      fbshipit-source-id: f9f12143f4d06fda1122c8a8d741d6381105253f
      6e255d2e
    • Jun Li (Core System)'s avatar
      Add BoundedQueue implementation to folly::coro · ed5e3082
      Jun Li (Core System) authored
      Summary: BoundedQueue can be useful in many places. This implementation uses MPMCQueue/ProduerConsumerQueue and folly semaphore to achieve bounded queue behavior.
      
      Reviewed By: iahs
      
      Differential Revision: D31782931
      
      fbshipit-source-id: 06aa12ee6e181641eb7b5acb0c6513b58e436d91
      ed5e3082
  6. 02 Nov, 2021 4 commits
    • Konstantin Tsoy's avatar
      Proper branch name for mvfst · ea734e9d
      Konstantin Tsoy authored
      Summary:
      Need to fix branch name for OSS mvfst so CI continues on.
      
      Ran opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh
      
      Reviewed By: lnicco
      
      Differential Revision: D32041656
      
      fbshipit-source-id: f72aab91a3084814a5d3fa8643ae5f405908c313
      ea734e9d
    • Jim Meyering's avatar
      folly/detail/MemoryIdler.cpp: fix platform010-exposed build errors · b89470b9
      Jim Meyering authored
      Summary:
      We should all be migrating to platform010, for the improved performance of its
      generated code and for its improved diagnostics/portability.
      
      cast PTHREAD_STACK_MIN to size_t, since with glibc-2.34, it now has type long
      
      This avoids the following errors:
      
        folly/detail/MemoryIdler.cpp:140:18: error: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'long' [-Werror,-Wsign-compare]
      
      Differential Revision: D32057242
      
      fbshipit-source-id: 1fae2aec5829a837e2aecf068f6c2c061c911e95
      b89470b9
    • Yicheng Wang's avatar
      Adding AcceptInfo into connectionAccepted of AsyncSocket callback · 8405118b
      Yicheng Wang authored
      Summary:
      AsyncServerSocket works like this:
      1) a socket is accepted from the fd
      2) the socket is pushed into the socket queue
      3) the connectionAccepted() callback is called with the socket is dequeued
      
      Things happening before connectionAccepted() are like black box to us currently. For example the time taken for a socket to sit in the socket queue is completely unknown. Adding this AcceptInfo into the callback could allow us to pass more information to downstream for instrumenting.
      
      This diff only pass timestamp of before the socket is being pushed into the queue.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31633582
      
      fbshipit-source-id: 66072955318388c2410b48313d3d54078edce478
      8405118b
    • Samuel Miller's avatar
      Add static helper to extract X509 from cert · 3e26416d
      Samuel Miller authored
      Summary:
      Since callsites wanting an X509 from an AsyncTransportCertificate will have to
      perform a dynamic cast to an OpenSSLTransportCertificiate, we might as well
      abstract that away into a static helper.
      
      This makes it potentially a bit easier to change implementation details in the
      future without having to update a bunch of existing callsites (and it hides the
      ugliness of the dynamic cast).
      
      Reviewed By: yfeldblum, mingtaoy
      
      Differential Revision: D31358948
      
      fbshipit-source-id: 9ed2f81db4cf746a83871961895a0c1f1fb5c297
      3e26416d
  7. 30 Oct, 2021 2 commits
  8. 29 Oct, 2021 3 commits
    • Giuseppe Ottaviano's avatar
      Account for allocated bytes in setEgressBufferBackpressureThreshold() · d2154674
      Giuseppe Ottaviano authored
      Summary:
      The intention of `setEgressBufferBackpressureThreshold()` is to bound memory usage, but it accounts for write size instead. The two can have very large discrepancy, because Thrift allocates large `IOBuf`s when serializing a message: the minimum allocation size is 16KiB, but in Thrift streaming the messages can be much smaller, which can lead to under-accounting by orders of magnitude.
      
      This diff changes the definition to account for allocated bytes.
      
      Differential Revision: D31927400
      
      fbshipit-source-id: 1172f91648b3b21d7d4076d51ec9f569b0e0ba22
      d2154674
    • Mohammad Husain's avatar
      Introduce conditional XLOG_EVERY_ macros · dc171587
      Mohammad Husain authored
      Differential Revision: D31976729
      
      fbshipit-source-id: 617cce3a62bad6efbfaea0e7ea6d0f120ae35dcf
      dc171587
    • Dan Melnic's avatar
      Fix typo · 5f371615
      Dan Melnic authored
      Summary: Fix typo
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31949755
      
      fbshipit-source-id: 31af70aa09a9b15a6bc9ed5567d0ab717956c886
      5f371615
  9. 28 Oct, 2021 5 commits
    • Shai Szulanski's avatar
      folly::coro::Promise · fefc6887
      Shai Szulanski authored
      Summary:
      A lightweight replacement for folly::Promise for use with coroutines.
      The main selling point is cancellation-awareness. It also has a smaller memory footprint (similar CPU cost) and can be `blockingWait`ed.
      
      Differential Revision: D31561034
      
      fbshipit-source-id: d803e02cfd7753ee1f532c8f696cf7fbf672938b
      fefc6887
    • Shai Szulanski's avatar
      Add mechanism to store data alongside CancellationState · 8cf4f930
      Shai Szulanski authored
      Summary: Enables implementing coro::Promise with a single allocation. Not intended for direct use by folly users.
      
      Differential Revision: D31561033
      
      fbshipit-source-id: ae9d350694c05710dcf7abd507f11795786da09d
      8cf4f930
    • Dan Melnic's avatar
      Switch AutoTimer looging to fmt::format · 3a7c07a8
      Dan Melnic authored
      Summary: Switch AutoTimer looging to fmt::format
      
      Reviewed By: mpark
      
      Differential Revision: D31980311
      
      fbshipit-source-id: 91e819a169a16540984e503f7562f9c2d36f75db
      3a7c07a8
    • Dan Melnic's avatar
      Fix C++20 compile issues · 99c72881
      Dan Melnic authored
      Summary: Fix C++20 compile issues
      
      Reviewed By: yfeldblum, mpark, ispeters
      
      Differential Revision: D31938070
      
      fbshipit-source-id: eed570b98685bfe5438a5d02f0417a40f495cb0f
      99c72881
    • Dmytro Stechenko's avatar
      Make Badge constructors noexcept · fe92e62e
      Dmytro Stechenko authored
      Summary:
      Make `folly::badge` and `folly::any_badge` noexcept constructible.
      Cannot make constructors explicitly defaulted due to classes becoming aggregate types.
      Aggregate type construction takes precedence over value constructors so we can construct them anywhere:
      ```
      class A{
          A() = default;
      };
      
      int main() {
          A a = {};
      }
      ```
      Cannot mark them `explicit` either, since we prefer to have conversions auto-magical where possible.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31983342
      
      fbshipit-source-id: ddeffc30e6332882a490afea5bf023522f96c18e
      fe92e62e
  10. 27 Oct, 2021 4 commits
    • Dmytro Stechenko's avatar
      Remove std::variant dependency from Badges · 8318a862
      Dmytro Stechenko authored
      Summary:
      Fixed some `folly::badge` docs to make them more precise.
      Renamed `folly::badges` to `folly::any_badge`.
      Made `folly::any_badge` an empty class that we can only lift into from a singular `folly::badge` that is a part of the badges allowed.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31905323
      
      fbshipit-source-id: f48394dda2c2c6b46169087c4231443552e056e4
      8318a862
    • Dan Melnic's avatar
      Fix C++20 compile issues · 80961096
      Dan Melnic authored
      Summary: Fix C++20 compile issues
      
      Reviewed By: ispeters
      
      Differential Revision: D31943747
      
      fbshipit-source-id: afafc8f2117c558401aaeb1a265b81fccaccf2a4
      80961096
    • Dan Melnic's avatar
      C++20 compile fixes · 84824a23
      Dan Melnic authored
      Summary: C++20 compile fixes
      
      Reviewed By: ispeters
      
      Differential Revision: D31943173
      
      fbshipit-source-id: 7b3a7e4dfe73dd9e258b96d8733d0eda78aeafa9
      84824a23
    • Giuseppe Ottaviano's avatar
      Prevent IOBufQueue::append() from packing into a shared tail · ba1a4c1b
      Giuseppe Ottaviano authored
      Summary:
      If a shared `IOBuf` is appended to `IOBufQueue` its tail is not considered writable, but we may write into it in the packing logic.
      
      Fix that, and share the implementation between the pointer and value versions.
      
      Reviewed By: luciang
      
      Differential Revision: D31890708
      
      fbshipit-source-id: 2b590cbcab7028af9421d29babb760869833bc65
      ba1a4c1b
  11. 26 Oct, 2021 2 commits
    • Keivaun Waugh's avatar
      Call undamped max latency callback on busy time · cd49bf89
      Keivaun Waugh authored
      Summary:
      D30916498 (https://github.com/facebook/folly/commit/a2792966998262d7aa2e64b5390b56ddf8ff12f4) added functionality to call a max latency callback using the undamped loop time (instead of the exponentially smoothed loop time). However, that diff used the total loop time to decide when to call the callback instead of the busy time. This means that if the event base is idle on an epoll, meaning it is waiting for work to do, then we'll end up calling the max latency callback. This doesn't make a lot of sense since what we really care about is the event base taking longer than expected to do one iteration of work. From my reading of the damped loop time callback, it looks like it's using the busy time, so this brings the undamped version in line with the damped version.
      
      Proxygen is currently the only consumer of the undamped max latency time: https://fburl.com/code/caheer2t, so this shouldn't change anyone else's counters.
      
      NOTE: the busy time doesn't seem to include the time for the [before loop callbacks](https://fburl.com/code/zsntr8lv) which I think is a mistake. I can address this in a following diff if others agree that this is a mistake.
      
      Reviewed By: jalopezsilva
      
      Differential Revision: D31905279
      
      fbshipit-source-id: 928add6243822a66ee7fa9cc0cbd93a0078ea0e9
      cd49bf89
    • Maged Michael's avatar
      hazard pointers: Optimize allocation of hazard pointers from domain · 0e92d3c2
      Maged Michael authored
      Summary:
      Optimize the management of available hazard pointers in the domain that are not in thread caches, by adding a linked list of available hazard pointers with a lock bit packed with the pointer to the head of the list. Pop operations are done in two atomic steps: (1) set lock bit, (2) pop hazard pointers and clear lock bit. Push operations are done in one atomic step, but can proceed only when the lock bit is clear.
      
      Microbenchmark results (in a process with 10K+ hazard pointers) show reduction in the latency of constructing destroying two 9-hazard-pointer-arrays that involve one TC hit and one TC miss/overflow from hundreds of microseconds to tens of nanoseconds.
      
      Before:
      ```
      1/1000 TC hit + miss & overflow                   550 ns    502 ns    468 ns
      ```
      
      After:
      ```
      TC hit + miss & overflow                           49 ns     48 ns     48 ns
      ```
      
      Reviewed By: yfeldblum
      
      Differential Revision: D31102053
      
      fbshipit-source-id: ae923bf3b05676e7572cca35179531e947846300
      0e92d3c2