1. 16 Oct, 2020 5 commits
    • Xavier Deguillard's avatar
      HazptrDomain: fix MSVC truncation warning · 2b5dedea
      Xavier Deguillard authored
      Summary:
      The MSVC compiler complains about truncation of size_t to int:
      
        Z:\installed\folly\include\folly/synchronization/HazptrDomain.h(210): warning C4267: 'argument': conversion from 'size_t' to 'int', possible loss of data
      
      MSVC being very verbose, this is followed by 25 lines of template expansion.
      
      Since `shard` is a size_t due to `calc_shard` returning that type, let's simply
      make the shard argument of hazptr_warning_list_too_large a size_t too to
      silence the warning.
      
      Reviewed By: magedm
      
      Differential Revision: D24365556
      
      fbshipit-source-id: b426cee7caaf593eb112691bc45b83e564342b0e
      2b5dedea
    • Lukas Piatkowski's avatar
      mononoke/integration: create a Makefile to run tests as part of getdeps.py build (#67) · 3f6324de
      Lukas Piatkowski authored
      Summary:
      Pull Request resolved: https://github.com/facebookexperimental/eden/pull/67
      
      With this change it will be possible to build dependencies of and run integration tests using getdeps.py.
      
      This is the first goal of Q4 as per https://fb.quip.com/v8YzAYNSYgot: "Get Open Source version of integration tests running on Legocastle".
      
      Before this diff:
      The OSS integration tests run now on GitHub by:
      - Building some test dependencies with getdeps.py
      - Building some test dependencies with homebrew/apt-get
      - Running tests via python script
      
      The OSS integration tests were not running on Sandcastle.
      
      After this diff:
      The OSS integration tests run on Github by:
      - Building and executing tests via getdeps.py (execution of tests happens by getdeps.py calling Make calling python script)
      
      The OSS integration tests run on Sandcastle using the same getdeps.py setup as Github.
      
      Reviewed By: krallin
      
      Differential Revision: D24253268
      
      fbshipit-source-id: cae249b72d076222673b8bbe4ec21866dcdbb253
      3f6324de
    • Christopher Kuklewicz's avatar
      Improve documentation comments on EventBaseManager · b26907ea
      Christopher Kuklewicz authored
      Summary:
      # What is the goal?
      
      Address the problem in the post at https://fb.workplace.com/groups/code.indexing/permalink/1929080303917865/ with the comment(s) in Codex for EventBaseManager.
      
      # What was not the way?
      
      Jie Jua noticed that the current comment on "EventBaseManager() {}" in Codex was
      
      ```
      encouraged. You should instead use the global singleton if possible.
      ```
      
      # What is the new way?
      
      Use documentation comments (doxygen) style `/** double starts */` instead of `//` ordinary comments. This should show the full comment which is:
      
      ```
      XXX Constructing a EventBaseManager directly is DEPRECATED and not
      encouraged. You should instead use the global singleton if possible.
      ```
      
      Which really needs that first line.
      
      Differential Revision: D24333522
      
      fbshipit-source-id: 8e84e0a49426844d5a6f6438b0d76fb9886742fe
      b26907ea
    • Orvid King's avatar
      Use the new name for await in the fibers test · 53947e04
      Orvid King authored
      Summary: The bare name isn't defined on windows at all, so we have to use the new internal name instead.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D24044971
      
      fbshipit-source-id: 24b37903ed0f309910d9f5141240b951bf5fba15
      53947e04
    • Alfred Fuller's avatar
      Move clang-format-next changes into clang-format-canary · 0deef031
      Alfred Fuller authored
      Reviewed By: yfeldblum
      
      Differential Revision: D23545773
      
      fbshipit-source-id: e93f84d0658166e875710a4c0fb5dc165a8628d8
      0deef031
  2. 15 Oct, 2020 9 commits
    • Andrew Huang's avatar
      Migrate AsyncSSLSocketTest to V2 Session API · 3fa19ab2
      Andrew Huang authored
      Summary: Migrate AsyncSSLSocketTest to new AsyncSSLSocket session API
      
      Reviewed By: mingtaoy
      
      Differential Revision: D24240249
      
      fbshipit-source-id: ea9bd7669f377c738e44d3348400a2cfbd3c2580
      3fa19ab2
    • Robin Cheng's avatar
      Fix livelock in AtForkTest when running under TSAN. · 08ee18c5
      Robin Cheng authored
      Summary: Use the forkInstrumented call when running under TSAN.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D23787959
      
      fbshipit-source-id: 159f8354ef9746b8374d6da12b22e389c2bc71f4
      08ee18c5
    • Yedidya Feldblum's avatar
      Fix is_constexpr_default_constructible under clang-10 · 068d3f6e
      Yedidya Feldblum authored
      Summary: [Folly] Fix `is_constexpr_default_constructible` under clang-10, which suffers a crash (https://bugs.llvm.org/show_bug.cgi?id=47620) and divergent behavior in certain constexpr evaluations.
      
      Reviewed By: mpark, Mizuchi
      
      Differential Revision: D24302661
      
      fbshipit-source-id: b6a58b5b675f367bf2dc1d810edb9926cfb87a2e
      068d3f6e
    • Dan Melnic's avatar
      TSAN fix for folly/io/test:shutdown_socket_set_test · 7c4ac999
      Dan Melnic authored
      Summary: TSAN fix for folly/io/test:shutdown_socket_set_test
      
      Differential Revision: D24321155
      
      fbshipit-source-id: 753a808c868da4c608c3ee95d16a3673d6f700b0
      7c4ac999
    • Dan Melnic's avatar
      Reduce io_uring test mem requirements · af60f10a
      Dan Melnic authored
      Summary: Reduce io_uring test mem requirements
      
      Reviewed By: xiaofeidu008, danobi
      
      Differential Revision: D24314788
      
      fbshipit-source-id: 35ec01475fbc1cf851a81e33cf8c7fec8dd88ddb
      af60f10a
    • Andrii Grynenko's avatar
      Unwrap Observer in SimpleObservable cache · fbfab7ff
      Andrii Grynenko authored
      Summary: Cache unwrapped observer instead of unwrapping it on every getObserver call.
      
      Reviewed By: jmswen
      
      Differential Revision: D24325934
      
      fbshipit-source-id: d8a023e59c933222a5f43aaacbd9c955236739b3
      fbfab7ff
    • Maged Michael's avatar
      ConcurrentHashMap: Remove comment about potential bottleneck under heavy removal · 2bf4a236
      Maged Michael authored
      Summary:
      Remove comment about potential high contention under sustained frequent removal from CHM-s.
      
      Recent changes to the hazard pointer library eliminated this problem:
      - Asynchronous reclamation of tagged objects (including CHM objects) no longer needs to be done while holding the lock on the list of tagged objects.
      - The list of tagged objects is now sharded and sharding can be easily increased if needed.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D24318407
      
      fbshipit-source-id: 3124e8d5651848a16084899a6f31d69cc235c702
      2bf4a236
    • John Reese's avatar
      apply black 20.8b1 formatting update · 68c04406
      John Reese authored
      Summary:
      allow-large-files
      
      black_any_style
      
      Reviewed By: zertosh
      
      Differential Revision: D24325133
      
      fbshipit-source-id: b4afe80d1e8b2bc993f4b8e3822c02964df47462
      68c04406
    • Robin Cheng's avatar
      Relax the limits for StackTraceSizeLimitTest. · 7cc28784
      Robin Cheng authored
      Summary: Apparently it's flaky...
      
      Differential Revision: D24179337
      
      fbshipit-source-id: 75416adaf8dbd37fc9ddeaa0500ddfaeb80cffa9
      7cc28784
  3. 14 Oct, 2020 4 commits
    • Yedidya Feldblum's avatar
      Cut linter suppression in TryTest.cpp · 98014f8b
      Yedidya Feldblum authored
      Summary: [Folly] Cut linter suppression in `TryTest.cpp`.
      
      Reviewed By: zertosh
      
      Differential Revision: D24294536
      
      fbshipit-source-id: 64b26ec4632ffdae8e246aa27a6c58c579e2db10
      98014f8b
    • Dan Melnic's avatar
      Move folly binary · 3f1e1ee8
      Dan Melnic authored
      Summary: Move folly binary
      
      Reviewed By: yfeldblum
      
      Differential Revision: D24294709
      
      fbshipit-source-id: e7cf0be6e45d3a132fc81143103f3b5cdedbb76f
      3f1e1ee8
    • Xiaofei Du's avatar
      Conserve resource for IoUringBackendTest to avoid rlimit issue · ef6f7573
      Xiaofei Du authored
      Summary:
      This is best effort. We cannot control how many tests are running
      at the same time. And if the user is running other tests that consumes resource
      
      Differential Revision: D24289056
      
      fbshipit-source-id: 0810ba0989e84a5bce6d695dd6df0dfe2618fbf7
      ef6f7573
    • Francesco Zoffoli's avatar
      CoReturn can return implicitly convertible values · bf475c74
      Francesco Zoffoli authored
      Summary:
      Gmock's `Return(x)` is a valid action as long as `x` is implicitly convertible to the return type of the function.
      
      The current implementation of `CoReturn(x)` does not support that, since `Task<T>` is not convertible to `Task<Q>` even if `Q` is convertible to `T`.
      
      This diff changes the `CoReturn` implementatio to match the way Gmock does, allowing for the implicit conversion.
      
      This is extremely valuable when using `Task<Expected<...>>`, as it allows to write `CoReturn(makeUnexpected(...))` instead of `CoReturn(Expected<...>(unexpected, ...))`.
      
      It also make the translation from a mock being sync to async more straightforward: add `Task` in the mock return type and replace `Return` with `CoReturn`.
      
      Reviewed By: lewissbaker
      
      Differential Revision: D24046013
      
      fbshipit-source-id: 9f166237a624600be51c00616f145ae4e1a45443
      bf475c74
  4. 13 Oct, 2020 2 commits
    • Lukas Piatkowski's avatar
      mononoke/integration tests: prepare dependencies for running tests by getdeps · 22ed347d
      Lukas Piatkowski authored
      Summary:
      This diff adds all third party dependencies that are required by getdeps to be able to build and runn Mononoke's integration tests.
      
      Also add a stub Makefile with no-op steps that will be filled in next diff.
      
      Reviewed By: ahornby
      
      Differential Revision: D24251894
      
      fbshipit-source-id: 67384ecfd0ced6762dddc3c6e61feb1240b1162d
      22ed347d
    • Andrii Grynenko's avatar
      Fix Core::hasCallback to return true for State::Empty · 8eba7be7
      Andrii Grynenko authored
      Summary:
      This specifically fixes a TSAN detected race where SemiFuture destructor was checking Core state to see if deferred executor has to be detached, while that state could become Empty if the callback is being proxied and both threads code be racing while accessing the executor.
      hasCallback() checks if the callback was ever set (not if the callback is stil alive) and it's true for State::Empty (similar to State::Done).
      
      Differential Revision: D24271346
      
      fbshipit-source-id: 5f52f20783d36745d1200497e0300bc086b65213
      8eba7be7
  5. 10 Oct, 2020 1 commit
    • Dan Melnic's avatar
      Add support for UDP RX timestamps · 1902d0ff
      Dan Melnic authored
      Summary: Add support for UDP RX timestamps
      
      Reviewed By: mjoras
      
      Differential Revision: D24146914
      
      fbshipit-source-id: 381839fc5402f13e428364c47c5752473acda6af
      1902d0ff
  6. 09 Oct, 2020 3 commits
    • Ivan Murashko's avatar
      `lint-ignore` processing for `clang-tidy` (xplat) · 343a70ae
      Ivan Murashko authored
      Summary: Disable CLANGTIDY checks for several places at the code where HOWTOEVEN checks are disabled.
      
      Reviewed By: suo
      
      Differential Revision: D24198276
      
      fbshipit-source-id: c52cd10941252fcacfea30589660c57eb7099011
      343a70ae
    • Ivan Murashko's avatar
      `lint-ignore` processing for `clang-tidy` (fbcode addons) (#1472) · 4f9a125a
      Ivan Murashko authored
      Summary:
      Pull Request resolved: https://github.com/facebook/folly/pull/1472
      
      Disable CLANGTIDY checks for several places at the code where HOWTOEVEN checks are disabled.
      
      Reviewed By: suo
      
      Differential Revision: D24197524
      
      fbshipit-source-id: e99cfe1a7440a275edd63643ad819ca9e2a07788
      4f9a125a
    • John Kearney's avatar
      Allow folly fibers to compile with windows asan · d15d9f55
      John Kearney authored
      Summary:
      - Folly FiberManager makes the assumption that asan always has dlfcn.h
         available.
         - This assumption is not true on windows.
       - This diff would mean that windows asan would not use:
         - `__sanitizer_start_switch_fiber`
         - `__sanitizer_finish_switch_fiber`
         - `__asan_unpoison_memory_region`
       - The behavior of `GetProcAddress` on windows is sufficiently different to `dlsym` to make switching to it without significant testing unappealing.
      
      (Note: this ignores all push blocking failures!)
      
      Reviewed By: andriigrynenko
      
      Differential Revision: D24167183
      
      fbshipit-source-id: 2587a4000901b5e3a8857e9aa5816270d2be34dc
      d15d9f55
  7. 08 Oct, 2020 7 commits
    • Robin Cheng's avatar
      Make ThreadCachedInts TSAN-friendly. · b984cc9d
      Robin Cheng authored
      Summary:
      ThreadCachedInts uses asymmetric barriers to heavily optimize the reader side. TSAN does not support (symmetric or asymmetric) barriers. In this particular case, this diff replaces the memory order for accessing increments and decrements with memory_order_seq_cst only when running under TSAN, and that, I think, is a valid alternative to the barriers because it ensures a total order between all atomic accesses to increments and decrements (so that, when we read all decrements before all increments, we make sure that any writes to increments are read as long as decrements are read, which seems to be the purpose of the barriers).
      
      Introduced additional tests to RcuTest and to AtomicReadMostlyMainPtrTest. The problem with existing tests is that this only triggers TSAN when the counter was already 0 when calling synchronize(), not when synchronize() had to invoke futexWait, because futexWait is itself a synchronization point.
      
      Reviewed By: davidtgoldblatt
      
      Differential Revision: D24029131
      
      fbshipit-source-id: 57c7bf32061868ccd5a4e20154c8c7db6e4eeef5
      b984cc9d
    • Lee Howes's avatar
      Add weakDetachOnGlobalCPUExecutorAfter · bb70495b
      Lee Howes authored
      Summary: Adds a form of detach that is safe to use in recursive code. Detaches a task after a delay, and checks for availability of the global executor at that point to decouple shutdown from the delayed detach.
      
      Reviewed By: andriigrynenko
      
      Differential Revision: D24174495
      
      fbshipit-source-id: e6b22485a817c18319c897fd51c3f8ba81d13041
      bb70495b
    • Maged Michael's avatar
      hazptr_domain: Add tagged list sharding · b0e23aef
      Maged Michael authored
      Summary:
      Changes:
      - Make the tagged list sharded.
      - Manage a common count for both tagged and untagged cohort objects.
      - Combine timed asynchronous reclamation for both tagged and untagged cohort objects.
      - Integrate checking the threshold for asynchronous reclamation with managing timed asynchronous reclamation.
      - Combine the asynchronous reclamation of both tagged and untagged objects.
      
      Reviewed By: davidtgoldblatt
      
      Differential Revision: D24136818
      
      fbshipit-source-id: 8df1b8eeef1df7f14b5d68ed5ad82d0459a381f5
      b0e23aef
    • Michael Suo's avatar
      Revert D24046011: `lint-ignore` processing for `clang-tidy` (fbcode addons) · 61788481
      Michael Suo authored
      Differential Revision:
      D24046011 (https://github.com/facebook/folly/commit/99dbd7aae8a57a477a1b3db2466dfe8db7157142)
      
      Original commit changeset: 91f52198242c
      
      fbshipit-source-id: 2041d1f6017fa833ff7bb8ac5c9adc2d3fe80553
      61788481
    • generatedunixname89002005325676's avatar
      Daily `arc lint --take CLANGFORMAT` · 921800d6
      generatedunixname89002005325676 authored
      Reviewed By: zertosh
      
      Differential Revision: D24188387
      
      fbshipit-source-id: a540cf6db895dda48723492aa843bf32d66011f3
      921800d6
    • Ivan Murashko's avatar
      `lint-ignore` processing for `clang-tidy` (fbcode addons) · 99dbd7aa
      Ivan Murashko authored
      Summary: Disable CLANGTIDY checks for several places at the code.
      
      Differential Revision: D24046011
      
      fbshipit-source-id: 91f52198242ceaa87e923a43a3dbe6610efe4e49
      99dbd7aa
    • Mingtao Yang's avatar
      Fix SSLCertificateIdentityVerifierTest for OpenSSL 1.1.1h · ad46f381
      Mingtao Yang authored
      Summary:
      This test was written in a way that relied on an internal OpenSSL implementation
      detail -- that OpenSSL would invoke the `handshakeVer` callback on the
      root certificate.
      
      OpenSSL commit https://github.com/openssl/openssl/commit/e2590c3a162eb118c36b09c2168164283aa099b4, which
      is part of OpenSSL 1.1.1h, alters the control flow of the X509_verify routine
      such that the handshakeVer callback is no longer called on self-signed
      certificates in the trust store (aka CA certificates).
      
      The purpose of this test was to ensure that a forced failed verification on the
      end entity certificate would elicit a particular behavior. This diff adjusts
      the implementation to match the original intention and removes the reliance
      on implementation detail specifics.
      
      Differential Revision: D24183002
      
      fbshipit-source-id: abc8337f76d3529966d276cae2337ad136456199
      ad46f381
  8. 07 Oct, 2020 2 commits
    • Lewis Baker's avatar
      Add data-structures for representing async stack-traces · c03ff623
      Lewis Baker authored
      Summary:
      Defines new data-structures folly::AsyncStackFrame and folly::AsyncStackRoot
      that can be used to build intrusive data-structures that represent
      asynchronous call-chains.
      
      These data-structures can be walked at either runtime from within the process
      or by external tooling, such as debuggers or profilers, allowing them to capture
      stack-traces that correctly attribute the calls to asynchonous callbacks to the
      asynchronous caller that initiaded the operation rather than to the immediate
      caller of the callback (typically an executor's event-loop).
      
      These data-structures are initially intended to be used by folly::coro coroutines
      to allow chains of these coroutines to be walked, but should also general enough
      to be applied to other types of asynchronous operations, such as futures or even
      asynchronous operations from other languages (eg. Rust).
      
      Reviewed By: andriigrynenko
      
      Differential Revision: D21118885
      
      fbshipit-source-id: 0d6130b40fd04ef330800eae4feb93bb79f48105
      c03ff623
    • Xiaoting Tang's avatar
      Use a beta version of libbpf to unblock perf_buffer__buffer API · 59aad4e9
      Xiaoting Tang authored
      Summary: It turns out that perf_buffer API is only available in libbpf 0.2.0, which hasn't been released yet. To unblock katran, I grabbed the latest libbpf revision and created a temporary manifest. To avoid relying on this "beta" version libbpf on other projects only katran uses it. I'll make sure to remove it in favor of the official libbpf 0.2.0 once it's released.
      
      Reviewed By: anakryiko
      
      Differential Revision: D24156655
      
      fbshipit-source-id: 32f6e04079a862fbfe96fd5475678cfa4ae1b3db
      59aad4e9
  9. 06 Oct, 2020 5 commits
    • Rodrigo Valle's avatar
      Add support for structured binding to folly::enumerate · 891965f5
      Rodrigo Valle authored
      Summary:
      Adding support for de-structuring the Proxy object inside a for loop, this will let users write code that looks like the following while still supporting the existing interface.
      
      ```
      for (const auto& [index, element] : folly::enumerate(collection)) {
          std::cout << index << ": " << element;
      }
      ```
      
      Also: update an existing unittest for c++17
      
      Please let me know if there's anything I'm failing to consider/you disagree with in this change and I'll do my best to make it right.
      
      Reviewed By: yfeldblum, ot
      
      Differential Revision: D23948526
      
      fbshipit-source-id: 6da83c9e9e0e911e33c6febc08ad2fd5b06498eb
      891965f5
    • Kevin Vigor's avatar
      fix raciness in AsyncBase::submit · 4d450003
      Kevin Vigor authored
      Summary:
      AsyncBase::submit() is racy: it calls submitOne() and then later calls op->start(). But
      op can be completed between return from submit and call to start(). In this case a CHECK
      will fail somewhere, either because we see op as COMPLETED in start(), or because we
      see op as INITALIZED in complete().
      
      Call start() to put ops into PENDING state *before* issuing them; add "unstart" method
      to put them back into INITIALIZED state when submit fails.
      
      In passing, fix bug: if submit submits fewer ops than expected, decrement pending count
      properly.
      
      Differential Revision: D24094495
      
      fbshipit-source-id: 2508dbd7ba5d306ceba34d62471d957f6cb20ff5
      4d450003
    • Dan Melnic's avatar
      Fix for a zerocopy AsyncSocket issue · 678a8567
      Dan Melnic authored
      Summary: Fix for a zerocopy AsyncSocket issue
      
      Reviewed By: kevin-vigor
      
      Differential Revision: D24083612
      
      fbshipit-source-id: 607c3daac882c6515bc9d031a84cd3efb8736177
      678a8567
    • Xiaoting Tang's avatar
      Use latest libbpf · e3933c88
      Xiaoting Tang authored
      Summary: Time to update libbpf version (the latest release is now 0.1.1).
      
      Reviewed By: udippant
      
      Differential Revision: D24063680
      
      fbshipit-source-id: 715ac74e9671f0f8ed5b8fe9174fe4070fc0f991
      e3933c88
    • Dan Melnic's avatar
      Add flags to set the io_uring mlock rlimit · d6ed060e
      Dan Melnic authored
      Summary: Add flags to set the io_uring mlock rlimit
      
      Reviewed By: danobi, kevin-vigor
      
      Differential Revision: D24063687
      
      fbshipit-source-id: be7cdad10797c087a21370ce42811a9752e4f61a
      d6ed060e
  10. 05 Oct, 2020 2 commits
    • Kevin Vigor's avatar
      fix SimpleAsyncIO build for fboss. · 9151f52b
      Kevin Vigor authored
      Summary:
      Evidently unlike folly, headers may be present in cmake even if they are excluded from
      build.
      
      Check that we have required headers before including AsyncIO.h or IoUring.h.
      
      Differential Revision: D24117788
      
      fbshipit-source-id: fec359c8298a21575a8714dd2abdbd9d4dc3e5c4
      9151f52b
    • TJ Yin's avatar
      add missing include to fix build failure · 2a60d871
      TJ Yin authored
      Reviewed By: jmswen
      
      Differential Revision: D24102346
      
      fbshipit-source-id: 9bb59bbce04285f93e9d6bb03e04841eaf10a882
      2a60d871