1. 08 Oct, 2020 1 commit
    • Mingtao Yang's avatar
      Fix SSLCertificateIdentityVerifierTest for OpenSSL 1.1.1h · ad46f381
      Mingtao Yang authored
      Summary:
      This test was written in a way that relied on an internal OpenSSL implementation
      detail -- that OpenSSL would invoke the `handshakeVer` callback on the
      root certificate.
      
      OpenSSL commit https://github.com/openssl/openssl/commit/e2590c3a162eb118c36b09c2168164283aa099b4, which
      is part of OpenSSL 1.1.1h, alters the control flow of the X509_verify routine
      such that the handshakeVer callback is no longer called on self-signed
      certificates in the trust store (aka CA certificates).
      
      The purpose of this test was to ensure that a forced failed verification on the
      end entity certificate would elicit a particular behavior. This diff adjusts
      the implementation to match the original intention and removes the reliance
      on implementation detail specifics.
      
      Differential Revision: D24183002
      
      fbshipit-source-id: abc8337f76d3529966d276cae2337ad136456199
      ad46f381
  2. 07 Oct, 2020 2 commits
    • Lewis Baker's avatar
      Add data-structures for representing async stack-traces · c03ff623
      Lewis Baker authored
      Summary:
      Defines new data-structures folly::AsyncStackFrame and folly::AsyncStackRoot
      that can be used to build intrusive data-structures that represent
      asynchronous call-chains.
      
      These data-structures can be walked at either runtime from within the process
      or by external tooling, such as debuggers or profilers, allowing them to capture
      stack-traces that correctly attribute the calls to asynchonous callbacks to the
      asynchronous caller that initiaded the operation rather than to the immediate
      caller of the callback (typically an executor's event-loop).
      
      These data-structures are initially intended to be used by folly::coro coroutines
      to allow chains of these coroutines to be walked, but should also general enough
      to be applied to other types of asynchronous operations, such as futures or even
      asynchronous operations from other languages (eg. Rust).
      
      Reviewed By: andriigrynenko
      
      Differential Revision: D21118885
      
      fbshipit-source-id: 0d6130b40fd04ef330800eae4feb93bb79f48105
      c03ff623
    • Xiaoting Tang's avatar
      Use a beta version of libbpf to unblock perf_buffer__buffer API · 59aad4e9
      Xiaoting Tang authored
      Summary: It turns out that perf_buffer API is only available in libbpf 0.2.0, which hasn't been released yet. To unblock katran, I grabbed the latest libbpf revision and created a temporary manifest. To avoid relying on this "beta" version libbpf on other projects only katran uses it. I'll make sure to remove it in favor of the official libbpf 0.2.0 once it's released.
      
      Reviewed By: anakryiko
      
      Differential Revision: D24156655
      
      fbshipit-source-id: 32f6e04079a862fbfe96fd5475678cfa4ae1b3db
      59aad4e9
  3. 06 Oct, 2020 5 commits
    • Rodrigo Valle's avatar
      Add support for structured binding to folly::enumerate · 891965f5
      Rodrigo Valle authored
      Summary:
      Adding support for de-structuring the Proxy object inside a for loop, this will let users write code that looks like the following while still supporting the existing interface.
      
      ```
      for (const auto& [index, element] : folly::enumerate(collection)) {
          std::cout << index << ": " << element;
      }
      ```
      
      Also: update an existing unittest for c++17
      
      Please let me know if there's anything I'm failing to consider/you disagree with in this change and I'll do my best to make it right.
      
      Reviewed By: yfeldblum, ot
      
      Differential Revision: D23948526
      
      fbshipit-source-id: 6da83c9e9e0e911e33c6febc08ad2fd5b06498eb
      891965f5
    • Kevin Vigor's avatar
      fix raciness in AsyncBase::submit · 4d450003
      Kevin Vigor authored
      Summary:
      AsyncBase::submit() is racy: it calls submitOne() and then later calls op->start(). But
      op can be completed between return from submit and call to start(). In this case a CHECK
      will fail somewhere, either because we see op as COMPLETED in start(), or because we
      see op as INITALIZED in complete().
      
      Call start() to put ops into PENDING state *before* issuing them; add "unstart" method
      to put them back into INITIALIZED state when submit fails.
      
      In passing, fix bug: if submit submits fewer ops than expected, decrement pending count
      properly.
      
      Differential Revision: D24094495
      
      fbshipit-source-id: 2508dbd7ba5d306ceba34d62471d957f6cb20ff5
      4d450003
    • Dan Melnic's avatar
      Fix for a zerocopy AsyncSocket issue · 678a8567
      Dan Melnic authored
      Summary: Fix for a zerocopy AsyncSocket issue
      
      Reviewed By: kevin-vigor
      
      Differential Revision: D24083612
      
      fbshipit-source-id: 607c3daac882c6515bc9d031a84cd3efb8736177
      678a8567
    • Xiaoting Tang's avatar
      Use latest libbpf · e3933c88
      Xiaoting Tang authored
      Summary: Time to update libbpf version (the latest release is now 0.1.1).
      
      Reviewed By: udippant
      
      Differential Revision: D24063680
      
      fbshipit-source-id: 715ac74e9671f0f8ed5b8fe9174fe4070fc0f991
      e3933c88
    • Dan Melnic's avatar
      Add flags to set the io_uring mlock rlimit · d6ed060e
      Dan Melnic authored
      Summary: Add flags to set the io_uring mlock rlimit
      
      Reviewed By: danobi, kevin-vigor
      
      Differential Revision: D24063687
      
      fbshipit-source-id: be7cdad10797c087a21370ce42811a9752e4f61a
      d6ed060e
  4. 05 Oct, 2020 2 commits
    • Kevin Vigor's avatar
      fix SimpleAsyncIO build for fboss. · 9151f52b
      Kevin Vigor authored
      Summary:
      Evidently unlike folly, headers may be present in cmake even if they are excluded from
      build.
      
      Check that we have required headers before including AsyncIO.h or IoUring.h.
      
      Differential Revision: D24117788
      
      fbshipit-source-id: fec359c8298a21575a8714dd2abdbd9d4dc3e5c4
      9151f52b
    • TJ Yin's avatar
      add missing include to fix build failure · 2a60d871
      TJ Yin authored
      Reviewed By: jmswen
      
      Differential Revision: D24102346
      
      fbshipit-source-id: 9bb59bbce04285f93e9d6bb03e04841eaf10a882
      2a60d871
  5. 03 Oct, 2020 1 commit
  6. 02 Oct, 2020 4 commits
    • Yedidya Feldblum's avatar
      Backport more of is_detected · 7a473ac6
      Yedidya Feldblum authored
      Summary: [Folly] Backport more of `is_detected`.
      
      Reviewed By: Mizuchi
      
      Differential Revision: D23984149
      
      fbshipit-source-id: 1e1ed3b0cf3e54999ac56aae11e7d5f0288cae4e
      7a473ac6
    • Xiaofei Du's avatar
      Fix a small bug in IoUringBackendTest · 6e905224
      Xiaofei Du authored
      Summary: small bug
      
      Differential Revision: D24080766
      
      fbshipit-source-id: bb03739014023aa98b55859d433f88149529e5e7
      6e905224
    • Chad Austin's avatar
      enable FOLLY_USE_SYMBOLIZER on all non-Windows builds · 16d1d55c
      Chad Austin authored
      Summary: The only thing FOLLY_USE_SYMBOLIZER guards now is whether to install signal handlers, and we can do that on any unix.
      
      Reviewed By: yfeldblum, luciang
      
      Differential Revision: D23881199
      
      fbshipit-source-id: 45631b59fe5df9656a227b68683f23d4ff65e223
      16d1d55c
    • Andrew Huang's avatar
      Add API for setting OpenSSL sessions in AsyncSSLSocket · 9773cb4a
      Andrew Huang authored
      Summary: Add API set (but not get) an OpenSSL session. This is intended to handle the situation where the user already has a TLS1.3 resumable session. For example, this currently occurs when the user obtains a session through wangle's SSLSessionCallbacks.
      
      Reviewed By: mingtaoy
      
      Differential Revision: D22285536
      
      fbshipit-source-id: 7ad8be501909dbb80ab74624991415e74441ef6e
      9773cb4a
  7. 01 Oct, 2020 7 commits
    • Andrew Huang's avatar
      Migrate to new SSLSession API in NetworkObserverSessionInfoCallback · aa990652
      Andrew Huang authored
      Summary: See title
      
      Reviewed By: yfeldblum
      
      Differential Revision: D22046592
      
      fbshipit-source-id: 289828fbecb83e086d75156a8696541dd3f7f194
      aa990652
    • Jon Janzen's avatar
      Remove --skip-project-specific flag · 97498b1a
      Jon Janzen authored
      Summary: This doesn't do anything anymore and is going to be removed in D23993306, let's remove it here first.
      
      Reviewed By: yns88
      
      Differential Revision: D23993954
      
      fbshipit-source-id: 4d7dd5f992e34be7a0da16ce7cf59810407649c4
      97498b1a
    • Luca Niccolini's avatar
      Disable tests on windows · 17f5cd91
      Luca Niccolini authored
      Summary: hitting https://github.com/google/googletest/issues/1373
      
      Reviewed By: mjoras
      
      Differential Revision: D23785392
      
      fbshipit-source-id: 114849ed966fb196bb8392bd71ee3c2972834279
      17f5cd91
    • Maged Michael's avatar
      hazptr: Move reclamation outside tagged list critical section · 3e6b4527
      Maged Michael authored
      Summary:
      Move the reclamation of tagged objects outside the critical section on the tagged list lock (in asynchronous reclamation of the domain tagged list).
      
      Tagged objects are subject to both synchronous and asynchronous reclamation. Each tagged object belongs to a cohort.
      
      Currently, asynchronous reclamation of the domain list of tagged objects reclaims tagged objects while holding the lock on the list, so that synchronous reclamation of a cohort's objects does not miss such objects.
      
      This diff adds a singly linked list of objects that are safe to reclaim to the cohort structure. Asynchronous reclamation pushes reclaimable tagged objects into the safe lists in their respective cohorts, instead of reclaiming such objects while holding the lock on the list of tagged objects. By pushing objects to their respective cohorts, synchronous reclamation can find not-yet-reclaimed objects in the cohort's own safe list.
      
      Reviewed By: davidtgoldblatt
      
      Differential Revision: D23973107
      
      fbshipit-source-id: 2a9758cc8b03bb98a45b5579bab2964d9ae3d60c
      3e6b4527
    • Laurent Stacul's avatar
      Fix find file 'Findfmt' (#1462) · 676fa1ed
      Laurent Stacul authored
      Summary:
      Hello,
      
      It seems I did a mistake in PR https://github.com/facebook/folly/issues/1328
      
      CMake find_package directive looks for Find<PackageName>.cmake.
      There is a mistake in the folly-deps.cmake leading to:
      
      CMake Error at CMake/folly-deps.cmake:248 (find_package):
        No "Findfmt.cmake" found in CMAKE_MODULE_PATH.
      
      Pull Request resolved: https://github.com/facebook/folly/pull/1462
      
      Reviewed By: yfeldblum
      
      Differential Revision: D24000731
      
      Pulled By: Orvid
      
      fbshipit-source-id: 2239750228f134776b57c794d04f65f69a92ed3a
      676fa1ed
    • Laurent Stacul's avatar
      Don't pollute CMAKE_REQUIRED_* variable (#1463) · 648bc92b
      Laurent Stacul authored
      Summary:
      Hello,
      
      Adding data in those variables can have an impact upon feature detections.
      
      For instance, using 'check_cxx_source_compiles' to detect libatomic can fail because the link command uses -rdynamic and the list of shared objects. But as libcrypto needs zlib which is not added in CMAKE_REQUIRED_LIBRARIES, the test will fail, as we can see here (from CMakeError.log, modified for readability):
      
      ```
      $ g++  -DNDEBUG -O3 -flto -ffat-lto-objects -fuse-linker-plugin -std=gnu++20 -fno-working-directory -ggdb3 -Wno-error=maybe-uninitialized -Wno-error=shadow=compatible-local -Wno-error=array-bounds -Wno-error=stringop-overflow -DFOLLY_CPP_ATOMIC_BUILTIN  -std=gnu++20  -DNDEBUG -O3 -Wl,-flto -ffat-lto-objects -fuse-linker-plugin -fno-working-directory -ggdb3 -L /path/to/mylibs  -pthread -lgflags -lglog -levent -levent_core -ldouble-conversion   -rdynamic CMakeFiles/cmTC_4a1a2.dir/src.cxx.o  -o cmTC_4a1a2  /path/to/mylibs/libgflags.so  /path/to/mylibs/libssl.so  /path/to/mylibs/libcrypto.so  -ldl
      ld: warning: libz.so.1, needed by /path/to/mylibs/libssl.so, not found (try using -rpath or -rpath-link)
      ld: /path/to/mylibs/libcrypto.so: undefined reference to `deflateInit_'
      ld: /path/to/mylibs/libcrypto.so: undefined reference to `deflate'
      ld: /path/to/mylibs/libcrypto.so: undefined reference to `deflateEnd'
      ld: /path/to/mylibs/libcrypto.so: undefined reference to `inflate'
      ld: /path/to/mylibs/libcrypto.so: undefined reference to `inflateInit_'
      ld: /path/to/mylibs/libcrypto.so: undefined reference to `inflateEnd'
      ld: /path/to/mylibs/libcrypto.so: undefined reference to `zError'
      ```
      I only give the example of the detection of libatomic but this error prevents the probing mechanism from detecting other features like the presence of ASN1_TIME_diff function or elf.h.
      
      I propose to avoid setting things in those variables or, if it needs to be modified, clean it up after it was useful.
      
      Pull Request resolved: https://github.com/facebook/folly/pull/1463
      
      Reviewed By: yfeldblum
      
      Differential Revision: D24000700
      
      Pulled By: Orvid
      
      fbshipit-source-id: ace1b5b5dee6362c4a30ab0a986984005e7dc569
      648bc92b
    • Dave Marchevsky's avatar
      Add cplus_demangle_v3 wrapper to Demangle.cpp · c668b205
      Dave Marchevsky authored
      Summary: This file contains a wrapper that is protected from `basename` fn name collision between `libiberty.h` and `libgen.h` (0fb6287719abb3cf52d7eda457efbe567985d1d4). I ran into this same issue again when trying to use `libiberty`'s `cplus_demangle_v3`, so let's add a similar wrapper for this function.
      
      Reviewed By: fjricci
      
      Differential Revision: D24008824
      
      fbshipit-source-id: 868e90579f753f0c5b62aaeb27af2ef0a2006155
      c668b205
  8. 30 Sep, 2020 3 commits
    • Kevin Vigor's avatar
      Add simple wrapper for AsyncIO that hides all the grunge. · 42b8f101
      Kevin Vigor authored
      Summary:
      Add SimpleAsyncIO class. This creates an instance of an AsyncIO
      and manages a completion thread and a pool of ops, so user does
      not have to worry about any of these details.
      
      Differential Revision: D23945993
      
      fbshipit-source-id: 90f9b83ca3447a19e3bc4aaf2a2f196ee4ea614d
      42b8f101
    • Genevieve Helsel's avatar
      fix use before assignment case · 19119237
      Genevieve Helsel authored
      Summary: If retries is 0, then `0 < 0` is false, meaning we will skip the `while` loop completely and just try to read retcode, which was never assigned.
      
      Reviewed By: fanzeyi
      
      Differential Revision: D23999523
      
      fbshipit-source-id: fac4a1104eba3585fb52fc8d83163cb1a87b8fee
      19119237
    • Jon Maltiel Swenson's avatar
      Add utility to make an observer with jittered updates · 45d40269
      Jon Maltiel Swenson authored
      Summary: This diff adds a utility `withJitter` that creates an observer proxying updates from the input observer, with the difference that each update will be propagated with a random lag.
      
      Reviewed By: andriigrynenko
      
      Differential Revision: D23795834
      
      fbshipit-source-id: cdaee7460c3c346e70f6c8c78f18467fe8cd4e47
      45d40269
  9. 29 Sep, 2020 4 commits
    • Dan Melnic's avatar
      Use separate b_, e_ range members · 588857cb
      Dan Melnic authored
      Summary: Use separate b_, e_ range members
      
      Reviewed By: yfeldblum
      
      Differential Revision: D23983016
      
      fbshipit-source-id: b17061d7c1022c4776a8a8cb76ad5d0b9da9ace2
      588857cb
    • Yedidya Feldblum's avatar
      A few tweaks to SFINAE in Function · 1fbecf3f
      Yedidya Feldblum authored
      Summary: [Folly] A few simplifying tweaks to SFINAE in `Function`.
      
      Reviewed By: aary
      
      Differential Revision: D23798698
      
      fbshipit-source-id: e5a3ecfe755f6eb371c253cfb925a23d4f5d29e4
      1fbecf3f
    • Wojciech Lis's avatar
      undef signal_set in folly/portability/Event.h · a34629de
      Wojciech Lis authored
      Summary:
      In the sync from fbsource to ovrsource, we found that signal_set macro from libevent
      has a conflict with boost::asio::signal_set function in boost_context.
      
      e.g. failure https://www.internalfb.com/intern/sandcastle/job/18014398839678371/
      
      This undef removes the build failures on the ovrsource side.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D23941679
      
      fbshipit-source-id: 25e208bdbc93f289d787ca742623fd6815863705
      a34629de
    • Koby Kahane's avatar
      Do not attempt to use __rdtsc on non-x64/x86 MSVC targets (#1461) · 6c4cef83
      Koby Kahane authored
      Summary:
      Currently, Hardware.h assumes the `__rdtsc` intrinsic is always available when building with MSVC. However, this is not the case when building for Windows for ARM64.
      
      This blocks successful compilation of https://github.com/facebook/rocksdb on Windows for ARM64, since that library embeds Folly and includes this specific header.
      
      Instead, use a condition similar to the check in `asm_volatile_pause` in Asm.h. This results in successful compilation and a fallback to `steady_clock::now()` on Windows for ARM64 targets.
      
      Pull Request resolved: https://github.com/facebook/folly/pull/1461
      
      Reviewed By: yfeldblum
      
      Differential Revision: D23952622
      
      Pulled By: Orvid
      
      fbshipit-source-id: 1f02d04cbbbde42c80ad56d8d7ccc4dd7fe86847
      6c4cef83
  10. 28 Sep, 2020 3 commits
    • Zeyi (Rice) Fan's avatar
      build zstd in static to avoid DLL dependency on Windows · a1ba5275
      Zeyi (Rice) Fan authored
      Summary: It seems after updating zstd to 1.4.5. `Dllexport` for zstd.dll was not being picked up correctly. Instead of having zstd being a runtime dependency let's try statically link it to avoid the DLL issue.
      
      Reviewed By: vitaut
      
      Differential Revision: D23970349
      
      fbshipit-source-id: 3b14dddb64d410cb9546c416f27d73b7604b21ba
      a1ba5275
    • Maged Michael's avatar
      hazptr_obj_cohort: Fix race condition · b29f0eb3
      Maged Michael authored
      Summary:
      Fix race condition in hazptr_obj_cohort deactivation
      
      Race condition:
      - Thread T1 is doing asynchronous reclamation of objects.
      - Thread T2 starts deactivating cohort C.
      - T1 reclaims an object that its destructor retires a tagged object A that belongs to cohort C.
      - T1 checks that C is active.
      - T2 sets C.active_ to false.
      - T2 pops and reclaims the objects in C's list.
      - T1 pushes A in C's list
      - T2 expects to find C's list empty but it is not. Assertion failure.
      
      With the fix.
      - Thread T1 is doing asynchronous reclamation of objects.
      - Thread T2 starts deactivating cohort C.
      - T1 reclaims an object that its destructor retires a tagged object A that belongs to cohort C.
      - T1 checks that C is active.
      - T2 sets C.active_ to false.
      - **T2 waits for T1 to release the lock on the domain tagged list**
      - T1 pushes A in C's list
      - **T1 releases the lock**
      - **T2 cleans up C's objects in the domain tagged list**
      - **No subsequent asynchronous reclamation of objects in the domain will involve objects that belong to C**
      - T2 pops and reclaims the objects in C's list **including A**
      - T2 finds C's list empty as expected.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D23947490
      
      fbshipit-source-id: 943664e61190da7b4ad1bb31ac47609bdea2314d
      b29f0eb3
    • Richard Tynan's avatar
      Fix typo in coments for SharedMutex · 005bacc2
      Richard Tynan authored
      Summary: Simple typo that I spotted.
      
      Differential Revision: D23963553
      
      fbshipit-source-id: 018e81c4878f4cb27b6676e88278d47d38706ed4
      005bacc2
  11. 27 Sep, 2020 1 commit
    • Andrii Grynenko's avatar
      Add upcasting support · 15906ea2
      Andrii Grynenko authored
      Summary: Store a raw pointer to the managed object outside of the ReadMostlySharedPtrCore to make casting easier.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D23920580
      
      fbshipit-source-id: 4d3d3423f4ef3a78ebbd5efd45bb365ca0f72a18
      15906ea2
  12. 26 Sep, 2020 2 commits
  13. 25 Sep, 2020 5 commits
    • Dan Melnic's avatar
      Skip deadlocking fork tests in TSAN mode · d586f9f8
      Dan Melnic authored
      Summary: Skip deadlocking fork tests in TSAN mode
      
      Differential Revision: D23933183
      
      fbshipit-source-id: a5fe2da954676f6ced2f00dea280e10ac88d99cd
      d586f9f8
    • Dan Melnic's avatar
      Add support for immediate SQ poll submit · aeea7370
      Dan Melnic authored
      Summary: Add support for immediate SQ poll submit
      
      Reviewed By: kevin-vigor
      
      Differential Revision: D23818098
      
      fbshipit-source-id: d8d626c51aef0cf8c0049e081d86313cb731429a
      aeea7370
    • Robin Cheng's avatar
      Make SymbolizerTest pass if compiled as PIE. · 0f0fe916
      Robin Cheng authored
      Summary:
      The symbolizer takes mapped addresses (address listed in ELF, offset by the binary offset (which is zero for non-PIE)) and symbolizes it into non-mapped addresses. In a couple of places in SymbolizerTest, we took the non-mapped addresses and symbolized them again, which wouldn't work if the binary was compiled with PIE. In another place in SymbolizerTest, the mapped address was used to look up DWARF information, whereas we needed the non-mapped address. This diff addresses these issues.
      
      Additionally, I discovered that r_debug::r_map->l_addr is the "right" way to get the PIE offset, so this diff pulls that common functionality to symbolizer/detail/Debug.h and use it for Symbolizer and the three tests so far that need PIE relocation adjustments.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D23492850
      
      fbshipit-source-id: 62353e576c50b44070b323b5477fea9bb4c0b500
      0f0fe916
    • Robin Cheng's avatar
      Suppress another TSAN false positive due to openssl not compiled with TSAN instrumentation. · e653781c
      Robin Cheng authored
      Summary: Similar to D23139413, this is another place where openssl causes TSAN issues widely in fbcode.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D23698225
      
      fbshipit-source-id: e883784500392c2d5be757c37cceb8bf8cc8b2aa
      e653781c
    • Chad Austin's avatar
      use backtrace(2) on all Apple platforms · 1d2bff8d
      Chad Austin authored
      Summary: I don't recall what previous intermediate state justified guarding use of backtrace with __APPLE__ and FOLLY_USE_SYMBOLIZER, but it no longer applies.
      
      Reviewed By: yfeldblum, luciang
      
      Differential Revision: D23881192
      
      fbshipit-source-id: b3ca829f9c738f50f7ab7e038348ec3df65ae4b9
      1d2bff8d