1. 10 Mar, 2021 1 commit
    • Yedidya Feldblum's avatar
      fix F14 fallback find() for stateful key-equal · 431c16c3
      Yedidya Feldblum authored
      Summary:
      The fallback implementation of F14 does a clever trick in its implementation of `find()` to support heterogeneous lookup. But this trick is UB and its implementation assumes that the key-equal type is empty, not specially-aligned, and not final. Relax these constraints.
      
      As it turns out, fbthrift streaming uses an F14 map with a stateful key-equal, triggering this UB in the fallback implementation: `StreamMap` uses key-equal type `StreamMapEquals`, which is nonempty and has a `StreamIdResolver` field, which itself is empty but which causes `StreamMapEquals` not to be empty. The simplest fix is, of course, to switch to using `StreamIdResolver` as a base, but this would be a bug in the F14 fallback regardless.
      
      `BottomKeyEqual` must have the same size, alignment, emptiness, and finality as `KeyEqual`.
      
      Reviewed By: aary
      
      Differential Revision: D26905527
      
      fbshipit-source-id: 023feb3963fc7b2e779fb591e737ea4b70ded476
      431c16c3
  2. 09 Mar, 2021 10 commits
    • Alex Snast's avatar
      add heterogeneous access support to EvictingCacheMap · 0efd32c4
      Alex Snast authored
      Summary:
      Heterogeneous access is described [here](https://github.com/facebook/folly/blob/master/folly/container/F14.md#heterogeneous-key-type-with-transparent-hash-and-equality) and this diff adds this capability to EvictingCacheMap.
      
      See `HeterogeneousAccess` test from EvictingCacheMapTest.cpp for usage example.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26879601
      
      fbshipit-source-id: 027352f2daa6773ab981b924732e1056550f6ea2
      0efd32c4
    • Tom Kwong's avatar
      Add StreamingStats functionality · 0fbe9c5b
      Tom Kwong authored
      Summary:
      This diff adds new capability to calculate statistics efficiently in a single pass. In particular, it uses Welford's algorithm to track an internal state such that variance and standard deviation can be obtained at O(1).
      
      The following statistics are available:
      - count
      - minimum
      - maximum
      - mean
      - sample and population variance
      - sample and population standard deviation
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26667517
      
      fbshipit-source-id: c4eb31e93646b9cff2f6516039170c8d88a5bf33
      0fbe9c5b
    • Philip Pronin's avatar
      define GENERATOR_REF · 438afd7d
      Philip Pronin authored
      Summary:
      Version of `GENERATOR` that we're already using in a few places where
      it doesn't leave its original scope.
      
      Reviewed By: ddrcoder, luciang
      
      Differential Revision: D26909115
      
      fbshipit-source-id: 5fbae8bd44c62d4d96427a4fd3ef60a8d9cb40e3
      438afd7d
    • Alan Frindell's avatar
      remove folly/io/coro/Socket.h shim · 3af4c3db
      Alan Frindell authored
      Summary: All dependencies have migrated to folly/io/coro/Transport.h
      
      Reviewed By: yairgott
      
      Differential Revision: D26837064
      
      fbshipit-source-id: abc46819e4d9bdec6ff5898d0295731781c7925d
      3af4c3db
    • Alan Frindell's avatar
      rename coro::Socket to coro::Transport · 8b4086af
      Alan Frindell authored
      Summary: This also renames the file and creates a Socket.h shim for compatabilty.
      
      Reviewed By: yairgott
      
      Differential Revision: D26837068
      
      fbshipit-source-id: 47c8f7b410540d1ad0aa03f4bc96816f8a93ef1c
      8b4086af
    • Alan Frindell's avatar
      coro::Socket wraps AsyncTransport · c3156ea8
      Alan Frindell authored
      Summary: This allows coro::Socket to be used with non-socket AsyncTransports, notably Fizz.  A subsequent diff renames the class and file, and creates a shim for callers.
      
      Reviewed By: yairgott
      
      Differential Revision: D26610473
      
      fbshipit-source-id: d64597ef0de3c90ab084249b20e85d97b34857a6
      c3156ea8
    • generatedunixname89002005325676's avatar
      Daily `arc lint --take CLANGFORMAT` · 2f7a1264
      generatedunixname89002005325676 authored
      Reviewed By: zertosh
      
      Differential Revision: D26908037
      
      fbshipit-source-id: 5c636fd281066f1d9af0d92f6f0b015ebf77437b
      2f7a1264
    • Dave Rigby's avatar
      Mark StaticMata as FOLLY_EXPORT (#1533) · 7c7531bd
      Dave Rigby authored
      Summary:
      When using folly::ThreadLocal<> to create thread-local variables, if a thread-local is set from two different libraries which are compiled with -fvisibility=hidden and using libc++, then a nullptr dereference is seen when attempting to set the thread-local from the second DLL.
      
      This is due to StaticMetaBase (which should be a singleton) getting created twice. This results in the thread local metadata getting corrupted and hence attempting to dereference a nullptr.
      
      Fix by marking StaticMeta as FOLLY_EXPORT, so only a single instance exists at runtime.
      
      Fixes https://github.com/facebook/folly/issues/1431
      
      Pull Request resolved: https://github.com/facebook/folly/pull/1533
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26818467
      
      Pulled By: Orvid
      
      fbshipit-source-id: 6b22627bc51060260e653885bdeb0df111207617
      7c7531bd
    • Yedidya Feldblum's avatar
      reduce inline code size of Executor::invokeCatchingExns · 5a37f8c8
      Yedidya Feldblum authored
      Summary: Some use-cases can be sensitive to inline code size, so reduce the inline code size of `Executor::invokeCatchingExns` just in case.
      
      Reviewed By: vitaut, zhxchen17
      
      Differential Revision: D26875990
      
      fbshipit-source-id: 0b822f7425bb47c98837faf08216092b49a7ed38
      5a37f8c8
    • Adam Simpkins's avatar
      update farmhash.cpp to avoid an unused function error on android · 4d235eee
      Adam Simpkins authored
      Summary:
      The `Fetch128()` function is only used in `#ifdef` sections that check for
      either `can_use_sse42` or `can_use_sse41 && x86_64`.  On Android builds this
      function was defined as a static function that was never used, causing
      compiler errors when compiled with `-Werror -Wunused-function`
      
      Reviewed By: mzlee
      
      Differential Revision: D26865431
      
      fbshipit-source-id: 5cd7ce12aa614f3d904df40ba21842223502a9ec
      4d235eee
  3. 08 Mar, 2021 5 commits
    • Kenny Yu's avatar
      Print async stack trace in signal handler · e85a11d5
      Kenny Yu authored
      Summary:
      If there is an active async operation on the current thread, then the signal
      handler will also print the async stack trace as well. Note that an async stack
      trace can have a mix of both non-async and async frames. This is an example
      of what an async stack trace might print:
      
      ```
      funcF      <- non async, top of stack
      funcE      <- non async
      co_funcD   <- async
      funcC      <- non async, blocking call. Second async stack root here
      co_funcB   <- async
      co_funcA   <- async
      main       <- non async. First async stack root here
      ```
      
      If there is no async operation in progress, there is no change in behavior
      (only normal stack trace is printed).
      
      This also adds a helper `getAsyncStackTraceStr()` to easily dump
      the async stack trace on demand. This helper is NOT async-signal-safe
      and is only meant for debugging purposes.
      
      Reviewed By: yfeldblum, andriigrynenko
      
      Differential Revision: D26661006
      
      fbshipit-source-id: f6e913910eb45c438a3e1e29d231e224eed15ef2
      e85a11d5
    • Seth Hinze's avatar
      Fix logging of AsyncSocket connection failed exceptions · 47667233
      Seth Hinze authored
      Summary:
      When log verbosity is sufficiently high, the logging of AsyncSocket connection exceptions triggers the following assert:
      ```
        assert(typeid(ex) == typeid(std::decay_t<Ex>) ||
             !"Dynamic and static exception types don't match. Exception would "
              "be sliced when storing in exception_wrapper.");
      ```
      Update the log to output `e.what()`, which all of the other exception logs already do.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26799863
      
      fbshipit-source-id: f9d98f81a77a654e78a153bc5c23e6ec8223fa1f
      47667233
    • Yedidya Feldblum's avatar
      use to_ascii_decimal in FOLLY_SAFE_CHECK · 9a5fc814
      Yedidya Feldblum authored
      Summary: Use `folly::to_ascii_decimal` in `FOLLY_SAFE_CHECK` and related facility implementations. This facility is intentionally async-signal-safe on its own and is light on its dependencies. Specifically, avoid the dependency on `folly/Conv.h` which is heavier.
      
      Reviewed By: luciang
      
      Differential Revision: D26597237
      
      fbshipit-source-id: e3cf0b3a3046cdbd3da89cc8a2e0627edb5a5a85
      9a5fc814
    • Cameron Pickett's avatar
      Fix CO_ASSERT_EQ for new gtest version · 32d05841
      Cameron Pickett authored
      Differential Revision: D26882407
      
      fbshipit-source-id: f7ab479aff6b8eaced65da333415efd33112ea0a
      32d05841
    • Aaryaman Sagar's avatar
      Explicitly qualify atomic_wait and atomic_notify function calls · 8477e256
      Aaryaman Sagar authored
      Summary:
      `std::atomic_wait()` and `std::atomic_notify_one()` are marked as unavailable.
      Not really sure what that means, but it seems to be breaking some open source
      builds https://github.com/facebook/folly/issues/1527.  Explicitly qualify the
      calls into those functions to try and fix the build break.
      
      Since we cannot conditionally import either of the above (because they are
      marked as unavailable), we can't rely on the standard implementations.  To
      prevent ADL from kicking in when the standard-library defines these, we fully
      qualify these and use `tag_invoke` for the customization points used in tests.
      
      Reviewed By: davidtgoldblatt
      
      Differential Revision: D26742072
      
      fbshipit-source-id: 9f44bbfd37530f5ecffa3e03d673cfb1df953299
      8477e256
  4. 07 Mar, 2021 2 commits
    • Yedidya Feldblum's avatar
      to_ascii, to_ascii_size · 89a3d4a7
      Yedidya Feldblum authored
      Summary:
      Generic ascii-ification functions `to_ascii` and `to_ascii_size` for varying unsigned integer types over varying integer bases.
      
      Has minimal dependencies. Is explicitly async-signal-safe.
      
      Reviewed By: kennyyu
      
      Differential Revision: D26593764
      
      fbshipit-source-id: b1432bee8ca8eb0a5304943c4e231e201425f14a
      89a3d4a7
    • Yedidya Feldblum's avatar
      Executor::invokeCatchingExns · 2f6126e6
      Yedidya Feldblum authored
      Summary: Executors just carry on after a task they are running fails, although they try to log something marginally useful. Deduplicate the logic.
      
      Reviewed By: aary, ot, philippv
      
      Differential Revision: D26745058
      
      fbshipit-source-id: 90fa558c89591b8a2d08e6820388970dee8811be
      2f6126e6
  5. 06 Mar, 2021 3 commits
    • Alfred Fuller's avatar
      Update include categories in clang-format canary · 688c8aaa
      Alfred Fuller authored
      Summary:
      This just fixes the include order within groups of includes, so is significantly less aggressive than the other codemod which regroups includes.
      
      I've manually ensured that:
      - groups are split when trivial.
      - glog is not reordered relative to other folly headers (seems to cause down stream issues)
      - portability headers are not reordered (seems to cause down stream issues)
      other diffs will finish the regrouping and ordering of those groups.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26405757
      
      fbshipit-source-id: 9b79a91a41a4e6ac677ed0cb7fb24dfb7c8093d2
      688c8aaa
    • Andrii Grynenko's avatar
      InlineLikeExecutor · dff3d40c
      Andrii Grynenko authored
      Summary: Tag all inline-like executors, so that there's a single way to detect them in places where we don't want inline executors.
      
      Differential Revision: D26564118
      
      fbshipit-source-id: 1a55c03bee0a0a56328175c016d3880a86eab8b1
      dff3d40c
    • Yedidya Feldblum's avatar
      no need to test for __has_include · 0bdb2b67
      Yedidya Feldblum authored
      Summary: Folly may assume all compilers have `__has_include`. Remove the single case of testing for it before using it.
      
      Reviewed By: simpkins
      
      Differential Revision: D26788728
      
      fbshipit-source-id: a11b47f6730700d81141d14c627e9d414ee6f68d
      0bdb2b67
  6. 05 Mar, 2021 10 commits
    • Alan Frindell's avatar
      Add AsyncSocket::UniquePtr ctor coro::Socket · b1ab506c
      Alan Frindell authored
      Summary: This diff is part of a series cleaning up coro::Socket
      
      Reviewed By: yairgott
      
      Differential Revision: D26837067
      
      fbshipit-source-id: e7ef587a6f859a167404b3eb2c8399544aa1d092
      b1ab506c
    • Ruslan Sayfutdinov's avatar
      Define S_IRXO, S_ISREG in SysStat.h · 696d65f7
      Ruslan Sayfutdinov authored
      Reviewed By: yfeldblum
      
      Differential Revision: D26848785
      
      fbshipit-source-id: 441aeac55f51ee8ec5e2316288f2c1f778a60dda
      696d65f7
    • Shai Szulanski's avatar
      folly::coro::makeUnorderedAsyncGeneratorFromAwaitable[Try]Range · 33b99b77
      Shai Szulanski authored
      Summary:
      Allows waiting for a range of SemiAwaitables while receiving progress incrementally.
      Often requested as a repeatedly-invokable collectAll.
      
      Uses an external AsyncScope to avoid the following sharp edge with using an internal scope (demonstrated in V1 of this diff): if the AsyncGenerator is destroyed without being fully drained we'd have to block on the scope, which can deadlock.
      
      Reviewed By: helfman
      
      Differential Revision: D25924855
      
      fbshipit-source-id: 46dd7c64d58560f57a7c18e35c159beb984286a7
      33b99b77
    • Yedidya Feldblum's avatar
      mod all folly::try_and_catch<std::exception> · e5ea61d1
      Yedidya Feldblum authored
      Summary: Mod all such cases not to pass any exception type list since such cases presumably want to catch all exceptions.
      
      Reviewed By: ericniebler
      
      Differential Revision: D26712856
      
      fbshipit-source-id: 1b5de869da5701a6bec85770f11c88195d7e62d8
      e5ea61d1
    • Henry Wang's avatar
      Use #if FOLLY_HAVE_SO_TIMESTAMPING · 70d84dd8
      Henry Wang authored
      Summary:
      D24094832 (https://github.com/facebook/folly/commit/842ecea531e8d6a90559f213be3793f7cd36781b) enabled code blocks which were currently not supported with SGX build mode, resulting in fbenclave build errs [T85891699].
      
      Our SGX targets depend on folly and are built with a custom toolchain. Because of our dep constraints we don't support all features. We typically try to toggle certain folly features, for example: https://www.internalfb.com/intern/diffusion/FBS/browsefile/master/tools/buckconfigs/fbcode/sgx.bcfg?commit=8981fc7187d8b2484a764e9277a1efa082a13f50&lines=36
      
      We were unable to disable these blocks of code because this macro was defined to be 1. We would typically just add the compiler flag: -DFOLLY_HAVE_SO_TIMESTAMPING=0. But doing causes redefinition.
      
      Reviewed By: bschlinker
      
      Differential Revision: D26765404
      
      fbshipit-source-id: 06d800efa7f23d9235a019ed7cc8fe4cde53c91c
      70d84dd8
    • Giuseppe Ottaviano's avatar
      Ensure that Function is noexcept-destructible · 694a416e
      Giuseppe Ottaviano authored
      Reviewed By: yfeldblum, philippv, luciang
      
      Differential Revision: D26832361
      
      fbshipit-source-id: 9f67a0af8c3c939cbfda1bd46806849e164e5354
      694a416e
    • Yedidya Feldblum's avatar
      try_and_catch with no type list · 94465a38
      Yedidya Feldblum authored
      Summary: Since most uses just specify `std::exception` and really just want to catch everything.
      
      Reviewed By: ericniebler
      
      Differential Revision: D26712827
      
      fbshipit-source-id: b33c1a311a839a50d81b9bc6c2b441908130dec6
      94465a38
    • Andrii Grynenko's avatar
      Don't discard tasks from executor when draining FiberManager · 2b8facde
      Andrii Grynenko authored
      Summary: FiberManager depends on those tasks to be run to know when it's safe to shut down (i.e. all outstanding tasks should be run).
      
      Differential Revision: D26811875
      
      fbshipit-source-id: 98b59ad28e25e7e47197e23ad3209349fdbfbb44
      2b8facde
    • Christopher Wei's avatar
      Add indent size for json pretty printing · 579e7cb2
      Christopher Wei authored
      Summary: Adds `pretty_formating_indent_size` option which is used to change the indentation size when used with `pretty_formatting`
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26793086
      
      fbshipit-source-id: 53efea33d7daa5c8f8840fc7f087d2d976ed3eda
      579e7cb2
    • Yedidya Feldblum's avatar
      revise demangle · 2690cff7
      Yedidya Feldblum authored
      Summary:
      Revise `folly::demangle` and its test suite.
      
      As a few items:
      * Minimize preprocessor conditional compilation.
      * Make the tests work for all combinations.
      
      Differential Revision: D26551208
      
      fbshipit-source-id: 422f6f96632c4dfc3f2e51f7b3ae8855cdb3ab30
      2690cff7
  7. 04 Mar, 2021 8 commits
    • Ruslan Sayfutdinov's avatar
      define NAME_MAX for Windows · 3fdabf98
      Ruslan Sayfutdinov authored
      Summary:
      If we have `PATH_MAX` we can probably have `NAME_MAX` as well.
      
      It should be exact equivalent of `NAME_MAX` on Windows:
      https://docs.microsoft.com/en-us/cpp/c-runtime-library/path-field-limits?view=msvc-160
      
      Reviewed By: Orvid
      
      Differential Revision: D26784071
      
      fbshipit-source-id: 1e28d5de4358efded182db85f0382939c96a4dce
      3fdabf98
    • Adam Simpkins's avatar
      fix the definition of FOLLY_HAS_STRING_VIEW on Windows · 4c227db0
      Adam Simpkins authored
      Summary:
      Update the check in `folly/Portability.h` to handle more recent versions of
      MSVC that do support `__has_include`.  The code on the `__has_include` code
      path was checking the value of `__cplusplus` rather than `_MSVC_LANG`.
      Microsoft only defines `__cplusplus` correctly when the compiler is invoked
      with the `/Zc:__cplusplus` flag.
      
      This updates the code to use the `FOLLY_CPLUSPLUS` macro instead.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26769080
      
      fbshipit-source-id: e40cfaef967b13d2fe1be2b40a109cb0ed39515a
      4c227db0
    • Dan Melnic's avatar
      Workaround for a clang -Wreorder issue · 618db6a4
      Dan Melnic authored
      Summary: Workaround for: "folly/Range.h:252:9: error: field 'b_' will be initialized after field 'e_' [-Werror,-Wreorder]"
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26788131
      
      fbshipit-source-id: 3664ea96adc26b0f97ad8d11259b018c39254692
      618db6a4
    • Maged Michael's avatar
      DynamicBoundedQueue: Round threshold up to avoid threshold 0 · 63c11c4e
      Maged Michael authored
      Summary: Ensure that threshold is not 0, otherwise when capacity < 10, threshold is 0, which can lead to consumers never transferring credit to producers.
      
      Differential Revision: D26791831
      
      fbshipit-source-id: 741bcf17de18198e43471d4b36d98c79288652fb
      63c11c4e
    • Andrew Gallagher's avatar
      Suppress unused parameter warnings on macos · 626e5caf
      Andrew Gallagher authored
      Summary: These cause unused variable warnings on other platforms.
      
      Reviewed By: mzlee
      
      Differential Revision: D26771614
      
      fbshipit-source-id: 9170c0da97375682624ee73fe422f34860f35a80
      626e5caf
    • Andrew Gallagher's avatar
      Suppress deprecation warnings on Apple platforms · a86f7668
      Andrew Gallagher authored
      Summary:
      On Apple platforms, some of the ucontext methods used here are
      deprecated.
      
      Reviewed By: yfeldblum, snufkinsnorka
      
      Differential Revision: D26765225
      
      fbshipit-source-id: 92c7e554b1d1e98b7b8566079a0cb3a0e8afc2d3
      a86f7668
    • Andrii Grynenko's avatar
      Patch AsyncStackTest.MixedStackWalk to work with the new <coroutine> header · 12593739
      Andrii Grynenko authored
      Summary: Depending on inlining of std library primitives (e.g. coroutine_handle) it's possible to have more frames in the stack trace.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26793792
      
      fbshipit-source-id: 44fb1344353e29ec92559ee34740157373112f8f
      12593739
    • Jason Rahman's avatar
      Fix incorrect purging in MemoryIdler · 64d8f3fd
      Jason Rahman authored
      Summary:
      When the given deadline to futexWaitUntil() is less than the idle
      timeout that controls purging, MemoryIdler should not purge JEMalloc arenas.
      Fix a bug where if the deadline is less than the idle timeout, purging happens
      100% of the time unconditionally.
      
      Reviewed By: mogeb, davidtgoldblatt, jalatif
      
      Differential Revision: D26784786
      
      fbshipit-source-id: 316ef96658d0cc2263b973b0b7c345605784eb36
      64d8f3fd
  8. 03 Mar, 2021 1 commit
    • Andrew Krieger's avatar
      dynamic constructor from anything String-like · e37262d4
      Andrew Krieger authored
      Summary:
      With the exception of std::string, nothing else string-like can be
      optimized further than a copy. So, create a generic Stringish constructor
      that copies .size() elements from .data(), where .data() has a value type
      of char.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D5407445
      
      fbshipit-source-id: acf709f07e73bb5bef3f124169c81d76edefa52c
      e37262d4