1. 13 Mar, 2021 2 commits
    • Yedidya Feldblum's avatar
      expand the erased_unique_ptr facility · b3cbe1d5
      Yedidya Feldblum authored
      Summary:
      The interface:
      
        erased_unique_ptr
        to_erased_unique_ptr
        make_erased_unique
        copy_to_erased_unique_ptr
        empty_erased_unique_ptr
      
      Reviewed By: iahs
      
      Differential Revision: D26980776
      
      fbshipit-source-id: c520993a2191e7f74c5056a4854e2b85731a09a0
      b3cbe1d5
    • Yedidya Feldblum's avatar
      avoid snprintf in Elf code · 63ee5137
      Yedidya Feldblum authored
      Summary: `snprintf` may not be async-signal-safe, but Elf code is intended for use in signal handlers.
      
      Reviewed By: luciang
      
      Differential Revision: D27000080
      
      fbshipit-source-id: 818f48787b5712b7334eca782e18c71c17d03361
      63ee5137
  2. 12 Mar, 2021 6 commits
  3. 11 Mar, 2021 5 commits
    • Roger Kim's avatar
      Replace if-statements with `if constexpr` statements · e741d8ef
      Roger Kim authored
      Summary:
      We have found that BitVectorCoding.h incorrectly outputs
      -Wdivision-by-zero errors when another file that includes it fails to compile.
      In the absence of other compilation errors, the -Wdivision-by-zero error does
      not appear (since the compiler correctly can tell that "division by zero" is
      not an issue in the code).
      
      In order to get rid of these erroneous error logs, we decided to replace the
      if-statements that surround the divisions (and check that the divisor is not
      zero) with `if constexpr` statements since those will stop the compiler from
      mistakenly thinking that division by zero is possible.
      
      Reviewed By: ot, philippv, luciang
      
      Differential Revision: D26961916
      
      fbshipit-source-id: f65ef6d7782c94e58d6d28b9f8b053f205e3b7c1
      e741d8ef
    • Srivatsan Ramesh's avatar
      A new NotificationQueue without read limit for server responses · 28799ed1
      Srivatsan Ramesh authored
      Summary: Using a separate queue for server replies without any limit on the number of items read at once.
      
      Differential Revision: D26612326
      
      fbshipit-source-id: d56c892c87bafa701c12b6da780e9b2d4710ee2c
      28799ed1
    • Giuseppe Ottaviano's avatar
      Make parseTo work with arbitrary string types · db94b222
      Giuseppe Ottaviano authored
      Summary: `to<T>` already works in simple cases when `IsSomeString<T>`; by generalizing `parseTo` any string-like `T` gets full functionality without having to copy-paste the overload.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26831450
      
      fbshipit-source-id: c09629bb7173dde1e0986654128b8e1d7c48b9ba
      db94b222
    • Brandon Schlinker's avatar
      Stop relying on net_tstamp.h in AsyncSocket · e9d7342e
      Brandon Schlinker authored
      Summary:
      `net_tstamp.h` is incomplete on some platforms, particularly OSS.
      
      One option would be to make `FOLLY_HAVE_SO_TIMESTAMPING` conditional on all of the timestamp flags being present. However, I'm instead going to just decouple this support from the underlying platform's headers.
      
      Reviewed By: rsunkad
      
      Differential Revision: D26971541
      
      fbshipit-source-id: 01b24f57ab827aca42889138e4cca4d488ae0940
      e9d7342e
    • Lee Howes's avatar
      Remove default argument for SerialExecutor::createUnique · df81d76c
      Lee Howes authored
      Summary: Remove the default argument for the deprecated createUnique operation for SerialExecutor.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26916321
      
      fbshipit-source-id: e6fb1766bfaae4d6e702982c5527d483e6d8c036
      df81d76c
  4. 10 Mar, 2021 4 commits
    • Yedidya Feldblum's avatar
      fix to_ascii under 32-bit msvc builds · 14c99ce6
      Yedidya Feldblum authored
      Summary:
      Under gcc and clang, we use the intrinsic `__builtin_clzll`. This builtin is available regardless of whether the `clz` instruction is available on the target platform - when it is, the backend chooses it; when it is not, the backend emits emulation code.
      
      Under msvc, the builtin is `__lzcnt64`. However, this builtin is available only in x64 builds; there is no fallback to emulation code, and 32-bit builds fail to compile.
      
      Differential Revision: D26918251
      
      fbshipit-source-id: a4d9c199f632c5a210bd6fd8782d327f7aac774e
      14c99ce6
    • Jason Fried's avatar
      get_running_executor a modern alternative · 749f7a4b
      Jason Fried authored
      Summary:
      In the long term I want to force all usage of get_executor to require that the call comes from a running asyncio eventloop as its safer/saner than allowing the default un-running loop to be used since it may never actually run.  Until then I have created get_running_executor that I can use in various places. When get_running_executor(True) is used everywhere we can codemod it back to get_executor() and have asyncio.get_running_loop() used exclusively.
      
      A folly::IOBuf can be built around a Python Buffer.  It supports three ways to ask python to cleanup that buffer.
      
      1. If it has the GIL just decref the buffer
      2. else if it has a Python Executor it can schedule the cleanup
      3. it can rely on some cPython magic called "Py_AddPendingCall"
      
      thrift-py3 deserialize wraps a python buffer in a IOBuf, but since ownership is not handed off to C++ it has the GIL when we destroy the IOBUF so it doesn't need an Executor.
      
      IOBuf calls folly.executor.get_executor() which will create a AsyncioExecutor and bind it to the default eventloop.
      
      So using deserialize right before a python fork, would cause a ABORT in the child since the AsyncioExecutor would try to drive() itself during its destruction.
      
      The asyncio default eventloop is considered a bad thing, new interfaces like asyncio.run() does not rely on a default loop but creating its own.
      To insure we only ever use the eventloop from a running loop, use get_running_loop.
      
      Reviewed By: sdunster
      
      Differential Revision: D26794162
      
      fbshipit-source-id: 8e7e53c453c9ccff7e81f02e0692c8bebad70edd
      749f7a4b
    • Zhengxu Chen's avatar
      Make XLOG_EVENT_MS capture variables inside expanded lambda. · 7d413635
      Zhengxu Chen authored
      Summary:
      Just capture ms value for XLOG_EVERY_MS since it's executed inline.
      
      The use case here is we need to pass a non constant value as the milliseconds param, e.g:
      ```
      XLOG_EVERY_MS(INFO, configValue) << msg;
      ```
      
      Reviewed By: yfeldblum, simpkins
      
      Differential Revision: D26746623
      
      fbshipit-source-id: 849ad9a20ff4088ffb5a2df99c9c424592b47867
      7d413635
    • Yedidya Feldblum's avatar
      fix F14 fallback find() for stateful key-equal · 431c16c3
      Yedidya Feldblum authored
      Summary:
      The fallback implementation of F14 does a clever trick in its implementation of `find()` to support heterogeneous lookup. But this trick is UB and its implementation assumes that the key-equal type is empty, not specially-aligned, and not final. Relax these constraints.
      
      As it turns out, fbthrift streaming uses an F14 map with a stateful key-equal, triggering this UB in the fallback implementation: `StreamMap` uses key-equal type `StreamMapEquals`, which is nonempty and has a `StreamIdResolver` field, which itself is empty but which causes `StreamMapEquals` not to be empty. The simplest fix is, of course, to switch to using `StreamIdResolver` as a base, but this would be a bug in the F14 fallback regardless.
      
      `BottomKeyEqual` must have the same size, alignment, emptiness, and finality as `KeyEqual`.
      
      Reviewed By: aary
      
      Differential Revision: D26905527
      
      fbshipit-source-id: 023feb3963fc7b2e779fb591e737ea4b70ded476
      431c16c3
  5. 09 Mar, 2021 10 commits
    • Alex Snast's avatar
      add heterogeneous access support to EvictingCacheMap · 0efd32c4
      Alex Snast authored
      Summary:
      Heterogeneous access is described [here](https://github.com/facebook/folly/blob/master/folly/container/F14.md#heterogeneous-key-type-with-transparent-hash-and-equality) and this diff adds this capability to EvictingCacheMap.
      
      See `HeterogeneousAccess` test from EvictingCacheMapTest.cpp for usage example.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26879601
      
      fbshipit-source-id: 027352f2daa6773ab981b924732e1056550f6ea2
      0efd32c4
    • Tom Kwong's avatar
      Add StreamingStats functionality · 0fbe9c5b
      Tom Kwong authored
      Summary:
      This diff adds new capability to calculate statistics efficiently in a single pass. In particular, it uses Welford's algorithm to track an internal state such that variance and standard deviation can be obtained at O(1).
      
      The following statistics are available:
      - count
      - minimum
      - maximum
      - mean
      - sample and population variance
      - sample and population standard deviation
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26667517
      
      fbshipit-source-id: c4eb31e93646b9cff2f6516039170c8d88a5bf33
      0fbe9c5b
    • Philip Pronin's avatar
      define GENERATOR_REF · 438afd7d
      Philip Pronin authored
      Summary:
      Version of `GENERATOR` that we're already using in a few places where
      it doesn't leave its original scope.
      
      Reviewed By: ddrcoder, luciang
      
      Differential Revision: D26909115
      
      fbshipit-source-id: 5fbae8bd44c62d4d96427a4fd3ef60a8d9cb40e3
      438afd7d
    • Alan Frindell's avatar
      remove folly/io/coro/Socket.h shim · 3af4c3db
      Alan Frindell authored
      Summary: All dependencies have migrated to folly/io/coro/Transport.h
      
      Reviewed By: yairgott
      
      Differential Revision: D26837064
      
      fbshipit-source-id: abc46819e4d9bdec6ff5898d0295731781c7925d
      3af4c3db
    • Alan Frindell's avatar
      rename coro::Socket to coro::Transport · 8b4086af
      Alan Frindell authored
      Summary: This also renames the file and creates a Socket.h shim for compatabilty.
      
      Reviewed By: yairgott
      
      Differential Revision: D26837068
      
      fbshipit-source-id: 47c8f7b410540d1ad0aa03f4bc96816f8a93ef1c
      8b4086af
    • Alan Frindell's avatar
      coro::Socket wraps AsyncTransport · c3156ea8
      Alan Frindell authored
      Summary: This allows coro::Socket to be used with non-socket AsyncTransports, notably Fizz.  A subsequent diff renames the class and file, and creates a shim for callers.
      
      Reviewed By: yairgott
      
      Differential Revision: D26610473
      
      fbshipit-source-id: d64597ef0de3c90ab084249b20e85d97b34857a6
      c3156ea8
    • generatedunixname89002005325676's avatar
      Daily `arc lint --take CLANGFORMAT` · 2f7a1264
      generatedunixname89002005325676 authored
      Reviewed By: zertosh
      
      Differential Revision: D26908037
      
      fbshipit-source-id: 5c636fd281066f1d9af0d92f6f0b015ebf77437b
      2f7a1264
    • Dave Rigby's avatar
      Mark StaticMata as FOLLY_EXPORT (#1533) · 7c7531bd
      Dave Rigby authored
      Summary:
      When using folly::ThreadLocal<> to create thread-local variables, if a thread-local is set from two different libraries which are compiled with -fvisibility=hidden and using libc++, then a nullptr dereference is seen when attempting to set the thread-local from the second DLL.
      
      This is due to StaticMetaBase (which should be a singleton) getting created twice. This results in the thread local metadata getting corrupted and hence attempting to dereference a nullptr.
      
      Fix by marking StaticMeta as FOLLY_EXPORT, so only a single instance exists at runtime.
      
      Fixes https://github.com/facebook/folly/issues/1431
      
      Pull Request resolved: https://github.com/facebook/folly/pull/1533
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26818467
      
      Pulled By: Orvid
      
      fbshipit-source-id: 6b22627bc51060260e653885bdeb0df111207617
      7c7531bd
    • Yedidya Feldblum's avatar
      reduce inline code size of Executor::invokeCatchingExns · 5a37f8c8
      Yedidya Feldblum authored
      Summary: Some use-cases can be sensitive to inline code size, so reduce the inline code size of `Executor::invokeCatchingExns` just in case.
      
      Reviewed By: vitaut, zhxchen17
      
      Differential Revision: D26875990
      
      fbshipit-source-id: 0b822f7425bb47c98837faf08216092b49a7ed38
      5a37f8c8
    • Adam Simpkins's avatar
      update farmhash.cpp to avoid an unused function error on android · 4d235eee
      Adam Simpkins authored
      Summary:
      The `Fetch128()` function is only used in `#ifdef` sections that check for
      either `can_use_sse42` or `can_use_sse41 && x86_64`.  On Android builds this
      function was defined as a static function that was never used, causing
      compiler errors when compiled with `-Werror -Wunused-function`
      
      Reviewed By: mzlee
      
      Differential Revision: D26865431
      
      fbshipit-source-id: 5cd7ce12aa614f3d904df40ba21842223502a9ec
      4d235eee
  6. 08 Mar, 2021 5 commits
    • Kenny Yu's avatar
      Print async stack trace in signal handler · e85a11d5
      Kenny Yu authored
      Summary:
      If there is an active async operation on the current thread, then the signal
      handler will also print the async stack trace as well. Note that an async stack
      trace can have a mix of both non-async and async frames. This is an example
      of what an async stack trace might print:
      
      ```
      funcF      <- non async, top of stack
      funcE      <- non async
      co_funcD   <- async
      funcC      <- non async, blocking call. Second async stack root here
      co_funcB   <- async
      co_funcA   <- async
      main       <- non async. First async stack root here
      ```
      
      If there is no async operation in progress, there is no change in behavior
      (only normal stack trace is printed).
      
      This also adds a helper `getAsyncStackTraceStr()` to easily dump
      the async stack trace on demand. This helper is NOT async-signal-safe
      and is only meant for debugging purposes.
      
      Reviewed By: yfeldblum, andriigrynenko
      
      Differential Revision: D26661006
      
      fbshipit-source-id: f6e913910eb45c438a3e1e29d231e224eed15ef2
      e85a11d5
    • Seth Hinze's avatar
      Fix logging of AsyncSocket connection failed exceptions · 47667233
      Seth Hinze authored
      Summary:
      When log verbosity is sufficiently high, the logging of AsyncSocket connection exceptions triggers the following assert:
      ```
        assert(typeid(ex) == typeid(std::decay_t<Ex>) ||
             !"Dynamic and static exception types don't match. Exception would "
              "be sliced when storing in exception_wrapper.");
      ```
      Update the log to output `e.what()`, which all of the other exception logs already do.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26799863
      
      fbshipit-source-id: f9d98f81a77a654e78a153bc5c23e6ec8223fa1f
      47667233
    • Yedidya Feldblum's avatar
      use to_ascii_decimal in FOLLY_SAFE_CHECK · 9a5fc814
      Yedidya Feldblum authored
      Summary: Use `folly::to_ascii_decimal` in `FOLLY_SAFE_CHECK` and related facility implementations. This facility is intentionally async-signal-safe on its own and is light on its dependencies. Specifically, avoid the dependency on `folly/Conv.h` which is heavier.
      
      Reviewed By: luciang
      
      Differential Revision: D26597237
      
      fbshipit-source-id: e3cf0b3a3046cdbd3da89cc8a2e0627edb5a5a85
      9a5fc814
    • Cameron Pickett's avatar
      Fix CO_ASSERT_EQ for new gtest version · 32d05841
      Cameron Pickett authored
      Differential Revision: D26882407
      
      fbshipit-source-id: f7ab479aff6b8eaced65da333415efd33112ea0a
      32d05841
    • Aaryaman Sagar's avatar
      Explicitly qualify atomic_wait and atomic_notify function calls · 8477e256
      Aaryaman Sagar authored
      Summary:
      `std::atomic_wait()` and `std::atomic_notify_one()` are marked as unavailable.
      Not really sure what that means, but it seems to be breaking some open source
      builds https://github.com/facebook/folly/issues/1527.  Explicitly qualify the
      calls into those functions to try and fix the build break.
      
      Since we cannot conditionally import either of the above (because they are
      marked as unavailable), we can't rely on the standard implementations.  To
      prevent ADL from kicking in when the standard-library defines these, we fully
      qualify these and use `tag_invoke` for the customization points used in tests.
      
      Reviewed By: davidtgoldblatt
      
      Differential Revision: D26742072
      
      fbshipit-source-id: 9f44bbfd37530f5ecffa3e03d673cfb1df953299
      8477e256
  7. 07 Mar, 2021 2 commits
    • Yedidya Feldblum's avatar
      to_ascii, to_ascii_size · 89a3d4a7
      Yedidya Feldblum authored
      Summary:
      Generic ascii-ification functions `to_ascii` and `to_ascii_size` for varying unsigned integer types over varying integer bases.
      
      Has minimal dependencies. Is explicitly async-signal-safe.
      
      Reviewed By: kennyyu
      
      Differential Revision: D26593764
      
      fbshipit-source-id: b1432bee8ca8eb0a5304943c4e231e201425f14a
      89a3d4a7
    • Yedidya Feldblum's avatar
      Executor::invokeCatchingExns · 2f6126e6
      Yedidya Feldblum authored
      Summary: Executors just carry on after a task they are running fails, although they try to log something marginally useful. Deduplicate the logic.
      
      Reviewed By: aary, ot, philippv
      
      Differential Revision: D26745058
      
      fbshipit-source-id: 90fa558c89591b8a2d08e6820388970dee8811be
      2f6126e6
  8. 06 Mar, 2021 3 commits
    • Alfred Fuller's avatar
      Update include categories in clang-format canary · 688c8aaa
      Alfred Fuller authored
      Summary:
      This just fixes the include order within groups of includes, so is significantly less aggressive than the other codemod which regroups includes.
      
      I've manually ensured that:
      - groups are split when trivial.
      - glog is not reordered relative to other folly headers (seems to cause down stream issues)
      - portability headers are not reordered (seems to cause down stream issues)
      other diffs will finish the regrouping and ordering of those groups.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D26405757
      
      fbshipit-source-id: 9b79a91a41a4e6ac677ed0cb7fb24dfb7c8093d2
      688c8aaa
    • Andrii Grynenko's avatar
      InlineLikeExecutor · dff3d40c
      Andrii Grynenko authored
      Summary: Tag all inline-like executors, so that there's a single way to detect them in places where we don't want inline executors.
      
      Differential Revision: D26564118
      
      fbshipit-source-id: 1a55c03bee0a0a56328175c016d3880a86eab8b1
      dff3d40c
    • Yedidya Feldblum's avatar
      no need to test for __has_include · 0bdb2b67
      Yedidya Feldblum authored
      Summary: Folly may assume all compilers have `__has_include`. Remove the single case of testing for it before using it.
      
      Reviewed By: simpkins
      
      Differential Revision: D26788728
      
      fbshipit-source-id: a11b47f6730700d81141d14c627e9d414ee6f68d
      0bdb2b67
  9. 05 Mar, 2021 3 commits
    • Alan Frindell's avatar
      Add AsyncSocket::UniquePtr ctor coro::Socket · b1ab506c
      Alan Frindell authored
      Summary: This diff is part of a series cleaning up coro::Socket
      
      Reviewed By: yairgott
      
      Differential Revision: D26837067
      
      fbshipit-source-id: e7ef587a6f859a167404b3eb2c8399544aa1d092
      b1ab506c
    • Ruslan Sayfutdinov's avatar
      Define S_IRXO, S_ISREG in SysStat.h · 696d65f7
      Ruslan Sayfutdinov authored
      Reviewed By: yfeldblum
      
      Differential Revision: D26848785
      
      fbshipit-source-id: 441aeac55f51ee8ec5e2316288f2c1f778a60dda
      696d65f7
    • Shai Szulanski's avatar
      folly::coro::makeUnorderedAsyncGeneratorFromAwaitable[Try]Range · 33b99b77
      Shai Szulanski authored
      Summary:
      Allows waiting for a range of SemiAwaitables while receiving progress incrementally.
      Often requested as a repeatedly-invokable collectAll.
      
      Uses an external AsyncScope to avoid the following sharp edge with using an internal scope (demonstrated in V1 of this diff): if the AsyncGenerator is destroyed without being fully drained we'd have to block on the scope, which can deadlock.
      
      Reviewed By: helfman
      
      Differential Revision: D25924855
      
      fbshipit-source-id: 46dd7c64d58560f57a7c18e35c159beb984286a7
      33b99b77