1. 15 Dec, 2017 1 commit
    • Dan Melnic's avatar
      Fix zerocopy AsyncSocket memory leaks · 23e45679
      Dan Melnic authored
      Summary:
      We currently leak ptr entries due to a missing map erase call in AsyncSocket::releaseZeroCopyBuf.
      Also addZeroCopyBuf called when the bytesWritten == 0 will cause and extra id to be allocated. This will delay the deletion of the last IOBuf sent via zerocopy. The number of buffers can accumulate over time if the same bytesWritten == 0 situation is encountered.
      Found when running the nbd-server zero downtime upgrade during heavy fio traffic.
      Add an AsyncSocket::processZeroCopyWriteInProgress so we can drain the zerocopy msg error queue even after we set the ReadCallback to nullptr.
      
      Reviewed By: djwatson
      
      Differential Revision: D6552982
      
      fbshipit-source-id: 3d2fdca84ec3b5fc46c3bed06c0c9ede66ed565a
      23e45679
  2. 14 Dec, 2017 8 commits
    • Chen Shen's avatar
      Fix xlog docs · f5882bb6
      Chen Shen authored
      Summary: When I was investigating the difference between glog/xlog I encountered this.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6572117
      
      fbshipit-source-id: 4130deb6cfed7cfb7d2433cdab292481beb2469f
      f5882bb6
    • Yedidya Feldblum's avatar
      Use explicit memory order in Baton::post · c5a4f4c8
      Yedidya Feldblum authored
      Summary: [Folly] Use explicit memory order in `Baton::post`.
      
      Reviewed By: djwatson
      
      Differential Revision: D6502268
      
      fbshipit-source-id: a379ea711f165e3645ac2127a49824f46225b80a
      c5a4f4c8
    • Kyle Nekritz's avatar
      Move security protocol and application protocol APIs to AsyncTransport. · 5acf7b92
      Kyle Nekritz authored
      Summary: They belong here rather than AsyncTransportWrapper.
      
      Reviewed By: siyengar
      
      Differential Revision: D6568209
      
      fbshipit-source-id: 0856dba0b2d527ae68d33942b28e2c485a5b1f46
      5acf7b92
    • Maged Michael's avatar
      Dynamic Bounded Queue · 17c08956
      Maged Michael authored
      Summary:
      DynamicBoundedQueue supports:
      - Dynamic memory usage that grows and shrink in proportion to the number of elements in the queue.
      - Adjustable capacity that helps throttle pathological cases of producer-consumer imbalance that may lead to excessive memory usage.
      - The adjustable capacity can also help prevent deadlock by allowing users to temporarily increase capacity substantially to guarantee accommodating producer requests that cannot wait.
      - SPSC, SPMC, MPSC, MPMC variants.
      - Blocking and spinning-only variants.
      - Inter-operable non-waiting, timed until, timed for, and waiting variants of producer and consumer operations.
      - Optional variable element weights.
      
      Reviewed By: djwatson
      
      Differential Revision: D6462123
      
      fbshipit-source-id: 5c7a7d88c3fa273211702af9ff78f6302afd2276
      17c08956
    • Maged Michael's avatar
      UnboundedQueue: Use alignas instead of FOLLY_ALIGNED · d5b67c25
      Maged Michael authored
      Summary: Use alignas instead of FOLLY_ALIGNED
      
      Reviewed By: Orvid
      
      Differential Revision: D6554984
      
      fbshipit-source-id: 5b11ac387faa137aa8dd0c35b6e9bb999cf0bf2d
      d5b67c25
    • Yedidya Feldblum's avatar
      Fix AsyncSocketTest.WriteErrorCallbackBytesWritten · 82d8337f
      Yedidya Feldblum authored
      Summary:
      [Folly] Fix `AsyncSocketTest.WriteErrorCallbackBytesWritten`.
      
      Thanks to congestion, especially when the tests are run concurrently, the expectations in the test were too restrictive.
      
      If only 20KB are read, it is possible that only 20KB are acknowledged. The expectation was that if 20KB are read and the recv buffer and send buffer are both 8KB and 24KB are written then all 24KB are acknowledged, but congestion control disagrees.
      
      It is possible that any number of bytes are written to the send buffer, from 28KB up to 40KB. And we have to explicitly wait for 28KB to be written even to know that (otherwise we only know that 20KB are written).
      
      Differential Revision: D6550804
      
      fbshipit-source-id: 100d086972c1526b909da0dbb6e609c144d7b17b
      82d8337f
    • Yedidya Feldblum's avatar
      constexpr_ceil · 8fba851f
      Yedidya Feldblum authored
      Summary: [Folly] `constexpr_ceil`, an integral rounding-up util.
      
      Reviewed By: Orvid
      
      Differential Revision: D6558042
      
      fbshipit-source-id: 6b42add9bf2e3605baf71391130c2a2c88cc4385
      8fba851f
    • Yedidya Feldblum's avatar
      Move max_align_v and max_align_t to folly/lang/Align.h · 9b2b633c
      Yedidya Feldblum authored
      Summary:
      [Folly] Move `max_align_v` and `max_align_t` to `folly/lang/Align.h`.
      
      And drop the macro `FOLLY_ALIGNED_MAX` - just use `alignas(max_align_v)` instead.
      
      Reviewed By: Orvid
      
      Differential Revision: D6554677
      
      fbshipit-source-id: cacb335b141623506a41e95e0740613374279c6f
      9b2b633c
  3. 13 Dec, 2017 6 commits
    • Yedidya Feldblum's avatar
      Remove unused #include <string.h> from folly/Portability.h · 0a1444e7
      Yedidya Feldblum authored
      Summary: [Folly] Remove unused `#include <string.h>` from `folly/Portability.h`.
      
      Reviewed By: Orvid
      
      Differential Revision: D6557374
      
      fbshipit-source-id: 369ef0b622dc057e882ed0f4ad25b47fa3f05100
      0a1444e7
    • Christopher Dykes's avatar
      Explicitly refer to the std::chrono namespace to avoid conflicts with the folly::chrono namespace · 4cbf7a64
      Christopher Dykes authored
      Summary:
      This is a hard requirement due to an upcoming change in Folly.
      
      The codemod was only run on files that referenced the `std::chrono` namespace with an unqualified `chrono` and also used both the `std` and `folly` namespaces.
      ```
      fbgr -sl "[^:]chrono::" | xargs grep -l "using namespace std;" | xargs grep -l "using namespace folly;" | xargs sed -r -i 's/([^:])chrono::([a-z])/\1std::chrono::\2/g'
      ```
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6547864
      
      fbshipit-source-id: 4a8230d311edbaa173722a09b2773e8d053fae7e
      4cbf7a64
    • RcRonco's avatar
      typo in io/Cursor.h · 9b471821
      RcRonco authored
      Summary: Closes https://github.com/facebook/folly/pull/721
      
      Differential Revision: D6558163
      
      Pulled By: yfeldblum
      
      fbshipit-source-id: ed03aa92b54c61f9980a4759da29910aa77d5db6
      9b471821
    • Dave Watson's avatar
      RCU · f885995c
      Dave Watson authored
      Summary: This adds an RCU implementation, matching http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0461r1.pdf as closely as pratical.  This implementation does not require thread registration or quiescence.
      
      Reviewed By: magedm
      
      Differential Revision: D6330631
      
      fbshipit-source-id: 2c729f3a4c0f151cde5d9a599ecd2a2c20c7da55
      f885995c
    • Adam Simpkins's avatar
      logging: minor clean up in Logger.h · 6283c759
      Adam Simpkins authored
      Summary:
      Update the comment describing the Logger class, and remove a `clang-format off`
      since recent versions of `clang-format` appear to do a good job formatting this
      macro.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6525996
      
      fbshipit-source-id: 6abbb3740afb53feb2418ca890cd78af1eb5f7a2
      6283c759
    • Adam Simpkins's avatar
      logging: add more documentation · 7fb9d9fd
      Adam Simpkins authored
      Summary: Add several new documents describing the behavior of the folly logging library.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6525997
      
      fbshipit-source-id: 8ba2ff99c70c7566987b3da321526a16755e72bf
      7fb9d9fd
  4. 12 Dec, 2017 6 commits
    • Yedidya Feldblum's avatar
      Switch to the try_wait_for and try_wait_until Baton APIs · a674aa6c
      Yedidya Feldblum authored
      Summary: Switch to the `try_wait_for` and `try_wait_until` `Baton` APIs.
      
      Reviewed By: davidtgoldblatt
      
      Differential Revision: D6532103
      
      fbshipit-source-id: aa3ce64152d167bb9c9cb1f266be0f9f8bd498f5
      a674aa6c
    • Andrii Nakryiko's avatar
      Store filename and provide detailed message on data access assertion failure. · eb7bc45f
      Andrii Nakryiko authored
      Summary:
      It looks like not having offset/size/filename information is way more
      harmful, than storing filename just for the sake of this error message.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6536616
      
      fbshipit-source-id: 469fbdf1deedd76ebd79cf98716c2c269cb10e4d
      eb7bc45f
    • Yedidya Feldblum's avatar
      Assorted tweaks to folly/String.h · c01c18b2
      Yedidya Feldblum authored
      Summary:
      [Folly] Assorted tweaks to `folly/String.h`.
      
      * Use `void_t` in the definition of `IsConvertible`.
      * Remove `AllConvertible`. Callers can use `StrictConjunction` instead.
      * Move the `static_assert`s from the header to the source file.
      
      Reviewed By: andrewjcg, ot
      
      Differential Revision: D6501515
      
      fbshipit-source-id: 472ecf23bf7f06be211480b0aceea95f7e60dc21
      c01c18b2
    • Neel Goyal's avatar
      Fix case where ssl cert does not match key · 3764b633
      Neel Goyal authored
      Summary: In some cases, SSLContextManager seg faults if a cert and key do not match.  This guards against that case when strictSSL = false, and throws a more useful error in the cases when SSL is required.
      
      Reviewed By: xybu
      
      Differential Revision: D6513964
      
      fbshipit-source-id: 8e63a22b346fd3f2a30d558a3659ab6794c7a105
      3764b633
    • Yedidya Feldblum's avatar
      folly::fibers::Baton API consistency with folly::Baton · a279ea6b
      Yedidya Feldblum authored
      Summary:
      [Folly] `folly::fibers::Baton` API consistency with `folly::Baton`.
      
      Specifically, the suite of `wait`, `try_wait`, `try_wait_for`, and `try_wait_until` members and member templates.
      
      Hold onto the `timed_wait` members for now, but mark them deprecated.
      
      Additionally, for consistency, offer main-context function params consistently for all `try_wait_for`, `try_wait_until`, and both variants of `timed_wait`.
      
      Reviewed By: andriigrynenko
      
      Differential Revision: D6531145
      
      fbshipit-source-id: 960fba48716b12b0ef53262786eacab88d8b2375
      a279ea6b
    • Nick Terrell's avatar
      Pass ZSTD_CONTENTSIZE_UNKNOWN · 2c8c7ee0
      Nick Terrell authored
      Summary:
      Content size 0 as unknown is deprecated.
      See code comments for details.
      
      Reviewed By: felixhandte
      
      Differential Revision: D6521582
      
      fbshipit-source-id: 876b94cc0123c1020b456cc8f4cd86997afe0619
      2c8c7ee0
  5. 10 Dec, 2017 4 commits
    • Yedidya Feldblum's avatar
      Remove multi-poster support from Baton · 712b8b8d
      Yedidya Feldblum authored
      Summary:
      [Folly] Remove multi-poster support from `Baton`.
      
      Code which needs multi-poster support may use `SaturatingSemaphore` instead.
      
      Reviewed By: magedm
      
      Differential Revision: D6529661
      
      fbshipit-source-id: d9dc053ca984ef3a404e9361910b0044817d4905
      712b8b8d
    • Andrew Gallagher's avatar
      Fix some clang build failures on aarch64 · 90ce64f9
      Andrew Gallagher authored
      Reviewed By: Orvid
      
      Differential Revision: D6387412
      
      fbshipit-source-id: 4c1424ed4fa49e69885de8cd72e78169314fc502
      90ce64f9
    • Maged Michael's avatar
      UnboundedBlockingQueue: Remove extra include · 87cfad8e
      Maged Michael authored
      Summary: Remove include not needed after recent change
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6529608
      
      fbshipit-source-id: 904072084e180125d68bd1ef6e4dffb2939cd1eb
      87cfad8e
    • Yedidya Feldblum's avatar
      Fix signed integer overflow in StaticTracepointTest.cpp · 01530690
      Yedidya Feldblum authored
      Summary:
      [Folly] Fix signed integer overflow in `StaticTracepointTest.cpp`.
      
      It is undefined behavior and is caught by UBSAN in some test runs.
      
      Reviewed By: Orvid
      
      Differential Revision: D6528445
      
      fbshipit-source-id: e4a199e9005b57bcd93c8c4b69f37ccb1e072bdf
      01530690
  6. 09 Dec, 2017 3 commits
    • Yedidya Feldblum's avatar
      Split Baton wait methods into fast and slow paths · bca34971
      Yedidya Feldblum authored
      Summary:
      [Folly] Split `Baton` wait methods into fast and slow paths.
      
      Force-inline the fast paths, and force-outline the slow paths.
      
      Reviewed By: magedm
      
      Differential Revision: D6501992
      
      fbshipit-source-id: 611e26b3cfeef01eb8d3a3500ae3ebc26bee6e86
      bca34971
    • Yedidya Feldblum's avatar
      Fix UBSAN failure in RangeTest.cpp with libstdc++-v4.9 · 2d3c3346
      Yedidya Feldblum authored
      Summary:
      [Folly] Fix UBSAN failure in `RangeTest.cpp` with `libstdc++-v4.9`.
      
      The implementation of `std::array` in `libstdc++-v4.9` has undefined behavior in the case of the zero-length array - dereferencing (but immediately taking the address again) of `nullptr`. This is fixed in `libstdc++-v5`.
      
      Reviewed By: ot
      
      Differential Revision: D6528614
      
      fbshipit-source-id: 26be6bdc28d9e6dc6e6293a4d4ecffd769c01413
      2d3c3346
    • Yedidya Feldblum's avatar
      replace std::dequeue with UMPMCQueue in UnboundedBlockingQueue · b92bbedc
      Yedidya Feldblum authored
      Summary: As above. Thanks magedm for giving us this beautiful piece of equipment!
      
      Reviewed By: magedm
      
      Differential Revision: D6488661
      
      fbshipit-source-id: 95aa9646ca1ea937bb1d055e9baa037896c3161e
      b92bbedc
  7. 08 Dec, 2017 4 commits
  8. 07 Dec, 2017 8 commits
    • Maged Michael's avatar
      FlatCombining: Use SaturatingSemaphore instead of multi-poster and non-blocking Baton · a9197170
      Maged Michael authored
      Summary: Use SaturatingSemaphore instead of multi-poster and non-blocking Baton, in preparation for restoring Baton to being single-poster and with blocking wait().
      
      Reviewed By: davidtgoldblatt
      
      Differential Revision: D6510619
      
      fbshipit-source-id: eed87beabc1b698080b8e41c746d570d30804ce7
      a9197170
    • Yedidya Feldblum's avatar
      Fix crash in exception_wrapper::get_exception<> · 39018ad5
      Yedidya Feldblum authored
      Summary:
      [Folly] Fix crash in `exception_wrapper::get_exception<>`.
      
      When the contained object is unthrown and does not inherit `std::exception`, `get_exception` templated over a type that does not inherit `std::exception` must throw and catch internally and may then return a pointer to the internally thrown object, which has since been deallocated.
      
      Attempting to dereference that pointer is undefined behavior is correctly caught by ASAN as heap-use-after-free.
      
      Fix it by storing objects not inheriting `std::exception` using only the `std::exception_ptr` representation.
      
      As a downside, we no longer get the small-object optimization or the non-throwing optimization for objects which do not inherit `std::exception`. But this is not likely to be terribly concerning in practice.
      
      Reviewed By: ericniebler
      
      Differential Revision: D6504911
      
      fbshipit-source-id: 0065de911733b5cab87be55e7e4e47f0a9c09140
      39018ad5
    • Yedidya Feldblum's avatar
      Use simpler tags for ctor dispatch in exception_wrapper · a335700f
      Yedidya Feldblum authored
      Summary:
      [Folly] Use simpler tags for ctor dispatch in `exception_wrapper`.
      
      The tags do not need to be `std::integral_constant`, but can just be arbitrary empty types.
      
      Also format the usage sites of the tags.
      
      Reviewed By: ericniebler
      
      Differential Revision: D6506417
      
      fbshipit-source-id: 377359086d94fc1a8bf8eebf6f8b058ff544829f
      a335700f
    • Stepan Palamarchuk's avatar
      Fix DCHECKs in IOBufQueue · fd0b3090
      Stepan Palamarchuk authored
      Summary: glog actually tries to pull srtings from these pointers when the checks fail, which obfuscates the error
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6506886
      
      fbshipit-source-id: 0283e02aecaa248b84fca1433d3f29f92c37b2e6
      fd0b3090
    • Mehdi Bouaziz's avatar
      Removed dead IsSomeVector · fe9d8cad
      Mehdi Bouaziz authored
      Summary: deadcode
      
      Reviewed By: ddrcoder
      
      Differential Revision: D6498754
      
      fbshipit-source-id: 9dda0ade6928d648b8938a06b185e430753f2318
      fe9d8cad
    • Andrii Grynenko's avatar
      Fix SignalHandlerTest with ASAN · f19e2b86
      Andrii Grynenko authored
      Reviewed By: yfeldblum
      
      Differential Revision: D6503762
      
      fbshipit-source-id: f82b88c401979761053004eadd2e4c6a74f05f9c
      f19e2b86
    • Adam Simpkins's avatar
      logging: allow partial updates to log handler settings · 171c3652
      Adam Simpkins authored
      Summary:
      This updates the LogHandlerConfig code to allow changing the settings on an
      existing log handler without listing all of its existing options from scratch.
      
      This also changes the syntax of the basic log handler configuration string to
      use a colon to separate the log handler name+type from the options list.  In
      other words, rather than specifying `default=stream,stream=stderr,async=true`
      you now say `default=stream:stream=stderr,async=true`.
      
      The primary motivation for this change is to make it easy for users to switch
      the async setting for the default log handler on or off.  Callers can now
      specify `default:async=true` to easily enable async mode on the default log
      handler without having to completely re-list the full settings for the default
      handler.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6494228
      
      fbshipit-source-id: 52a296f800a5456f0c3aa10546298139c8db52fc
      171c3652
    • Adam Simpkins's avatar
      logging: split FileHandlerFactory into two classes · 9e1a1ce1
      Adam Simpkins authored
      Summary:
      Split FileHandlerFactory into separate FileHandlerFactory and
      StreamHandlerFactory classes, where FileHandlerFactory only handles logging to
      files by path name, and StreamHandlerFactory only supports logging to stdout or
      stderr.
      
      The primary motivation for this is to allow logging to stdout or stderr in some
      cases without allowing arbitrary files to be opened by FileHandlerFactory.
      This can be achieved now by registering StreamHandlerFactory but not
      FileHandlerFactory.  This makes it safer to allow controlling logging
      configuration via command line flags even in setuid binaries.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6494226
      
      fbshipit-source-id: a3ec371ca4266424d07dff20be18e6e13c057593
      9e1a1ce1