1. 13 Jul, 2020 1 commit
    • Eric Niebler's avatar
      replace the use of boost::variant with a simple union · ed2cb6d2
      Eric Niebler authored
      Summary:
      `boost::variant` is an expensive template, and `boost/variant.hpp` is an expensive header. In the one place it is used in futures/detail/Core.h (a commonly-included header), it can be trivially replaced with a `union`, so do so.
      
      As a drive-by, I mark as `noexcept` those members of `KeepAliveOrDeferred` that can be done so unconditionally.
      
      Reviewed By: yfeldblum, kirkshoop, ot
      
      Differential Revision: D22460453
      
      fbshipit-source-id: 5a01f873058273d1a20265507d87796450cc008b
      ed2cb6d2
  2. 12 Jul, 2020 1 commit
  3. 10 Jul, 2020 10 commits
    • Mark Santaniello's avatar
      Use sized deallocation in SysAllocator and Arena (2nd try) · b99077fa
      Mark Santaniello authored
      Summary:
      Use sized-deallocation (`sdallocx`) if possible in `folly::SysAllocator` and `folly::Arena`.
      
      `Arena` has always allocated two types of blocks:
      1. Normal (fixed-sized): size is the "goodSize adjusted" `minBlockSize`
      2. Large (variable-sized): when #1 is too small
      
      Type #2 makes sized-deallocation tricky -- we need somewhere to remember the allocated sizes.
      
      The old code used a single type `Block` and kept a single list. Here I change to have two types and two lists.  The `LargeBlock` has an additional `allocSize` data member.
      
      This makes the Arena object itself 16B larger, but seems better than adding a 4B `allocSize` to each and every block, regardless of type.
      
      Note that, prior to this change, it was possible to `merge()` arenas with different `minBlockSize`.  This is no longer possible.
      
      Reviewed By: davidtgoldblatt
      
      Differential Revision: D22466906
      
      fbshipit-source-id: 719f357a0a1f6cfcda3208391837195c3859ab69
      b99077fa
    • Lukasz Piatkowski's avatar
      edenscm/hg: add GitHub Actions with CI for HG plus add fixes for getdeps (#25) · e9a34c04
      Lukasz Piatkowski authored
      Summary:
      Fixes include:
      1. Passing "GETDEPS_BUILD_DIR" and "GETDEPS_INSTALL_DIR" env variable and using them in eden/scm/Makefile rather than assuming the source code is always in the same place regardless getdeps arguments (it isn't).
      2. Added "fbthrift-source" and "fb303-source" to avoid unnecessary compilation (at least of fb303) and to put fbthrift and fb303 source code in an easy to locate place inside getdeps' "installed" folder.
      
      Pull Request resolved: https://github.com/facebookexperimental/eden/pull/25
      
      Test Plan: sandcastle, check oss-eden_scm-darwin-getdeps
      
      Reviewed By: farnz
      
      Differential Revision: D22431872
      
      Pulled By: lukaspiatkowski
      
      fbshipit-source-id: 8ccbb090713ec085a5dd56df509eb58ab6fb9e34
      e9a34c04
    • Ian Petersen's avatar
      Fix array length calculation in ConvTest.cpp · 287dd830
      Ian Petersen authored
      Summary:
      The build with Clang 10 on iOS broke with the following error:
      
      ```
      folly/test/ConvTest.cpp:321:45: error: expression does not compute the number of elements in this array; element type is 'const char *const', not 'const char *const [4]' [-Werror,-Wsizeof-array-div]
          FOR_EACH_RANGE (i, 0, sizeof(uStrings2) / sizeof(uStrings2)) {
                                       ~~~~~~~~~  ^
      ```
      
      Looking at nearby uses of `FOR_EACH_RANGE`, it looks like the new compiler has caught a typo, which this diff fixes.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D22470932
      
      fbshipit-source-id: 3090ed9824af488fc429becf2c1084c7725daf5a
      287dd830
    • Phil Willoughby's avatar
      clang coro handling · cbdf3df7
      Phil Willoughby authored
      Summary:
      On windows `<experimental/coroutine>` comes from the Microsoft SDK and uses the MSVC intrinsics. If you are compiling with clang this header exists but nothing from it will work because clang does not implement the MSVC intrinsics.
      
      Further, the clang and MSVC intrinsics are ABI incompatible with each other to the extent that there is no way I can find to implement the MSVC intrinsics behavior as inline functions in a clang TU.
      
      MS have indicated publicly that they will work with the LLVM project to make sure that clang and MSVC will be compatible for `<coroutine>`, but until then we have to say that folly does not have coroutines when clang is used on Windows.
      
      (Note: this ignores all push blocking failures!)
      
      Reviewed By: yfeldblum
      
      Differential Revision: D22301662
      
      fbshipit-source-id: 96b1c3909a3916fe300073af34429eeb5c08d1fd
      cbdf3df7
    • Sotirios Delimanolis's avatar
      Set SSLContext minimum version through SSL_CTX_set_min_proto_version for OpenSSL >= 1.1.0 · 2fa292de
      Sotirios Delimanolis authored
      Summary:
      Attempt to set minimum version of an OpenSSL context through `SSL_CTX_set_min_proto_version` instead of disabling them explicitly through options, as recommended here: https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_min_proto_version.html
      
      The test checks that the `SSLContext`'s `SSL_CTX` can keep a lower version regardless of the OpenSSL's configured minimum.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D22338879
      
      fbshipit-source-id: 356328c2ffba2a6a9d4243300a11fc823bc345d7
      2fa292de
    • Giuseppe Ottaviano's avatar
      Fix race between setProxy() and setCallback() · bff1bb83
      Giuseppe Ottaviano authored
      Summary:
      D22371898 (https://github.com/facebook/folly/commit/4981497ad3333ab084e18b2a02a574cbf9438585) introduced a race by attempting to share the storage between `proxy_` and `callback_`: `setProxy()` and `setCallback()` may concurrently try to set both, and there's no good ordering we can use in `setCallback()` to fetch the proxy pointer before constructing the callback.
      
      So revert to the logic pre-D22371898 (https://github.com/facebook/folly/commit/4981497ad3333ab084e18b2a02a574cbf9438585). This means increasing the `Core` footprint by a further 8 bytes, but D22474230 should recover that.
      
      Differential Revision: D22474532
      
      fbshipit-source-id: c63ca6ecd166fb71dcbeddbb2c04eb07494f99ea
      bff1bb83
    • Lu Pan's avatar
      set max deferred readers for folly::SharedMutex dynamically · 1ad7426b
      Lu Pan authored
      Summary: Instead of hard coding the max deferred readers allowed to be 64, statically allocate large enough slots and pick max deferred readers allowed dynamically based on the platform running the service. Specifically, we set the `maxDeferredReaders = 2 * nextPowTwo(numCPU)`, which four times the number of physical cores, to allow faster reads. We are effectively giving each HW thread two slots.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D22407478
      
      fbshipit-source-id: 4001cf96dc502e00f00a27d57c63ba0028a52671
      1ad7426b
    • Giuseppe Ottaviano's avatar
      Outline most of Core implementation · ca9225f8
      Giuseppe Ottaviano authored
      Summary: After D22371898 we can move the `CoreBase` implementation out of the header.
      
      Reviewed By: luciang
      
      Differential Revision: D22371900
      
      fbshipit-source-id: bd4fe6df298c0ba02988b357cd7413eb8f7d8b67
      ca9225f8
    • Giuseppe Ottaviano's avatar
      Type-erase most of Core implementation · 4981497a
      Giuseppe Ottaviano authored
      Summary:
      `Core<T>` instantiations share a large part of the implementation, but in the current shape the code is duplicated for each instantiation. This diff moves whatever possible to a common base class.
      
      This diff only splits the typed and untyped parts, for ease of review. The untyped parts are moved to the cpp file in D22371900.
      
      Reviewed By: andriigrynenko
      
      Differential Revision: D22371898
      
      fbshipit-source-id: db48c202d0bdcbefbebe150e7c7d9f35e06687d0
      4981497a
    • Dan Melnic's avatar
      Fix jemalloc folly weak symbol issue · aac9d77f
      Dan Melnic authored
      Summary:
      Fix jemalloc folly weak symbol issue
      
      (Note: this ignores all push blocking failures!)
      
      Reviewed By: davidtgoldblatt
      
      Differential Revision: D22420581
      
      fbshipit-source-id: 136fa6a8c74d8fea3fc96ba1d205266f2f2fa02c
      aac9d77f
  4. 09 Jul, 2020 4 commits
    • Hasnain Lakhani's avatar
      Extend findAddress to provide function parameter names · 63cdc1c1
      Hasnain Lakhani authored
      Summary:
      We provide an optional callback that callers can use to save parameter names
      where desired.
      
      Differential Revision: D22399275
      
      fbshipit-source-id: 7f9440a46cbcc4085d48a2387ca3d2db51f2fe7e
      63cdc1c1
    • Omer Strulovich's avatar
      Revert D22189916: Use sized deallocation in SysAllocator and Arena · 5ea745d4
      Omer Strulovich authored
      Differential Revision:
      D22189916 (https://github.com/facebook/folly/commit/0d2bdbf0e0618bd9366d3c35345e091ee24f0fae)
      
      Original commit changeset: e6fba48eaae0
      
      fbshipit-source-id: 4ea03c56fda84240e178b2b4baf83a57d3afa344
      5ea745d4
    • Mark Santaniello's avatar
      Use sized deallocation in SysAllocator and Arena · 0d2bdbf0
      Mark Santaniello authored
      Summary:
      Use sized-deallocation (`sdallocx`) if possible in `folly::SysAllocator` and `folly::Arena`.
      
      `Arena` has always allocated two types of blocks:
      1. Normal (fixed-sized): size is the "goodSize adjusted" `minBlockSize`
      2. Large (variable-sized): when #1 is too small
      
      Type #2 makes sized-deallocation tricky -- we need somewhere to remember the allocated sizes.
      
      The old code used a single type `Block` and kept a single list. Here I change to have two types and two lists.  The `LargeBlock` has an additional `allocSize` data member.
      
      This makes the Arena object itself 16B larger, but seems better than adding a 4B `allocSize` to each and every block, regardless of type.
      
      Note that, prior to this change, it was possible to `merge()` arenas with different `minBlockSize`.  This is no longer possible.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D22189916
      
      fbshipit-source-id: e6fba48eaae0b5cc8456b856b02d2cfc71c03834
      0d2bdbf0
    • Dan Melnic's avatar
      Replace loop with clear_and_dispose() · c2a5cc0f
      Dan Melnic authored
      Summary: Replace loop with clear_and_dispose()
      
      Reviewed By: yfeldblum
      
      Differential Revision: D22450280
      
      fbshipit-source-id: cad040abd3f40e0fbb1f2ab27d06bf5174760c2c
      c2a5cc0f
  5. 08 Jul, 2020 6 commits
    • Dan Melnic's avatar
      IoUringBackend free mempool rework · 9b29fac9
      Dan Melnic authored
      Summary: IoUringBackend free mempool rework
      
      Reviewed By: kevin-vigor
      
      Differential Revision: D22424753
      
      fbshipit-source-id: 5f8482702ec17561058ea841493e8a5a57b642a6
      9b29fac9
    • Matt Galloway's avatar
      folly | Fix building folly's FileUtil.cpp in iOS 14 or macOS 11 SDK · eaf22e04
      Matt Galloway authored
      Summary: Fix issue where `preadv` and `pwritev` are now included in iOS 14 and macOS 11 SDKs, so we need to do runtime checks for those platforms.
      
      Reviewed By: Orvid
      
      Differential Revision: D22275221
      
      fbshipit-source-id: 12e45c55a0fcf10f540f2d4cb9a3c392d7e182cb
      eaf22e04
    • Giuseppe Ottaviano's avatar
      Outline some of Core.h guts · e4869845
      Giuseppe Ottaviano authored
      Summary: `Future.h` is widely included; `KeepAliveOrDeferred` and `DeferredExecutor` are not templates, we can move all the definitions to a cpp file.
      
      Reviewed By: yfeldblum, luciang
      
      Differential Revision: D22371899
      
      fbshipit-source-id: 23a77b2e0560d82f6c3006597eef85d11527cb13
      e4869845
    • Dan Melnic's avatar
      processSubmit rework · 64058698
      Dan Melnic authored
      Summary: processSubmit rework
      
      Reviewed By: mokomull
      
      Differential Revision: D22405528
      
      fbshipit-source-id: 99db524e62264d80556e6ac771f6d504cb31bd37
      64058698
    • Zeyi (Rice) Fan's avatar
      update GitHub actions · fb4af641
      Zeyi (Rice) Fan authored
      Summary:
      The GitHub actions have been failing because `rsocket-cpp` has been deleted.
      
      https://github.com/facebook/watchman/runs/843777632
      
      This commit is generated with `./opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh`
      
      Reviewed By: wez
      
      Differential Revision: D22417304
      
      fbshipit-source-id: e64a8d011c753bf8ac90ed2c9c90036baa8cf950
      fb4af641
    • Pranav Thulasiram Bhat's avatar
      Turn await into a CPO · bf54ecec
      Pranav Thulasiram Bhat authored
      Summary: Implement await as a CPO - Allow customization for user defined types outside of folly.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D22268765
      
      fbshipit-source-id: 4badba9274b7206afc339e9d94f6ca991dc5674e
      bf54ecec
  6. 07 Jul, 2020 2 commits
    • Sotirios Delimanolis's avatar
      Add FOLLY_OPENSSL_PREREQ macro function · 4d04d3d7
      Sotirios Delimanolis authored
      Summary:
      This diff introduces a new macro function to resolve whether the current OpenSSL version available is greater than or equal to the given (major, minor, fix) version tuple.
      
      We used the rules defined in the [OPENSSL_VERSION_NUMBER man page](https://www.openssl.org/docs/man1.1.1/man3/OPENSSL_VERSION_NUMBER.html), ie. nibbles for each of
      ```
       MNNFFPPS: major minor fix patch status
      ```
      
      We choose to ignore `patch` and `status` for simplicity and because we don't expect to build against such versions.
      
      The existing `FOLLY_OPENSSL_IS_110` variable already checks for greater than or equal to version number 1.1.0, so I didn't feel the need to modify it (nor the other variables). We can do some clean up in a future diff.
      
      Reviewed By: yfeldblum, mingtaoy
      
      Differential Revision: D22346692
      
      fbshipit-source-id: 156ee69ecd619de12319d7d63239f28c323820a4
      4d04d3d7
    • Kirk Shoop's avatar
      create explicit instantiations for folly::Unit · 3e9186f1
      Kirk Shoop authored
      Summary:
      move template instantiations for folly::Unit of several types into explicit instantiations in cpp files.
      
      `folly::Future<folly::Unit>`
      `folly::SemiFuture<folly::Unit>`
      `folly::Promise<folly::Unit>`
      `folly::SharedPromise<folly::Unit>`
      `folly::futures::detail::FutureBase<folly::Unit>`
      `folly::futures::detail::Core<folly::Unit>`
      `folly::Try<folly::Unit>`
      
      Reviewed By: yfeldblum, ericniebler
      
      Differential Revision: D22092858
      
      fbshipit-source-id: b8faa1c8aedfc193c17e2c3bc2e5b0f429ae6b0f
      3e9186f1
  7. 06 Jul, 2020 5 commits
    • Zeyi (Rice) Fan's avatar
      add retry flag to getdeps test · 9426d302
      Zeyi (Rice) Fan authored
      Summary: This commit adds a flag `--retry` to getdeps and teach it to run retry failed test. This allows us to still pass the tests when there are some flaky tests presents.
      
      Reviewed By: wez
      
      Differential Revision: D22291063
      
      fbshipit-source-id: 572af48a52ceb4a9abbf530cc0154ded0120c0de
      9426d302
    • Pranav Thulasiram Bhat's avatar
      Add Task<void> specialization for taskWait · f9dec043
      Pranav Thulasiram Bhat authored
      Summary:
      `folly::coro::blockingWait(...)` returns `void` for `Task<void>`.
      
      This diff implements a template specialization on taskWait for `Task<void>`
      
      Differential Revision: D22265598
      
      fbshipit-source-id: ff13d2b18b8706a6e8257ef706e407b315daf98a
      f9dec043
    • Pranav Thulasiram Bhat's avatar
      Faster collectAll(args...) · bd41a1ed
      Pranav Thulasiram Bhat authored
      Summary:
      The variadic implementation of collectAll uses the futures APIs, and is likely to be slow.
      
      This diff adds a fiber only version similar to fibers::foreach
      
      Reviewed By: yfeldblum
      
      Differential Revision: D22212232
      
      fbshipit-source-id: f71765c5411d2c6ca4388fe373baafad65f1b01c
      bd41a1ed
    • Zeyi (Rice) Fan's avatar
      avoid using relative path in fb_py_win_main · 18372331
      Zeyi (Rice) Fan authored
      Summary:
      After some experimenting, it is a little awkward if we want to specify a relative path based on the executable location. We'd need to add a bunch of path calculations to make it right, and I don't think the added complexity is really worth the effort.
      
      As a result, let's just remove the use of relative path, and if we ever want to ship a copy of Python distribution, we can place it under the same directory as the binary.
      
      Reviewed By: chadaustin
      
      Differential Revision: D22394180
      
      fbshipit-source-id: 86d27f6d16a03fe08826b5e5eafcef2a1c77997f
      18372331
    • Shai Szulanski's avatar
      fix clang-format nits · 5b5359fe
      Shai Szulanski authored
      Reviewed By: yfeldblum
      
      Differential Revision: D22393040
      
      fbshipit-source-id: 920a7dfbf1d4334aecb475add96d3799c89aa211
      5b5359fe
  8. 03 Jul, 2020 2 commits
    • Giuseppe Ottaviano's avatar
      IOBufQueue::moveAsValue() · fc6f6e43
      Giuseppe Ottaviano authored
      Summary: Added for symmetry with `IOBuf::cloneAsValue()`
      
      Reviewed By: luciang
      
      Differential Revision: D21737695
      
      fbshipit-source-id: 40f5695565ffc7d7c389c9953bffc3ab85508ed1
      fc6f6e43
    • Lukas Piatkowski's avatar
      eden/scm: provide getdeps.py way of building eden/scm on GitHub · 63977f90
      Lukas Piatkowski authored
      Summary:
      In order to do what the title says, this diff does:
      1. Add the `eden/oss/.../third-party/rust/.../Cargo.toml` files. As mentioned in the previous diff, those are required by GitHub so that the third party dependencies that are local in fbsource are properly defined with a "git" dependency in order for Cargo to "link" crates properly.
      2. Changes to `eden/scm/Makefile` to add build/install commands for getdeps to invoke. Those command knowing that they are called from withing getdeps context they link the dependencies brought by getdeps into their proper places that match their folder layout in fbsource. Those Makefile commands also pass a GETDEPS_BUILD env to the setup.py invocations so that it knows it is being called withing a getdeps build.
      3. Changes to `eden/scm/setup.py` that add "thriftasset" that makes use of the getdeps.py provided "thrift" binary to build .py files out of thrift files.
      4. Changes to `distutils_rust` to use the vendored crates dir provided by getdeps.
      5. Changes to `getdeps/builder.py` and `getdeps/manifest.py` that enable more fine-grained configuratior of how Makefile builds are invoked.
      6. Changes to `getdeps/buildopts.py` and `getdeps/manifest.py` to disable overriding PATH and pkgconfig env, so that "eden/scm" builds in getdeps using system libraries rather than getdeps-provided ones (NOTE: I've tried to use getdeps provided libraries, but the trickiest bit was that Rust links with Python, which is currently not providable by getdeps, so if you try to build everything the system provided Python libraries will collide with getdeps provided ones)
      7. Added `opensource/fbcode_builder/manifests/eden_scm` for the getdeps build.
      
      Reviewed By: quark-zju
      
      Differential Revision: D22336485
      
      fbshipit-source-id: 244d10c9e06ee83de61e97e62a1f2a2184d2312f
      63977f90
  9. 02 Jul, 2020 3 commits
  10. 01 Jul, 2020 4 commits
    • Zeyi (Rice) Fan's avatar
      make fb_py_win_main to dynamically find Python3.dll · 59701670
      Zeyi (Rice) Fan authored
      Summary:
      In EdenFS's latest Windows package. We are seeing DLL import errors coming from `asyncio` as it requires a system native module `_overlapped.pyd`.
      
      The underlying cause is because when we build EdenFS CLI on Sandcastle, we are linking with Python 3.6.2. The Python36.dll shipped with the EdenFS package is also coming from that version.
      
      However, on Windows laptop. We have Python 3.6.3. Since we are not shipping the Python system libraries with us. It uses the libraries installed in the system, and it attempts to import the `_overlapped.pyd` located at `C:\Pythone36\DLLs\`. This version is compiled against Python 3.6.3, which is incompatible with the Python36.dll we are using.
      
      ----
      
      To resolve this, we need either ship an embedded copy of Python along with EdenFS, or teach EdenFS to use the Python distribution installed in the system. This commit tweaks the executable we prepend to the archive created with zipapp to locate `Python3.dll` dynamically. This allows us to use the Python installed in the system so we can avoid the version mismatch issue.
      
      With this setup, we can also be shipping an embedded Python version along with EdenFS, and the Python loader can look for that path. This is demonstrated with the relative DLL loading `..\python`.
      
      In theory, we can have a package structure like this:
      
      ```
      .
      ├── python
      │   ├── ....
      │   └── python3.dll
      └── bin
          ├── ...
          ├── edenfsctl.exe
          └── edenfs.exe
      ```
      
      Reviewed By: xavierd
      
      Differential Revision: D22325210
      
      fbshipit-source-id: 96a3f9503e7865a5f9d95710ff13f019afcf04f1
      59701670
    • Jon Maltiel Swenson's avatar
      Remove rsocket-cpp dependency from OSS projects depending on fbthrift · 0c4dd654
      Jon Maltiel Swenson authored
      Summary: fbthrift no longer depends on rsocket-cpp
      
      Reviewed By: simpkins
      
      Differential Revision: D22275231
      
      fbshipit-source-id: c1f217f1ce97591b8ebca002bf8ae3af701be641
      0c4dd654
    • Yedidya Feldblum's avatar
      Small benchmark for coarse_steady_clock · c3e79e82
      Yedidya Feldblum authored
      Summary: [Folly] Small benchmark for `coarse_steady_clock`.
      
      Differential Revision: D22320499
      
      fbshipit-source-id: dbd93e886b9e984d7469288258eefaa4a3fafb10
      c3e79e82
    • Giuseppe Ottaviano's avatar
      Fix overflow in EliasFanoReader · cf316095
      Giuseppe Ottaviano authored
      Summary:
      The upper bitvector size (in bits) can exceed the domain of `SizeType` even if the list otherwise respects the contract.
      
      Also improve some of the tests.
      
      Reviewed By: yfeldblum, philippv
      
      Differential Revision: D22304614
      
      fbshipit-source-id: bf846730c59451457d3e2d3cf14d90e3df012e0b
      cf316095
  11. 30 Jun, 2020 2 commits
    • Matthieu Martin's avatar
      Add addFiberRemote · 77a38ddb
      Matthieu Martin authored
      Summary: Complete support of the FiberManager::addTask*
      
      Reviewed By: pranavtbhat
      
      Differential Revision: D22305146
      
      fbshipit-source-id: 94e4a02f754b306d8b5fe4994832a7e2dc2cf4a5
      77a38ddb
    • Matthieu Martin's avatar
      Specify return type of async:: functions · 6284a28e
      Matthieu Martin authored
      Summary: Simplify API understanding for code reader
      
      Reviewed By: pranavtbhat
      
      Differential Revision: D22272546
      
      fbshipit-source-id: 341625ed9079f3bc4aee09851eadc02fd05b2434
      6284a28e