1. 13 May, 2020 2 commits
    • Matt Joras's avatar
      Add recvmmsg weak symbol. · ee1e6c7b
      Matt Joras authored
      Summary:
      As it turns out, to use this reliably on Android we need to define a weak symbol. The headers are only exported after API level 21, so apps that compile with 16 cannot use it with a compile time flag.
      
      Instead we can utilize a weak symbol to link with it, and check if it's there at runtime.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21535034
      
      fbshipit-source-id: 8b68a803d1e9f2f293e9f3efa16fb2a18abee53d
      ee1e6c7b
    • Yedidya Feldblum's avatar
      Explicitly enforce that RequestToken is trivially-destructible · a3e63b14
      Yedidya Feldblum authored
      Summary:
      [Folly] Explicitly enforce that `RequestToken` is trivially-destructible to permit its use as a Meyers Singleton without requiring the use of Leaky Meyers Singleton to avoid destruction-order problems.
      
      (Note: this ignores all push blocking failures!)
      
      Differential Revision: D21533605
      
      fbshipit-source-id: fe334fd5257b1d0a07efd849c4532b28801e54c4
      a3e63b14
  2. 12 May, 2020 3 commits
    • Christopher Wei's avatar
      Add virtual event base interface for getFiberManager · 7c0d945b
      Christopher Wei authored
      Summary: Adds a version of getFiberManager that takes a VirtualEventBase + FrozenOptions
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21507009
      
      fbshipit-source-id: 2bffd1704650b25e78fc388b3e033da4f2c16b9a
      7c0d945b
    • Matt Joras's avatar
      Fix recvmmsg wrapper. · d33c823f
      Matt Joras authored
      Summary: It needs to set the msg_len, otherwise the caller doesn't know how long the datagram is.
      
      Reviewed By: yangchi
      
      Differential Revision: D21512788
      
      fbshipit-source-id: f24c2bd9e86acf93ebf12c48afcf7658c8e8d54e
      d33c823f
    • Dan Melnic's avatar
      Fix sendmmsg wrapper · efb57da2
      Dan Melnic authored
      Summary: Fix sendmmsg wrapper
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21513176
      
      fbshipit-source-id: ddc1bdd93801136994ef62fd5793da7371e9e22a
      efb57da2
  3. 11 May, 2020 4 commits
    • Spencer Baumgardner's avatar
      add support for reference types · fe4c0d46
      Spencer Baumgardner authored
      Summary:
      This diff adds support for reference types to the Async abstraction.
      
      One example use-case is memoization where you generate the value asynchronously on cache-miss. You want may want to return a reference to avoid the expense of copying.
      
      Some additional context is in D21438851.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21478238
      
      fbshipit-source-id: 53e1b997abed2d40f0a2f9ab935925ad9bbc91ab
      fe4c0d46
    • JiaJie Ye's avatar
      VirtualEventBase is a SequencedExecutor · 0ef1daa2
      JiaJie Ye authored
      Summary:
      VirtualEventBase is backed by a EventBase and byitself is also sequenced, thus shoudl be an SequencedExecutor
      
      (Note: this ignores all push blocking failures!)
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21493098
      
      fbshipit-source-id: 97fd09161117e77422a5786584db89af98dfe314
      0ef1daa2
    • Xavier Deguillard's avatar
      getdeps: add filtering of tests · 7e79b25f
      Xavier Deguillard authored
      Summary:
      For large projects, with lots of tests, running all the tests can take a lot
      of time, but for quick development iteration, only a subset of the tests may
      be needed to run.
      
      On non-Windows platforms, this can be easily achieved by manually executing
      the individual tests binaries and use the builtin filtering mechanism of that
      test binary to achieve the goal. On Windows, this can quickly become
      impossible as DLLs might not be available, and the right PATH would need to
      be manually specified by hand to execute the tests binaries[0].
      
      To solve this, let's simply provide a unified way of running specific tests
      by passing in a regexp. Both testpilot and CTest do support regex to execute
      specific tests. My understanding is that cargo doesn't yet allows regex, but
      will in the future.
      
      [0]: And a missing DLLs would produce no output when executed from
           PowerShell, which makes this very confusing.
      
      Reviewed By: wez
      
      Differential Revision: D21484774
      
      fbshipit-source-id: ee32e950e25bb2a498a2b364a447955a917b0590
      7e79b25f
    • Ian Petersen's avatar
      Modify folly::coro's Malloc.cpp to support iOS <10.0 · 4103ac13
      Ian Petersen authored
      Summary:
      Sized deallocation (introduced in C++14) is only available on iOS after 10.0.
      This diff changes `folly_coro_async_free` to use `operator delete(void*)` when
      `operator delete(void*, size_t)` is unavailable.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21492467
      
      fbshipit-source-id: 2afef1d3d2aa6c70fec5f49e5d62a4a594e6da83
      4103ac13
  4. 10 May, 2020 1 commit
  5. 09 May, 2020 1 commit
    • Xavier Deguillard's avatar
      getdeps: silence inherits via dominance warnings · aa8735e9
      Xavier Deguillard authored
      Summary:
      Previously, the Windows build was litered with warnings of the form (typo included):
      
        warning C4250: 'C1': inherits 'C2::C2::method' via dominance
        note: see declaration of 'C2::method'
      
      Microsoft doesn't offer any recommendation, and the internet suggest that the
      right `using` should silence it. That's unfortunately not the case, adding:
        using C2::method
      In `C1` doesn't do anything, and the compiler still complains :(.
      
      Since the warning appears to be non-actionable, and looks more like a
      "notice" than a warning, let's just silence it.
      
      Reviewed By: wez
      
      Differential Revision: D21395095
      
      fbshipit-source-id: ae661b3ed61303e6361b8a15d9e7c6b9627ea8c1
      aa8735e9
  6. 08 May, 2020 6 commits
    • Wez Furlong's avatar
      getdeps: try harder to avoid ccache on sandcastle · 47e3c03f
      Wez Furlong authored
      Summary:
      A small number of hosts are impacted by some local misconfiguration like this:
      
      ```
      ccache /usr/bin/c++  -DHAVE_PCLMUL -DHAVE_SSE42 -DOS_LINUX -DROCKSDB_FALLOCATE_PRESENT -DROCKSDB_LIB_IO_POSIX -DROCKSDB_MALLOC_USABLE_SIZE -DROCKSDB_NO_DYNAMIC_EXTENSION -DROCKSDB_PLATFORM_POSIX -DROCKSDB_PTHREAD_ADAPTIVE_MUTEX -DROCKSDB_RANGESYNC_PRESENT -DROCKSDB_SCHED_GETCPU_PRESENT -DROCKSDB_SUPPORT_THREAD_LOCAL -DSNAPPY -I/var/sandcastle/temp/fbcode_builder_getdeps/extracted/rocksdb-v6.8.1.tar.gz/rocksdb-6.8.1 -I/var/sandcastle/temp/fbcode_builder_getdeps/extracted/rocksdb-v6.8.1.tar.gz/rocksdb-6.8.1/include -I/var/sandcastle/temp/fbcode_builder_getdeps/extracted/rocksdb-v6.8.1.tar.gz/rocksdb-6.8.1/third-party/folly -isystem /var/sandcastle/temp/fbcode_builder_getdeps/extracted/rocksdb-v6.8.1.tar.gz/rocksdb-6.8.1/third-party/gtest-1.8.1/fused-src -isystem /var/sandcastle/temp/fbcode_builder_getdeps/installed/snappy-8kzrVQJKWlYwtUKTyQTv0R3_bltgjXecrUEMwSpi-vI/include -W -Wextra -Wall -Wsign-compare -Wshadow -Wno-unused-parameter -Wno-unused-variable -Woverloaded-virtual -Wnon-virtual-dtor -Wno-missing-field-initializers -Wno-strict-aliasing -std=c++11 -fno-omit-frame-pointer -momit-leaf-frame-pointer -fno-builtin-memcmp -O2 -g -DNDEBUG -MD -MT CMakeFiles/rocksdb.dir/cache/lru_cache.cc.o -MF CMakeFiles/rocksdb.dir/cache/lru_cache.cc.o.d -o CMakeFiles/rocksdb.dir/cache/lru_cache.cc.o -c /var/sandcastle/temp/fbcode_builder_getdeps/extracted/rocksdb-v6.8.1.tar.gz/rocksdb-6.8.1/cache/lru_cache.cc
      ccache: error: /home/facebook/.ccache/ccache.conf: No such file or directory
      ```
      
      let's try to avoid it!
      
      Reviewed By: simpkins
      
      Differential Revision: D21460408
      
      fbshipit-source-id: 95b1fe019ff36eb8acb8476af997c220a7947436
      47e3c03f
    • Lukas Piatkowski's avatar
      rust: remove abandoned heapsize crate · 9285064e
      Lukas Piatkowski authored
      Summary: This also unblocks the MacOS Mononoke builds, so enabling them back
      
      Reviewed By: farnz
      
      Differential Revision: D21455422
      
      fbshipit-source-id: 4eae10785db5b93b1167f580a1c887ee4c8a96a2
      9285064e
    • Lukas Piatkowski's avatar
      mononoke/fbcode_builder: bring back mononoke manifests for getdeps build · a0458616
      Lukas Piatkowski authored
      Summary:
      The breakage has been fixed, so bring back the manifest, but only the Linux one, because the Mac version is failing due to another issue.
      
      Also to make it easier to debug issues on GitHub Actions separate out the dependencies build from Mononoke and rust-shed builds.
      
      Reviewed By: krallin
      
      Differential Revision: D21448412
      
      fbshipit-source-id: 68d89c858d1692727a7fd66bca114920e6dfb4dc
      a0458616
    • Nathan Bronson's avatar
      callWithExtractedKey fix for pair of lvalue ref, improvements · 7d618e4f
      Nathan Bronson authored
      Summary:
      callWithExtractedKey previously incorrectly translated
      std::pair<X&,Y&>&& into X&&, Y&&, when reference collapsing rules call
      instead for X& and Y&.  This diff fixes the problem, adds unit tests,
      and also targets the 2-argument std::pair constructor when possible.
      This means that callWithExtractedKey can be safely used on top of
      libc++'s std::unordered_map::emplace without disabling the key-finding
      optimization in that method.  (That lib's emplace is clever enough to
      be able to search on the key before constructing a node, but does not
      include the logic to unwrap the std::pair 3-arg piecewise constructor.)
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21419822
      
      fbshipit-source-id: d3c61e45d7fe7d8c6805342bbe63a8c866f4e306
      7d618e4f
    • Maged Michael's avatar
      ConcurrentHashMap: Add clarifying comments · fa2e7912
      Maged Michael authored
      Summary: Add comments about the performance of frequent removal and mention clear() in the comment about the timing of destruction of removed key-value pairs.
      
      Reviewed By: davidtgoldblatt
      
      Differential Revision: D21455366
      
      fbshipit-source-id: be9f80136260f2de3b68d6fff54ccdadc5544046
      fa2e7912
    • Lewis Baker's avatar
      Add folly::coro::AsyncScope · b7457e50
      Lewis Baker authored
      Summary:
      This class makes it easier to spawn an unbounded number of tasks
      over time within a given scope and then later wait for all of
      those tasks to complete. e.g. during shutdown.
      
      Reviewed By: kirkshoop
      
      Differential Revision: D21391413
      
      fbshipit-source-id: c91c66aa61a8ae30e04d0608426037d8adba14d7
      b7457e50
  7. 07 May, 2020 7 commits
    • Adam Simpkins's avatar
      update AsyncFileWriter to avoid writevFull() on Windows · 8c357b33
      Adam Simpkins authored
      Summary:
      Change folly/logging's AsyncFileWriter code to avoid calling writevFull() on
      Windows.  On POSIX system we use this function to minimize the number of
      system calls we make.  However, on Windows there is no `writev()` function
      provided by the system, so folly emulates this by performing multiple separate
      `write()` calls.  Given that we are just going to perform multiple `write()`
      calls there is no reason to construct the iobuf data structure here.
      
      Differential Revision: D21367424
      
      fbshipit-source-id: 8f6ba9d5fb36318a33d293a416008cd7fa03cb70
      8c357b33
    • Adam Simpkins's avatar
      update AsyncLogWriter::performIO() to take the vector by reference · f121101a
      Adam Simpkins authored
      Summary:
      Update the `AsyncLogWriter::performIO()` API to take the vector of messages by
      const reference, rather than by modifiable pointer.  This parameter is purely
      an input parameter, and shouldn't be modified by the `performIO()` code, so it
      makes more sense to take this as const reference.
      
      Differential Revision: D21367425
      
      fbshipit-source-id: 4bbf3a58dd48bb620101ec3982b189bea656b301
      f121101a
    • Andrii Grynenko's avatar
      Fix blockingWait to schedule task on the DrivableExecutor · 439c7b22
      Andrii Grynenko authored
      Summary: The first portion of the task (before the first suspension point) was run inline.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21460275
      
      fbshipit-source-id: 31d50e15d5890fbc7127dfff424a4e814dcb2320
      439c7b22
    • Yedidya Feldblum's avatar
      copy_to_unique_ptr, copy_to_shared_ptr · 34ea33d2
      Yedidya Feldblum authored
      Summary: [Folly] `copy_to_unique_ptr`, `copy_to_shared_ptr` - type-deducing wrappers around `make_unique` and `make_shared`.
      
      Reviewed By: vitaut
      
      Differential Revision: D21332222
      
      fbshipit-source-id: f41dac0fcfa972118d3a721194c941e14511a80c
      34ea33d2
    • Ravindra Sunkad's avatar
      Update FBOSS OSS build to link with v4.12.0 of iproute2 · 4fffe6ad
      Ravindra Sunkad authored
      Summary:
      fboss-oss build links to hash that corresponds to tag v4.4.0 released on Jan 11 2016
      
      ```
      repo_url = https://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git
      rev = 92a0236a3cdf3438000834121b7ea8a09f1f52b1
      ```
      
      The change is to update the iproute2 version to ```v4.12.0``` (July 5 2017) to match with the version used internally to Facebook
      
      Reviewed By: shri-khare
      
      Differential Revision: D21411714
      
      fbshipit-source-id: fac606396e284193bb7199cf60d2601594bfa5f0
      4fffe6ad
    • Hasnain Lakhani's avatar
      Add manifest for fatal · 15cf7e67
      Hasnain Lakhani authored
      Reviewed By: wez
      
      Differential Revision: D21417542
      
      fbshipit-source-id: f7f0726e84db3dc903bf11c9045a9975a3e90234
      15cf7e67
    • Adam Simpkins's avatar
      define FOLLY_XLOG_STRIP_PREFIXES correctly for all targets · 2c064a01
      Adam Simpkins authored
      Summary:
      The `FOLLY_XLOG_STRIP_PREFIXES` macro tells folly::logging how to strip off
      the absolute portion of the path when automatically computing log category
      names from the source file name.
      
      Previously this was set in FollyCompilerUnix.cmake, which caused it to only be
      applied on Linux/Mac platforms, and only for targets that explicitly used
      `apply_folly_compile_options_to_target()`.
      
      This changes the code to apply this definition with a directory property, so
      it affects all targets under the top-level directory, and it is also applied
      on Windows.
      
      Reviewed By: wez
      
      Differential Revision: D21333014
      
      fbshipit-source-id: 9256e14ab4fd03d19f28957e7ca298c930a22b18
      2c064a01
  8. 06 May, 2020 7 commits
    • Kyle Nekritz's avatar
      Back out "Give ownership of new session callbacks to SSLContext" · 82070c96
      Kyle Nekritz authored
      Summary:
      Original commit changeset: f1dc8bd17150
      
      This accidentally enabled the openssl internal session cache which can use
      excessive memory.
      
      Reviewed By: jalopezsilva, mingtaoy
      
      Differential Revision: D21433514
      
      fbshipit-source-id: 91358f6fd2decc0d18eff7ca8194b677f3c8629c
      82070c96
    • Adam Simpkins's avatar
      build and run FileUtilTest in CMake builds · 6808eb56
      Adam Simpkins authored
      Summary:
      Remove the `HANGING` tag from this test, so that it is run by default in the
      CMake build.
      
      Differential Revision: D21367512
      
      fbshipit-source-id: 4bb707c83005ee8cd39ed2ce6b82b7f0c53f2c88
      6808eb56
    • Adam Simpkins's avatar
      disable the writeFileAtomic() function on Windows · 06302af7
      Adam Simpkins authored
      Summary:
      This puts the `writeFileAtomic()` function behind and `#ifndef _WIN32` check,
      to disable it on Windows.
      
      While this function currently compiles on Windows, it does not behave
      correctly: it often fails with "Permission denied" errors when trying to use
      folly's emulated `fchmod()` function.  Additionally, even the `fchmod()`
      succeeds (or is skipped) it fails if the destination path already exists, as
      `rename()` on Windows does not allow replacing an existing file.  While we
      could attempt to replace the existing file, doing so would not be atomic.  We
      might be able to achieve this with transactional NTFS features if the
      underlying file system is NTFS, but according to Microsoft these APIs are
      discouraged and may be removed in future versions of Windows.
      
      Therefore for now simply disable `writeFileAtomic()` on Windows.
      
      Differential Revision: D21367511
      
      fbshipit-source-id: 50a5a1004feacc8bdb0a24421bea339962e1d546
      06302af7
    • Adam Simpkins's avatar
      build the logging example in CMake builds · 6af9edb3
      Adam Simpkins authored
      Summary:
      Add a CMakeLists.txt file to build the logging example program in CMake
      builds.
      
      This also disables the `AsyncFileWriter.fork` test on Windows, since `fork()`
      isn't available on Windows.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21332968
      
      fbshipit-source-id: b578df92576e79d34b7fe2bf79dc8f30e0fd6d6b
      6af9edb3
    • Adam Simpkins's avatar
      update the logging example to reflect the current default level · 282cc3f6
      Adam Simpkins authored
      Summary:
      D8941725 changed the default log level from WARNING to INFO.  This updates the
      comments in the example logging program to reflect that.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21332969
      
      fbshipit-source-id: 88b206bf4dc891d8868a34ea02d91cc79b7c5c48
      282cc3f6
    • Wez Furlong's avatar
      getdeps: fix builds with cached project downloads · aef5dd9e
      Wez Furlong authored
      Summary:
      D21364132 accidentally broke this; we can't run the fetcher
      for projects for which we pulled the build out of cache, because there
      is no source to update in that case.
      
      This commit adjusts the logic so that we write out a marker file
      to indicate that we installed a build from cache and to look for
      that file being present to gate the new update logic.
      
      Reviewed By: lnicco
      
      Differential Revision: D21419122
      
      fbshipit-source-id: 304670848add22531d88549d66f22c40ff255140
      aef5dd9e
    • Dan Melnic's avatar
      Return 0 from PollIoBackend::eb_event_add if the event type is not set · 911311ab
      Dan Melnic authored
      Summary: Return 0 from PollIoBackend::eb_event_add if the event type is not set
      
      Reviewed By: danobi
      
      Differential Revision: D21416811
      
      fbshipit-source-id: 5825e473dca06d33017785e4d73947aade5f554c
      911311ab
  9. 05 May, 2020 9 commits
    • Maged Michael's avatar
      hazptr: Add warnings for pathological performance. · 831d84b5
      Maged Michael authored
      Summary:
      Add warnings for pathological cases:
      - Too large retired lists
      - Thread cache overflow
      
      Move the warning for executor backlog to a separate function for clearer profiling.
      
      Reviewed By: davidtgoldblatt
      
      Differential Revision: D21314798
      
      fbshipit-source-id: e07d6182309a0940adbdf76774a9c44fb1aa200c
      831d84b5
    • Andrew Huang's avatar
      Give ownership of new session callbacks to SSLContext · 2ee65365
      Andrew Huang authored
      Summary: Giving SSLContext ownership over the new session callback attached to SSL_CTX, so that it may do additional work with the new session (i.e. forward the session to its corresponding abstracted SSLSession).
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21021333
      
      fbshipit-source-id: f1dc8bd17150d45498ae1c22ef4eed1d2ca00c01
      2ee65365
    • Andrew Huang's avatar
      Add SSLContext to OpenSSL SSL_CTX ex data · d908fe0b
      Andrew Huang authored
      Summary: Adding SSLContext members to SSL_CTX ex data to access them inside callbacks attached to SSL_CTX (e.g. session callbacks)
      
      Reviewed By: yfeldblum, mingtaoy
      
      Differential Revision: D21021353
      
      fbshipit-source-id: 6aa3995f7d719ca7e87bad798876a92dd5765b86
      d908fe0b
    • Lewis Baker's avatar
      Clarify documentation in folly::coro::Task about when RequestContext is captured · aee68ea3
      Lewis Baker authored
      Summary:
      Clarify that the RequestContext is captured at the time the coroutine is
      started (e.g. due to co_awaiting it or calling .start()) rather than at
      the time the coroutine is called.
      
      Reviewed By: andriigrynenko
      
      Differential Revision: D21353947
      
      fbshipit-source-id: 40ec7b25d84e96a611a1f63b6184a0d6a016948d
      aee68ea3
    • Shannon Zhu's avatar
      Convert type check targets in folly to use configuration · c230732e
      Shannon Zhu authored
      Summary:
      Migrating buck integration to use configurations, which is more performant and is the primary supported way to run Pyre.
      Mypy is no longer supported at Facebook, so mypy buck integration is covered in this migration as well.
      
      Reviewed By: grievejia
      
      Differential Revision: D21398810
      
      fbshipit-source-id: 12c0cd395e243703f4380b31527d0c202d58cec2
      c230732e
    • Adam Simpkins's avatar
      implement readvFull() and writevFull() using read()/write() on Windows · b88ccd9e
      Adam Simpkins authored
      Summary:
      On Windows, implement `readvFull()` and `writevFull()` by calling `read()` and
      `write()` repeatedly, rather than `readv()` and `writev()`.
      
      Windows does not provide native implementations of `readv()` and `writev()`,
      so the code was previously using implementations provided by folly in
      `folly/portability/SysUio.cpp`.  The folly implementations attempted to lock
      the file to provide similar atomicity guarantees that `readv()` and `writev()`
      provide on POSIX.  However, callers of `readvFull()` and `writevFull()` don't
      care about this atomicity (since they are willing to do multiple `readv()` and
      `writev()` calls if necessary).  Therefore this locking behavior is not
      needed.  Locking in this case is undesirable because it adds extra overhead,
      and fails in some cases.  For instance, attempting to lock stderr blocks for 9
      seconds before eventually failing.
      
      Differential Revision: D21367446
      
      fbshipit-source-id: b2ae3c9c5da977402336c750d3d21ba9825527d9
      b88ccd9e
    • Dan Melnic's avatar
      Use zerocopy only if all the IOBufs in the chain have a SharedInfo · e5c56348
      Dan Melnic authored
      Summary: Use zerocopy only if all the IOBufs in the chain have a SharedInfo
      
      Reviewed By: knekritz
      
      Differential Revision: D21342852
      
      fbshipit-source-id: 458c32ef336b76ed0dc226ea670b7c04c1427bee
      e5c56348
    • Andrew Huang's avatar
      Handle session resumption with abstracted SSLSession · 3bd6e353
      Andrew Huang authored
      Summary: Add session resumption functionality in AsyncSSLSocket for the new abstracted SSLSession. Add logic to resume sessions correctly no matter which API is used, allowing for a piecewise transition to using the abstracted SSLSession.
      
      Reviewed By: mingtaoy
      
      Differential Revision: D20600380
      
      fbshipit-source-id: bb81d5be95ccaa6486a7431391676f3e5c0b5b8e
      3bd6e353
    • Nate Stedman's avatar
      Fix some broken documentation comments · 1972a4b5
      Nate Stedman authored
      Summary: I temporarily enabled Clang's `-Wdocumentation` flag, which reports things like using an `param` for a parameter that doesn't exist. [Full explanation with examples here](https://clang.llvm.org/docs/DiagnosticsReference.html#wdocumentation).
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21396362
      
      fbshipit-source-id: d7b34bc14af7bb1b0901a8f195c02674eb587fbc
      1972a4b5