1. 29 Apr, 2020 5 commits
    • Wez Furlong's avatar
      getdeps: split monolithic GH actions CI spec up (#1360) · 43eb92d9
      Wez Furlong authored
      Summary:
      Rather than have a single main.yml file containing all off the different
      builds, split that up so that we have one file per build environment
      (linux, mac, windows).
      
      This has a couple of advantages:
      
      * It is quicker to see the status of just one of the platforms
      * Artifact collection for one platform is not blocked pending completion
        of the builds for all systems
      * It's a little easier to understand what is happening for a single platform
      
      To support having multiple files I've changed the output-file option to
      be an output-dir.
      
      I've included the rm of main.yml in this commit.  Once this gets
      imported back to the FB internal system I'll amend in an update to
      the helper script that updates all of our opensource builds and run
      and amend that.
      Pull Request resolved: https://github.com/facebook/folly/pull/1360
      
      Test Plan:
      the GH action status on this PR should show three different
      actions running, one for each platform.
      
      I updated and ran
      `fbcode/opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh`
      to regenerate all the actions files for FB.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21310991
      
      Pulled By: wez
      
      fbshipit-source-id: 604ef652c8f746781a4b410c6b996cdee4524e0d
      43eb92d9
    • Wez Furlong's avatar
      fb_py_win_main.c: fix File Not Found errors on windows · b2003abb
      Wez Furlong authored
      Summary:
      This commit resolves an issue with our zipapp executables
      on Windows that meant that the only reliable way to start them was
      to use the fully path to the executable.
      
      The root cause is that the __wargv array is produced by parsing the
      process command line into an array, and if you ran `watchman-wait -h`
      __wargv[0] would have `watchman-wait` rather than the fully qualified
      path to the executable that the zipapp plumbing requires.
      
      The fix is to ask the system for the fully qualified path and ensure that
      that gets set as both argv[0] AND argv[1].
      
      Reviewed By: xavierd
      
      Differential Revision: D21190350
      
      fbshipit-source-id: eeb95084592d30a028a93b2b03877f8cc6c72729
      b2003abb
    • Wez Furlong's avatar
      GH actions: fixup boost windows environment (#1359) · fca12003
      Wez Furlong authored
      Summary:
      The environment changed since I tested  D20740410 and now it appears
      that we'll need to re-export a versioned variable in order for cmake
      to detect boost on the GH actions hosts.
      Pull Request resolved: https://github.com/facebook/folly/pull/1359
      
      Test Plan:
      the GH actions status of this diff.
      
      The workflow was updated via:
      
      ```
      python3 build/fbcode_builder/getdeps.py generate-github-actions folly --output-file .github/workflows/main.yml
      ```
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21307640
      
      Pulled By: yfeldblum
      
      fbshipit-source-id: 1555cbcade822775379cd9054be37fdbc17b4d93
      fca12003
    • Wez Furlong's avatar
      getdeps: allow satisfying deps from system packages · 5de9cfc6
      Wez Furlong authored
      Summary:
      From the outset, we wanted to be sure that getdeps was able
      to source and build the dependencies so that we knew that we'd have
      a repeatable build.  This came at the cost of build times: having
      to build boost on each CI run is a bit of a chore.
      
      This commit adds three new elements to the manifest files:
      
      * `rpms` - a list of RPM names that are all required to be present
        in order to consider the dependency satisfied
      * `debs` - like `rpms` above, but scoped to debian package names
      * `preinstalled.env` - a list of environment variables that if they
        are all set and non-empty will satisfy the dependency.
      
      A new `--allow-system-packages` option to getdeps enables the new
      logic that looks for system packages; it is off by default, but
      enabled in the generated GitHub Actions workflows.
      
      A new `install-system-deps` subcommand is provided that will attempt
      to install the system packages needed to satisfy the build.  This
      typically needs to be run via sudo and is thus broken out separately
      from the main getdeps build flow.
      
      I made a pass over the manifest files and added package names that
      satisfy the build on ubuntu-18 and fedora-31.
      
      shri-khare: I renamed the `Python3.7.6` manifest to just `python` as
      part of this change; the version of python that it pulls in through
      the normal build is the same and I believe that an equal or newer
      version of python3 is available in the GH actions builder.
      
      The `preinstalled.env` is used only by the boost manifest: it references
      the name of an environment variable that is set by the github
      windows hosts and that points to a pre-built and pre-installed
      copy of boost.  Since there is no package manager that we can
      easily query for this sort of thing, probing from the environment
      seems like a reasonable and fast way to check for this.  We
      may need to evolve this over time to become more feature rich,
      but this seems like a good starting point.
      
      This commit has the potential to save 20 minutes of build time
      from each public CI build just due to the boost dependency alone!
      
      Refs: https://github.com/facebook/watchman/pull/797
      
      Reviewed By: yfeldblum
      
      Differential Revision: D20740410
      
      fbshipit-source-id: 6c38019449c54465127656c3d18a6ff1f30adaea
      5de9cfc6
    • Xavier Deguillard's avatar
      Explicitly cast to long in chrono::durationToPosixTime · 3c553b49
      Xavier Deguillard authored
      Summary:
      The MSVC compiler was complaining that an implicit conversion from intmax_t
      to long was being performed. Let's use static_cast to silence it.
      
      Reviewed By: simpkins
      
      Differential Revision: D21267752
      
      fbshipit-source-id: e466f107ca8e861c12e0abd1f8fa7448f22adfd9
      3c553b49
  2. 28 Apr, 2020 3 commits
    • Xavier Deguillard's avatar
      Do not use int as the return type for distance_if_multipass · 81b08b7e
      Xavier Deguillard authored
      Summary:
      The MSVC compiler complains about a potential lossy conversion happening,
      which feels mostly right due to Iterators being potentially far apart.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21280438
      
      fbshipit-source-id: 8fcb18d865688aa1fae869a41256ad3dc3ad2a97
      81b08b7e
    • Yedidya Feldblum's avatar
      identity · c9f2df1b
      Yedidya Feldblum authored
      Summary: [Folly] `identity`, an initialized function object matching `Identity`, which is renamed `identity_fn`.
      
      Reviewed By: markisaa, lewissbaker
      
      Differential Revision: D21265889
      
      fbshipit-source-id: 5cc9f2e624da9a4acd0f65985ff10a4dd306efdc
      c9f2df1b
    • Mingtao Yang's avatar
      Update test certs · bb79aaa5
      Mingtao Yang authored
      Summary:
      The test cert "test-cert.pem" is signed with a SHA1 based signature algorithm, and newer distros such as
      CentOS 8 will fail loading this certificate due to its default security policy.
      
      This diff regenerates the test certificates, following the old certificates
      names as closely as possible. I included the script to generate these in case
      they need to be changed in the future.
      
      Reviewed By: AjanthanAsogamoorthy
      
      Differential Revision: D21271202
      
      fbshipit-source-id: de4227ae81dfffaf00f8d355e83936db6eb5fe31
      bb79aaa5
  3. 27 Apr, 2020 3 commits
    • Adam Simpkins's avatar
      fix how folly and its downstream projects find boost · 6e85117e
      Adam Simpkins authored
      Summary:
      Folly links explicitly links against boost statically on Windows.  However,
      many downstream projects that use folly independently performed their own
      discovery of boost and normally used the default option of linking against it
      dynamically.  This resulted in boost being listed twice on the link line on
      Windows for some downstream projects: both its static libraries and dynamic
      libraries would be present.  This ends up causing linking failures due to
      duplicate definitions.
      
      This updates folly's installed CMake file to correctly call
      `find_dependency(Boost)` so that downstream projects don't need to perform
      their own independent discovery.  This ensures that all downstream projects
      use a consistent method of linking against Boost.  I updated many downstream
      projects to remove their explicit separate discovery, and rely only on this
      behavior from folly.
      
      I also added a configuration setting to allow explicitly selecting whether to
      link against boost statically at folly configure time.
      
      Reviewed By: wez
      
      Differential Revision: D21232164
      
      fbshipit-source-id: 9ecc3ce988add48905252297e979403c42e7e148
      6e85117e
    • Pranav Thulasiram Bhat's avatar
      Async annotation wrapper · 4162d2f4
      Pranav Thulasiram Bhat authored
      Summary:
      ## Why is this diff needed?
      This diff presents an annotation framework designed to address some of the issues we've had working with fibers in NodeAPI:
      - It's difficult to identify functions doing I/O. As a result, we've had I/Os happening sequentially in for loops.
      - It's difficult to tell if a function is running on fibers. This is especially hard in large code bases such as ours
      
      ## What does this diff do?
      The `Async<T>` type can be used as the return type of a function. This accomplishes a couple of things:
      - It indicates that this function should run on fibers and that a fiber can suspend in this function (or in one of it's children)
      - It forces calling functions to `await` the result of this function. When coupled with a good lint rule, this enforces annotations right to the top of the stack.
      
      Reviewed By: A5he
      
      Differential Revision: D21159049
      
      fbshipit-source-id: ee922093b140b22d8e7a7587b87aa0e783055b6c
      4162d2f4
    • Zeyi (Rice) Fan's avatar
      directly reference vendored Rust crates instead of copying it · f5d380b4
      Zeyi (Rice) Fan authored
      Reviewed By: wez
      
      Differential Revision: D21238976
      
      fbshipit-source-id: 13a6def8d6c8d25057409956cc883589b205cd04
      f5d380b4
  4. 25 Apr, 2020 1 commit
  5. 24 Apr, 2020 3 commits
    • Wez Furlong's avatar
      getdeps: partially educate getdeps about EdenFS on Windows · d8e90515
      Wez Furlong authored
      Summary:
      I noticed that copytree was taking forever and realized
      that it wasn't issuing a prefetch call so I started looking in here;
      this commit teaches getdeps how to recognize and EdenFS repo on
      Windows but skips calling prefetch on Windows.
      
      Currently the prefetch implementation triggers some very slow
      processing in mercurial that is slower to start than just
      enumerating the files in the opensource build.
      
      It turned out that my original problem was just that my credentials
      had expired and we weren't surfacing that error on Windows yet.
      
      Reviewed By: simpkins
      
      Differential Revision: D20755905
      
      fbshipit-source-id: 8d3695cdd1f04199d1d409895482b8c706285d5f
      d8e90515
    • Eric Niebler's avatar
      Some portability fixes discovered with gcc-10 -fcoroutines · ae4d15fd
      Eric Niebler authored
      Summary:
      By and large, trying to test folly::coro with gcc-10's (very green) coroutines implementation was a failure, but it did turn up a couple of issues:
      
      * Friends of friends are not friends with gcc
      * -Wshadow picks up a few more instances of shadowing
      * `std::exchange` needs `#include <utility>`.
      
      Reviewed By: lewissbaker
      
      Differential Revision: D21190145
      
      fbshipit-source-id: d3f3148baa26bb8a2c0c1954fc930fd8124672e1
      ae4d15fd
    • Lukas Piatkowski's avatar
      mononoke/configerator structs: add shipit and autocargo configs for all configerator structs · 78efaa67
      Lukas Piatkowski authored
      Summary: The configerator structs are used in many top level functions in Mononoke and are required in order to build all the code on github
      
      Reviewed By: ahornby
      
      Differential Revision: D21130546
      
      fbshipit-source-id: 7f17d92173f5ecf7c3406ae4202359a0db8df84a
      78efaa67
  6. 23 Apr, 2020 5 commits
    • Lucas Dobson-Defenbaugh's avatar
      Fix deprecation warnings for collectXSemiFuture · b646e5df
      Lucas Dobson-Defenbaugh authored
      Summary:
      We're recommending collectAny as a replacement for
      collectAllSemiFutures :(
      
      Reviewed By: LeeHowes
      
      Differential Revision: D21197163
      
      fbshipit-source-id: 0055e0feb5cc74a767370bc1dc9fe8e4d8dffcfc
      b646e5df
    • Dan Melnic's avatar
      Let AsyncUDPSocket send to multiple addresses · 43b0911a
      Dan Melnic authored
      Summary:
      Add support for variable number of addrs so we can send data to multiple destinations.
      
      (Note: this ignores all push blocking failures!)
      
      Reviewed By: kevin-vigor
      
      Differential Revision: D21032857
      
      fbshipit-source-id: f73b98d44f5d7d92f3692dfddb9b1c76ebcc51c5
      43b0911a
    • Andrew Doran's avatar
      Disable TSAN instrumentation across StaticMetaBase::onThreadExit(). · d08b8a41
      Andrew Doran authored
      Summary:
      Disable TSAN instrumentation across StaticMetaBase::onThreadExit() to avoid a shortcoming in TSAN on Linux that produces sprurious failures.
      
      This was recently worked around by ignoring TSAN failures in ld-linux's _dl_deallocate_tsd() (D20597534), but that turns out to be insufficient, because other TSAN failures can occur on the TLS area after the dynamic linker has called free() on it.  This diff backs out the change from D20597534, and works around the problem in a more direct & complete way.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D20671620
      
      fbshipit-source-id: 05a74e709b620391f18467c7a5e4d14dcf9f4a1e
      d08b8a41
    • Michael Park's avatar
      Fix inaccurate deprecation message for `collectAllSemiFuture`. · bd44e567
      Michael Park authored
      Summary: Fixes a copy-paste mistake from the `collectAnySemiFuture` deprecation message.
      
      Reviewed By: Orvid
      
      Differential Revision: D21199692
      
      fbshipit-source-id: e61b740ea238dcbd6ca8481e1ba578a5073fe7bf
      bd44e567
    • Yedidya Feldblum's avatar
      Remove virtual inheritance in AsyncSocket · 97abe3c9
      Yedidya Feldblum authored
      Summary: [Folly] Remove unused virtual inheritance in `AsyncSocket`.
      
      Differential Revision: D20966977
      
      fbshipit-source-id: 502ab00438eebed0d8ebcf32ef602af50b05df47
      97abe3c9
  7. 22 Apr, 2020 3 commits
    • James Donald's avatar
      Fix -Winvalid-return warnings in Singleton · 1864bd5f
      James Donald authored
      Summary:
      Resolves these warnings:
      ```
      folly\Singleton.cpp(102,1): warning: function declared 'noreturn' should not return [-Winvalid-noreturn]
      }
      ^
      folly\Singleton.cpp(108,1): warning: function declared 'noreturn' should not return [-Winvalid-noreturn]
      }
      ^
      folly\Singleton.cpp(126,1): warning: function declared 'noreturn' should not return [-Winvalid-noreturn]
      }
      ^
      folly\Singleton.cpp(134,1): warning: function declared 'noreturn' should not return [-Winvalid-noreturn]
      }
      ^
      folly\Singleton.cpp(145,1): warning: function declared 'noreturn' should not return [-Winvalid-noreturn]
      }
      ```
      
      Reviewed By: mzlee, yfeldblum
      
      Differential Revision: D21130719
      
      fbshipit-source-id: a0f198d69c4fde44f9a5250b6915e36b6ee68642
      1864bd5f
    • Chris Sarbora's avatar
      Correctly handle Android not having malloc_usable_size below API17 · 5ead4c5e
      Chris Sarbora authored
      Summary: Android doesn't expose `malloc_usable_size` below API-level 17. Correctly reflect this in folly-config.h, and then ensure that all code paths that use `malloc_usable_size` are unreachable when compiling without it.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D20721326
      
      fbshipit-source-id: d60b3caa1910dc2bf8ed0844177f5cb93a9fd067
      5ead4c5e
    • Chris Sarbora's avatar
      Let small_vector pop_back a non-assignable type · eb877c83
      Chris Sarbora authored
      Summary: Relying on `erase()`, which must be able to move elements when erasing in front of other valid elements, causes compilation failures when trying to erase non-assignable elements at the end of the vector even though no elements actually need to be moved (and thus assigned) in that case. Here we help the compiler out and extract the destruction/resizing logic away from the moving logic.
      
      Reviewed By: yfeldblum, ot, mkatsevVR
      
      Differential Revision: D20718712
      
      fbshipit-source-id: 51c4e6636bc33577494c0c10e95da5f92b58d4b6
      eb877c83
  8. 21 Apr, 2020 4 commits
  9. 20 Apr, 2020 1 commit
    • Andrew Krieger's avatar
      Change DCHECK to assert to remove glog dep from Range · 97fc9b61
      Andrew Krieger authored
      Summary:
      This one DCHECK is the only thing keeping Range from
      being free of external deps. Lets make it not be a thing, so
      people are more inclined to pull in folly because Range is an
      easy gateway class to the rest of folly.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21121743
      
      fbshipit-source-id: 2a54e94074135c951526c50d4bb63ebf2076a72f
      97fc9b61
  10. 18 Apr, 2020 2 commits
    • Jonathan Sailor's avatar
      update public README about supported platforms, compilers · fb0e252e
      Jonathan Sailor authored
      Summary:
      The README stated "folly requires gcc 5.1+ and a version of boost..." which
      some people might thing means that folly only supports gcc.
      
      Edit the README with more details about supported compilers and platforms.
      I tried to strike a balance between informative (FB uses folly on many
      platforms), conservative (the CMake build might not support all of them),
      and not mentioning specific versions (likely to get outdated.)
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21093741
      
      fbshipit-source-id: 3a541f22b30736201361d2a10d861d61bd278e47
      fb0e252e
    • Scott Wolchok's avatar
      Convert throw in IPAddress to throw_exception · 4ee17e23
      Scott Wolchok authored
      Summary: Unbreak -fno-exceptions support for this file.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D21043400
      
      fbshipit-source-id: aceea45c32d143c723ad6bae68a7403f14d7e0f2
      4ee17e23
  11. 17 Apr, 2020 1 commit
  12. 16 Apr, 2020 4 commits
    • Andrew Huang's avatar
      Implement abstracted SSL session in folly · 52a8633f
      Andrew Huang authored
      Summary: Repurpose the existing `folly::ssl::SSLSession` to be an abstraction for SSL sessions, with the internal implementation hidden. The intention is to soon move to using these abstracted sessions instead of directly using OpenSSL sessions.
      
      Reviewed By: yfeldblum, mingtaoy
      
      Differential Revision: D20143848
      
      fbshipit-source-id: 01e175668c0afcf8b91d19a289368496717d3150
      52a8633f
    • Maged Michael's avatar
      RequestContext: Fix overwrite of null data · 0e29494c
      Maged Michael authored
      Summary:
      Fix a bug in overwrite of null data.
      
      Null old data should be erased so that the insert of the new data can succeed.
      
      Reviewed By: davidtgoldblatt
      
      Differential Revision: D20908574
      
      fbshipit-source-id: 6873d9c76daf1289f1d5b97b001c21473c41ea3f
      0e29494c
    • Dan Melnic's avatar
      Fix typo · e1cdd680
      Dan Melnic authored
      Summary:
      Fix typo
      
      (Note: this ignores all push blocking failures!)
      
      Reviewed By: mokomull
      
      Differential Revision: D21056053
      
      fbshipit-source-id: 3bf4295f9c6be1c50cdc64f6bbdf1b34cce6bece
      e1cdd680
    • Adam Simpkins's avatar
      getdeps: make several improvements to the eden manifest · d2a4722e
      Adam Simpkins authored
      Summary:
      - Indicate that the EDEN_VERSION_OVERRIDE environment variable affects the
        build.
      - Exclude eden/config.py from the shipit  path map, as this file is
        auto-generated in open source builds.
      - Include pexpect as a dependency, as this is needed for some of the
        integration tests.
      
      Reviewed By: genevievehelsel
      
      Differential Revision: D21000163
      
      fbshipit-source-id: 8eec378f66487229c995f637c4787eae530c7845
      d2a4722e
  13. 14 Apr, 2020 1 commit
    • Misha Shneerson's avatar
      Lift RequestContext root-id into task stats · 7469e0b5
      Misha Shneerson authored
      Summary: We want to use observer API to correlate tasks across same Thrift request. `RequestContext::getRootId` is the thing for it.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D20600485
      
      fbshipit-source-id: 1a79bde0f6c5b3692b4fb3805c98f377b1b317c9
      7469e0b5
  14. 10 Apr, 2020 2 commits
    • Felix Handte's avatar
      Make the Zstd Codec use the Context Singletons · 0938cf41
      Felix Handte authored
      Summary: Use pre-existing cached local contexts, rather than creating our own.
      
      Reviewed By: bimbashrestha
      
      Differential Revision: D18890550
      
      fbshipit-source-id: d53d0bdf35ebfcfa5dfb45da6b99a886022d5f8b
      0938cf41
    • Dan Melnic's avatar
      Fix PollIoBackend compile error · d1ec5727
      Dan Melnic authored
      Summary:
      Fix PollIoBackend compile error
      
      (Note: this ignores all push blocking failures!)
      
      Reviewed By: yfeldblum, kevin-vigor
      
      Differential Revision: D20955224
      
      fbshipit-source-id: a2c7a34d9129cb282d3652607e79dcee1ce1cc78
      d1ec5727
  15. 09 Apr, 2020 2 commits
    • Wez Furlong's avatar
      getdeps: export fbsource commit date into environment · 4a29e9cb
      Wez Furlong authored
      Summary:
      This diff extracts the fbsource commit hash and the date of that
      commit and maintains that in place of just the commit hash that
      we were previously extracting.
      
      This data is exported into the environment that we pass on to
      builders so that it is available if they choose to use it.
      
      In a follow on diff I'll use this to default a version number
      in the watchman project.
      
      Reviewed By: fanzeyi
      
      Differential Revision: D20949666
      
      fbshipit-source-id: dc12bffe5f0efc4297b15ba0140c4c67a23ab0fd
      4a29e9cb
    • Kirk Shoop's avatar
      Add Support for nested MasterPtr and Cleanup tasks · 8045a2a0
      Kirk Shoop authored
      Summary:
      Allow a T that derives from EnableMasterFromThis<T> to use masterLockFromThis() to get a non-owning shared_ptr to this and to use masterRefFromThis() to get a MasterPtrRef<> from this.
      
      Adds MasterPtr::cleanup() that returns SemiFuture<Unit>. join() just does a blocking wait on the SemiFuture returned from cleanup().
      
      Allows a T to provide a T::cleanup() method that will be composed into the MasterPtr::cleanup() work.
      
      MasterPtr now uses SemiFuture<Unit> instead of Baton. This allows users of MasterPtr::cleanup() to compose cleanup work with other tasks.
      
      Andrii suggested that the cleanup feature be extracted out of MasterPtr
      
      Adds cleanup traits (that MasterPtr satisfies) and a Cleanup type that satisfies the cleanup traits and allows objects that are not heap-allocated to participate in structured concurrency by deriving from Cleanup.
      
      Reviewed By: andriigrynenko
      
      Differential Revision: D19584561
      
      fbshipit-source-id: aa2d608effe613ec84b08f902a1c61561f3458bb
      8045a2a0