1. 21 Sep, 2019 1 commit
    • Yedidya Feldblum's avatar
      Optimize Function::operator() codegen · 44833c41
      Yedidya Feldblum authored
      Summary:
      [Folly] Optimize `Function::operator()` codegen for size and speed.
      
      * Avoid translating between values and references for trivially-copyable values.
      * Avoid shifting all arguments to make room for the function object address.
      
      In the optimal case, the codegen for calling a `Function` with many arguments translates into just a `jmp`.
      
      See:
      * https://github.com/thecppzoo/zoo/commits/master/inc/zoo/AnyCallable.h
      * https://github.com/bloomberg/bde/blob/3.38.0.1/groups/bsl/bslstl/bslstl_function.h
      * https://github.com/bloomberg/bde/blob/3.38.0.1/groups/bsl/bslmf/bslmf_forwardingtype.h
      
      Given this example code:
      
      ```lang=c++,name=check.cpp
      extern "C" void check_0(folly::Function<void()>& f) { f(); }
      extern "C" void check_1(int i, folly::Function<void(int)>& f) { f(i); }
      extern "C" void check_2(int i, int j, folly::Function<void(int, int)>& f) { f(i, j); }
      extern "C" void check_3(int i, int j, int k, folly::Function<void(int, int, int)>& f) { f(i, j, k); }
      extern "C" void check_4(int i, int j, int k, int l, folly::Function<void(int, int, int, int)>& f) { f(i, j, k, l); }
      ```
      
      Before:
      
      ```name=check.o
      0000000000000000 <check_0>:
         0:   ff 67 30                jmp    QWORD PTR [rdi+0x30]
      0000000000000000 <check_1>:
         0:   55                      push   rbp
         1:   48 89 f0                mov    rax,rsi
         4:   48 89 e5                mov    rbp,rsp
         7:   48 83 ec 10             sub    rsp,0x10
         b:   89 7d fc                mov    DWORD PTR [rbp-0x4],edi
         e:   48 8d 75 fc             lea    rsi,[rbp-0x4]
        12:   48 89 c7                mov    rdi,rax
        15:   ff 50 30                call   QWORD PTR [rax+0x30]
        18:   c9                      leave
        19:   c3                      ret
      0000000000000000 <check_2>:
         0:   55                      push   rbp
         1:   48 89 d0                mov    rax,rdx
         4:   48 89 e5                mov    rbp,rsp
         7:   48 83 ec 10             sub    rsp,0x10
         b:   89 7d f8                mov    DWORD PTR [rbp-0x8],edi
         e:   89 75 fc                mov    DWORD PTR [rbp-0x4],esi
        11:   48 8d 55 fc             lea    rdx,[rbp-0x4]
        15:   48 8d 75 f8             lea    rsi,[rbp-0x8]
        19:   48 89 c7                mov    rdi,rax
        1c:   ff 50 30                call   QWORD PTR [rax+0x30]
        1f:   c9                      leave
        20:   c3                      ret
      0000000000000000 <check_3>:
         0:   55                      push   rbp
         1:   48 89 c8                mov    rax,rcx
         4:   48 89 e5                mov    rbp,rsp
         7:   48 83 ec 10             sub    rsp,0x10
         b:   89 7d f4                mov    DWORD PTR [rbp-0xc],edi
         e:   89 75 f8                mov    DWORD PTR [rbp-0x8],esi
        11:   89 55 fc                mov    DWORD PTR [rbp-0x4],edx
        14:   48 8d 4d fc             lea    rcx,[rbp-0x4]
        18:   48 8d 55 f8             lea    rdx,[rbp-0x8]
        1c:   48 8d 75 f4             lea    rsi,[rbp-0xc]
        20:   48 89 c7                mov    rdi,rax
        23:   ff 50 30                call   QWORD PTR [rax+0x30]
        26:   c9                      leave
        27:   c3                      ret
      0000000000000000 <check_4>:
         0:   55                      push   rbp
         1:   4c 89 c0                mov    rax,r8
         4:   48 89 e5                mov    rbp,rsp
         7:   48 83 ec 10             sub    rsp,0x10
         b:   89 7d f0                mov    DWORD PTR [rbp-0x10],edi
         e:   89 75 f4                mov    DWORD PTR [rbp-0xc],esi
        11:   89 55 f8                mov    DWORD PTR [rbp-0x8],edx
        14:   89 4d fc                mov    DWORD PTR [rbp-0x4],ecx
        17:   4c 8d 45 fc             lea    r8,[rbp-0x4]
        1b:   48 8d 4d f8             lea    rcx,[rbp-0x8]
        1f:   48 8d 55 f4             lea    rdx,[rbp-0xc]
        23:   48 8d 75 f0             lea    rsi,[rbp-0x10]
        27:   48 89 c7                mov    rdi,rax
        2a:   ff 50 30                call   QWORD PTR [rax+0x30]
        2d:   c9                      leave
        2e:   c3                      ret
      ```
      
      After:
      
      ```name=check.o
      0000000000000000 <check_0>:
         0:   ff 67 30                jmp    QWORD PTR [rdi+0x30]
      0000000000000000 <check_1>:
         0:   ff 66 30                jmp    QWORD PTR [rsi+0x30]
      0000000000000000 <check_2>:
         0:   ff 62 30                jmp    QWORD PTR [rdx+0x30]
      0000000000000000 <check_3>:
         0:   ff 61 30                jmp    QWORD PTR [rcx+0x30]
      0000000000000000 <check_4>:
         0:   41 ff 60 30             jmp    QWORD PTR [r8+0x30]
      ```
      
      Reviewed By: mpark, ericniebler, ot, luciang
      
      Differential Revision: D17416587
      
      fbshipit-source-id: cdb76314197f8979339e31435fff997a46f48f1c
      44833c41
  2. 20 Sep, 2019 8 commits
    • Adam Simpkins's avatar
      fbcode_builder: implement automatic project detection from the current repo · 571f3268
      Adam Simpkins authored
      Summary:
      This updates fbcode_builder to try and automatically detect the current
      repository's project name by looking for a `.projectid` file in the repository
      root.  If the project name can be detected:
      - The current repository will automatically be used as the source directory
        for this project (instead of fetching the sources into the scratch
        directory).
      - If an explicit project name was not specified on the command line, the
        current project will be built by default.
      
      This also changes the repository detection logic to use the current working
      directory, rather than the directory where the fbcode_builder code lives.
      This will allow this logic to work even if we move fbcode_builder into its own
      repository in the future, and have projects depend on it using git submodules.
      This does mean that callers need to invoke fbcode_builder.py from inside the
      repository.
      
      Reviewed By: wez
      
      Differential Revision: D17088938
      
      fbshipit-source-id: f14d28fdcfaa330ff837ea52b8bdd4e358b81c61
      571f3268
    • Adam Simpkins's avatar
      improve run_cmake.py on Windows · 233f10ae
      Adam Simpkins authored
      Summary:
      Update the generated `run_cmake.py` script to use `subprocess.run()` instead
      of `os.execve()`.  The `os.execve()` call doesn't really do what we want on
      Windows: this causes the script to exit while CMake is still running,
      resulting in confusing output.  During the build step it also did not work
      correctly with `vcvarsall.bat`, and using `subprocess` also solves this.
      
      Reviewed By: wez
      
      Differential Revision: D17493897
      
      fbshipit-source-id: e0477627fc1824b0efcb1fa5a782d207853bcae8
      233f10ae
    • Adam Simpkins's avatar
      fix calculation of whether a project is cachable · 5dec9568
      Adam Simpkins authored
      Summary:
      Never cache first-party projects that use ShipIt.  Previously the code checked
      the `shipit_fbcode_builder` property, which controlled whether or not
      the `fbcode_builder` sources should be included in the project's ShipIt
      mapping.  This setting is enabled for most but not all projects that use
      ShipIt.
      
      This resulted in projects that use ShipIt but that do not include the fbcode
      builder sources being incorrectly cached.  This caused getdeps.py to not
      run the SimpleShipitTransformerFetcher properly when their sources changed.
      
      Reviewed By: wez
      
      Differential Revision: D17493522
      
      fbshipit-source-id: 57be5ac94ae44f56ccb3ce60ba23fac5d68bce0f
      5dec9568
    • Adam Simpkins's avatar
      the edenfsctl program depends on python-toml · bb96fea7
      Adam Simpkins authored
      Summary:
      Update the getdeps manifest and Eden's CMake files to indicate that the
      edenfsctl program depends on python-toml.
      
      Reviewed By: chadaustin
      
      Differential Revision: D17401215
      
      fbshipit-source-id: f512678d8bca9c7b2b4d25bf9c3ecd7eed825de9
      bb96fea7
    • Zeyi (Rice) Fan's avatar
      getdeps: move manifests · e41b0ce1
      Zeyi (Rice) Fan authored
      Reviewed By: simpkins
      
      Differential Revision: D17403339
      
      fbshipit-source-id: 2a6b2eb073d54e080f6a313948afd2815f58bba9
      e41b0ce1
    • Alex Guzman's avatar
      Fix order of operations ambiguity · 361701c4
      Alex Guzman authored
      Summary: Fixes a warning about ambiguity between & and ==
      
      Reviewed By: knekritz
      
      Differential Revision: D17500996
      
      fbshipit-source-id: 9a3d1da499bf25aff3d97bc3ea40bf5a1d19257d
      361701c4
    • Alex Guzman's avatar
      Fix portable versions of X509_get_extension_flags and X509_get_key_usage · 3a5914c6
      Alex Guzman authored
      Summary:
      These two functions are currently not correctly implemented when compared to the documented behavior.
      
      1. These functions aren't guaranteed to return correct information until you've called X509_check_purpose on the certificate. Before then, these flag fields are uninitialized.
      2. X509_get_key_usage should return UINT32_MAX when the key usage flag isn't present.
      
      This fixes both of these issues
      
      Reviewed By: yfeldblum
      
      Differential Revision: D15767405
      
      fbshipit-source-id: 013bc4c2f694ba7cbd7395626418a0d423d6c1c7
      3a5914c6
    • Alex Guzman's avatar
      Add function for converting ASN1_TIME to std::chrono::system_clock::time_point · 125e0ed6
      Alex Guzman authored
      Summary: As it says on tin.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D15409159
      
      fbshipit-source-id: a99b17b54e0c36d79956213655385d234190ff52
      125e0ed6
  3. 19 Sep, 2019 7 commits
    • Zeyi (Rice) Fan's avatar
      getdeps: throw exceptions when found duplicate manifests and name mismatches · a18cdcd5
      Zeyi (Rice) Fan authored
      Summary:
      Throws an exception when:
      
      * The name specified in the manifest mismatches the filename
      * Duplicated manifest -- with the sub-directory support it is now able to have multiple manifest files with the same name
      
      Reviewed By: chadaustin
      
      Differential Revision: D17438460
      
      fbshipit-source-id: ac7ad0b701beb15f0e91bb05cd1ec8fe378ad5b6
      a18cdcd5
    • Zeyi (Rice) Fan's avatar
      getdeps: include subdirectories when searching manifest · 9691120a
      Zeyi (Rice) Fan authored
      Summary: Make getdeps to look for subdirectories for manifest files.
      
      Reviewed By: simpkins
      
      Differential Revision: D17222388
      
      fbshipit-source-id: e13503beccd9edf6d80f78fbc3238b2a8d2053dd
      9691120a
    • Adam Simpkins's avatar
      the libraries in thrift/lib/py depend on six · 0bfa67ca
      Adam Simpkins authored
      Summary:
      The libraries in thrift/lib/py support both Python 2 and Python 3, and rely on
      the Python six module for some of this compatibility support.
      
      Update the getdeps manifest for fbthrift to indicate this dependency, and
      update fbthrift's CMakeLists.txt file to find and reference python-six
      properly.  This will ensure that the python-six code is built into any python
      executable that uses the thrift/lib/py libraries.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D17401218
      
      fbshipit-source-id: 0007dda8974ae9bd87e4d7e256c74908c9a30d8f
      0bfa67ca
    • Adam Simpkins's avatar
      add a builder that can re-package python wheel files · 6e1ff036
      Adam Simpkins authored
      Summary:
      Add a new builder that can extract Python wheel files, and re-package them
      for consumption by our add_fb_python_library() and add_fb_python_executable()
      CMake functions.  This is useful for dependencies on packages from PyPI.
      
      At the moment this code only handles architecture-independent pure-Python
      packages.  It shouldn't be too hard to extend this to handle more complex
      wheels, but for now I only need to use it for some pure-Python wheels and so I
      haven't tested with more complex wheel files.
      
      This also includes two new manifests for python-six and python-toml that take
      use this new builder.
      
      Reviewed By: wez
      
      Differential Revision: D17401216
      
      fbshipit-source-id: d6f74565887c3f004e1c06503dc9ec81599dd697
      6e1ff036
    • Yedidya Feldblum's avatar
      Support -Werror=unused-function · f14e7cb7
      Yedidya Feldblum authored
      Summary: Support `-Werror=unused-function`.
      
      Reviewed By: Orvid, guangyfb
      
      Differential Revision: D17252831
      
      fbshipit-source-id: ccf3829ebaa6992341ff5b1ab84c173922eeae03
      f14e7cb7
    • Adam Simpkins's avatar
      update make_fbpy_archive.py to replace the output on Windows · d4b04b02
      Adam Simpkins authored
      Summary:
      Update the code to use `os.replace()` rather than `os.rename()` so that it
      won't fail on Windows if the destination path already exists.
      
      Reviewed By: chadaustin
      
      Differential Revision: D17462716
      
      fbshipit-source-id: cbc06319ccb2d73868f80ab1874890ebec5a621b
      d4b04b02
    • Maged Michael's avatar
      hazptr: Add microbenchmark for unused tagged obj batch · 3ea63e0b
      Maged Michael authored
      Summary:
      Adds a microbenchmark for the lifetime of an unused tagged hazptr_obj_batch in the presence of unrelated objects in the domain's tagged list.
      
      This diff also makes the calculation of the number of operations and the scalability measure for the cleanup microbenchmark more consistent with those for the other microbenchmarks.
      
      Reviewed By: davidtgoldblatt
      
      Differential Revision: D17416088
      
      fbshipit-source-id: b59fd1b48114d591da5d9432df7910d940683006
      3ea63e0b
  4. 18 Sep, 2019 7 commits
  5. 17 Sep, 2019 6 commits
    • Amol Bhave's avatar
      create unorderedReduceSemiFuture method · 92dd0f71
      Amol Bhave authored
      Summary:
      create a unorderedReduceSemiFuture method that returns a SemiFuture
      instead of just a Future.
      
      Reviewed By: LeeHowes
      
      Differential Revision: D16946582
      
      fbshipit-source-id: e28b4cb055b3a9c67adebb373c60a775f7005762
      92dd0f71
    • Adam Simpkins's avatar
      fix the thrift CMake rules to add dependencies on the thrift compiler · e01c5188
      Adam Simpkins authored
      Summary:
      Update the thrift C++ and Python CMake rules to indicate that the output also
      depends on the thrift compiler itself.
      
      Previously the C++ rule indicated that the output depended on the thrift
      template files, which caught most cases when the thrift compiler was updated,
      but wasn't fully correct.  The thrift templates were also removed and baked
      into the thrift compiler binary in D16356056.
      
      Reviewed By: yfeldblum, chadaustin
      
      Differential Revision: D17401217
      
      fbshipit-source-id: ae5cde7a7e5e07a74406a1b6f4469124187bc12f
      e01c5188
    • Adam Simpkins's avatar
      add a command line option to disable the build cache · c9299818
      Adam Simpkins authored
      Summary:
      This is useful when working on changes to some of the builder functions,
      to skip ever trying to use cached results.
      
      Reviewed By: chadaustin
      
      Differential Revision: D17401219
      
      fbshipit-source-id: fb7d5ea45618653957b9bd6a62eed91d8334824d
      c9299818
    • Rosen Penev's avatar
      libfolly.pc: Fix for cross compilation (#1226) · eb473c29
      Rosen Penev authored
      Summary:
      Changes libfolly.pc to a more standard format.
      
      Allows overriding of the variables by pkgconfig.
      Signed-off-by: default avatarRosen Penev <rosenp@gmail.com>
      
      I'm not sure of anything that uses pkgconfig to find libfolly but might as well fix it.
      Pull Request resolved: https://github.com/facebook/folly/pull/1226
      
      Reviewed By: simpkins
      
      Differential Revision: D17415620
      
      Pulled By: yfeldblum
      
      fbshipit-source-id: d015c3e83783b1c5e9ac48a64ad35c0c07120dbd
      eb473c29
    • Wez Furlong's avatar
      getdeps: adjust fixup-dyn-deps to optionally use absolute paths · 9dc9ee90
      Wez Furlong authored
      Summary:
      This commit teaches fixup-dyn-deps how to generate correct
      absolute paths in the context of the ultimate install path (specified
      via the `--final-install-prefix` option)
      
      Absolute paths are desirable if you have, for example, an executable
      that you wish to install with the setuid bit set.
      
      Reviewed By: simpkins
      
      Differential Revision: D17361491
      
      fbshipit-source-id: 4c4f3f15208266300622f84dc9cd1ba83883dfb7
      9dc9ee90
    • Andrew Krieger's avatar
      s/if _WIN32/ifdef _WIN32/g · fa32bb0f
      Andrew Krieger authored
      Summary:
      Avoids -Wundef on non-Windows builds.
      folly/system/ThreadId.h had #elif _WIN32 which was changed to
      #elif defined(_WIN32)
      
      Reviewed By: Orvid
      
      Differential Revision: D17399760
      
      fbshipit-source-id: 76583374c33d5d59e7c64b9a7a05c660a817bb92
      fa32bb0f
  6. 16 Sep, 2019 8 commits
    • Chad Austin's avatar
      fbcode_builder: add a license header to FBBuildOptions.cmake · a783cdd3
      Chad Austin authored
      Summary:
      Add a license header to satisfy the open source linter. Use the same
      header the other .cmake files have.
      
      Reviewed By: mhlakhani
      
      Differential Revision: D17404782
      
      fbshipit-source-id: 66679d72c9e680f8bb8b27869e981a046b3520cf
      a783cdd3
    • Wez Furlong's avatar
      Let's try a github action to build things (#743) · a92b821d
      Wez Furlong authored
      Summary:
      This commit adds a getdeps command that is able to generate
      a workflow file for the GitHub Actions CI environment.
      
      The workflow file could be expressed more simply using the matrix
      syntax and with three steps (checkout, build, test), but I chose to
      break out the steps for each of the dependencies because the UX
      while waiting on the build is much nicer that way: the steps show
      during and live log tailing for the section of the build that is
      underway.  If they were all lumped into a single build step then
      the logs from the boost section of the build dominate and make
      the github UI work very hard.
      
      Pull Request resolved: https://github.com/facebook/watchman/pull/743
      
      Test Plan:
      https://github.com/facebook/watchman/pull/743 successfully
      executes the github actions CI flow.
      
      ```
      $ opensource/fbcode_builder/getdeps.py generate-github-actions --output-file watchman/.github/workflows/main.yml watchman
      ```
      
      Reviewed By: simpkins
      
      Differential Revision: D17384915
      
      Pulled By: wez
      
      fbshipit-source-id: 9a9e5a3e806c18f6cc38ba1cb7059740cda01ad4
      a92b821d
    • Wez Furlong's avatar
      getdeps: teach builder how to find vs 2019 · fe1ec310
      Wez Furlong authored
      Summary:
      GitHub Actions CI `windows-latest` environment has only VS 2019
      installed, so we need to expand our logic to be able to locate it.
      
      Note that Boost 1.69 doesn't know how to locate VS 2019 so we are effectively
      tied to VS 2017 at the moment; the search order in this diff reflects that.
      
      (That means that we can't target `windows-latest` on GitHub Actions, but that
      is really a concern for a later diff in this stack)
      
      Reviewed By: simpkins
      
      Differential Revision: D17385052
      
      fbshipit-source-id: 9bb0612154f42d425a625406488f39bb4ec3d8ae
      fe1ec310
    • Wez Furlong's avatar
      getdeps: prefer MSVC over gcc when building on GitHub Actions CI · 116dea9d
      Wez Furlong authored
      Summary:
      while testing https://github.com/facebook/watchman/pull/743 I
      noticed that the cmake builds were picking up the installed mingw GCC
      compiler rather than the MSVC compiler.  That would be fine except that
      boost is built with MSVC and its generated libraries cannot be subsequently
      found by a cmake gcc build that uses FindBoost.
      
      This commit forces cmake to pick cl.exe rather than gcc.  This is probably
      fine to do unconditionally on windows, but since I've only observed this
      particular problem with GitHub Actions I'm keeping it constrained to that
      environment for now.
      
      Reviewed By: simpkins
      
      Differential Revision: D17385050
      
      fbshipit-source-id: 90bef898b138e5d4bbd28a155ed1bd468acee9de
      116dea9d
    • Wez Furlong's avatar
      getdeps: restructure bison+flex dependencies · c108f833
      Wez Furlong authored
      Summary:
      We've been squeaking by with assuming that flex is installed already
      on posix systems, but that isn't the case on the github actions default
      configuration.
      
      Adjust the bison recipe: on windows it deploys both flex and bison.  We use the
      same source for both flex and bison but install flex to a separate install
      prefix to make it easier to consume the flex dependency distinct from the bison
      dependency.
      
      The latest flex release segfaults during compilation on linux unless we
      force -DGNU_SOURCE, so the manifest does that on linux.
      
      Reviewed By: simpkins
      
      Differential Revision: D17385051
      
      fbshipit-source-id: 9f31b07849af9de50099d1b20bedba517bbbdf2f
      c108f833
    • Wez Furlong's avatar
      getdeps: fix openssl dep for libevent on macos · 691ed9ec
      Wez Furlong authored
      Summary:
      while testing https://github.com/facebook/watchman/pull/743 on macOS
      I noticed that the libevent build failed to find openssl.
      
      openssl is special on macos because apple do not ship the headers.
      We already build and depend on openssl for the folly manifest on
      macos, so this is really just adding a missing edge.
      
      Reviewed By: simpkins
      
      Differential Revision: D17385053
      
      fbshipit-source-id: 1b688537fef422d81a959fc5749c871b9e868baa
      691ed9ec
    • Tom Jackson's avatar
      toPrettyJson() sorts keys for deterministic output · 800f602c
      Tom Jackson authored
      Summary: Diffing logs would be easier if the lines which pretty-printed JSON were identical if their values were identical, and they're just easier to visually scan for a value if they're sorted anyway.
      
      Reviewed By: yfeldblum, luciang
      
      Differential Revision: D17367669
      
      fbshipit-source-id: 91188f23576ce39b9edc2307be018a43f30c786d
      800f602c
    • Yedidya Feldblum's avatar
      Use a fixed mempool capacity in LifoSem · a821d1c9
      Yedidya Feldblum authored
      Summary: [Folly] Use a fixed mempool capacity in `LifoSem`, simplifying usage.
      
      Reviewed By: nbronson
      
      Differential Revision: D17251982
      
      fbshipit-source-id: e3bce99204daef178418268f1d195365400df885
      a821d1c9
  7. 13 Sep, 2019 2 commits
    • Nanshu Chen's avatar
      refactor AsyncGeneratorWrapper to use AsyncGenerator::NextResult · f65c05bb
      Nanshu Chen authored
      Summary: AsyncGeneratorWrapper uses folly::Optional as the return type for "getNext". Now the AsyncGenerator API has changed by adding `next()` method which returns a `NextResult` type, which works like folly::Optional while supporting both values and references, it's best to use `NextResult` for the wrapper as well.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D16695661
      
      fbshipit-source-id: 84e9d74e8d6b17b93bd3c22e2aaf56c7b9e4b789
      f65c05bb
    • Hasnain Lakhani's avatar
      Add CMake option to fetch deps statically · 0c5ceda6
      Hasnain Lakhani authored
      Summary:
      We would like to build a version of proxygen that has minimal
      dependencies on dynamic libraries.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D17228181
      
      fbshipit-source-id: cfd61afdfa978c49a536184f426502196241fb8a
      0c5ceda6
  8. 12 Sep, 2019 1 commit