1. 13 Jun, 2019 5 commits
    • Michael Park's avatar
      Named an unnamed thread in ObserverManager · 03ef648d
      Michael Park authored
      Summary: This patch names the thread inside `ObserverManager::NextQueue`, similar to the vector of threads in `ObserverManager::CurrentQueue`.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D15746165
      
      fbshipit-source-id: 02df5616dcc2af27f318088c051a49d14df9a4df
      03ef648d
    • Aaryaman Sagar's avatar
      Optimizations, support for exceptions and big return values for lock_combine · 886a70d9
      Aaryaman Sagar authored
      Summary:
      Some optimizations and changes to make mutex migrations easier:
      - Add exception handling support, this allows using lock_combine pretty much
        anywhere a unique_lock would be used and makes transitioning between lock
        methods easier and more efficient as users aren't required to maintain their
        own unions anywhere (eg. with folly::Try)
      - Add support for big return values so people can return anything from the
        critical section.  Without this, users would have to use code of the following
        form, which is prone to false sharing with metadata for the waiting thread
      ```
      auto value = ReturnValue{};
      mutex.lock_combine([&]() {
        value = critical_section();
      });
      ```
      - Add some optimizations like inlining the combine codepath and an optimistic
        load to elide a branch.  This gets us a ~8% throughput improvement from
        before.  More importantly, This prevents compilers from messing up the
        generated code to dereference the waiter node whenever they feel like.
      - Defer time publishing for combinable threads until a preemption.  This gets us
        to the same level of efficiency as std::atomic even on broadwell, takes us to
        7x of the baseline (std::mutex) on the NUMA-less machines and almost perfectly
        parallel in the moderate concurrency levels.  I suspect we can do better with
        NUMA-awareness, but that's for another diff
      
      Reviewed By: yfeldblum
      
      Differential Revision: D15522658
      
      fbshipit-source-id: 420f4202503305d57b6bd59a9a4ecb67d4dd3c2e
      886a70d9
    • Yedidya Feldblum's avatar
      Rename to_underlying_type to to_underlying · 92c4c7e2
      Yedidya Feldblum authored
      Summary:
      [Folly] Rename `to_underlying_type` to `to_underlying`, following the pattern of `to_unsigned` and `to_signed`.
      
      (Note: this ignores all push blocking failures!)
      
      Reviewed By: mzlee
      
      Differential Revision: D15776448
      
      fbshipit-source-id: d1f87e917c22a1aa47fb5e37a3ecf7032cec0d48
      92c4c7e2
    • Alexey Spiridonov's avatar
      Use GTEST_SKIP for SKIP · 9b9aadfc
      Alexey Spiridonov authored
      Summary:
      The anticipated (and desired) effect of this change is to make it so that `SKIP` in `SetUp` will actually always skip the test. Right now, test behavior depends on the setting `FOLLY_SKIP_AS_FAILURE` — when this is false, the test will get **executed** despite being "skipped".
      
      I believe it's unlikely that tests rely on the current behavior of doing `SKIP` from `SetUp` and running the test anyway. So, I expect this to be a relatively safe change.
      
      Reviewed By: simpkins
      
      Differential Revision: D15703771
      
      fbshipit-source-id: d32e1d9c4cb8cb99c667882b81946154c6ca906b
      9b9aadfc
    • Aaryaman Sagar's avatar
      Use folly::ThreadLocal in DeterministicSchedule · 9fc5d328
      Aaryaman Sagar authored
      Summary:
      DeterministicSchedule does not work on mobile because __thread or thread_local
      support is not present.  folly::ThreadLocal helps because it uses
      pthread_getspecific for mobile and platforms where support for this is not there
      
      This does mean that we can't add DSched tests for folly::ThreadLocal, but that
      does not have DSched tests anyway.  When we add DSched tests for that, we can
      move to using a lock + map keyed by std::this_thread::get_id
      
      Reviewed By: yfeldblum
      
      Differential Revision: D15770455
      
      fbshipit-source-id: 2a3cc6b3b1c116469cce6295a641d784e5bdfd50
      9fc5d328
  2. 12 Jun, 2019 4 commits
  3. 11 Jun, 2019 5 commits
    • Adam Simpkins's avatar
      update the watchman manifest to include all Eden thrift files · 348f8727
      Adam Simpkins authored
      Summary:
      This is a hack for now to allow fbsource-based watchman getdeps builds to
      succeed.  In the long run we need to update Watchman's CMakeLists.txt to
      depend on a proper EdenFS build, rather than copying Eden's thrift files into
      Watchman's source tree.
      
      Reviewed By: wez
      
      Differential Revision: D15753320
      
      fbshipit-source-id: 2dea483c2053c4ea1cf64021c4bbc3239fce645d
      348f8727
    • Yedidya Feldblum's avatar
      Support various operations on empty IPAddress and SocketAddress · a34214c4
      Yedidya Feldblum authored
      Summary: [Folly] Support various operations on empty `IPAddress` and `SocketAddress`, i.e. default-constructed instances, such as `operator==` and `hash`. V.s. throwing exceptions.
      
      Reviewed By: simpkins
      
      Differential Revision: D15750100
      
      fbshipit-source-id: 89853635f6aab3c295f6fca4e66e6435b0839357
      a34214c4
    • Shrikrishna Khare's avatar
      fbcode_builder: getdeps: fboss: add FBOSS fetcher · 3fe63e48
      Shrikrishna Khare authored
      Summary:
      The previous diffs in the stack add fbcode_builder manifests for FBOSS
      dependencies iproute2 and OpenNSL. The other FBOSS dependencies already have
      fbcode_builder support. Thus, add FBOSS manifest.
      
      Additionally, this patch also modifies fboss CMakeLists.txt to get FBOSS to
      build using fbcode_builder.
      
      Reviewed By: wez
      
      Differential Revision: D15626298
      
      fbshipit-source-id: 31802ef695ae69b08f526bbeb299c59f3b9d44be
      3fe63e48
    • Shrikrishna Khare's avatar
      fbcode_builder: getdeps: fboss: update libcurl manifest to disable lldp · fc8b6513
      Shrikrishna Khare authored
      Summary:
      D15683387 introduced manifest for libcurl and used cmake instead of autoconf
      (rationale: having cmake build curl would make it easy as we want curl to build
      on windows and macos).
      
      if libcurl finds ldap.h, then it builds with ldap enabled.
      On my devserver, ldap library is not installed, and thus libcurl builds without
      it, and links with FBOSS binaries fine.
      
      In sandbox, ldap seems to be installed, thus libcurl builds with ldap. However,
      linking fails for FBOSS binaries as those don't link against ldap - FBOSS does
      not depend on ldap.
      
      Thus, we need a way to configure libcurl with ldap disabled. [cmake.defines]
      CURL_DISABLE_LDAP=OFF, does not quite do it. Thus, am going with autoconf, with
      args --disable-ldap for now for Linux and cmake for non-Linux environments.
      
      Reviewed By: wez
      
      Differential Revision: D15730947
      
      fbshipit-source-id: 2ce48976ee785401d550720cbdccb22f9d416675
      fc8b6513
    • Shrikrishna Khare's avatar
      fbcode_builder: getdeps: fboss: extend iproute2 builder to copy lib/* · c9e0baad
      Shrikrishna Khare authored
      Summary:
      D15588809 introduced manifest and customer builder for iproute2. While the
      Iproute2Builder copies include/* from build to install directory, it does not
      copy lib/* which contains the libnetlink.a
      
      As a result, the build system looks up libnetlink.a on local machine (which it
      may not find).
      
      Fix it by explicitly coping lib/* as well.
      
      Reviewed By: wez
      
      Differential Revision: D15708835
      
      fbshipit-source-id: 3f5906eb48ce982b6e887cca0d3c11314425beab
      c9e0baad
  4. 10 Jun, 2019 1 commit
    • Jingyi Yang's avatar
      add fiber support · 9bded589
      Jingyi Yang authored
      Summary: This diff allows zmq socket to run on folly::fibers
      
      Reviewed By: saifhhasan
      
      Differential Revision: D15113447
      
      fbshipit-source-id: 8a0e1f91d719a9deda48837d4ff1065408280b72
      9bded589
  5. 09 Jun, 2019 2 commits
  6. 08 Jun, 2019 1 commit
    • Yedidya Feldblum's avatar
      Fixed constexpr append in FixedString tests · 336bfca0
      Yedidya Feldblum authored
      Summary:
      [Folly] Fixed constexpr append in `FixedString` tests after {D15618465}.
      
      ```
      folly/test/FixedStringTest.cpp: In member function 'virtual void FixedStringAssignTest_ConstexprAppendLiteral_Test::TestBody()':
      folly/test/FixedStringTest.cpp:360:3: error: non-constant condition for static assertion
         static_assert(constexpr_append_literal_test() == "hello world!", "");
         ^~~~~~~~~~~~~
      In file included from folly/Range.h:25:0,
                       from folly/FixedString.h:33,
                       from folly/test/FixedStringTest.cpp:20:
      folly/test/FixedStringTest.cpp:360:47:   in constexpr expansion of 'constexpr_append_literal_test()'
      folly/test/FixedStringTest.cpp:355:18:   in constexpr expansion of '#'result_decl' not supported by dump_expr#<expression error>.folly::BasicFixedString<char, 20>::append((& s[7]))'
      folly/FixedString.h:1277:48:   in constexpr expansion of 'folly::constexpr_strlen<char>(that)'
      folly/portability/Constexpr.h:68:21: error: 'strlen((& s[7]))' is not a constant expression
         return std::strlen(s);
                ~~~~~~~~~~~^~~
      ```
      
      Reviewed By: simpkins
      
      Differential Revision: D15643402
      
      fbshipit-source-id: c7e26a0c37c956300364c566f211c866985d9dc2
      336bfca0
  7. 07 Jun, 2019 3 commits
    • Nathan Bronson's avatar
      switch vectorized and fallback implementation order · 74f760d6
      Nathan Bronson authored
      Summary:
      It's common for people reading the F14 code to
      see the fallback implementation without noticing the #if
      !FOLLY_F14_VECTOR_INTRINSICS_AVAILABLE at the top, which leads them to
      incorrect conclusions about the code. This diff flips the order in the
      file to reduce confusion.
      
      Reviewed By: shixiao
      
      Differential Revision: D15571969
      
      fbshipit-source-id: 10d2aa29da3f1b8dd48af490a53f1f845f103fa8
      74f760d6
    • Yedidya Feldblum's avatar
      Merge folly/futures/helpers.h into folly/futures/Future.h · 11c136c8
      Yedidya Feldblum authored
      Summary: [Folly] Merge `folly/futures/helpers.h` into `folly/futures/Future.h` to avoid the cyclic include trap.
      
      Differential Revision: D15600550
      
      fbshipit-source-id: a0d14c0092bb433f7af797201d9fafebab370502
      11c136c8
    • Anton Likhtarov's avatar
      getThreadName(): handle bad std::thread::id gracefully · a5fd94a4
      Anton Likhtarov authored
      Summary: pthread_getname_np() tries to dereference 0 when called with it, let's turn it into an error instead
      
      Reviewed By: yfeldblum
      
      Differential Revision: D15708351
      
      fbshipit-source-id: aa2dd92e12e33174c8998daed056de50d4111774
      a5fd94a4
  8. 06 Jun, 2019 14 commits
    • Zeyi (Rice) Fan's avatar
      add level settings to handler · d1c34f8e
      Zeyi (Rice) Fan authored
      Summary:
      This diff adds an option "level" to standard log handler configuration so handlers can have individual logging level set.
      
      This allows us to have multiple handler that handles messages with different level settings.
      
      For example,
      
      > eden=DBG3:default:other; default=stream; other=stream:level=WARN;
      
      This configuration creates two log handlers "default" and "other". Both of the handlers will be receiving log messages level "DBG3" from "eden" log category. In this example, "other" handler will only be receiving log messages that is >= "WARN" while the "default" handler still receives all messages >= "DBG3".
      
      However, one catch of this diff is that, when the log category is set to a higher logging level and the log handler has a lower logging level. The log handler will NOT receive any log messages that is lower than the log category logging level. The test `LogLevelReverseTest` illustrate this situation.
      
      Reviewed By: simpkins
      
      Differential Revision: D15541101
      
      fbshipit-source-id: 741aa97d2d5e93c40324bf6a3dd1d9e38f4dc6a5
      d1c34f8e
    • Dan Melnic's avatar
      Change SharedInfoObserverEntryBase::beforeFreeExtBuffer to... · d7e9357c
      Dan Melnic authored
      Change SharedInfoObserverEntryBase::beforeFreeExtBuffer to SharedInfoObserverEntryBase::afterFreeExtBuffer
      
      Summary: Change SharedInfoObserverEntryBase::beforeFreeExtBuffer to SharedInfoObserverEntryBase::afterFreeExtBuffer
      
      Reviewed By: yfeldblum
      
      Differential Revision: D15687084
      
      fbshipit-source-id: 8f77a123f1bdc0e0ee7391dee7da47d9b9addc6d
      d7e9357c
    • Shrikrishna Khare's avatar
      fbcode_builder: getdeps: fboss: add libnl manifest · 873fd33e
      Shrikrishna Khare authored
      Summary:
      This is towards getting open source FBOSS to build using fbcode_builder.
      libnl is one of the dependencies for FBOSS. This patch adds a manifest
      file to build libnl.
      
      Reviewed By: wez
      
      Differential Revision: D15686853
      
      fbshipit-source-id: f7549df6dc1005630193b024be6e1ea330cc6646
      873fd33e
    • Shrikrishna Khare's avatar
      fbcode_builder: getdeps: fboss: add libcurl manifest · c8fb6d47
      Shrikrishna Khare authored
      Summary:
      This is towards getting open source FBOSS to build using fbcode_builder.
      libcurl is one of the dependencies for FBOSS. This patch adds a manifest
      file to build libcurl.
      
      Reviewed By: wez
      
      Differential Revision: D15683387
      
      fbshipit-source-id: 8df5c413e7dff06d1a19e0ce3b1706bff2f1ab54
      c8fb6d47
    • Shrikrishna Khare's avatar
      fbcode_builder: getdeps: fboss: add libusb manifest · f76eb394
      Shrikrishna Khare authored
      Summary:
      This is towards getting open source FBOSS to build using fbcode_builder.
      libusb is one of the dependencies for FBOSS. This patch adds a manifest
      file to build libusb. FBOSS does not need libudev, so we avoid unncessary
      dependency by passing right argument to ./configure.
      
      Reviewed By: wez
      
      Differential Revision: D15683386
      
      fbshipit-source-id: 1d35caecf4012ddc0df24a7305019d522b4763fa
      f76eb394
    • Andrii Grynenko's avatar
      Disable exception checks if it wasn't thrown by a fiber · 1b24cbe4
      Andrii Grynenko authored
      Summary: If fiber was activated with an exception it's safe for that fiber to be preempted.
      
      Differential Revision: D15640534
      
      fbshipit-source-id: d908aacd81e1ec57874324a58b180587a1fe2eeb
      1b24cbe4
    • Yedidya Feldblum's avatar
      Replace inclusions of folly/futures/helpers.h with folly/futures/Future.h · dff7ae96
      Yedidya Feldblum authored
      Summary: Replace inclusions of `folly/futures/helpers.h` with `folly/futures/Future.h` to avoid the cyclic include trap.
      
      Differential Revision: D15600549
      
      fbshipit-source-id: 19950be24a7437fb1fbec293e24058adf17343ca
      dff7ae96
    • Wez Furlong's avatar
      getdeps: disable ccache on sandcastle · 9add78ae
      Wez Furlong authored
      Summary:
      We sometimes see some flakeyness in our internal CI, so we can live
      without it there.
      
      Reviewed By: pkaush
      
      Differential Revision: D15695124
      
      fbshipit-source-id: 1d76ae89e245d9c95937e8818826c544c0ae8fc3
      9add78ae
    • Takshak Chahande's avatar
      AsyncSocket's ConnectCallback provide additional preConnect · 754409b4
      Takshak Chahande authored
      Summary:
      Currently, AsyncSocket's `connect` API creates new socket, set various
      options, bind it to passed argument address and used it for connecting to
      server.
      
      While most of the things are handled inside that single connect routine; sometimes
      if the caller would like to do few more additional operations before actual
      connect happens; caller would like to have hold on the underneath used socket.
      
      In order to do that additional operation on the socket, this diff provides the
      facility to provide custom pre-connect hook `preConnect(NetworkSocket fd)` in
      the `ConnectCallback` which will be invoked just before the actual server
      connect.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D15282727
      
      fbshipit-source-id: 5beac55c77a9537ee4f26856dd5f78c5224a1f49
      754409b4
    • Uladzislau Paulovich's avatar
      folly | Remove unnecessary reinterpret_cast in Executor · 088fab7c
      Uladzislau Paulovich authored
      Summary: Input and output types for the cast are the same (except const qualifier) and therefore it's not required.
      
      Reviewed By: milend
      
      Differential Revision: D15694981
      
      fbshipit-source-id: d5a811d97cc145ba9c232b79f6c8fa52041b0cf0
      088fab7c
    • Yedidya Feldblum's avatar
      XLOG_EVERY_N_THREAD · 6c7906af
      Yedidya Feldblum authored
      Summary:
      [Folly] `XLOG_EVERY_N_THREAD`.
      
      Implemented with a single per-thread map for all `XLOG_EVERY_N_THREAD` collectively. This approach has better TLS overhad than a `thread_local` counter per `XLOG_EVERY_N_THREAD`, at the cost of integer-keyed map lookups.
      
      While the `thread_local` counter approach is twice as fast as this approach for a noop log-handler, it is expected that the runtime overhead of real log-handlers will make this type of saving immaterial.
      
      Reviewed By: simpkins, luciang
      
      Differential Revision: D15293529
      
      fbshipit-source-id: 7af535a0d329473126a0e4dcffc19d750e8b8b10
      6c7906af
    • Miroslav Crnic's avatar
      LifoSem make tryRemoveNode protected · 8042bf57
      Miroslav Crnic authored
      Summary:
      Lifo sem exposes allocateNode and tryWaitOrPush as protected which allows inherited classes custom Node implementation, creation and enqueueing.
      tryRemoveNode was private which prevented inherited classes from dequeueing a Node without signaling shutdown.
      This diff fixes it
      
      Reviewed By: shixiao
      
      Differential Revision: D15647287
      
      fbshipit-source-id: 5b26a9616f6de559224f9d8c38230780c78a7cf5
      8042bf57
    • Yedidya Feldblum's avatar
      Let KeepAlive::copy() allow dummy executors to expire · d3233089
      Yedidya Feldblum authored
      Summary: [Folly] Let `KeepAlive::copy()` allow dummy executors to expire, and not attempt to dereference them to perform an acquire - which, as is known in advance, would fail and return false were the executor still to be alive.
      
      Reviewed By: andrewcox
      
      Differential Revision: D15683240
      
      fbshipit-source-id: e2026493af6936b8f0f1b8fe3c1cb97d2bf0f96e
      d3233089
    • Yedidya Feldblum's avatar
      Split KeepAlive concepts of dummy and alias · 8d2d67e3
      Yedidya Feldblum authored
      Summary: [Folly] Split KeepAlive concepts of dummy and alias. A dummy KeepAlive is one for an executor which does not actually support keep-alive semantics. An alias KeepAlive is one for which there is another KeepAlive, with a surrounding lifetime, to the same executor.
      
      Reviewed By: andrewcox
      
      Differential Revision: D15683241
      
      fbshipit-source-id: a5809b06c90ed4a655a6973fac67137b5e1981dc
      8d2d67e3
  9. 05 Jun, 2019 5 commits
    • Yedidya Feldblum's avatar
      Avoid gcc7 bug in SerialExecutor test · 86b2ff29
      Yedidya Feldblum authored
      Summary:
      [Folly] Avoid gcc7 lambda-captures bug in `SerialExecutor` test.
      
      ```
      folly/executors/test/SerialExecutorTest.cpp: In lambda function:
      folly/executors/test/SerialExecutorTest.cpp:66:40: error: declaration of 'i' shadows a previous local [-Werror=shadow]
      folly/executors/test/SerialExecutorTest.cpp:60:12: note: shadowed declaration is here
      folly/executors/test/SerialExecutorTest.cpp: In function 'void simpleTest(const std::shared_ptr<folly::Executor>&)':
      folly/executors/test/SerialExecutorTest.cpp:74:45: error: already captured 'i' in lambda expression [-Werror]
      folly/executors/test/SerialExecutorTest.cpp: In lambda function:
      folly/executors/test/SerialExecutorTest.cpp:74:62: error: 'i' is not captured
      folly/executors/test/SerialExecutorTest.cpp:74:46: note: the lambda has no capture-default
      folly/executors/test/SerialExecutorTest.cpp:60:12: note: 'int i' declared here
      ```
      
      Fixes #1154.
      
      Reviewed By: andriigrynenko
      
      Differential Revision: D15629657
      
      fbshipit-source-id: fec82e9d54b2c79a7f54f366ec8a19ad7dc198e9
      86b2ff29
    • Joe Loser's avatar
      Cut FOLLY_HAVE_COMPARE_EQUIVALENT from StringKeyedBenchmark.cpp (#1157) · 59ca4e5c
      Joe Loser authored
      Summary:
      - Cut `FOLLY_HAVE_COMPARE_EQUIVALENT` as this is `1` for Clang 3.4
        onward (regardless of `libstdc++` or `libc++`) and also `1` for GCC
        versions we support.
      Pull Request resolved: https://github.com/facebook/folly/pull/1157
      
      Reviewed By: Orvid
      
      Differential Revision: D15641688
      
      Pulled By: yfeldblum
      
      fbshipit-source-id: 0602913b375a406cce6337e43ba708362a94d444
      59ca4e5c
    • Shrikrishna Khare's avatar
      fbcode_builder: getdeps: fboss: add libmnl manifest · 93d5da3c
      Shrikrishna Khare authored
      Summary:
      This is towards getting open source FBOSS to build using fbcode_builder.
      libmnl is one of the dependencies for FBOSS. This patch adds a manifest file
      to build the specific version of libmnl for FBOSS.
      
      Reviewed By: wez
      
      Differential Revision: D15633176
      
      fbshipit-source-id: dd51997ae950f53842711fbf2d044e84cdf437e7
      93d5da3c
    • Shrikrishna Khare's avatar
      fbcode_builder: getdeps: fboss: add OpenNSL fetcher · d6678a05
      Shrikrishna Khare authored
      Summary:
      This is towards getting open source FBOSS to build using fbcode_builder.
      OpenNSL is one of the dependencies for FBOSS.
      
      OpenNSL is github repository with headers and prebuilt opaque library. Thus, we
      can't/don't need to build it. Thus, OpenNSL manifest uses NopBuilder.
      
      fbcode_builder expects the library to be in installed/lib and headers to be in
      installed/include, but OpenNSL lib structure is different.  Thus, add explicit
      [install.fields] rule for bin/wedge => lib. Once a rule(s) is provided, we must
      provide rule(s) for all, so add rule for include => include as well.
      
      Reviewed By: wez
      
      Differential Revision: D15593639
      
      fbshipit-source-id: facb36ff7b9a31f5952ecbc1a9fdb834c0e68d11
      d6678a05
    • Shrikrishna Khare's avatar
      fbcode_builder: getdeps: fboss: add iproute2 fetcher · cfed36bd
      Shrikrishna Khare authored
      Summary:
      This is towards getting open source FBOSS to build using fbcode_builder.
      iproute2 is one of the dependencies for FBOSS. This patch adds a manifest file
      to build the specific version of iproute2 needed for FBOSS.
      
      Additionally, the default git clone depth of 100 is insufficient for the
      version of iproute2 FBOSS depends on. Thus, this patch extends the git SCHEMA
      to add optional argument depth. The default remains 100.
      
      The usual /configure --prefix does not work for iproute2. Thus, we need to add
      a custom builder that:
        - copies sources to build directory, builds, and
        - installs to installed directory using DEST_DIR.
        - it must also explicitly copy include from build dir to install dir
      
      Reviewed By: wez
      
      Differential Revision: D15588809
      
      fbshipit-source-id: ac5eab24134e078d88b85b4be433c78b05ef8ce5
      cfed36bd