1. 30 Nov, 2017 3 commits
    • Adam Simpkins's avatar
      logging: don't clamp the log level to DFATAL in debug builds · 2fa201bf
      Adam Simpkins authored
      Summary:
      Remove the logic that clamps the log level to DFATAL in debug builds.
      
      This behavior logically makes some sense, but results in subtle and potentially
      confusing behavior changes.  It seems confusing that after calling
      `setLevel(LogLevel::MAX_LEVEL)` on a log category, calling `getLevel()` on that
      object would return `DFATAL` rather than `MAX_LEVEL`.  We also weren't clamping
      the level to DFATAL consistently: `setLevel()` would clamp the value, but on
      construction the initial level was still set to MAX_LEVEL rather than DFATAL.
      This resulted in some issues when implementing `LoggerDB::getConfig()` since we
      could not consistently detect if a log category was using the default level
      settings.  Rather than fix this inconsistency it seems better to simply remove
      this clamping behavior.
      
      This means that it is possible for users to disable DFATAL log messages even in
      debug builds if they really want to.  Previously this was only allowed in
      release builds.  This protection doesn't really seem all that
      valuable--presumably most developers won't really want to do this, and if they
      really do request this configuration it doesn't seem all that bad to honor it.
      
      Reviewed By: bolinfest, yfeldblum
      
      Differential Revision: D6200569
      
      fbshipit-source-id: 83ef8e2e4d3b61bc5b105038cbe3132979e9ac67
      2fa201bf
    • Adam Simpkins's avatar
      logging: add a LogHandler::getConfig() method · 95d99350
      Adam Simpkins authored
      Summary:
      Add a method to LogHandler to return its current configuration.  This will
      make it possible to query the LoggerDB for its current configuration state.
      
      Reviewed By: bolinfest
      
      Differential Revision: D6200563
      
      fbshipit-source-id: 2b8b9752bbeb26c8aac28d1a73b7e2312fd198c8
      95d99350
    • Adam Simpkins's avatar
      logging: add a LogHandler registry to LoggerDB · ae69959c
      Adam Simpkins authored
      Summary:
      Update the LoggerDB to track a list of LogHandlers by name, and
      LogHandlerFactories by handler type.
      
      This will be needed to support updating the LoggerDB configuration from a
      LogConfig object.
      
      Reviewed By: bolinfest
      
      Differential Revision: D6200562
      
      fbshipit-source-id: e365b4e0df65aa5aaa34e118eb3cee9c9c45cb05
      ae69959c
  2. 29 Nov, 2017 4 commits
    • Yedidya Feldblum's avatar
      Synchronize coupled caches in folly::threadlocal_detail::StaticMeta · 73c52b9f
      Yedidya Feldblum authored
      Summary:
      [Folly] Synchronize coupled caches in `folly::threadlocal_detail::StaticMeta`.
      
      The caches should be set together, and only together, because they are coupled. This prevents bugs where one function that sets one cache but not the other cache is inlined into the caller in one module, and another function that reads both caches is inlined into the caller in another module.
      
      Reviewed By: djwatson
      
      Differential Revision: D6435175
      
      fbshipit-source-id: 846c4972b40e525f2c04da6e6609c2ad54f674c0
      73c52b9f
    • Adam Simpkins's avatar
      logging: add a FileHandlerFactory class · 70230b7a
      Adam Simpkins authored
      Summary:
      Add a new LogHandlerFactory interface for creating LogHandler objects from a
      LogHandlerConfig.
      
      Also add an initial FileHandlerFactory implementation capable of creating
      LogHandler objects that write to a file descriptor.
      
      Reviewed By: bolinfest
      
      Differential Revision: D6200567
      
      fbshipit-source-id: 14b86fc14ad475223aa4b57d45c40638b48c7594
      70230b7a
    • Andrii Nakryiko's avatar
      Add unit tests for opening non-ELF files. · 4c6781bf
      Andrii Nakryiko authored
      Summary:
      Just testing that ElfFile returns expected error code and message for
      non-ELF files.
      
      Depends on D6410210
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6418365
      
      fbshipit-source-id: aaab3b9f3ca1a12d384ae98a1772b7f640115192
      4c6781bf
    • Sarang Masti's avatar
      Fix incorrect usages of folly::Synchronized · ecb501b7
      Sarang Masti authored
      Summary:
      We need to keep LockedPtr returned by rlock/wlock alive for the entire duration
      of the loop. Else we are working on a snapshot of the data structure, which
      might not be what we want.
      
      Reviewed By: aary, ot, luciang
      
      Differential Revision: D6426613
      
      fbshipit-source-id: 294c0a7b9d756f3116d5cdb3a7b3678f693e0754
      ecb501b7
  3. 28 Nov, 2017 6 commits
    • Adam Simpkins's avatar
      logging: convert assert() checks to FOLLY_SAFE_DCHECK() · d017a3ff
      Adam Simpkins authored
      Summary:
      Replace all assert() checks in the folly logging code with
      FOLLY_SAFE_DCHECK().
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6422885
      
      fbshipit-source-id: 5cb12dd59f2fe5d346f15b9d317abede8654a879
      d017a3ff
    • Alexander Kabaev's avatar
      Unbreak folly on x86-32 · 6e6f346d
      Alexander Kabaev authored
      Summary:
      _mm_cvtsi64_si128 and _mm_cvtsi128_si64 intrinsics are available on
      x86_64 only, so do not enable SSE instructions when compiling for 32 bits.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6422271
      
      fbshipit-source-id: a3c5cf3f2ee4b47d5a1edcf93cfff3d1f3fe907e
      6e6f346d
    • Yedidya Feldblum's avatar
      Remove some custom test main()'s in folly/experimental/symbolizer/ · 327ee770
      Yedidya Feldblum authored
      Summary:
      [Folly] Remove some custom test `main()`'s in `folly/experimental/symbolizer/`.
      
      In one of the two cases, we need to do some extra work to give `ElfFile` knowledge of the program path.
      
      Reviewed By: ot, Orvid, philippv, luciang
      
      Differential Revision: D6420517
      
      fbshipit-source-id: 11c694a8342abb6fff05c368f3cda2e6287ccb4b
      327ee770
    • Pingjia Shan's avatar
      Reset context shared_ptr in AsyncTimeout::cancelTimeout() · acbd746f
      Pingjia Shan authored
      Summary:
      This seems to fix issue in the attached task.
      
      `context_` is set in `scheduleTimeout()` and never gets reset. So when an AsyncTimeout object is held across requests by objects like `IOThreadPool0` in I/O Thread Pool used by wangle/acceptor/ConnectionManager, RequestContext object created for the last request is leaked until `IOThreadPool0` handles another request and overrides `context_`. In the issue described in attached task, unit test has single request, next request never comes in, RequestContext doesn't end until test service stops, logging feature relying on RequestContext dtor doesn't get called in time, and thus unit test fails.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6402268
      
      fbshipit-source-id: 200c6d358dfa6d7d9aa68ab05f6f1c7f4117b0ec
      acbd746f
    • Adam Simpkins's avatar
      logging: fix unused variable warning in non-debug builds · db13010c
      Adam Simpkins authored
      Summary:
      Use FOLLY_SAFE_DCHECK() rather than assert() to avoid compiler errors about
      this variable being unused in builds where assert() statements are disabled.
      
      Reviewed By: bolinfest, yfeldblum
      
      Differential Revision: D6421978
      
      fbshipit-source-id: ab1019fa1fa420856e1bf3bc39c441662c4efd75
      db13010c
    • Tom Jackson's avatar
      Use "auto" with GENERATOR instead of std::function · 2fef5f70
      Tom Jackson authored
      Summary: Now that we're using C++14 more broadly, we should use it to improve generator performance. This speeds up a microbenchmark >5x by removing type erasure.
      
      Reviewed By: philippv
      
      Differential Revision: D6398730
      
      fbshipit-source-id: 5809058a3b5ff0e66fd4b1e8954698944e1a7d09
      2fef5f70
  4. 27 Nov, 2017 4 commits
    • Xiangyu Bu's avatar
      Revise API to load cert/key in SSLContext. · 3e19d28a
      Xiangyu Bu authored
      Summary:
      When loading cert/key pair, order matters:
      (a) Wrong key will fail to load if a cert is loaded;
      (b) Wrong cert will succeed to load even if a private key is loaded.
      
      So this diff adds:
      (1) SSLContext::checkPrivateKey() -- must call for case (b).
      (2) SSLContext::loadCertKeyPairFromBufferPEM() -- use this if one loads both cert and key. Guaranteed to throw if cert/key mismatch.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6416280
      
      fbshipit-source-id: 8ae370883d46e9b5afb69c506c09fbf7ba82b1b9
      3e19d28a
    • Andrii Nakryiko's avatar
      Fix ElfFile crashing when opening short (<64 bytes) files. · 36ac1032
      Andrii Nakryiko authored
      Summary:
      ElfFile in init() assumes file is long enough to contain complete
      ElfHeader, which is 64 bytes long. This is not true for valid non-ELF files like
      short scripts. They shouldn't cause crash.
      
      Reviewed By: myreg
      
      Differential Revision: D6410210
      
      fbshipit-source-id: 28fd017d8de17c431d7d006a1655ade8a95994bd
      36ac1032
    • Dave Watson's avatar
      Use hazptr_local and hazptr_array · 0fd99413
      Dave Watson authored
      Summary: Use newest hazptr hotness in concurrenthashmap.  Shaves ~10% off of the single-thread find performance.
      
      Reviewed By: magedm
      
      Differential Revision: D6259947
      
      fbshipit-source-id: 7ecf99d38fdf8e311fca3313137e0fca5af3f165
      0fd99413
    • Bin Liu's avatar
      make `usingJEMalloc()` a constexpr when USE_JEMALLOC is defined · ee207f19
      Bin Liu authored
      Summary: This would save some function calls and reduce code size if we know we will use jemalloc.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6411810
      
      fbshipit-source-id: 9db471678e473945f5f00354c0a1285ecdb91de6
      ee207f19
  5. 26 Nov, 2017 1 commit
    • Giuseppe Ottaviano's avatar
      Add folly::hasher support for floating point types · 13e576ac
      Giuseppe Ottaviano authored
      Summary:
      Move `folly::hasher` closer to feature parity with `std::hash`.
      This is in order to replace some instances of `folly::hash::hash_combine(...)` with `folly::Hash()(...)` (`std::hash` is the identity for integers, which makes it an unsafe default for more sophisticated hash data structures, including open-addressing hash tables).
      
      The implementation is similar to `libstdc++`'s implementation, in that we handle separately the `0` case, because `0` and `-0` have different binary representations but are equal according to `operator==`, and hash the bytes otherwise. It is probably a little faster than `libstdc++`'s implementation, that delegates a out-of-line Murmur hash routine for arbitrary buffers, while this uses a minimal inlineable machine word hashing routine.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6410713
      
      fbshipit-source-id: 86d9e4ed8da04fffe283949825852e539ec7d5cf
      13e576ac
  6. 23 Nov, 2017 1 commit
    • Yedidya Feldblum's avatar
      QueuedImmediateExecutor tweaks · c45f2aa5
      Yedidya Feldblum authored
      Summary:
      [Folly] `QueuedImmediateExecutor` tweaks.
      
      * Add a leaky meyers singleton instance.
      * Make the thread-local queue an instance variable. Callers which want the singleton thread-local queue can use the singleton executor instance instance, for the same effect.
      * Simplify the body of `add`, and perform the thread-local lookup only once per invocation.
      
      Reviewed By: djwatson
      
      Differential Revision: D6399067
      
      fbshipit-source-id: 03904885a70c4b943141bd83868414d27232fd6a
      c45f2aa5
  7. 22 Nov, 2017 4 commits
    • Fuat Geleri's avatar
      Enable EventBase switching for AsyncSocket even if it has registered events · 6612bf09
      Fuat Geleri authored
      Summary:
      AsyncSocket will return isDetachable() -> false if there is a read callback set.
      So,
       - ignore event registration status
       - unregister the events when detaching and
       - register them back when attaching again.
      
      Reviewed By: afrind, eduardo-elizondo
      
      Differential Revision: D6331787
      
      fbshipit-source-id: fca4e6101f2d5666edbafa09116aa50f34cb084d
      6612bf09
    • Guo Xiao's avatar
      Link to unwind when using symbolizer · 277bd447
      Guo Xiao authored
      Summary:
      Fix #673
      Closes https://github.com/facebook/folly/pull/715
      
      Reviewed By: meyering
      
      Differential Revision: D6394731
      
      Pulled By: yfeldblum
      
      fbshipit-source-id: fb92828d0c633278601125dbc64cadf0cdbaa7fe
      277bd447
    • Dave Watson's avatar
      Add a fast path to folly::ThreadLocal · dc4be288
      Dave Watson authored
      Summary:
      Currently folly::ThreadLocal[Ptr] is pretty heavy-weight for a get():
      
      1) call instance(), take a static init guard, branch
      2) call getThreadEntry, check if thread_local is not null, branch
      3) check if id < threadEntry->capacity, branch
      4) Finally, return threadEntry->elements[id]
      
      If we have real thread_locals, we can do better by caching the capacity directly,
      combining all three checks:
      
      1) checkif id < threadLocalCapacityCheck, branch.  If not, do slow path.
      2) return threadEntry->elements[id].  Threadentry is never null if capacity > 0, and
          instance() setup work is called during the first getThreadEntry call when threadlocalcapacity == 0.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6379878
      
      fbshipit-source-id: 4fc7564bbb2f319d65875124026aef28d910ef06
      dc4be288
    • Andrii Grynenko's avatar
      Fix ASAN failure in FutureDAG test · 8bfce3ed
      Andrii Grynenko authored
      Summary: dogscience
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6387604
      
      fbshipit-source-id: 3e33da3e3c6801d6e2186637c8fb6adaf1eef6f6
      8bfce3ed
  8. 21 Nov, 2017 7 commits
    • Aaron Orenstein's avatar
      Add -Werror to -Wshadow-local and -Wshadow-compatible-local warning check · f9953b79
      Aaron Orenstein authored
      Summary:
      Without -Werror Apple LLVM will print a warning about -Wshadow-local
      and -Wshadow-compatible-local but won't fail.  This causes problems for
      downstream code which uses -Werror.
      
      Reviewed By: meyering
      
      Differential Revision: D6375771
      
      fbshipit-source-id: 4ac3a4ff46bdb76f1f07b0c8ba479a7d404c8c9b
      f9953b79
    • Adam Simpkins's avatar
      logging: add a LogConfig class and parsing code · 8af0eef9
      Adam Simpkins authored
      Summary:
      Add a new LogConfig class to represent the current configuration state of the
      LoggerDB.  This also includes code to parse config strings in both JSON and a
      simpler more human-writable format (intended primarily for use in command line
      arguments).
      
      I generally expect the human-writable format to be used mainly to configure log
      levels.  It also supports configuring log handler details as well, but the
      format for this data is slightly more cumbersome and will probably be harder
      for people to easily remember.
      
      The parsing code is intentionally kept as part of the 'init' library rather
      than the core 'logging' library so that other libraries that simply wish to log
      messages do not need to depend on it.  For instance, this would allow the folly
      JSON library to use the logging library without causing a circular dependency.
      
      Reviewed By: bolinfest
      
      Differential Revision: D6200560
      
      fbshipit-source-id: e4e3c7f941808251b6c7bcbbdac0210118675fb0
      8af0eef9
    • Dave Watson's avatar
      improve ThreadLocalBenchmark · 65d63573
      Dave Watson authored
      Summary:
      Benchmark appears to be memory bound, and is affected by different cpus.
      Add a new benchmark that does more work.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6380904
      
      fbshipit-source-id: 5cfbaab70379aa4a2923d957e1147d8486deeff7
      65d63573
    • Dave Watson's avatar
      fix hazptr_array move · 29b40639
      Dave Watson authored
      Summary: Broken, found via build failures in D6259947
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6374215
      
      fbshipit-source-id: 3a438b5416683c602966c2f029c6b7787acdccaa
      29b40639
    • Giuseppe Ottaviano's avatar
      Revert D6366352: [folly] Split get_default() into two for deferred default construction · 0f122a82
      Giuseppe Ottaviano authored
      Summary:
      This reverts commit db55b944ca63e565997094c11b90c4ebe98531ce
      
      bypass-lint
      
      Differential Revision: D6366352
      
      fbshipit-source-id: e25906409186b077ef9117aa524cc7c86314ae12
      0f122a82
    • Aaryaman Sagar's avatar
      Split get_default() into two for deferred default construction · d4d49d6c
      Aaryaman Sagar authored
      Summary:
      As it stood currently folly::get_default() would unnecessarily
      construct a value into the third parameter, which was unnecessary in the fast
      path where the element was found in the map
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6366352
      
      fbshipit-source-id: db55b944ca63e565997094c11b90c4ebe98531ce
      d4d49d6c
    • Dylan Yudaken's avatar
      correct usage of namespace std for coroutines_trait specialization · fe1ff9ef
      Dylan Yudaken authored
      Summary: When specializing traits you should not use the libcpp inline std namespace, but rather the plain one.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6373787
      
      fbshipit-source-id: d62ef9ccbf90c6f952db5f93f2377c920e68a809
      fe1ff9ef
  9. 20 Nov, 2017 4 commits
    • Christopher Dykes's avatar
      Work around a bug in MSVC name lookup within templated friend contexts · f5edd7dc
      Christopher Dykes authored
      Summary: MSVC has a bug that causes it to think that `once_flag` in this context resolves to the `using` statement further up, rather than `detail::once_flag`.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6376501
      
      fbshipit-source-id: 481dbd75ed21f3d519bd920258fa743f314668ad
      f5edd7dc
    • Yedidya Feldblum's avatar
      No need for a wrapping structure for posixTimeToDuration · 3c9a6e9a
      Yedidya Feldblum authored
      Summary:
      [Folly] No need for a wrapping structure for `posixTimeToDuration`.
      
      We can just use a variant of tag dispatch. In this variant, we pass to `posixTimeToDuration` a default-initialized value of the desired return type and we write overload templates for each possible variant. The argument is used purely for overload resolution and return type deduction, not for its runtime value. It is slightly different from tag dispatch because we do not use separate types which are purely tag types.
      
      Reviewed By: simpkins
      
      Differential Revision: D6371572
      
      fbshipit-source-id: 1987dee31fceec8733caa61495e96489dbf1ca39
      3c9a6e9a
    • Dave Watson's avatar
      Add free-function retire · 7046d43c
      Dave Watson authored
      Summary:
      Adds a hazptr_retire and domain::retire methods to hazptr.  They both allocate memory instead of inheriting.
      
      This will make implementation of google's cell proposal trivial, vs. D4754972 which felt overwraught.
      
      http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0561r0.html
      
      Reviewed By: magedm
      
      Differential Revision: D6361162
      
      fbshipit-source-id: 9f259f434139f960483b2ab7c5190d3807edcf52
      7046d43c
    • Tianjiao Yin's avatar
      fix SingletonTest · 79376dc7
      Tianjiao Yin authored
      Summary: There is no guarantee that this code will finish in 6 seconds (nor other threads will release singleton in 5 seconds), especially in ASAN mode. Though I don't have better idea, this diff will relax the condition in ASAN mode to make the unit-test less flaky.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6371692
      
      fbshipit-source-id: 58dd15cc0b3273719314c8b323ba88ee47e8ff61
      79376dc7
  10. 19 Nov, 2017 2 commits
    • Adam Simpkins's avatar
      implement to() conversions for std::chrono to timespec/timeval · dc91b004
      Adam Simpkins authored
      Summary:
      Add folly::to() conversions to convert between std::chrono::duration or
      std::chrono::time_point types and struct timespec or struct timeval types.
      
      To conform to the behavior of the existing arithmetic-to-arithmetic
      conversions, this code performs proper overflow checking and throws a
      `ConversionError` on overflow.  This unfortunately does make the code rather
      complicated compared to a non-checking implementation.
      
      Conversions between some unusual duration types is not implemented yet, and
      will fail at compile time if someone tries to use it.  This happens for
      durations where neither the numerator nor the denominator of the ratio is 1.
      For instance, 7/13ths of a second.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6356700
      
      fbshipit-source-id: 9dce8ab8f32d8c18089f32c7176a8abf3c3f11f7
      dc91b004
    • Pingjia Shan's avatar
      get rid of redundant calls to RequestContext::saveContext() · aa7f8dcd
      Pingjia Shan authored
      Summary:
      In the past, these calls were required.
      
      Used to solve static destruction ordering issue.  Any static object that uses RequestContext must call this function in its constructor.
      
      That is when we were using `static folly::ThreadLocal<std::shared_ptr<RequestContext>>`, which was non-leaky.
      
      The problem being addressed is when we have some code of the form:
      
      ```lang=c++
      void doWork() {
        static EventBase eb;
      }
      ```
      
      But now we are using `SingletonThreadLocal<std::shared_ptr<RequestContext>>`, which is leaky.
      
      So the issue that these calls were there to address seems to have been resolved.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6332597
      
      fbshipit-source-id: c6aba6620ef2fb3a344ea20f56c8b9c0cdf42c70
      aa7f8dcd
  11. 18 Nov, 2017 2 commits
    • Pádraig Brady's avatar
      folly: support FOLLY_FALLTHROUGH on GCC · 13cc05ea
      Pádraig Brady authored
      Summary:
      This is required to avoid new GCC 7 -Wimplict-fallthrough warnings.
      We also update to use the C++17 [[fallthrough]] attribute if supported.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6367140
      
      fbshipit-source-id: d5380983cb300f944df9c2885d0faa0155994be7
      13cc05ea
    • Tianjiao Yin's avatar
      reduce the number of iteration when test SharedMutexTest in ASAN mode · bcd0d782
      Tianjiao Yin authored
      Summary: This unit-test takes too long to finish in ASAN mode from heavily loaded system (more than 10 minutes).
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6362111
      
      fbshipit-source-id: b097eff60f88ace4fb869132598806700804e267
      bcd0d782
  12. 17 Nov, 2017 2 commits
    • Pingjia Shan's avatar
      Remove unnecessary white line · ba4282e4
      Pingjia Shan authored
      Summary:
      Seems to be causing rendering error:
      
      {F114885894}
      
      Created from Diffusion's 'Open in Editor' feature.
      
      Reviewed By: lskuff
      
      Differential Revision: D6358188
      
      fbshipit-source-id: 3c437007b425bcadb79a71807ad92d520a70e8cb
      ba4282e4
    • Jim Meyering's avatar
      folly/fibers/test/FibersTest.cpp: accommodate ASAN's detect_stack_use_after_return=1 · a447cb9e
      Jim Meyering authored
      Summary:
      With ASAN enabled (actually, only with ASAN *and* its detect_stack_use_after_return=1 option),
      the addTaskFinally test would fail.  This adapts to accommodate the larger stack offsets.
      Also, use EXPECT_GT and EXPECT_LT rather than EXPECT_TRUE.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6353666
      
      fbshipit-source-id: 39e04caffa7b24cde97c749686c7e651a071dcec
      a447cb9e