1. 02 Dec, 2017 1 commit
    • Adam Simpkins's avatar
      logging: update initialization code to use the new LogConfig logic · 6072ce3b
      Adam Simpkins authored
      Summary:
      Replace the initLoggingGlogStyle() function with a more generic initLogging()
      function that accepts a log config string to be parsed with parseLogConfig().
      
      Reviewed By: bolinfest, yfeldblum
      
      Differential Revision: D6342086
      
      fbshipit-source-id: fb1bffd11f190b70e03e2ccbf2b30be08d655242
      6072ce3b
  2. 01 Dec, 2017 4 commits
    • Igor Sugak's avatar
      unify FOLLY_SANITIZE existing uses and add a few new · d08cf162
      Igor Sugak authored
      Reviewed By: yfeldblum
      
      Differential Revision: D6453972
      
      fbshipit-source-id: fa083c0b782188f5f45a2a354045aee03e528829
      d08cf162
    • Adam Simpkins's avatar
      implement chrono conversions for unusual duration types · 730db44f
      Adam Simpkins authored
      Summary:
      Implement conversions between std::chrono::duration and POSIX-style time
      structures even when neither the numerator nor the denominator of the duration
      ratio are 1.
      
      Both of these are done by first converting to an intermediate type where the
      numerator is 1, and then using the conversion routines for that case.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6366647
      
      fbshipit-source-id: 8f9495fb4101cac6d8b4cf0353a679107007b298
      730db44f
    • Dave Watson's avatar
      Move threadlocal_detail::Atfork to its own file · f089f1fd
      Dave Watson authored
      Summary: As title
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6440723
      
      fbshipit-source-id: 3168d7bb616ae0ff3fe42f7584c5a255c4953875
      f089f1fd
    • Bin Liu's avatar
      fix ASAN build · df9c2e3e
      Bin Liu authored
      Summary:
      Fix bug in non-jemalloc build.
      
      (Note: this ignores all push blocking failures!)
      
      Reviewed By: igorsugak
      
      Differential Revision: D6451342
      
      fbshipit-source-id: 44725d4e0c685a59325fa3d08877d0f62120c5c2
      df9c2e3e
  3. 30 Nov, 2017 13 commits
    • Yedidya Feldblum's avatar
      Let SaturatingSemaphore::try_wait be non-const and add ready · 48d3f5e1
      Yedidya Feldblum authored
      Summary:
      [Folly] Let `SaturatingSemaphore::try_wait` be non-`const` and add `ready`.
      
      For internal API consistency.
      
      Reviewed By: magedm
      
      Differential Revision: D6450089
      
      fbshipit-source-id: 65b9b723672521710a69719b192eb2922a27b778
      48d3f5e1
    • Igor Sugak's avatar
      fix build when sanitizers are enabled and jemalloc is disabled · b71a1b76
      Igor Sugak authored
      Summary: Add preprocessor checks to guard jemalloc headers if any sanitizer is enabled.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6441578
      
      fbshipit-source-id: 85364fca5af33c95cd05f232d1e3f9cbac9a4120
      b71a1b76
    • Yedidya Feldblum's avatar
      Some fixes for custom conversions of enums · 96ac8f8f
      Yedidya Feldblum authored
      Summary:
      [Folly] Some fixes for custom conversions of enums.
      
      Of note, `to` was defined for enum -> all conversions, including enum -> string conversions, but we actually want enum -> string conversions to be done via ADL-discovered toAppend.
      
      Reviewed By: ot
      
      Differential Revision: D6411250
      
      fbshipit-source-id: 852b64309e6adf1c68e5153635cb29632e2d86d4
      96ac8f8f
    • Maged Michael's avatar
      Saturating semaphore · 644a73aa
      Maged Michael authored
      Summary:
      SaturatingSemaphore is a flag that allows:
      - multiple concurrent posters
      - multiple concurrent waiters
      - idempotent posting
      - non-destructive waiting
      - blocking and spinning
      - pre-block spin time control
      
      ```
      /// SaturatingSemaphore is a flag that allows concurrent posting by
      /// multiple posters and concurrent non-destructive waiting by
      /// multiple waiters.
      ///
      /// A SaturatingSemaphore allows one or more waiter threads to check,
      /// spin, or block, indefinitely or with timeout, for a flag to be set
      /// by one or more poster threads. By setting the flag, posters
      /// announce to waiters (that may be already waiting or will check
      /// the flag in the future) that some condition is true. Posts to an
      /// already set flag are idempotent.
      ```
      
      Reviewed By: djwatson
      
      Differential Revision: D6379704
      
      fbshipit-source-id: 59aed76caa2d159639e75425a778a9c63f18f375
      644a73aa
    • Igor Sugak's avatar
      add FOLLY_SANITIZE macro · 614eb717
      Igor Sugak authored
      Summary: Most of the time we want to guard some code when any of the sanitizers are enabled not just one.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6429720
      
      fbshipit-source-id: 7157a0a13f2b2acabd36a19c61b9fdbf38ba96b8
      614eb717
    • Adam Simpkins's avatar
      logging: add a LogConfig::update() method · 153c4233
      Adam Simpkins authored
      Summary:
      Add a method for merging the settings from two LogConfig objects.
      This allows LogConfig objects to be merged before applying them to the
      LoggerDB.  The effects are the same as two sequential LoggerDB::updateConfig()
      calls, but without having to apply the intermediate state to the LoggerDB.
      
      Reviewed By: bolinfest
      
      Differential Revision: D6342085
      
      fbshipit-source-id: 0f8a1b7d8d195a80bc74342444dd3152d331fcb6
      153c4233
    • Adam Simpkins's avatar
      logging: add an assert for the FLAG_INHERIT bit · b26334e5
      Adam Simpkins authored
      Summary:
      Add a static_assert() to check that the FLAG_INHERIT bit does not conflict with
      valid LogLevel values.
      
      Reviewed By: bolinfest
      
      Differential Revision: D6341242
      
      fbshipit-source-id: cec284623c8a612a3c54f1b593d169310790616e
      b26334e5
    • Adam Simpkins's avatar
      logging: remove comment about LogLevel::ERROR · 89353dca
      Adam Simpkins authored
      Summary:
      The logging code used to support a LogLevel::ERROR value on non-Windows
      platforms.  I removed it in D5288600 to make all platforms consistent, but I
      forgot to remove this comment describing it.
      
      Reviewed By: bolinfest
      
      Differential Revision: D6341244
      
      fbshipit-source-id: 054427d342066f354c859b12611b907dc2d4fa35
      89353dca
    • Adam Simpkins's avatar
      logging: add LoggerDB::updateConfig() and resetConfig() · 01d4b7d6
      Adam Simpkins authored
      Summary: Add methods for applying config changes from a LogConfig object to the LoggerDB.
      
      Reviewed By: bolinfest
      
      Differential Revision: D6200564
      
      fbshipit-source-id: a25eb99e84b2885bf6853e2222db0d7432a6c37b
      01d4b7d6
    • Adam Simpkins's avatar
      logging: add a LoggerDB::getConfig() method · b6e14b7b
      Adam Simpkins authored
      Summary: Add a method to get the current LogConfig state from the LoggerDB.
      
      Reviewed By: bolinfest
      
      Differential Revision: D6200596
      
      fbshipit-source-id: 3bc57d498a5d25d19099d861376d71ea9f7e4039
      b6e14b7b
    • Adam Simpkins's avatar
      logging: don't clamp the log level to DFATAL in debug builds · 2fa201bf
      Adam Simpkins authored
      Summary:
      Remove the logic that clamps the log level to DFATAL in debug builds.
      
      This behavior logically makes some sense, but results in subtle and potentially
      confusing behavior changes.  It seems confusing that after calling
      `setLevel(LogLevel::MAX_LEVEL)` on a log category, calling `getLevel()` on that
      object would return `DFATAL` rather than `MAX_LEVEL`.  We also weren't clamping
      the level to DFATAL consistently: `setLevel()` would clamp the value, but on
      construction the initial level was still set to MAX_LEVEL rather than DFATAL.
      This resulted in some issues when implementing `LoggerDB::getConfig()` since we
      could not consistently detect if a log category was using the default level
      settings.  Rather than fix this inconsistency it seems better to simply remove
      this clamping behavior.
      
      This means that it is possible for users to disable DFATAL log messages even in
      debug builds if they really want to.  Previously this was only allowed in
      release builds.  This protection doesn't really seem all that
      valuable--presumably most developers won't really want to do this, and if they
      really do request this configuration it doesn't seem all that bad to honor it.
      
      Reviewed By: bolinfest, yfeldblum
      
      Differential Revision: D6200569
      
      fbshipit-source-id: 83ef8e2e4d3b61bc5b105038cbe3132979e9ac67
      2fa201bf
    • Adam Simpkins's avatar
      logging: add a LogHandler::getConfig() method · 95d99350
      Adam Simpkins authored
      Summary:
      Add a method to LogHandler to return its current configuration.  This will
      make it possible to query the LoggerDB for its current configuration state.
      
      Reviewed By: bolinfest
      
      Differential Revision: D6200563
      
      fbshipit-source-id: 2b8b9752bbeb26c8aac28d1a73b7e2312fd198c8
      95d99350
    • Adam Simpkins's avatar
      logging: add a LogHandler registry to LoggerDB · ae69959c
      Adam Simpkins authored
      Summary:
      Update the LoggerDB to track a list of LogHandlers by name, and
      LogHandlerFactories by handler type.
      
      This will be needed to support updating the LoggerDB configuration from a
      LogConfig object.
      
      Reviewed By: bolinfest
      
      Differential Revision: D6200562
      
      fbshipit-source-id: e365b4e0df65aa5aaa34e118eb3cee9c9c45cb05
      ae69959c
  4. 29 Nov, 2017 4 commits
    • Yedidya Feldblum's avatar
      Synchronize coupled caches in folly::threadlocal_detail::StaticMeta · 73c52b9f
      Yedidya Feldblum authored
      Summary:
      [Folly] Synchronize coupled caches in `folly::threadlocal_detail::StaticMeta`.
      
      The caches should be set together, and only together, because they are coupled. This prevents bugs where one function that sets one cache but not the other cache is inlined into the caller in one module, and another function that reads both caches is inlined into the caller in another module.
      
      Reviewed By: djwatson
      
      Differential Revision: D6435175
      
      fbshipit-source-id: 846c4972b40e525f2c04da6e6609c2ad54f674c0
      73c52b9f
    • Adam Simpkins's avatar
      logging: add a FileHandlerFactory class · 70230b7a
      Adam Simpkins authored
      Summary:
      Add a new LogHandlerFactory interface for creating LogHandler objects from a
      LogHandlerConfig.
      
      Also add an initial FileHandlerFactory implementation capable of creating
      LogHandler objects that write to a file descriptor.
      
      Reviewed By: bolinfest
      
      Differential Revision: D6200567
      
      fbshipit-source-id: 14b86fc14ad475223aa4b57d45c40638b48c7594
      70230b7a
    • Andrii Nakryiko's avatar
      Add unit tests for opening non-ELF files. · 4c6781bf
      Andrii Nakryiko authored
      Summary:
      Just testing that ElfFile returns expected error code and message for
      non-ELF files.
      
      Depends on D6410210
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6418365
      
      fbshipit-source-id: aaab3b9f3ca1a12d384ae98a1772b7f640115192
      4c6781bf
    • Sarang Masti's avatar
      Fix incorrect usages of folly::Synchronized · ecb501b7
      Sarang Masti authored
      Summary:
      We need to keep LockedPtr returned by rlock/wlock alive for the entire duration
      of the loop. Else we are working on a snapshot of the data structure, which
      might not be what we want.
      
      Reviewed By: aary, ot, luciang
      
      Differential Revision: D6426613
      
      fbshipit-source-id: 294c0a7b9d756f3116d5cdb3a7b3678f693e0754
      ecb501b7
  5. 28 Nov, 2017 6 commits
    • Adam Simpkins's avatar
      logging: convert assert() checks to FOLLY_SAFE_DCHECK() · d017a3ff
      Adam Simpkins authored
      Summary:
      Replace all assert() checks in the folly logging code with
      FOLLY_SAFE_DCHECK().
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6422885
      
      fbshipit-source-id: 5cb12dd59f2fe5d346f15b9d317abede8654a879
      d017a3ff
    • Alexander Kabaev's avatar
      Unbreak folly on x86-32 · 6e6f346d
      Alexander Kabaev authored
      Summary:
      _mm_cvtsi64_si128 and _mm_cvtsi128_si64 intrinsics are available on
      x86_64 only, so do not enable SSE instructions when compiling for 32 bits.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6422271
      
      fbshipit-source-id: a3c5cf3f2ee4b47d5a1edcf93cfff3d1f3fe907e
      6e6f346d
    • Yedidya Feldblum's avatar
      Remove some custom test main()'s in folly/experimental/symbolizer/ · 327ee770
      Yedidya Feldblum authored
      Summary:
      [Folly] Remove some custom test `main()`'s in `folly/experimental/symbolizer/`.
      
      In one of the two cases, we need to do some extra work to give `ElfFile` knowledge of the program path.
      
      Reviewed By: ot, Orvid, philippv, luciang
      
      Differential Revision: D6420517
      
      fbshipit-source-id: 11c694a8342abb6fff05c368f3cda2e6287ccb4b
      327ee770
    • Pingjia Shan's avatar
      Reset context shared_ptr in AsyncTimeout::cancelTimeout() · acbd746f
      Pingjia Shan authored
      Summary:
      This seems to fix issue in the attached task.
      
      `context_` is set in `scheduleTimeout()` and never gets reset. So when an AsyncTimeout object is held across requests by objects like `IOThreadPool0` in I/O Thread Pool used by wangle/acceptor/ConnectionManager, RequestContext object created for the last request is leaked until `IOThreadPool0` handles another request and overrides `context_`. In the issue described in attached task, unit test has single request, next request never comes in, RequestContext doesn't end until test service stops, logging feature relying on RequestContext dtor doesn't get called in time, and thus unit test fails.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6402268
      
      fbshipit-source-id: 200c6d358dfa6d7d9aa68ab05f6f1c7f4117b0ec
      acbd746f
    • Adam Simpkins's avatar
      logging: fix unused variable warning in non-debug builds · db13010c
      Adam Simpkins authored
      Summary:
      Use FOLLY_SAFE_DCHECK() rather than assert() to avoid compiler errors about
      this variable being unused in builds where assert() statements are disabled.
      
      Reviewed By: bolinfest, yfeldblum
      
      Differential Revision: D6421978
      
      fbshipit-source-id: ab1019fa1fa420856e1bf3bc39c441662c4efd75
      db13010c
    • Tom Jackson's avatar
      Use "auto" with GENERATOR instead of std::function · 2fef5f70
      Tom Jackson authored
      Summary: Now that we're using C++14 more broadly, we should use it to improve generator performance. This speeds up a microbenchmark >5x by removing type erasure.
      
      Reviewed By: philippv
      
      Differential Revision: D6398730
      
      fbshipit-source-id: 5809058a3b5ff0e66fd4b1e8954698944e1a7d09
      2fef5f70
  6. 27 Nov, 2017 4 commits
    • Xiangyu Bu's avatar
      Revise API to load cert/key in SSLContext. · 3e19d28a
      Xiangyu Bu authored
      Summary:
      When loading cert/key pair, order matters:
      (a) Wrong key will fail to load if a cert is loaded;
      (b) Wrong cert will succeed to load even if a private key is loaded.
      
      So this diff adds:
      (1) SSLContext::checkPrivateKey() -- must call for case (b).
      (2) SSLContext::loadCertKeyPairFromBufferPEM() -- use this if one loads both cert and key. Guaranteed to throw if cert/key mismatch.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6416280
      
      fbshipit-source-id: 8ae370883d46e9b5afb69c506c09fbf7ba82b1b9
      3e19d28a
    • Andrii Nakryiko's avatar
      Fix ElfFile crashing when opening short (<64 bytes) files. · 36ac1032
      Andrii Nakryiko authored
      Summary:
      ElfFile in init() assumes file is long enough to contain complete
      ElfHeader, which is 64 bytes long. This is not true for valid non-ELF files like
      short scripts. They shouldn't cause crash.
      
      Reviewed By: myreg
      
      Differential Revision: D6410210
      
      fbshipit-source-id: 28fd017d8de17c431d7d006a1655ade8a95994bd
      36ac1032
    • Dave Watson's avatar
      Use hazptr_local and hazptr_array · 0fd99413
      Dave Watson authored
      Summary: Use newest hazptr hotness in concurrenthashmap.  Shaves ~10% off of the single-thread find performance.
      
      Reviewed By: magedm
      
      Differential Revision: D6259947
      
      fbshipit-source-id: 7ecf99d38fdf8e311fca3313137e0fca5af3f165
      0fd99413
    • Bin Liu's avatar
      make `usingJEMalloc()` a constexpr when USE_JEMALLOC is defined · ee207f19
      Bin Liu authored
      Summary: This would save some function calls and reduce code size if we know we will use jemalloc.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6411810
      
      fbshipit-source-id: 9db471678e473945f5f00354c0a1285ecdb91de6
      ee207f19
  7. 26 Nov, 2017 1 commit
    • Giuseppe Ottaviano's avatar
      Add folly::hasher support for floating point types · 13e576ac
      Giuseppe Ottaviano authored
      Summary:
      Move `folly::hasher` closer to feature parity with `std::hash`.
      This is in order to replace some instances of `folly::hash::hash_combine(...)` with `folly::Hash()(...)` (`std::hash` is the identity for integers, which makes it an unsafe default for more sophisticated hash data structures, including open-addressing hash tables).
      
      The implementation is similar to `libstdc++`'s implementation, in that we handle separately the `0` case, because `0` and `-0` have different binary representations but are equal according to `operator==`, and hash the bytes otherwise. It is probably a little faster than `libstdc++`'s implementation, that delegates a out-of-line Murmur hash routine for arbitrary buffers, while this uses a minimal inlineable machine word hashing routine.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6410713
      
      fbshipit-source-id: 86d9e4ed8da04fffe283949825852e539ec7d5cf
      13e576ac
  8. 23 Nov, 2017 1 commit
    • Yedidya Feldblum's avatar
      QueuedImmediateExecutor tweaks · c45f2aa5
      Yedidya Feldblum authored
      Summary:
      [Folly] `QueuedImmediateExecutor` tweaks.
      
      * Add a leaky meyers singleton instance.
      * Make the thread-local queue an instance variable. Callers which want the singleton thread-local queue can use the singleton executor instance instance, for the same effect.
      * Simplify the body of `add`, and perform the thread-local lookup only once per invocation.
      
      Reviewed By: djwatson
      
      Differential Revision: D6399067
      
      fbshipit-source-id: 03904885a70c4b943141bd83868414d27232fd6a
      c45f2aa5
  9. 22 Nov, 2017 4 commits
    • Fuat Geleri's avatar
      Enable EventBase switching for AsyncSocket even if it has registered events · 6612bf09
      Fuat Geleri authored
      Summary:
      AsyncSocket will return isDetachable() -> false if there is a read callback set.
      So,
       - ignore event registration status
       - unregister the events when detaching and
       - register them back when attaching again.
      
      Reviewed By: afrind, eduardo-elizondo
      
      Differential Revision: D6331787
      
      fbshipit-source-id: fca4e6101f2d5666edbafa09116aa50f34cb084d
      6612bf09
    • Guo Xiao's avatar
      Link to unwind when using symbolizer · 277bd447
      Guo Xiao authored
      Summary:
      Fix #673
      Closes https://github.com/facebook/folly/pull/715
      
      Reviewed By: meyering
      
      Differential Revision: D6394731
      
      Pulled By: yfeldblum
      
      fbshipit-source-id: fb92828d0c633278601125dbc64cadf0cdbaa7fe
      277bd447
    • Dave Watson's avatar
      Add a fast path to folly::ThreadLocal · dc4be288
      Dave Watson authored
      Summary:
      Currently folly::ThreadLocal[Ptr] is pretty heavy-weight for a get():
      
      1) call instance(), take a static init guard, branch
      2) call getThreadEntry, check if thread_local is not null, branch
      3) check if id < threadEntry->capacity, branch
      4) Finally, return threadEntry->elements[id]
      
      If we have real thread_locals, we can do better by caching the capacity directly,
      combining all three checks:
      
      1) checkif id < threadLocalCapacityCheck, branch.  If not, do slow path.
      2) return threadEntry->elements[id].  Threadentry is never null if capacity > 0, and
          instance() setup work is called during the first getThreadEntry call when threadlocalcapacity == 0.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6379878
      
      fbshipit-source-id: 4fc7564bbb2f319d65875124026aef28d910ef06
      dc4be288
    • Andrii Grynenko's avatar
      Fix ASAN failure in FutureDAG test · 8bfce3ed
      Andrii Grynenko authored
      Summary: dogscience
      
      Reviewed By: yfeldblum
      
      Differential Revision: D6387604
      
      fbshipit-source-id: 3e33da3e3c6801d6e2186637c8fb6adaf1eef6f6
      8bfce3ed
  10. 21 Nov, 2017 2 commits
    • Aaron Orenstein's avatar
      Add -Werror to -Wshadow-local and -Wshadow-compatible-local warning check · f9953b79
      Aaron Orenstein authored
      Summary:
      Without -Werror Apple LLVM will print a warning about -Wshadow-local
      and -Wshadow-compatible-local but won't fail.  This causes problems for
      downstream code which uses -Werror.
      
      Reviewed By: meyering
      
      Differential Revision: D6375771
      
      fbshipit-source-id: 4ac3a4ff46bdb76f1f07b0c8ba479a7d404c8c9b
      f9953b79
    • Adam Simpkins's avatar
      logging: add a LogConfig class and parsing code · 8af0eef9
      Adam Simpkins authored
      Summary:
      Add a new LogConfig class to represent the current configuration state of the
      LoggerDB.  This also includes code to parse config strings in both JSON and a
      simpler more human-writable format (intended primarily for use in command line
      arguments).
      
      I generally expect the human-writable format to be used mainly to configure log
      levels.  It also supports configuring log handler details as well, but the
      format for this data is slightly more cumbersome and will probably be harder
      for people to easily remember.
      
      The parsing code is intentionally kept as part of the 'init' library rather
      than the core 'logging' library so that other libraries that simply wish to log
      messages do not need to depend on it.  For instance, this would allow the folly
      JSON library to use the logging library without causing a circular dependency.
      
      Reviewed By: bolinfest
      
      Differential Revision: D6200560
      
      fbshipit-source-id: e4e3c7f941808251b6c7bcbbdac0210118675fb0
      8af0eef9