1. 11 Aug, 2016 5 commits
    • Max Wang's avatar
      Use _r_debug instead of /proc/<pid>/maps for folly::symbolizer · ddeb302a
      Max Wang authored
      Summary:
      Using _r_debug offers a number of benefits:
      - It allows us to symbolize addresses in data segments like .bss that
        are marked "[heap]" in /proc/<pid>/maps.
      - It requires a lot less code.
      
      It also fixes a preexisting bug where we would fail to symbolize
      addresses that fell in any section past the first in any position-
      independent objects.  Since `elfFile->getBaseAddress()` should always
      return 0 for any PIEs, we wouldn't correctly adjust the address to be
      ELF-relative when performing symbol lookup.  (For the first section,
      we just used the start of the range we found in /perf/<pid>/maps.)
      
      There is a minor downside:
      - We have to open an ELF header for each object in order to check if a
        given address is in the range of some segment.  Before, we used
        /proc/<pid>/maps to make the range check, and only opened the header
        once we knew it was the one we wanted.  In the common case, however,
        where the addresses are from our own executable, we don't open any
        more files than before (and, in fact, one fewer).
      
      Reviewed By: luciang
      
      Differential Revision: D3695872
      
      fbshipit-source-id: 9bdcc77da4d658ffad5c671f9b8ea65a471ed64f
      ddeb302a
    • Yedidya Feldblum's avatar
      Remove ConditionallyExistent, because it violates the C++ spec · 7ef3c1b2
      Yedidya Feldblum authored
      Summary:
      [Folly] Remove `ConditionallyExistent`, because it violates the C++ spec.
      
      The spec says that struct and class types occupy at least one byte. But the point of this class is to occupy zero bytes when the condition is false. That can't be made to work.
      
      GCC and Clang support an extension allowing, in some cases, structs to occupy no space at all. But it violates the spec, and MSVC does not support the extension.
      
      There is, sort of, the possibility of empty-base-class-optimization. But it will be very painful to use, and it will only work in some cases.
      
      Since this is broken now, and fixing it would violate the C++ spec and break this under MSVC, it's better just to remove it.
      
      Reviewed By: nbronson, Orvid
      
      Differential Revision: D3696371
      
      fbshipit-source-id: c475c6e15d9ff1bc4c44dc7e336ce74e6db640ef
      7ef3c1b2
    • Sven Over's avatar
      Let applyTuple accept any number of tuples · efc54c28
      Sven Over authored
      Summary:
      Instead of passing exactly one tuple of arguments to applyTuple,
      this diff allows to pass any number (zero or more) of tuples.
      
      This can be very handy when piecing together arguments for
      a function call.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3681499
      
      fbshipit-source-id: a75a448636759f71db8d303e9dccada5b559af54
      efc54c28
    • Lucian Grijincu's avatar
      __google_stl_debug_vector: erase() at invalid position:... · ef50b189
      Lucian Grijincu authored
      __google_stl_debug_vector: erase() at invalid position: folly/experimental/test:future_dag_test - FutureDAGTest.RemoveNodeComplex
      
      Summary:
      ```
      [ RUN      ] FutureDAGTest.RemoveNodeComplex
      terminate called after throwing an instance of 'std::out_of_range'
        what():  erase() at invalid position
      *** Aborted at 1470867850 (Unix time, try 'date -d 1470867850') ***
      *** Signal 6 (SIGABRT) (0x133c001ed107) received by PID 2019591 (pthread TID 0x7ff1e6df5980) (linux TID 2019591) (maybe from PID 2019591, UID 4924), stack trace: ***
          @ 00007ff1e702020f folly::symbolizer::(anonymous namespace)::innerSignalHandler(int, siginfo_t*, void*)
          @ 00007ff1e701faf1 folly::symbolizer::(anonymous namespace)::signalHandler(int, siginfo_t*, void*)
          @ 00007ff1e87c714f (unknown)
          @ 00007ff1e7cf6d02 gsignal
          @ 00007ff1e7cf8af5 abort
          @ 00007ff1e73880e4 __gnu_cxx::__verbose_terminate_handler()
          @ 00007ff1e7385ee5 __cxxabiv1::__terminate(void (*)())
          @ 00007ff1e7385f30 std::terminate()
          @ 00007ff1e73861c0 __cxa_throw
          @ 00007ff1e740ae00 std::__throw_out_of_range(char const*)
          @ 00000000004483db std::vector<folly::FutureDAG::Node, std::allocator<folly::FutureDAG::Node> >::_M_erase(__gnu_cxx::__normal_iterator<folly::FutureDAG::Node*, std::vector<folly::FutureDAG::Node, std::allocator<folly::FutureDAG::Node> > >)
          @ 00000000004478fa std::vector<folly::FutureDAG::Node, std::allocator<folly::FutureDAG::Node> >::erase(__gnu_cxx::__normal_iterator<folly::FutureDAG::Node const*, std::vector<folly::FutureDAG::Node, std::allocator<folly::FutureDAG::Node> > >)
          @ 000000000044770a folly::FutureDAG::remove(unsigned long)
          @ 0000000000424bc2 FutureDAGTest::remove(unsigned long)
          @ 000000000041f7ed FutureDAGTest_RemoveNodeComplex_Test::TestBody()
          @ 00007ff1e9b19361 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)
                             /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/src/gtest.cc:2420
                             -> /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/src/gtest-all.cc
          @ 00007ff1e9b029de testing::Test::Run()
                             /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/src/gtest.cc:2437
                             -> /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/src/gtest-all.cc
          @ 00007ff1e9b02aca testing::TestInfo::Run()
                             /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/src/gtest.cc:2612
                             -> /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/src/gtest-all.cc
          @ 00007ff1e9b02cfc testing::TestCase::Run()
                             /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/src/gtest.cc:2730
                             -> /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/src/gtest-all.cc
          @ 00007ff1e9b0d2a7 testing::internal::UnitTestImpl::RunAllTests()
                             /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/src/gtest.cc:4602
                             -> /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/src/gtest-all.cc
          @ 00007ff1e9b0d60c testing::UnitTest::Run()
                             /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/src/gtest.cc:2420
                             -> /home/engshare/third-party2/gtest/1.7.0/src/gtest-1.7.0/src/gtest-all.cc
          @ 00007ff1eb64ec70 RUN_ALL_TESTS()
          @ 00007ff1eb64ec3c main
          @ 00007ff1e7ce10f5 __libc_start_main
          @ 000000000041f2d2 (unknown)
      Summary (total time 0.07s):
        PASS: 0
        FAIL: 0
        SKIP: 0
        FATAL: 1
          folly/experimental/test:future_dag_test - FutureDAGTest.RemoveNodeComplex
        TIMEOUT: 0
      ````
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3700956
      
      fbshipit-source-id: dffca9460ccca1d8069218c5b7310e4d6e061697
      ef50b189
    • Shayan Mohanty's avatar
      Polymorphic Functor implementation in Folly::FutureDAG · 61aaac89
      Shayan Mohanty authored
      Summary:
      Implements a polymorphic functor for FutureDAGs. In order for FutureDAGs to be stateful they must be wrapped by a class of some sort. This is a really common pattern which we've been using in Gossit (and further - across the RedWood stack) in order to maintain state, and we feel it's generalized enough to be useful elsewhere.
      
      `state` is an instance of the type declared in the template, and the exec* methods wrap go().get() so client-side implementations only have to touch the functor after construction in order to drive their DAGs.
      
      Reviewed By: tjkswaine
      
      Differential Revision: D3685651
      
      fbshipit-source-id: 81169aefcff13ac8cc6cbb6bef6d90047732ad8a
      61aaac89
  2. 10 Aug, 2016 11 commits
    • Michael Lee's avatar
      Gate SysMembarrier to not FOLLY_MOBILE platforms · 18aab924
      Michael Lee authored
      Summary: Gate `kIsLinux` to not include `FOLLY_MOBILE` because "mobile" linux does not really behave like real linux.
      
      Differential Revision: D3697717
      
      fbshipit-source-id: 1b0f4208d2f71c35399c30f20a71bfa4ba4724e8
      18aab924
    • Dave Watson's avatar
      Fix scheduling bug · 09a84b28
      Dave Watson authored
      Summary:
      Noticed this while debugging other timerwheel changes.  Scheduling new events in callbacks may result in us *running them right away* if they land in a bucket we are currently processing.
      
      Instead, round up all the timeouts, then run them separately.  This means you can never schedule a timeout that will run immediately (0ticks), but that's probably fine.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3679413
      
      fbshipit-source-id: 7e18632f23ea33c072c2718e30b378641895b094
      09a84b28
    • Philip Pronin's avatar
      fix bug in FBString::find · 28c41287
      Philip Pronin authored
      Summary:
      Standard (21.4.7.2 / 1):
      
      > Determines the lowest position xpos, if possible, such that both of the following conditions obtain:
      - pos <= xpos and xpos + str.size() <= size();
      - traits::eq(at(xpos+I), str.at(I)) for all elements I of the string controlled
      by str.
      
      The existing logic violates the first requirement for `str.size() == 0` by
      unconditionally returning `pos`.
      
      Reviewed By: ot, Gownta
      
      Differential Revision: D3698862
      
      fbshipit-source-id: 9622f1b99b259d2d81ae83795dff9cd94619c725
      28c41287
    • Christopher Dykes's avatar
      Mark FormatArg::error as noreturn · b62a976d
      Christopher Dykes authored
      Summary: Because, if no inlining is enabled, the absence of this attribute caused compile errors due to functions that were expected to return a value not returning a value.
      
      Reviewed By: meyering
      
      Differential Revision: D3698413
      
      fbshipit-source-id: 8f3505b17a2fa7b9710e3fb56d18c6ca00feacb3
      b62a976d
    • Christopher Dykes's avatar
      Don't attempt to call getsockname before binding an async socket · 76a4de97
      Christopher Dykes authored
      Summary: With WinSock, calling `getsockname` on a socket before it's been bound will always result in an error, so make sure to bind the socket first.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3698112
      
      fbshipit-source-id: e9efe05323b242add3808ee1a6fec2593beb04ac
      76a4de97
    • Christopher Dykes's avatar
      Special-case /dev/null in open in the portability header · c326c636
      Christopher Dykes authored
      Summary: `/dev/null` doesn't exist on Windows, but thankfully, `NUL` does, and has the same semantics.
      
      Reviewed By: meyering
      
      Differential Revision: D3698007
      
      fbshipit-source-id: 5ef31c6576f988dd747ea3c39e296c244bc640b7
      c326c636
    • Andrii Grynenko's avatar
      Use ReadMostlyMainPtrDeleter in folly::Singleton · aee84333
      Andrii Grynenko authored
      Reviewed By: yfeldblum
      
      Differential Revision: D3691541
      
      fbshipit-source-id: 9488c1487ebd0500a23300292215456ca3220f03
      aee84333
    • Andrii Grynenko's avatar
      ReadMostlyMainPtrDeleter · c1dd3ab1
      Andrii Grynenko authored
      Summary:
      Deleter helper object, which may release multiple ReadMostlyMainPtrs at once.
      This allows underlying ref count implementation to perform expensive synchronization operations (e.g. asymmetric heavy barrier only once).
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3691524
      
      fbshipit-source-id: 3ac593b0d813345daba3a81ff4e2eb71b4db292e
      c1dd3ab1
    • Christopher Dykes's avatar
      Wait for all threads to finish executing before leaving in an EventBase test · 10e9cd3e
      Christopher Dykes authored
      Summary:
      If one of the assertions failed, it would result in any threads that are still running accessing already-disposed data, so wait for the threads to exit, even when an exception is thrown.
      
      This also slightly cleans up the handling of threads a bit further down, because the mess it was doing was completely unnecessary.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3692438
      
      fbshipit-source-id: 9082ba248b2cf1062e46c97faf0bc062312ee9ae
      10e9cd3e
    • Christopher Dykes's avatar
      Add a utility for disabling some CRT assertions in debug mode · 52286a8b
      Christopher Dykes authored
      Summary:
      See the comment on `msvcReturnInvalidParams` for more info on the issue.
      This also adds a use of it to TestUtilTest.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3691526
      
      fbshipit-source-id: f0f596e9b4c830e1d31de01926162636757329e8
      52286a8b
    • Christopher Dykes's avatar
      Support read and write from blocking and non-blocking pipes and sockets · 60136d87
      Christopher Dykes authored
      Summary: Also switch `pipe` to return a socket pair instead, as libevent can't poll against a pipe on Windows. And lastly, fix the file descriptor for sockets leaking when close is called.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3691255
      
      fbshipit-source-id: c684242d255ac5158a4c805d432d345dac1b3bd8
      60136d87
  3. 09 Aug, 2016 9 commits
    • Christopher Dykes's avatar
      Allow the time remaining parameter to nanosleep to be null · cc01a8a4
      Christopher Dykes authored
      Summary: Pah! This definitely wasn't causing the lock tests to segfault. Nope, not at all <_>.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3691120
      
      fbshipit-source-id: 6601db637f22c7b0bc326907a2a7976f6df7c159
      cc01a8a4
    • Christopher Dykes's avatar
      Properly support socketpair and reading and writing non-blocking sockets · 4a0d0c51
      Christopher Dykes authored
      Summary:
      Also handle invalid file descriptors correctly and switch away from `WSASendMsg` for the implementation of `sendmsg` due to limitations imposed by WinSock.
      This also fixes up a few places that were explicitly referencing the global version of some socket functions, which was bypassing the socket portability layer.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3692358
      
      fbshipit-source-id: 05f394dcc9bac0591df7581345071ba2501b7695
      4a0d0c51
    • Christopher Dykes's avatar
      Implement setenv correctly and support setting values to empty strings · 5d8ca09a
      Christopher Dykes authored
      Summary: Just calling `SetEnvironmentVariableA` wasn't updating `_environ`, which meant that calls to `getenv` weren't reflecting the changes made via `setenv`. The correct way to implement it is using `_putenv_s`, but there's one problem with that: passing an empty string as the value to `_putenv_s` results in the environment variable being unset. To make it possible to set the environment variable to an empty string, we shall dive head-first into the implementation details of the CRT and emerge victorious by blatently ignoring the documentation of `getenv` and modifying the string it returns to terminate it early.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3691007
      
      fbshipit-source-id: 350c2ec72ec90b9178a9a45b2c2ed2659b788e37
      5d8ca09a
    • Christopher Dykes's avatar
      Look for the PATH environment variable rather than USER · 4d777293
      Christopher Dykes authored
      Summary: Windows is weird and calls it USERNAME instead, so just use one that everything agrees on: PATH
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3691072
      
      fbshipit-source-id: 579c6484736ef47e130049c29bef8b59c66a4482
      4d777293
    • Christopher Dykes's avatar
      Support using fcntl to mark pipes as non-blocking · 703fb237
      Christopher Dykes authored
      Summary: Because the comment was a lie; sockets are blocking by default, not non-blocking.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3691145
      
      fbshipit-source-id: 5d3c62b3573205fe416d77fe4b5b9fbd593ffd93
      703fb237
    • Subodh Iyengar's avatar
      Remove getTFOSucceeded · 70618258
      Subodh Iyengar authored
      Summary:
      The getTFOsucceeded doesn't really work right now
      as intended. Currently we check right after sendmsg,
      however we can only know whether or not the server
      has acked the data after 1-RTT.
      
      This removes getTFOSucceeded. Will fix in another diff.
      
      Differential Revision: D3679235
      
      fbshipit-source-id: 6b2bb01d3743ea7e68ad3cc9a26be6806f17ffbe
      70618258
    • Yedidya Feldblum's avatar
      constexpr_abs · cf93e807
      Yedidya Feldblum authored
      Summary:
      [Folly] `constexpr_abs`.
      
      Is `constexpr`.
      
      Works over integral and floating types.
      
      If given an integral type, the return type is the usigned version of that integral type, thereby avoiding the undefined behavior of `std::abs(std::numeric_limits<int>::min())`.
      
      Reviewed By: simpkins
      
      Differential Revision: D3654072
      
      fbshipit-source-id: 24fefc0c3b055f78ba3e07472c38fb9c550e0f31
      cf93e807
    • Aaryaman Sagar's avatar
      Removing noexcept specifications in constructors for Synchronized that call... · b26ef8ac
      Aaryaman Sagar authored
      Removing noexcept specifications in constructors for Synchronized that call contextualLock() and contextualRLock()
      
      Summary:
      Most mutex lock() functions do not have a noexcept guarantee, saying that the
      constructor for Synchronized based on whether the underlying constructor for
      the type stored is not enough.  Although this will *rarely* cause bugs, it
      probably can be fixed
      
      Reviewed By: simpkins
      
      Differential Revision: D3682974
      
      fbshipit-source-id: ec0bb701d0af41ffc79128fe8db7935a5f19bc70
      b26ef8ac
    • Subodh Iyengar's avatar
      Add MSG_NOSIGNAL to AsyncSSLSocket · 6b554b1e
      Subodh Iyengar authored
      Summary:
      We are definitely not prepared to handle
      SIGPIPEs, so add MSG_NOSIGNAL to sendmsg
      of AsyncSSLSocket.
      
      This is a problem which exists in openssl
      as well which calls send with flags = 0.
      We recently made a change to move the send
      into our control, so we can now supply the
      flag
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3686679
      
      fbshipit-source-id: ff8fe662e62923c25876bdfd516352639505dca6
      6b554b1e
  4. 08 Aug, 2016 3 commits
    • Adam Simpkins's avatar
      Update documentation for Synchronized · 218a45f5
      Adam Simpkins authored
      Summary:
      Update the documentation for Synchronized to remove references to the
      various SYNCHRONIZED macros, and document the lock() and withLock() APIs
      instead.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3618871
      
      fbshipit-source-id: e970f9a23e45831d62232eea2ba133a55e5a4a49
      218a45f5
    • Michael Lee's avatar
      Include sys/syscall.h to SysMembarrier · b8c8b1f8
      Michael Lee authored
      Summary: It calls syscall, but does not include the header.
      
      Reviewed By: knekritz
      
      Differential Revision: D3586636
      
      fbshipit-source-id: 2ef05fc4b88f236a84b79afa708e98fe21529685
      b8c8b1f8
    • Maged Michael's avatar
      Methodology for using DeterministicSchedule support for auxiliary data and global invariants · 58ecc903
      Maged Michael authored
      Summary:
      Depends on D3648195
      
      This test example is intended to demonstrate the methodology for using DeterministicSchedule support for auxiliary data and global invariants.
      The main goal is fine-grained invariant checking, ideally after every shared update.
      The secondary goals are:
      - Minimize intrusion in the original code. In this proposed methodology, it is adding a friend.
      - Minimize duplication of original tested code. Unfortunately, depending on the original code, it seems that significant duplication may be unavoidable if we don't want to change the original code.
      
      This diff is primarily about the methodology for testing already developed code. I plan to apply what we agree on through this diff to the dynamic MPMCQueue code (D3462592).
      
      A future goal to keep in mind is creating a methodology for developing new code with hooks for DSched aux. data and invariant checking integrated in it, in order to minimize or eliminate duplication of tested code.  In past projects, I used non-standard source code (basically algorithm code) that is automatically translatable through scripts and macros to input to a DSched-like tool as well as to compilable code. The main challenge for such a methodology is to allow the original source code to be standard readable C++ code.
      
      Reviewed By: djwatson
      
      Differential Revision: D3675447
      
      fbshipit-source-id: aae2c9f0550af88dc3a5dcbe53318a75a86b6e2b
      58ecc903
  5. 05 Aug, 2016 7 commits
    • Christopher Dykes's avatar
      Use the socket portability layer when needed. · c98aec4e
      Christopher Dykes authored
      Summary: This switches the places in Folly that need to explicitly reference the socket portability implementation to do exactly that.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3299984
      
      fbshipit-source-id: 57cd8ebe66c9055aba66581a8c0fcf6c125d96f9
      c98aec4e
    • Dave Watson's avatar
      Upper level tests · f3d9a55f
      Dave Watson authored
      Summary:
      Add some tests to test the other levels of the timer wheel.
      
      They are slow by necessity.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3637132
      
      fbshipit-source-id: badf8d37d726dbeb5a8220d50c60b8efdaee7989
      f3d9a55f
    • Dave Watson's avatar
      remove catchupEveryN · 07b497a7
      Dave Watson authored
      Summary: This feature doesn't make sense when wheeltimer doesn't constantly tick - we always have to get the current clock time.  On the plus side, we'll only be grabbing the clock on timer schedule or timeout, never for individual ticks.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3637088
      
      fbshipit-source-id: ed8fe52419259332a14b6dc1d357979dcf258a20
      07b497a7
    • Christopher Dykes's avatar
      Handle creating the default crypto context if it doesn't already exist · cf7b6169
      Christopher Dykes authored
      Summary: It's perfectly possible that the default crypto context simply hasn't been created yet, so try to create it if the initial acquisition fails.
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3673138
      
      fbshipit-source-id: 122955df04055ff4f99513b182375d4388dd0305
      cf7b6169
    • Christopher Dykes's avatar
      Don't throw in the Watchdog destructor · 86b83461
      Christopher Dykes authored
      Summary: As-of C++11, destructors are assumed to be `noexcept` unless explicitly marked otherwise. `Watchdog`'s destructor throws, so switch it to a `LOG(FATAL)` instead.
      
      Reviewed By: andriigrynenko
      
      Differential Revision: D3672621
      
      fbshipit-source-id: 5224ecf85d101462e02e12da257e033bab4db1a1
      86b83461
    • Aaryaman Sagar's avatar
      Adding policies for all the upgrade and downgrade mutex transitions that are... · 7dd2b580
      Aaryaman Sagar authored
      Adding policies for all the upgrade and downgrade mutex transitions that are going to be supported by Synchronized
      
      Summary:
      This diff contains the lock policies that will be used by Synchronized to
      implement mutex transitions by means of heterogenous RAII mutex wrappers
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3665020
      
      fbshipit-source-id: a5509dfd58a1dd6cd60a7d3afe929d0da860926d
      7dd2b580
    • Aaryaman Sagar's avatar
      Adding support for upgradable mutexes to LockTraits · 84f24236
      Aaryaman Sagar authored
      Summary:
      This diff adds support for upgradable mutexes to the LockTraits
      abstraction used by folly::Synchronized
      
      Reviewed By: yfeldblum
      
      Differential Revision: D3645453
      
      fbshipit-source-id: 30f16eb3fbebc687a4136256f1103962c0e4c465
      84f24236
  6. 04 Aug, 2016 5 commits