Summary: Sometimes variable shadowing is fine, sometimes it can cause subtle mistakes.

Reviewed By: yfeldblum

Differential Revision: D3140450

fb-gh-sync-id: acf7195ad65614d4f012bef8ce7dccb3a3038456
fbshipit-source-id: acf7195ad65614d4f012bef8ce7dccb3a3038456

Summary:We are jumping through some ugly hoops in the implementation of
fibers to allow for non-copyable functors/lambdas. We can get rid of
those by using folly::Function instead of std::function to store
functions in folly::fibers::Fiber.

This involves one observable interface change: the virtual function
folly::fibers::InlineFunctionRunner::run must take a
folly::Function<void()> argument instead of a std::function<void()>.

Reviewed By: andriigrynenko

Differential Revision: D3102711

fb-gh-sync-id: 56705b972dc24cc0da551109ed44732b97cb6e13
fbshipit-source-id: 56705b972dc24cc0da551109ed44732b97cb6e13

Summary: For exceptions aggregation conveniece

Reviewed By: philippv

Differential Revision: D3128132

fb-gh-sync-id: 48c72df364228c1d2c8f29161c2b85c131b4fea8
fbshipit-source-id: 48c72df364228c1d2c8f29161c2b85c131b4fea8

Summary:This is an ugly but occassionally useful hack to accomodate cases
where we want to propagate folly::Function but run into the brick wall that is
std::function

@override-unit-failures

Reviewed By: spacedentist

Differential Revision: D3118432

fb-gh-sync-id: 80ed56494dfcf60e0f266013d880107acabc7dcf
fbshipit-source-id: 80ed56494dfcf60e0f266013d880107acabc7dcf

Summary: Allow ASan to detect access to invalidated strings even when they are small (in-situ). See D3114022 for details.

Reviewed By: luciang

Differential Revision: D3118161

fb-gh-sync-id: 3b83e3309d510ee9721dd505d176b09bdb7fd42d
fbshipit-source-id: 3b83e3309d510ee9721dd505d176b09bdb7fd42d

Summary:Now that folly::Future uses folly::Function, we can use non-copyable
callbacks. That allows us to get rid of folly::MoveWrapper in the
implementaion.

This diff also enforces perfect forwarding in the implementation of
folly::Future, thereby reducing the number of times that a callable
that is passed to Future::then et al. gets passed by value.

Before folly::Function, Future::then(callback) has invoked the move
constructor of the callback type 5 times for small callback objects
(fitting into the in-place storage inside folly::detail::Core) and
6 times for large callback objects. This has been reduced to 5 times
in all cases with the switch to UniqueFunction. This diff reduces it
to 2 times.

Reviewed By: yfeldblum

Differential Revision: D2976647

fb-gh-sync-id: 9da470d7e9130bd7ad8af762fd238ef9a3ac5892
fbshipit-source-id: 9da470d7e9130bd7ad8af762fd238ef9a3ac5892

Summary:By using folly::Function instead of std::function to internally store
functions in FunctionScheduler, this diff allows users to pass
non-copyable lambdas to FunctionScheduler::addFunction.

All exisiting unit tests still pass. Also, passing std::function to
addFunction still works, as the std::function will be implicitly
converted (i.e. wrapped) in a folly::Function. However, this implies
a performance penalty.

Reviewed By: fugalh

Differential Revision: D3092944

fb-gh-sync-id: 1e8081e70d4717025f2eadbb6b6da173460d4373
fbshipit-source-id: 1e8081e70d4717025f2eadbb6b6da173460d4373

Summary:Instead of using preprocessor to disable SSO, use a default argument. Also
reimplement `insertImplDiscr` to mirror `insertImpl`.

Reviewed By: philippv

Differential Revision: D3130901

fb-gh-sync-id: a5b0ba97b3c7d91323be01ab617ca9b09dbb0fd2
fbshipit-source-id: a5b0ba97b3c7d91323be01ab617ca9b09dbb0fd2

Summary: The only thing that matters in it is dirname, and Windows doesn't have it.

Reviewed By: yfeldblum

Differential Revision: D2977655

fb-gh-sync-id: eb5703485089f5c66882fbc9cc142686214c1148
fbshipit-source-id: eb5703485089f5c66882fbc9cc142686214c1148

Summary: Windows doesn't have it, so stub it out for now, because not getting output to the system log is a minor inconvience, not a breaking issue.

Reviewed By: yfeldblum

Differential Revision: D2977537

fb-gh-sync-id: b05d9d3c722ad4f3953a979f91024602d17d3ed0
fbshipit-source-id: b05d9d3c722ad4f3953a979f91024602d17d3ed0

Summary: MSVC doesn't like this and complains about ambigious overloads.

Reviewed By: ericniebler

Differential Revision: D3107438

fb-gh-sync-id: 597ba85fdb7640b0a0fda352bf53ce31cf0e38b7
fbshipit-source-id: 597ba85fdb7640b0a0fda352bf53ce31cf0e38b7

Summary:The swap function belongs in the same namespace as Function: folly.
Also, to avoid amibiguities with a generic swap function in
<algorithm>, we need two variants: one for identical types of
folly::Function, and one for folly::Functions with different
configurations.

Reviewed By: ericniebler

Differential Revision: D3106429

fb-gh-sync-id: 11b04e9bc709d52016ac94c078278410f5ee43c6
fbshipit-source-id: 11b04e9bc709d52016ac94c078278410f5ee43c6

Summary:Update setFromPath() to accept a StringPiece rather than just std::string
or a plain const char*.

Also fix two other minor issues:
- Leave the old address untouched on failure.  Previously it could leave the
  SocketAddress in a partially updated state.
- Don't assume the input is nul terminated.  Previously the input code read
  one past the specified input length, and copied this into the address,
  assuming it was a nul terminator.  The new code explicitly writes a 0 byte.

Reviewed By: yfeldblum

Differential Revision: D3119882

fb-gh-sync-id: 3e2258f42034b4f470ade0a23ea085e132a3dd0f
fbshipit-source-id: 3e2258f42034b4f470ade0a23ea085e132a3dd0f

Summary: Because we don't have these builtins under MSVC, and not having to deal with the differences in API in the places that use these builtins is good.

Reviewed By: yfeldblum

Differential Revision: D2984842

fb-gh-sync-id: 34db7455debf81e4abffe57c154eb731ae097ff6
fbshipit-source-id: 34db7455debf81e4abffe57c154eb731ae097ff6

Summary: We don't have dirent.h on Windows, but we can emulate its behavior.

Reviewed By: yfeldblum

Differential Revision: D2978570

fb-gh-sync-id: af5ade0ea64ba22900440250e7125aa039a77f62
fbshipit-source-id: af5ade0ea64ba22900440250e7125aa039a77f62

Summary: The primary issue is that the pthread implementation we use for Windows defines `pthread_t` as a struct (yes, it is allowed to do this), which breaks a lot of things.

Reviewed By: yfeldblum

Differential Revision: D2862671

fb-gh-sync-id: 551569b6a9e2e374cf77e186e148b6b397f8f25f
fbshipit-source-id: 551569b6a9e2e374cf77e186e148b6b397f8f25f

Summary: `strndup` is defined on modern versions of OSX

Reviewed By: Orvid, grp

Differential Revision: D3122635

fb-gh-sync-id: 678792765addf2fb226e835d3b7a67e155ed6dc5
fbshipit-source-id: 678792765addf2fb226e835d3b7a67e155ed6dc5

Summary:Most header files in folly already used "#pragma once" to
protect against multiple inclusion. This diff removes old-style
ifndef/define/endif header guards and replaces them with
pragma once.

In some cases the defined symbol is tested in other header
files. In those cases the "#define" is kept.

Reviewed By: igorsugak

Differential Revision: D3054492

fb-gh-sync-id: 20aa0b9b926a30dd021e4b8f5440e8888874681c
fbshipit-source-id: 20aa0b9b926a30dd021e4b8f5440e8888874681c

Summary: This diff adds folly/docs/Function.md

Reviewed By: yfeldblum

Differential Revision: D3120617

fb-gh-sync-id: fecc0e507e05016aaac43ba981eab49431229bd7
fbshipit-source-id: fecc0e507e05016aaac43ba981eab49431229bd7

Summary:This adds basic print functions for FiberManager, Fiber and FiberManager map.
It also adds a global list of fibers to FiberManager. Fibers are only removed from that list on Fiber object destruction, so it shouldn't have any perf impact.

Inspired by tao/server/scripts/fiber_bt.gdb

FiberManager map example:
  (gdb) print_folly_fiber_manager_map
    Global FiberManager map has 2 entries.
      (folly::EventBase*)0x7fffffffdb60 -> (folly::fibers::FiberManager*)0x7ffff5b58480
      (folly::EventBase*)0x7fffffffd930 -> (folly::fibers::FiberManager*)0x7ffff5b58300

FiberManager example:
  (gdb) print_folly_fiber_manager &manager
    (folly::fibers::FiberManager*)0x7fffffffdbe0

    Fibers active: 3
    Fibers allocated: 3
    Fibers pool size: 0
    Active fiber: (folly::fibers::Fiber*)(nil)
    Current fiber: (folly::fibers::Fiber*)(nil)

    Active fibers:
      (folly::fibers::Fiber*)0x7ffff5b5b000   State: Awaiting
      (folly::fibers::Fiber*)0x7ffff5b5b300   State: Awaiting
      (folly::fibers::Fiber*)0x7ffff5b5b600   State: Awaiting

Fiber example: P56244621

Reviewed By: yfeldblum

Differential Revision: D3119616

fb-gh-sync-id: defa27b84aafbd4333b2ca301f07c226f0386f44
fbshipit-source-id: defa27b84aafbd4333b2ca301f07c226f0386f44

Summary:gcc-5.0 introduced non-relocatable strings in libgcc. Only treat gcc < 5 strings as relocatable. Enable relocation for fbstrings via typedef.

Re https://github.com/facebook/folly/issues/316, thanks tomhughes for reporting this.

Reviewed By: snarkmaster, ot

Differential Revision: D3115580

fb-gh-sync-id: d8aff947d55a0a0c57f6b997f651a190e05f2779
fbshipit-source-id: d8aff947d55a0a0c57f6b997f651a190e05f2779

Summary:The current implementation of `insertImpl` assumes that
`expand_noinit` does not reallocate if the `size() + delta <=
capacity()`, but D3114022 makes this assumption invalid when compiling
with ASan. It also doesn't guarantee exponential growth, so repeated
inserting at the end could trigger quadratic behavior.

The new implementation fixes the problems above, and it's much
simpler.

Reviewed By: yfeldblum, Orvid

Differential Revision: D3119813

fb-gh-sync-id: 946ebeeaf924a531f7f03fb7e79c75e352a50c58
fbshipit-source-id: 946ebeeaf924a531f7f03fb7e79c75e352a50c58

Summary: Alerts may be sent by clients, potentially letting us know why connections fail.

Reviewed By: siyengar

Differential Revision: D3117395

fb-gh-sync-id: bddf51f2399eb9e7e397981d5440adb3e815d6d2
fbshipit-source-id: bddf51f2399eb9e7e397981d5440adb3e815d6d2

Summary: Windows doesn't have it, so be nice and at least stub it out.

Reviewed By: yfeldblum

Differential Revision: D2984232

fb-gh-sync-id: 3e8871ab78c5d7c8785a52af24548245f842f19b
fbshipit-source-id: 3e8871ab78c5d7c8785a52af24548245f842f19b

Summary:Let's break OSX!

Alright, maybe not. Neither OSX nor Windows define malloc_usable_size, so we implement them based on what is available on the respective platforms.
This moves the implementation for OSX out of Portability.h and into the new header, so it likely breaks something on OSX, although I'm not sure what.

Reviewed By: yfeldblum

Differential Revision: D3019938

fb-gh-sync-id: df95faef09535098fb73b7b3479d7a73f6b49712
fbshipit-source-id: df95faef09535098fb73b7b3479d7a73f6b49712

Summary: Adding a benchmark for one the most common scenarios (used in Futures, EventBase, fibers etc).

Reviewed By: yfeldblum

Differential Revision: D3106365

fb-gh-sync-id: 8cb55959b3803e8836ab5a5bdf43fdfc1db02d4c
fbshipit-source-id: 8cb55959b3803e8836ab5a5bdf43fdfc1db02d4c

Summary:It was mistakenly assuming the length passed in included the null terminator.
This also makes the portability implementation of `strndup` available to OSX and FreeBSD, where they weren't present, and where HHVM had a wrapper for them.
This also removes the extra pair of conditions around `memrchr`, as the main define should always be getting set.

Reviewed By: yfeldblum

Differential Revision: D3116467

fb-gh-sync-id: 243dd4dace219efab2c2bf2f383202e70fbec4de
fbshipit-source-id: 243dd4dace219efab2c2bf2f383202e70fbec4de

Summary:MSVC has support in the compiler for the intrinsics required, but both refuses to tell us that, and also gives them proper names.
The code already checks for runtime support, this just enables compiling the SSE 4.2 version in the first place.

Reviewed By: yfeldblum

Differential Revision: D3104296

fb-gh-sync-id: 9143240bede9b756817691fdd86818001267dac1
fbshipit-source-id: 9143240bede9b756817691fdd86818001267dac1

Summary: OpenSSLPtrTypes.h uses CHECK_EQ which is included in glog/logging.

Reviewed By: knekritz

Differential Revision: D3118577

fb-gh-sync-id: f8a00aa5a523bf88a3f783433c6699d555799225
fbshipit-source-id: f8a00aa5a523bf88a3f783433c6699d555799225

Summary: This adds Windows support to the portability header for sys/syscall.h, which was previously named portability/Syscall.h, which is inconsistent with the naming of the other headers, so it's now also been renamed to SysSyscall.h, and the one place that used it updated to reflect the new name.

Reviewed By: yfeldblum

Differential Revision: D2984383

fb-gh-sync-id: c0e4ce55c7bc2aa6db9b084e09d9762ba587002d
fbshipit-source-id: c0e4ce55c7bc2aa6db9b084e09d9762ba587002d

Summary:`fbstring`'s small string optimization prevents ASan to catch invalid
accesses to the data of a destroyed string, for example if a
`StringPiece` is initialized from a temporary string.

This diff disables building a string with the small category when
compiled with ASan: small strings will be constructed as
`Medium`-category strings and heap-allocated. This is done by only
changing the behavior of construction and resizing, so that the ABI is
preserved and it is still possible to link an ASan-enabled object file
with a library that wasn't compiled with ASan.

The diff also fixes a blind spot in `fbstring_core`'s constructor,
which disabled ASan altogether in order to allow fast word-aligned
copy of small strings. Since small string construction is now disabled
under ASan, we don't need to disable it anymore.

Lastly, it always clears moved-from strings, even when they are small.
This improves the performance of the move constructor (no more conditional
needed) and it uncovers another class of potential bugs.

Reviewed By: luciang

Differential Revision: D3114022

fb-gh-sync-id: 4e180fbf2b8aced3b977afc985d26fdf244d9598
fbshipit-source-id: 4e180fbf2b8aced3b977afc985d26fdf244d9598

Summary: Some compiler will complain about this when calling std::min with a size_t and a uint64_t. So use unit64_t for numElements in HHWheelTimer.

Reviewed By: mzlee

Differential Revision: D3116346

fb-gh-sync-id: 67a9eebf4f9e8fe0e732a7292af55122be04163b
fbshipit-source-id: 67a9eebf4f9e8fe0e732a7292af55122be04163b

Summary:We should be able to use HHWheelTimer in fibers. So it should use way less stack.

Alternative solution to D3112305

Reviewed By: haijunz

Differential Revision: D3112558

fb-gh-sync-id: 3a52a37d9f9347639005fdf84524f7f8c3041918
fbshipit-source-id: 3a52a37d9f9347639005fdf84524f7f8c3041918

Summary:[Folly] `IPAddressV[46]::validate(StringPiece)`.

Just perform the check without allocations or throws.

Reviewed By: meyering

Differential Revision: D3103545

fb-gh-sync-id: 2918c1398935738f597b9045cf7dadbe42da2527
fbshipit-source-id: 2918c1398935738f597b9045cf7dadbe42da2527

Summary:Use a ThreadLocalPRNG insteaad of a per context RNG.  This avoids
calls to Random::seed on context creation which can get expensive
when many are created in an application.

Reviewed By: siyengar

Differential Revision: D3105501

fb-gh-sync-id: 92d987c27a1f190a98035ca25c23b994ca915007
fbshipit-source-id: 92d987c27a1f190a98035ca25c23b994ca915007

Summary:Per https://github.com/facebook/folly/issues/379, github user pmalek determined that fbvector's insert function was a lot slower than std::vector's. While insert is often imagined to be a second-class vector function (if you're inserting in the middle, don't use a vector!), it is the correct function to use when inserting multiple elements at the back of an array (since the standard lacks an ##append(multiple-elements)## function). The code, therefore, required optimization.

There are three things that were slowing down fbvector:
- Excessive asserts. The fbvector code contains internal asserts that ensure proper calling of private methods. These are mostly unnecessary, given the extensive test suite that batters fbvector, in addition to fbvector's battle tested history. I have removed these.
- Internal data checks. When inserting into a vector, the elements being inserted may not reference the contents of the vector in question. If the client supplies internal references, then the standard allows for undefined behavior. An old design decision for fbvector was to be forgiving of this error; we check for internal data before performing internal data moves. This is expensive. Further, it is unnecessary when the insertion point is at the end of the vector (which is the valid case we are optimizing) or if the vector is being reallocated. I've rearchitected the insert macros to only perform the internal-data-check when it is absolutely necessary. While rearchitecting the checks, I also reorganized the n==0 base case checking.
- The 'window' function is pretty expensive when called with the end() iterator. It is effectively a no-op, but the function (and some of its called functions) are too big to inline, so the call overhead is non-trivial. I've put window calls behind a conditional to save this overhead.

I added a benchmark test to FBVectorTestBenchmark.cpp.h, but the particular pattern in that file caused the results to be completely optimized away, voiding the test. Oh well.

Running the github benchmarks show a 4x speedup for inserting a single element on the back, bringing it quite close to that of std::vector. The multi-insert function got a bit faster, though the exact number of iterations seems to be having an effect on the numbers (running the tests 10,000 times, intead of only 1,000 times, yields results that are 90% as fast as std::vector, instead of 75%). Both of these cases are now looking quite a bit better.

I resurrected the old StlVectorTest suite, since I was mucking with the complicated insert code. The suite had one latent compilation error (caught with newer compiler warnings - fortunately this particular case was benign). The test suite caught an error in shrink_to_fit, where a clean-slate optimization for empty vectors (see D2696314) that trampled the vector's allocator. I have fixed that small bug in this diff, too.

Reviewed By: ot

Differential Revision: D3105962

fb-gh-sync-id: ac9fa9d7ca79335cf0ff6bb9010af1ed8bd7916b
fbshipit-source-id: ac9fa9d7ca79335cf0ff6bb9010af1ed8bd7916b

Summary:This diff makes it possible to pass callables (such as lambdas) to
folly::Future::then that are not copy-constructible. As a consequence, move-only
types (such as folly::Promise or std::unique_ptr) can now be captured in lambdas
passed to folly::Future::then.

Using C++14 notation, the following is now possible:
  Future<Unit>().then([promise = std::move(promise)]() mutable {
    promise.setValue(123);
  });
See folly/futures/test/NonCopyableLambdaTest.cpp for more examples.

folly::Future uses std::function to store callback functions. (More precisely,
the callback function is stored in a std::function in folly::detail::Core.)
std::function is a copy-constructible type and it requires that the callable
that it stores is copy constructible as well.

This diff changes the implementation of folly::detail::Core to use
folly::Function instead of std::function. It also simplifies the code in
folly::detail::Core a little bit: Core had a reserved space of size
8*sizeof(void*) to store callbacks in-place. Only larger callbacks (capturing
more data) would be stored in the std::function, which puts those on the heap.
folly::Function has a template parameter to set the size of the in-place
callable storage. In Core, it is set to 8*sizeof(void*), so all callbacks that
used to be stored in-place inside Core, are now stored in-place inside
folly::Function. This even reduces the size of a Core object: the
folly::Function object occupies 80 bytes, which is 16 bytes less than what was
used before for a std::function (32 bytes) and the callback storage (64
bytes). The resulting size of a Core<Unit> goes down from 192 to 176 bytes (on
x86_64).

Reviewed By: fugalh

Differential Revision: D2884868

fb-gh-sync-id: 35548890c392f80e6c680676b0f98e711bc03ca3
fbshipit-source-id: 35548890c392f80e6c680676b0f98e711bc03ca3

Summary:AsyncSocket has a few cases where
we might overwrite errno, with another function
call that is done during use.

For example withAddr could cause an EBADF if the fd
was bad.

This fixes these cases by copying errno, before doing
anything else

Reviewed By: afrind

Differential Revision: D3101932

fb-gh-sync-id: 73d4810314c90cb987936a0a3a9dc977af07243a
fbshipit-source-id: 73d4810314c90cb987936a0a3a9dc977af07243a

Summary:[Folly] `s/MAX_STATIC_CONSTRUCTOR_PRIORITY/FOLLY_STATIC_CTOR_PRIORITY_MAX/g`.

Namespace it with a `FOLLY_` prefix - when Folly introduces its own symbols, it is best when they are namespaced.

And stick `_MAX` at the end. Reads a bit more hierarchically to me that way.

Reviewed By: andriigrynenko

Differential Revision: D3095964

fb-gh-sync-id: 4ac8c00c03c8bd146fad9be96ce3849830504d96
fbshipit-source-id: 4ac8c00c03c8bd146fad9be96ce3849830504d96

Summary: This was on for mcrouter for very long time. We should make it default for everyone.

Reviewed By: yfeldblum

Differential Revision: D3023133

fb-gh-sync-id: 6401f86df754e9490d46beb5fd9d0cf4b3675208
fbshipit-source-id: 6401f86df754e9490d46beb5fd9d0cf4b3675208