Summary:
Provide translations for gcc packed

MSVC has a very very ugly syntax for packing using pragmas which is very different from the gcc attribute
this makes it very hard to macro around, but lots of ifdefs are also painful - cleanest of the hackery is to simply have push, pop, and attribute macros for packed that define properly depending on the compiler

Test Plan: fbmake runtests

Reviewed By: njormrod@fb.com

FB internal diff: D1279966

Summary:
Allow mutable functors in Promise::fulfil.

Facebook:
Works both with EventBase and without it (thanks to wangle::Future thread-safety). CacheClientImpl lifetime is guaranteed using shared_ptr.

Test Plan: unit test

Reviewed By: stepan@fb.com

FB internal diff: D1291024

Summary:
Fix an off-by-one error causing an ASAN abort.  When calling
longestCommonPrefix() with a full mask, it would compare one byte past
the end of the address data, and would end up writing past the end of
the ba array on the stack.

Test Plan:
Built with ASAN, ran the unit tests, and verified the ASAN failure was
gone.

Reviewed By: jasmeetbagga@fb.com

FB internal diff: D1284750

Summary:
Moving our internal IP/Mac address libraries to folly/

Facebook:
We want to get rid of common/ dependencies in Mcrouter since we're going to open source it. Also looking at the original commit, seems like it's been the intention all along, so I just did it.

I tried to keep dependencies intact as much as possible. Changing projects to use this directly should be in separate diffs.

Test Plan:
Run folly/network and common/network unit tests.

Generate the list of targets with:

```
fbgs /$FILE.h | cut -f1 -d: | xargs -L1 dirname | cut -f2- -d/ | sort | uniq
```

Then fbconfig + fbmake. Will fix contbuild failures.

Revert Plan:

Reviewed By: simpkins@fb.com

FB internal diff: D1261089

Summary: - On iOS, iovec::iov_len is a size_t, which is a uint32_t; convert

Test Plan:
- fbconfig -r folly && fbmake runtests
- Builds in fbobjc

Reviewed By: simpkins@fb.com

FB internal diff: D1284931

Summary:
Headers whose includes are not sufficient cause problems when included
from other files. An easy way to prevent this is to include the header
first in its associated cpp file. In fact, we have a lint rule for this.
However, there are places where this rule is ignored. This diff fixes
many of them.

Note that a few top-level directories were excluded.

Reviewed By: robbert@fb.com

FB internal diff: D1281032

Summary: rwspinlock.h was missing the portability.h header which made the x64 detection go wonky, but only shows up when used with isolated rwspinlock use

Test Plan: fbmake runtests

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1284387

Blame Revision: D1282140

Summary: Another cross platform fun macro - this one detects x64 status and requires some portability.h includes (which should only define portability stuff and not actually include anything)

Test Plan: fbmake runtests

Reviewed By: delong.j@fb.com

FB internal diff: D1282140

Summary: doh
@override-unit-failures

Test Plan: compiled under android

Reviewed By: subodh@fb.com

FB internal diff: D1283177

Summary: the fbstring specific unlikely/likely defs were missing the ifdef blocks to avoid breaking things without builtin_expect

Test Plan: fbmake runtests

Reviewed By: njormrod@fb.com

FB internal diff: D1282166

Summary:
The functor allows to create ranges and sequences that advance the
iterator by more than 1 per iteration.

Test Plan: Unit test.

Reviewed By: tjackson@fb.com

FB internal diff: D1228065

Summary:
Clang chokes because it tries to instantiate both versions of `from<const int*>`, one of which calls `std::begin(const int*)`, which doesn't work. By casting the intializer list to the right type, it can pick the overload.

Clang, because it makes debugging these templates so much better.

Test Plan: `fbconfig --clang folly/gen/test && fbmake runtests_dbg`

Reviewed By: tjackson@fb.com

FB internal diff: D1280888

Summary:
change from using __thread to using FOLLY_THREAD_LOCAL macro, this will allow abstraction over gcc and msvc implementations of thread local (__thread and __declspec(thread)) which have the same semantices and will also allow drop in replacement of thread_local when compiler support for the feature is complete  This doesn't do anything about apple, however, which still has broken __thread support

This doesn't actually change any implementation for now, simply allows for correct compilation

Test Plan: fbmake runtests

Reviewed By: delong.j@fb.com

FB internal diff: D1278726

Summary: MSVC is a stricter compiler and needs additional includes explicitly declared

Test Plan: fbmake runtests

Reviewed By: njormrod@fb.com

FB internal diff: D1277004

Summary:
Adding `... | parallel(my | pipe | line) | ...` for parallelizing a portion of a generator pipeline.

```lang=cpp
auto factored = from(values)
| parallel(filter(isEven) | map(square) | sub(count))
| sum;
```

Work is divided evenly among a fixed number of threads using a `MPMCQueue`.

Test Plan: Unit tests and benchmarks testing for a variety of workloads and performance characteristics, including sub-linear (blocking) workloads, linear (mostly math) workloads, and superlinear (sleeping) workloads to simulate real-world use.

Reviewed By: lucian@fb.com

FB internal diff: D638551

Summary: A LIFO semaphore is useful for building a thread pool because we can keep cpu caches warmer by continually reusing the last used thread. This diff introduces a LIFO semaphore to folly.

Test Plan: unit tests

Reviewed By: davejwatson@fb.com

FB internal diff: D1280405

Summary: folly Uri::host() returns leading and trailing brackets for IPv6 host, for example, for URI "http:://[::1]:8080/index.html", host() method returns "[::1]". But square brackets are not part of host in fact, this diff is going to remove brackets for IPv6 host.

Test Plan: fbconfig -r folly/test && fbmake runtests_dbg

Reviewed By: chip@fb.com

FB internal diff: D1276513

Summary:
Provide translations for gcc noreturn attribute

__attribute__((noreturn)) is gcc specific, clang understands it on nix systems, but for msvc __declspec(noreturn) is the compiler specific version, which clang will imitate/use on windows.  There was already a FOLLY_NORETURN in portability.h, however because of __declspec limitations it must prefix the declaration, not postfix.  The gcc noreturn attribute does not have this limitation and will work prefixed OR postfixed, so to keep from turning code into spaghetti nonsense, the macro was moved to the front of declations where it is currently used.  This will allow it to work with the proper definitions on clang, clang on windows, gcc, and msvc

Test Plan: fbmake runtests

Reviewed By: delong.j@fb.com

FB internal diff: D1279466

Summary:
Provide translations for gcc always_inline and noinline attribute

Change to using a macro from portability.h for attributes for using always_inline and noinline for cross platform support

Test Plan: fbmake runtests

Reviewed By: delong.j@fb.com

FB internal diff: D1279652

Summary:
Using StringPiece makes it easier than std::string

Facebook: Motivation for this diff is found in D1253595

Test Plan:
Added new unittest

fbconfig folly/io/test && fbmake runtests_opt

Reviewed By: simpkins@fb.com

FB internal diff: D1276185

Summary: As part of the windows port, uint is not defined as a type and is not standard, it makes msvc choke, a simple change to unsigned int fixes the issue

Test Plan: fbmake runtests

Reviewed By: delong.j@fb.com

FB internal diff: D1278487

Test Plan: used it

Reviewed By: lucian@fb.com

FB internal diff: D1277063

@override-unit-failures

Summary:
as per discussion from https://phabricator.fb.com/D1274352

After changing the queue length to be a small value (100) I didn't see
improvement from cache alignment of atomics (too much noise). Dropping
that and just fixing the benchmark.

Test Plan: n/a

Reviewed By: ngbronson@fb.com

FB internal diff: D1276591

Summary:
The code originally contained the line

EXPECT_EQ((rcount == lineCount), r); // EOF iff at lineCount

Which asserted that, on the last iteration, EOF was being read (and thus
r was true).

This line has been causing errors in contbuild. There seem to be two
issues at play.

First, the left hand side of the EXPECT_EQ is always false. rcount is
incremented three lines down, and the (implicit) loop ends when rcount
equals lineCount. This means that, on the last iteration, rcount is
actually one less than lineCount. Hence the check is effectively
asserting that r is false.

The second issue is that r is rarely true. Empirically, a necessary but
not sufficient condition is that there be lots of other processes
running at the same time (such as when running fbconfig -r folly &&
fbmake runtests_opt). This would explain why the test passes during
development but fails in contbuild.

I am guessing that, when there are few processes running, Subprocess is
predictably ceasing to read before whichever other process yields EOF,
and that this does not always happen when there is resource contention.

As an immediate fix, I am asserting that r may only be true on the last
iteration. Could someone (cc @tudorb) who is more familiar with the
intricacies of Subprocess please check to see if this is an idiosyncrasy
of the test, or if there is actually an underlying bug.

Test Plan:
Before the change, run fbconfig -r folly && fbmake runtests_opt
repeatedly. Notice that most of the iterations yield an error from line
376 (which is consistent with contbuild error logs).

After the change, notice that the error disappears.

Reviewed By: andrewjcg@fb.com

FB internal diff: D1269124

Summary:
same as map, but runs it's argument in parallel over a number of threads.

@override-unit-failures

Test Plan: added test

Reviewed By: tjackson@fb.com

FB internal diff: D1258364

Summary: Changing the parent's WD is prone to race conditions of various sorts, and needlessly alters parent state. Python's subprocess also has this feature.

Test Plan: fbmake dbg _bin/folly/test/subprocess_test ; ../_bin/folly/test/subprocess_test

Reviewed By: agoder@fb.com

FB internal diff: D1269526

Summary:
* folly/test/CacheLocalityTest.cpp (contentionAtWidth): Work around
clang's prohibition against using flexible arrays in a lambda.

Test Plan:
fbconfig --clang --with-project-version clang:dev folly/test \
&& fbmake runtests

Reviewed By: ngbronson@fb.com

FB internal diff: D1263620

Summary: Allow specifying custom alignment.

Test Plan: fbconfig -r folly/test && fbmake opt -j32

@override-unit-failures

Reviewed By: lucian@fb.com

FB internal diff: D1264851

Summary:
D1261546 introduced regression in `sizeof(ConcurrentSkipList)`.
In case of `NodeAlloc = folly::SysAlloc`, size of an empty `NodeAlloc` struct
is 1 byte, due to the alignment requirements of the next field, it got
translated into wasting 8 bytes.

Test Plan: fbconfig -r folly/test && fbmake opt -j32

@override-unit-failures

Reviewed By: lucian@fb.com

FB internal diff: D1264730

Summary:
* folly/folly-config.h (FOLLY_HAVE_CONSTEXPR_STRLEN): Define to
zero when compiling with clang, since clang's strlen is not constexpr.

Test Plan:
Run this:
fbconfig --clang --with-project-version clang:dev folly/test && fbconfig dbg
and observe that the following no longer strikes:
folly/test/RangeTest.cpp:304:25: error: constexpr variable 'hello1' must be initialized by a constant expression

I also tried using clang-3.4 (the current default),
fbconfig --clang folly/test && fbmake dbg
with these results:
https://phabricator.fb.com/P8583443

Reviewed By: pgriess@fb.com

FB internal diff: D1262840

Summary: Fix make check.  Benchmark.cpp was included twice, eventfd doesn't exist anymore.

Test Plan: autoreconf -i; ./configure; make check on ubuntu

Reviewed By: pgriess@fb.com

FB internal diff: D1259706

Summary:
Automake complains there are two Malloc.cpp rules - rename one of them.

Facebook: Actually this file doesn't even build in fbconfig - is it still necessary?

Test Plan: autoreconf -i on ubuntu doens't fail

Reviewed By: pgriess@fb.com

FB internal diff: D1257372

Summary:
When all the variants I am dreaming of work, there's a little more than
100 tests generated.

Test Plan: fbmake runtests

Reviewed By: hannesr@fb.com

FB internal diff: D1258754

Summary: Now compiles in 18 seconds. Took two minutes before.

Test Plan: `fbmake runtests_opt`

Reviewed By: smith@fb.com

FB internal diff: D1243689

Summary:
Allow specifying `SimpleAllocator` to use for node allocation.
Added optimizations for arena allocators.

Test Plan: fbconfig -r folly/test && fbmake opt -j32

Reviewed By: soren@fb.com

FB internal diff: D1261546

Summary:
There is no reason to force heap allocation for
`ConcurrentSkipList`.

Test Plan: fbconfig -r unicorn/test && fbmake opt -j32

@override-unit-failures

Reviewed By: lucian@fb.com

FB internal diff: D1261183

Summary: For use with Thrift::Frozen, especially.

Test Plan: Unit tests

Reviewed By: philipp@fb.com

FB internal diff: D1255257

Summary:
Subtle bugfix to fbvector

Reproduction requirements:
- emplace into a vector at a non-end position
- the vector has enough capacity for the extra elements
- the value type's internal state is poisoned after a move

What should happen, explained by picture:

INITIAL SETUP

abc1234XY____
^
want to insert two elements here

FIRST STEP: uninitialized move-construction
x and y are initialized, but could be in an invalid state

abc1234xyXY__

SECOND STEP: move-assignment, in reverse order

abc123xy4XY__
abc12xy34XY__
abc1xy234XY__
abcxy1234XY__

What actually happens:
An indexing error causes the entire tail (##1234xy##) to be
move-assigned, instead of the intended head of the tail (##1234##).

If the data type's move operator does not invalidate its own data (i.e.
move is essentially a copy, as is the case with atomic types) then the
move assignment of ##y## onto ##Y## and ##x## onto ##X## is basically a
no-op. However, for types that do invalidate their own data when being
the source of a move (e.g. non-empty vectors and fbvectors), then the
end result will have bad data in place of ##X## and ##Y##.

Detection:
This bug has lain dormant for a while.

Very few people emplace into a vector, and those few who do could be
saved by using a copy-movable type like int.

The original test case did not use a data type which poisoned itself
after being the source of a move, so the tests did not notice any
oddities.

A new testsuite, which does poison itself, did notice.

Test Plan:
re-enable the test/stl_test mega-test. Run it. All tests
pass.

fbconfig -r folly && fbmake runtests_opt

Run fbvector through the new test suite (which initially caught the
error). It now passes.

Reviewed By: robbert@fb.com

FB internal diff: D1257258

Summary: These were required to build folly on OSX.

Test Plan: build

Reviewed By: davejwatson@fb.com

FB internal diff: D1256856