Summary:
My previous change to re-implement IOBuf's internal storage mechanism
introduced a build failure when compiling with clang.

This fixes the new compilation error in IOBuf.cpp, as well as two
existing build failures in some of the unit tests.

Test Plan: Built the folly/io code with clang.

Reviewed By: andrewjcg@fb.com

FB internal diff: D1082086

Summary:
This removes kFlagExt, and instead implements the internal buffer
optimization using operator new and delete.

IOBuf::createCombined() will allocate a single buffer for the IOBuf,
SharedInfo, and the actual data itself.  Each heap allocated IOBuf now
contains a set of flags indicating if the memory segment indicating when
it can actually be deleted.  The delete operator won't actually free the
storage until both the IOBuf and the data buffer are unused (the
SharedInfo object always becomes unused at the same time as the data
buffer).

This has a couple advantages over the old mechanism:

- Other IOBufs can continue to use and share the data storage space even
after the original IOBuf associated with it is deleted.  With the old
internal buffer mechanism, internal buffers could never be shared.

- This simplifies several parts of the code, since kFlagExt no longer
exists.  Code that previously required special handling for internal
buffers no longer needs to be aware of them.

One downside is that this can result in wasted space in some cases if
the original IOBuf is changed to point at a different buffer, since the
space for the data buffer cannot be freed until the IOBuf itself is also
destroyed.  The old internal buffer mechanism also had this problem,
which we mitigated simply by disallowing internal buffers for larger
than ~200 bytes.  With the new mechanism we currently allocate an
internal buffer for up to 1024 bytes by default, but we also allow
callers to explicitly decide if they want an internal buffer or not.

Test Plan:
Added some new unit tests for the combined buffer behavior.  Also ran
all of the existing IOBuf unit tests, with and without ASAN.  (ASAN
performs additional memory checking, but also changes IOBuf's behavior
slightly as usingJEMalloc() is false with ASAN.)

Reviewed By: davejwatson@fb.com

FB internal diff: D1072336

@override-unit-failures

Summary: Because the global cache is slow and contends on locks.

Test Plan: testinproduction

Reviewed By: philipp@fb.com

FB internal diff: D1081259

Summary: It's convenient to get back the fd, similar to unique_ptr::release().

Test Plan: unittest

Reviewed By: tudorb@fb.com

FB internal diff: D1080426

Test Plan: test added

Reviewed By: lucian@fb.com

FB internal diff: D1076170

@override-unit-failures

Summary: To be used in the new fatal signal handler.

Test Plan: test added

Reviewed By: lucian@fb.com

FB internal diff: D1076169

@override-unit-failures

Summary: To be used from the (new) fatal signal handler.

Test Plan: test added

Reviewed By: lucian@fb.com

FB internal diff: D1076168

@override-unit-failures

Summary:
Improve moveToFbString()'s code which determines if we actually have a
buffer that was allocated with malloc().

The old code simply checked if flags_ == kFlagExt.  This check is rather
fragile, and relies on very specific behavior from the existing
construction methods.  It also unnecessarily forced reallocation in some
cases.

This updates the code to have more specific checks for the flags and
fields that actually matter.  In particular, even if we have an external
buffer, if sharedInfo->freeFn is set then it is not managed by malloc().
The old check happened to work because takeOwnership() was the only
function that set set freeFn, and it also set kFlagFreeSharedInfo.

This also improves the code so that it won't unnecessarily reallocate
the buffer if kFlagMaybeShared is set but the buffer isn't really
shared.  The code will also no longer unnecessarily reallocates the
buffer just because kFlagFreeSharedInfo is set.

Test Plan:
Updated the moveToFbString() test to also test with buffers created
using takeOwnership() and wrapBuffer().

Reviewed By: davejwatson@fb.com

FB internal diff: D1072304

@override-unit-failures

Summary:
The IOBuf::reserveSlow() code assumed that external buffers were always
allocated with malloc.  This would cause problems when if reserve() was
ever used on a buffer created with IOBuf::takeOwnership().

This changes the code to now check if a user-specified free function has
been supplied.  If so, then it does not try to use realloc()/rallocm(),
and it now frees the old buffer using the specified free function rather
than free().

User-supplied buffers also have a separately allocated SharedInfo
object, which must be freed when we no longer need it.

Test Plan:
Added additional unit tests to check calling reserve() on IOBufs created
with IOBuf::takeOwnership().

Reviewed By: davejwatson@fb.com

FB internal diff: D1072292

Summary:
A dumb&slow implementation. I just needed something that outputs keys in a sorted order.
No easy to use API to discourage the use due to perf implications.

Test Plan: unit test

Reviewed By: delong.j@fb.com

FB internal diff: D1073757

Summary: Rename the 'capacity' argument so g++ will not complain when -Wshadow is enabled ('capacity' is also a method on MPMCQueue)

Test Plan: compiles, unit tests pass

Reviewed By: ngbronson@fb.com

FB internal diff: D1076220

Test Plan: unittest

Reviewed By: tudorb@fb.com

FB internal diff: D1071715

Summary:
The IOBuf.StringOperations test contained two buggy tests.  According to
the comments, they were intended to test a string spanning multiple
buffers.  They created a chain of two buffers, but used Appender to
write the string, which only writes in the final buffer in the chain.

Additionally, the test didn't request enough space in the final buffer
for the string data, and didn't allow Appender to grow the buffer.  (The
code also didn't check the return value from pushAtMost() to see if it
actually wrote all of the string data.)  The test only passed because
IOBuf would normally allocate more capacity than the caller requested.

I fixed the tests to request sufficient capacity for the string data,
and use push() rather than pushAtMost() to ensure that all data was
written correctly.  I also added two new tests which actually test with
strings spanning multiple buffers.

Test Plan:
Ran this test with some changes to IOBuf that cause it to only allocate
the requested capacity in some cases, and verified that the
StringOperations test pass now.

(The changes I was testing with use goodMallocSize() to determine the
allocation size even for small internal buffers.  When not using
jemalloc, goodMallocSize() returns the original size, so it only
allocates the capacity requested by the caller.)

Reviewed By: pgriess@fb.com

FB internal diff: D1072283

Summary:
- This diff introduces errors when compiling with clang in fbcode.
Revert until they get fixed.

Test Plan: .

Reviewed By: tconerly@fb.com

FB internal diff: D1074481

Blame Revision: D998595

Summary:
- D1030008 added Subprocess to libfolly in automake builds. This
surfaced some ambient compilation errors that slipped through in my
prior run through porting this.
- Mac OS X uses a nonstandard location for wait.h
- Non-Linux platforms don't support prctl; gate that on Linux

Test Plan:
- fbmake runtests in fbcode
- make check on Mac OS X

Reviewed By: davejwatson@fb.com

FB internal diff: D1066273

Blame Revision: D1030008

Summary:
- It looks like Xcode 5.0.2 / clang 500.2.79 changed this behavior. The
prior behavior was seen against Xcode 5.0.1 / clang 500.2.75. Blerg.
If we really have to support different Xcode flavors we can come up
with a real solution for this, but for now just take advantage of the
fact that clang and gcc seem to agree on this.

Test Plan: - make check on Mac OS X

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1066272

Blame Revision: D998590

Summary:
- Add generated .cpp files to CLEANFILES so that they get swept up
during 'make clean' and friends

Test Plan: - autoreconf -i && ./configure && make check && make clean

Reviewed By: davejwatson@fb.com

FB internal diff: D1066270

Summary:
- Prior to this fix, the test relied upon begin() being evaluated after
the fbstring constructor (even thoug the order of evaluation of
function arguments is un-defined). When this assumption was violated,
begin() ended up with an iterator that was invalid since it points to
data internal to fbstring, and the fbstring copy constructor can end
up triggering re-allocation in the source.

Test Plan:
- fbconfig -r folly && fbmake runtests
- ./configure && make check on Ubuntu/FC/Mac

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1014093

Summary:
- Apple platforms either have different __thread behavior than Linux: on
i386 __thread values are zeroed out before the destructor provided to
pthread_key_create(3) is called; on ARM, __thread isn't supported at
all. To handle this, use pthread_getspecific(3) to manage the array of
IDs on these platforms only.

Test Plan:
- fbconfig -r folly && fbmake runtests
- ./configure && make check on Ubuntu/FC/Mac

Reviewed By: simpkins@fb.com

FB internal diff: D1008923

Summary:
- It turns out that it's not always desirable to perform runtime
resolution of weak symbols. For example, on iOS, weak symbols are
resolved at runtime only if *all* symbol resolution is deferred util
then, which is undesirable for othe reasons. Detect such platforms at
configure time and use that information to populate detail/Malloc.h
with the correct declarations: weak symbols or extern symbols with
a value of nullptr.

Test Plan:
- fbconfig -r folly && fbmake runtests
- ./configure && make check on Ubuntu/FC/Mac

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1002959

Summary:
- Clang's libc++ doesn't provide these header files. Detect libc++ via
the _LIBCPP_VERSION symbol (pulling it in by sourcing some header
files earlier if necessary) and avoid using these files.

Test Plan:
- fbconfig -r folly && fbmake runtests
- ./configure && make check on Ubuntu/FC/Mac

Reviewed By: andrewjcg@fb.com

FB internal diff: D998595

Summary: Adding const avoids a copy constuctor.

Test Plan:
Unit tests, new benchmark:
toJson        1.83us  546.15K
toJson        1.54us  649.98K

Reviewed By: tudorb@fb.com

FB internal diff: D1071781

Summary: Broken in D1054291

Test Plan: build adpublisher in clang mode which was one of the two projects that failed to build.

Reviewed By: bmaurer@fb.com

FB internal diff: D1067652

@override-unit-failures

Summary:
Port TimeseriesHistogram from common/stats to folly. Similarly to
MultiLevelTimeSeries and Histogram classes we've converted before.

Test Plan:
Ported the old unittest for TimeseriesHistogram from common/stats to
folly/test. The same unittest is left in tact and is still passing as well. Ran
all tests in folly/test and common/stats

Reviewed By: simpkins@fb.com

FB internal diff: D1054291

Summary: Missed these on the first pass because they're generated

Test Plan: Built open source folly, links

Reviewed By: pgriess@fb.com

FB internal diff: D1062451

Summary: Test compressing low-entropy (constant) data as well.

Test Plan: ran it

Reviewed By: tuomas.pelkonen@fb.com

FB internal diff: D1061444

@override-unit-failures

Summary: See https://www.facebook.com/groups/fbcode/permalink/601496126553897/

Test Plan: contbuild

Reviewed By: ldbrandy@fb.com

FB internal diff: D1055852

@override-unit-failures
some hphp_dbg tests postponed for 1.5 days, all other hphp tests
(including some hphp_dbg ones) are passing

Summary: There's a bit of a gotcha here, let's provide at least a note on this.

Test Plan: Unit tests

Reviewed By: tudorb@fb.com

FB internal diff: D1058244

Summary:
Added SCOPE_FAIL and SCOPE_SUCCESS macros in non-portable C++. The macros are similar to D's scope(failure) and scope(success).
Currently the supported platforms are GCC and MSVC. For all others, std::uncaught_exception() is used, which will fail if the macros are used in a destructor called during stack unwinding.
@override-unit-failures

Test Plan:
1. Added new unit test to ScopeGuardTest.cpp.
2. Ran fbconfig -r folly && fbmake dbg
3. Ran _build/dbg/folly/test/scope_guard_test to make sure my unit test was running and passing.

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1033621

Summary:
This two header files are missing from the Makefile which make the
open source version broken.

Test Plan: code using open source version compiles

Reviewed By: meyering@fb.com

FB internal diff: D1050326

Summary: Because I needed it.

Test Plan: test added

Reviewed By: delong.j@fb.com

FB internal diff: D1032179

Summary: just added few words about Subprocess.

Reviewed By: delong.j@fb.com

Test Plan: none

Summary:
as per summary
@override-unit-failures

Test Plan: run unit tests

Reviewed By: delong.j@fb.com

FB internal diff: D1031726

Summary:
GCC can't compile code that passes a MoveWrapper into a lambda
in some cases. For example if the capture list has
[someConstObject, myMoveWrapper], gcc 4.7 fails. This specific case
works with gcc 4.8, but other cases (e.g. [this, myMoveWrapper]) still
fail.

This diff is a hack that makes code like that compile. It can be
removed, along with MoveWrapper itself once we move to C++14 with its
extended lambda syntax.

Test Plan: fbmake

Reviewed By: hans@fb.com

FB internal diff: D1032585

Summary:
Update Makefile.am with recent files.

Leaves out io/Compression because it uses lz4, of which there are no prebuilt packages for ubuntu (didn't check fedora, but I assume it is the same story there)

Test Plan: on ubuntu 12.10, ./configure, make, make install

Reviewed By: tudorb@fb.com

FB internal diff: D1030008

Summary: Add MoveWrapper to folly.

@override-unit-failures

Test Plan: arc unit

Reviewed By: hans@fb.com

FB internal diff: D1030005

Summary:
Other inclusions of folly-config.h watch for this.
It must have been missed when this file was added.

Test Plan: Fixes Mac OSX build.

Reviewed By: seanc@fb.com

FB internal diff: D1028354

Summary:
MemoryMapping.cpp uses DEFINE_int64() from gflags,
but doesn't include it anywhere.

@override-unit-failures

Test Plan: OSS build failure goes away

Reviewed By: seanc@fb.com

FB internal diff: D1026364

Summary: Pretty sure it was meant to be this way...

@override-unit-failures

Test Plan: n/a

Reviewed By: tudorb@fb.com

FB internal diff: D1020789

Summary:
Amortized cost of `TypedIOBuf::push()` is `O(N)` at this moment.
Unless we're using jemalloc where we can rely on `realloc()`.  In this
diff we switch to exponential growth otherwise.

Test Plan:
built unicorn/test with ASan (and thus without jemalloc), ran
end-to-end tests and verified we no longer spend minutes to build di4

@override-unit-failures

Reviewed By: tudorb@fb.com

FB internal diff: D1015233