Summary: We've intentionally eliminated support for gcc's old variadic macro syntax so as to bring our close in line with compatbile standards. This diff enables folly to build using warp.

Test Plan: built folly

Reviewed By: delong.j@fb.com

FB internal diff: D1176956

Summary:
- We need to be able to disable FOLLY_HAVE_WEAK_SYMBOLS via
-DFOLLY_HAVE_WEAK_SYMBOLS=0. Switch the #ifdef checks to #if.

Test Plan:
- fbconfig -r folly && fbmake runtests
- Build in fbobjc

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1150036

Summary:
Without this change, the sig-handling test's deliberate
invalid access would trigger an ASAN-abort, which differed
from the expected SEGV.  Skip this test when ASAN is enabled.
* folly/experimental/symbolizer/test/SignalHandlerTest.cpp: Include
CPortability.h for definion of FOLLY_SANITIZE_ADDRESS.
(SignalHandler) [FOLLY_SANITIZE_ADDRESS]: Provide a different
regexp to match the first line of output from an ASAN-enabled binary.

Test Plan:
fbconfig --sanitize=address --platform-all=gcc-4.8.1-glibc-2.17 \
folly/experimental/symbolizer/test:signal_handler_test \
&& fbmake --fast runtests \
&& fbmake --fast runtests_opt

Sample output, before this change: https://phabricator.fb.com/P5428975
(search down to first AddressSanitizer abort.
With this change, expect that test to pass.

Reviewed By: lucian@fb.com

FB internal diff: D1164768

Test Plan: contbuild

Reviewed By: philipp@fb.com

FB internal diff: D1164211

Summary:
This diff is a split from D1157286 for folly part;
adds a substract function which substracts a histogram data from another;
modifies some of the const specifiers in the Histogram class.

Test Plan: tested on the thrift perftest by adding x-th percentile latency stats

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1158270

Summary: Fix OS X compilation

Test Plan: compiled on OS X

Reviewed By: delong.j@fb.com

FB internal diff: D1163199

Summary:
scanHaystackBlock may read-overrun the needle.data() buffer by
up to 15 bytes, but that overrun will never cross a page boundary.
The fix is to turn off ASAN-checking for this function, but since
that attribute is accompanied by a "noinline" one (which conflicts
with the function's own "inline"), I have also removed the "inline"
attribute on both decl and defn.  That is a good thing, regardless:
these days, there are very few cases in which we should be trying to
tell the compiler to inline.

Test Plan:
Before, this would elicit an ASAN abort.  Now it passes 100%:

fbconfig --platform-all=gcc-4.8.1-glibc-2.17 --sanitize=address \
folly/test:range_test && fbmake runtests

Reviewed By: philipp@fb.com

FB internal diff: D1162982

Summary:
Rather than opening and closing Elf files every time we symbolize them, we open
the first time and cache.

We're using two caches: one for the signal handler (fixed size, slow, but
async-signal-safe) and one for exception tracing and general use.

Also, unrelated, removed two useless frames from the stack trace dump in the
signal handler.

Test Plan: tests added

Reviewed By: lucian@fb.com

FB internal diff: D1161444

Test Plan: fbconfig -r folly/experimental/test:eliasfano_test && fbmake runtests_opt

Reviewed By: lucian@fb.com

FB internal diff: D1161670

Summary:
Changing all the include paths following D1151911.
Depends on D1151911

Test Plan: contbuild

Reviewed By: ldbrandy@fb.com

FB internal diff: D1159003

Facebook:
Contbuild is mostly passed, remaining failures are unrelated. I'm confident other failures are unrelated.
@override-unit-failures

Summary:
At long last, promoting this out of experimental. Also, while I'm at
it, I've separated the tests and benchmarks into their corresponding parts.

Redirect headers provided.

Test Plan: Unit tests, contbuild.

Reviewed By: marcelo.juchem@fb.com

FB internal diff: D1151911

Summary:
NaN or INF is not allowed according to JSON specification.
This will cause problems when other places try to encode the output
of toJson with NaN or INF.
raise error when parsing object with NaN or INF to JSON.

Test Plan: manually test, raise error for object w/ NaN

Reviewed By: delong.j@fb.com

FB internal diff: D1143853

Summary: Since I don't want to pay the cost when we access array elements. If I did, I would just use Java.

Test Plan: fbconfig folly/test/ && fbmake runtests

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1158505

Summary:
In a rare case, the current thread's would be inserted in the StaticMeta linked
list twice, causing the list to be corrupted, leading to code spinning forever.

After a fork, in the child, only the current thread survives, so all other threads
must be forcefully removed from StaticMeta. We do that by clearing the list
and re-adding the current thread, but we didn't check whether the current thread
was already there. It is possible for the current thread to not be in the list
if it never used any ThreadLocalPtr objects with the same tag.

Now, when the thread in the child tries to use a ThreadLocalPtr with the same
tag, it adds itself to the list (##if (prevCapacity == 0)
meta.push_back(threadEntry)##), but it had already been added by the post-fork
handler, boom.

Fix by adding the necessary check in onForkChild.

Test Plan: @durham's test case, also added new test for this

Reviewed By: delong.j@fb.com

FB internal diff: D1158672

@override-unit-failures

Summary:
This diff causes Baton to reduce a thread's memory footprint when it
blocks for an extended period (by default 5 to 7.5 seconds).  Reductions
are achieved by flushing the thread-local jemalloc caches (if jemalloc
is in use) and by calling madvise(MADV_DONTNEED) on the portion of the
thread's stack that isn't active.  Once the thread resumes execution
both of these resources will be reallocated.  Configuration is via
system-wide default.

Test Plan:
1. new unit tests
2. manual execution of existing unit tests with very low idleTimeout
3. peek and poke with gdb to observe madvise discarding the page

Reviewed By: davejwatson@fb.com

FB internal diff: D1146966

Summary:
FB_LOG_EVERY_MS was using clock() as its source of time, which
measures the elapsed CPU time of the process.  The name and the docs
suggest that wall time was intended, so that's what this diff does.

Test Plan: new unit test

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1157926

Summary:
Change the way forward / skip pointers are encoded, so we can
expect that `uint32_t` will be enough to address ~4B elements instead
of ~2B as it is now.

Test Plan:
Ran benchmarks for both versions, saw no significant
difference.

Added tests.

Reviewed By: lucian@fb.com

FB internal diff: D1156950

Summary:
- Use the BOOST_STD_EXTENSION_NAMESPACE macro to pick the namespace to
use for hash specializations

Test Plan:
- fbconfig -r unicorn/utils/ && fbmake opt
- fbconfig -r folly && fbmake runtests
- Build in fbobjc with libc++
@override-unit-failures

Reviewed By: rajatr@fb.com

FB internal diff: D1153422

Blame Revision: D1152140

Summary:
Add wrapper functions for directly appending to a string, similar to the
existing format() and vformat() wrappers.

Test Plan:
Converted some existing code using these format() and vformat() wrappers
to use formatChecked() and vformatChecked().

Reviewed By: jon.coens@fb.com

FB internal diff: D1151496

@override-unit-failures

Summary:
- The fbobjc builds complain about use of deprecated <ext/hash_set> and
friends, and then fail because of -Werror. Just use the C++ standard
collections.

Test Plan:
- fbconfig -r folly && fbmake runtests
- Build for iOS and Android
@override-unit-failures

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1152140

Summary: - Update the pre-computed folly-config.h file for Apple builds

Test Plan:
- fbconfig -r folly && fbmake runtests
- Build in fbobjc

Reviewed By: delong.j@fb.com

FB internal diff: D1150188

Summary: ...and stop lint complaining at everyone.

@override-unit-failures

Test Plan: Inspection. Build folly.

Reviewed By: delong.j@fb.com

FB internal diff: D1151216

Summary:
- In libstdc++, existing code uses the internal std::__ostream_insert()
method to write a formatted string that can include '\0' characters.
This internal method doesn't exist in libc++. Instead, use the
relevant internal bits.

Test Plan:
- fbconfig -r folly && fbmake runtests
- ./configure && make check on Mac OS X

Reviewed By: njormrod@fb.com

FB internal diff: D1108540

Summary:
- The fbobjc codebase defines this macro as well, and to the same thing.
However, Clang complains on macro redefinitions if paramater names are
different, even if the macros expand to the same thing. Normalize the
Folly version to 'x', which is what fbobjc uses.

Test Plan:
- fbconfig -r folly && fbmake runtests
- Builds in fbobjc

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1149985

Summary: this is a simple method that allows a folly::Range `[b, e)` to be split on a character at position `i` (where b <= i < e) in an incremental manner.
@override-unit-failures

Test Plan: unit tests added

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1145631

Summary:
- On iOS, off_t is an int64_t, and as such std::min() doesn't compile
since the types don't match. Normalize to size_t and fail with an
error if this conversion can't be made

Test Plan:
- fbconfig -r folly && fbmake runtests
- Built on iOS

Reviewed By: tudorb@fb.com

FB internal diff: D1142795

Summary: explicit ctor

Test Plan: contbuild

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1134033

Blame Revision: D1133938

Summary:
This restore's format()'s behavior of crashing on invalid format
strings, and adds an alternative formatChecked() function to throw
instead of crashing.

Format strings are commonly programmer defined, and fixed at compile
time.  Bad format strings are usually programmer errors, and crashing is
desirable to help catch these bugs early.

However, it is also useful to support dynamic format strings which are
not guaranteed to be well formed at compile time.  formatChecked() makes
it safe to use dynamic format strings, as a bad format strings will not
crash the program.

This does change the throwing/crashing behavior slightly: the old
format() code also used to crash if the format string referred to a
non-existent key in one of the argument containers.  I removed this,
since it seems like the argument containers are likely to be dynamic.

I also changed the code to crash on std::range_errors as well.  Various
problems in format string arguments are caught in the Conv.h code, which
throws range_errors.  The old crashing code did not crash on these
errors, but it seems useful to do so.  The only minor concern here is
that this may also crash unintentionally if the Output callback throws a
range_error.  This seems low-risk, but we can remove this behavior if
needed.

Test Plan:
Updated the BogusFormatString test to check both format() and
formatChecked().

Reviewed By: tudorb@fb.com

FB internal diff: D1144301

Summary:
This reverts 61e20daa, which changed the format code to abort on error.

I do plan to restore the crashing behavior, but I plan to make it
optional.  I will add a formatSafe() function that throws on error,
while format() catches these exceptions and crashes.

This diff is an intermediate diff to make it easier to review the
changes.  This is close to a straight revert of 61e20daa and 61a41c9b.
However, I did leave the new test case, as well as the
FormatArg::errorStr() method added by those diffs.

Test Plan:
Verified that the existing format tests pass.

Also added a handful of new tests for bad format strings.  This did
catch one bug in the code where it wasn't correctly checking for a null
return value from memchr().

Reviewed By: delong.j@fb.com

FB internal diff: D1144298

Summary:
rallocm sets the value of the second argument even if it has failed to allocate the requested size:
https://github.com/jemalloc/jemalloc/blob/898960247a8b2e6534738b7a3a244855f379faf9/src/jemalloc.c#L1903-L1906

As a result newAllocatedCapacity was being set to a value less than the original value and the logic was broken.

Test Plan: unit tests pass and my code which was crashing before now works!

Reviewed By: soren@fb.com

FB internal diff: D1144314

Test Plan: fbconfig -r folly/experimental/test && fbmake runtests_opt

Reviewed By: chaoyc@fb.com

FB internal diff: D1141124

Summary:
* Added benchmark functions to the checksum unit tests
* While in the code, fixed the order of the arguments to
the EXPECT_EQ statements so the tests will report the
actual and expected values properly upon error

Test Plan:
fbconfig -r folly && fbmake opt
_build/opt/folly/test/checksum_test --benchmark

Reviewed By: rajat@fb.com

FB internal diff: D1134930

Summary:
Removed duplicate #includes.

Facebook: The removals were examined by hand: several duplicate includes
were not removed because they were/seemed to
be in different preprocessor branches of the source.

@override-unit-failures

Test Plan: arc unit

Reviewed By: robbert@fb.com

FB internal diff: D1136491

Summary:
Adjusted the ifdefs to allow the use of the hardware CRC instructions
when compiling with gcc-4.7.x

Test Plan:
Built with gcc-4.7.1, ran the unit tests, verified that they ran the
hardware-accelerated code path

Reviewed By: saramg@fb.com

FB internal diff: D1134567

Summary:
Many callers of byLine method use it by providing arguments for File
explicit constructors. Added function breaks byLine(file desciptor)
use case. Commenting it out until File constructors are made exlicit to
fix failing tests.

Test Plan: unittest

Reviewed By: lucian@fb.com

FB internal diff: D1133938

Blame Revision: D1129497

Summary:
Previously, all IOBuf APIs required that IOBufs always be allocated on
the heap.  The only methods provided to create IOBufs returned
unique_ptr<IOBuf>.

This adds new methods to support creating IOBufs on the stack.  This is
useful in cases where the IOBuf will be short-lived, and the overhead of
the heap allocation is undesirable.  (One use case is to wrap an
existing buffer in a short-lived IOBuf so that it can be used with the
Cursor API.)

I have currently made IOBufs movable but not copyable.  (Move operations
clearly should move only a single IOBuf, but it is not clear if the copy
operators should copy only a single IOBuf or the entire chain.)

Test Plan:
Updated the unit tests to test the new CREATE, WRAP_BUFFER,
TAKE_OWNERSHIP, and COPY_BUFFER constructors, as well as the move
constructor and assignment operator.

Reviewed By: davejwatson@fb.com

FB internal diff: D1067341

Summary:
A Baton allows a thread to block once and be awoken: it captures
a single handoff.  During its lifecycle (from construction/reset to
destruction/reset) a baton must either be post()ed and wait()ed exactly
once each, or not at all.  Batons may be reused after a call to
recycle().

Baton includes no internal padding, and is only 4 bytes in size.
Any alignment or padding to avoid false sharing is up to the user.

This is basically a stripped-down semaphore that supports only a single
call to sem_post.  The current posix semaphore sem_t isn't too bad, but
this provides more a bit more speed, checking, inlining, smaller size,
a guarantee that the implementation won't change, and compatibility
with DeterministicSchedule.  Baton is directly implemented on top of
futex, and takes care to avoid system calls.

Test Plan:
1. new unit tests
2. this code has already been in production use in TAO for many months

Reviewed By: davejwatson@fb.com

FB internal diff: D1130407

Summary: -1LL<<63 was passed to folly::to() without '-' so it overflowed. Fixed that

Test Plan:
tried this:

int main() {
stringstream s;
s << "{\"int\":" << -1LL<<63 << "}";
string str = s.str();
cout << "string to parse: " << endl << str << endl;

int64_t sample = folly::parseJson(str.c_str())["int"].asInt();
LOG(INFO) << "test result = " << sample;

return 0;
}

it returns right value.

Also tried it with "-Infinity" and double type for sample. In works fine as well.

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1130155

Test Plan: unittest

Reviewed By: lucian@fb.com

FB internal diff: D1129497

Summary:
It is useful when writing additional specializations of of
toAppend() and related functions to use IsSomeString, the same way the
ones in Conv.h do.  Move IsSomeString out of detail, so outside code
doesn't access a detail namespace inappropriately.

Test Plan: fbmake runtests; arc lint

Reviewed By: tudorb@fb.com

FB internal diff: D1130910