Summary: explicit ctor

Test Plan: contbuild

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1134033

Blame Revision: D1133938

Summary:
This restore's format()'s behavior of crashing on invalid format
strings, and adds an alternative formatChecked() function to throw
instead of crashing.

Format strings are commonly programmer defined, and fixed at compile
time.  Bad format strings are usually programmer errors, and crashing is
desirable to help catch these bugs early.

However, it is also useful to support dynamic format strings which are
not guaranteed to be well formed at compile time.  formatChecked() makes
it safe to use dynamic format strings, as a bad format strings will not
crash the program.

This does change the throwing/crashing behavior slightly: the old
format() code also used to crash if the format string referred to a
non-existent key in one of the argument containers.  I removed this,
since it seems like the argument containers are likely to be dynamic.

I also changed the code to crash on std::range_errors as well.  Various
problems in format string arguments are caught in the Conv.h code, which
throws range_errors.  The old crashing code did not crash on these
errors, but it seems useful to do so.  The only minor concern here is
that this may also crash unintentionally if the Output callback throws a
range_error.  This seems low-risk, but we can remove this behavior if
needed.

Test Plan:
Updated the BogusFormatString test to check both format() and
formatChecked().

Reviewed By: tudorb@fb.com

FB internal diff: D1144301

Summary:
This reverts 61e20daa, which changed the format code to abort on error.

I do plan to restore the crashing behavior, but I plan to make it
optional.  I will add a formatSafe() function that throws on error,
while format() catches these exceptions and crashes.

This diff is an intermediate diff to make it easier to review the
changes.  This is close to a straight revert of 61e20daa and 61a41c9b.
However, I did leave the new test case, as well as the
FormatArg::errorStr() method added by those diffs.

Test Plan:
Verified that the existing format tests pass.

Also added a handful of new tests for bad format strings.  This did
catch one bug in the code where it wasn't correctly checking for a null
return value from memchr().

Reviewed By: delong.j@fb.com

FB internal diff: D1144298

Summary:
rallocm sets the value of the second argument even if it has failed to allocate the requested size:
https://github.com/jemalloc/jemalloc/blob/898960247a8b2e6534738b7a3a244855f379faf9/src/jemalloc.c#L1903-L1906

As a result newAllocatedCapacity was being set to a value less than the original value and the logic was broken.

Test Plan: unit tests pass and my code which was crashing before now works!

Reviewed By: soren@fb.com

FB internal diff: D1144314

Test Plan: fbconfig -r folly/experimental/test && fbmake runtests_opt

Reviewed By: chaoyc@fb.com

FB internal diff: D1141124

Summary:
* Added benchmark functions to the checksum unit tests
* While in the code, fixed the order of the arguments to
the EXPECT_EQ statements so the tests will report the
actual and expected values properly upon error

Test Plan:
fbconfig -r folly && fbmake opt
_build/opt/folly/test/checksum_test --benchmark

Reviewed By: rajat@fb.com

FB internal diff: D1134930

Summary:
Removed duplicate #includes.

Facebook: The removals were examined by hand: several duplicate includes
were not removed because they were/seemed to
be in different preprocessor branches of the source.

@override-unit-failures

Test Plan: arc unit

Reviewed By: robbert@fb.com

FB internal diff: D1136491

Summary:
Adjusted the ifdefs to allow the use of the hardware CRC instructions
when compiling with gcc-4.7.x

Test Plan:
Built with gcc-4.7.1, ran the unit tests, verified that they ran the
hardware-accelerated code path

Reviewed By: saramg@fb.com

FB internal diff: D1134567

Summary:
Many callers of byLine method use it by providing arguments for File
explicit constructors. Added function breaks byLine(file desciptor)
use case. Commenting it out until File constructors are made exlicit to
fix failing tests.

Test Plan: unittest

Reviewed By: lucian@fb.com

FB internal diff: D1133938

Blame Revision: D1129497

Summary:
Previously, all IOBuf APIs required that IOBufs always be allocated on
the heap.  The only methods provided to create IOBufs returned
unique_ptr<IOBuf>.

This adds new methods to support creating IOBufs on the stack.  This is
useful in cases where the IOBuf will be short-lived, and the overhead of
the heap allocation is undesirable.  (One use case is to wrap an
existing buffer in a short-lived IOBuf so that it can be used with the
Cursor API.)

I have currently made IOBufs movable but not copyable.  (Move operations
clearly should move only a single IOBuf, but it is not clear if the copy
operators should copy only a single IOBuf or the entire chain.)

Test Plan:
Updated the unit tests to test the new CREATE, WRAP_BUFFER,
TAKE_OWNERSHIP, and COPY_BUFFER constructors, as well as the move
constructor and assignment operator.

Reviewed By: davejwatson@fb.com

FB internal diff: D1067341

Summary:
A Baton allows a thread to block once and be awoken: it captures
a single handoff.  During its lifecycle (from construction/reset to
destruction/reset) a baton must either be post()ed and wait()ed exactly
once each, or not at all.  Batons may be reused after a call to
recycle().

Baton includes no internal padding, and is only 4 bytes in size.
Any alignment or padding to avoid false sharing is up to the user.

This is basically a stripped-down semaphore that supports only a single
call to sem_post.  The current posix semaphore sem_t isn't too bad, but
this provides more a bit more speed, checking, inlining, smaller size,
a guarantee that the implementation won't change, and compatibility
with DeterministicSchedule.  Baton is directly implemented on top of
futex, and takes care to avoid system calls.

Test Plan:
1. new unit tests
2. this code has already been in production use in TAO for many months

Reviewed By: davejwatson@fb.com

FB internal diff: D1130407

Summary: -1LL<<63 was passed to folly::to() without '-' so it overflowed. Fixed that

Test Plan:
tried this:

int main() {
stringstream s;
s << "{\"int\":" << -1LL<<63 << "}";
string str = s.str();
cout << "string to parse: " << endl << str << endl;

int64_t sample = folly::parseJson(str.c_str())["int"].asInt();
LOG(INFO) << "test result = " << sample;

return 0;
}

it returns right value.

Also tried it with "-Infinity" and double type for sample. In works fine as well.

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1130155

Test Plan: unittest

Reviewed By: lucian@fb.com

FB internal diff: D1129497

Summary:
It is useful when writing additional specializations of of
toAppend() and related functions to use IsSomeString, the same way the
ones in Conv.h do.  Move IsSomeString out of detail, so outside code
doesn't access a detail namespace inappropriately.

Test Plan: fbmake runtests; arc lint

Reviewed By: tudorb@fb.com

FB internal diff: D1130910

Summary: Our script wasn't picking up removed files.

Summary: A function was getting inlined in opt mode but not in debug mode.

Test Plan: ran test

Reviewed By: philipp@fb.com

FB internal diff: D1130949

@override-unit-failures
test fix only

Summary: Include guard had FOLLY_DETAIL_.. even though code is in folly

Test Plan: unit tests

Reviewed By: davejwatson@fb.com

FB internal diff: D1130269

@override-unit-failures

Summary:
Instances of IndexedMemPool dynamically allocate and then pool
their element type (T), returning 4-byte integer indices that can be
passed to the pool's operator[] method to access or obtain pointers to
the actual elements.  Once they are constructed, elements are never
destroyed.  These two features are useful for lock-free algorithms.
The indexing behavior makes it easy to build tagged pointer-like-things,
since a large number of elements can be managed using fewer bits than a
full pointer.  The pooling behavior makes it safe to read from T-s even
after they have been recycled

Test Plan:
1. unit tests
2. unit tests using DeterministicSchedule
3. this code is moved from tao/queues where it is in production use

Reviewed By: davejwatson@fb.com

FB internal diff: D1089053

Summary:
Detailed output is a bit hard to parse visually. Add some colors for
clarity (clownyness?). Enabled only when printing stack traces from a
signal-handler to TTY.

Also:
- terse output printed empty lines if it could not find a symbol for an address; fixed by printing "(unknown)".
- added a dummy ##Crash## program to test colorization easily

Test Plan: n/a

Reviewed By: tudorb@fb.com

FB internal diff: D1128303

Summary: AHM::find checks that the returned SimpleRetT index is >= numMapsAllocated_ but that value may have changed and find pick the first element in the next sub map. Use success instead.

@override-unit-failures

Test Plan: unit tests

Reviewed By: delong.j@fb.com

FB internal diff: D1126684

Test Plan: eyeballed it

@override-unit-failures

Reviewed By: soren@fb.com

FB internal diff: D1125180

Summary:
The __builtin_ia32_crc[qd]i() functions are GCC >= 4.8 specific
folly-config.h includes must be wrapped in a FOLLY_NO_CONFIG guard
FOLLY_HAVE_WEAK_SYMBOLS needs to apply to the cpp file as well as the header

Closes #44
Closes #42

Test Plan: HHVM builds with recent folly

Reviewed By: seanc@fb.com

FB internal diff: D1117181

Summary:
The invariant after Cursor::insert is that the cursor points to the buffer
after the insert. That invariant was not followed in the branch where the
new buffer was just prepended. This change fixes the bug.

Test Plan:
Unit test modified, all tests run with
fbconfig -r folly && fbmake runtests_opt

Reviewed By: davejwatson@fb.com

FB internal diff: D1114749

Summary:
timed_mutex::try_lock_for is broken in gcc 4.8 (see
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54562), so this diff adds
a workaround using try_lock_until.  The internal implementation of
try_lock_for converts to try_lock_until, but it uses the steady_clock
instead of the system_clock as its time base.  In some gcc versions
these are the same clock so it works okay, but not in all.

Test Plan: unit tests

Reviewed By: delong.j@fb.com

FB internal diff: D1108584

Summary:
- Remove FOLLY_NORETURN from function definitions, which was causing
compilation errors on some platforms.

Test Plan: - Compile on Mac OS X with Clang

Reviewed By: tudorb@fb.com

FB internal diff: D1111416

Summary:
* Added a new crc32c() function, with a portable implementation
and an optimized version for x86 with SSE4.2

Test Plan: New unit test included

Reviewed By: tudorb@fb.com

FB internal diff: D1111515

Summary:
* folly/test/PackedSyncPtrTest.cpp: Avoid an operator new [] vs
operator delete mismatch.

Test Plan:
fbconfig --platform-all=gcc-4.8.1-glibc-2.17 --sanitize=address \
folly/test:packed_sync_ptr_test && fbmake runtests

Reviewed By: tudorb@fb.com

FB internal diff: D1111227

Summary:
- Add autoconf check for cplus_demangle_v3_callback() in libiberty and
avoid is usage when not available. Clang/libc++ on Mac OS X doesn't
support this.

Test Plan:
- fbconfig -r folly && fbmake runtests
- ./configure && make check on Mac OS X

Reviewed By: tudorb@fb.com

FB internal diff: D1108543

Summary: This reverts commit 2c4f8da739ba0294f2c947753a0e30fee291feb5.

Test Plan: Compiles

Reviewed By: tudorb@fb.com

FB internal diff: D1108840

Test Plan: Tested with subsequent thrift diff

Reviewed By: davejwatson@fb.com

FB internal diff: D1108428

Summary:
- Clang's libc++ doesn't provide these header files. Detect libc++ via
the _LIBCPP_VERSION symbol (pulling it in by sourcing some header files
earlier if necessary) and avoid using these files.
- This is another attempt at D1074481.

Test Plan: .

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1074723

Summary:
- This fixes a bug introduced in D1002959 that broke -fb platform
compilation: there was noone to forward-declare these symbols.

Test Plan:
- Built the gcc-4.8.1-glibc-2.17-fb platform
- fbconfig -r folly && fbmake runtests
- configure/make check on Mac OS X

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1074720

Blame Revision: D1002959

Test Plan: fbconfig -r folly/experimental/symbolizer folly/experimental/exception_tracer && fbmake runtests_opt

Reviewed By: wez@fb.com

FB internal diff: D1104426

Summary:
Yes, I know about boost::starts_with, but I think the convenience is worth it.
Also, I've written then equivalent of removePrefix / removeSuffix way too
many times.

Test Plan: test added

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1106425

Summary:
Even though it should be according to the docs; tdep_trace allocates memory on
first use from each thread.

Wrote a slow loop that we can use from the signal handler. The exception tracer
still uses the fast version.

Test Plan: fbconfig -r folly/experimental/symbolizer folly/experimental/exception_tracer && fbmake runtests_opt

Reviewed By: philipp@fb.com

FB internal diff: D1101095

Summary:
Add futexTimedWait to Futex which allows callers to wait on the futex
for a specified max duration.

Test Plan: -- Ran all unitests

Reviewed By: ngbronson@fb.com

FB internal diff: D1090115

Summary:
AtomicStruct<T> acts like std::atomic<T>, but it supports any
trivial or trivially-copyable type up to 8 bytes in size.  I'm not sure
why these types weren't included in the original std::atomic standard,
since this construct removes a lot of boilerplate for some uses of
std::atomic.

Test Plan:
1. unit tests
2. this code is adapted from production use in tao/queues/LifoSem

Reviewed By: davejwatson@fb.com

FB internal diff: D1085106

Summary:
folly::json::serialize by default doesn't check for valid UTF8, and as a result can generate invalid JSON.  There is an option to check for valid UTF8, which throws on an error.

This diff introduces a new option, `skip_invalid`, which replaces invalid chars with U+FFFD. http://en.wikipedia.org/wiki/Specials_(Unicode_block) seems to suggest that this is the correct replacement.

@override-unit-failures

Test Plan: g-unittest

Reviewed By: delong.j@fb.com

FB internal diff: D1102923

Summary:
Adds LZMA2 and LZMA2_VARINT_SIZE compression support for
folly::io::Compression.  This format shows some big wins for
compressing ELF object files and is useful in our modified
ccache client.

Test Plan:
Compression unittests.  Also, tested compressing object files built
in fbcode.  On average, the compression percentage improved from
~16.5% to ~12%.  But we save a lot more as object files get bigger,
which can help make bigger object files fit over fewer memcache
keys.

Reviewed By: njormrod@fb.com

FB internal diff: D1092576

Summary:
... even though Google's signal handler does it
(https://code.google.com/p/google-glog/source/browse/trunk/src/signalhandler.cc?r=16#235)

Assuming the existence of an invalid pthread_t is a lesser evil than reading
from another thread's stack, IMO.

Test Plan: folly/experimental/symbolizer/test

Reviewed By: simpkins@fb.com

FB internal diff: D1096620