Summary:
It is useful when writing additional specializations of of
toAppend() and related functions to use IsSomeString, the same way the
ones in Conv.h do.  Move IsSomeString out of detail, so outside code
doesn't access a detail namespace inappropriately.

Test Plan: fbmake runtests; arc lint

Reviewed By: tudorb@fb.com

FB internal diff: D1130910

Summary: Our script wasn't picking up removed files.

Summary: A function was getting inlined in opt mode but not in debug mode.

Test Plan: ran test

Reviewed By: philipp@fb.com

FB internal diff: D1130949

@override-unit-failures
test fix only

Summary: Include guard had FOLLY_DETAIL_.. even though code is in folly

Test Plan: unit tests

Reviewed By: davejwatson@fb.com

FB internal diff: D1130269

@override-unit-failures

Summary:
Instances of IndexedMemPool dynamically allocate and then pool
their element type (T), returning 4-byte integer indices that can be
passed to the pool's operator[] method to access or obtain pointers to
the actual elements.  Once they are constructed, elements are never
destroyed.  These two features are useful for lock-free algorithms.
The indexing behavior makes it easy to build tagged pointer-like-things,
since a large number of elements can be managed using fewer bits than a
full pointer.  The pooling behavior makes it safe to read from T-s even
after they have been recycled

Test Plan:
1. unit tests
2. unit tests using DeterministicSchedule
3. this code is moved from tao/queues where it is in production use

Reviewed By: davejwatson@fb.com

FB internal diff: D1089053

Summary:
Detailed output is a bit hard to parse visually. Add some colors for
clarity (clownyness?). Enabled only when printing stack traces from a
signal-handler to TTY.

Also:
- terse output printed empty lines if it could not find a symbol for an address; fixed by printing "(unknown)".
- added a dummy ##Crash## program to test colorization easily

Test Plan: n/a

Reviewed By: tudorb@fb.com

FB internal diff: D1128303

Summary: AHM::find checks that the returned SimpleRetT index is >= numMapsAllocated_ but that value may have changed and find pick the first element in the next sub map. Use success instead.

@override-unit-failures

Test Plan: unit tests

Reviewed By: delong.j@fb.com

FB internal diff: D1126684

Test Plan: eyeballed it

@override-unit-failures

Reviewed By: soren@fb.com

FB internal diff: D1125180

Summary:
The __builtin_ia32_crc[qd]i() functions are GCC >= 4.8 specific
folly-config.h includes must be wrapped in a FOLLY_NO_CONFIG guard
FOLLY_HAVE_WEAK_SYMBOLS needs to apply to the cpp file as well as the header

Closes #44
Closes #42

Test Plan: HHVM builds with recent folly

Reviewed By: seanc@fb.com

FB internal diff: D1117181

Summary:
The invariant after Cursor::insert is that the cursor points to the buffer
after the insert. That invariant was not followed in the branch where the
new buffer was just prepended. This change fixes the bug.

Test Plan:
Unit test modified, all tests run with
fbconfig -r folly && fbmake runtests_opt

Reviewed By: davejwatson@fb.com

FB internal diff: D1114749

Summary:
timed_mutex::try_lock_for is broken in gcc 4.8 (see
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=54562), so this diff adds
a workaround using try_lock_until.  The internal implementation of
try_lock_for converts to try_lock_until, but it uses the steady_clock
instead of the system_clock as its time base.  In some gcc versions
these are the same clock so it works okay, but not in all.

Test Plan: unit tests

Reviewed By: delong.j@fb.com

FB internal diff: D1108584

Summary:
- Remove FOLLY_NORETURN from function definitions, which was causing
compilation errors on some platforms.

Test Plan: - Compile on Mac OS X with Clang

Reviewed By: tudorb@fb.com

FB internal diff: D1111416

Summary:
* Added a new crc32c() function, with a portable implementation
and an optimized version for x86 with SSE4.2

Test Plan: New unit test included

Reviewed By: tudorb@fb.com

FB internal diff: D1111515

Summary:
* folly/test/PackedSyncPtrTest.cpp: Avoid an operator new [] vs
operator delete mismatch.

Test Plan:
fbconfig --platform-all=gcc-4.8.1-glibc-2.17 --sanitize=address \
folly/test:packed_sync_ptr_test && fbmake runtests

Reviewed By: tudorb@fb.com

FB internal diff: D1111227

Summary:
- Add autoconf check for cplus_demangle_v3_callback() in libiberty and
avoid is usage when not available. Clang/libc++ on Mac OS X doesn't
support this.

Test Plan:
- fbconfig -r folly && fbmake runtests
- ./configure && make check on Mac OS X

Reviewed By: tudorb@fb.com

FB internal diff: D1108543

Summary: This reverts commit 2c4f8da739ba0294f2c947753a0e30fee291feb5.

Test Plan: Compiles

Reviewed By: tudorb@fb.com

FB internal diff: D1108840

Test Plan: Tested with subsequent thrift diff

Reviewed By: davejwatson@fb.com

FB internal diff: D1108428

Summary:
- Clang's libc++ doesn't provide these header files. Detect libc++ via
the _LIBCPP_VERSION symbol (pulling it in by sourcing some header files
earlier if necessary) and avoid using these files.
- This is another attempt at D1074481.

Test Plan: .

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1074723

Summary:
- This fixes a bug introduced in D1002959 that broke -fb platform
compilation: there was noone to forward-declare these symbols.

Test Plan:
- Built the gcc-4.8.1-glibc-2.17-fb platform
- fbconfig -r folly && fbmake runtests
- configure/make check on Mac OS X

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1074720

Blame Revision: D1002959

Test Plan: fbconfig -r folly/experimental/symbolizer folly/experimental/exception_tracer && fbmake runtests_opt

Reviewed By: wez@fb.com

FB internal diff: D1104426

Summary:
Yes, I know about boost::starts_with, but I think the convenience is worth it.
Also, I've written then equivalent of removePrefix / removeSuffix way too
many times.

Test Plan: test added

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1106425

Summary:
Even though it should be according to the docs; tdep_trace allocates memory on
first use from each thread.

Wrote a slow loop that we can use from the signal handler. The exception tracer
still uses the fast version.

Test Plan: fbconfig -r folly/experimental/symbolizer folly/experimental/exception_tracer && fbmake runtests_opt

Reviewed By: philipp@fb.com

FB internal diff: D1101095

Summary:
Add futexTimedWait to Futex which allows callers to wait on the futex
for a specified max duration.

Test Plan: -- Ran all unitests

Reviewed By: ngbronson@fb.com

FB internal diff: D1090115

Summary:
AtomicStruct<T> acts like std::atomic<T>, but it supports any
trivial or trivially-copyable type up to 8 bytes in size.  I'm not sure
why these types weren't included in the original std::atomic standard,
since this construct removes a lot of boilerplate for some uses of
std::atomic.

Test Plan:
1. unit tests
2. this code is adapted from production use in tao/queues/LifoSem

Reviewed By: davejwatson@fb.com

FB internal diff: D1085106

Summary:
folly::json::serialize by default doesn't check for valid UTF8, and as a result can generate invalid JSON.  There is an option to check for valid UTF8, which throws on an error.

This diff introduces a new option, `skip_invalid`, which replaces invalid chars with U+FFFD. http://en.wikipedia.org/wiki/Specials_(Unicode_block) seems to suggest that this is the correct replacement.

@override-unit-failures

Test Plan: g-unittest

Reviewed By: delong.j@fb.com

FB internal diff: D1102923

Summary:
Adds LZMA2 and LZMA2_VARINT_SIZE compression support for
folly::io::Compression.  This format shows some big wins for
compressing ELF object files and is useful in our modified
ccache client.

Test Plan:
Compression unittests.  Also, tested compressing object files built
in fbcode.  On average, the compression percentage improved from
~16.5% to ~12%.  But we save a lot more as object files get bigger,
which can help make bigger object files fit over fewer memcache
keys.

Reviewed By: njormrod@fb.com

FB internal diff: D1092576

Summary:
... even though Google's signal handler does it
(https://code.google.com/p/google-glog/source/browse/trunk/src/signalhandler.cc?r=16#235)

Assuming the existence of an invalid pthread_t is a lesser evil than reading
from another thread's stack, IMO.

Test Plan: folly/experimental/symbolizer/test

Reviewed By: simpkins@fb.com

FB internal diff: D1096620

Summary: Makes clang nervous

@override-unit-failures

Test Plan: fbconfig -r folly && fbmake dbg succeeds

Reviewed By: delong.j@fb.com

FB internal diff: D1094209

Summary:
If a previous handler was found, the code was missing the step of
actually re-raising the handler after restoring the old handler.

Test Plan: Verified that our service actually exits after receiving SIGTERM.

Reviewed By: tudorb@fb.com

FB internal diff: D1096360

Summary:
We want to install the signal handler first and then give users an opportunity
to add their callbacks.

Test Plan: built

Reviewed By: lucian@fb.com

FB internal diff: D1092083

@override-unit-failures

Summary:
(not committing yet, but I want to trigger unittests)

The glog symbolizer that we use has a few bugs (abort()s on certain small
shared libraries) and doesn't allow us to distinguish between template
specializations and function overloads (which, given that our code is more
template-heavy than Google's, has in fact been an issue).

Luckily, we have our own in folly, which doesn't have these problems and also
supports mappings from address to file and line number.

Switch translateFrames (aka the fb303 call that cpprof uses) to our symbolizer.
Also, removed a lot of dead code in common/process.

Test Plan: common/process, tested cpprof by hand

Reviewed By: lucian@fb.com

FB internal diff: D1090907

@override-unit-failures

Summary: SCOPE_SUCCESS may legitimately throw, so changed the `noexcept` attribute to a conditional one. Also added the noexcept promise to the lambdas in the hope the compiler will check code during compilation appropriately. (It currently doesn't.)

Test Plan: Added a unittest that failed, not passes.

Reviewed By: simpkins@fb.com

FB internal diff: D1093328

@override-unit-failures

Summary:
The alignment requirements and details required to avoid false
sharing belong in CacheLocality, so this diff moves them there.

@override-unit-failures

Test Plan: fbmake runtests

Reviewed By: davejwatson@fb.com

FB internal diff: D1086090

Summary:
D1090936 noticed some problems with fbstring iomanip behavior.
The root cause is that os.write(ostream, char*, size_t) is an
UnformattedOutputFunction, so disregards setw(), setfill(), and
left/right alignment.

The use of os.write instead of os << str.data() is intentional:
D367009 switched from the latter to the former so that strings
containing a '\0' are printed properly.

There does not seem to be a public function to write with formatting.
Where needed in libgcc, the function __ostream_insert is used. Since
FBString already uses such 'private' functions, __ostream_insert is an
appropriate solution.

@override-unit-failures

Test Plan:
Added test cases to FBStringTest.cpp to cover iomanip.
fbconfig -r folly && fbmake opt && fbmake runtests_opt

Reviewed By: andrei.alexandrescu@fb.com

FB internal diff: D1091474

Summary:
Also, switch to the simpler unw_backtrace(), which has the nice advantage of
actually doing IP adjustment (-1 in certain frames) correctly, unlike me :)

This is in preparation for the faster backtrace in libunwind 1.1.

Test Plan: folly/experimental/exception_tracer, folly/experimental/symbolizer, admarket/lib/util:memory_tracker_test

Reviewed By: lucian@fb.com

FB internal diff: D1088357

Summary:
CacheLocality reads cache sharing information from sysfs to
determine how CPUs should be grouped to minimize contention, Getcpu
provides fast access to the current CPU via __vdso_getcpu, and
AccessSpreader uses these two to optimally spread accesses among a
predetermined number of stripes.

AccessSpreader<>::current(n) microbenchmarks at 22 nanos, which is
substantially less than the cost of a cache miss.  This means that we
can effectively use it to reduce cache line ping-pong on striped data
structures such as IndexedMemPool or statistics counters.

Because CacheLocality looks at all of the cache levels, it can be used for
different levels of optimization.  AccessSpreader<>::stripeByChip.current()
uses as few stripes as possible to avoid off-chip communication,
AccessSpreader<>::stripeByCore.current() uses as many stripes as necessary
to get the optimal speedup, but no more.

@override-unit-failures

Test Plan: new unit tests

Reviewed By: davejwatson@fb.com

FB internal diff: D1076718

Summary:
Per-thread caching calls malloc, which means it doesn't work for malloc's
profiling (and also it isn't async-signal-safe, despite the documentation).

Test Plan: compiled

Reviewed By: philipp@fb.com

FB internal diff: D1085368

@override-unit-failures

Summary:
Also, C++ify the interface and switch to per-thread caching in libunwind as per
D1081259

Test Plan: folly/experimental/symbolizer/test

Reviewed By: lucian@fb.com

FB internal diff: D1081272

Summary:
If the compare part of cmpxchg fails, it writes the unexpected value
to %rax. At certain optimization levels this was causing me to hit an
incorrectly failed assertion in some thrift code. I also cleaned up the asm
statement to use named operands.

Test Plan: Added new test that fails before this diff.

Reviewed By: delong.j@fb.com

FB internal diff: D1083222
@override-unit-failures

Summary:
Update IOBuf::gather() and RWCursor::gather() to check their arguments
more carefully, and throw std::overflow_errors if the caller attempts to
gather more data than is available.

The comments for IOBuf::gather() claimed that it ensured that maxLength
bytes would always be available when it returned.  However, if the total
chain length was less than maxLength, it would simply coalesce the full
chain and return successfully, even though fewer than maxLength bytes
were available.  This fixes the code to throw a std::overflow_error
rather than coalescing the chain.

Additional, this updates RWCursor::gather() to ensure that it does not
attempt to gather past the end of the IOBuf chain.  Previously it could
gather past the end of the chain, coalescing the head of the chain into
the tail.  This would free the IOBuf head, which was owned by an
external caller.

A new RWCursor::gatherAtMost() API is provided for callers that wish to
gather as much as requested if possible, but still succeed if less than
this is available.

Test Plan:
Updated the unit tests to test calling gather() with more bytes than
actually available.

Reviewed By: davejwatson@fb.com

FB internal diff: D1081995