Summary:
Fixes compilation on non x86.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Pull Request resolved: https://github.com/facebook/folly/pull/1202

Differential Revision: D16760427

Pulled By: yfeldblum

fbshipit-source-id: 6f31d5a17b9fe9c1f91d884eb94967ba06756f3e

Summary:
This diff fixes the initial conditions for SwapTrackingAlloc,
which fixes F14 tests on platforms that do not have intrinsics enabled.

Differential Revision: D16755857

fbshipit-source-id: a76d8093100cbc0c5ce8723daa0976673d942299

Summary:
This diff extracts the key destructuring logic in F14's emplace
functions into standalone helpers that can be used by other container
types (like sorted_vector_{set,map}).  It also adds better PMR compliance, by
using a stateful allocator to construct a temporary if that is required
to be able to perform the search.

This diff also cleans up some of the testing code in folly/container/test
that had issues if included in multiple places.

Reviewed By: yfeldblum

Differential Revision: D16655131

fbshipit-source-id: 5a6f57ac346d1fd5928e0f737110760b0c518974

Summary:
In response to review feedback for D16477400 and D16477401, update
`ManifestLoader.load_all_manifests()` to only update its data for projects
that have not previously been loaded.  This helps ensure that code using a
single `ManifestLoader` object cannot have two in-memory `Manifest` objects
for the same project, and that existing data (such as project hashes) can't be
invalidated if a manifest is later loaded from updated on-disk data.

Reviewed By: pkaush

Differential Revision: D16586682

fbshipit-source-id: 50b1979ec55f2ad6901629cd852293a8f6ca903f

Summary:
This is a missing piece from https://github.com/facebook/folly/pull/1178, but more nasty as it can cause runtime rather than compile-time failures.

Given an interface `ICat` with `void meow() const`, if we have `struct cat` with `void meow() const noexcept`, then constructing a `folly::Poly<ICat const&>` for an instance of `cat` will have a null pointer in the vtable entry for the `meow` method.

The culprit seems to be in the template specialization for [`ThunkFn` ](https://github.com/cstratopoulos/folly/blob/feature/poly-update/folly/detail/PolyDetail.h#L539) which may be dispatched on `IsConstMember`.

I've added a fairly trivial test case to `PolyTest.cpp`; but notably it is the only instance of _any_ test case in that file having a `noexcept` method. Without the change in `PolyDetail.h`, the call to `cref->meow()` will cause a null pointer dereference. If something more explicit is desired, maybe we could `EXPECT` that the corresponding vtable entry is non-null, but that doesn't exactly fit the pattern of other test cases in that file.

(Aside: In the linked PR I used `__cpp_noexcept_function_type` directly, but I have since realized that a macro for this is already present in [`Portability.h`](https://github.com/facebook/folly/blob/master/folly/Portability.h#L503) so I'm using that one now)
Pull Request resolved: https://github.com/facebook/folly/pull/1201

Reviewed By: ericniebler

Differential Revision: D16686139

Pulled By: yfeldblum

fbshipit-source-id: 73c7ac132d6e772ffd49f450b5fcd0f8a0dbdb26

Summary:
Added boundary support in folly/io/Cursor.h.

The bounded Cursor can have a boundary that is shorter than the underlying IOBuf. This gives a custom view of the IOBuf.

Reviewed By: spalamarchuk

Differential Revision: D16347838

fbshipit-source-id: 16d91e42e2acee788b31bc2a9b6cc8716b1d22f2

Summary: Switch defer to use inline continuations by default, if bound executors match.

Reviewed By: andriigrynenko

Differential Revision: D16643666

fbshipit-source-id: d3cdf86761c378d7ca138b46322c55a9755bdc48

Summary: As both coro.h and futures.h defines the function getExecutor, redefinition error occurs when both of them are included. This change moves the getExecutor inline function to executor.h so it can be shared.

Reviewed By: yfeldblum

Differential Revision: D16676456

fbshipit-source-id: 33f7558a3d5dbc996c4efef4fe90918de32ad8b2

Summary:
When inserting an item with a hint, the item will be always copied, even
if it is already present in the map. This could cause an exception to be
thrown when it doesn't need to be. The standard doesn't require this, but
it is an optimization anyways.

Reviewed By: yfeldblum

Differential Revision: D16585971

fbshipit-source-id: f0a8eeb0bd86f07ef110f91cbef8ab422ff55480

Summary:
The current folly ExceptionTracer only keeps the stack traces in thread local storage for as long as the exception is currently active or caught. When using folly futures we almost never handle exceptions within a catch block or even the same thread with thenError.

With C++ 11, we have exception_ptr which ref counts the exception so being able to keep the stack trace as long as something still holds a pointer to the exception is very useful. This allows us to get a stack trace for an exception returned by futures.

This tracer adds a callback that runs when an exception is thrown and captures the stack trace in a synchronized map. To manage the lifetime, we override the deleter function with our own wrapper that will remove the trace from the map when the exception is no longer needed.

Two things of note:
* We're attempting to allocate memory in the exception thrown callback so if we run out of memory this code won't be able to handle it and trigger a terminate via the noexcept.
* Every exception thrown requires acquiring a lock. We should be able to mitigate this via folly::ConcurrentHashMap if it becomes a problem. Initial benchmarking seems fine as is. I don't believe it'll become a problem since we only throw exceptions in exceptional cases and they shouldn't be performance sensitive.

Reviewed By: yfeldblum

Differential Revision: D16533296

fbshipit-source-id: 7d8832c24c79bde98d1bb213d4b70f259fb3d4bc

Summary: We can't use futures::WaitExecutor here, because it may drop some work if detached, which is not supported by coro::Task.

Reviewed By: lewissbaker

Differential Revision: D16664921

fbshipit-source-id: 9797b2c59af156ec96e64244d5810a5924cfb3cc

Summary: More broadly compatible implementation of variant in future core using a boost variant.

Reviewed By: andriigrynenko

Differential Revision: D16676402

fbshipit-source-id: 19784ab1815e81b9afc49e46b5425e55111cd530

Summary: Add PicoSpinLock TSAN annotations

Reviewed By: yfeldblum

Differential Revision: D16653917

fbshipit-source-id: d388a31824f1278fe79ab4a42e21a66781b879df

Summary: [Folly] Cut enforcement of copy-ctor counts in sorted-vector sets/maps tests, since whether the copy-ctor is elidable, and therefore the actual values of the counts, depends on the compiler and on the optimizations in use.

Reviewed By: nbronson

Differential Revision: D16638747

fbshipit-source-id: b7a03a46afc065bd93595026df1aab46c9e1295e

To reduce risk of stack overflow when catching up with deferred work, make blocking operation runOnMainContext

Summary: Shift wait execution off of a (potential) fiber stack to reduce risk of stack overflows while running deferred work.

Reviewed By: andriigrynenko

Differential Revision: D16643654

fbshipit-source-id: 3ab1fe762b8f93c011cdb926e7689281018f54bc

Reviewed By: yfeldblum

Differential Revision: D16639497

fbshipit-source-id: ddefeb12e2f83bc86e410092233efb2cdeaacbd0

Summary: This can be useful to avoid the cost of re-throwing exceptions.

Reviewed By: yfeldblum, LeeHowes

Differential Revision: D16628722

fbshipit-source-id: 434842d99369d0293c4eeb894e6feac15ec16173

Summary: Defer uses inline continuations by default, if bound executors match.

Reviewed By: andriigrynenko

Differential Revision: D15300706

fbshipit-source-id: fc17c70ed9956442007a1cc2f6a082f75c555ad4

Summary:
After this diff, DeferredExecutor participates consistently in executor inline behaviour by being special cased in the core.

DeferredExecutor is no longer an executor, and is hence no longer special cased in the Future code. This is replaced with a variant of DeferredExecutor and Executor in Core.

Reviewed By: yfeldblum, andriigrynenko

Differential Revision: D15836529

fbshipit-source-id: 8324ba1de57e85fc757ecc3b431bf71858868a0d

Summary: Simple code reorg. DeferredExecutor class moved into Core as it is later becoming functionality of Core rather than of the Future.

Reviewed By: andriigrynenko

Differential Revision: D15836530

fbshipit-source-id: 5a14a6c9332666b275e39796cae5f32c96edacd5

Summary: addFrom on DeferredExecutor allows the DeferredExecutor to know what executor is running the work enqueuing to it, which it can check against its internal state to run the work inline if the executors match.

Reviewed By: andriigrynenko

Differential Revision: D15836532

fbshipit-source-id: 40c35d976b90072c88bb544360b059c1b151d4d0

Summary:
Allows adding directly to a KeepAlive if the KeepAlive is consumed in the process.

This allows simultaneous consumption and propagation of the KeepAlive into the callback, propagating information about the executor running the callback without the cost of reference counting the KeepAlive.

Reviewed By: andriigrynenko

Differential Revision: D15836528

fbshipit-source-id: 8f9e6f6ec47ad294391741d25fce68a85a429919

Summary: [Folly] `cacheline_align_t`, `cacheline_align_v` - since it is theoretically possible on some platforms for `alignas(hardware_destructive_interference_size)` to be a compile-time error.

Reviewed By: aary

Differential Revision: D16598497

fbshipit-source-id: d0dfe99b726cec5dba96e26147e73e494e62a895

Summary:
[Folly] Add unsafe-unlocked getter to `Synchronized`.

Provided as a backdoor for call-sites where it is known safe to be used. For example, when it is known that only one thread has access to the `Synchronized` instance.

To be used with care - this method explicitly overrides the normal safety guarantees provided by the rest of the `Synchronized` API.

Reviewed By: chadaustin, simpkins

Differential Revision: D8178205

fbshipit-source-id: 648ee392a43a06333452f72d992ec42ba4a1a73c

Summary: [Folly] Fix visibility attributes that gcc objects to in xlog by moving them from lambdas to proper functions.

Reviewed By: nbronson

Differential Revision: D16578895

fbshipit-source-id: 021fd10ef01bb610dc6f20d9a9a98a5a8e11df71

Summary: The TSX intrinsics should only be compiled and used when compilers and platforms support them. This diff wraps the intrinsics and adds checks on TSX support to server this purpose.

Reviewed By: nbronson

Differential Revision: D16585187

fbshipit-source-id: 6490c0da7f6221caaa4529bfcebb261cfd99cc7d

Summary:
The Python chained `IOBuf` has undesirable behavior on cyclic pattern data. For example, when we have a data chain with
```
chain = make_chain([IOBuf(b"aaa"), IOBuf(b"aaaa")])
```
`b"".join(chain)` will yield `b"aaa"` rather than `b"aaaaaaa"`.

The root cause to this bug is because in the `__iter__` method of the Python `IOBuf`, the original code checks whether the circular chain has been traversed by the `!=` operator (`self != next`), which has been overridden by `__richcmp__` function. The rich comparator then invokes the comparator in C++, which compares the underlying data in a `IOBuf` chain rather than their reference locations. In the above example, therefore, `chain == chain.next` would return `True`. However, in `__iter__`, in order to check whether the traversal is back to the head of the chain, we should compare by reference rather value.

Reviewed By: yfeldblum

Differential Revision: D16589600

fbshipit-source-id: 3b03c4d502bdc385edca3502949be03440543a21

Summary: [Folly] Extract Executor::KeepAlive bitmask constants to a non-template base class so that they can be used from elsewhere.

Reviewed By: LeeHowes

Differential Revision: D16617648

fbshipit-source-id: eb3ceddfa8c492bb4ddd7de0046bb5c2c6b4e762

Reviewed By: yfeldblum

Differential Revision: D16594071

fbshipit-source-id: 78755bc64f22b28f89958064184c1fdf35b1bb85

Summary: needed for openr

Reviewed By: simpkins

Differential Revision: D16010070

fbshipit-source-id: 6d485fa7e4e321e6cd23d9894b38c1ecc7574665

Summary: Don't constanly rebuild exceptions that are always the same.

Reviewed By: jrahman

Differential Revision: D16016383

fbshipit-source-id: b0b24ed39e6c41f95c61369c90a3ac83abfd1a66

Summary:
While developing on a project it is often convenient to be able to invoke its
build manually, rather than always needing to re-run `getdeps.py`.  This
updates the CMakeBuilder to also emit a script that can be used to manually
run CMake outside of `getdeps.py`.

The CMakeBuilder is the only builder that this really matters for right now,
as pretty much all of the projects where we do first-party development use
CMake for their build system.

Reviewed By: pkaush

Differential Revision: D16477399

fbshipit-source-id: c8a14af158af7b32d6c799ef685b037e68b748ff

Summary:
Move code that computes project hashes to ManifestLoader.  ManifestLoader is
the only class that has all of the information necessary to compute the
project hashes correctly.  The ManifestLoader object can also cache previously
computed hashes, so that we don't have to keep computing hashes for projects
over and over again.  Previously the `BuildOptions.compute_dirs()` function
would end up re-computing hashes for all dependencies each time it was called.

Reviewed By: strager

Differential Revision: D16477401

fbshipit-source-id: ce03642114f91ce4f859f612e6b2e747cf1653be

Summary:
The ManifestLoader contains all of the state needed to create a fetcher
object, so define a helper method on this object to create a fetcher.

Reviewed By: strager

Differential Revision: D16477395

fbshipit-source-id: 6de0942fe6b8de26c18c82bf99343f5467dc006a

Summary:
Add a new ManifestLoader class to handle loading manifests and computing
dependencies.

For now the main thing this class does is maintain the `manifest_by_name`
mapping.  In subsequent diffs we should be able to move some additional logic
into this class, which will help clean up the code and eliminate some redudant
work.  In particular, we can have this class cache project hashes, which will
avoid re-computing hashes over and over again for the same projects as we do
in many cases today.  We should also be able to save and re-use some of the
project dependency ordering computation in some cases as well.

Reviewed By: strager

Differential Revision: D16477400

fbshipit-source-id: f06f62f77d8443fccaa69fe4c1306e39c395b325

Summary:
Update `BuildOpts.compute_dirs()` to use the correct project-specific manifest
context when computing project hashes.  Previously it was incorrectly using
the initial project's context when evaluating all dependencies.  This would
result in some projects potentially seeing the wrong values for variables that
may change from project to project (like `test`).

Reviewed By: pkaush

Differential Revision: D16477398

fbshipit-source-id: 6c23f5e5e19b2402000a138b3920b79044446041

Summary:
Add a ContextGenerator class so that we actually use the correct per-project
context when loading projects and computing dependencies.

Previously commands like `build` and `test` would change the contexts for each
project as they iterated through and performed the build.  However, they did
not do this when first loading the projects.  This could cause them to use
different context values when loading dependencies than when performing the
build.  For instance, this could cause issues if a project depends on
`googletest` only when testing is enabled, as the code previously did not set
the "test" parameter when evaluating dependencies.

Reviewed By: pkaush

Differential Revision: D16477396

fbshipit-source-id: c1e055f07de1cb960861d19594e3bda20a2ccd87

Summary:
Check that all variable names are valid when loading manifest files.
This ensures that getdeps.py will error out if someone makes a typo in a
variable name, rather than treating it as never equal to anything.

Reviewed By: pkaush

Differential Revision: D16477397

fbshipit-source-id: 030e0642ff4a08db8eb74a0a0223e03d53e4880f

Summary:
This allows us to use a custom deleter to track the lifetime of exceptions in
the case of smart pointers which may be persisted outside catch blocks.

Reviewed By: yfeldblum

Differential Revision: D16572937

fbshipit-source-id: ad2f0a981a2b31ef4b17a902719b684f538c29c7

Reviewed By: JunqiWang

Differential Revision: D16497024

fbshipit-source-id: aa0e42b4bfba3c70769514131daa30df84851126