Summary:
Fix the fbzmq pathmap listed in its manifest file to match the path map
actually used by ShipIt.  The fact that this was broken was why internal
getdeps builds did not detect that D16577367 broke the build.

Reviewed By: wez, jstrizich

Differential Revision: D17005360

fbshipit-source-id: 046ff58ad70c03b860c3fccebaba975808df244d

Summary:
Update ThriftCppLibrary.cmake to record that the output depends on the thrift
`cpp2` template files.  The `THRIFT_TEMPLATE_FILES` variable was defined in
D16062657 but doesn't appear to be used anywhere.  This uses it, and also
updates it to only match the C++ template files.

Reviewed By: wez

Differential Revision: D16738442

fbshipit-source-id: aa4097e9ec1d009ef1e3ee2efd03ffa19a82ed12

Summary:
Add explicit double quotes around arguments that are intended to be exactly a
single parameter.  Also line wrap to 80 characters.

Reviewed By: wez

Differential Revision: D16738443

fbshipit-source-id: b8c9b6d50da72b44e23eaf5effbc384b6cbbc1c0

Summary:
Change ThriftCppLibrary to use `fb_cmake_parse_args()`

This code doesn't really care much about handling empty arguments, so we could
use the standard `cmake_parse_arguments()` function here instead of
`fb_cmake_parse_args()`.  However, it is slightly more convenient that
`fb_cmake_parse_args()` automatically errors out on unhandled arguments, so
that we don't have to check for this ourselves.

Reviewed By: wez

Differential Revision: D16738445

fbshipit-source-id: 67b9761f2ceaf7665bd336290bb44af32fec6f4a

Summary:
The manifest file for iproute2 previously listed the commit to check out as
7ca63aef7d1b0c808da0040c6b366ef7a61f38c1.  However, this is not a commit, but
a tag object that refers to commit 92a0236a3cdf3438000834121b7ea8a09f1f52b1.

This caused getdeps to think that iproute2 was always out-of-date, even when
it was already on the correct commit, since it was comparing current commit ID
against the tag ID.

This updates the manifest to refer to the commit ID instead of the tag ID.

Reviewed By: shri-khare

Differential Revision: D17005418

fbshipit-source-id: c7145c73b66ae93a2e9097ddcf8c53237b135d32

Summary: Enable FOLLY_TLS on FOLLY_MOBILE too

Reviewed By: guangyfb

Differential Revision: D16883165

fbshipit-source-id: 1a8f0991088f6963dc2acf84b74b294be998b877

Summary: [Folly] Avoid `__forceinline` for `catch_exception` under MSVC, which warns about functions which contain `try`-and-`catch` blcoks having the `__forceinline` modifier.

Reviewed By: akrieger

Differential Revision: D16982707

fbshipit-source-id: 68c443c4ce8ca77edd49d509a88ef4b0ab9488a9

Summary:
The definition of RLockedPtr should use a non-const Subclass type, and not a
const one.

Reviewed By: ot

Differential Revision: D15356827

fbshipit-source-id: b8ad41e263f0e15ffa25b0698aa85eab8ca2ccb8

Summary: This moves within to be in a more logical spot. In the parent diff we kept it where it was to make it easier to see the differences.

Reviewed By: LeeHowes

Differential Revision: D16967106

fbshipit-source-id: 9f9cd2ea38a996ac12d3ca8256c1ec050d57dfc0

Summary:
This fixes a deadlock bug where the after continuation runs inline.

We resolve this by using defer to delay execution until both futures are created thus avoiding any extra synchronization.

Reviewed By: LeeHowes

Differential Revision: D16952748

fbshipit-source-id: a88a40d3135bdabc43bb51f9b5ea072f8e6564a3

Summary:
`ConcurrentHashMap::insert_or_assign` does not follow the interface for `std::map::insert_or_assign`.  Making this clear for future users to avoid confusion.

Code:
```
int main(int argc, char** argv) {
  folly::ConcurrentHashMap<int, std::string> map;
  std::cout << "Insertion took place?: " << map.insert_or_assign(1, "hey").second << std::endl;
  std::cout << "Insertion took place?: "
<< map.insert_or_assign(0, "wow").second << std::endl;
  std::cout << "Insertion took place?: "
<< map.insert_or_assign(1, "wow").second << std::endl;
}
```

Output:
```
Insertion took place?: 1
Insertion took place?: 1
Insertion took place?: 1
```

Also see: https://github.com/facebook/folly/blob/master/folly/concurrency/test/ConcurrentHashMapTest.cpp#L86-L87

Reviewed By: magedm

Differential Revision: D16967939

fbshipit-source-id: ce0d32813371b31a94e14e404665d4fb2e7ffdfe

Summary:
Adds folly::coro::sleep() function as an alternative to using folly::futures::sleep().

The coro version returns a Task<void> and supports cancellation of the sleep operation when awaited within another Task.

Reviewed By: kirkshoop

Differential Revision: D16816328

fbshipit-source-id: 46bc4ee2475e6bd0bdfd0f7f30f3e0f1ea54d4d5

Summary:
The `folly::coro::Task` coroutine type now has an associated `CancellationToken` that is implicitly passed down to child operations that it awaits.

This is a first step towards supporting cancellation of persistent async streams.

This adds a new `co_withCancellation()` customisation point that awaitable types can customise to allow them to opt-in to cancellation. Currently only `Task` customises this operation.

Also provided a new `co_current_cancellation_token` object that can be awaited within a `Task` to retrieve the current `CancellationToken`.

Note that I have not yet hooked up the `Future` or `SemiFuture` to integrate with this cancellation mechanism (most `Future`-based code is not cancellation-aware). So a coroutine that is currently suspended awaiting for a `Future` may not currently respond to a cancellation request.

Reviewed By: andriigrynenko

Differential Revision: D16610810

fbshipit-source-id: 72a31d7a3ba4c281db54c1942ab41d4ea2f34d21

Summary: `LockFlags` is a set of flags that mirrors `mlock2(..., int flags)`, but since this is not a C API we don't need a bitset, we can just use a more idiomatic struct of bools (also used for the other options in the class). Furthermore, having an entry with value 0 (`LOCK_PREFAULT`) does not make sense in a bitset.

Reviewed By: yfeldblum

Differential Revision: D16900127

fbshipit-source-id: 6477854121c0626e0f36cb7a50d50df73323bd4e

Summary:
This simplifies things for the OSS CI situation in
the short term, especially since Eden isn't quite buildable
or usable outside FB at this time.

Reviewed By: chadaustin

Differential Revision: D16953346

fbshipit-source-id: 028150bb3cfc9a8ccbe1197321214b56ad359463

Summary: [Folly] Drop workaround for clang-3.6 in `Range` ctor.

Reviewed By: gkmhub, Orvid

Differential Revision: D16945307

fbshipit-source-id: d70d7560375d0c020b19fbac227767605459a834

Summary:
bzero is deprecated and can be compile time disabled for uClibc-ng
Pull Request resolved: https://github.com/facebook/folly/pull/1195

Reviewed By: Orvid

Differential Revision: D16451837

Pulled By: yfeldblum

fbshipit-source-id: 929a8e8cd7f8ddb3a904c1b1956a5f8ca115244f

Summary:
Simplified the interface for consuming an AsyncGenerator to now just have a single async next() method instead of an iterator-based API.

The iterator-based API was added to integrate with the 'for co_await' syntax present in the Coroutines TS, however this syntax has been removed from C++20 to allow for future exploration of the AsyncRanges design-space.

The `co_await gen.next()` expression produces an optional-like object that lets you query whether the result is a sentinel or contains a value.

Old:
```
Task<void> consume(AsyncGenerator<T> gen) {
  for (auto it = co_await gen.begin();
       it != gen.end();
       co_await ++it) {
    use(*it);
  }
}
```

New:
```
Task<void> consume(AsyncGenerator<T> gen) {
  while (auto item = co_await gen.next()) {
    use(*item);
  }
}
```

Reviewed By: andriigrynenko, kirkshoop

Differential Revision: D16586151

fbshipit-source-id: 4b0bf31ba9291d894a18e9553513eddee0cde33a

Reviewed By: Orvid

Differential Revision: D16640245

fbshipit-source-id: 3a184e4d877ea27a9400de5993b494095f6bd5e5

Summary:
does not affect the github build.
this is for fb CI

Reviewed By: dddmello

Differential Revision: D16935982

fbshipit-source-id: 61656fa600f19438b5aa89486486ed4ee103dea4

Summary:
folly::Hash did not work with pointer-like types like std::unique_ptr<T>,
std::shared_ptr<T> and T*.  Add specializations for those

Reviewed By: yfeldblum

Differential Revision: D16134842

fbshipit-source-id: 78d3cf9c6f5ef4f49b008c0d1c489d3c6203ba89

Summary: As far as I can tell, folly/test/FBVectorTestUtil.h does not need to include folly/Benchmark.h. Move it to folly/testFBVectorBenchmark where it is actually used.

Reviewed By: yfeldblum

Differential Revision: D16931137

fbshipit-source-id: 8ef5f99a783537672d55883adadd08270e318a61

Summary: Fix : iterating through ConcurrentHashMapSIMD results in infinite loop

Reviewed By: nbronson

Differential Revision: D16923826

fbshipit-source-id: c8b98ef13034240c162c155cf0e5c3f097b73c7f

Summary: Underflow/overflow of signed integers is fraught with dangers. So this diff removes the potential for overflow while adding or subtracting buckets in folly stats.

Reviewed By: yfeldblum

Differential Revision: D16898755

fbshipit-source-id: 4d3df1024b4c01ea0b276278105071e62e27fa27

Summary:
In the Facebook infrastructure the testpilot runner talks
to a service to determine whether a given test should be run locally.

The remote service was returning an error for some tests because they
didn't have enough configuration specified.

This diff associates some configuration with the tests based on the
host type and makes things happier.

Reviewed By: Ben0mega

Differential Revision: D16894682

fbshipit-source-id: b6f59b112991160ef26cc126e98041d8908a6273

Summary:
Sized deallocation via ::operator delete with size == 0 is
broken in some allocators.  This diff adds a unit test that ensures we
are either disabling sized deallocator or running with a good allocator.

Reviewed By: interwq

Differential Revision: D10857654

fbshipit-source-id: 981c7bcc1b021d4cd1ea1725ccf510d1958842fd

Reviewed By: pixelb

Differential Revision: D15301144

fbshipit-source-id: f94a805427b6fc8b20e7c01fcf6e1853fbebf7f1

Summary: Make `AsyncServerSocket` not fail on `EINPROGRESS` in `bind` and fix handling of `errno` there.

Reviewed By: yfeldblum

Differential Revision: D16791566

fbshipit-source-id: 1aeaefb6bae47bf41fd2d20238b716b84d0566d1

Summary: T52681488

Reviewed By: Orvid

Differential Revision: D16901429

fbshipit-source-id: 0a5fc218edf6ef29b100365163102dbdc2abf313

Summary:
[Folly] Suppress a spurious MSVC warning in `folly/Traits.h`.

```
folly\traits.h(645): warning C4287: '>': unsigned/negative constant mismatch
folly\traits.h(646): warning C4287: '<=': unsigned/negative constant mismatch
folly\traits.h(647): warning C4287: '<': unsigned/negative constant mismatch
```

Reviewed By: Orvid

Differential Revision: D16889045

fbshipit-source-id: 31edd58bf046143092a5f006ee0ac5b071b539a0

Summary:
[Folly] Fix MSVC conversion warning in `Optional` test.

```
folly\optional.h(372): warning C4244: 'argument': conversion from 'double' to 'int', possible loss of data
folly\optional.h(358): note: see reference to function template instantiation 'void folly::Optional<Type>::construct<std::initializer_list<_Ty>&,double>(std::initializer_list<_Ty> &,double &&)' being compiled
          with
          [
              _Ty=int
          ]
folly\optional.h(359): note: see reference to function template instantiation 'void folly::Optional<Type>::construct<std::initializer_list<_Ty>&,double>(std::initializer_list<_Ty> &,double &&)' being compiled
          with
          [
              _Ty=int
          ]
folly\optional.h(449): note: see reference to function template instantiation 'folly::Optional<Type>::Optional<std::initializer_list<_Ty>&,double>(folly::Optional<Type>::PrivateConstructor,std::initializer_list<_Ty> &,double &&) noexcept' being compiled
          with
          [
              _Ty=int
          ]
folly\optional.h(451): note: see reference to function template instantiation 'folly::Optional<Type>::Optional<std::initializer_list<_Ty>&,double>(folly::Optional<Type>::PrivateConstructor,std::initializer_list<_Ty> &,double &&) noexcept' being compiled
          with
          [
              _Ty=int
          ]
folly\test\optionaltest.cpp(659): note: see reference to function template instantiation 'folly::Optional<Type> folly::make_optional<Type,int,double>(std::initializer_list<_Ty>,double &&)' being compiled
          with
          [
              _Ty=int
          ]
```

Reviewed By: Orvid

Differential Revision: D16889085

fbshipit-source-id: c59f5e3fa0d4402e1b836841dff66d73b111235f

Summary:
Building a project that uses folly and `target_compile_features` may fail
because folly requires C++14 and CMake doesn't know about it. So if a project
has only C++11 features in `target_compile_features` CMake will pick
`-std=gnu++11` or similar and compilation will fail on an assertion in
`folly/Portability.h`. To fix this, add `target_compile_features` to the
`folly` target pushing the minimum required standard to C++14.

Reviewed By: Orvid

Differential Revision: D16884888

fbshipit-source-id: 0e5ac4abde10b1ea757011bcd05b17d508859b4f

Summary:
Add some CMake functions for building standalone executables from Python
source files.  This generates executables similar to PEX
(https://github.com/pantsbuild/pex).

In the future this could potentially be leveraged to directly build XAR files
(https://github.com/facebookincubator/xar).

The main advantages of these functions is that they allow easily defining
"libraries" of python files, and their dependencies, which can then be used
and packaged as part of multiple different standalone executables.

Reviewed By: wez

Differential Revision: D16722499

fbshipit-source-id: e1d829b911dc428e5438b5cf9cebf99b3fb6ce24

Summary: Avoid SIGSEGV if getIOExecutor() returns null

Reviewed By: yfeldblum

Differential Revision: D16859157

fbshipit-source-id: 00c7a45677c61d84a10486f11b4f792093ac2ff5

Summary:
It's spammy and is just informational.

Created from Diffusion's 'Open in Editor' feature.

Reviewed By: yfeldblum

Differential Revision: D16886741

fbshipit-source-id: 29aecb8a62cac9a60cbc4cabc596fd1ee00cfc04

Summary:
[Folly] Cut IsRelocatable markings of containers, removing forward declarations of the containers to avoid over-inclusion and to avoid undefined behavior.

> 16.5.4.2.1 [namespace.std]\1
> Unless otherwise specified, the behavior of a C++ program is undefined if it adds declarations or definitions to namespace std or to a namespace within namespace std.

Closes #1203.
Closes #1204.

Reviewed By: igorsugak

Differential Revision: D16855473

fbshipit-source-id: 0e6acc4f589707aa79b0001b415bcadfab652d62

Summary: [Folly] Revise `atomic_fetch_set`, `atomic_fetch_reset` comments, enforcing structure and switching comment style.

Reviewed By: aary

Differential Revision: D16877442

fbshipit-source-id: 0c4c163f2daa14616f20c4e0cac1002c83ec5fac

Summary:
http://man7.org/linux/man-pages/man2/mlock.2.html

       MLOCK_ONFAULT
              Lock pages that are currently resident and mark the entire
              range so that the remaining nonresident pages are locked when
              they are populated by a page fault.

The kernel still checks a few things for `MLOCK_ONFAULT`: rlimits RLIMIT_MEMLOCK / CAP_IPC_LOCK => EPERM or ENOMEM

`TRY_LOCK` & `MUST_LOCK` are only different in in their handling of EPERM, ENOMEN: whether to fail or ignore the errors, so I added a separate enum.

https://github.com/torvalds/linux/blob/v5.0/mm/mlock.c#L29-L37
```
bool can_do_mlock(void)
{
	if (rlimit(RLIMIT_MEMLOCK) != 0)
		return true;
	if (capable(CAP_IPC_LOCK))
		return true;
	return false;
}
EXPORT_SYMBOL(can_do_mlock);
```

https://github.com/torvalds/linux/blob/v5.0/mm/mlock.c#L671-L714
```
static __must_check int do_mlock(unsigned long start, size_t len, vm_flags_t flags)
{
	unsigned long locked;
	unsigned long lock_limit;
	int error = -ENOMEM;

	if (!can_do_mlock())
		return -EPERM;

	len = PAGE_ALIGN(len + (offset_in_page(start)));
	start &= PAGE_MASK;

	lock_limit = rlimit(RLIMIT_MEMLOCK);
	lock_limit >>= PAGE_SHIFT;
	locked = len >> PAGE_SHIFT;

	if (down_write_killable(&current->mm->mmap_sem))
		return -EINTR;

	locked += current->mm->locked_vm;
	if ((locked > lock_limit) && (!capable(CAP_IPC_LOCK))) {
		/*
		 * It is possible that the regions requested intersect with
		 * previously mlocked areas, that part area in "mm->locked_vm"
		 * should not be counted to new mlock increment count. So check
		 * and adjust locked count if necessary.
		 */
		locked -= count_mm_mlocked_page_nr(current->mm,
				start, len);
	}

	/* check against resource limits */
	if ((locked <= lock_limit) || capable(CAP_IPC_LOCK))
		error = apply_vma_lock_flags(start, len, flags);

	up_write(&current->mm->mmap_sem);
	if (error)
		return error;

	error = __mm_populate(start, len, 0);
	if (error)
		return __mlock_posix_error_return(error);
	return 0;
}
```

https://github.com/torvalds/linux/blob/v5.0/mm/mlock.c#L721-L732
```
SYSCALL_DEFINE3(mlock2, unsigned long, start, size_t, len, int, flags)
{
	vm_flags_t vm_flags = VM_LOCKED;

	if (flags & ~MLOCK_ONFAULT)
		return -EINVAL;

	if (flags & MLOCK_ONFAULT)
		vm_flags |= VM_LOCKONFAULT;

	return do_mlock(start, len, vm_flags);
}

```

Reviewed By: yfeldblum

Differential Revision: D16760748

fbshipit-source-id: 54937046f5faaceac2cfb9dd6a0f207ad653fe6b

Summary:
Currently the timekeeper callback set by the within method isn't cleaned up when the future resolves and always waits the full duration. This can cause OOM under high QPS and long timeout situations.

This resolves this by cancelling the callback.

Reviewed By: mpark

Differential Revision: D16835816

fbshipit-source-id: c400afc37dbc66447aaa8fd557a770d717d9ab02

Summary:
RocksDB's TSAN version was not able to instrument atomic_fetch_set and
atomic_fetch_reset; and as a result the DistributedMutex tests were failing
under TSAN

Also TSAN seems to have some issue when and exception_ptr is placed into an
aligned_storage_t instance. This verifiably correct program (I think) also fails
with the same TSAN error P78696506. So
disable that test

This is the code with the same false-negative
```
namespace {
class ExceptionWithConstructionTrack : public std::exception {
 public:
  explicit ExceptionWithConstructionTrack(int id)
      : id_{folly::to<std::string>(id)}, constructionTrack_{id} {}

  const char* what() const noexcept override {
    return id_.c_str();
  }

 private:
  std::string id_;
  TestConstruction constructionTrack_;
};

template <typename Storage, typename Atomic>
void transferCurrentException(Storage& storage, Atomic& produced) {
  assert(std::current_exception());
  new (&storage) std::exception_ptr(std::current_exception());
  produced->store(true, std::memory_order_release);
}

void concurrentExceptionPropagationStress(
    int numThreads,
    std::chrono::milliseconds milliseconds) {
  auto&& stop = std::atomic<bool>{false};
  auto&& exceptions = std::vector<std::aligned_storage<48, 8>::type>{};
  auto&& produced = std::vector<std::unique_ptr<std::atomic<bool>>>{};
  auto&& consumed = std::vector<std::unique_ptr<std::atomic<bool>>>{};
  auto&& consumers = std::vector<std::thread>{};
  for (auto i = 0; i < numThreads; ++i) {
    produced.emplace_back(new std::atomic<bool>{false});
    consumed.emplace_back(new std::atomic<bool>{false});
    exceptions.push_back({});
  }

  auto producer = std::thread{[&]() {
    auto counter = std::vector<int>(numThreads, 0);
    for (auto i = 0; true; i = ((i + 1) % numThreads)) {
      try {
        throw ExceptionWithConstructionTrack{counter.at(i)++};
      } catch (...) {
        transferCurrentException(exceptions.at(i), produced.at(i));
      }

      while (!consumed.at(i)->load(std::memory_order_acquire)) {
        if (stop.load(std::memory_order_acquire)) {
          return;
        }
      }

      consumed.at(i)->store(false, std::memory_order_release);
    }
  }};

  for (auto i = 0; i < numThreads; ++i) {
    consumers.emplace_back([&, i]() {
      auto counter = 0;
      while (true) {
        while (!produced.at(i)->load(std::memory_order_acquire)) {
          if (stop.load(std::memory_order_acquire)) {
            return;
          }
        }
        produced.at(i)->store(false, std::memory_order_release);

        try {
          auto storage = &exceptions.at(i);
          auto exc = folly::launder(
            reinterpret_cast<std::exception_ptr*>(storage));
          auto copy = std::move(*exc);
          exc->std::exception_ptr::~exception_ptr();
          std::rethrow_exception(std::move(copy));
        } catch (std::exception& exc) {
          auto value = std::stoi(exc.what());
          EXPECT_EQ(value, counter++);
        }

        consumed.at(i)->store(true, std::memory_order_release);
      }
    });
  }

  std::this_thread::sleep_for(milliseconds);
  stop.store(true);
  producer.join();
  for (auto& thread : consumers) {
    thread.join();
  }
}
} // namespace
```

Reviewed By: yfeldblum

Differential Revision: D16739396

fbshipit-source-id: 0eba191ffc07d2e64a3ea550c5fe9a7d01ce2f18