Summary:
The LZ4 Frame codec encodes data using the LZ4 frame format.
One advantage of the LZ4 frame format is that it has 4 magic bytes in the header, so users can transparently determine compression type.
It also allows the user to interop with the lz4 command line tool.

Reviewed By: yfeldblum

Differential Revision: D4715918

fbshipit-source-id: 689833fef526b1cfe98685179e7b494380d49cba

Summary:
Extract the environment-as-STL-map logic out of `test::EnvVarSaver` and into a
separate class `experimental::EnvironmentState` so that other code which needs
to manipulate the environment can do so more easily.

Add routines to set the process environment from the state of an
`EnvironmentState`'s map, and to extract the environment in the forms required
by `Subprocess` and the UNIX routines `execve` etc.

Reviewed By: yfeldblum

Differential Revision: D4713307

fbshipit-source-id: 6b1380dd29b9ba41c97b886814dd3eee91fc1c0f

Summary:
Looks like some tests for the `io` directory were left behind when they were moved out of experimental. Re-enabled them in the new location, except for the compression test because it takes a lot longer. It's still built and can be executed manually.

Also added some preprocessor guards around compression codecs that might not be compiled in (and therefore fail during test SetUp).

Depends on #547 because the path to gtest appears in the new Makefile.am's `CPPFLAGS`.
Closes https://github.com/facebook/folly/pull/550

Reviewed By: yfeldblum

Differential Revision: D4631244

Pulled By: Orvid

fbshipit-source-id: 496b2c272e4f7293822f159776a02f7dd0c9e04d

Reviewed By: yfeldblum, ot

Differential Revision: D4742262

fbshipit-source-id: 7137bb8e1cb39c74e8ba726e76e9a8a6f3e1984c

Randomize the Unix socket name in AsyncSocketTest.SendMessageAncillaryData test to avoid collisions.

Summary:
Our test framework reports frequent failures of AsyncSocketTest.SendMessageAncillaryData. According to the logs the socket fails to bind:

    folly/io/async/test/AsyncSocketTest2.cpp:3098: Failure
    Expected: (bind(lfd, (struct sockaddr*)&addr, sizeof(addr))) != (-1), actual: -1 vs -1
    Bind failed: 98

This diff adds the socket name randomization to avoid name collisions between tests running concurrently on the same test box.

Reviewed By: yfeldblum

Differential Revision: D4758942

fbshipit-source-id: 6066dbc18222a4521c40b2ff218cb7dab8bd789d

Summary:
Change every instance of EXPECT_EQ(false, ... to the simpler/shorter EXPECT_FALSE(...
Do the same for ASSERT_EQ, for consistency, and for true/TRUE.

Reviewed By: Orvid

Differential Revision: D4751397

fbshipit-source-id: 045293dbf8be1167f0cb1b85a2f9cd59a6a30f73

Summary:
Flat combining template that takes the following template parameters:
 T         Concurrent data structure using FC interface
 Mutex Mutex type (default std::mutex)
 Atom  Atomic template (default std::atomic)
 Req    Optional request structure to hold custom info (default dummy type bool)

Flat combining (FC) was introduced in the SPAA 2010 paper Flat Combining and the Synchronization-Parallelism Tradeoff, by Danny Hendler, Itai Incze, Nir Shavit, and Moran Tzafrir.
http://mcg.cs.tau.ac.il/projects/projects/flat-combining

Reviewed By: djwatson

Differential Revision: D4602402

fbshipit-source-id: 38327f752a3e92bb01e5496c321d8c87c818087a

Summary: The "l" in "Haskell" is not a singleton. Also, friends don't let friends rely on operator precedence.

Reviewed By: yfeldblum

Differential Revision: D4756653

fbshipit-source-id: 7b5360bfc1b26fac06062e4adbfae3a1e477c2dc

Summary: D4719112 changed the signature of the constructor but didn't update this callsite...

Reviewed By: djwatson

Differential Revision: D4754745

fbshipit-source-id: ae538a3ffb64cdc211f23220ea5e0d7bb7eff9b3

Summary: On Android, we should also use a relaxed singleton

Reviewed By: andriigrynenko

Differential Revision: D4754227

fbshipit-source-id: 157844e618d216b768bbb07f36c380ba2f4c9ab8

Summary: Sometimes tools need to output a little more help context before or after help.

Reviewed By: yfeldblum

Differential Revision: D4719112

fbshipit-source-id: e6f59312d722e0669053644d862c81758855b538

Summary: The gflags portability header should always be used in Folly. There were a couple of places gflags was being used directly. This fixes that.

Reviewed By: yfeldblum

Differential Revision: D4748674

fbshipit-source-id: ba3c0564a39f870f1f5f9371e32d06eb84591ba7

Summary:
sorted_vector_{set,map} can silently drop insert if the hinted
insertion position is too small.  This diff fixes it.

Reviewed By: yfeldblum

Differential Revision: D4747319

fbshipit-source-id: 31e399d07a0b77b700edf034dc723cb997dc8e16

Summary:
`LZ4Codec::doCompress()` and `doUncompress()` need to coalesce a `const IOBuf*`, so they `clone()` and then `coalesce()`.
It is more efficient to do this in one step, as shown by the benchmarks.

Reviewed By: yfeldblum, simpkins

Differential Revision: D4727802

fbshipit-source-id: f8d5f6187f2ee804301232b3c75b848330169fc5

Summary: Submitting mostly to see what breaks.

Reviewed By: yfeldblum

Differential Revision: D4249032

fbshipit-source-id: d4e3fdfd57750c8dcabdb4c01ab3b528c4818624

Summary: VS 2017 is here and the bugs these were hitting have been fixed, so re-enable them.

Reviewed By: yfeldblum

Differential Revision: D4742829

fbshipit-source-id: 898e7cfea9de9ffa813c562c12e0e18e0577c8c3

In AsyncSocketTest.SendMessageFlags test use folly::test::TemporaryFile object instead of directly creating a file with O_TMPFILE flag

Summary: O_TMPFILE flag is not available on older Linux systems (supported starting Linux 3.11). Got complains about it on GitHub (https://fburl.com/9d848m7i) Will use folly::test::TemporaryFile instead.

Reviewed By: yfeldblum

Differential Revision: D4729733

fbshipit-source-id: 903563825c5b939e35c61725da559e33d21212c4

Summary: This code looks really weird and is otherwise confusing.

Reviewed By: andriigrynenko

Differential Revision: D4730812

fbshipit-source-id: d9f4e73145e81f085ce959a76cdb568d3ae4eb52

Summary: Re-landing D4618623 after it had to be reverted due to it's use of `FixedString`, which had broken GCC 4.8. `FixedString` should now be compiling correctly on GCC 4.8 so this should be fine now.

Reviewed By: yfeldblum

Differential Revision: D4727022

fbshipit-source-id: f85c6952fd82e6cd6042f8055ef54d9422f8b0f9

Summary:
folly/python/futures.h provides some helper functions for bridging folly::future to asyncio.Future.

folly/python/NotificationQueueExecutor.h is a Driveable executor that has a fileno() method that can be monitored using (select, epoll) to determine if the drive method should be called.

folly/python/executor.pyx is an implementation of a "driver" for the NotificationQueueExecutor from the python asyncio side. It tracks also keeps track of asyncio eventloops to Executor mappings.

the getExecutor() from folly/python/futures.h uses that mapping to return the correct executor for this python thread.

Reviewed By: andriigrynenko, yfeldblum

Differential Revision: D4687029

fbshipit-source-id: e79314606ffa18cb6933fe6b749991bfea646cde

Summary:
Fix warnings exposed by gcc's -Wshadow-compatible-local.
Rename lambda parameter "ts" to "timeSeries".

This avoids the following errors:

  folly/test/TimeseriesTest.cpp:802:27: error: declaration of 'ts' shadows a previous local [-Werror=shadow-compatible-local]
  folly/test/TimeseriesTest.cpp:809:31: error: shadowed declaration is here [-Werror=shadow-compatible-local]

Reviewed By: yfeldblum, liuxuli

Differential Revision: D4735180

fbshipit-source-id: 59b3fecf4031b3c6615e5b764fbc4ea04754eb50

Summary: Use perfect forwarding in `EventBaseLocal::getOrCreate` to make it work with noncopyable but moveable arguments.

Reviewed By: yfeldblum

Differential Revision: D4730566

fbshipit-source-id: fa02348b7a9217fef980ec5e743b5990b9d19e9a

Summary: This is more in line with "traditional" output from cli tools on *nix.

Reviewed By: yfeldblum

Differential Revision: D4719055

fbshipit-source-id: 2389a4a2a49f18d4b3d0257f68036f73f8fbc49e

Summary:
Adds a default inline implementation for `std::hash` and `std::equal_to` to `folly::Uri`.
Both do the comparison / hash based upon the return value of `folly::Uri::toString`.

Reviewed By: yfeldblum

Differential Revision: D4711506

fbshipit-source-id: f4c2a9de8d4302fd315a9f31329cc8ba9f5e0409

Summary:
I'm working on a project where the target platform is tiny a bit out of
date:

    $ openssl version
    OpenSSL 1.0.0-fips 29 Mar 2010

For various "reasons" I'm not able to update it at this time, however I
would like to use Proxygen (and hence Folly). This patch allows Folly to
compile with the above version of OpenSSL.
Closes https://github.com/facebook/folly/pull/562

Reviewed By: Orvid

Differential Revision: D4715116

Pulled By: yfeldblum

fbshipit-source-id: be38ffb78f1e5cee971ce8cb81936b7f16efe050

Summary: Add getPeerAddress/getLocalAddress folly::SocketAddress() function

Reviewed By: yfeldblum

Differential Revision: D4706693

fbshipit-source-id: 534e862f4168bc9e7bb02e41eac8e2ced6a02c59

Summary:
To GCC 4.8, because it's not *quite* dead yet.

This is needed to get FixedString working properly on GCC 4.8

Reviewed By: yfeldblum

Differential Revision: D4646026

fbshipit-source-id: 075df04e479bd2b11f6538da2ed3e78b59956621

Summary:
I encountered a situation where "pkg-config --libs openssl" included
some linker flags, i.e.

    $ pkg-config --libs openssl
    -Wl,-z,relro -lssl -lcrypto -ldl -lz

This resulted in these linker flags being passed to sed. This change
filters out the linker flags from the pkg-config output.
Closes https://github.com/facebook/folly/pull/559

Reviewed By: pixelb

Differential Revision: D4700120

Pulled By: yfeldblum

fbshipit-source-id: 530d524caf965919418d19a1df53d2b418f67717

Summary:
This diff adds string-based compress and uncompress support to
folly::io::Codec. By default, the string API is implemented by translating to
and from IOBufs. Room is left for individual implementations to override that
behavior and go straight to the underlying library.

Reviewed By: yfeldblum

Differential Revision: D4690770

fbshipit-source-id: 7e2615da9add8328654505cdda6a2c79583b4558

Summary: Closes https://github.com/facebook/folly/pull/560

Reviewed By: pixelb

Differential Revision: D4700198

Pulled By: yfeldblum

fbshipit-source-id: 5e251e5719c6783c9f8a4fd9bab357ed362340c0

Summary:
This option is useful when compiling on systems that do not want to
include libunwind.
Closes https://github.com/facebook/folly/pull/558

Reviewed By: andrewjcg

Differential Revision: D4701878

Pulled By: yfeldblum

fbshipit-source-id: 61765e2e04ea1da510f06d7805c2c54381aa771e

Summary:
Use default constructor directly, instead of copying the element introduced by default parameter value.
This behavior is compatible with `std::vector`.

Reviewed By: ot, bixue2010, luciang

Differential Revision: D4700611

fbshipit-source-id: d3bd82c46c2857f5abcbec5dd2a64aaa85eaefe4

Summary: Folly supports being compiled without all of the compression libraries, so we need a way to determine at runtime what libraries Folly has been compiled to support.

Reviewed By: yfeldblum

Differential Revision: D4692556

fbshipit-source-id: 0ec459db70a4b1d64f1e07c87a1f51ae584bccd0

Summary:
Modify `QueueAppender::pushAtMost(const uint8_t*, size_t)` to fill the end of
the current buffer before it allocates another.

Reviewed By: yfeldblum

Differential Revision: D4687832

fbshipit-source-id: 7c21f4359da9f0cf26d9577abddfd640b0c877e9

Summary:
Since this is a not-uncommon pattern now:

```lang=cpp
if (auto found1 = get_ptr(m, k1)) {
  if (auto found2 = get_ptr(*found, k2)) {
    return use(*found2);
  }
}
```

This diff enables:

```lang=cpp
if (auto found = get_ptr(m, k1, k2)) {
  return use(*found);
}

```

Note that it works for const and non-const maps, returning a correspondingly mutable pointer.

Reviewed By: luciang, yfeldblum

Differential Revision: D3812701

fbshipit-source-id: 77ace9f5dca6cdc4cae7e6dfb9e5fc1075b5b571

Summary:
With more experience of using folly::Singleton in our code we have found new
recommendations for how it should be used. This change updates the
documentation block to include those recommendations.

Reviewed By: yfeldblum

Differential Revision: D4688700

fbshipit-source-id: 40350c2330084a28f1dc0744176d0ac8a0e64d39

Summary: EventBase support is achieved through default VirtualEventBase mechanism.

Reviewed By: yfeldblum

Differential Revision: D4685948

fbshipit-source-id: 15c8c789a55776984aa9087455e3f7b79d7604ad

Summary:
IndexedMemPool uses regular memory for the global and local next links. There can be concurrent reads and writes of such links. In order to avoid C++ undefined behavior due to the data races, these links should be atomic. Relaxed loads and stores are sufficient for correctness.

Depends on D4680286

Reviewed By: nbronson

Differential Revision: D4680317

fbshipit-source-id: 27c0d888e08bd5d862aee4c6dc6ee393033b32e3

Summary: It's a pain, so rely on macros to figure a few things about the build configuration.

Reviewed By: yfeldblum

Differential Revision: D4229128

fbshipit-source-id: 78b414c21cae6ba51ade2ab75b117cccad5c608c

Summary:
This switches mcrouter to use VirtualEventBase as the unified way of managing its threads. This allows multiple mcrouters to be run on the same thread-pool, without having to keep separate shutdown logic.

As an immediate win - this eliminates any special shutdown logic from tcc and standalone mcrouter.

Reviewed By: jmswen

Differential Revision: D4570982

fbshipit-source-id: 3974279005150e3e1db16230fde6b5951ae016b9