Summary:
This switches mcrouter to use VirtualEventBase as the unified way of managing its threads. This allows multiple mcrouters to be run on the same thread-pool, without having to keep separate shutdown logic.

As an immediate win - this eliminates any special shutdown logic from tcc and standalone mcrouter.

Reviewed By: jmswen

Differential Revision: D4570982

fbshipit-source-id: 3974279005150e3e1db16230fde6b5951ae016b9

Summary: The new AsyncSocket is already constructed at this point so just use the event base from it.

Reviewed By: djwatson

Differential Revision: D4689395

fbshipit-source-id: aac898c1c6e6e0e0ffcb20b16bbf2b842cc31d54

Implementing a callback interface for folly::AsyncSocket allowing to supply an ancillary data buffer with msghdr structure to sendmsg() system call

Summary: Implementing a callback interface for folly::AsyncSocket allowing to supply an ancillary data buffer with msghdr structure to sendmsg() system call.

Reviewed By: afrind

Differential Revision: D4422168

fbshipit-source-id: 29a23b05f704aff796d368f4ac9514c49b7ce578

Summary: Existing runInLoop implementation was generally not safe (it did not capture the KeepAlive token). runInLoop() is mostly used in library code though, and often times it's ok not capture the KeepAlive token. It's better to just have the caller make the decision, rather then keep runInLoop() and make it always capture the KeepAlive token.

Reviewed By: yfeldblum

Differential Revision: D4684025

fbshipit-source-id: 65907bbe9c774e2a7b92580d0c0387d346b495d5

Summary:
Even though an ostream operator<<() is defined for StringPiece, gtest doesn't
use it when printing StringPiece values in test failure messages.  Because
StringPiece defines a nested iterator type gtest instead treats StringPiece as
a container, and prints each of its elements (characters) one-by-one.  This is
fairly awkward to read.

This diff defines an explicit PrintTo() function for StringPiece in
folly/test/TestUtils.h.  This makes gtest print StringPiece values nicely if
you include TestUtils.h in your test sources.

Reviewed By: yfeldblum

Differential Revision: D4672257

fbshipit-source-id: 4b39ccc116e5382c29c37c2abe879293d310faf5

Summary: IndexedMemPool uses regular memory for the global and local next links. There is a data race as there can be concurrent reads and writes of such links. Added a test for data races.

Reviewed By: nbronson

Differential Revision: D4680286

fbshipit-source-id: 9ef3ed439b9df07d69afe570e516c146caa53a6c

Summary: MSG_PEEK was difficult if not impossible to use well since we do not provide a way wait for more data to arrive. If you are using setPeek on AsyncSocket, and you do not receive the amount of data you want, you must either abandon your peek attempt, or spin around the event base waiting for more data. This diff replaces the peek interface on AsyncSocket with a pre-received data interface, allowing users to insert data back onto the front of connections after reading some data in another layer.

Reviewed By: djwatson

Differential Revision: D4626315

fbshipit-source-id: c552e64f5b3ac9e40ea3358d65b4b9db848f5d74

Reviewed By: yfeldblum

Differential Revision: D4671818

fbshipit-source-id: 80154adfef238f0be62da12fb843e22de1ae65ca

Summary:
GCC & Clang allow the use of a non-captured constexpr local within a lambda, MSVC does not.

All 3 compilers allow this when the local has static storage.

Reviewed By: yfeldblum

Differential Revision: D4675147

fbshipit-source-id: 3e07eb77731cbd3ebda6aee4b0b57ac19ac34de5

Summary:
With the way it is currently, if `jemalloc.h` is included after `folly/Malloc.h`, `jemalloc.h` ends up redefining these.
This changes it to just include `jemalloc.h` if we know we're compiling with JEMalloc.

Reviewed By: yfeldblum

Differential Revision: D4660285

fbshipit-source-id: 814e36432676e89cf7ba2bc5247b7dde2bfdc14a

Summary: D4624754 broke mcrouter-oss because that uses older OpenSSL (1.0.1).

Reviewed By: yfeldblum, jmswen

Differential Revision: D4670767

fbshipit-source-id: f297c8d3e013525bb36bd059eb40c70f0c4c2dcc

Summary: Title.

Reviewed By: yfeldblum

Differential Revision: D4674125

fbshipit-source-id: 71ad38d7a52ab1c27fcc7ff14d130978b0ece382

Summary:
Make each EventBase have a defuault VirtualEventBase which is attached to it and is joined on destruction. Default VirtualEventBase is lazily created on first request.
This makes it trivial to use code switched to VirtualEventBase both with VirtualEventBase and EventBase.

Reviewed By: yfeldblum

Differential Revision: D4644639

fbshipit-source-id: cf28a3632463a1c61404c225ce1186f5a4a062a3

Summary:
There is no need for EventBase to log every loop iteration on a relatively low VLOG level.
Just move it up to 11 like the other verbose logging in EventBase.cpp

Reviewed By: yfeldblum

Differential Revision: D4642379

fbshipit-source-id: 351636afb186ba94b2447b206d7d1389cbdba110

Summary:
Add a few more getters (subject, issuer, notbefore, notafter) to
OpenSSLCertUtils.

Also add an additional API to AsyncSSLSocket to add application-generated
alert strings during the handshake, for e.g., during certificate verification

Reviewed By: knekritz

Differential Revision: D4624754

fbshipit-source-id: f01998b9e0e58b88ece8c6dc51ab590988bf0a8f

Summary:
This file only compiles with MSVC after I made the change in this diff.

The code was difficult to follow, so I am not sure if the template is needed here?

Reviewed By: yfeldblum

Differential Revision: D4656575

fbshipit-source-id: 19415623bdb1757b573f12cdc9d8544d716ce6f2

Summary:
Fix warnings exposed by gcc's -Wshadow-compatible-local.
Don't use a hard-coded names that would cause shadowing with each nested use of this macro.

This avoids errors like the following:

  folly/Foreach.h:94:18: error: declaration of 'FOR_EACH_privateCount' shadows a previous local [-Werror=shadow-compatible-local]
  folly/Foreach.h:91:12: error: shadowed declaration is here [-Werror=shadow-compatible-local]

Reviewed By: yfeldblum

Differential Revision: D4656783

fbshipit-source-id: d7cd820fb04ccd78242f7487e2b219dc300709b6

Summary:
AsyncSSLSocket can be connected using connect with
total timeouts.

This adds a method to get the total timeout that
was set.

Reviewed By: ngoyal, yfeldblum

Differential Revision: D4656331

fbshipit-source-id: a55ad9f081449d358b8133e9598a2063f625a2e6

Summary:
It is not unusual to write a RequestData that will send a request to a backend service upon destruction (for example, to log the request data). This typically involves calling EventBase::runInEventBaseThread() to do the logging. Normally, if you wait until the RequestContext is destructing for this to happen, everything works fine. However, if you explicitly call clearContextData(), then this can cause a deadlock. clearContextData() holds the write lock on the data_. Imagine the NotificationQueue thread tries to set the context (for some other reason), and it is waiting on the read lock on the data_. It does so while holding a lock on its own queue. Now, the clearContextData() calls the destructor while holding the write lock on data_, and the destructor calls runInEventBaseThread, which grabs the queue's lock. Deadlock.

This can either be fixed in NotificationQueue or RequestContext. I decided to fix it in RequestContext because there may be other deadlocks that don't involve NotificationQueue (see the test plan).

There are other deadlocks in RequestContext if you call write methods in RequestData::onSet or onUnset. But fixing this would require a massive perf regression (I imagine using shared_ptr instead of unique_ptr, and copying the ptrs under readlock and calling the callbacks after releasing the lock - yuck!) So instead, warn people against doing that.

Reviewed By: yfeldblum

Differential Revision: D4655400

fbshipit-source-id: cfebe696953a19dc4dba49976bbcd0ec1363bc42

Summary:
[Folly] Tweak size handling in `Endian`.

Either multiply by 8 in one place, or divide by 8 in another place. Go with the latter, because `sizeof` in C++ emits byte sizes, not bit sizes.

Reviewed By: simpkins

Differential Revision: D4645020

fbshipit-source-id: cb78600ba4c20bebc66aed506d4b5d6c378fc998

Summary: This reverts commit 21b86944f513da0700039a3e3736937fd72e8887

Differential Revision: D4618623

fbshipit-source-id: 6ab6498a2ce22895d7052e258d1af216d46ca178

Summary:
The current method to supply connect timeouts to AsyncSSLSocket
is to supply only the max connect + ssl connect time.

However in some cases when connect time is known, it is desirable
to supply it so that if connect does not succeed in connect time
we can error out quicker and retry.

This adds a new method for connect time in addition to the total
Connect time.

An alternative to this would be to create a AsyncSocket, connect it
and then pass it's fd to AsyncSSLSocket, however that approach does
not work well when TFO is being used, because TFO State is a part of
the AsyncSocket's state and that is not encapsulated with the fd transfer.

We could move the state around, but that is error prone, and this is much
simpler and isolated to AsyncSSLSocket.

Reviewed By: knekritz

Differential Revision: D4626924

fbshipit-source-id: d802d035efbced68873ab59314d9f0e661e5509b

Summary:
[Folly] Refactor `Endian`.

Define `swap` for all types, instead of only declaring it for all types, but have its implementation call a function that only has a given set of non-template overloads for a given set of supported types.

The effective change is that now `Endian::swap<std::string>` will fail to compile, rather than compile but fail to link.

And support floating-point types.

Reviewed By: nbronson

Differential Revision: D4615706

fbshipit-source-id: 5034e1e4466b8118a258d917ce8f4094460ca01a

Summary: detect_stack_use_after_return enables fake stack logic. Previous implementation was relying on fiber releasing its fake stack before switch, which is not safe, because that fake stack may actually be in use. This diff implements a hacky workaround by building a freeFakeStack function, which can release fake stack outside of the fiber. Ideally this function should be provided by ASAN itself.

Reviewed By: yfeldblum

Differential Revision: D4631962

fbshipit-source-id: 6213e2a6b3cbc35fae3793fcf86b0614dd8ff825

Summary: It doesn't work nicely with fake stacks (which are used with detect_stack_use_after_return).

Reviewed By: yfeldblum

Differential Revision: D4631995

fbshipit-source-id: f139041314f8f64c0cf0a15874d6a4710fb2a5b6

Summary:
Parameterizing Range by the template arg T is potentially
confusing because it violates the typical container pattern of T being
the value type.  This diff uses Range<Iter> consistently in Range.h when
the template arg refers to a fully general iterator.

Reviewed By: yfeldblum

Differential Revision: D4635270

fbshipit-source-id: f70bb2e3e23ab9875fba3dbf7b11492613a037b8

Summary:
C++14 has no direct means of creating a std::weak_ptr, one must always create a (temporary) std::shared_ptr first. C++17 adds weak_from_this() to std::enable_shared_from_this to avoid that overhead.
Alas code that is meant to compile under different language versions cannot rely on std::enable_shared_from_this::weak_from_this().
This new utility class uses SFINAE to call std::enable_shared_from_this<T>::weak_from_this() if available. Falls back to std::enable_shared_from_this<T>::shared_from_this() otherwise.
Use as a drop-in replacement for std::enable_shared_from_this where weak_from_this() is desired.

Reviewed By: yfeldblum

Differential Revision: D4616394

fbshipit-source-id: 73bf6cd8852d4a33478a9280cc69c81e7ea73423

Summary: As title, convenient for debugging. Asked for from D4542856

Reviewed By: nkg-

Differential Revision: D4618623

fbshipit-source-id: 21b86944f513da0700039a3e3736937fd72e8887

Summary:
`make check` was failing because <gmock.h> is included but not supplied. Updating to googletest 1.8.0 includes both googletest & googlemock, so `make check` now runs (and passes).
Closes https://github.com/facebook/folly/pull/547

Reviewed By: igorsugak

Differential Revision: D4593403

Pulled By: Orvid

fbshipit-source-id: 02f80e7b67e17562ff981e9c80a37f985eb685be

Summary: I think this ended up encompassing more than intended.

Reviewed By: yfeldblum, maxgeorg

Differential Revision: D4622645

fbshipit-source-id: 26597098a479b85d6ecc64abef2ecc53637ee759

Summary: Fixes "undefined behavior" in ubsan builds when using an empty ByteRange.

Reviewed By: siyengar

Differential Revision: D4613663

fbshipit-source-id: 4b53ba764609acc986340f10613f84585fa697cf

Summary: They are checks that aren't actually used anywhere to guard anything, so kill them. Many are for things that have long since been standard, or aren't even used.

Reviewed By: yfeldblum

Differential Revision: D4615947

fbshipit-source-id: 2179caff4ef8583c89efedbff3136ae5fc552424

Summary: This is the only place `inttypes.h` is used in Folly and it's used in a context where `cstdint` would have sufficed.

Reviewed By: yfeldblum

Differential Revision: D4619248

fbshipit-source-id: e67dccb7d8bf1ad5f7539d4b039590253e8d8c79

Summary: These are the only includes of `sched.h` in Folly, and they really are unused:

Reviewed By: yfeldblum

Differential Revision: D4615501

fbshipit-source-id: 6a3a1aad7a91a92a322fd93d18e1284aa3f86b89

Summary:
It currently takes ~32kB of stack trace to run symbolizer, which is
very close to ASan alt stack size (32 kB).

If we exclude `demangle` (which can use unbound stack size in extreme cases),
the heaviest path in symbolizer includes `FrameArray<100>` and three arrays of
`PATH_MAX` (4 kB) size.  This diff removes the former and one of the latters,
reducing this code path from 32 kB to ~10 kB.

Reviewed By: ot, yfeldblum

Differential Revision: D4618467

fbshipit-source-id: e6a53b61b3d5f6e8b892216d2e9b839ed8430d0e

Summary: It was only needed for GCC 4.7 support. GCC 4.8 and above support it.

Reviewed By: yfeldblum

Differential Revision: D4616044

fbshipit-source-id: 177d7b3b7090134f9d8084a3ba7a984446c87d5e

Summary: `last` was already passed in as a variable, so switch this so there isn't a local variable at all.

Reviewed By: mlogan, yfeldblum

Differential Revision: D4615206

fbshipit-source-id: dd8e3157e54dc2abe3dfd97837ce327371f4f7b2

Summary: [Folly] Drop support for gcc 4.7 with byteswap shim

Reviewed By: igorsugak, Orvid

Differential Revision: D4610746

fbshipit-source-id: 092cd432c7bda9ca5507d0230b6fcaa43b96c500

Summary:
`previousValue` makes the incorrect assumption that `outer_`
is a multiple of the word size. This is incorrect because `skipToNext`
can reposition at an arbitrary point.

The bug is very rare because it can only happen if there is a gap
larger than the skip quantum after the first element in the upper
bits.

Reviewed By: philippv

Differential Revision: D4607270

fbshipit-source-id: ff016f09f3f1f87314b68370e3dc137b82694f45

Summary:
antonl noticed that bulk_insert doesn't remove duplicates.
We now run a std::unique() pass after merging.

Reviewed By: yfeldblum

Differential Revision: D4595304

fbshipit-source-id: 538364150aeea64b95488da158c09e07a6597e7c