InlineFunctionRef for inline FunctionRef storage

Summary:
InlineFunctionRef is a semantically the same as folly::FunctionRef but has the
additional benefit of being able to store the function it was instantiated
with inline in a buffer of the given capacity.  If there is not enough in-situ
capacity for the callable, this has the same semantics as FunctionRef.

This helps give a perf boost in the case where the data gets separated from
the point of invocation.  If, for example, at the point of invocation, the
InlineFunctionRef object is not cached, a remote memory/cache read might be
required to invoke the original callable.  Customizable inline storage helps
tune storage so we can store a type-erased callable with better performance
and locality.  A real-life example of this might be a folly::FunctionRef with
a function pointer.  The folly::FunctionRef would point to the function
pointer object in a remote location.  This causes a double-indirection at the
point of invocation, and if that memory is dirty, or not cached, it would
cause additional cache misses.  On the other hand with InlineFunctionRef,
inline storage would store the value of the function pointer, avoiding the
need to do a remote lookup to fetch the value of the function pointer.

To prevent misuse, InlineFunctionRef disallows construction from an lvalue
callable.  This is to prevent usage where a user relies on the callable's
state after invocation through InlineFunctionRef.  This has the potential to
copy the callable into inline storage when the callable is small, so we might
not use the same function when invoking, but rather a copy of it.

Also note that InlineFunctionRef will always invoke the const qualified
version of the call operator for any callable that is passed.  Regardless of
whether it has a non-const version.  This is done to enforce the logical
constraint of function state being immutable.

This class is always trivially-copyable (and therefore
trivially-destructible), making it suitable for use in a union without
requiring manual destruction.

Reviewed By: yfeldblum, ot

Differential Revision: D14029799

fbshipit-source-id: 2cff3ce27d564f3d524095189f847c14911f9402

folly/synchronization/detail/InlineFunctionRef.h

+/*
+ * Copyright 2004-present Facebook, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include <folly/Traits.h>
+#include <folly/Utility.h>
+#include <folly/functional/Invoke.h>
+#include <folly/lang/Launder.h>
+
+namespace folly {
+namespace detail {
+
+/**
+ * InlineFunctionRef is similar to folly::FunctionRef but has the additional
+ * benefit of being able to store the function it was instantiated with inline
+ * in a buffer of the given capacity.  Inline storage is only used if the
+ * function object and a pointer (for type-erasure) are small enough to fit in
+ * the templated size.  If there is not enough in-situ capacity for the
+ * callable, this just stores a reference to the function object like
+ * FunctionRef.
+ *
+ * This helps give a perf boost in the case where the data gets separated from
+ * the point of invocation.  If, for example, at the point of invocation, the
+ * InlineFunctionRef object is not cached, a remote memory/cache read might be
+ * required to invoke the original callable.  Customizable inline storage
+ * helps tune storage so we can store a type-erased callable with better
+ * performance and locality.  A real-life example of this might be a
+ * folly::FunctionRef with a function pointer.  The folly::FunctionRef would
+ * point to the function pointer object in a remote location.  This causes a
+ * double-indirection at the point of invocation, and if that memory is dirty,
+ * or not cached, it would cause additional cache misses.  On the other hand
+ * with InlineFunctionRef, inline storage would store the value of the
+ * function pointer, avoiding the need to do a remote lookup to fetch the
+ * value of the function pointer.
+ *
+ * To prevent misuse, InlineFunctionRef disallows construction from an lvalue
+ * callable.  This is to prevent usage where a user relies on the callable's
+ * state after invocation through InlineFunctionRef.  This has the potential
+ * to copy the callable into inline storage when the callable is small, so we
+ * might not use the same function when invoking, but rather a copy of it.
+ *
+ * Also note that InlineFunctionRef will always invoke the const qualified
+ * version of the call operator for any callable that is passed.  Regardless
+ * of whether it has a non-const version.  This is done to enforce the logical
+ * constraint of function state being immutable.
+ *
+ * This class is always trivially-copyable (and therefore
+ * trivially-destructible), making it suitable for use in a union without
+ * requiring manual destruction.
+ */
+template <typename FunctionType, std::size_t Size>
+class InlineFunctionRef;
+
+template <typename ReturnType, typename... Args, std::size_t Size>
+class InlineFunctionRef<ReturnType(Args...), Size> {
+  using Storage = std::aligned_storage_t<Size - 8, 8>;
+  using Call = ReturnType (*)(const Storage&, Args&&...);
+
+  struct InSituTag {};
+  struct RefTag {};
+
+  static_assert((Size % 8) == 0, "Size has to be a multiple of 8");
+  static_assert(Size >= 16, "This doesn't work");
+  static_assert(alignof(Call) == alignof(Storage), "Mismatching alignments");
+
+  // This defines a mode tag that is used in the construction of
+  // InlineFunctionRef to determine the storage and indirection method for the
+  // passed callable.
+  //
+  // This requires that the we pass in a type that is not ref-qualified.
+  template <typename Func>
+  using ConstructMode = std::conditional_t<
+      folly::is_trivially_copyable<Func>{} &&
+          (sizeof(Func) <= sizeof(Storage)) &&
+          (alignof(Func) <= alignof(Storage)),
+      InSituTag,
+      RefTag>;
+
+ public:
+  /**
+   * InlineFunctionRef can be constructed from a nullptr, callable or another
+   * InlineFunctionRef with the same size.  These are the constructors that
+   * don't take a callable.
+   *
+   * InlineFunctionRef is meant to be trivially copyable so we default the
+   * constructors and assignment operators.
+   */
+  InlineFunctionRef(std::nullptr_t) : call_{nullptr} {}
+  InlineFunctionRef() : call_{nullptr} {}
+  InlineFunctionRef(const InlineFunctionRef& other) = default;
+  InlineFunctionRef(InlineFunctionRef&&) = default;
+  InlineFunctionRef& operator=(const InlineFunctionRef&) = default;
+  InlineFunctionRef& operator=(InlineFunctionRef&&) = default;
+
+  /**
+   * Constructors from callables.
+   *
+   * If all of the following conditions are satisfied, then we store the
+   * callable in the inline storage:
+   *
+   *  1) The function has been passed as an rvalue, meaning that there is no
+   *     use of the original in the user's code after it has been passed to
+   *     us.
+   *  2) Size of the callable is less than the size of the inline storage
+   *     buffer.
+   *  3) The callable is trivially constructible and destructible.
+   *
+   * If any one of the above conditions is not satisfied, we fall back to
+   * reference semantics and store the function as a pointer, and add a level
+   * of indirection through type erasure.
+   */
+  template <
+      typename Func,
+      std::enable_if_t<
+          !std::is_same<std::decay_t<Func>, InlineFunctionRef>{} &&
+          !std::is_reference<Func>{} &&
+          folly::is_invocable_r<ReturnType, Func&&, Args&&...>{}>* = nullptr>
+  InlineFunctionRef(Func&& func) {
+    // We disallow construction from lvalues, so assert that this is not a
+    // reference type.  When invoked with an lvalue, Func is a lvalue
+    // reference type, when invoked with an rvalue, Func is not ref-qualified.
+    static_assert(
+        !std::is_reference<Func>{},
+        "InlineFunctionRef cannot be used with lvalues");
+    static_assert(std::is_rvalue_reference<Func&&>{}, "");
+    construct(ConstructMode<Func>{}, folly::as_const(func));
+  }
+
+  /**
+   * The call operator uses the function pointer and a reference to the
+   * storage to do the dispatch.  The function pointer takes care of the
+   * appropriate casting.
+   */
+  ReturnType operator()(Args... args) const {
+    return call_(storage_, static_cast<Args&&>(args)...);
+  }
+
+  /**
+   * We have a function engaged if the call function points to anything other
+   * than null.
+   */
+  operator bool() const noexcept {
+    return call_;
+  }
+
+ private:
+  friend class InlineFunctionRefTest;
+
+  /**
+   * Inline storage constructor implementation.
+   */
+  template <typename Func>
+  void construct(InSituTag, Func& func) {
+    using Value = std::remove_reference_t<Func>;
+
+    // Assert that the following two assumptions are valid
+    //    1) fit in the storage space we have and match alignments, and
+    //    2) be invocable in a const context, it does not make sense to copy a
+    //       callable into inline storage if it makes state local
+    //       modifications.
+    static_assert(alignof(Value) <= alignof(Storage), "");
+    static_assert(is_invocable<const std::decay_t<Func>, Args&&...>{}, "");
+    static_assert(folly::is_trivially_copyable<Value>{}, "");
+
+    new (&storage_) Value{func};
+    call_ = &callInline<Value>;
+  }
+
+  /**
+   * Ref storage constructor implementation.  This is identical to
+   * folly::FunctionRef.
+   */
+  template <typename Func>
+  void construct(RefTag, Func& func) {
+    // store a pointer to the function
+    using Pointer = std::add_pointer_t<std::remove_reference_t<Func>>;
+    new (&storage_) Pointer{&func};
+    call_ = &callPointer<Pointer>;
+  }
+
+  template <typename Func>
+  static ReturnType callInline(const Storage& object, Args&&... args) {
+    // The only type of pointer allowed is a function pointer, no other
+    // pointer types are invocable.
+    static_assert(
+        !std::is_pointer<Func>::value ||
+            std::is_function<std::remove_pointer_t<Func>>::value,
+        "");
+    return folly::invoke(
+        *folly::launder(reinterpret_cast<const Func*>(&object)),
+        static_cast<Args&&>(args)...);
+  }
+
+  template <typename Func>
+  static ReturnType callPointer(const Storage& object, Args&&... args) {
+    // When the function we were instantiated with was not trivial, the given
+    // pointer points to a pointer, which pointers to the callable.  So we
+    // cast to a pointer and then to the pointee.
+    static_assert(std::is_pointer<Func>::value, "");
+    return folly::invoke(
+        **folly::launder(reinterpret_cast<const Func*>(&object)),
+        static_cast<Args&&>(args)...);
+  }
+
+  Call call_;
+  Storage storage_;
+};
+
+} // namespace detail
+} // namespace folly

folly/synchronization/detail/test/InlineFunctionRefTest.cpp

+/*
+ * Copyright 2019-present Facebook, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include <folly/synchronization/detail/InlineFunctionRef.h>
+
+#include <folly/portability/GTest.h>
+
+#include <cstring>
+
+namespace folly {
+namespace detail {
+
+class InlineFunctionRefTest : public ::testing::Test {
+ public:
+  template <typename InlineFRef>
+  static auto& storage(InlineFRef& fref) {
+    return fref.storage_;
+  }
+
+  template <typename InlineFRef>
+  static auto& call(InlineFRef& fref) {
+    return fref.call_;
+  }
+};
+
+TEST_F(InlineFunctionRefTest, BasicInvoke) {
+  {
+    auto func = [](auto integer) { return integer; };
+    auto copy = func;
+    auto fref = InlineFunctionRef<int(int), 24>{std::move(func)};
+    EXPECT_EQ(fref(1), 1);
+    EXPECT_EQ(fref(2), 2);
+    EXPECT_EQ(fref(3), 3);
+
+    EXPECT_EQ(sizeof(copy), 1);
+    EXPECT_EQ(std::memcmp(&storage(fref), &copy, 1), 0);
+  }
+}
+
+TEST_F(InlineFunctionRefTest, InvokeWithCapture) {
+  {
+    auto data = std::uint64_t{1};
+    auto func = [&](auto integer) { return integer + data; };
+    auto copy = func;
+    auto fref = InlineFunctionRef<int(int), 24>{std::move(func)};
+
+    EXPECT_EQ(fref(1), 2);
+    EXPECT_EQ(fref(2), 3);
+    EXPECT_EQ(fref(3), 4);
+
+    data = 2;
+    EXPECT_EQ(fref(1), 3);
+    EXPECT_EQ(fref(2), 4);
+    EXPECT_EQ(fref(3), 5);
+
+    data = 3;
+    EXPECT_EQ(fref(1), 4);
+    EXPECT_EQ(fref(2), 5);
+    EXPECT_EQ(fref(3), 6);
+
+    EXPECT_EQ(sizeof(copy), 8);
+    EXPECT_EQ(std::memcmp(&storage(fref), &copy, 8), 0);
+  }
+}
+
+TEST_F(InlineFunctionRefTest, InvokeWithFunctionPointer) {
+  {
+    using FPtr = int (*)(int);
+    auto func = FPtr{[](auto integer) { return integer; }};
+    auto copy = func;
+
+    // we move into InlineFunctionRef but the move doesn't actually do anything
+    // destructive to the parameter
+    auto fref = InlineFunctionRef<int(int), 24>{std::move(func)};
+
+    EXPECT_EQ(fref(1), 1);
+    EXPECT_EQ(fref(2), 2);
+    EXPECT_EQ(fref(3), 3);
+
+    EXPECT_EQ(sizeof(func), 8);
+    EXPECT_EQ(std::memcmp(&storage(fref), &copy, 8), 0);
+  }
+}
+
+TEST_F(InlineFunctionRefTest, InvokeWithBigLambda) {
+  {
+    auto data = std::array<std::uint8_t, 128>{};
+    for (auto i = std::size_t{0}; i < data.size(); ++i) {
+      data[i] = i;
+    }
+    auto func = [data](auto integer) { return integer + data[integer]; };
+    auto copy = func;
+    auto address = &func;
+    auto fref = InlineFunctionRef<int(int), 24>{std::move(func)};
+
+    EXPECT_EQ(fref(1), 2);
+    EXPECT_EQ(fref(2), 4);
+    EXPECT_EQ(fref(3), 6);
+
+    EXPECT_EQ(sizeof(copy), 128);
+    EXPECT_EQ(sizeof(copy), sizeof(func));
+    EXPECT_EQ(std::memcmp(&storage(fref), &address, 8), 0);
+    EXPECT_EQ(std::memcmp(&copy, &func, sizeof(copy)), 0);
+  }
+}
+
+TEST_F(InlineFunctionRefTest, Nullability) {
+  auto fref = InlineFunctionRef<void(), 24>{nullptr};
+  EXPECT_FALSE(fref);
+}
+
+TEST_F(InlineFunctionRefTest, CopyConstruction) {
+  {
+    auto data = std::uint64_t{1};
+    auto func = [&](auto integer) { return integer + data; };
+    auto one = InlineFunctionRef<int(int), 24>{std::move(func)};
+    auto two = one;
+    EXPECT_EQ(std::memcmp(&one, &two, sizeof(one)), 0);
+
+    EXPECT_EQ(two(1), 2);
+    EXPECT_EQ(two(2), 3);
+    EXPECT_EQ(two(3), 4);
+
+    data = 2;
+    EXPECT_EQ(two(1), 3);
+    EXPECT_EQ(two(2), 4);
+    EXPECT_EQ(two(3), 5);
+
+    data = 3;
+    EXPECT_EQ(two(1), 4);
+    EXPECT_EQ(two(2), 5);
+    EXPECT_EQ(two(3), 6);
+  }
+
+  {
+    auto data = std::array<std::uint8_t, 128>{};
+    for (auto i = std::size_t{0}; i < data.size(); ++i) {
+      data[i] = i;
+    }
+    auto func = [data](auto integer) { return integer + data[integer]; };
+    auto one = InlineFunctionRef<int(int), 24>{std::move(func)};
+    auto two = one;
+    EXPECT_EQ(std::memcmp(&one, &two, sizeof(one)), 0);
+
+    EXPECT_EQ(two(1), 2);
+    EXPECT_EQ(two(2), 4);
+    EXPECT_EQ(two(3), 6);
+
+    EXPECT_EQ(sizeof(func), 128);
+    auto address = &func;
+    EXPECT_EQ(std::memcmp(&storage(two), &address, 8), 0);
+  }
+}
+
+TEST_F(InlineFunctionRefTest, TestTriviality) {
+  {
+    auto lambda = []() {};
+    auto fref = InlineFunctionRef<void(), 24>{std::move(lambda)};
+    EXPECT_TRUE(std::is_trivially_destructible<decltype(fref)>{});
+    EXPECT_TRUE(folly::is_trivially_copyable<decltype(fref)>{});
+  }
+  {
+    auto integer = std::uint64_t{0};
+    auto lambda = [&]() { static_cast<void>(integer); };
+    auto fref = InlineFunctionRef<void(), 24>{std::move(lambda)};
+    EXPECT_TRUE(std::is_trivially_destructible<decltype(fref)>{});
+    EXPECT_TRUE(folly::is_trivially_copyable<decltype(fref)>{});
+  }
+  {
+    auto data = std::array<std::uint8_t, 128>{};
+    auto lambda = [data]() { static_cast<void>(data); };
+    auto fref = InlineFunctionRef<void(), 24>{std::move(lambda)};
+    EXPECT_TRUE(std::is_trivially_destructible<decltype(fref)>{});
+    EXPECT_TRUE(folly::is_trivially_copyable<decltype(fref)>{});
+  }
+}
+
+namespace {
+template <typename Data>
+class ConstQualifiedFunctor {
+ public:
+  int operator()() {
+    return 0;
+  }
+  int operator()() const {
+    return 1;
+  }
+
+  Data data_;
+};
+} // namespace
+
+TEST_F(InlineFunctionRefTest, CallConstQualifiedMethod) {
+  {
+    auto small = ConstQualifiedFunctor<std::uint8_t>{};
+    auto fref = InlineFunctionRef<int(), 24>{std::move(small)};
+    EXPECT_EQ(fref(), 1);
+  }
+  {
+    const auto small = ConstQualifiedFunctor<std::uint8_t>{};
+    auto fref = InlineFunctionRef<int(), 24>{std::move(small)};
+    EXPECT_EQ(fref(), 1);
+  }
+  {
+    auto big = ConstQualifiedFunctor<std::array<std::uint8_t, 128>>{};
+    auto fref = InlineFunctionRef<int(), 24>{std::move(big)};
+    EXPECT_EQ(fref(), 1);
+  }
+  {
+    const auto big = ConstQualifiedFunctor<std::array<std::uint8_t, 128>>{};
+    auto fref = InlineFunctionRef<int(), 24>{std::move(big)};
+    EXPECT_EQ(fref(), 1);
+  }
+}
+
+namespace {
+template <std::size_t Size>
+using InlineFRef = InlineFunctionRef<void(), Size>;
+} // namespace
+
+TEST_F(InlineFunctionRefTest, TestSizeAlignment) {
+  EXPECT_EQ(sizeof(storage(std::declval<InlineFRef<16>&>())), 8);
+  EXPECT_EQ(alignof(decltype(storage(std::declval<InlineFRef<16>&>()))), 8);
+
+  EXPECT_EQ(sizeof(storage(std::declval<InlineFRef<24>&>())), 16);
+  EXPECT_EQ(alignof(decltype(storage(std::declval<InlineFRef<24>&>()))), 8);
+
+  EXPECT_EQ(sizeof(storage(std::declval<InlineFRef<32>&>())), 24);
+  EXPECT_EQ(alignof(decltype(storage(std::declval<InlineFRef<32>&>()))), 8);
+}
+
+namespace {
+int foo(int integer) {
+  return integer;
+}
+} // namespace
+
+TEST_F(InlineFunctionRefTest, TestFunctionPointer) {
+  auto fref = InlineFunctionRef<int(int), 24>{&foo};
+  EXPECT_EQ(fref(1), 1);
+  EXPECT_EQ(fref(2), 2);
+  EXPECT_EQ(fref(3), 3);
+}
+
+namespace {
+class alignas(16) MaxAligned {};
+class alignas(32) MoreThanMaxAligned {};
+} // namespace
+
+TEST_F(InlineFunctionRefTest, TestMaxAlignment) {
+  {
+    auto aligned = MaxAligned{};
+    auto func = [aligned]() { static_cast<void>(aligned); };
+    auto fref = InlineFunctionRef<void(), 24>{std::move(func)};
+    auto address = &func;
+    EXPECT_EQ(std::memcmp(&storage(fref), &address, 8), 0);
+  }
+  {
+    auto aligned = MoreThanMaxAligned{};
+    auto func = [aligned]() { static_cast<void>(aligned); };
+    auto fref = InlineFunctionRef<void(), 24>{std::move(func)};
+    auto address = &func;
+    EXPECT_EQ(std::memcmp(&storage(fref), &address, 8), 0);
+  }
+}
+
+TEST_F(InlineFunctionRefTest, TestLValueConstructibility) {
+  auto lambda = []() {};
+  EXPECT_TRUE((!std::is_constructible<
+               InlineFunctionRef<void(), 24>,
+               decltype(lambda)&>{}));
+}
+
+} // namespace detail
+} // namespace folly

folly/test/FunctionRefBenchmark.cpp

@@ -16,6 +16,7 @@
 #include <folly/Benchmark.h>
 #include <folly/Function.h>
 #include <folly/Random.h>
+#include <folly/synchronization/detail/InlineFunctionRef.h>
 
 #include <cstdint>
 #include <functional>
@@ -59,6 +60,7 @@ void runSmallInvokeBenchmark(std::size_t iters, MakeFunction make) {
     folly::makeUnpredictable(i);
     return i;
   };
+  folly::makeUnpredictable(lambda);
   auto func = make(lambda);
   folly::makeUnpredictable(func);
 
@@ -73,6 +75,7 @@ void runSmallCreateAndInvokeBenchmark(std::size_t iters, MakeFunction make) {
     folly::makeUnpredictable(i);
     return i;
   };
+  folly::makeUnpredictable(lambda);
 
   for (auto i = std::size_t{iters}; --i;) {
     auto func = make(lambda);
@@ -92,6 +95,7 @@ void runBigAndInvokeBenchmark(std::size_t iters, MakeFunction make) {
     folly::makeUnpredictable(i);
     return i;
   };
+  folly::makeUnpredictable(lambda);
   auto func = make(lambda);
   folly::makeUnpredictable(func);
 
@@ -113,6 +117,7 @@ void runBigCreateAndInvokeBenchmark(std::size_t iters, MakeFunction make) {
     folly::makeUnpredictable(i);
     return i;
   };
+  folly::makeUnpredictable(lambda);
 
   suspender.dismissing([&] {
     for (auto i = std::size_t{iters}; --i;) {
@@ -145,6 +150,11 @@ BENCHMARK(SmallFunctionFollyFunctionRefInvoke, iters) {
   runSmallInvokeBenchmark(
       iters, [](auto& f) { return folly::FunctionRef<size_t(size_t&)>{f}; });
 }
+BENCHMARK(SmallFunctionFollyInlineFunctionRefInvoke, iters) {
+  runSmallInvokeBenchmark(iters, [](auto f) {
+    return detail::InlineFunctionRef<size_t(size_t&), 24>{std::move(f)};
+  });
+}
 
 BENCHMARK_DRAW_LINE();
 BENCHMARK(SmallFunctionFunctionPointerCreateInvoke, iters) {
@@ -168,6 +178,11 @@ BENCHMARK(SmallFunctionFollyFunctionRefCreateInvoke, iters) {
   runSmallCreateAndInvokeBenchmark(
       iters, [](auto& f) { return folly::FunctionRef<size_t(size_t&)>{f}; });
 }
+BENCHMARK(SmallFunctionFollyInlineFunctionRefCreateInvoke, iters) {
+  runSmallInvokeBenchmark(iters, [](auto f) {
+    return detail::InlineFunctionRef<size_t(size_t&), 24>{std::move(f)};
+  });
+}
 
 BENCHMARK_DRAW_LINE();
 BENCHMARK(BigFunctionStdFunctionInvoke, iters) {
@@ -187,6 +202,11 @@ BENCHMARK(BigFunctionFollyFunctionRefInvoke, iters) {
   runBigAndInvokeBenchmark(
       iters, [](auto& f) { return folly::FunctionRef<size_t(size_t&)>{f}; });
 }
+BENCHMARK(BigFunctionFollyInlineFunctionRefInvoke, iters) {
+  runSmallInvokeBenchmark(iters, [](auto f) {
+    return detail::InlineFunctionRef<size_t(size_t&), 24>{std::move(f)};
+  });
+}
 
 BENCHMARK_DRAW_LINE();
 BENCHMARK(BigFunctionStdFunctionCreateInvoke, iters) {
@@ -206,6 +226,11 @@ BENCHMARK(BigFunctionFollyFunctionRefCreateInvoke, iters) {
   runBigCreateAndInvokeBenchmark(
       iters, [](auto& f) { return folly::FunctionRef<size_t(size_t&)>{f}; });
 }
+BENCHMARK(BigFunctionFollyInlineFunctionRefCreateInvoke, iters) {
+  runSmallInvokeBenchmark(iters, [](auto f) {
+    return detail::InlineFunctionRef<size_t(size_t&), 24>{std::move(f)};
+  });
+}
 } // namespace folly
 
 int main(int argc, char** argv) {

folly/test/FunctionRefTest.cpp

@@ -19,8 +19,7 @@
 #include <folly/Function.h>
 #include <folly/portability/GTest.h>
 
-using folly::Function;
-using folly::FunctionRef;
+namespace folly {
 
 TEST(FunctionRef, Traits) {
   static_assert(std::is_literal_type<FunctionRef<int(int)>>::value, "");
@@ -224,3 +223,5 @@ TEST(FunctionRef, ForEach) {
 
   EXPECT_EQ(55, sum);
 }
+
+} // namespace folly