folly Singleton: "eager" option to initialize upfront

Summary: Instead of the default lazy-loading behavior (still the default) some singletons might need to get initialized at startup time.  This would be for singletons which take a long time for the instance's constructor to run, e.g. expensive initialization by reading some large dataset, talking to an outside service, and so on.

Provides a way for singletons to opt-in to this, and get populated at the time  `registrationComplete()` is called, instead of lazily.

Some notes about the way I implemented here, mainly, why I did this as a "builder-pattern" kind of thing and not some other way.  I could probably be convinced to do otherwise. :)

* Changing the constructor: the constructor's already slightly fiddly with the two optional -- well, one optional construct function, and another optional-but-only-if-construct-provided, destruct function.  Didn't want to pile more into the ctor.
* New superclass called `EagerLoadedSingleton`; just didn't want to add more classes, esp. if it's just to add one more option.
* Method like `void setEagerLoad()` that makes this eager-load; not sure where one would write the `shouldEagerLoad()` call, probably in some central initialization spot in `main()`, but all the maintenance would have to go there.  I like that it's "attached" to the singleton being defined.  (Though you can still do this.)  Bonus #2; the rule that builds the cpp containing "main" doesn't need to import this dependency and the cpp doesn't have to include Singleton just to do this eager-load call, nor the header for the type itself.
* Omitting this altogether and just saying `folly::Singleton<Foo>::get_weak()` to "ping" the singleton and bring into existence: see last point.  Still might need to have the file containing this initialization decorum include/link against Foo, as well as have one place to maintain the list of things to load up-front.

Reviewed By: @meyering

Differential Revision: D2449081

folly/Singleton-inl.h

@@ -135,20 +135,35 @@ SingletonHolder<T>::SingletonHolder(TypeDescriptor type__,
     type_(type__), vault_(vault) {
 }
 
+template <typename T>
+bool SingletonHolder<T>::creationStarted() {
+  // If alive, then creation was of course started.
+  // This is flipped after creating_thread_ was set, and before it was reset.
+  if (state_.load(std::memory_order_acquire) == SingletonHolderState::Living) {
+    return true;
+  }
+
+  // Not yet built.  Is it currently in progress?
+  if (creating_thread_.load(std::memory_order_acquire) != std::thread::id()) {
+    return true;
+  }
+
+  return false;
+}
+
 template <typename T>
 void SingletonHolder<T>::createInstance() {
-  // There's no synchronization here, so we may not see the current value
-  // for creating_thread if it was set by other thread, but we only care about
-  // it if it was set by current thread anyways.
-  if (creating_thread_ == std::this_thread::get_id()) {
+  if (creating_thread_.load(std::memory_order_acquire) ==
+        std::this_thread::get_id()) {
     LOG(FATAL) << "circular singleton dependency: " << type_.name();
   }
 
   std::lock_guard<std::mutex> entry_lock(mutex_);
-  if (state_ == SingletonHolderState::Living) {
+  if (state_.load(std::memory_order_acquire) == SingletonHolderState::Living) {
     return;
   }
-  if (state_ == SingletonHolderState::NotRegistered) {
+  if (state_.load(std::memory_order_acquire) ==
+        SingletonHolderState::NotRegistered) {
     auto ptr = SingletonVault::stackTraceGetter().load();
     LOG(FATAL) << "Creating instance for unregistered singleton: "
                << type_.name() << "\n"
@@ -156,7 +171,7 @@ void SingletonHolder<T>::createInstance() {
                << "\n" << (ptr ? (*ptr)() : "(not available)");
   }
 
-  if (state_ == SingletonHolderState::Living) {
+  if (state_.load(std::memory_order_acquire) == SingletonHolderState::Living) {
     return;
   }
 
@@ -164,10 +179,10 @@ void SingletonHolder<T>::createInstance() {
     // Clean up creator thread when complete, and also, in case of errors here,
     // so that subsequent attempts don't think this is still in the process of
     // being built.
-    creating_thread_ = std::thread::id();
+    creating_thread_.store(std::thread::id(), std::memory_order_release);
   };
 
-  creating_thread_ = std::this_thread::get_id();
+  creating_thread_.store(std::this_thread::get_id(), std::memory_order_release);
 
   RWSpinLock::ReadHolder rh(&vault_.stateMutex_);
   if (vault_.state_ == SingletonVault::SingletonVaultState::Quiescing) {
@@ -216,7 +231,7 @@ void SingletonHolder<T>::createInstance() {
 
   // This has to be the last step, because once state is Living other threads
   // may access instance and instance_weak w/o synchronization.
-  state_.store(SingletonHolderState::Living);
+  state_.store(SingletonHolderState::Living, std::memory_order_release);
 
   {
     RWSpinLock::WriteHolder wh(&vault_.mutex_);

folly/Singleton.h

@@ -71,6 +71,26 @@
 // Where create and destroy are functions, Singleton<T>::CreateFunc
 // Singleton<T>::TeardownFunc.
 //
+// The above examples detail a situation where an expensive singleton is loaded
+// on-demand (thus only if needed).  However if there is an expensive singleton
+// that will likely be needed, and initialization takes a potentially long time,
+// e.g. while initializing, parsing some files, talking to remote services,
+// making uses of other singletons, and so on, the initialization of those can
+// be scheduled up front, or "eagerly".
+//
+// In that case the singleton can be declared this way:
+//
+// namespace {
+// auto the_singleton =
+//     folly::Singleton<MyExpensiveService>(/* optional create, destroy args */)
+//     .shouldEagerInit();
+// }
+//
+// This way the singleton's instance is built at program initialization
+// time, or more accurately, when "registrationComplete()" or
+// "startEagerInit()" is called. (More about that below; see the
+// section starting with "A vault goes through a few stages of life".)
+//
 // What if you need to destroy all of your singletons?  Say, some of
 // your singletons manage threads, but you need to fork?  Or your unit
 // test wants to clean up all global state?  Then you can call
@@ -89,6 +109,7 @@
 #include <folly/Memory.h>
 #include <folly/RWSpinLock.h>
 #include <folly/Demangle.h>
+#include <folly/Executor.h>
 #include <folly/io/async/Request.h>
 
 #include <algorithm>
@@ -98,6 +119,7 @@
 #include <condition_variable>
 #include <string>
 #include <unordered_map>
+#include <unordered_set>
 #include <functional>
 #include <typeinfo>
 #include <typeindex>
@@ -197,6 +219,8 @@ class SingletonHolderBase {
 
   virtual TypeDescriptor type() = 0;
   virtual bool hasLiveInstance() = 0;
+  virtual void createInstance() = 0;
+  virtual bool creationStarted() = 0;
   virtual void destroyInstance() = 0;
 
  protected:
@@ -220,15 +244,15 @@ struct SingletonHolder : public SingletonHolderBase {
 
   void registerSingleton(CreateFunc c, TeardownFunc t);
   void registerSingletonMock(CreateFunc c, TeardownFunc t);
-  virtual TypeDescriptor type();
-  virtual bool hasLiveInstance();
-  virtual void destroyInstance();
+  virtual TypeDescriptor type() override;
+  virtual bool hasLiveInstance() override;
+  virtual void createInstance() override;
+  virtual bool creationStarted() override;
+  virtual void destroyInstance() override;
 
  private:
   SingletonHolder(TypeDescriptor type, SingletonVault& vault);
 
-  void createInstance();
-
   enum class SingletonHolderState {
     NotRegistered,
     Dead,
@@ -246,7 +270,7 @@ struct SingletonHolder : public SingletonHolderBase {
   std::atomic<SingletonHolderState> state_{SingletonHolderState::NotRegistered};
 
   // the thread creating the singleton (only valid while creating an object)
-  std::thread::id creating_thread_;
+  std::atomic<std::thread::id> creating_thread_;
 
   // The singleton itself and related functions.
 
@@ -308,26 +332,99 @@ class SingletonVault {
     singletons_[entry->type()] = entry;
   }
 
+  /**
+   * Called by `Singleton<T>.shouldEagerInit()` to ensure the instance
+   * is built when registrationComplete() is called; see that method
+   * for more info.
+   */
+  void addEagerInitSingleton(detail::SingletonHolderBase* entry) {
+    RWSpinLock::ReadHolder rh(&stateMutex_);
+
+    stateCheck(SingletonVaultState::Running);
+
+    if (UNLIKELY(registrationComplete_)) {
+      throw std::logic_error(
+        "Registering for eager-load after registrationComplete().");
+    }
+
+    RWSpinLock::ReadHolder rhMutex(&mutex_);
+    CHECK_THROW(singletons_.find(entry->type()) != singletons_.end(),
+                std::logic_error);
+
+    RWSpinLock::UpgradedHolder wh(&mutex_);
+    eagerInitSingletons_.insert(entry);
+  }
+
   // Mark registration is complete; no more singletons can be
-  // registered at this point.
-  void registrationComplete() {
+  // registered at this point.  Kicks off eagerly-initialized singletons
+  // (if requested; default behavior is to do so).
+  void registrationComplete(bool autoStartEagerInit = true) {
     RequestContext::saveContext();
     std::atexit([](){ SingletonVault::singleton()->destroyInstances(); });
 
-    RWSpinLock::WriteHolder wh(&stateMutex_);
+    {
+      RWSpinLock::WriteHolder wh(&stateMutex_);
 
-    stateCheck(SingletonVaultState::Running);
+      stateCheck(SingletonVaultState::Running);
 
-    if (type_ == Type::Strict) {
-      for (const auto& p: singletons_) {
-        if (p.second->hasLiveInstance()) {
-          throw std::runtime_error(
-            "Singleton created before registration was complete.");
+      if (type_ == Type::Strict) {
+        for (const auto& p: singletons_) {
+          if (p.second->hasLiveInstance()) {
+            throw std::runtime_error(
+              "Singleton created before registration was complete.");
+          }
         }
       }
+
+      registrationComplete_ = true;
+    }
+
+    if (autoStartEagerInit) {
+      startEagerInit();
+    }
+  }
+
+ /**
+  * If eagerInitExecutor_ is non-nullptr (default is nullptr) then
+  * schedule eager singletons' initializations through it.
+  * Otherwise, initializes them synchronously, in a loop.
+  */
+  void startEagerInit() {
+    std::unordered_set<detail::SingletonHolderBase*> singletonSet;
+    {
+      RWSpinLock::ReadHolder rh(&stateMutex_);
+      stateCheck(SingletonVaultState::Running);
+      if (UNLIKELY(!registrationComplete_)) {
+        throw std::logic_error(
+          "registrationComplete() not yet called");
+      }
+      singletonSet = eagerInitSingletons_;  // copy set of pointers
     }
 
-    registrationComplete_ = true;
+    auto *exe = eagerInitExecutor_;  // default value is nullptr
+    for (auto *single : singletonSet) {
+      if (exe) {
+        eagerInitExecutor_->add([single] {
+          if (!single->creationStarted()) {
+            single->createInstance();
+          }
+        });
+      } else {
+        single->createInstance();
+      }
+    }
+  }
+
+  /**
+   * Provide an executor through which startEagerInit would run tasks.
+   * If there are several singletons which may be independently initialized,
+   * and their construction takes long, they could possibly be run in parallel
+   * to cut down on startup time.  Unusual; default (synchronous initialization
+   * in a loop) is probably fine for most use cases, and most apps can most
+   * likely avoid using this.
+   */
+  void setEagerInitExecutor(folly::Executor *exe) {
+    eagerInitExecutor_ = exe;
   }
 
   // Destroy all singletons; when complete, the vault can't create
@@ -417,6 +514,8 @@ class SingletonVault {
 
   mutable folly::RWSpinLock mutex_;
   SingletonMap singletons_;
+  std::unordered_set<detail::SingletonHolderBase*> eagerInitSingletons_;
+  folly::Executor* eagerInitExecutor_{nullptr};
   std::vector<detail::TypeDescriptor> creation_order_;
   SingletonVaultState state_{SingletonVaultState::Running};
   bool registrationComplete_{false};
@@ -484,6 +583,30 @@ class Singleton {
     vault->registerSingleton(&getEntry());
   }
 
+  /**
+   * Should be instantiated as soon as "registrationComplete()" is called.
+   * Singletons are usually lazy-loaded (built on-demand) but for those which
+   * are known to be needed, to avoid the potential lag for objects that take
+   * long to construct during runtime, there is an option to make sure these
+   * are built up-front.
+   *
+   * Use like:
+   *   Singleton<Foo> gFooInstance = Singleton<Foo>(...).shouldEagerInit();
+   *
+   * Or alternately, define the singleton as usual, and say
+   *   gFooInstance.shouldEagerInit()
+   *
+   * at some point prior to calling registrationComplete().
+   * Then registrationComplete can be called (by default it will kick off
+   * init of the eager singletons); alternately, you can use
+   * startEagerInit().
+   */
+  Singleton& shouldEagerInit() {
+    auto vault = SingletonVault::singleton<VaultTag>();
+    vault->addEagerInitSingleton(&getEntry());
+    return *this;
+  }
+
   /**
   * Construct and inject a mock singleton which should be used only from tests.
   * Unlike regular singletons which are initialized once per process lifetime,

folly/test/SingletonTest.cpp

@@ -17,11 +17,13 @@
 #include <thread>
 
 #include <folly/Singleton.h>
+#include <folly/io/async/EventBase.h>
 
 #include <folly/Benchmark.h>
 
 #include <glog/logging.h>
 #include <gtest/gtest.h>
+#include <boost/thread/barrier.hpp>
 
 using namespace folly;
 
@@ -482,6 +484,129 @@ TEST(Singleton, SingletonConcurrencyStress) {
   }
 }
 
+namespace {
+struct EagerInitSyncTag {};
+}
+template <typename T, typename Tag = detail::DefaultTag>
+using SingletonEagerInitSync = Singleton<T, Tag, EagerInitSyncTag>;
+TEST(Singleton, SingletonEagerInitSync) {
+  auto& vault = *SingletonVault::singleton<EagerInitSyncTag>();
+  bool didEagerInit = false;
+  auto sing = SingletonEagerInitSync<std::string>(
+                  [&] {didEagerInit = true; return new std::string("foo"); })
+              .shouldEagerInit();
+  vault.registrationComplete();
+  EXPECT_TRUE(didEagerInit);
+  sing.get_weak();  // (avoid compile error complaining about unused var 'sing')
+}
+
+namespace {
+struct EagerInitAsyncTag {};
+}
+template <typename T, typename Tag = detail::DefaultTag>
+using SingletonEagerInitAsync = Singleton<T, Tag, EagerInitAsyncTag>;
+TEST(Singleton, SingletonEagerInitAsync) {
+  auto& vault = *SingletonVault::singleton<EagerInitAsyncTag>();
+  bool didEagerInit = false;
+  auto sing = SingletonEagerInitAsync<std::string>(
+                  [&] {didEagerInit = true; return new std::string("foo"); })
+              .shouldEagerInit();
+  folly::EventBase eb;
+  vault.setEagerInitExecutor(&eb);
+  vault.registrationComplete();
+  EXPECT_FALSE(didEagerInit);
+  eb.loop();
+  EXPECT_TRUE(didEagerInit);
+  sing.get_weak();  // (avoid compile error complaining about unused var 'sing')
+}
+
+namespace {
+class TestEagerInitParallelExecutor : public folly::Executor {
+ public:
+  explicit TestEagerInitParallelExecutor(const size_t threadCount) {
+    eventBases_.reserve(threadCount);
+    threads_.reserve(threadCount);
+    for (size_t i = 0; i < threadCount; i++) {
+      eventBases_.push_back(std::make_shared<folly::EventBase>());
+      auto eb = eventBases_.back();
+      threads_.emplace_back(std::make_shared<std::thread>(
+          [eb] { eb->loopForever(); }));
+    }
+  }
+
+  virtual ~TestEagerInitParallelExecutor() override {
+    for (auto eb : eventBases_) {
+      eb->runInEventBaseThread([eb] { eb->terminateLoopSoon(); });
+    }
+    for (auto thread : threads_) {
+      thread->join();
+    }
+  }
+
+  virtual void add(folly::Func func) override {
+    const auto index = (counter_ ++) % eventBases_.size();
+    eventBases_[index]->add(func);
+  }
+
+ private:
+  std::vector<std::shared_ptr<folly::EventBase>> eventBases_;
+  std::vector<std::shared_ptr<std::thread>> threads_;
+  std::atomic<size_t> counter_ {0};
+};
+}  // namespace
+
+namespace {
+struct EagerInitParallelTag {};
+}
+template <typename T, typename Tag = detail::DefaultTag>
+using SingletonEagerInitParallel = Singleton<T, Tag, EagerInitParallelTag>;
+TEST(Singleton, SingletonEagerInitParallel) {
+  const static size_t kIters = 1000;
+  const static size_t kThreads = 20;
+
+  std::atomic<size_t> initCounter;
+
+  auto& vault = *SingletonVault::singleton<EagerInitParallelTag>();
+
+  auto sing = SingletonEagerInitParallel<std::string>(
+                  [&] {++initCounter; return new std::string(""); })
+              .shouldEagerInit();
+
+  for (size_t i = 0; i < kIters; i++) {
+    SCOPE_EXIT {
+      // clean up each time
+      vault.destroyInstances();
+      vault.reenableInstances();
+    };
+
+    initCounter.store(0);
+
+    {
+      boost::barrier barrier(kThreads + 1);
+      TestEagerInitParallelExecutor exe(kThreads);
+      vault.setEagerInitExecutor(&exe);
+      vault.registrationComplete(false);
+
+      EXPECT_EQ(0, initCounter.load());
+
+      for (size_t j = 0; j < kThreads; j++) {
+        exe.add([&] {
+          barrier.wait();
+          vault.startEagerInit();
+          barrier.wait();
+        });
+      }
+
+      barrier.wait();  // to await all threads' readiness
+      barrier.wait();  // to await all threads' completion
+    }
+
+    EXPECT_EQ(1, initCounter.load());
+
+    sing.get_weak();  // (avoid compile error complaining about unused var)
+  }
+}
+
 // Benchmarking a normal singleton vs a Meyers singleton vs a Folly
 // singleton.  Meyers are insanely fast, but (hopefully) Folly
 // singletons are fast "enough."