Commit 2adb6eb0 authored by Vladimir Slaykovskiy's avatar Vladimir Slaykovskiy Committed by Facebook Github Bot 0

Reduce footprint of ScribeClient

Summary:
I've found all code paths that contributed to the number of threads in new ScribeClient. Here's a list of flags that reduce the number of thread:
  --default_tls_thread_count=1 # was 32
  --sr2_event_base_pool_size=1
  --has_thrift_dispatcher_reporter=0 # creates a thrift server with a few extra threads
  --observer_manager_pool_size=1 # default is 4

Optimizations in this diff:
- Don't initialize OBCClient object if it is not used
- Added FLAG_observer_manager_pool_size to control size of the pool in ObserverManager

Currently OBC counters are switched off globally in ScribeCliean, it means that new code path that creates SR instances is disabled, but eventually we plan to switch it on. Clients that don't use SR and want to keep their RSS/threads profile low, should pass flags listed above. I'll announce this in ScribeUsers group as well.

Differential Revision: D3870704

fbshipit-source-id: 0efad6b3dc43c072ab11cac7e9461c09532ea11c
parent a610249b
......@@ -25,8 +25,12 @@ FOLLY_TLS bool ObserverManager::inManagerThread_{false};
FOLLY_TLS ObserverManager::DependencyRecorder::Dependencies*
ObserverManager::DependencyRecorder::currentDependencies_{nullptr};
DEFINE_int32(
observer_manager_pool_size,
4,
"How many internal threads ObserverManager should use");
namespace {
constexpr size_t kCurrentThreadPoolSize{4};
constexpr size_t kCurrentQueueSize{10 * 1024};
constexpr size_t kNextQueueSize{10 * 1024};
}
......@@ -34,7 +38,11 @@ constexpr size_t kNextQueueSize{10 * 1024};
class ObserverManager::CurrentQueue {
public:
CurrentQueue() : queue_(kCurrentQueueSize) {
for (size_t i = 0; i < kCurrentThreadPoolSize; ++i) {
if (FLAGS_observer_manager_pool_size < 1) {
LOG(ERROR) << "--observer_manager_pool_size should be >= 1";
FLAGS_observer_manager_pool_size = 1;
}
for (int32_t i = 0; i < FLAGS_observer_manager_pool_size; ++i) {
threads_.emplace_back([&]() {
ObserverManager::inManagerThread_ = true;
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment