Commit 68a78d99 authored by Brandon Schlinker's avatar Brandon Schlinker Committed by Facebook GitHub Bot

TcpInfo, an abstraction layer to capture and access TCP state

Summary:
An cross-platform abstraction layer for capturing current TCP and congestion control state.

Fetches information from four different resources:
- `TCP_INFO` (state of TCP)
- `TCP_CONGESTION` (name of congestion control algorithm)
- `TCP_CC_INFO` (details for a given congestion control algorithm)
- `SIOCOUTQ`/`SIOCINQ` (socket buffers)

`TcpInfo` is designed to solve two problems:

**(1) `TcpInfo` unblocks use of the latest `tcp_info` struct and related structs.**

As of 2020, the `tcp_info` struct shipped with glibc (sysdeps/gnu/netinet/tcp.h) has not been updated since 2007 due to compatibility concerns; see commit titled "Update netinet/tcp.h from Linux 4.18" in glibc repository. This creates scenarios where fields that have long been available in the kernel ABI cannot be accessed.

Even if glibc does eventually update the `tcp_info` shipped, we don't want to be limited to their update cycle. `TcpInfo` solves this in two ways:
   - First, `TcpInfoTypes.h` contains a copy of the latest `tcp_info` struct for Linux, and `TcpInfo` always uses this struct for lookups; this decouples `TcpInfo` from glibc's / the platform's `tcp_info`.
   - Second, `TcpInfo` determines which fields in the struct are populated (and thus valid) based on the number of bytes the kernel ABI copies into the struct during the corresponding getsockopt operation. When a field is accessed through `getFieldAsOptUInt64` or through an accessor, `TcpInfo` returns an empty optional if the field is unavailable at run-time.

In this manner, `TcpInfo` enables the latest struct to always be used while ensuring that programs can determine at runtime which fields are available for use --- there's no risk of a program assuming that a field is valid when it in fact was never initialized/set by the ABI.

**(2) `TcpInfo` abstracts platform differences while still keeping details available.**

The `tcp_info` structure varies significantly between Apple and Linux. `TcpInfo` exposes a subset of `tcp_info` and other fields through accessors that hide these differences, and reduce potential errors (e.g., Apple stores srtt in milliseconds, Linux stores in microseconds, `TcpInfo::srtt` does the conversions needed to always return in microseconds). When a field is unavailable on a platform, the accessor returns an empty optional.

In parallel, the underlying structures remain accessible and can be safely accessed through the appropriate `getFieldAsOptUInt64(...)`. This enables platform-specific code to have full access to the underlying structure while also benefiting from `TcpInfo`'s knowledge of whether a given field was populated by the ABI at run-time.

Support for FreeBSD will be added in a subsequent diff.

Differential Revision: D22134355

fbshipit-source-id: accae8762aa88c187cc473b8121df901c6ffb456
parent 16ac56e4
/*
* Copyright (c) Facebook, Inc. and its affiliates.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <glog/logging.h>
#include <folly/String.h>
#include <folly/net/TcpInfo.h>
#include <folly/portability/Sockets.h>
#if defined(__linux__)
#include <linux/sockios.h>
#include <sys/ioctl.h>
#endif
namespace folly {
namespace tcpinfo {
namespace {
constexpr std::array<
folly::StringPiece,
static_cast<std::underlying_type_t<TcpInfo::CongestionControlName>>(
TcpInfo::CongestionControlName::NumCcTypes)>
kCcNames{
"UNKNOWN",
"CUBIC",
"BIC",
"DCTCP",
"DCTCP_RENO",
"BBR",
"RENO",
"DCTCP_CUBIC",
"VEGAS"};
static_assert(
kCcNames.size() ==
static_cast<std::underlying_type_t<TcpInfo::CongestionControlName>>(
TcpInfo::CongestionControlName::NumCcTypes),
"kCcNames and folly::TcpInfo::CongestionControlName should have "
"the same number of values");
} // namespace
using ms = std::chrono::milliseconds;
using us = std::chrono::microseconds;
using namespace tcpinfo;
TcpInfo::IoctlDispatcher* TcpInfo::IoctlDispatcher::getDefaultInstance() {
static TcpInfo::IoctlDispatcher dispatcher = {};
return &dispatcher;
}
int TcpInfo::IoctlDispatcher::ioctl(int fd, unsigned long request, void* argp) {
#if defined(__linux__)
return ::ioctl(fd, request, argp);
#else
return -1; // no cross platform for ioctl operations
#endif
}
Expected<TcpInfo, std::errc> TcpInfo::initFromFd(
const NetworkSocket& fd,
const LookupOptions& options,
netops::Dispatcher& netopsDispatcher,
IoctlDispatcher& ioctlDispatcher) {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::makeUnexpected(std::errc::invalid_argument);
#else
if (NetworkSocket() == fd) {
return folly::makeUnexpected(std::errc::invalid_argument);
}
// try to get TCP_INFO
TcpInfo info = {};
socklen_t len = sizeof(TcpInfo::tcp_info);
info.tcpInfoBytesRead = netopsDispatcher.getsockopt(
fd,
IPPROTO_TCP,
folly::tcpinfo::tcp_info_sock_opt,
(void*)&info.tcpInfo,
&len);
if (info.tcpInfoBytesRead < 0) {
int errnoCopy = errno;
VLOG(4) << "Error calling getsockopt(): " << folly::errnoStr(errnoCopy);
return folly::makeUnexpected(static_cast<std::errc>(errnoCopy));
}
// if enabled, try to get information about the congestion control algo
if (options.getCcInfo) {
initCcInfoFromFd(fd, info, netopsDispatcher);
}
// if enabled, try to get memory buffers
if (options.getMemInfo) {
initMemInfoFromFd(fd, info, ioctlDispatcher);
}
return info;
#endif
}
/**
*
* Accessor definitions.
*
*/
Optional<std::chrono::microseconds> TcpInfo::minrtt() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
const auto ptr = getFieldAsPtr(&tcp_info::tcpi_min_rtt);
return (ptr) ? us(*CHECK_NOTNULL(ptr)) : folly::Optional<us>();
#elif defined(__APPLE__)
return folly::none;
#else
return folly::none;
#endif
}
Optional<std::chrono::microseconds> TcpInfo::srtt() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
const auto ptr = getFieldAsPtr(&tcp_info::tcpi_rtt);
return (ptr) ? us(*CHECK_NOTNULL(ptr)) // __linux__ stores in us
: folly::Optional<us>();
#elif defined(__APPLE__)
const auto ptr = getFieldAsPtr(&tcp_info::tcpi_srtt);
return (ptr) ? us(ms(*CHECK_NOTNULL(ptr))) // __APPLE__ stores in ms
: folly::Optional<us>();
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::bytesSent() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_info::tcpi_bytes_sent);
#elif defined(__APPLE__)
return getFieldAsOptUInt64(&tcp_info::tcpi_txbytes);
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::bytesReceived() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_info::tcpi_bytes_received);
#elif defined(__APPLE__)
return getFieldAsOptUInt64(&tcp_info::tcpi_rxbytes);
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::bytesRetransmitted() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_info::tcpi_bytes_retrans);
#elif defined(__APPLE__)
return getFieldAsOptUInt64(&tcp_info::tcpi_txretransmitbytes);
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::bytesNotSent() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_info::tcpi_notsent_bytes);
#elif defined(__APPLE__)
return folly::none;
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::packetsSent() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_info::tcpi_segs_out);
#elif defined(__APPLE__)
return getFieldAsOptUInt64(&tcp_info::tcpi_txpackets);
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::packetsWithDataSent() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_info::tcpi_data_segs_out);
#elif defined(__APPLE__)
return folly::none;
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::packetsReceived() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_info::tcpi_segs_in);
#elif defined(__APPLE__)
return getFieldAsOptUInt64(&tcp_info::tcpi_rxpackets);
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::packetsWithDataReceived() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_info::tcpi_data_segs_in);
#elif defined(__APPLE__)
return folly::none;
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::packetsRetransmitted() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_info::tcpi_total_retrans);
#elif defined(__APPLE__)
return getFieldAsOptUInt64(&tcp_info::tcpi_txretransmitpackets);
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::packetsInFlight() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
// tcp_packets_in_flight is defined in kernel as:
// (tp->packets_out - tcp_left_out(tp) + tp->retrans_out)
//
// tcp_left_out is defined as:
// (tp->sacked_out + tp->lost_out)
//
// mapping from tcp_info fields to tcp_sock fields:
// info->tcpi_unacked = tp->packets_out;
// info->tcpi_retrans = tp->retrans_out;
// info->tcpi_sacked = tp->sacked_out;
// info->tcpi_lost = tp->lost_out;
const auto packetsOutOpt = getFieldAsOptUInt64(&tcp_info::tcpi_unacked);
const auto retransOutOpt = getFieldAsOptUInt64(&tcp_info::tcpi_retrans);
const auto sackedOutOpt = getFieldAsOptUInt64(&tcp_info::tcpi_sacked);
const auto lostOutOpt = getFieldAsOptUInt64(&tcp_info::tcpi_lost);
if (packetsOutOpt && retransOutOpt && sackedOutOpt && lostOutOpt) {
return (*packetsOutOpt - (*sackedOutOpt + *lostOutOpt) + *retransOutOpt);
}
return folly::none;
#elif defined(__APPLE__)
return folly::none;
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::cwndInPackets() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_info::tcpi_snd_cwnd);
#elif defined(__APPLE__)
return getFieldAsOptUInt64(&tcp_info::tcpi_snd_cwnd);
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::cwndInBytes() const {
auto cwndInPacketsOpt = cwndInPackets();
auto mssOpt = mss();
if (cwndInPacketsOpt || mssOpt) {
return cwndInPacketsOpt.value() * mssOpt.value();
}
return folly::none;
}
Optional<uint64_t> TcpInfo::ssthresh() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_info::tcpi_snd_ssthresh);
#elif defined(__APPLE__)
return getFieldAsOptUInt64(&tcp_info::tcpi_snd_ssthresh);
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::mss() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
return tcpInfo.tcpi_snd_mss;
#elif defined(__APPLE__)
return tcpInfo.tcpi_maxseg;
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::deliveryRateBitsPerSecond() const {
return bytesPerSecondToBitsPerSecond(deliveryRateBytesPerSecond());
}
Optional<uint64_t> TcpInfo::deliveryRateBytesPerSecond() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_info::tcpi_delivery_rate);
#elif defined(__APPLE__)
return folly::none;
#else
return folly::none;
#endif
}
Optional<bool> TcpInfo::deliveryRateAppLimited() const {
#ifndef FOLLY_HAVE_TCP_INFO
return folly::none;
#elif defined(__linux__)
// have to check if delivery rate is available for two reasons
// (1) can't use getTcpInfoFieldAsPtr on bit-field
// (2) tcpi_delivery_rate_app_limited was added in earlier part of tcp_info
// to take advantage of 1-byte gap; must check if we have the delivery
// rate field to determine if the app limited field is available
if (deliveryRateBytesPerSecond().has_value()) {
return tcpInfo.tcpi_delivery_rate_app_limited;
}
return folly::none;
#elif defined(__APPLE__)
return folly::none;
#else
return folly::none;
#endif
}
Optional<std::string> TcpInfo::ccNameRaw() const {
#ifndef FOLLY_HAVE_TCP_CC_INFO
return folly::none;
#elif defined(__linux__)
return maybeCcNameRaw;
#elif defined(__APPLE__)
return folly::none;
#else
return folly::none;
#endif
}
Optional<TcpInfo::CongestionControlName> TcpInfo::ccNameEnum() const {
#ifndef FOLLY_HAVE_TCP_CC_INFO
return folly::none;
#elif defined(__linux__)
return maybeCcEnum;
#elif defined(__APPLE__)
return folly::none;
#else
return folly::none;
#endif
}
Optional<folly::StringPiece> TcpInfo::ccNameEnumAsStr() const {
const auto maybeCcNameEnum = ccNameEnum();
if (!maybeCcNameEnum.has_value()) {
return folly::none;
}
const auto ccEnumAsInt =
static_cast<std::underlying_type_t<CongestionControlName>>(
maybeCcNameEnum.value());
CHECK_GE(
static_cast<std::underlying_type_t<TcpInfo::CongestionControlName>>(
TcpInfo::CongestionControlName::NumCcTypes),
ccEnumAsInt);
CHECK_GE(kCcNames.size(), ccEnumAsInt);
return kCcNames[ccEnumAsInt];
}
Optional<uint64_t> TcpInfo::bbrBwBitsPerSecond() const {
return bytesPerSecondToBitsPerSecond(bbrBwBytesPerSecond());
}
Optional<uint64_t> TcpInfo::bbrBwBytesPerSecond() const {
#ifndef FOLLY_HAVE_TCP_CC_INFO
return folly::none;
#elif defined(__linux__)
auto bbrBwLoOpt = getFieldAsOptUInt64(&tcp_bbr_info::bbr_bw_lo);
auto bbrBwHiOpt = getFieldAsOptUInt64(&tcp_bbr_info::bbr_bw_hi);
if (bbrBwLoOpt && bbrBwHiOpt) {
return ((int64_t)*bbrBwHiOpt << 32) + *bbrBwLoOpt;
}
return folly::none;
#elif defined(__APPLE__)
return folly::none;
#else
return folly::none;
#endif
}
Optional<std::chrono::microseconds> TcpInfo::bbrMinrtt() const {
#ifndef FOLLY_HAVE_TCP_CC_INFO
return folly::none;
#elif defined(__linux__)
auto opt = getFieldAsOptUInt64(&tcp_bbr_info::bbr_min_rtt);
return (opt) ? us(*opt) : folly::Optional<us>();
#elif defined(__APPLE__)
return folly::none;
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::bbrPacingGain() const {
#ifndef FOLLY_HAVE_TCP_CC_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_bbr_info::bbr_pacing_gain);
#elif defined(__APPLE__)
return folly::none;
#else
return folly::none;
#endif
}
Optional<uint64_t> TcpInfo::bbrCwndGain() const {
#ifndef FOLLY_HAVE_TCP_CC_INFO
return folly::none;
#elif defined(__linux__)
return getFieldAsOptUInt64(&tcp_bbr_info::bbr_cwnd_gain);
#elif defined(__APPLE__)
return folly::none;
#else
return folly::none;
#endif
}
Optional<size_t> TcpInfo::sendBufInUseBytes() const {
return maybeSendBufInUseBytes;
}
Optional<size_t> TcpInfo::recvBufInUseBytes() const {
return maybeRecvBufInUseBytes;
}
void TcpInfo::initCcInfoFromFd(
const NetworkSocket& fd,
TcpInfo& wrappedInfo,
netops::Dispatcher& netopsDispatcher) {
#ifndef FOLLY_HAVE_TCP_CC_INFO
return; // platform not supported
#elif defined(__linux__)
if (NetworkSocket() == fd) {
return;
}
// identification strings returned by Linux Kernel for TCP_CONGESTION
static constexpr auto kLinuxCcNameStrReno = "reno";
static constexpr auto kLinuxCcNameStrCubic = "cubic";
static constexpr auto kLinuxCcNameStrBic = "bic";
static constexpr auto kLinuxCcNameStrBbr = "bbr";
static constexpr auto kLinuxCcNameStrVegas = "vegas";
static constexpr auto kLinuxCcNameStrDctcp = "dctcp";
static constexpr auto kLinuxCcNameStrDctcpReno = "dctcp_reno";
static constexpr auto kLinuxCcNameStrDctcpCubic = "dctcp_cubic";
std::array<char, (unsigned int)kLinuxTcpCaNameMax> tcpCongestion{{0}};
socklen_t optlen = tcpCongestion.size();
if (netopsDispatcher.getsockopt(
fd, IPPROTO_TCP, TCP_CONGESTION, tcpCongestion.data(), &optlen) < 0) {
VLOG(4) << "Error calling getsockopt(): " << folly::errnoStr(errno);
return;
}
{
auto ccStr = std::string(tcpCongestion.data());
if (ccStr == kLinuxCcNameStrReno) {
wrappedInfo.maybeCcEnum = CongestionControlName::RENO;
} else if (ccStr == kLinuxCcNameStrCubic) {
wrappedInfo.maybeCcEnum = CongestionControlName::CUBIC;
} else if (ccStr == kLinuxCcNameStrBic) {
wrappedInfo.maybeCcEnum = CongestionControlName::BIC;
} else if (ccStr == kLinuxCcNameStrBbr) {
wrappedInfo.maybeCcEnum = CongestionControlName::BBR;
} else if (ccStr == kLinuxCcNameStrVegas) {
wrappedInfo.maybeCcEnum = CongestionControlName::VEGAS;
} else if (ccStr == kLinuxCcNameStrDctcp) {
wrappedInfo.maybeCcEnum = CongestionControlName::DCTCP;
} else if (ccStr == kLinuxCcNameStrDctcpReno) {
wrappedInfo.maybeCcEnum = CongestionControlName::DCTCP_RENO;
} else if (ccStr == kLinuxCcNameStrDctcpCubic) {
wrappedInfo.maybeCcEnum = CongestionControlName::DCTCP_CUBIC;
} else {
wrappedInfo.maybeCcEnum = CongestionControlName::UNKNOWN;
}
wrappedInfo.maybeCcNameRaw.emplace(std::move(ccStr));
}
// get TCP_CC_INFO if supported for the congestion control algorithm
switch (wrappedInfo.maybeCcEnum.value_or(CongestionControlName::UNKNOWN)) {
case CongestionControlName::UNKNOWN:
case CongestionControlName::RENO:
case CongestionControlName::CUBIC:
case CongestionControlName::BIC:
return; // no TCP_CC_INFO for these congestion controls, exit out
case CongestionControlName::BBR:
case CongestionControlName::VEGAS:
case CongestionControlName::DCTCP:
case CongestionControlName::DCTCP_RENO:
case CongestionControlName::DCTCP_CUBIC:
break; // supported, proceed
case CongestionControlName::NumCcTypes:
LOG(FATAL) << "CongestionControlName::NumCcTypes is not a valid CC type";
}
tcpinfo::tcp_cc_info ccInfo = {};
socklen_t len = sizeof(tcpinfo::tcp_cc_info);
int bytesRead = netopsDispatcher.getsockopt(
fd, IPPROTO_TCP, TCP_CC_INFO, (void*)&ccInfo, &len);
if (bytesRead < 0) {
int errnoCopy = errno;
VLOG(4) << "Error calling getsockopt(): " << folly::errnoStr(errnoCopy);
return;
}
wrappedInfo.maybeCcInfo = ccInfo;
wrappedInfo.tcpCcInfoBytesRead = bytesRead;
#else
return;
#endif
}
void TcpInfo::initMemInfoFromFd(
const NetworkSocket& fd,
TcpInfo& wrappedInfo,
IoctlDispatcher& ioctlDispatcher) {
#if defined(__linux__)
if (NetworkSocket() == fd) {
return;
}
size_t val = 0;
if (ioctlDispatcher.ioctl(fd.toFd(), SIOCOUTQ, &val) == 0) {
wrappedInfo.maybeSendBufInUseBytes = val;
}
if (ioctlDispatcher.ioctl(fd.toFd(), SIOCINQ, &val) == 0) {
wrappedInfo.maybeRecvBufInUseBytes = val;
}
#endif
}
} // namespace tcpinfo
} // namespace folly
/*
* Copyright (c) Facebook, Inc. and its affiliates.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#pragma once
#include <chrono>
#include <folly/Expected.h>
#include <folly/FixedString.h>
#include <folly/Optional.h>
#include <folly/net/NetOpsDispatcher.h>
#include <folly/net/NetworkSocket.h>
#include <folly/net/TcpInfoTypes.h>
namespace folly {
namespace tcpinfo {
/**
* Structure specifying options for TcpInfo::initFromFd.
*/
struct LookupOptions {
// On supported platforms, whether to fetch the name of the congestion
// control algorithm and any information exposed via TCP_CC_INFO.
bool getCcInfo{true};
// On supported platforms, whether to fetch socket buffer utilization.
bool getMemInfo{true};
};
/**
* Abstraction layer for capturing current TCP and congestion control state.
*
* Fetches information from four different resources:
* - TCP_INFO (state of TCP)
* - TCP_CONGESTION (name of congestion control algorithm)
* - TCP_CC_INFO (details for a given congestion control algorithm)
* - SIOCOUTQ/SIOCINQ (socket buffers)
*
* To save space, the structure only allocates fields for which the underlying
* platform supports lookups. For instance, if TCP_CONGESTION is not supported,
* then the TcpInfo structure will not have fields to store the CC name / type.
*
* This abstraction layer solves two problems:
*
* 1. It unblocks use of the latest tcp_info structs and related structs.
*
* As of 2020, the tcp_info struct shipped with glibc
* (sysdeps/gnu/netinet/tcp.h) has not been updated since 2007 due to
* compatibility concerns; see commit titled "Update netinet/tcp.h from
* Linux 4.18" in glibc repository. This creates scenarios where fields
* that have long been available in the kernel ABI cannot be accessed.
* Even if glibc does eventually update the tcp_info shipped, we don't
* want to be limited to their update cycle.
*
* folly::TcpInfo solves this in two ways:
* - First, TcpInfoTypes.h contains a copy of the latest tcp_info struct
* for Linux, and folly::TcpInfo always uses this struct for lookups;
* this decouples TcpInfo from glibc's / the platform's tcp_info.
*
* - Second, folly::TcpInfo determines which fields in the struct the
* kernel ABI populated (and thus which fields are valid) based on the
* number of bytes the kernel ABI copies into the struct during the
* corresponding getsockopt operation. When a field is accessed
* through getFieldAsOptUInt64 or through an accessor, folly::TcpInfo
* returns an empty optional if the field is unavailable at run-time.
* In this manner, folly::TcpInfo enables the latest struct to always
* be used while ensuring that programs can determine at run-time
* which fields are available for use --- there's no risk of a program
* assuming that a field is valid when it in fact was never
* initialized/set by the ABI.
*
* 2. Eliminates platform differences while still retaining details.
*
* The tcp_info structure varies significantly between Apple and Linux.
* folly::TcpInfo exposes a subset of tcp_info and other fields through
* accessors that abstract these differences, and reduce potential errors
* (e.g., Apple stores srtt in milliseconds, Linux stores in microseconds).
* When a field is unavailable on a platform, the accessor returns an empty
* optional.
*
* In parallel, the underlying structures remain accessible and can be
* safely accessed through the appropriate getFieldAsOptUInt64(...). This
* enables platform-specific code to have full access to the structure
* while also benefiting from folly::TcpInfo's knowledge of whether a
* given field was populated by the ABI at run-time.
*/
struct TcpInfo {
enum class CongestionControlName {
UNKNOWN = 0,
CUBIC = 1,
BIC = 2,
DCTCP = 3,
DCTCP_RENO = 4,
BBR = 5,
RENO = 6,
DCTCP_CUBIC = 7,
VEGAS = 8,
NumCcTypes,
};
/**
* Dispatcher that enables calls to ioctl to be intercepted for tests.
*
* Also enables ioctl calls to be disabled for unsupported platforms.
*/
class IoctlDispatcher {
public:
static IoctlDispatcher* getDefaultInstance();
virtual int ioctl(int fd, unsigned long request, void* argp);
protected:
IoctlDispatcher() = default;
virtual ~IoctlDispatcher() = default;
};
/**
* Initializes and returns TcpInfo struct.
*
* @param fd Socket file descriptor encapsulated in NetworkSocket.
* @param options Options for lookup.
* @param netopsDispatcher Dispatcher to use for netops calls;
* facilitates mocking during unit tests.
* @param ioctlDispatcher Dispatcher to use for ioctl calls;
* facilitates mocking during unit tests.
*/
static Expected<TcpInfo, std::errc> initFromFd(
const NetworkSocket& fd,
const LookupOptions& options = LookupOptions(),
netops::Dispatcher& netopsDispatcher =
*netops::Dispatcher::getDefaultInstance(),
IoctlDispatcher& ioctlDispatcher =
*IoctlDispatcher::getDefaultInstance());
/**
* Accessors for tcp_info.
*
* These accessors are always available regardless of platform, they return
* folly::none if the underlying field is unavailable.
*/
Optional<std::chrono::microseconds> minrtt() const;
Optional<std::chrono::microseconds> srtt() const;
Optional<uint64_t> bytesSent() const;
Optional<uint64_t> bytesReceived() const;
Optional<uint64_t> bytesRetransmitted() const;
Optional<uint64_t> bytesNotSent() const;
Optional<uint64_t> packetsSent() const;
Optional<uint64_t> packetsWithDataSent() const;
Optional<uint64_t> packetsReceived() const;
Optional<uint64_t> packetsWithDataReceived() const;
Optional<uint64_t> packetsRetransmitted() const;
Optional<uint64_t> packetsInFlight() const;
Optional<uint64_t> cwndInPackets() const;
Optional<uint64_t> cwndInBytes() const;
Optional<uint64_t> ssthresh() const;
Optional<uint64_t> mss() const;
Optional<uint64_t> deliveryRateBitsPerSecond() const;
Optional<uint64_t> deliveryRateBytesPerSecond() const;
Optional<bool> deliveryRateAppLimited() const;
/**
* Accessors for congestion control information.
*
* These accessors are always available regardless of platform, they return
* folly::none if the underlying field is unavailable.
*/
Optional<std::string> ccNameRaw() const;
Optional<CongestionControlName> ccNameEnum() const;
Optional<folly::StringPiece> ccNameEnumAsStr() const;
Optional<uint64_t> bbrBwBitsPerSecond() const;
Optional<uint64_t> bbrBwBytesPerSecond() const;
Optional<std::chrono::microseconds> bbrMinrtt() const;
Optional<uint64_t> bbrPacingGain() const;
Optional<uint64_t> bbrCwndGain() const;
/**
* Accessors for memory info information.
*
* These accessors are always available regardless of platform, they return
* folly::none if the underlying field is unavailable.
*/
Optional<size_t> sendBufInUseBytes() const;
Optional<size_t> recvBufInUseBytes() const;
private:
/**
* Returns pointer containing requested field from passed struct.
*
* If field is unavailable, returns a nullptr.
*/
template <typename T1, typename T2>
static const T1* getFieldAsPtr(
const T2& tgtStruct, const int tgtBytesRead, T1 T2::*field) {
if (field != nullptr && tgtBytesRead > 0 &&
getFieldOffset(field) + sizeof(tgtStruct.*field) <=
(unsigned long)tgtBytesRead) {
return &(tgtStruct.*field);
}
return nullptr;
}
/**
* Get the offset of a field in a struct.
*
* Requires that struct (T1) be POD (else undefined behavior).
*
* Alternative to `offsetof` that enables us to avoid use of macro.
* Approach from:
* https://gist.github.com/graphitemaster/494f21190bb2c63c5516
*/
template <typename T1, typename T2>
static size_t constexpr getFieldOffset(T1 T2::*field) {
static_assert(std::is_pod_v<T1>);
constexpr T2 dummy{};
return size_t(&(dummy.*field)) - size_t(&dummy);
}
/**
* Converts an optional containing a value with units bits/s to byte/s.
*
* If input optional is empty, returns empty.
*/
static Optional<uint64_t> bytesPerSecondToBitsPerSecond(
const Optional<uint64_t>& bytesPerSecondOpt) {
if (bytesPerSecondOpt.hasValue()) {
return bytesPerSecondOpt.value() * 8;
}
return folly::none;
}
/**
* Initializes the congestion control fields in passed WrappedTcpInfo.
*/
static void initCcInfoFromFd(
const NetworkSocket& fd,
TcpInfo& tcpInfo,
netops::Dispatcher& netopsDispatcher =
*netops::Dispatcher::getDefaultInstance());
/**
* Initializes the socker buffer memory fields in passed WrappedTcpInfo.
*/
static void initMemInfoFromFd(
const NetworkSocket& fd,
TcpInfo& tcpInfo,
IoctlDispatcher& ioctlDispatcher =
*IoctlDispatcher::getDefaultInstance());
#if defined(FOLLY_HAVE_TCP_INFO)
public:
using tcp_info = folly::tcpinfo::tcp_info;
/**
* Returns pointer containing requested field from tcp_info struct.
*
* The tcp_info struct type is platform specific (and thus templated). If
* no tcp_info is supprted for this platform, this accessor is unavailable.
*
* To access tcp_info fields without needing to consider platform specifics,
* use accessors, such as bytesSent().
*/
template <typename T1>
const T1* getFieldAsPtr(T1 tcp_info::*field) const {
return getFieldAsPtr(tcpInfo, tcpInfoBytesRead, field);
}
/**
* Returns Optional<uint64_t> containing requested field from tcp_info struct.
*
* The tcp_info struct type is platform specific (and thus templated). If
* no tcp_info is supprted for this platform, this accessor is unavailable.
*
* To access tcp_info fields without needing to consider platform specifics,
* use accessors such as bytesSent().
*/
template <typename T1>
folly::Optional<uint64_t> getFieldAsOptUInt64(T1 tcp_info::*field) const {
if (auto ptr = getFieldAsPtr(field)) {
return *ptr;
}
return folly::none;
}
private:
// tcp_info struct for this system, may be backported.
tcp_info tcpInfo = {};
// number of bytes read during getsockopt for TCP_INFO.
int tcpInfoBytesRead{0};
#endif
#if defined(FOLLY_HAVE_TCP_CC_INFO)
public:
using tcp_cc_info = folly::tcpinfo::tcp_cc_info;
// TCP_CA_NAME_MAX from <net/tcp.h> (Linux) or <netinet/tcp.h> (FreeBSD)
static constexpr socklen_t kLinuxTcpCaNameMax = 16;
/**
* Returns Optional<uint64_t> containing requested field from BBR struct.
*
* If tcp_cc_info is unavailable or the congestion controller is not BBR,
* returns folly::none for all fields.
*
* To access tcp_cc_info fields without needing to consider platform
* specifics, use accessors such as bbrBwBitsPerSecond().
*/
template <typename T1>
folly::Optional<uint64_t> getFieldAsOptUInt64(
T1 tcpinfo::tcp_bbr_info::*field) const {
if (maybeCcInfo.has_value() && ccNameEnum() == CongestionControlName::BBR) {
return getFieldAsOptUInt64(maybeCcInfo.value().bbr, field);
}
return folly::none;
}
/**
* Returns Optional<uint64_t> containing requested field from Vegas struct.
*
* If tcp_cc_info is unavailable or the congestion controller is not Vegas,
* returns folly::none for all fields.
*/
template <typename T1>
folly::Optional<uint64_t> getFieldAsOptUInt64(
T1 tcpinfo::tcpvegas_info::*field) const {
if (maybeCcInfo.hasValue() &&
ccNameEnum() == CongestionControlName::VEGAS) {
return getFieldAsOptUInt64(maybeCcInfo.value().vegas, field);
}
return folly::none;
}
/**
* Returns Optional<uint64_t> containing requested field from DCTCP struct.
*
* If tcp_cc_info is unavailable or the congestion controller is not DCTCP,
* returns folly::none for all fields.
*/
template <typename T1>
const folly::Optional<uint64_t> getFieldAsOptUInt64(
T1 tcpinfo::tcp_dctcp_info::*field) const {
if (maybeCcInfo.has_value() &&
(ccNameEnum() == CongestionControlName::DCTCP ||
ccNameEnum() == CongestionControlName::DCTCP_CUBIC ||
ccNameEnum() == CongestionControlName::DCTCP_RENO)) {
return getFieldAsOptUInt64(maybeCcInfo.value().dctcp, field);
}
return folly::none;
}
private:
/**
* Returns Optional<uint64_t> containing requested field from tcp_cc_info.
*
* The tcp_cc_info struct type is platform specific (and thus templated). If
* no tcp_cc_info is supprted for this platform, this accessor is unavailable.
*
* To access tcp_cc_info fields without needing to consider platform
* specifics, use accessors such as bbrBwBitsPerSecond().
*/
template <typename T1, typename T2>
folly::Optional<uint64_t> getFieldAsOptUInt64(
const T2& tgtStruct, T1 T2::*field) const {
if (field != nullptr && tcpCcInfoBytesRead > 0 &&
getFieldOffset(field) + sizeof(tgtStruct.*field) <=
(unsigned long)tcpCcInfoBytesRead) {
return folly::Optional<uint64_t>(tgtStruct.*field);
}
return folly::none;
}
// raw congestion control algorithm name returned by underlying platform
folly::Optional<std::string> maybeCcNameRaw;
// enum for congestion control algorithm.
folly::Optional<CongestionControlName> maybeCcEnum;
// additional information from congestion control algorithm.
folly::Optional<tcp_cc_info> maybeCcInfo;
// number of bytes read during getsockopt for TCP_CC_INFO.
//
// if TCP_CC_INFO was not able to be fetched, will be 0.
int tcpCcInfoBytesRead{0};
#endif // #if defined(FOLLY_HAVE_TCP_CC_INFO)
private:
folly::Optional<size_t> maybeSendBufInUseBytes;
folly::Optional<size_t> maybeRecvBufInUseBytes;
};
} // namespace tcpinfo
} // namespace folly
/*
* Copyright (c) Facebook, Inc. and its affiliates.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#pragma once
#if defined(__linux__) || defined(__APPLE__)
#include <netinet/tcp.h>
#include <sys/types.h>
#include <unistd.h>
#endif
#if defined(__linux__)
#include <asm/types.h>
#endif
namespace folly {
namespace tcpinfo {
/**
*
* tcp_info structures.
*
*/
#if defined(__linux__)
#define FOLLY_HAVE_TCP_INFO 1
const int tcp_info_sock_opt = TCP_INFO;
/**
* tcp_info as of kernel 5.7.
*
* The kernel ABI is fully backwards compatible. Thus, if a new structure has
* been released, this structure can (and should be) upgraded.
*
* Having a copy of the latest available structure decouples compilation from
* whatever is in the header files available to the compiler. These may be
* very outdated; see discussion of glibc below. WrappedTcpInfo determines which
* fields are supported by the kernel running on the machine based on the size
* of the tcp_info object returned and exposes only those fields.
*/
struct tcp_info {
__u8 tcpi_state;
__u8 tcpi_ca_state;
__u8 tcpi_retransmits;
__u8 tcpi_probes;
__u8 tcpi_backoff;
__u8 tcpi_options;
__u8 tcpi_snd_wscale : 4, tcpi_rcv_wscale : 4;
__u8 tcpi_delivery_rate_app_limited : 1;
__u32 tcpi_rto;
__u32 tcpi_ato;
__u32 tcpi_snd_mss;
__u32 tcpi_rcv_mss;
__u32 tcpi_unacked;
__u32 tcpi_sacked;
__u32 tcpi_lost;
__u32 tcpi_retrans;
__u32 tcpi_fackets;
/* Times. */
__u32 tcpi_last_data_sent;
__u32 tcpi_last_ack_sent; /* Not remembered, sorry. */
__u32 tcpi_last_data_recv;
__u32 tcpi_last_ack_recv;
/* Metrics. */
__u32 tcpi_pmtu;
__u32 tcpi_rcv_ssthresh;
__u32 tcpi_rtt;
__u32 tcpi_rttvar;
__u32 tcpi_snd_ssthresh;
__u32 tcpi_snd_cwnd;
__u32 tcpi_advmss;
__u32 tcpi_reordering;
__u32 tcpi_rcv_rtt;
__u32 tcpi_rcv_space;
__u32 tcpi_total_retrans;
__u64 tcpi_pacing_rate;
__u64 tcpi_max_pacing_rate;
__u64 tcpi_bytes_acked; /* RFC4898 tcpEStatsAppHCThruOctetsAcked */
__u64 tcpi_bytes_received; /* RFC4898 tcpEStatsAppHCThruOctetsReceived */
__u32 tcpi_segs_out; /* RFC4898 tcpEStatsPerfSegsOut */
__u32 tcpi_segs_in; /* RFC4898 tcpEStatsPerfSegsIn */
__u32 tcpi_notsent_bytes;
__u32 tcpi_min_rtt;
__u32 tcpi_data_segs_in; /* RFC4898 tcpEStatsDataSegsIn */
__u32 tcpi_data_segs_out; /* RFC4898 tcpEStatsDataSegsOut */
__u64 tcpi_delivery_rate;
__u64 tcpi_busy_time; /* Time (usec) busy sending data */
__u64 tcpi_rwnd_limited; /* Time (usec) limited by receive window */
__u64 tcpi_sndbuf_limited; /* Time (usec) limited by send buffer */
__u32 tcpi_delivered;
__u32 tcpi_delivered_ce;
__u64 tcpi_bytes_sent; /* RFC4898 tcpEStatsPerfHCDataOctetsOut */
__u64 tcpi_bytes_retrans; /* RFC4898 tcpEStatsPerfOctetsRetrans */
__u32 tcpi_dsack_dups; /* RFC4898 tcpEStatsStackDSACKDups */
__u32 tcpi_reord_seen; /* reordering events seen */
__u32 tcpi_rcv_ooopack; /* Out-of-order packets received */
__u32 tcpi_snd_wnd; /* peer's advertised receive window after
* scaling (bytes)
*/
};
/**
* Legacy tcp_info used to confirm backwards compatibility.
*
* We use this structure in test cases where the kernel has an older version of
* tcp_info to verify that the wrapper returns unsupported fields as empty
* optionals.
*
* This tcp_info struct is what shipped in 3.x kernels, and is still shipped
* with glibc as of 2020 in sysdeps/gnu/netinet/tcp.h. glibc has not updated the
* tcp_info struct since 2007 due to compatibility concerns; see commit titled
* "Update netinet/tcp.h from Linux 4.18" in glibc repository.
*/
struct tcp_info_legacy {
__u8 tcpi_state;
__u8 tcpi_ca_state;
__u8 tcpi_retransmits;
__u8 tcpi_probes;
__u8 tcpi_backoff;
__u8 tcpi_options;
__u8 tcpi_snd_wscale : 4, tcpi_rcv_wscale : 4;
__u32 tcpi_rto;
__u32 tcpi_ato;
__u32 tcpi_snd_mss;
__u32 tcpi_rcv_mss;
__u32 tcpi_unacked;
__u32 tcpi_sacked;
__u32 tcpi_lost;
__u32 tcpi_retrans;
__u32 tcpi_fackets;
/* Times. */
__u32 tcpi_last_data_sent;
__u32 tcpi_last_ack_sent; /* Not remembered, sorry. */
__u32 tcpi_last_data_recv;
__u32 tcpi_last_ack_recv;
/* Metrics. */
__u32 tcpi_pmtu;
__u32 tcpi_rcv_ssthresh;
__u32 tcpi_rtt;
__u32 tcpi_rttvar;
__u32 tcpi_snd_ssthresh;
__u32 tcpi_snd_cwnd;
__u32 tcpi_advmss;
__u32 tcpi_reordering;
__u32 tcpi_rcv_rtt;
__u32 tcpi_rcv_space;
__u32 tcpi_total_retrans;
};
#elif defined(__APPLE__)
#define FOLLY_HAVE_TCP_INFO 1
using tcp_info = ::tcp_connection_info;
const int tcp_info_sock_opt = TCP_CONNECTION_INFO;
#endif
/**
* extra structures used to communicate congestion control information.
*/
#if defined(__linux__) && defined(TCP_CONGESTION) && defined(TCP_CC_INFO)
#define FOLLY_HAVE_TCP_CC_INFO 1
struct tcpvegas_info {
__u32 tcpv_enabled;
__u32 tcpv_rttcnt;
__u32 tcpv_rtt;
__u32 tcpv_minrtt;
};
struct tcp_dctcp_info {
__u16 dctcp_enabled;
__u16 dctcp_ce_state;
__u32 dctcp_alpha;
__u32 dctcp_ab_ecn;
__u32 dctcp_ab_tot;
};
struct tcp_bbr_info {
/* u64 bw: max-filtered BW (app throughput) estimate in Byte per sec: */
__u32 bbr_bw_lo; /* lower 32 bits of bw */
__u32 bbr_bw_hi; /* upper 32 bits of bw */
__u32 bbr_min_rtt; /* min-filtered RTT in uSec */
__u32 bbr_pacing_gain; /* pacing gain shifted left 8 bits */
__u32 bbr_cwnd_gain; /* cwnd gain shifted left 8 bits */
};
union tcp_cc_info {
struct tcpvegas_info vegas;
struct tcp_dctcp_info dctcp;
struct tcp_bbr_info bbr;
};
#endif
} // namespace tcpinfo
} // namespace folly
/*
* Copyright (c) Facebook, Inc. and its affiliates.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <cstring>
#include <folly/net/TcpInfo.h>
#include <folly/net/test/MockNetOpsDispatcher.h>
#include <folly/portability/GMock.h>
#include <folly/portability/GTest.h>
using namespace folly;
using namespace testing;
using namespace folly::tcpinfo;
using us = std::chrono::microseconds;
const auto kTestSiocoutqVal = 10;
const auto kTestSiocinqVal = 100;
const auto kTestUnknownCcName = "coolNewCCA"; // it's cool, new, and a CCA!
// tests are only supported on Linux right now
#ifdef __linux__
#include <linux/sockios.h>
#include <sys/ioctl.h>
class TcpInfoTest : public Test {
public:
/**
* Mock to enable testing of socket buffer lookups.
*/
class MockIoctlDispatcher : public TcpInfo::IoctlDispatcher {
public:
MockIoctlDispatcher() = default;
virtual ~MockIoctlDispatcher() = default;
/**
* Configures mocked methods to forward calls to default implementation.
*/
void forwardToDefaultImpl() {
ON_CALL(*this, ioctl(testing::_, testing::_, testing::_))
.WillByDefault(
testing::Invoke([](int fd, unsigned long request, void* argp) {
return ::ioctl(fd, request, argp);
}));
}
MOCK_METHOD3(ioctl, int(int fd, unsigned long request, void* argp));
};
template <typename T1>
void setupExpectCallTcpInfo(NetworkSocket& s, const T1& tInfo) {
EXPECT_CALL(
mockNetOpsDispatcher_, getsockopt(s, IPPROTO_TCP, TCP_INFO, _, _))
.WillOnce(
WithArgs<3, 4>(Invoke([tInfo](void* optval, socklen_t* optlen) {
auto copied = std::min((unsigned int)sizeof tInfo, *optlen);
std::memcpy(optval, (void*)&tInfo, copied);
return copied;
})));
}
void setupExpectCallCcName(NetworkSocket& s, const std::string& ccName) {
EXPECT_CALL(
mockNetOpsDispatcher_,
getsockopt(
s,
IPPROTO_TCP,
TCP_CONGESTION,
NotNull(),
Pointee(Eq(TcpInfo::kLinuxTcpCaNameMax))))
.WillOnce(WithArgs<3, 4>(Invoke([ccName](
void* optval, socklen_t* optlen) {
EXPECT_THAT(optlen, Pointee(Ge(ccName.size())));
std::copy(
ccName.begin(),
ccName.end(),
((std::array<char, (unsigned int)TcpInfo::kLinuxTcpCaNameMax>*)
optval)
->data());
return ccName.size();
})));
}
void setupExpectCallCcInfo(
NetworkSocket& s, const folly::tcpinfo::tcp_cc_info& ccInfo) {
EXPECT_CALL(
mockNetOpsDispatcher_,
getsockopt(
s,
IPPROTO_TCP,
TCP_CC_INFO,
NotNull(),
Pointee(Eq(sizeof(folly::tcpinfo::tcp_cc_info)))))
.WillOnce(
WithArgs<3, 4>(Invoke([ccInfo](void* optval, socklen_t* optlen) {
auto copied = std::min((unsigned int)sizeof ccInfo, *optlen);
std::memcpy(optval, (void*)&(ccInfo), copied);
return copied;
})));
}
struct ExpectCallMemInfoConfig {
size_t siocoutq{0};
size_t siocinq{0};
};
void setupExpectCallMemInfo(
NetworkSocket& s, const ExpectCallMemInfoConfig& config) {
EXPECT_CALL(mockIoctlDispatcher_, ioctl(s.toFd(), SIOCOUTQ, testing::_))
.WillOnce(WithArgs<2>(Invoke([config](void* argp) {
size_t* val = static_cast<size_t*>(argp);
*val = config.siocoutq;
return 0;
})));
EXPECT_CALL(mockIoctlDispatcher_, ioctl(s.toFd(), SIOCINQ, testing::_))
.WillOnce(WithArgs<2>(Invoke([config](void* argp) {
size_t* val = static_cast<size_t*>(argp);
*val = config.siocinq;
return 0;
})));
}
static folly::tcpinfo::tcp_info getTestLatestTcpInfo() {
folly::tcpinfo::tcp_info tInfo = {};
tInfo.tcpi_state = 1;
tInfo.tcpi_ca_state = 2;
tInfo.tcpi_retransmits = 3;
tInfo.tcpi_probes = 4;
tInfo.tcpi_backoff = 5;
tInfo.tcpi_options = 6;
tInfo.tcpi_snd_wscale = 7;
tInfo.tcpi_rcv_wscale = 8;
tInfo.tcpi_delivery_rate_app_limited = 1;
tInfo.tcpi_rto = 9;
tInfo.tcpi_ato = 10;
tInfo.tcpi_snd_mss = 11;
tInfo.tcpi_rcv_mss = 12;
tInfo.tcpi_unacked = 113; // 113 instead of 13 for packets in flight
tInfo.tcpi_sacked = 14;
tInfo.tcpi_lost = 15;
tInfo.tcpi_retrans = 16;
tInfo.tcpi_fackets = 17;
tInfo.tcpi_last_data_sent = 18;
tInfo.tcpi_last_ack_sent = 19;
tInfo.tcpi_last_data_recv = 20;
tInfo.tcpi_last_ack_recv = 21;
tInfo.tcpi_pmtu = 22;
tInfo.tcpi_rcv_ssthresh = 23;
tInfo.tcpi_rtt = 24;
tInfo.tcpi_rttvar = 25;
tInfo.tcpi_snd_ssthresh = 26;
tInfo.tcpi_snd_cwnd = 27;
tInfo.tcpi_advmss = 28;
tInfo.tcpi_reordering = 29;
tInfo.tcpi_rcv_rtt = 30;
tInfo.tcpi_rcv_space = 31;
tInfo.tcpi_total_retrans = 32;
tInfo.tcpi_pacing_rate = 33;
tInfo.tcpi_max_pacing_rate = 34;
tInfo.tcpi_bytes_acked = 35;
tInfo.tcpi_bytes_received = 36;
tInfo.tcpi_segs_out = 37;
tInfo.tcpi_segs_in = 38;
tInfo.tcpi_notsent_bytes = 39;
tInfo.tcpi_min_rtt = 40;
tInfo.tcpi_data_segs_in = 41;
tInfo.tcpi_data_segs_out = 42;
tInfo.tcpi_delivery_rate = 43;
tInfo.tcpi_busy_time = 44;
tInfo.tcpi_rwnd_limited = 45;
tInfo.tcpi_sndbuf_limited = 46;
tInfo.tcpi_delivered = 47;
tInfo.tcpi_delivered_ce = 48;
tInfo.tcpi_bytes_sent = 49;
tInfo.tcpi_bytes_retrans = 50;
tInfo.tcpi_dsack_dups = 51;
tInfo.tcpi_reord_seen = 52;
tInfo.tcpi_rcv_ooopack = 53;
tInfo.tcpi_snd_wnd = 54;
return tInfo;
}
static folly::tcpinfo::tcp_info_legacy getTestLegacyTcpInfo() {
folly::tcpinfo::tcp_info_legacy tInfo = {};
tInfo.tcpi_state = 1;
tInfo.tcpi_ca_state = 2;
tInfo.tcpi_retransmits = 3;
tInfo.tcpi_probes = 4;
tInfo.tcpi_backoff = 5;
tInfo.tcpi_options = 6;
tInfo.tcpi_snd_wscale = 7;
tInfo.tcpi_rcv_wscale = 8;
tInfo.tcpi_rto = 9;
tInfo.tcpi_ato = 10;
tInfo.tcpi_snd_mss = 11;
tInfo.tcpi_rcv_mss = 12;
tInfo.tcpi_unacked = 113; // 113 instead of 13 for packets in flight
tInfo.tcpi_sacked = 14;
tInfo.tcpi_lost = 15;
tInfo.tcpi_retrans = 16;
tInfo.tcpi_fackets = 17;
tInfo.tcpi_last_data_sent = 18;
tInfo.tcpi_last_ack_sent = 19;
tInfo.tcpi_last_data_recv = 20;
tInfo.tcpi_last_ack_recv = 21;
tInfo.tcpi_pmtu = 22;
tInfo.tcpi_rcv_ssthresh = 23;
tInfo.tcpi_rtt = 24;
tInfo.tcpi_rttvar = 25;
tInfo.tcpi_snd_ssthresh = 26;
tInfo.tcpi_snd_cwnd = 27;
tInfo.tcpi_advmss = 28;
tInfo.tcpi_reordering = 29;
tInfo.tcpi_rcv_rtt = 30;
tInfo.tcpi_rcv_space = 31;
tInfo.tcpi_total_retrans = 32;
return tInfo;
}
static folly::tcpinfo::tcp_cc_info getTestBbrInfo() {
folly::tcpinfo::tcp_cc_info ccInfo = {};
ccInfo.bbr.bbr_bw_lo = 1;
ccInfo.bbr.bbr_bw_hi = 2;
ccInfo.bbr.bbr_min_rtt = 3;
ccInfo.bbr.bbr_pacing_gain = 4;
ccInfo.bbr.bbr_cwnd_gain = 5;
return ccInfo;
}
static folly::tcpinfo::tcp_cc_info getTestVegasInfo() {
folly::tcpinfo::tcp_cc_info ccInfo = {};
ccInfo.vegas.tcpv_enabled = 6;
ccInfo.vegas.tcpv_rttcnt = 7;
ccInfo.vegas.tcpv_rtt = 8;
ccInfo.vegas.tcpv_minrtt = 9;
return ccInfo;
}
static folly::tcpinfo::tcp_cc_info getTestDctcpInfo() {
folly::tcpinfo::tcp_cc_info ccInfo = {};
ccInfo.dctcp.dctcp_enabled = 10;
ccInfo.dctcp.dctcp_ce_state = 11;
ccInfo.dctcp.dctcp_alpha = 12;
ccInfo.dctcp.dctcp_ab_ecn = 13;
ccInfo.dctcp.dctcp_ab_tot = 14;
return ccInfo;
}
static void checkNoTcpInfo(const TcpInfo& wrappedTcpInfo) {
// check a few accessors to make sure nothing available
EXPECT_FALSE(wrappedTcpInfo.bytesRetransmitted().has_value());
EXPECT_FALSE(wrappedTcpInfo.packetsRetransmitted().has_value());
EXPECT_FALSE(wrappedTcpInfo.srtt().has_value());
EXPECT_FALSE(wrappedTcpInfo.cwndInBytes().has_value());
EXPECT_FALSE(wrappedTcpInfo.cwndInPackets().has_value());
EXPECT_FALSE(
wrappedTcpInfo
.getFieldAsOptUInt64(&folly::tcpinfo::tcp_info::tcpi_total_retrans)
.has_value());
EXPECT_FALSE(
wrappedTcpInfo
.getFieldAsOptUInt64(&folly::tcpinfo::tcp_info::tcpi_snd_cwnd)
.has_value());
}
static void checkTcpInfoAgainstLegacy(const TcpInfo& wrappedTcpInfo) {
const auto& controlTcpInfo = getTestLegacyTcpInfo();
EXPECT_FALSE(wrappedTcpInfo.minrtt());
EXPECT_EQ(us(controlTcpInfo.tcpi_rtt), wrappedTcpInfo.srtt());
EXPECT_FALSE(wrappedTcpInfo.bytesSent());
EXPECT_FALSE(wrappedTcpInfo.bytesReceived());
EXPECT_FALSE(wrappedTcpInfo.bytesRetransmitted());
EXPECT_FALSE(wrappedTcpInfo.packetsSent());
EXPECT_FALSE(wrappedTcpInfo.packetsWithDataSent());
EXPECT_FALSE(wrappedTcpInfo.packetsReceived());
EXPECT_FALSE(wrappedTcpInfo.packetsWithDataReceived());
EXPECT_EQ(
controlTcpInfo.tcpi_total_retrans,
wrappedTcpInfo.packetsRetransmitted());
EXPECT_EQ(
controlTcpInfo.tcpi_unacked + controlTcpInfo.tcpi_retrans -
(controlTcpInfo.tcpi_sacked + controlTcpInfo.tcpi_lost),
wrappedTcpInfo.packetsInFlight());
EXPECT_NE(0, wrappedTcpInfo.packetsInFlight());
EXPECT_EQ(controlTcpInfo.tcpi_snd_cwnd, wrappedTcpInfo.cwndInPackets());
EXPECT_EQ(
controlTcpInfo.tcpi_snd_cwnd * controlTcpInfo.tcpi_snd_mss,
wrappedTcpInfo.cwndInBytes());
EXPECT_EQ(controlTcpInfo.tcpi_snd_ssthresh, wrappedTcpInfo.ssthresh());
EXPECT_EQ(controlTcpInfo.tcpi_snd_mss, wrappedTcpInfo.mss());
EXPECT_FALSE(wrappedTcpInfo.deliveryRateBitsPerSecond());
EXPECT_FALSE(wrappedTcpInfo.deliveryRateBytesPerSecond());
EXPECT_FALSE(wrappedTcpInfo.deliveryRateAppLimited());
// try using getTcpInfoFieldAsOpt to get one of the older fields
// this field _should_ be available in legacy
EXPECT_EQ(
controlTcpInfo.tcpi_snd_cwnd,
wrappedTcpInfo.getFieldAsOptUInt64(
&folly::tcpinfo::tcp_info::tcpi_snd_cwnd));
// try using getTcpInfoFieldAsOpt to get one of the newer fields
// this field should _not_ be available in legacy
EXPECT_FALSE(
wrappedTcpInfo
.getFieldAsOptUInt64(&folly::tcpinfo::tcp_info::tcpi_delivery_rate)
.hasValue());
}
static void checkTcpInfoAgainstLatest(const TcpInfo& wrappedTcpInfo) {
const auto& controlTcpInfo = getTestLatestTcpInfo();
EXPECT_EQ(us(controlTcpInfo.tcpi_min_rtt), wrappedTcpInfo.minrtt());
EXPECT_EQ(us(controlTcpInfo.tcpi_rtt), wrappedTcpInfo.srtt());
EXPECT_EQ(controlTcpInfo.tcpi_bytes_sent, wrappedTcpInfo.bytesSent());
EXPECT_EQ(
controlTcpInfo.tcpi_bytes_received, wrappedTcpInfo.bytesReceived());
EXPECT_EQ(
controlTcpInfo.tcpi_bytes_retrans, wrappedTcpInfo.bytesRetransmitted());
EXPECT_EQ(controlTcpInfo.tcpi_segs_out, wrappedTcpInfo.packetsSent());
EXPECT_EQ(
controlTcpInfo.tcpi_data_segs_out,
wrappedTcpInfo.packetsWithDataSent());
EXPECT_EQ(controlTcpInfo.tcpi_segs_in, wrappedTcpInfo.packetsReceived());
EXPECT_EQ(
controlTcpInfo.tcpi_data_segs_in,
wrappedTcpInfo.packetsWithDataReceived());
EXPECT_EQ(
controlTcpInfo.tcpi_total_retrans,
wrappedTcpInfo.packetsRetransmitted());
EXPECT_EQ(
controlTcpInfo.tcpi_unacked + controlTcpInfo.tcpi_retrans -
(controlTcpInfo.tcpi_sacked + controlTcpInfo.tcpi_lost),
wrappedTcpInfo.packetsInFlight());
EXPECT_NE(0, wrappedTcpInfo.packetsInFlight());
EXPECT_EQ(controlTcpInfo.tcpi_snd_cwnd, wrappedTcpInfo.cwndInPackets());
EXPECT_EQ(
controlTcpInfo.tcpi_snd_cwnd * controlTcpInfo.tcpi_snd_mss,
wrappedTcpInfo.cwndInBytes());
EXPECT_EQ(controlTcpInfo.tcpi_snd_ssthresh, wrappedTcpInfo.ssthresh());
EXPECT_EQ(controlTcpInfo.tcpi_snd_mss, wrappedTcpInfo.mss());
EXPECT_EQ(
controlTcpInfo.tcpi_delivery_rate * 8,
wrappedTcpInfo.deliveryRateBitsPerSecond());
EXPECT_EQ(
controlTcpInfo.tcpi_delivery_rate,
wrappedTcpInfo.deliveryRateBytesPerSecond());
EXPECT_EQ(
controlTcpInfo.tcpi_delivery_rate_app_limited,
wrappedTcpInfo.deliveryRateAppLimited());
// try using getFieldAsOptUInt64 directly
EXPECT_EQ(
controlTcpInfo.tcpi_delivery_rate,
wrappedTcpInfo.getFieldAsOptUInt64(
&folly::tcpinfo::tcp_info::tcpi_delivery_rate));
}
static void checkNoCcNameType(const TcpInfo& wrappedTcpInfo) {
EXPECT_FALSE(wrappedTcpInfo.ccNameRaw().has_value());
EXPECT_FALSE(wrappedTcpInfo.ccNameEnum().has_value());
EXPECT_FALSE(wrappedTcpInfo.ccNameEnumAsStr().has_value());
}
static void checkNoCcInfo(const TcpInfo& wrappedTcpInfo) {
// should get false for all three types
EXPECT_FALSE(wrappedTcpInfo.bbrCwndGain().has_value());
EXPECT_FALSE(
wrappedTcpInfo.getFieldAsOptUInt64(&tcp_bbr_info::bbr_pacing_gain)
.has_value());
EXPECT_FALSE(wrappedTcpInfo.getFieldAsOptUInt64(&tcpvegas_info::tcpv_rtt)
.has_value());
EXPECT_FALSE(
wrappedTcpInfo.getFieldAsOptUInt64(&tcp_dctcp_info::dctcp_alpha)
.has_value());
}
static void checkCcFieldsAgainstBbr(const TcpInfo& wrappedTcpInfo) {
EXPECT_EQ("BBR", wrappedTcpInfo.ccNameEnumAsStr());
EXPECT_EQ(TcpInfo::CongestionControlName::BBR, wrappedTcpInfo.ccNameEnum());
const auto controlCcInfo = getTestBbrInfo().bbr;
EXPECT_EQ(
controlCcInfo.bbr_pacing_gain,
wrappedTcpInfo.getFieldAsOptUInt64(&tcp_bbr_info::bbr_pacing_gain));
const uint64_t bbrBwBytesPerSecond =
(uint64_t(controlCcInfo.bbr_bw_hi) << 32) + controlCcInfo.bbr_bw_lo;
EXPECT_EQ(bbrBwBytesPerSecond, wrappedTcpInfo.bbrBwBytesPerSecond());
EXPECT_EQ(bbrBwBytesPerSecond * 8, wrappedTcpInfo.bbrBwBitsPerSecond());
EXPECT_EQ(us(controlCcInfo.bbr_min_rtt), wrappedTcpInfo.bbrMinrtt());
EXPECT_EQ(controlCcInfo.bbr_pacing_gain, wrappedTcpInfo.bbrPacingGain());
EXPECT_EQ(controlCcInfo.bbr_cwnd_gain, wrappedTcpInfo.bbrCwndGain());
// try using getFieldAsOptUInt64 directly to get one of the BBR fields
EXPECT_EQ(
controlCcInfo.bbr_pacing_gain,
wrappedTcpInfo.getFieldAsOptUInt64(&tcp_bbr_info::bbr_pacing_gain));
// no CC info for the other types
EXPECT_FALSE(wrappedTcpInfo.getFieldAsOptUInt64(&tcpvegas_info::tcpv_rtt));
EXPECT_FALSE(
wrappedTcpInfo.getFieldAsOptUInt64(&tcp_dctcp_info::dctcp_alpha));
}
static void checkCcFieldsAgainstVegas(const TcpInfo& wrappedTcpInfo) {
EXPECT_EQ("VEGAS", wrappedTcpInfo.ccNameEnumAsStr());
EXPECT_EQ(
TcpInfo::CongestionControlName::VEGAS, wrappedTcpInfo.ccNameEnum());
const auto controlCcInfo = getTestVegasInfo().vegas;
EXPECT_EQ(
controlCcInfo.tcpv_rtt,
wrappedTcpInfo.getFieldAsOptUInt64(&tcpvegas_info::tcpv_rtt));
// no CC info for the other types
EXPECT_FALSE(wrappedTcpInfo.bbrCwndGain().has_value());
EXPECT_FALSE(
wrappedTcpInfo.getFieldAsOptUInt64(&tcp_bbr_info::bbr_pacing_gain)
.has_value());
EXPECT_FALSE(
wrappedTcpInfo.getFieldAsOptUInt64(&tcp_dctcp_info::dctcp_alpha)
.has_value());
}
static void checkCcFieldsAgainstDctcp(
const TcpInfo& wrappedTcpInfo,
const TcpInfo::CongestionControlName dctcpType) {
switch (dctcpType) {
case TcpInfo::CongestionControlName::DCTCP:
EXPECT_EQ("DCTCP", wrappedTcpInfo.ccNameEnumAsStr());
break;
case TcpInfo::CongestionControlName::DCTCP_RENO:
EXPECT_EQ("DCTCP_RENO", wrappedTcpInfo.ccNameEnumAsStr());
break;
case TcpInfo::CongestionControlName::DCTCP_CUBIC:
EXPECT_EQ("DCTCP_CUBIC", wrappedTcpInfo.ccNameEnumAsStr());
break;
case TcpInfo::CongestionControlName::UNKNOWN:
case TcpInfo::CongestionControlName::RENO:
case TcpInfo::CongestionControlName::CUBIC:
case TcpInfo::CongestionControlName::BIC:
case TcpInfo::CongestionControlName::BBR:
case TcpInfo::CongestionControlName::VEGAS:
case TcpInfo::CongestionControlName::NumCcTypes:
FAIL();
}
EXPECT_EQ(dctcpType, wrappedTcpInfo.ccNameEnum());
const auto controlCcInfo = getTestDctcpInfo().dctcp;
EXPECT_EQ(
controlCcInfo.dctcp_alpha,
wrappedTcpInfo.getFieldAsOptUInt64(&tcp_dctcp_info::dctcp_alpha));
EXPECT_EQ(
controlCcInfo.dctcp_enabled,
wrappedTcpInfo.getFieldAsOptUInt64(&tcp_dctcp_info::dctcp_enabled));
// no CC info for the other types
EXPECT_FALSE(wrappedTcpInfo.bbrCwndGain().has_value());
EXPECT_FALSE(
wrappedTcpInfo.getFieldAsOptUInt64(&tcp_bbr_info::bbr_pacing_gain)
.has_value());
EXPECT_FALSE(wrappedTcpInfo.getFieldAsOptUInt64(&tcpvegas_info::tcpv_rtt)
.has_value());
}
static void checkNoMemoryInfo(const TcpInfo& wrappedTcpInfo) {
EXPECT_FALSE(wrappedTcpInfo.sendBufInUseBytes().has_value()); // siocoutq
EXPECT_FALSE(wrappedTcpInfo.recvBufInUseBytes().has_value()); // siocinq
}
protected:
StrictMock<folly::netops::test::MockDispatcher> mockNetOpsDispatcher_;
StrictMock<MockIoctlDispatcher> mockIoctlDispatcher_;
};
TEST_F(TcpInfoTest, LegacyStruct) {
NetworkSocket s(0);
setupExpectCallTcpInfo(s, getTestLegacyTcpInfo());
auto wrappedTcpInfoExpect = TcpInfo::initFromFd(
s,
LookupOptions{.getCcInfo = false, .getMemInfo = false},
mockNetOpsDispatcher_);
ASSERT_TRUE(wrappedTcpInfoExpect.hasValue());
const auto& wrappedTcpInfo = wrappedTcpInfoExpect.value();
checkTcpInfoAgainstLegacy(wrappedTcpInfo);
// no CC name/type or info; no memory info
checkNoCcNameType(wrappedTcpInfo);
checkNoCcInfo(wrappedTcpInfo);
checkNoMemoryInfo(wrappedTcpInfo);
}
TEST_F(TcpInfoTest, LatestStruct) {
NetworkSocket s(0);
setupExpectCallTcpInfo(s, getTestLatestTcpInfo());
auto wrappedTcpInfoExpect = TcpInfo::initFromFd(
s,
LookupOptions{.getCcInfo = false, .getMemInfo = false},
mockNetOpsDispatcher_);
ASSERT_TRUE(wrappedTcpInfoExpect.hasValue());
const auto& wrappedTcpInfo = wrappedTcpInfoExpect.value();
checkTcpInfoAgainstLatest(wrappedTcpInfo);
// no CC name/type or info; no memory info
checkNoCcNameType(wrappedTcpInfo);
checkNoCcInfo(wrappedTcpInfo);
checkNoMemoryInfo(wrappedTcpInfo);
}
TEST_F(TcpInfoTest, LatestStructWithCcInfo) {
NetworkSocket s(0);
setupExpectCallTcpInfo(s, getTestLatestTcpInfo());
setupExpectCallCcName(s, "bbr");
setupExpectCallCcInfo(s, getTestBbrInfo());
auto wrappedTcpInfoExpect = TcpInfo::initFromFd(
s,
LookupOptions{.getCcInfo = true, .getMemInfo = false},
mockNetOpsDispatcher_,
mockIoctlDispatcher_);
ASSERT_TRUE(wrappedTcpInfoExpect.hasValue());
const auto& wrappedTcpInfo = wrappedTcpInfoExpect.value();
checkTcpInfoAgainstLatest(wrappedTcpInfo);
checkCcFieldsAgainstBbr(wrappedTcpInfo);
// no memory info
checkNoMemoryInfo(wrappedTcpInfo);
}
TEST_F(TcpInfoTest, LatestStructWithMemInfo) {
NetworkSocket s(0);
setupExpectCallTcpInfo(s, getTestLatestTcpInfo());
setupExpectCallMemInfo(
s,
ExpectCallMemInfoConfig{
.siocoutq = kTestSiocoutqVal, .siocinq = kTestSiocinqVal});
auto wrappedTcpInfoExpect = TcpInfo::initFromFd(
s,
LookupOptions{.getCcInfo = false, .getMemInfo = true},
mockNetOpsDispatcher_,
mockIoctlDispatcher_);
ASSERT_TRUE(wrappedTcpInfoExpect.hasValue());
const auto& wrappedTcpInfo = wrappedTcpInfoExpect.value();
checkTcpInfoAgainstLatest(wrappedTcpInfo);
EXPECT_EQ(kTestSiocoutqVal, wrappedTcpInfo.sendBufInUseBytes()); // siocoutq
EXPECT_EQ(kTestSiocinqVal, wrappedTcpInfo.recvBufInUseBytes()); // siocinq
// no CC name/type or info
checkNoCcNameType(wrappedTcpInfo);
checkNoCcInfo(wrappedTcpInfo);
}
TEST_F(TcpInfoTest, LatestStructWithCcInfoAndMemInfo) {
NetworkSocket s(0);
setupExpectCallTcpInfo(s, getTestLatestTcpInfo());
setupExpectCallCcName(s, "bbr");
setupExpectCallCcInfo(s, getTestBbrInfo());
setupExpectCallMemInfo(
s,
ExpectCallMemInfoConfig{
.siocoutq = kTestSiocoutqVal, .siocinq = kTestSiocinqVal});
auto wrappedTcpInfoExpect = TcpInfo::initFromFd(
s,
LookupOptions{.getCcInfo = true, .getMemInfo = true},
mockNetOpsDispatcher_,
mockIoctlDispatcher_);
ASSERT_TRUE(wrappedTcpInfoExpect.hasValue());
const auto& wrappedTcpInfo = wrappedTcpInfoExpect.value();
checkTcpInfoAgainstLatest(wrappedTcpInfo);
checkCcFieldsAgainstBbr(wrappedTcpInfo);
EXPECT_EQ(kTestSiocoutqVal, wrappedTcpInfo.sendBufInUseBytes()); // siocoutq
EXPECT_EQ(kTestSiocinqVal, wrappedTcpInfo.recvBufInUseBytes()); // siocinq
}
TEST_F(TcpInfoTest, LatestStructWithCcInfoAndMemInfoUnknownCc) {
NetworkSocket s(0);
setupExpectCallTcpInfo(s, getTestLatestTcpInfo());
setupExpectCallCcName(s, kTestUnknownCcName);
setupExpectCallMemInfo(
s,
ExpectCallMemInfoConfig{
.siocoutq = kTestSiocoutqVal, .siocinq = kTestSiocinqVal});
auto wrappedTcpInfoExpect = TcpInfo::initFromFd(
s,
LookupOptions{.getCcInfo = true, .getMemInfo = true},
mockNetOpsDispatcher_,
mockIoctlDispatcher_);
ASSERT_TRUE(wrappedTcpInfoExpect.hasValue());
const auto& wrappedTcpInfo = wrappedTcpInfoExpect.value();
checkTcpInfoAgainstLatest(wrappedTcpInfo);
EXPECT_EQ(kTestSiocoutqVal, wrappedTcpInfo.sendBufInUseBytes()); // siocoutq
EXPECT_EQ(kTestSiocinqVal, wrappedTcpInfo.recvBufInUseBytes()); // siocinq
// the CC name and enum should be set, but there should be no other CC info
EXPECT_EQ(kTestUnknownCcName, wrappedTcpInfo.ccNameRaw());
EXPECT_EQ("UNKNOWN", wrappedTcpInfo.ccNameEnumAsStr());
EXPECT_EQ(
TcpInfo::CongestionControlName::UNKNOWN, wrappedTcpInfo.ccNameEnum());
checkNoCcInfo(wrappedTcpInfo);
}
TEST_F(TcpInfoTest, FailUninitializedSocket) {
NetworkSocket s;
auto wrappedTcpInfoExpect = TcpInfo::initFromFd(
s,
LookupOptions{.getCcInfo = true, .getMemInfo = true},
mockNetOpsDispatcher_,
mockIoctlDispatcher_);
ASSERT_FALSE(wrappedTcpInfoExpect.hasValue()); // complete failure
}
TEST_F(TcpInfoTest, FailTcpInfo) {
NetworkSocket s(0);
EXPECT_CALL(mockNetOpsDispatcher_, getsockopt(s, IPPROTO_TCP, TCP_INFO, _, _))
.WillOnce(Return(-1));
auto wrappedTcpInfoExpect = TcpInfo::initFromFd(
s,
LookupOptions{.getCcInfo = true, .getMemInfo = true},
mockNetOpsDispatcher_,
mockIoctlDispatcher_);
ASSERT_FALSE(wrappedTcpInfoExpect.hasValue()); // complete failure
}
TEST_F(TcpInfoTest, FailCcName) {
NetworkSocket s(0);
setupExpectCallTcpInfo(s, getTestLatestTcpInfo());
setupExpectCallMemInfo(
s,
ExpectCallMemInfoConfig{
.siocoutq = kTestSiocoutqVal, .siocinq = kTestSiocinqVal});
// ensure that we try to fetch TCP name via TCP_CONGESTION
// return -1 when trying to get CC name to mimic socket error state
EXPECT_CALL(
mockNetOpsDispatcher_,
getsockopt(
s,
IPPROTO_TCP,
TCP_CONGESTION,
NotNull(),
Pointee(Eq(TcpInfo::kLinuxTcpCaNameMax))))
.WillOnce(Return(-1));
auto wrappedTcpInfoExpect = TcpInfo::initFromFd(
s,
LookupOptions{.getCcInfo = true, .getMemInfo = true},
mockNetOpsDispatcher_,
mockIoctlDispatcher_);
ASSERT_TRUE(wrappedTcpInfoExpect.hasValue());
const auto& wrappedTcpInfo = wrappedTcpInfoExpect.value();
checkTcpInfoAgainstLatest(wrappedTcpInfo);
EXPECT_EQ(kTestSiocoutqVal, wrappedTcpInfo.sendBufInUseBytes()); // siocoutq
EXPECT_EQ(kTestSiocinqVal, wrappedTcpInfo.recvBufInUseBytes()); // siocinq
// no CC name/type or info due to failed lookup for TCP_CONGESTION
checkNoCcNameType(wrappedTcpInfo);
checkNoCcInfo(wrappedTcpInfo);
}
TEST_F(TcpInfoTest, FailCcInfo) {
NetworkSocket s(0);
setupExpectCallTcpInfo(s, getTestLatestTcpInfo());
setupExpectCallCcName(s, "bbr");
setupExpectCallMemInfo(
s,
ExpectCallMemInfoConfig{
.siocoutq = kTestSiocoutqVal, .siocinq = kTestSiocinqVal});
// ensure that we try to fetch TCP info via TCP_CC_INFO
// return -1 when trying to get CC name to mimic socket error state
EXPECT_CALL(
mockNetOpsDispatcher_,
getsockopt(
s,
IPPROTO_TCP,
TCP_CC_INFO,
NotNull(),
Pointee(Eq(sizeof(TcpInfo::tcp_cc_info)))))
.WillOnce(Return(-1));
auto wrappedTcpInfoExpect = TcpInfo::initFromFd(
s,
LookupOptions{.getCcInfo = true, .getMemInfo = true},
mockNetOpsDispatcher_,
mockIoctlDispatcher_);
ASSERT_TRUE(wrappedTcpInfoExpect.hasValue());
const auto& wrappedTcpInfo = wrappedTcpInfoExpect.value();
checkTcpInfoAgainstLatest(wrappedTcpInfo);
EXPECT_EQ(kTestSiocoutqVal, wrappedTcpInfo.sendBufInUseBytes()); // siocoutq
EXPECT_EQ(kTestSiocinqVal, wrappedTcpInfo.recvBufInUseBytes()); // siocinq
// the CC name and enum should be set, but there should be no other CC info,
// despite how this is BBR (and thus triggered a TCP_CC_INFO lookup) given
// that the TCP_CC_INFO lookup failed
EXPECT_EQ("BBR", wrappedTcpInfo.ccNameEnumAsStr());
EXPECT_EQ(TcpInfo::CongestionControlName::BBR, wrappedTcpInfo.ccNameEnum());
checkNoCcInfo(wrappedTcpInfo);
}
TEST_F(TcpInfoTest, FailSiocoutq) {
NetworkSocket s(0);
setupExpectCallTcpInfo(s, getTestLatestTcpInfo());
setupExpectCallCcName(s, "bbr");
setupExpectCallCcInfo(s, getTestBbrInfo());
EXPECT_CALL(mockIoctlDispatcher_, ioctl(s.toFd(), SIOCOUTQ, testing::_))
.WillOnce(Return(-1));
EXPECT_CALL(mockIoctlDispatcher_, ioctl(s.toFd(), SIOCINQ, testing::_))
.WillOnce(WithArgs<2>(Invoke([toSet = kTestSiocinqVal](void* argp) {
size_t* val = static_cast<size_t*>(argp);
*val = toSet;
return 0;
})));
auto wrappedTcpInfoExpect = TcpInfo::initFromFd(
s,
LookupOptions{.getCcInfo = true, .getMemInfo = true},
mockNetOpsDispatcher_,
mockIoctlDispatcher_);
ASSERT_TRUE(wrappedTcpInfoExpect.hasValue());
const auto& wrappedTcpInfo = wrappedTcpInfoExpect.value();
checkTcpInfoAgainstLatest(wrappedTcpInfo);
checkCcFieldsAgainstBbr(wrappedTcpInfo);
// should have SIOCINQ, but no SIOCOUTQ given failure during lookup
EXPECT_FALSE(wrappedTcpInfo.sendBufInUseBytes().has_value()); // siocoutq
EXPECT_EQ(kTestSiocinqVal, wrappedTcpInfo.recvBufInUseBytes()); // siocinq
}
TEST_F(TcpInfoTest, FailSiocinq) {
NetworkSocket s(0);
setupExpectCallTcpInfo(s, getTestLatestTcpInfo());
setupExpectCallCcName(s, "bbr");
setupExpectCallCcInfo(s, getTestBbrInfo());
EXPECT_CALL(mockIoctlDispatcher_, ioctl(s.toFd(), SIOCOUTQ, testing::_))
.WillOnce(WithArgs<2>(Invoke([toSet = kTestSiocoutqVal](void* argp) {
size_t* val = static_cast<size_t*>(argp);
*val = toSet;
return 0;
})));
EXPECT_CALL(mockIoctlDispatcher_, ioctl(s.toFd(), SIOCINQ, testing::_))
.WillOnce(Return(-1));
auto wrappedTcpInfoExpect = TcpInfo::initFromFd(
s,
LookupOptions{.getCcInfo = true, .getMemInfo = true},
mockNetOpsDispatcher_,
mockIoctlDispatcher_);
ASSERT_TRUE(wrappedTcpInfoExpect.hasValue());
const auto& wrappedTcpInfo = wrappedTcpInfoExpect.value();
checkTcpInfoAgainstLatest(wrappedTcpInfo);
checkCcFieldsAgainstBbr(wrappedTcpInfo);
// should have SIOCOUTQ, but no SIOCINQ given failure during lookup
EXPECT_EQ(kTestSiocoutqVal, wrappedTcpInfo.sendBufInUseBytes()); // siocoutq
EXPECT_FALSE(wrappedTcpInfo.recvBufInUseBytes().has_value()); // siocinq
}
struct TcpInfoTestCcParamRawStrAndEnum {
TcpInfoTestCcParamRawStrAndEnum(
std::string ccNameRaw, TcpInfo::CongestionControlName ccNameEnum)
: ccNameRaw(std::move(ccNameRaw)), ccNameEnum(ccNameEnum) {}
const std::string ccNameRaw; // raw name returned by kernel
const TcpInfo::CongestionControlName ccNameEnum; // expected
};
class TcpInfoTestCcParam
: public TcpInfoTest,
public testing::WithParamInterface<TcpInfoTestCcParamRawStrAndEnum> {
public:
static std::vector<TcpInfoTestCcParamRawStrAndEnum> getTestingValues() {
return std::vector<TcpInfoTestCcParamRawStrAndEnum>{
{kTestUnknownCcName, TcpInfo::CongestionControlName::UNKNOWN},
{"cubic", TcpInfo::CongestionControlName::CUBIC},
{"bic", TcpInfo::CongestionControlName::BIC},
{"dctcp", TcpInfo::CongestionControlName::DCTCP},
{"dctcp_reno", TcpInfo::CongestionControlName::DCTCP_RENO},
{"bbr", TcpInfo::CongestionControlName::BBR},
{"reno", TcpInfo::CongestionControlName::RENO},
{"dctcp_cubic", TcpInfo::CongestionControlName::DCTCP_CUBIC},
{"vegas", TcpInfo::CongestionControlName::VEGAS}};
}
};
INSTANTIATE_TEST_CASE_P(
CcParamTests,
TcpInfoTestCcParam,
::testing::ValuesIn(TcpInfoTestCcParam::getTestingValues()));
TEST_P(TcpInfoTestCcParam, FetchAllAndCheck) {
const auto testParams = GetParam();
NetworkSocket s(0);
setupExpectCallTcpInfo(s, getTestLatestTcpInfo());
setupExpectCallMemInfo(
s,
ExpectCallMemInfoConfig{
.siocoutq = kTestSiocoutqVal, .siocinq = kTestSiocinqVal});
// setup CC specifics for the test
setupExpectCallCcName(s, testParams.ccNameRaw);
switch (testParams.ccNameEnum) {
case TcpInfo::CongestionControlName::UNKNOWN:
case TcpInfo::CongestionControlName::RENO:
case TcpInfo::CongestionControlName::CUBIC:
case TcpInfo::CongestionControlName::BIC:
break; // CC_INFO not supported
case TcpInfo::CongestionControlName::BBR:
setupExpectCallCcInfo(s, getTestBbrInfo());
break;
case TcpInfo::CongestionControlName::VEGAS:
setupExpectCallCcInfo(s, getTestVegasInfo());
break;
case TcpInfo::CongestionControlName::DCTCP:
case TcpInfo::CongestionControlName::DCTCP_RENO:
case TcpInfo::CongestionControlName::DCTCP_CUBIC:
setupExpectCallCcInfo(s, getTestDctcpInfo());
break;
case TcpInfo::CongestionControlName::NumCcTypes:
FAIL();
}
auto wrappedTcpInfoExpect = TcpInfo::initFromFd(
s,
LookupOptions{.getCcInfo = true, .getMemInfo = true},
mockNetOpsDispatcher_,
mockIoctlDispatcher_);
ASSERT_TRUE(wrappedTcpInfoExpect.hasValue());
const auto& wrappedTcpInfo = wrappedTcpInfoExpect.value();
checkTcpInfoAgainstLatest(wrappedTcpInfo);
EXPECT_EQ(kTestSiocoutqVal, wrappedTcpInfo.sendBufInUseBytes()); // siocoutq
EXPECT_EQ(kTestSiocinqVal, wrappedTcpInfo.recvBufInUseBytes()); // siocinq
// check CC information
EXPECT_EQ(testParams.ccNameRaw, wrappedTcpInfo.ccNameRaw());
EXPECT_EQ(testParams.ccNameEnum, wrappedTcpInfo.ccNameEnum());
switch (testParams.ccNameEnum) {
case TcpInfo::CongestionControlName::UNKNOWN:
EXPECT_EQ("UNKNOWN", wrappedTcpInfo.ccNameEnumAsStr());
break;
case TcpInfo::CongestionControlName::RENO:
EXPECT_EQ("RENO", wrappedTcpInfo.ccNameEnumAsStr());
break;
case TcpInfo::CongestionControlName::CUBIC:
EXPECT_EQ("CUBIC", wrappedTcpInfo.ccNameEnumAsStr());
break;
case TcpInfo::CongestionControlName::BIC:
EXPECT_EQ("BIC", wrappedTcpInfo.ccNameEnumAsStr());
break;
case TcpInfo::CongestionControlName::BBR:
EXPECT_EQ("BBR", wrappedTcpInfo.ccNameEnumAsStr());
break;
case TcpInfo::CongestionControlName::VEGAS:
EXPECT_EQ("VEGAS", wrappedTcpInfo.ccNameEnumAsStr());
break;
case TcpInfo::CongestionControlName::DCTCP:
EXPECT_EQ("DCTCP", wrappedTcpInfo.ccNameEnumAsStr());
break;
case TcpInfo::CongestionControlName::DCTCP_RENO:
EXPECT_EQ("DCTCP_RENO", wrappedTcpInfo.ccNameEnumAsStr());
break;
case TcpInfo::CongestionControlName::DCTCP_CUBIC:
EXPECT_EQ("DCTCP_CUBIC", wrappedTcpInfo.ccNameEnumAsStr());
break;
case TcpInfo::CongestionControlName::NumCcTypes:
FAIL();
}
switch (testParams.ccNameEnum) {
case TcpInfo::CongestionControlName::UNKNOWN:
case TcpInfo::CongestionControlName::RENO:
case TcpInfo::CongestionControlName::CUBIC:
case TcpInfo::CongestionControlName::BIC:
checkNoCcInfo(wrappedTcpInfo); // should be no extra CC_INFO
break;
case TcpInfo::CongestionControlName::BBR:
checkCcFieldsAgainstBbr(wrappedTcpInfo);
break;
case TcpInfo::CongestionControlName::VEGAS:
checkCcFieldsAgainstVegas(wrappedTcpInfo);
break;
case TcpInfo::CongestionControlName::DCTCP:
case TcpInfo::CongestionControlName::DCTCP_RENO:
case TcpInfo::CongestionControlName::DCTCP_CUBIC:
checkCcFieldsAgainstDctcp(wrappedTcpInfo, testParams.ccNameEnum);
break;
case TcpInfo::CongestionControlName::NumCcTypes:
FAIL();
}
}
#endif // #ifdef __linux__
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment