Bpftune

lastmod 2024-11-20created 2024-11-201 edits

bpftune uses BPF to auto-tune Linux systems

A lightweight tool designed to use the observability features of BPF (eBPF) to continuously and automatically tune Linux system performance.

Problems it aims to solve

Too many tuning options: the Linux kernel has more than 1600 tunable sysctl parameters. Finding settings suited to diverse environments is complex and difficult
Reduced administrator involvement: modern cloud systems need automated management rather than manual tuning. Most systems see little administrator involvement after initial setup
Limits of static settings: since the system environment keeps changing, tuning with fixed values can be inefficient

Minimal overhead: avoid tracing high-frequency events, and use observability only when strictly necessary
Clear policy explanations: log "what" was changed and "why" to syslog
Administrator first: when an administrator sets a value, auto-tuning is disabled
Zero-configuration principle: tunes automatically without any configuration. Avoids magic numbers where possible
Dynamic adjustment: when TCP memory usage rises, reduces buffer sizes to keep the system balanced

Tuner: each tuner handles events raised by BPF programs and manages the related tunables
Strategy: a tuner can have several strategies, and an effective strategy is chosen through each strategy's evaluation function
Event: contains the tuner ID, scenario, network namespace information, etc., and the tuner takes appropriate action based on this information

bpftune is a daemon that loads and manages multiple plugin tuners in .so format.
Each tuner has a unique ID and handles events in BPF and in user space.
BPF components include bpftune.bpf.h and use shared variables and map definitions.

TCP connection tuner: automatically tunes the congestion control algorithm
neighbour table tuner: automatically grows the neighbour table size
route table tuner: automatically grows the route table size
sysctl tuner: disables a tuner when a tuned sysctl value conflicts with a manual setting
TCP buffer tuner: automatically adjusts TCP buffer sizes
net buffer tuner: automatically adjusts core networking-related tuning
netns tuner: detects network namespaces being added and removed

Bpftune uses BPF to auto-tune Linux systems | Hacker News
Concern that diagnosing and fixing system problems could become harder
The concept of a "feedback loop" is important for explaining how problems could arise
- Not because of problems in the Linux kernel, BPF, or the program itself, but because of how it works
- There may be no risk, there may be controls in place, and even if problems occur it may converge to a stable state
Control theory is somewhat underapplied in software engineering
Would like to hear from people using this tool
- Wondering whether it is effective and whether it is worth the setup time
bpftune was designed as a tool that needs no configuration
- Welcome automated help, since there is no need to learn lots of settings
Available out of the box on CachyOS
- After installation, accessible via CachyOS Hello -> Apps/Tweaks
Curious about its effectiveness in multi-tenant environments such as shared k8s clusters
- Each application has a different purpose and moves between nodes over time
- However, there are likely similarities across most applications
Wondering whether TCP buffer sizing is worthwhile
Question: if tuning parameters to optimal values were easy, shouldn't the kernel have done so from the start?
Great that it has evolved from simple packet filtering to tracing and monitoring
- A great tool that most people should know about
- Have been using it for several years

author Zer0 sitemap