Sizing
How much disk, memory and CPU does obserae need? This page gives the figures we measure in practice, the arithmetic to apply them to your own traffic, and a worked example.
Every number here is an order of magnitude. Traffic mixes differ, and a sizing that is exactly right is a sizing with no headroom. Always take a margin.
Tools → Sizing in the web GUI runs this arithmetic for you, and on a running instance it can replace the figures below with what your own deployment actually stores. Sizes are binary throughout — 1 GB means 1024 MB, matching the Storage page.
Where these numbers come from
Being clear about this matters more than the numbers themselves:
flowsandsessionsare measured. They come from a real production instance: 50 M flow records occupied 831 MB, and 25 M sessions occupied 2.7 GB.sessions_consolidatedandsession_matchesare estimates. Plausible, never observed.session_matchesin particular depends on your Flow Matrix far more than on your traffic.- The memory and CPU figures have not been measured at all. They are comfortable working numbers, not the output of a benchmark.
The GUI labels each figure the same way, so you always know which kind you are looking at.
What one exchange produces
Understanding the storage figures starts with what a single round trip creates. Take a DNS query and its response:
| Records created | Store |
|---|---|
| 2 flow records — one per direction | flows |
| 1 session — the bidirectional conversation, per exporter that saw it | sessions |
| 1 consolidated conversation — the real exchange, whichever exporters saw it | sessions_consolidated |
If the conversation is covered by your Flow Matrix, it also produces a match
row (session_matches).
How many sessions per flow?
This ratio is the single biggest lever on your disk footprint, and it depends entirely on how long your conversations last.
- Short exchanges (DNS, monitoring probes, API calls) tend towards two flow records per session — so asymptotically half as many sessions as flows. This is the worst case for storage.
- Long-lived sessions (file transfers, backups, streams, database connections) emit many flow records over their lifetime, all folded into a single session. The more long sessions you have, the far fewer sessions you get per flow.
When in doubt, size with the short-exchange assumption (sessions = flows / 2)
and treat anything better as margin.
Disk
Per million rows
| Store | Per million rows | What moves it |
|---|---|---|
flows | 15 MB | Raw ingestion volume. |
sessions | 110 MB | Your flow-to-session ratio, and the number of exporters that see the same traffic. |
sessions_consolidated | 45 MB | Lower per real conversation the more exporters you have — see below. |
session_matches | 66 MB | Your session count, the size of your Flow Matrix, and how many of its rows actually match. |
Two notes worth reading before you multiply:
Consolidation gets cheaper as you add exporters. When several devices report the same conversation, they each produce a session, but they consolidate into one conversation. Aggregation makes both the processing and the storage more efficient — the redundant views collapse instead of accumulating. A second exporter on the same traffic is not a second full cost.
Matches are yours to control. session_matches is the one store whose size
you set by design rather than by traffic: a large Flow Matrix, or one with many
broadly-scoped rows that match everything, produces many more match rows than a
tight one.
Fixed footprint
| Item | Size |
|---|---|
| Enrichment metadata (cloud ranges, threat intel, GeoIP, ASN) | ~500 MB typically |
| DuckDB database (cartography, rules, settings, baselines) | Varies; 1 GB is a generous allowance |
| Audit log and alerts | Anecdotal — do not bother budgeting for them |
The database size depends on your rules, your cartography and your anomaly baselines rather than on traffic. It does not grow with flow volume.
Estimating your own
At F flows per second, per day, with the short-exchange assumption:
flows = F × 86,400 / 1,000,000 × 15 MB
sessions = F/2 × 86,400 / 1,000,000 × 110 MB
Which works out to roughly 1.3 GB of flows and 4.7 GB of sessions per day at 1,000 flows/s — sessions dominate, so that ratio is where your attention belongs. Multiply by your retention window, add the consolidated and match stores, then add the fixed footprint.
Retention is on in the configuration obserae ships with, and off in the code
defaults — so a config file that omits the retention: block entirely will
accumulate forever, while the obserae.yaml the installer, the tarball and the
Docker image deploy has enabled: true. Whichever you started from, collect a
few days first, read the real daily growth off the
Storage page, and only then tune the horizons. See
Lifecycle for the retention keys.
Worked example
A small site: around thirty machines and exporters, roughly 700 flows/s, and three days of history.
| Store | Rows | Size |
|---|---|---|
flows | 181 M (700 × 86,400 × 3) | 2.7 GB |
sessions | 91 M (short-exchange bound) | 9.8 GB |
sessions_consolidated | 45 M | 2.0 GB |
session_matches | 45 M | 2.9 GB |
| Enrichment + database | — | 1.5 GB |
| Total | ~20 GB |
Sessions are two thirds of the bill. If that site ran mostly long transfers rather than short queries, the same traffic would fit in a fraction of the space.
Disk performance matters
A large part of obserae’s work is writing files. The ingestion pipeline runs in RAM and lands Parquet files on disk continuously, and queries read them back. At meaningful traffic volumes, disk throughput and latency are a real constraint — often more so than CPU.
Prefer NVMe or a decent SSD. Avoid a microSD card for anything but a small homelab: it will bottleneck ingestion and wear out under the write pattern.
Do not forget backups
Backups live on disk too. Size the target volume for the number of snapshots you intend to keep, not for one — rotation policy and disk size have to be decided together. See Lifecycle.
Memory and CPU
All the heavy processing — decoding, enrichment, sessionization, correlation — happens in RAM, by design. Nothing is staged through a database on the hot path.
| Traffic | Memory | vCores |
|---|---|---|
| Small business, low volume | 4 GB | 4 |
| Around 1,000 flows/s | 8 GB | 8 |
These are comfortable figures, not floors: obserae runs on less. They are chosen to leave headroom for traffic bursts and for queries running while ingestion continues. They are also the least certain numbers on this page — nothing has measured obserae’s memory or CPU against a known flow rate, so treat them as starting points and watch the Monitoring page on your own traffic.
CPU scales automatically. obserae’s architecture parallelises across the cores it is given, up to around 50 cores — so on a larger machine you do not have to tune anything to make use of it. Give it more vCores and it will use them.
Small deployments
obserae is frugal software. For many small companies, a Raspberry Pi 5 or equivalent with 4–8 GB of RAM and an NVMe drive is genuinely sufficient — see the Raspberry Pi notes in the FAQ.
The storage medium matters more than the board. A Pi with an NVMe hat will outperform a bigger machine writing to a tired SD card.
Take a margin
To restate the one thing that matters most: these are orders of magnitude. Your traffic mix, the length of your conversations, the number of exporters watching the same links and the size of your Flow Matrix all move the result.
Size generously, run for a few days, then read the real numbers off the Storage page and tune retention with actual data in hand. Once the instance has collected a little, Tools → Sizing can do that for you: its “use my instance” button replaces the published coefficients with the ones your own deployment produces.