Sizing
How much disk, memory and CPU does obserae need? This page gives the figures we measure in practice, the arithmetic to apply them to your own traffic, and a worked example.
Every number here is an order of magnitude. Traffic mixes differ, and a sizing that is exactly right is a sizing with no headroom. Always take a margin.
What one exchange produces
Understanding the storage figures starts with what a single round trip creates. Take a DNS query and its response:
| Records created | Store |
|---|---|
| 2 flow records — one per direction | flows |
| 1 session — the bidirectional conversation, per exporter that saw it | sessions |
| 1 consolidated conversation — the real exchange, whichever exporters saw it | sessions_consolidated |
If the conversation is covered by your Flow Matrix, it also produces a match
row (session_matches).
How many sessions per flow?
This ratio is the single biggest lever on your disk footprint, and it depends entirely on how long your conversations last.
- Short exchanges (DNS, monitoring probes, API calls) tend towards two flow records per session — so asymptotically half as many sessions as flows. This is the worst case for storage.
- Long-lived sessions (file transfers, backups, streams, database connections) emit many flow records over their lifetime, all folded into a single session. The more long sessions you have, the far fewer sessions you get per flow.
When in doubt, size with the short-exchange assumption (sessions = flows / 2)
and treat anything better as margin.
Disk
Per million rows
| Store | Per million rows | What moves it |
|---|---|---|
flows | 15 MB | Raw ingestion volume. |
sessions | 110 MB | Your flow-to-session ratio, and the number of exporters that see the same traffic. |
sessions_consolidated | 45 MB | Lower per real conversation the more exporters you have — see below. |
session_matches | 66 MB | Your session count, the size of your Flow Matrix, and how many of its rows actually match. |
Two notes worth reading before you multiply:
Consolidation gets cheaper as you add exporters. When several devices report the same conversation, they each produce a session, but they consolidate into one conversation. Aggregation makes both the processing and the storage more efficient — the redundant views collapse instead of accumulating. A second exporter on the same traffic is not a second full cost.
Matches are yours to control. session_matches is the one store whose size
you set by design rather than by traffic: a large Flow Matrix, or one with many
broadly-scoped rows that match everything, produces many more match rows than a
tight one.
Fixed footprint
| Item | Size |
|---|---|
| Enrichment metadata (cloud ranges, threat intel, GeoIP, ASN) | ~500 MB typically |
| DuckDB database (cartography, rules, settings, baselines) | Varies; 1 GB is a generous allowance |
| Audit log and alerts | Anecdotal — do not bother budgeting for them |
The database size depends on your rules, your cartography and your anomaly baselines rather than on traffic. It does not grow with flow volume.
Estimating your own
At F flows per second, per day, with the short-exchange assumption:
flows = F × 86,400 / 1,000,000 × 15 MB
sessions = F/2 × 86,400 / 1,000,000 × 110 MB
Which works out to roughly 1.3 GB of flows and 4.7 GB of sessions per day at 1,000 flows/s — sessions dominate, so that ratio is where your attention belongs. Multiply by your retention window, add the consolidated and match stores, then add the fixed footprint.
Retention is off by default, precisely so you can collect a few days, look at the real daily growth on the Storage page, and only then choose a policy. See Lifecycle for the retention keys.
Worked example
A small site: around thirty machines and exporters, roughly 700 flows/s, and three days of history.
| Store | Rows | Size |
|---|---|---|
flows | 181 M (700 × 86,400 × 3) | 2.7 GB |
sessions | 91 M (short-exchange bound) | 9.8 GB |
sessions_consolidated | 45 M | 2.0 GB |
session_matches | 45 M | 2.9 GB |
| Enrichment + database | — | 1.5 GB |
| Total | ~20 GB |
Sessions are two thirds of the bill. If that site ran mostly long transfers rather than short queries, the same traffic would fit in a fraction of the space.
Disk performance matters
A large part of obserae’s work is writing files. The ingestion pipeline runs in RAM and lands Parquet files on disk continuously, and queries read them back. At meaningful traffic volumes, disk throughput and latency are a real constraint — often more so than CPU.
Prefer NVMe or a decent SSD. Avoid a microSD card for anything but a small homelab: it will bottleneck ingestion and wear out under the write pattern.
Do not forget backups
Backups live on disk too. Size the target volume for the number of snapshots you intend to keep, not for one — rotation policy and disk size have to be decided together. See Lifecycle.
Memory and CPU
All the heavy processing — decoding, enrichment, sessionization, correlation — happens in RAM, by design. Nothing is staged through a database on the hot path.
| Traffic | Memory | vCores |
|---|---|---|
| Small business, low volume | 4 GB | 4 |
| Around 1,000 flows/s | 8 GB | 8 |
These are comfortable figures, not floors: obserae runs on less. They are chosen to leave headroom for traffic bursts and for queries running while ingestion continues.
CPU scales automatically. obserae’s architecture parallelises across the cores it is given, up to around 50 cores — so on a larger machine you do not have to tune anything to make use of it. Give it more vCores and it will use them.
Small deployments
obserae is frugal software. For many small companies, a Raspberry Pi 5 or equivalent with 4–8 GB of RAM and an NVMe drive is genuinely sufficient — see the Raspberry Pi notes in the FAQ.
The storage medium matters more than the board. A Pi with an NVMe hat will outperform a bigger machine writing to a tired SD card.
Take a margin
To restate the one thing that matters most: these are orders of magnitude. Your traffic mix, the length of your conversations, the number of exporters watching the same links and the size of your Flow Matrix all move the result.
Size generously, run for a few days, then read the real numbers off the Storage page and tune retention with actual data in hand.