Operations

This page is for the person who runs obserae as a service: where files live, how to expose the GUI safely, what to monitor, how to back up the instance and what to check first when something looks wrong.

For a first hands-on setup, start with Quickstart. For package and Docker installation, see Installation.

Filesystem Layout

A typical systemd install uses:

/etc/obserae/obserae.yaml          # configuration you maintain

/var/lib/obserae/
  db/obserae.duckdb                # main database file
  data/                            # traffic history, audit log, enrichment, backups, master key
  run/obserae.sock                 # local control socket for obserae-cli

Keep /var/lib/obserae on a disk sized for your retention policy. Traffic history grows with the number of flow records you collect; retention is what makes that growth plateau.

Deploy As A Systemd Service

1. Install the binaries

sudo install -m 0755 obserae      /usr/local/bin/
sudo install -m 0755 obserae-cli  /usr/local/bin/

2. Create the service user and directories

sudo useradd --system --home /var/lib/obserae --shell /usr/sbin/nologin obserae
sudo install -d -o obserae -g obserae -m 0750 \
    /var/lib/obserae \
    /var/lib/obserae/data \
    /var/lib/obserae/db \
    /var/lib/obserae/run

3. Create the config

sudo install -d -m 0755 /etc/obserae
sudo tee /etc/obserae/obserae.yaml > /dev/null <<'YAML'
listen:
  netflow:
    enabled: true
    address: "0.0.0.0:2055"
  ipfix:
    enabled: true
    address: "0.0.0.0:4739"

storage:
  data_dir: "/var/lib/obserae/data"
  duckdb_path: "/var/lib/obserae/db/obserae.duckdb"
  memory_limit: "50%"

control:
  socket: "/var/lib/obserae/run/obserae.sock"

web:
  enabled: true
  address: "127.0.0.1:8080"

logging:
  verbosity: 0
YAML

The web GUI stays on loopback here. Put a TLS reverse proxy in front before you make it reachable from another machine.

4. Install the unit

sudo tee /etc/systemd/system/obserae.service > /dev/null <<'EOF'
[Unit]
Description=obserae NetFlow/IPFIX collector
After=network-online.target
Wants=network-online.target

[Service]
Type=exec
User=obserae
Group=obserae
ExecStart=/usr/local/bin/obserae --config /etc/obserae/obserae.yaml
Restart=on-failure
RestartSec=5

NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/obserae
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
LockPersonality=true
RestrictRealtime=true

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now obserae
sudo journalctl -u obserae -f

If you bind a privileged UDP port, grant CAP_NET_BIND_SERVICE in the unit.

Expose The Web GUI Safely

The GUI requires a login. On first boot, obserae creates the admin user and prints the generated password once in the daemon log. Change it after signing in.

obserae serves HTTP; it does not terminate TLS itself. For remote access, keep obserae bound to 127.0.0.1:8080 and put a reverse proxy with HTTPS in front.

Caddy

obserae.example.com {
    reverse_proxy 127.0.0.1:8080
}

Caddy forwards the original scheme, so obserae can keep the session cookie secure.

nginx

server {
    listen 443 ssl;
    server_name obserae.example.com;

    ssl_certificate     /etc/letsencrypt/live/obserae.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/obserae.example.com/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_http_version 1.1;
        proxy_set_header Host              $host;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header Upgrade           $http_upgrade;
        proxy_set_header Connection        "upgrade";
    }
}

The Upgrade and Connection headers are required for live cockpit and cartography updates.

If the GUI appears to accept the password but sends you back to the login page, you are probably reaching a non-loopback HTTP URL without TLS. Use HTTPS, or for a trusted lab-only plain-HTTP deployment set web.secure_cookies: false.

To record the real client IP in the audit log when a reverse proxy is used, set web.trusted_proxies to the proxy address or CIDR. Without it, forwarded client IP headers are ignored.

CLI Access

obserae-cli talks to the local control socket. Give operators access by adding them to the obserae group:

sudo usermod -aG obserae alice
# alice must log out and back in

obserae-cli --socket /var/lib/obserae/run/obserae.sock status

A shell alias keeps commands short:

alias obserae-cli='/usr/local/bin/obserae-cli --socket /var/lib/obserae/run/obserae.sock'

Treat socket access as privileged. It is the administrative control path.

Monitoring

The quickest health check is:

obserae-cli --socket /var/lib/obserae/run/obserae.sock status

Use --json when integrating with a metrics stack:

obserae-cli --socket /var/lib/obserae/run/obserae.sock status --json

Watch these signals:

The GUI Monitoring page shows the same operational view for administrators. The Cockpit stays focused on analyst-facing health and alert coverage.

Audit Log

Enterprise feature, free during the alpha.

The Audit log records sensitive actions: logins, denied requests, cartography changes, rule changes, lifecycle changes, user changes and output/device edits. It is searchable in the GUI and exportable for incident response.

The audit log is tamper-evident. If entries are edited, deleted or reordered, integrity verification reports where the chain broke.

Use:

obserae-cli verify-auditlog

Keep audit retention deliberate. By default, audit entries are kept forever; if you set an age limit, make sure it matches your investigation and compliance needs.

Master Key

obserae creates one master key on first boot, normally under <data_dir>/masterkey.bin. It protects stored secrets such as output tokens, device API secrets and session-signing material.

Back it up to a secret manager:

obserae-cli masterkey export --output /secure/obserae-masterkey.b64

If you rebuild an instance, restore the master key before first boot. Without the matching key, encrypted connector/output secrets cannot be recovered and users will need to sign in again.

You can rotate the key from Config I/O or the CLI:

obserae-cli masterkey import new-masterkey.b64

Backups

Use the built-in backup runner from the Lifecycle page or the CLI. A backup is a recoverable snapshot of the instance, not just a copy of one file.

Useful commands:

obserae-cli backup status
obserae-cli backup run
obserae-cli backup list
obserae-cli backup plan --point-in-time 2026-05-28T13:00:00Z
obserae-cli backup restore --point-in-time 2026-05-28T13:00:00Z --dry-run --confirm
obserae-cli backup restore --point-in-time 2026-05-28T13:00:00Z --confirm

Before upgrades or large imports, run a manual backup:

obserae-cli backup run

Also export configuration regularly:

obserae-cli config export > /var/backups/obserae/$(date +%F).config.yml

Config export is useful for review and version control. Backup is what you use for disaster recovery.

Upgrades

For a binary install:

obserae-cli backup run
sudo systemctl stop obserae
sudo install -m 0755 obserae /usr/local/bin/
sudo install -m 0755 obserae-cli /usr/local/bin/
sudo systemctl start obserae

For Docker:

docker pull ghcr.io/spartan-conseil/obserae:latest
docker restart obserae

Read the release notes before upgrading production. They call out any migration, configuration or compatibility notes.

Troubleshooting

The CLI cannot connect to the socket

Check that the service is running and that the CLI uses the configured socket:

sudo systemctl status obserae
obserae-cli --socket /var/lib/obserae/run/obserae.sock status

Flow count stays at zero

Check the exporter path in this order:

1. The device is configured to send NetFlow/IPFIX to the obserae host.
2. UDP 2055 or 4739 is reachable through firewalls.
3. Packets arrive on the host.
4. NetFlow v9/IPFIX templates have arrived.

Useful checks:

sudo ss -ulnp | grep -E '2055|4739'
sudo tcpdump -ni any udp port 2055 -c 5
obserae-cli --socket /var/lib/obserae/run/obserae.sock status

For a brand-new NetFlow v9/IPFIX exporter, data packets may arrive before the template that describes them. Wait for the next template refresh or restart the exporter.

Login loops back to the login page

Use HTTPS through a reverse proxy. A browser will drop a secure session cookie received over remote plain HTTP. For a trusted lab-only HTTP deployment, set:

web:
  secure_cookies: false

Disk keeps filling

Enable retention or shorten the retention windows. Then run a cleanup from the Lifecycle page.

df -h /var/lib/obserae
obserae-cli retention status

A rule is quarantined

A rule usually becomes quarantined because it references a cartography entity that was renamed or removed. Either restore the missing entity, update the rule, or delete it.

obserae-cli rule ls
obserae-cli rule show NAME

Memory keeps growing on a small host

Set a memory limit and enable retention:

storage:
  memory_limit: "512MB"
  max_threads: 2

retention:
  enabled: true
  flows_max_age: 168h
  sessions_max_age: 720h

If heap metrics grow steadily for hours, enable the profiler briefly on localhost and investigate from there:

debug:
  pprof_enabled: true
  pprof_address: "127.0.0.1:6060"

Do not expose the profiler on the network.

See Also

• Configuration - YAML keys and tuning recipes.
• Lifecycle - retention and backup settings.
• Monitoring - GUI operational dashboard.
• CLI - command reference.
• Installation - Docker and binary install paths.