Detection Rules

Flow Matrix rules are the detection half of obserae’s NDR engine. A rule is one legitimate (or one forbidden) pattern of traffic that obserae’s matcher looks for in your sessions. Each rule is authored symbolically — src: backends, dst: postgres, dst_service: postgres — and the engine takes care of turning it into the underlying IP/port comparisons. What the rules surface then feeds the response side — alerting and outputs.

Together, your rules form the Flow Matrix: the auditable list of traffic you have declared as expected. The interesting signals are:

• a session that matches no rule at all (unexpected traffic);
• a session that matches a rule it shouldn’t (policy violation).

The GUI’s Sessions page filters directly on the first case (set the Match state toggle to Unmatched). The CLI uses an NFQL anti-pivot — see NFQL.

Not to be confused with alert rules. Flow Matrix rules (this page, managed on the Flow Matrix page) describe allowed connectivity and are evaluated by the matcher against every session. Alert rules (Alerting, managed on the Rules page) run a saved NFQL query on a schedule and raise alerts. The two are complementary.

Common tasks

• Write your first rule — YAML, CLI or GUI.
• Understand the port/service field.
• Find a rule in a large matrix.
• List the traffic no rule accounts for.
• Clean up redundant rules.
• Fix a quarantined rule.

Why sessions, not flows?

obserae folds raw NetFlow records into bidirectional sessions before the matcher runs. One conversation = one session row, regardless of how many flow records described it. That means:

• The matcher inspects 10×–100× fewer rows per tick.
• Each match already carries the inferred role (server_ip, role_method, role_conf) — see Sessions.

The trade-off: a match fires only after the session has closed (FIN / RST / idle timeout), so worst-case detection latency is matcher.interval plus the protocol’s idle window. Default tuning puts that around 1 minute.

YAML format

rules:
  - name: backends-to-postgres
    description: "Backends opening PostgreSQL connections"
    src: backends                  # any cartography reference
    src_iface: eth1                # optional — only valid for host refs
    src_service: "*"               # port/service on src side (see below)
    dst: postgres
    dst_iface: eth1
    dst_service: postgres
    enabled: true                  # default true; matcher skips disabled rules
    tags: [critical, datastore]    # optional — free-form labels, searchable

There is no separate protocol: field. The protocol is derived from the src_service / dst_service values (see The port/service field below).

A rule has no orientation field. Sessions are non-oriented (endpoints are folded into a canonical IP order internally; NFQL exposes them reoriented to client_ip / server_ip), so the matcher’s predicate is symmetric: a rule src=A, dst=B matches a session whether A landed on the low or the high side of that internal ordering. If you need to know which side initiated, look at the session’s server_ip and role_method.

Required fields

• name — unique across rules.
• src, dst — both required; either can be any4 / any6, internet4 / internet6 or internal4 / internal6 (family-specific keywords — see Cartography). internal4 / internal6 are the exact complement of internet4 / internet6 (the non-routable side: RFC1918, ULA, loopback, link-local, …).
• src_service, dst_service — required; use "*" for “any port”. See The port/service field.

Optional fields

The port/service field

src_service and dst_service carry both the port and the protocol — there is no separate protocol: field. Each side accepts one of:

Supported protocol names (case-insensitive): TCP, UDP, ICMP, IGMP, GRE, AH, ESP, OSPF, SCTP, ICMPv6. A port is only meaningful for TCP, UDP and SCTP — a form like 80/IGMP is rejected because IGMP does not carry an L4 port. Use */IGMP, */GRE, … to match those protocols.

A bare port without a protocol (53) and port ranges are rejected.

The protocol is derived from the two sides:

• You must pin the protocol on at least one side, otherwise the rule is rejected (there is no ANY protocol any more).
• If both sides pin a protocol and they conflict (*/TCP on one side, 53/UDP on the other), the rule is rejected.

internet4 / internet6, internal4 / internal6, any4 / any6 and network references accept *, */PROTO and PORT/PROTO (so internet4 -> 53/UDP works), but not a service name — service names are reserved for host / group references. With */PROTO (or *), the compiler does not restrict the destination IPs to interfaces that bind a matching service, since there is no service name to look up: every CIDR in the endpoint is emitted as-is.

Reference syntax for src / dst

Same as NFQL — bare names (looked up across hosts/groups/networks), host:NAME[:IFACE], group:NAME, network:NAME, the DHCP projections network:NAME.dhcp / network:NAME.static (when the network has a DHCP range), and the family-specific reserved keywords any4 / any6, internet4 / internet6 and internal4 / internal6 (the non-routable complement of internet). See Cartography for the full table.

For example, “office DHCP clients may only reach internal DNS”:

rules:
  - name: office-dhcp-dns-only
    src: "office.dhcp"
    src_service: "*"
    dst: "network:dns-servers"
    dst_service: 53/UDP

In the Flow Matrix page the source / destination picker is autocomplete-driven — you click a suggestion, you don’t type free text. The DHCP projections show up on demand: type a dot (office.) or the keyword dhcp / static and the picker offers network:office.dhcp and network:office.static for every network that has a declared range. Typing just office keeps the picker focused on the bare network, so the common case stays uncluttered.

Reading a rule

obserae-cli rule show backends-to-postgres

name:            backends-to-postgres
description:     Backends opening PostgreSQL connections
src:             backends
src_service:     *
dst:             postgres
dst_iface:       eth1
dst_service:     postgres
enabled:         true
expansions:      24
tags:            [critical, datastore]
last_compile_error:

The expansions count is the number of concrete IP/port tuples the rule compiled to. A high count is fine; symbolic names are resolved once so matching can stay fast at runtime.

last_compile_error is empty for a healthy rule. A non-empty value means the rule is quarantined (see below).

Managing rules

Bulk YAML

Rules are the flow_matrix: section of the single consolidated config bundle (CLI obserae-cli config …, or the GUI’s Backup and Restore pages):

obserae-cli config import   config.yml
obserae-cli config export   --output config.yml

When the bundle carries a flow_matrix: section it is a full atomic replacement of the rule set — every rule not in it is removed. Use the per-entity commands for incremental edits.

Per-entity

obserae-cli rule add backends-to-redis \
    --src backends --dst redis \
    --src-service '*' --dst-service redis \
    --description "Backends caching to Redis"

obserae-cli rule ls
obserae-cli rule update backends-to-redis --enabled=false
obserae-cli rule rm     backends-to-redis [--yes] [--dry-run]

In the GUI

The Flow Matrix page offers create / edit / enable-disable / delete actions with chip pickers for src, dst and tags. See Web GUI.

To scope a rule to a single interface of a host, type the host name in the src/dst entity field: the autocomplete offers interface-qualified refs such as host:web-01:eth0 and host:web-01:eth1 alongside the bare host:web-01. Pick one and the rule targets just that NIC’s IP. (There is no separate interface field — the qualifier lives in the entity ref.)

Searching rules

The Flow Matrix search box understands a small set of operators. Multiple terms are AND-ed together; a term without a known prefix is a free-text search.

Examples:

tag:critical                    # every rule tagged critical
proto:tcp port:5432             # TCP rules touching port 5432
host:srv-web service:https      # rules covering srv-web on https
tag:edge tag:external           # rules tagged both edge AND external

Typing in the search input debounces 200 ms before re-querying, so the table follows your typing smoothly.

Spotting redundant rules

When two rules cover overlapping traffic — for example group:web → internet4:443 (wide) and host:web-01 → internet4:443 (narrow, a member of the group) — the daemon detects it and exposes it in the GUI.

• In the table, the narrow rule’s name shows a small ⊂ N badge meaning “covered by N other rule(s)”. Click the row to see who.
• In the drawer, a Relations section lists the rules that cover the current one (“covered by”) and the rules the current one covers (“covers”). Each entry is a clickable button that pivots the drawer to that rule.
• When the current rule is strictly covered by another rule that is still enabled, a “Disable this redundant rule” button appears. Clicking it flips enabled=false — it never deletes, and the wider rule keeps catching the traffic.

The detector recognises three kinds of relations: subset (every match of A is also a match of B), equal (two rules cover the exact same flow set) and overlap (the rules intersect without containment).

The relations are recomputed automatically after every rule mutation and after every cartography mutation — moving a host into a group is enough to surface new relations on the very next table reload.

Inspecting what a rule has caught

obserae-cli matches ls                                # last 50, all rules
obserae-cli matches ls --rule public-https --limit 200
obserae-cli matches ls --since 1h
obserae-cli matches ls --since 2026-05-01T12:00:00Z
obserae-cli matches ls --json | jq '.[]'

Each match row carries the session’s internal canonical endpoint pair (ip_a:port_a ↔ ip_b:port_b — the physical low/high-IP ordering this dedicated endpoint denormalises, not the NFQL client/server surface), the protocol, and the inferred role columns (server_ip, server_port, role_method, role_conf) that give the client/server orientation.

In the GUI, click any rule on the Flow Matrix page to see a chart of its matches over the last 24 hours plus a table of recent matches.

Asking the matrix about one flow

An automation platform reacting to an east-west alert asks one question first: was this flow supposed to be allowed? POST /api/policy/check answers it directly, without writing a query.

curl -sX POST https://obserae.corp.example/api/policy/check \
  -H "Authorization: Bearer obs_…" -H "Content-Type: application/json" \
  -d '{"src":"10.0.0.51","dst":"10.20.4.11","port":5432,"protocol":"tcp"}'

{
  "covered": false,
  "matched_rule": null,
  "nearest_rules": [
    { "name": "build-to-db", "why_not": "source not in scope" }
  ],
  "src_entity": { "ip": "10.0.0.51", "carto": { "resolved": true, "host": "laptop" } },
  "dst_entity": { "ip": "10.20.4.11", "carto": { "resolved": true, "host": "db-01" } }
}

It takes the same references as a rule’s own endpoints — host:NAME, group:NAME, network:NAME or a bare address — and returns the same verdict as the GUI’s Find covering rule. Both run the same query; a shared test fixture fails the build if they ever disagree.

why_not: what would have to change

{"covered": false} tells an analyst nothing they can put in a ticket. nearest_rules names the rules that came closest and the one dimension standing between each of them and the flow:

The list is closed — an integration may switch on those four values and will never meet a fifth. A rule failing on several dimensions is reported once, under the first of the list. The list is capped at ten, and nearest_rules_truncated: true says it was cut, so a short list is never mistaken for a complete one.

A covered flow carries an empty nearest_rules: there is nothing to explain.

Common patterns

Stage a rule before turning it on

Add it with enabled: false, inspect its expansions and what it would have caught (run an equivalent NFQL query on closed sessions for a feeling), then rule update --enabled=true.

Service binding constrains interfaces

When a rule names a src_service or dst_service (other than "*"), the compiler only emits the IPs of interfaces the service is bound to. Example:

- name: bastion-to-prod-ssh
  src: bastion              # 1 host × 1 admin iface  = 1 CIDR
  dst: production           # 9 hosts × 1 ssh iface  = 9 IPs (only ssh-bound!)
  src_service: "*"
  dst_service: ssh          # service name pins port + protocol (TCP/22)

That produces 9 expansions rather than 26 you’d get with dst_service: "*" (because the rule only targets the ssh-bound interfaces, not every interface of every host).

Catch traffic that matched no rule

The canonical “what shouldn’t be happening?” question. Run this in NFQL:

FROM session_matches | LAST 3600
> FROM sessions      | LAST 3600 | WHERE state == "closed"
                     | PIVOT NOT session_id == session_id
                     | KEEP client_ip, server_ip, role_method

In the GUI, set the Sessions page’s Match state toggle to Unmatched — it runs this for you.

Disable a noisy rule temporarily

obserae-cli rule update X --enabled=false

Re-enabling later resumes the matcher cursor where it stopped — no historical re-match storm. To remove the rule and its history entirely, use rule rm X.

Force a rebuild

There is no separate “recompile” command. Every supported edit (rule update, cartography mutation) triggers it implicitly. If you really need to force a rebuild without changing content (rare — for example, after a daemon upgrade), edit the rule’s description:

obserae-cli rule update X --description "..."

That bumps the revision and re-emits expansions.

Quarantined rules

A rule references entities in the cartography. If a cartography mutation removes one of those entities (or a typo references a missing host), the next recompile fails, and the rule is quarantined:

• It stays in the database with enabled=true.
• Its matcher cursor is paused — it produces zero matches.
• last_compile_error is populated with the reason.

obserae-cli rule ls shows the column:

NAME                  ENABLED  EXPANSIONS  LAST_COMPILE_ERROR
backends-to-postgres  true            24
ssh-from-old-host     true             0  host "old-host": not found

To clear it, either restore the missing entity in the cartography, update the rule to point at a different reference, or delete the rule.

Where to next

• Sessions — the session model the matcher works on.
• Alerting — alert rules, the complementary NFQL-based detections.
• NFQL — query the matches and the unmatched sessions.
• Web GUI — the Flow Matrix page in the interface tour.