Outputs

Outputs send your alerts out of obserae — to a chat channel, an on-call app, a ticketing system or a SIEM. Where Alerting decides when an alert fires, an output decides where it goes.

obserae ships these kinds of output:

• Webhook — an HTTP POST (or PUT/PATCH) to any URL. Works with incoming webhooks, automation platforms, or your own endpoint.
• Gotify — a push to a Gotify server.
• Slack — a message to a channel via the Slack API (bot token).
• Mattermost — a post to a channel via the Mattermost API (access token).
• Telegram — a message to a chat or channel via the Telegram Bot API.
• Syslog — an RFC5424 message to a SIEM over UDP/TCP/TLS, as JSON, CEF (ArcSight) or LEEF (QRadar).
• Splunk HEC — an event to a Splunk HTTP Event Collector.
• Elasticsearch / OpenSearch — one document per alert into an index.
• PagerDuty — raises an on-call incident (Events API v2).
• Opsgenie — raises an Opsgenie alert (US or EU region).
• Email (SMTP) — sends an email through your SMTP server.
• ntfy — a push to a topic on ntfy, self-hosted or not.
• Discord — a card in a channel, through a channel webhook.
• Microsoft Teams — a card in a channel, through a Workflows or legacy connector webhook.
• TheHive 5 — a SIRP alert whose addresses arrive as typed observables.
• MISP — publishes the public addresses an alert saw as a MISP event.
• Microsoft Sentinel — writes into a custom Obserae_CL table (Logs Ingestion API).
• Jira — opens an issue in a project (Cloud or Server / Data Center).
• GLPI — opens a ticket in the GLPI helpdesk.

Open the Outputs page from the Connectors group in the sidebar.

Community includes Webhook, Discord, Telegram, Email (SMTP), and Gotify. Business, Business+, and Enterprise licenses include every output type. If a license expires beyond its 30-day grace period, existing professional destinations remain visible and stored but stop receiving deliveries and cannot be changed. Renewing the license restores them; obserae never deletes or rewrites them during a license transition.

Wiring obserae to a SOAR, a SIRP or an automation engine? Response Integrations is the section written for that reader: what obserae does and does not do, the alert contract field by field, how to verify the signature, the API operations a playbook calls, and three worked playbooks.

Create an output

Click + New output and fill in the form.

For a webhook:

• URL — where to POST the alert (if it’s on your own network, see Internal destinations below).
• Method — POST (default), PUT or PATCH.
• Custom headers — one Key: Value per line (e.g. Authorization: Bearer …).
• Body template — leave empty to send obserae’s default JSON (id, rule_name, severity, fired_at, matched_count, detail, plus the enriched rule_tags / observed_value / key when the rule has them — see What’s in an alert below), or write your own using {{.RuleName}}, {{.Severity}}, {{.FiredAt}}, {{.MatchedCount}}, {{.Detail}}, {{.ID}}.
• Signing secret — optional. When set, every request is signed so the receiver can verify it really came from obserae (see Verifying webhooks below).

For Gotify:

• Server URL — your Gotify base URL (e.g. https://gotify.example.com).

• App token — the Gotify application token.

• Priority — leave at 0 to derive it from the alert severity, or pin a fixed 0–10.

• Title / Message templates — optional. Leave empty for the defaults:

  • title → [HIGH] postgres-present ([<SEVERITY>] <rule name>)
  • message → Alert "postgres-present" fired at 2026-06-04T12:00:00Z (7 matched).

  Or write your own with the same fields as the webhook body: {{.RuleName}}, {{.Severity}}, {{.FiredAt}}, {{.MatchedCount}}, {{.Detail}}, {{.ID}}. The default message does not show the matched rows — to include them, add {{.Detail}} to your message template, e.g. {{.MatchedCount}} match(es): {{.Detail}}.

For Slack:

• Channel — the channel id or #name your bot is allowed to post to.
• Bot token — the bot user OAuth token (xoxb-…) with chat:write.
• Message template — optional; leave empty for a compact one-line default ([HIGH] postgres-present — 7 matched at …).

For Mattermost:

• Server URL — your Mattermost base URL (e.g. https://mm.example.com).
• Channel ID — the channel’s id (not its display name; copy it from the channel’s View Info).
• Access token — a personal or bot access token.
• Message template — optional (same default as Slack).

For Telegram:

• Chat ID — the numeric chat id, or @channelusername for a public channel. (Message @userinfobot to find a numeric id.)
• Bot token — the token @BotFather gave you. The token is kept secret and is never shown again after saving.
• Parse mode — Plain text, MarkdownV2, HTML or legacy Markdown.
• Message template — optional (same default as Slack).

The message template for Slack/Mattermost/Telegram uses the same fields as the webhook body — {{.RuleName}}, {{.Severity}}, {{.FiredAt}}, {{.MatchedCount}}, {{.Detail}}, {{.ID}}.

For Syslog:

• Collector address — host:port of your syslog receiver / SIEM.

• Transport — UDP, TCP, or TCP + TLS (use TLS for an encrypted feed; the TLS options below then apply).

• Format — JSON, CEF (for ArcSight) or LEEF (for QRadar).

• Facility — syslog facility 0–23 (default 16 = local0).

• App name — the RFC5424 APP-NAME (default obserae).

  Syslog has no token to enter. (Internal collector on your LAN? See Internal destinations below.)

For Splunk HEC:

• HEC base URL — e.g. https://splunk.example.com:8088.
• HEC token — your HTTP Event Collector token.
• Index / Sourcetype / Source — optional routing for the event.

For Elasticsearch / OpenSearch:

• Cluster base URL — e.g. https://es.example.com:9200.
• Index — where to write (one document per alert).
• Authentication — Basic (username + password) or API key.
• Password / API key — the credential for the chosen mode.

For PagerDuty:

• Integration (routing) key — the Events API v2 key from your PagerDuty service.

• Source — optional label shown on the incident (default obserae).

  Repeated fires of the same rule update one incident (de-duplicated on the rule name) instead of paging on every tick.

For Opsgenie:

• API key — your Opsgenie API integration key.

• Region — US (api.opsgenie.com) or EU (api.eu.opsgenie.com).

  The alert priority is derived from severity (critical→P1 … info→P5), and repeated fires de-duplicate onto one alert (by rule name).

For Email (SMTP):

• SMTP host / Port — your mail server, e.g. smtp.example.com / 587.
• Connection security — STARTTLS (587, recommended), Implicit TLS (465), or None (plaintext — lab only).
• From — the sender address.
• To / Cc — recipients, comma-separated.
• Auth username / password — leave the username empty to send without authentication; otherwise the password is stored securely.
• Subject / Body templates — optional. Defaults to [HIGH] rule-name and a compact summary; use {{.RuleName}}, {{.Severity}}, {{.MatchedCount}}, {{.Detail}}, etc. for your own.

For Discord:

• Webhook URL — create it in Discord under Server Settings → Integrations → Webhooks, then paste the whole URL here. It ends in its own token, so obserae stores it as a secret and never shows it again: anyone holding that URL can post to the channel. If it leaks, delete the webhook in Discord and make a new one — rotating it here is not enough.
• Username — optional, overrides the name the webhook posts under.
• Message template — optional. Leave it empty and obserae sends a card: the severity colours its left edge, and the rule, severity and match count are fields. Fill it in and you get plain text instead — the card is replaced, not decorated, because showing both prints the alert twice.

For GLPI:

• GLPI URL — e.g. https://glpi.example.com.

• Application token — from Setup → General → API → API client. Enable the REST API there first, and allow the address obserae calls from.

• User token — the API user’s personal token, from that user’s profile under Remote access keys. The two tokens are different things: the application token identifies the client, the user token authenticates the account tickets are filed as.

• Entity ID — optional; leave empty to use the API user’s default entity.

  Tickets are created as incidents, with urgency and priority following the alert severity, and lead with a link back to the alert. As with Jira, obserae opens the ticket and never reads it back.

For Jira:

• Jira URL / Project key — e.g. https://acme.atlassian.net and SEC.

• Issue type — optional; defaults to Task.

• API version — v3 for Jira Cloud, v2 for Server / Data Center. They are not interchangeable: Cloud requires the description as an Atlassian Document Format tree, and Server/DC has no v3 at all.

• Account email — Cloud only. Setting it also selects the authentication: with an email obserae sends Basic email:token; without one it sends a Bearer personal access token, which is what Server / Data Center expects.

• API token — the Cloud API token, or the Server/DC personal access token.

  obserae opens a ticket and stops there. It never polls the ticket, never reads its status back and never closes it: watching a ticket is a workflow, and obserae deliberately hosts no workflow engine. The ticket leads with a link back to the alert, so the analyst is one click from the evidence.

For Microsoft Sentinel:

• Data Collection Endpoint URL — the DCE’s ingestion URI.

• DCR immutable ID — the Data Collection Rule’s immutable id, not its resource name. Find it in the rule’s JSON view. Using the name instead produces a 404 at delivery time with nothing pointing back at this field, which is the one mistake worth calling out here.

• Stream name — optional; defaults to Custom-Obserae_CL.

• Tenant ID / Application (client) ID / Client secret — an Entra ID app registration. Give it the Monitoring Metrics Publisher role on the Data Collection Rule.

  Alerts land in the Obserae_CL table. Sentinel also ingests OCSF natively, so you can set the output’s event schema to ocsf and receive Detection Findings instead of obserae’s own document — see Send it in your SIEM’s own language.

For MISP:

• MISP URL / API key — your instance and an automation key.

• Distribution — who sees the event. Defaults to Your organisation only. Raising it sends a record of your network’s activity to other people, so it is a decision, not a default to leave alone.

• Mark attributes to_ids — off by default. That flag tells every downstream consumer the address is fit to block on. obserae reports what its rules saw; deciding an address deserves a block belongs to whoever runs the feed.

• Publish immediately — off by default, so a human reviews the event first.

  Only publicly routable addresses are published. A private address says nothing to anyone outside your network, and shipping one to a sharing community leaks your internal topology for no benefit. An alert with nothing public in it creates no event at all rather than an empty one.

  This is the push twin of GET /api/indicators, which is pull. Like it, publishing a list is not enforcement: obserae blocks nothing either way.

For TheHive 5:

• TheHive URL — e.g. https://thehive.example.com.

• API key — an organisation API key with permission to create alerts.

• Alert type / Source — optional; both default to obserae and are how TheHive classifies where an alert came from.

  Each address in the alert arrives as a typed observable rather than as text in the description. That is the difference between an analyst who can pivot — search other cases, run a Cortex analyzer, drag it into a case — and one who has to retype the address by hand. Each observable carries obserae’s own context: the cartography host name as its message, and obserae:asset=…, obserae:group=…, obserae:role=… and obserae:intel=… as tags. An address a feed lists is flagged as an IOC.

  The alert’s sourceRef is obserae’s incident key, so TheHive deduplicates on the same identity obserae does — including after you rename the rule. Lifecycle events are off by default for TheHive: once an analyst has taken an alert, its state belongs to the SIRP, and obserae does not reach back into it.

For Microsoft Teams:

• Webhook URL — from a Workflows (Power Automate) trigger, or from a legacy Office 365 connector. As with Discord it carries its own signature, so obserae stores it as a secret and never shows it again.
• Card format — this is the field to get right. Teams has had two webhook generations and they take different bodies: a Workflows URL (they look like https://prod-….logic.azure.com/workflows/…) takes an Adaptive Card; an Office 365 connector takes a MessageCard. Sending the wrong one does not fail — Teams accepts the request and renders nothing, so a silent empty channel is the symptom. New outputs default to Adaptive Card, because Microsoft has retired the connectors.
• Message template — optional; replaces the card with plain text. Both generations accept plain text, so a template works either way.

For ntfy:

• Server URL — https://ntfy.sh, or your own server.
• Topic — the topic to publish to, e.g. obserae-alerts. It becomes the URL path, so it is also the address your phone subscribes to.
• Access token — optional, because ntfy allows public topics. Leave it empty only if you mean it: anyone who knows the name of a public topic can read every alert you publish to it, and the name is the only thing protecting it. On a public server, use a token and a name nobody would guess.
• Priority — leave at 0 to derive it from the severity (critical → 5, high → 4, medium → 3, low → 2, info → 1), or pin a fixed 1–5. On a phone, 5 overrides Do Not Disturb — keep it for what deserves waking someone.
• Tags — optional, comma-separated. ntfy clients render a tag as an emoji when the name matches one, so warning shows a ⚠ next to the title.
• Title / Message templates — optional. Defaults to [HIGH] postgres-present and the compact alert line. The default message does not include the matched rows; add {{.Detail}} to show them.

Internal destinations. For every output where you supply the address — webhook, Gotify, Mattermost, Splunk, Elasticsearch, syslog, SMTP, ntfy, Discord, Teams, TheHive, MISP, Sentinel, Jira and GLPI — obserae refuses an internal target by default (localhost, 10.x / 172.16.x / 192.168.x, the 169.254.x cloud-metadata address, …) as an SSRF safety guard: saving one returns an error. To reach a destination on your own network, add its CIDR to outputs.egress_allow_cidrs in the config (see Configuration). The fixed-endpoint outputs (Slack, Telegram, PagerDuty, Opsgenie) always go to the provider and are unaffected.

Then set the routing (below) and tick Enabled.

What’s in an alert

Every output sends the same alert content, shaped to fit the destination — a JSON body for a webhook, Splunk event or Elasticsearch document; a chat message for Slack / Mattermost / Telegram / Gotify; a syslog line (JSON / CEF / LEEF); an incident for PagerDuty / Opsgenie; an email for SMTP.

There are two payload formats, and each output picks one:

• v2 (obserae.alert/2) — what every output created today speaks. Fields are named, the rule has a stable id, and the alert says how to reproduce itself. Use this one.
• v1 (legacy) — what outputs created before this format existed still receive, unchanged. Nothing was migrated: your existing receivers keep getting the exact bytes they always got. An output on v1 shows a banner in its drawer with a before/after preview and a one-click switch.

The v2 alert, field by field

{
  "schema_version": "obserae.alert/2",
  "event_type": "alert.fired",
  "id": "018f1c00-0000-7000-8000-000000000001",
  "dedup_key": "a41f9c…",
  "fired_at": "2026-06-04T12:00:00Z",
  "sent_at": "2026-06-04T12:00:01Z",

  "instance": {
    "id": "8f2c0f1e-…",
    "name": "obserae-paris-dc1",
    "version": "0.34.0",
    "base_url": "https://obserae.corp.example"
  },

  "rule": {
    "id": "3f5b0c4e-…",
    "name": "postgres-present",
    "type": "presence",
    "severity": "high",
    "severity_score": 70,
    "tags": ["prod", "segmentation"],
    "url": "https://obserae.corp.example/alerting/rules/3f5b0c4e-…"
  },

  "alert": {
    "status": "open",
    "matched_count": 7,
    "first_seen_at": "2026-06-04T11:58:00Z",
    "url": "https://obserae.corp.example/alerts/018f1c00-…"
  },

  "query": {
    "nfql": "FROM sessions | LAST 300 | WHERE server_port == 5432 | KEEP client_ip, server_ip, server_port",
    "as_of": "2026-06-04T12:00:00Z"
  },

  "matches": {
    "columns": [
      {"name": "client_ip", "type": "VARCHAR"},
      {"name": "server_ip", "type": "VARCHAR"},
      {"name": "server_port", "type": "INTEGER"}
    ],
    "rows": [
      ["10.0.0.50", "10.20.4.11", 5432],
      ["10.0.0.51", "10.20.4.11", 5432]
    ],
    "records": [
      {"client_ip": "10.0.0.50", "server_ip": "10.20.4.11", "server_port": 5432},
      {"client_ip": "10.0.0.51", "server_ip": "10.20.4.11", "server_port": 5432}
    ],
    "sample_size": 2,
    "truncated": true
  }
}

The four things worth knowing before you write a playbook:

1. Correlate on rule.id, never on rule.name. The id is fixed when the rule is created and survives every rename. The name is a label your colleagues will edit.
2. dedup_key is the incident, id is the occurrence. A fire and the alert.resolved that closes it share a dedup_key; open your ticket on that, and the close will find it. alert.first_seen_at is when that incident opened, which for a rule re-firing hourly is not the same as fired_at.
3. Read matches.records. It is matches.rows with the column names applied — no counting positions, and a change to the rule’s KEEP can no longer silently shift detail[0] under your automation.
4. query.nfql + query.as_of replay the alert. Post them back to /api/query and you get the same rows the alert showed, anchored on the instant the rule ran.

matches.sample_size is how many rows travelled; alert.matched_count is the real total, and truncated says the two differ. The sample is 10 rows by default (outputs.sample_rows, up to 100).

Fields that appear only for the rule shape that produced them: alert.observed_value and alert.threshold for a threshold rule, alert.key for a group-by rule. A heartbeat rule fires on an empty result, so its matches.rows is [] — with columns still filled in, so you know what it was looking for.

An alert.resolved event is the one shape with no matches block at all. It carries resolved_at, duration_seconds and fire_count instead — see Alerts that close.

If Public URL is not configured, obserae emits no url field at all rather than a link nobody can open. No link ever carries a token.

One example per rule type

Threshold — alert carries the measurement and the bound it crossed:

"rule":  { "type": "threshold", "severity": "critical", "severity_score": 90 },
"alert": { "status": "open", "matched_count": 1,
           "observed_value": 4213, "threshold": 1000 }

Group-by — one alert per entity, and the entity is in alert.key:

"alert":   { "status": "open", "matched_count": 1,
             "key": {"client_ip": "10.0.0.50"} },
"matches": { "sample_size": 1, "truncated": false }

Heartbeat — fired because the query went silent:

"rule":    { "type": "heartbeat" },
"matches": { "columns": [{"name": "host", "type": "VARCHAR"}],
             "rows": [], "records": [], "sample_size": 0, "truncated": false }

First-seen — the sample holds only the rows that are new:

"rule":    { "type": "first_seen" },
"matches": { "rows": [["10.0.0.77", "10.20.4.11", 5432]], "sample_size": 1 }

Asset context: the entities block

This is the part that saves you the follow-up API calls. Instead of two bare addresses, the alert already tells you whose machines they are, what the internet knows about the public one, and whether your Flow Matrix allows the conversation at all:

"entities": [
  {
    "ip": "10.0.0.50",
    "role": "client",
    "scope": "internal",
    "carto": {
      "resolved": true,
      "host": "ci-runner-prod",
      "interface": "eth0",
      "network": "build",
      "group": "build",
      "groups": ["build"]
    },
    "policy": {
      "covered": false,
      "matched_rule": null,
      "checked_pair": "10.0.0.50 → 10.20.4.11:5432/tcp"
    }
  },
  {
    "ip": "203.0.113.9",
    "role": "server",
    "scope": "external",
    "carto": { "resolved": false },
    "service": { "port": 443, "protocol": "tcp" },
    "public": {
      "asn": 15169,
      "as_org": "Google LLC",
      "country": "US",
      "cloud": { "provider": "aws", "service": "EC2", "region": "eu-west-3" }
    },
    "threat_intel": [
      { "source": "firehol_level1", "verdict": "match",
        "updated_at": "2026-06-04T05:00:00Z" }
    ]
  }
],
"entities_truncated": false

Read it like this:

• role is client, server, or both — taken from the rule’s own columns, not guessed. A column the rule kept that is not one of the two sides carries no role at all.
• carto.resolved: false means the address is not on your map. When it is false, no other key is present — there is no empty host to mistake for a host name, so if entity.carto.host is a safe test.
• threat_intel: [] is not the same as no threat_intel. An empty array means the feeds were consulted and none lists this address. A single {"verdict": "unavailable"} means they could not be consulted. Do not treat the second as clean.
• threat_intel[].updated_at is when that feed last refreshed. It is there so your playbook can weigh the listing: a Tor exit list refreshed two hours ago is worth acting on, the same listing from a feed that has not refreshed in three weeks is worth a human look first. See enrichment.
• policy.covered: false with checked_pair is your segmentation finding, already phrased: this conversation is not allowed by any rule in your Flow Matrix. When it is covered, matched_rule names the rule.
• entities_truncated: true means more distinct addresses appeared than outputs.max_entities allows (20 by default). The ones kept are those most frequent in the matched sample — the addresses the alert is about.

Two guarantees worth knowing.

The context never delays your notification. The lookups get 250 ms per alert. If they run out, whatever was resolved is sent as-is, anything left carries "enrichment": "timeout", and the alert goes out on time. The same holds if the cartography, the feeds or the Flow Matrix are unavailable: the matching part of the block degrades and the alert is still delivered.

The context is frozen. It is captured at the moment the rule fired and stored with the alert. If a delivery fails and retries an hour later — or you resend it by hand next week — you receive the original context, not a fresh reading. Rename the host, refresh the feeds, edit the Flow Matrix: the alert still says what was true when it fired. That is what makes it evidence.

Choosing how much context to send

In obserae.yaml, outputs.entities:

outputs:
  entities: full
  max_entities: 20

off is a true off switch: nothing is looked up and the document is byte-identical to what it was without the block.

Three things obserae will not tell you here, on purpose: the operating system of a host (it is a map icon you chose, not something obserae observed), the reverse DNS of a public address (obserae does no PTR lookups), and a clickable link to a host on the map (there is no per-host URL yet). Each would be a guess or a dead link in a document you may attach to an incident report.

Version policy

• Adding a field is not a breaking change and does not bump the version. Write your playbook to ignore fields it does not know.
• Removing a field, or changing its type, bumps the major (obserae.alert/3).
• Two majors are supported in parallel for at least two obserae minor releases, so you always have a window to migrate.

The schema itself is published by your own instance at GET /api/schemas/alert/2 — point your validator at that rather than a copy, and it can never drift from what you receive.

The v1 alert (legacy outputs)

An output still on v1 receives the original flat shape:

[["10.0.0.50", 5432], ["10.0.0.51", 5432]]

Positional rows are exactly what v2 fixes: change a rule’s KEEP and detail[0] starts meaning something else, without anything failing loudly. Switching an output to v2 is one click and changes nothing else about it.

Send it in your SIEM’s own language

Writing into a SIEM is not the same as integrating with it. An Elasticsearch index full of obserae’s field names supports no Elastic Security dashboard, no detection rule and no correlation — the data is there, and nothing in the product can see it. A Splunk index without a CIM mapping is the same story.

Two settings on the output fix that.

Your existing outputs do not move

An output that already exists keeps sending exactly what it sends today, down to the byte, when you upgrade. Nothing changes shape behind you — an index that Elasticsearch has already mapped one way cannot be re-mapped without a reindex, and your saved searches and dashboards are built on the field names that are in there now.

Switching is a decision, and it is yours. When you make it, apply the index template first (below) and point the output at a new index; the old one keeps serving your existing searches until you retire it.

A new Elasticsearch output is created in ECS, because an Elasticsearch output that does not speak ECS integrates with nothing.

ECS — what it buys you

With mapping: ecs, an obserae alert arrives with the field names Elastic itself uses:

• Elastic Security dashboards populate. Network views key on event.category and event.kind, which the alert now carries.
• Detection rules you already have keep working on source.ip, destination.ip, destination.port and network.transport.
• Correlation across your other sources works, because everything is on the same field names. related.ip and related.hosts are what the “pivot to this host” actions in Elastic use.
• Asset context comes along: host.name from your cartography, source.as.organization.name and source.geo.country_iso_code from the enrichment, threat.enrichments[] from the intel feeds.

Nothing is lost. The complete obserae alert — the matched rows, the NFQL, the entity block — travels with it under obserae.*, so the evidence is still there when an analyst opens the document.

Apply the index template first. Your instance serves it:

curl -s https://obserae.example.com/api/integrations/elastic/template \
     -H "Authorization: Bearer $TOKEN" \
  | curl -XPUT https://es.example.com:9200/_index_template/obserae \
     -H 'Content-Type: application/json' -d @-

Adjust index_patterns (shipped as obserae-*) to match your index. If you skip this step, Elasticsearch guesses a mapping from the first alert it receives — and a wrong guess does not fail, it just quietly makes some queries return nothing. You find out weeks later.

OCSF — what it buys you

With format: ocsf, the alert arrives as an OCSF Detection Finding (class 2004). That is the format Microsoft Sentinel, Google SecOps and AWS Security Lake ingest natively, so the alert lands in their finding views with no transformation of yours in between.

The obserae alert travels with it under unmapped, so nothing is lost here either.

Two things worth knowing:

• The webhook’s X-Obserae-Schema header says ocsf.detection_finding/2004 instead of obserae.alert/2, so a receiver that routes on the header still routes correctly.
• OCSF and a custom body template cannot be combined. The template replaces the payload entirely, so it would silently override your OCSF setting. obserae refuses the pair rather than letting you believe you have OCSF.

Splunk: sourcetype and the add-on

obserae ships a Splunk add-on, TA-obserae, that maps alerts onto the CIM Alerts and Network Traffic models. Field extractions only — no inputs, no scripts, no credentials, and nothing that reaches back to obserae.

Set the output’s Sourcetype to match the payload you send:

Set it on the output, not on the HEC token — a token-level sourcetype applies to everything that token receives.

Install it from Splunkbase as TA-obserae, on the search head only: everything is search-time, so your indexers need nothing. See Splunk for the rest.

One field is worth explaining: CIM’s action is always unknown. obserae is out-of-band — it reads NetFlow and IPFIX, it is not on the path of anything and it blocks nothing — so it never observed an allow-or-deny decision. Reporting allowed would be claiming knowledge it does not have.

TLS (internal / self-signed endpoints)

Many outputs talk to a server over TLS — an HTTPS webhook, Gotify, Mattermost, Splunk or Elasticsearch endpoint, a syslog TCP + TLS feed, or SMTP over STARTTLS / Implicit TLS. If that server presents a certificate obserae doesn’t trust by default (a private/internal CA, or a self-signed certificate), the form offers two options at the bottom — they apply to whichever of those encrypted transports the output uses:

• Custom CA certificate — paste the PEM of your CA (or the self-signed cert). obserae will trust it in addition to the public roots and verify the connection normally. Recommended.
• Skip TLS certificate verification (insecure) — turns off certificate checking entirely. A warning appears as soon as you tick it: with verification off, obserae cannot confirm it’s really talking to your server, and the connection could be intercepted. Use it only for a trusted lab/internal endpoint while getting started, and switch to a custom CA for anything that matters.

(Outputs with a fixed public endpoint — Slack, Telegram, PagerDuty, Opsgenie — always use the provider’s certificate, so these options don’t apply to them.)

Choose which alerts go where (routing)

Each output receives an alert only when all of these match:

• Minimum severity — e.g. set high so an output only gets high/critical alerts.
• Rule names (optional) — a comma-separated allow-list; leave empty for any rule.
• Rule tags (optional) — comma-separated; the alert’s rule must carry at least one of these tags. Leave empty for any tag.

So you can send everything to an audit webhook (min severity = info), while only paging the on-call Gotify on high+ alerts tagged prod.

Alerts that close, not just alerts that fire

An alert opens an incident and closes when the condition goes away (obserae waits for three clean evaluations — see Alerting). Destinations hear about both ends, not only the first:

What each destination does with a close:

Chat and mail start off on purpose: a “resolved” message in a channel doubles the noise for the person reading it, who can already see the incident closed wherever they acknowledged it. Turn it on with Send lifecycle events in the output form if you want it — the trigger message is unaffected either way.

Why renaming a rule no longer opens a second incident

An incident is identified by a key built from your instance, the rule’s immutable id and the alert’s group key — never the rule’s name. So:

• renaming a firing rule keeps updating the same incident instead of opening a second one and orphaning the first;
• a rule grouped by IP opens one incident per IP, each closing on its own evidence;
• two obserae installations feeding one SOAR never close each other’s incidents.

A resolution deliberately carries no matched rows. The condition is gone, so the rows it fired on are minutes or hours old, and shipping them as proof of a resolution would be more misleading than shipping nothing. It carries how long the condition held and how many times it re-fired instead.

Test it

Open an output and click Send test. obserae delivers a sample alert straight away and tells you whether it succeeded — or shows the exact error (bad URL, auth rejected, server down). Use it to confirm everything is wired before a real alert depends on it.

Watch deliveries

The output’s drawer shows Recent deliveries with a status for each:

• sent — delivered successfully.
• failed — the last attempt errored; obserae will retry automatically with increasing back-off.
• dead — given up (too many failures, or a permanent error like a 404/auth failure that won’t fix itself).

Deliveries are reliable: every alert is queued the instant it fires and retried until it lands, so a brief network blip or a receiver restart never loses a notification. Retrying happens in the background and never slows down alerting.

Verifying webhooks

If you set a signing secret, each webhook request carries:

• X-Obserae-Timestamp — when it was sent (Unix seconds).
• X-Obserae-Delivery — a unique id you can use to ignore duplicates.
• X-Obserae-Signature — sha256=<hex>, an HMAC-SHA256 of "<timestamp>.<rawBody>" using your secret.

A v2 webhook also carries, so you can route without parsing the body:

• X-Obserae-Event — alert.fired (or test for a Send test).
• X-Obserae-Schema — obserae.alert/2.
• X-Obserae-Instance — which obserae sent it.
• User-Agent — obserae/<version>.

Your receiver verifies a request by recomputing that HMAC over the timestamp and the exact raw body and comparing it to the header. Because the timestamp is signed, an old captured request can’t be replayed.

Verify the raw body, before any JSON parsing — re-serialising changes the bytes and the signature will not match.

Python

import hashlib, hmac, time

def verify(secret: str, headers, raw_body: bytes, max_age: int = 300) -> bool:
    ts = headers.get("X-Obserae-Timestamp", "")
    sig = headers.get("X-Obserae-Signature", "")
    if not ts.isdigit() or abs(time.time() - int(ts)) > max_age:
        return False                      # too old: a replayed capture
    mac = hmac.new(secret.encode(), (ts + ".").encode() + raw_body, hashlib.sha256)
    return hmac.compare_digest("sha256=" + mac.hexdigest(), sig)

Go

func verify(secret string, h http.Header, rawBody []byte, maxAge time.Duration) bool {
	ts := h.Get("X-Obserae-Timestamp")
	sent, err := strconv.ParseInt(ts, 10, 64)
	if err != nil {
		return false
	}
	if d := time.Since(time.Unix(sent, 0)); d > maxAge || d < -maxAge {
		return false // too old (or too far in the future): a replayed capture
	}
	mac := hmac.New(sha256.New, []byte(secret))
	mac.Write([]byte(ts + "."))
	mac.Write(rawBody)
	want := "sha256=" + hex.EncodeToString(mac.Sum(nil))
	return hmac.Equal([]byte(want), []byte(h.Get("X-Obserae-Signature")))
}

Both use a constant-time comparison: a plain == leaks, one byte at a time, how close a forged signature is.

A note on secrets

Every credential you enter — a webhook signing secret, a Gotify / Slack / Mattermost / Telegram bot token, a Splunk HEC token, an Elasticsearch password or API key, a PagerDuty routing key, an Opsgenie API key, an SMTP password — is stored by obserae and used only to sign or authenticate its outgoing requests. They are never shown again in the interface or the API (you’ll see “secret is set”), and never appear in logs. When editing an output, leave the secret field blank to keep the existing one, or type a new value to replace it. (Syslog has no credential to store.)

See also: Alerting · NFQL · Configuration.