UPDATED · 17 JUN 2026 · EDIT ON GITHUB
ARCHITECTURE · DATA FABRIC

The data fabric.

The fabric is a schemaless, append-only event log. Every signal — endpoint telemetry, cloud audit log, identity event, network flow — lands in the same store in its native shape and is decorated with enrichments (asset, identity, geo, threat-intel) on the way in. There is no “normalization pass” that loses information.

Why It Matters

  • One join key — every event carries a stable entity_id for both the actor and the target, so cross-source correlation is a JOIN, not an integration.
  • Decoupled compute — detections, search, and replay all run against the same store. New detection? It can replay yesterday's events without re-ingesting anything.
  • Per-tenant cells — each customer's data lives in an isolated cell with customer-managed keys. No shared multi-tenant tables.

Stream catalog

StreamSourcep50 latencyRetention
endpoint.processEDR agents180 ms365 d hot
identity.signinIdP audit log240 ms365 d hot
cloud.auditCloudTrail, Activity, Audit210 ms365 d hot
network.flowVPC / NetFlow / ZTNA320 ms90 d hot · 7 y cold
saas.auditM365 · Okta · GitHub …260 ms365 d hot

The canonical event

Every signal becomes the same envelope: a typed event with the original payload preserved under raw and enrichments attached under ctx. Detections and search read the enriched fields; an investigator can always drop back to raw. Nothing is discarded on the way in.

EVENT · endpoint.process
{
  "id":     "ev_8X2k9Qd",
  "ts":     "2026-06-20T14:21:08.402Z",
  "type":   "endpoint.process",
  "entity": { "actor": "usr_a91", "target": "host_3f2" },
  "ctx":    { "asset": "fin-db-02", "geo": "US-VA", "intel": ["IRONVEIL"] },
  "raw":    { "ppid": 644, "cmd": "rundll32 …", "signed": false }
}

Storage tiers & retention

The single logical store is backed by three physical tiers. Detections and search span them transparently; you choose the boundaries per stream.

TierBacked byTypical windowServes
HotIn-memory + NVMe0–365 dLive detection, interactive search
WarmObject store90 d–2 yInvestigation, replay, back-test
ColdArchival + WORMup to 7 yCompliance, legal hold

Replay & search

Because compute is decoupled from storage, any detection plan can be replayed over historical events — to back-test a new rule before it ships, or to re-score the past after threat intelligence updates. Search runs against the same store, so there is no separate index to keep in sync and no window where new data is unsearchable.

Correlation is a JOIN, not an integration. Because every stream shares entity_id, asking “what else did this host do in the five minutes around the alert?” is one query across endpoint, identity, cloud, and network — no per-source connectors to reconcile.
← PREV System Overview NEXT → Detection Pipeline