# Enrichment, Normalization, and Routing

Three capabilities turn raw, inconsistent input into useful, well-targeted security data: **normalization** makes heterogeneous records consistent, **enrichment** adds context to them, and **routing** sends them to the right destinations. All three happen inside a pipeline, between your inputs and outputs.

A useful way to think about the order: **normalize first** so records share a consistent shape, **enrich second** using those now-consistent fields as lookup keys, and **route last** based on record content — including the results of enrichment.

## Normalization

Normalization is the job of [transforms](/docs/transforms) — the processing components that sit between input and output and let you clean, normalize, and reshape data before it reaches its destination. Transforms work **one record at a time**: there are no aggregation, grouping, or windowing operations, and a single transform holds at most 20 operations (chain more transforms if you need more). Most operations use GJSON/SJSON path syntax; the `jq` operation is the exception and runs a full query engine.

Common normalization operations:

- **[Convert timestamp](/docs/transforms/convert_timestamp)** — parse timestamps in many formats (auto-detected by default) and convert them to a single canonical format and timezone. This is the go-to for making event times consistent across sources.
- **[Rename key](/docs/transforms/rename_key)** — move a field to a canonical name.
- **[Mutate type](/docs/transforms/mutate_type)** — cast a value to a consistent type, e.g. the string `"123"` to the number `123`.
- **[Flatten](/docs/transforms/flatten)** and **[Flatten all](/docs/transforms/flatten-all)** — lift nested structures into a single-level keyspace.
- **[jq](/docs/transforms/jq)** — reshape a record wholesale: build new objects, restructure fields, and map a record into a target schema. Mapping records to schemas like OCSF, ECS, or CEF is done here rather than with a dedicated operator.

Normalization is also how you respond when an upstream source changes its shape — see [Schema drift detection](/docs/guides/schema-detection), which recommends `rename_key`, `mutate_type`, and `jq` as remediation transforms.

## Enrichment

[Enrichments](/docs/enrichments) are middleware components that augment records in flight with context from internal and external sources. Every enrichment follows the same pattern: extract a value from the record using a configurable **join key** (for example `source.ip`), look it up, and write the result to a **destination path**. Lookups degrade gracefully — a failed enrichment does not stop the record — and results carry a status of `success`, `no_match`, or `error`.

Documented enrichments fall into two categories:

- **Internal data** — enrich against data your organization controls. [KV Lookup](/docs/enrichments/monad/kv-lookup) joins records against values stored in a Monad KV Lookup output, useful for asset inventories, user context, or your own threat-intel tables.
- **External intelligence** — enrich against third-party data. [Geolocus](/docs/enrichments/Onyphe/geolocus) resolves an IP address to geolocation details, and [GreyNoise Community](/docs/enrichments/greynoise/greynoise-community) adds internet-scanner threat classification for an IP.

Because enrichment writes its result back into the record, downstream routing can make decisions based on it.

## Routing

Routing decides which records reach which destination, and it is configured on the **edges** between nodes in a pipeline. Each edge is a routing rule. A node has a single incoming edge but can fan out to many outgoing edges, and every edge is evaluated independently for every record — so one record can match several edges and be delivered to several destinations at once, regardless of order.

An edge condition has a two-level structure: a **logical operator** (`always`, `never`, `and`, `or`, `nor`, `xor`) wrapping one or more **leaf conditions**. The root of a condition must always be a logical operator — even a single leaf is wrapped — and conditions nest up to three levels deep. Leaf conditions test record content and all support a `not` modifier:

[key_exists](/docs/conditionals/key-exists), [equals](/docs/conditionals/equals), [equals_any](/docs/conditionals/equals-any), [contains](/docs/conditionals/contains), [starts_with](/docs/conditionals/starts-with), [ends_with](/docs/conditionals/ends-with), [matches_regex](/docs/conditionals/matches-regex), [greater_than](/docs/conditionals/greater-than), [less_than](/docs/conditionals/less-than), and [sample](/docs/conditionals/sample).

Common patterns include a simple pass-through (`always`), multi-criteria filtering (`and` combining `key_exists` and `equals_any`), and priority fan-out (separate critical, standard, and archive edges from one node). See [Data Routing](/docs/routing) and [Conditionals](/docs/conditionals) for the full operator reference and key-path syntax.

## Putting it together

In a typical pipeline, a record is first normalized so its fields are predictable, then enriched so it carries the context your team needs, and finally routed so it lands in the right place. For how these components fit into the pipeline as a whole, see [What are Pipelines?](/docs/guides/what-are-pipelines).

## Related

- [Transforms](/docs/transforms) · [Enrichments](/docs/enrichments)
- [Data Routing](/docs/routing) · [Conditionals](/docs/conditionals)
- [Schema drift detection](/docs/guides/schema-detection)
- [What are Pipelines?](/docs/guides/what-are-pipelines)