Copy page

Core Concepts

Monitoring and Alerting

Once a pipeline is running, you need to know it stays healthy and be told when something needs attention. Monad gives you pipeline health and per-node status for monitoring, a set of alert rules for notification, and schema drift detection for catching upstream changes before they cause silent data loss.

Monitoring pipeline health

Monad reports a single status for a pipeline and for each node inside it, drawn from the same set of values. A pipeline's status summarizes its nodes: it shows the most significant state among them — an error on any node surfaces as Erroring — reads Running only once every enabled node is running, and reads Disabled when the whole pipeline is turned off.

• Pending — enabled, but not yet reporting a running state. This is the brief state before a node's components begin processing; it resolves to Running.
• Initializing — starting up but not yet processing data. For push-based inputs (HTTP, OTLP, Splunk HEC, syslog), ingest requests return 503 Service Unavailable with pipeline is not ready and a Retry-After header until the node is running — no data is lost. This is expected briefly after a pipeline is created or redeployed.
• Running — actively processing data. This is the desired steady state.
• Paused — not processing, but incoming edges stay open, so data buffers upstream and drains once the node is resumed. The buffer is self-limiting: as it fills, the node enters Throttled. Monad does not drop buffered data.
• Throttled — back-pressure from high data volume has halted ingestion to keep the buffer bounded. Pull-based inputs pause and resume automatically; push-based inputs receive 429 Too Many Requests. Buffered data is retained — records are lost only if the source's own retention window is exceeded before ingestion resumes, or a push client ignores the 429.
• Erroring — an error occurred while processing: an input error (credentials, misconfiguration, network, or upstream throttling), a transformation error (a wrong operation for a key, or a data-format mismatch), or an output error (a schema or format mismatch at the destination). Check the node logs for details.
• Disabled — turned off; it will not process until re-enabled. A disabled node closes its incoming edges, so data is dropped rather than buffered — the key difference from Paused, which buffers it.
• Unknown — the status could not be determined. If a status stays unknown for more than five minutes, contact support@monad.com.

Paused buffers data while Disabled drops it. For the durability and delivery guarantees behind these states — burst back-pressure, durable buffering, and no-loss failover — see Reliability & Delivery.

The full reference, including sync types, is on the Pipeline Status Indicators page. For the delivery guarantees behind these states — durable buffering, failover, and at-least-once delivery — see Reliability & Delivery. For diagnosing a specific problem, see Troubleshooting.

Alert rules

Alert rules watch your pipelines and notify you when a condition is met. Rules are evaluated on a regular cycle: Monad collects the relevant data, evaluates the condition, generates an alert, and delivers it. Each alert carries a severity of critical, high, medium, low, or info.

You configure these alert types yourself:

• Threshold — a metric (bytes, records, or errors) crossing a value you set over a time window.
• Error rate — the error percentage over a window exceeding a threshold.
• Volume anomaly — statistically unusual ingress or egress volume, detected against learned history.
• Pipeline status — a pipeline that stays in an Erroring or Throttled state for the whole window.
• Organization logs — individual log events matching a level and/or a message filter.
• Cost budget — an organization's current spend exceeding a budget you set.

You create and scope these rules — to specific pipelines or all of them — in the UI or via the API. Once an alert fires for a given rule and resource it won't re-fire for at least an hour, and when the condition clears a RESOLVED event follows. See the Alerts overview for configuration details, deduplication, and resolution behavior.

Monad also provides one system-managed alert rule, Schema Drift Detection (below). It is created automatically for every organization and its settings are fixed — you can't create another or delete it — though you can turn it off if you don't want drift alerts.

Routing alerts to your team

Alert events are delivered through the Monad Alerts input, which needs no configuration or credentials. Because alerts arrive as records in a pipeline, you route them the same way you route any data — by severity, rule type, resource, or status — to destinations like PagerDuty, Slack, a SIEM, or storage.

Each alert is a JSON record with a consistent top-level shape — id, name, rule_id, rule_type, severity, description, created_at, a resource object (resource_type, resource_id), a status object whose state is FIRING or RESOLVED, and a metadata object whose fields vary by alert type. Those top-level fields are what your edge conditions match on — for example severity, rule_type, resource.resource_id, or status.state. See Record Structure for the full schema and a sample record.

If a destination expects a specific payload — a PagerDuty event, a Slack message, a particular SIEM schema — add a transform before the output to reshape the alert record into that format. The jq operation is the most flexible for building a new object from the alert's fields; see Enrichment, Normalization, and Routing for the full set of formatting tools.

Schema drift detection

Schema drift detection watches the data flowing across a pipeline edge and tells you when its shape changes — a new field appears, or an existing field changes type — which is a common cause of silent, downstream data loss.

It works in two phases. In learning, Monad builds a model of the edge's schema (fields, types, and how often they appear). Once the model has seen enough data it moves to detecting, comparing each incoming record against the learned schema. Detection is enabled per edge in the pipeline editor, and runs automatically with no transforms required. A new_field change is reported at low severity and a type_change at medium.

Drift alerts are system-managed: Monad creates the alert rule automatically for every organization, and its settings are fixed — you can't create another or delete it, though you can turn it off if you don't want drift alerts. It fires for any edge where you have enabled schema detection, and drift events appear in the alert stream with a before/after diff. Handling drift is a matter of triage: if the change was expected, rebaseline by resetting the schema; if it wasn't, remediate with a transform — rename_key for a renamed field, mutate_type for a type change, or drop_key for an unwanted field.

Schema drift detection is currently in beta — contact support@monad.com to enable it. The full walkthrough is in Schema drift detection, and the alert itself is documented at Schema Drift Detection.

Related

• Pipeline Status Indicators · Reliability & Delivery · Troubleshooting
• Alerts overview
• Schema drift detection
• Monad Alerts input
• What are Pipelines?