Copy page

Getting Started

Status Indicators

This document provides an overview of the health statuses and status indicators used in Monad to monitor the health and state of pipelines. These indicators are essential for understanding the current behavior of the pipeline and identifying any issues that may need attention.

For the durability and delivery guarantees behind these states — burst back-pressure, durable buffering, and no-loss failover — see Reliability & Delivery.

Health Statuses

The health status provides a high-level overview of the pipeline's condition:

• Healthy: The pipeline is functioning as expected, and data is being processed without issues.
• Unhealthy: The pipeline is experiencing issues, and one or more components have errored.

Status Indicators

The status indicators give a more detailed look into the current operational state of the pipeline at various nodes. These indicators help you understand whether the pipeline is actively processing data or if there are issues that need to be addressed.

1. Initializing

• The pipeline node is starting up but hasn't begun processing data yet. This is a transitional state and should resolve to Running once resource are done initializing.
• Push-based inputs (HTTP, OTEL, Splunk HEC, syslog): until the pipeline reaches Running, ingest requests return a 503 Service Unavailable with pipeline is not ready. This is expected briefly after a pipeline is created or redeployed, and no data is lost - the endpoint refuses traffic it cannot yet route rather than dropping it. The response includes a Retry-After header; retry until it succeeds, or poll the pipeline status and send data once it reports Running.

2. Running

• The pipeline node is actively running and processing data. This is the desired state when the pipeline is operating normally.

3. Paused

• The node is paused: it is not processing data, but its incoming edges remain open, so data buffers upstream and drains once the node is re-enabled. Monad does not drop buffered data.
• The buffer is self-limiting: as it fills, the pipeline enters the Throttled state (below), which halts input ingestion so the buffer cannot grow without bound — there is no disk-exhaustion risk. Data is only at risk if a node is left paused longer than the source can retain it (see Throttled).

4. Disabled

• The pipeline node or pipeline is disabled. It will not process any data until re-enabled.
• For a disabled node, the incoming edges are closed, so data is dropped rather than buffered. On re-enable, the node resumes from live data with no backlog. (This is the key difference from Paused, which buffers data instead of dropping it.)

5. Erroring

• Errors have occurred while processing data. Check the node logs for specific details on the error and other indicators to resolve the issue.
• Input Errors: Occur when trying to establish a connection or retrieve data from the input. This could include issues like incorrect credentials, misconfigurations, or network problems when setting up the connection, or throttling by external services.
• Transformation Errors: Happen during the data transformation stage. For example, if the transformation applies the wrong modification to a key (e.g., converting instead of masking a key), or if there is a mismatch in the expected data format (e.g.,flattening missing arrays).
• Output Errors: Occur when trying to send data to the output. These could be caused by mismatches between the data being sent and the expected format or schema of the destination.

6. Throttled

• The pipeline is experiencing backpressure due to high data volume and is temporarily not accepting new data until it can catch up. Throttling halts ingestion at the input to keep the buffer bounded; already-buffered data is retained, not dropped.
• Pull-based inputs: The pipeline saves its current state and pauses data ingestion. It will automatically resume processing once the backlog is cleared.
• Push-based inputs (HTTP, OTEL, Splunk, etc.): The pipeline returns a 429 Too Many Requests error and will not accept data until capacity is available.
• When data can be lost: Monad holds buffered data until the backlog clears, so no data is lost on Monad's side. Records are only lost if the pipeline stays throttled longer than the source's own retention window (pull-based inputs age out data at the source before ingestion resumes), or if a push-based client does not honor the 429 responses.

7. Unknown

• The current status of the pipeline cannot be determined. This occurs if the system is unable to retrieve the health status for this node. If a status remains unknown for more than 5 minutes, please contact support@monad.com

Sync Types

Monad input components use different synchronization methods to retrieve and process data. Understanding these sync types is important for proper expectations of pipeline behavior:

1. Incremental

• Components with incremental sync maintain state and continue data processing from where they left off in the previous run.
• Incremental syncs use techniques like timestamps, watermarks, or other tracking mechanisms to identify and process only new or changed data.
• Characteristics: More efficient resource usage, reduced data transfer, and faster processing times.
• These are commonly used in components handling continuous data streams or large datasets.

2. Full Synchronisation

• Components with full synchronisation ingest ALL of the data each time they run.
• These syncs completely refresh the destination data, replacing previous data with a fresh copy.
• Full synchronisation is usually controlled via a cron job or scheduled trigger to limit resource usage.
• Characteristics: Ensures complete data consistency but requires more resources per run.
• This approach is typically used when the source system doesn't support bookmarking features or other state-tracking mechanisms.

The sync type is an inherent property of each component and cannot be changed by the end user. Each component's documentation specifies which sync type it uses. Understanding a component's sync type helps to set appropriate expectations for data processing behavior and resource usage.

Summary

• Healthy/Unhealthy: Provide a quick overview of the pipeline's overall condition.
• Status Indicators: (Initializing, Running, Paused, Disabled, Erroring, Throttled, Unknown) give detailed insight into the operational state of the pipeline. Paused buffers data while Disabled drops it.
• Sync Types: (Incremental, Full Synchronisation) define how components retrieve and process data over time.

Monitoring the health status, status indicators, and understanding the sync types will provide a clear understanding of the pipeline's performance and any necessary actions.