Pipelines
Pipeline management endpoints
Get node retry queue
Required Permissions:
pipeline:data:read
Return records currently queued for retry on a pipeline node
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
node_idNode ID
query Parameters
limitMax records to return (1-10, default 10)
meta_onlyOmit record payloads — a cheap presence check
Get node retry queue › Responses
Records queued for retry
created_atdatadata_bytesDataBytes is the payload size in bytes. Omitted is set when the payload was left out for exceeding maxRetryRecordBytes. (meta_only leaves out every payload without setting Omitted — it isn't a size decision.)
eligible_atEligibleAt is a rough estimate of the earliest retry time (see estEligibleAt), not an authoritative floor — it under-estimates output/enrichment nodes.
error_node_idlast_errorlast_seen_atomittedretry_countstream_seqList pipelines
Get pipeline
Required Permissions:
pipeline:read
Get pipeline
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
Get pipeline › Responses
Pipeline details
component_tiercreated_atcron_scheduledescriptionenabledidinput_idmanaged_bynameorganization_idtagscustomer tag names
updated_atUpdate pipeline
Required Permissions:
pipeline:write
Update pipeline
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
Update pipeline › Request Body
Decision Table
| Variant | Matching Criteria |
|---|---|
| type = object | |
| type = object |
Update pipeline › Responses
Response body for updating a pipeline
component_tiercreated_atcron_scheduledescriptionenabledidinput_idmanaged_bynameorganization_idtagscustomer tag names
updated_atGet pipeline metrics
Get time series metrics for a pipeline
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
query Parameters
metricMetric to retrieve (ingress_bytes|egress_bytes|ingress_records|egress_records|errors)
startISO3339 start time, default 6 hours ago
endISO3339 end time, default now
resolutionResolution of the data, default determined by time window
Get pipeline metrics › Responses
Pipeline metrics
end_atmetricnode_idnode_slugorganization_idorganization_namepipeline_idpipeline_nameresolutionstart_atGet pipeline node metrics
Get pipeline node metrics
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
node_idNode ID
query Parameters
metricMetric to retrieve (ingress_bytes|egress_bytes|ingress_records|egress_records|errors)
startISO3339 start time, default 6 hours ago
endISO3339 end time, default now
resolutionResolution of the data, default determined by time window
Get pipeline node metrics › Responses
Pipeline node metrics
end_atmetricnode_idnode_slugorganization_idorganization_namepipeline_idpipeline_nameresolutionstart_atGet status of all pipelines for an organization
Required Permissions:
pipeline:read
Get status of all pipelines for an organization
path Parameters
organization_idOrganization ID
query Parameters
startISO3339 start time, default 24 hours ago
endISO3339 end time, default now
Get status of all pipelines for an organization › Responses
Organization Pipeline Summary
organization_idList pipelines
Required Permissions:
pipeline:read
List pipelines
path Parameters
organization_idOrganization ID
query Parameters
limitLimit
offsetOffset
include_statusInclude the status of the pipeline nodes
List pipelines › Responses
List of pipelines
Pagination metadata.
Create pipeline
Required Permissions:
pipeline:write
Create a new pipeline with specified configuration
path Parameters
organization_idOrganization ID
Create pipeline › Request Body
Decision Table
| Variant | Matching Criteria |
|---|---|
| type = object | |
| type = object · requires: edges, name, nodes |
Create pipeline › Responses
Pipeline created successfully
billingAccountIdcomponent_tiercreatedAtcron_scheduledescriptionenabledidis_syntheticmanaged_bynamenext_cron_run_atorganizationIdorganizationNameDeprecated: Never set. Removed along with pkg/datastore/postgres.
routableRoutable is set by the api from the inputs registry when the pipeline's input is a push (HTTP, OTEL, TCP) input, and the pipeline-operator labels the Pipeline from it. omitempty keeps it off customer reads and leaves the ConfigHash unchanged for non-push pipelines.
tagsTags is the pipeline's customer tag names, populated only on customer reads (never on the operator path), so omitempty keeps the ConfigHash unchanged and tag edits don't roll pods.
updatedAtGet metrics for specific pipelines
Required Permissions:
pipeline:read
Get aggregated ingress and egress metrics for specific pipelines
path Parameters
organization_idOrganization ID
query Parameters
pipeline_idsComma-separated list of pipeline IDs
resolutionResolution for metrics (default: 5m)
Get metrics for specific pipelines › Responses
Pipeline metrics data
Get pipeline status
Required Permissions:
pipeline:read
Retrieve the status of pipelines within an organization for a specified time period.
path Parameters
organization_idOrganization ID
query Parameters
limitMaximum number of pipelines to return (default is 10)
offsetOffset for pagination (default is 0)
startStart time (RFC3339 format) for status metrics (default: 24 hours ago)
endEnd time (RFC3339 format) for status metrics (default: now)
Get pipeline status › Responses
List of pipelines with status
idstatuslast_ingested_timeGet pipeline configuration
Required Permissions:
pipeline:read
Retrieve a specific pipeline configuration by pipeline ID
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
query Parameters
include_statusInclude the status of the pipeline nodes
Get pipeline configuration › Responses
Pipeline configuration retrieved successfully
billingAccountIdcomponent_tiercreatedAtcron_scheduledescriptionenabledidis_syntheticmanaged_bynamenext_cron_run_atorganizationIdorganizationNameDeprecated: Never set. Removed along with pkg/datastore/postgres.
routableRoutable is set by the api from the inputs registry when the pipeline's input is a push (HTTP, OTEL, TCP) input, and the pipeline-operator labels the Pipeline from it. omitempty keeps it off customer reads and leaves the ConfigHash unchanged for non-push pipelines.
tagsTags is the pipeline's customer tag names, populated only on customer reads (never on the operator path), so omitempty keeps the ConfigHash unchanged and tag edits don't roll pods.
updatedAtUpdate pipeline
Required Permissions:
pipeline:write
Update an existing pipeline with the specified configuration
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
Update pipeline › Request Body
Decision Table
| Variant | Matching Criteria |
|---|---|
| type = object | |
| type = object |
Update pipeline › Responses
Pipeline updated successfully
billingAccountIdcomponent_tiercreatedAtcron_scheduledescriptionenabledidis_syntheticmanaged_bynamenext_cron_run_atorganizationIdorganizationNameDeprecated: Never set. Removed along with pkg/datastore/postgres.
routableRoutable is set by the api from the inputs registry when the pipeline's input is a push (HTTP, OTEL, TCP) input, and the pipeline-operator labels the Pipeline from it. omitempty keeps it off customer reads and leaves the ConfigHash unchanged for non-push pipelines.
tagsTags is the pipeline's customer tag names, populated only on customer reads (never on the operator path), so omitempty keeps the ConfigHash unchanged and tag edits don't roll pods.
updatedAtUpdate pipeline edge
Required Permissions:
pipeline:write
Enable or disable a pipeline edge
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
edge_idEdge ID
Update pipeline edge › Request Body
Decision Table
| Variant | Matching Criteria |
|---|---|
| type = object | |
| type = object |
Update pipeline edge › Responses
Edge updated successfully
Force graduate schema state
Required Permissions:
pipeline:write
Force an edge in learning mode to immediately graduate to detection mode
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
edge_idEdge ID
Force graduate schema state › Responses
Schema state graduated to detection mode
Reset schema state
Required Permissions:
pipeline:write
Delete the current schema state for an edge and signal the sidecar to restart in learning mode
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
edge_idEdge ID
Reset schema state › Responses
Schema state reset successfully
Get pipeline status
Required Permissions:
pipeline:read
Get pipeline status
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
query Parameters
metricsMetrics to retrieve (all|health|ingress_bytes|egress_bytes|ingress_records|egress_records|backpressure)
startISO3339 start time, default 24 hours ago
endISO3339 end time, default now
Get pipeline status › Responses
Pipeline status
average_size_egressedaverage_size_ingestederrorslast_ingested_timelast_updated_atorganization_idorganization_namepipeline_idpipeline_namestatusGet pipeline node status
Required Permissions:
pipeline:read
Get pipeline node status
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
node_idNode ID
query Parameters
metricsMetrics to retrieve (all|health|ingress_bytes|egress_bytes|ingress_records|egress_records|backpressure)
startISO3339 start time, default 24 hours ago
endISO3339 end time, default now
Get pipeline node status › Responses
Pipeline Node status
avg_bytes_per_record_egressavg_bytes_per_record_ingresscomponent_typecomponent_type_iderrorslast_ingested_timelast_record_processed_timelast_updated_atnode_idnode_slugstatusTest a pipeline node connection
Required Permissions:
pipeline:write
Tests the connection for a node's submitted config (effective config plus one-shot ephemeral values). Nothing is persisted.
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
Test a pipeline node connection › Request Body
Decision Table
| Variant | Matching Criteria |
|---|---|
| type = object | |
| type = object |
Test a pipeline node connection › Responses
Connection test successful
messageGet pipeline node metrics
Required Permissions:
pipeline:read
Get pipeline node metrics
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
node_idNode ID
query Parameters
metricMetric to retrieve (ingress_bytes|egress_bytes|ingress_records|egress_records|errors)
startISO3339 start time, default 6 hours ago
endISO3339 end time, default now
resolutionResolution of the data, default determined by time window
Get pipeline node metrics › Responses
Pipeline node metrics
end_atmetricnode_idnode_slugorganization_idorganization_namepipeline_idpipeline_nameresolutionstart_atGet schema state
Get the current schema state for a pipeline edge
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
edge_idEdge ID
Get schema state › Responses
Current schema state
edge_idlearning_startmodepipeline_idtotal_records_observedupdated_atList schema history
List schema drift events for a pipeline edge
path Parameters
organization_idOrganization ID
pipeline_idPipeline ID
edge_idEdge ID
List schema history › Responses
Schema history
created_atedge_idevent_tagsid