Monad Organization Logs
Collects internal logs from your Monad organization to monitor user activity and identify issues within your pipelines.
Sync Type: Incremental
Requirements
- You must have an active Monad organization.
- Proper access permissions to retrieve organization logs are required.
Details
The Monad Logs input retrieves logs from your Monad organization, allowing you to monitor activities and detect potential issues in your pipelines. This input leverages NATS JetStream to consume logs efficiently and can resume from the last processed log, ensuring no data is lost between executions.
Log Types
- API Logs record requests made to the Monad API for your organization, from the UI, API keys, or tools such as Terraform. Each entry records when the request happened, who made it (actor, authentication method, and role), what was requested (HTTP method, resource path, source IP, and user agent), and the response status and duration. Request and response bodies are not included. API Logs cover calls made to Monad; they do not record the calls Monad's connectors make to your own systems or cloud accounts.
- Pipeline Logs record operational messages from your pipeline nodes, such as startup, status, and error messages, tagged with the pipeline and node that produced them. They describe connector status and errors and do not include the content of your records.
Implementation
The Monad Logs input is designed to seamlessly integrate with your Monad organization. Once configured, it automatically retrieves logs from your organization, ensuring that you have a continuous and up-to-date view of all actions and issues within your pipelines.
This input operates efficiently, processing logs in real-time while maintaining a record of the last retrieved log to prevent any data loss. It handles errors gracefully and ensures that your logging data is always consistent and accurate.
Whether you're monitoring ongoing activities or troubleshooting pipeline issues, the Monad Logs input provides a reliable and straightforward solution for log management.
NOTE: Monad keeps track of the last 1,000 logs per pipeline node and 1,000 logs for API activity. Once the limit is reached, the oldest logs are removed to make room for new logs.
Configuration
The following configuration defines the input parameters. Each field's specifications, such as type, requirements, and descriptions, are detailed below.
Settings
| Setting | Type | Required | Default | Description |
|---|---|---|---|---|
| Log Type | string | Yes | "All" | Select which types of logs to collect: API Logs record requests made to the Monad API for your organization, Pipeline Logs show execution details from your pipeline nodes, or choose All Logs to collect both types. See Log Types. Possible values: "API", "Pipeline", "All" |
Secrets
None.
Sample Record
Code
Sync frequency
By default this input polls approximately every 10 seconds, with each sync beginning after the previous one completes. A cron schedule configured on the pipeline overrides this cadence. See Input Sync Frequency for details.
Community transforms
The Monad Transforms Repository has ready-made transforms for this input. Import one into your organization, then add it to a pipeline after this input.
| Format | Transform |
|---|---|
| OCSF v1.9.0 | Monad Organization Logs to OCSF v1.9.0 |
| ECS v8.11.0 | Monad Organization Logs to ECS |
| ECS v9.5.0 | Monad Organization Logs to ECS v9.5.0 |
| CEF | Monad Organization Logs to CEF |
Older OCSF versions (v1.1.0) are also in the repository.
CEF transforms emit an intermediate record; add the CEF Serializer transform after one to produce the CEF:0 string.