Copy page

AWS

EKS Audit Logs

Ingests Kubernetes API server audit logs from Amazon EKS clusters via CloudWatch Logs. These audit logs provide detailed records of all API requests made to the Kubernetes control plane, essential for compliance, security monitoring, and troubleshooting.

Sync Type: Incremental

Overview

Amazon EKS control plane audit logging captures all API requests made to your Kubernetes API server. This connector polls CloudWatch Logs to retrieve these audit events, maintaining an incremental cursor to ensure only new logs are ingested on subsequent syncs.

How It Works

1. AWS EKS publishes Kubernetes API server audit logs to CloudWatch Logs under the log group /aws/eks/<cluster-name>/cluster
2. Monad polls this log group at regular intervals using the CloudWatch Logs FilterLogEvents API
3. An incremental timestamp cursor tracks the last processed log, ensuring only new events are fetched on subsequent runs
4. On first run, audit logging must be enabled on the EKS cluster; Monad will begin ingesting logs from that point forward (or from a specified backfill start time)

Requirements

Before using this connector, ensure the following prerequisites are met:

1. Enable Control Plane Audit Logging on Your EKS Cluster

By default, control plane audit logging is disabled on EKS clusters. You must explicitly enable it for this connector to receive data.

Using AWS Console:

1. Navigate to Amazon EKS in the AWS Console
2. Select your cluster
3. Go to the Observability tab
4. Under Control plane logging, toggle the Audit log type to On
5. Click Save changes

Using AWS CLI:

Code
aws eks update-cluster-config \
  --name <cluster-name> \
  --region <region> \
  --logging '{"clusterLogging":[{"types":["audit"],"enabled":true}]}'

After enabling, AWS will create the CloudWatch log group /aws/eks/<cluster-name>/cluster and begin publishing audit logs within a few minutes.

2. Grant IAM Permissions

The role or credentials used must have permissions to read from the CloudWatch log group containing the audit logs.

Minimum Required Permissions:

Code
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "logs:DescribeLogGroups",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "logs:FilterLogEvents",
      "Resource": "arn:aws:logs:<region>:<account-id>:log-group:/aws/eks/<cluster-name>/cluster:*"
    }
  ]
}

Replace:

• <region> with your AWS region (e.g., us-east-1)
• <account-id> with your 12-digit AWS account ID
• <cluster-name> with your EKS cluster name

For Role ARN Authentication:

1. Create or update an IAM role in your AWS account
2. Add the above permissions to the role
3. Update the trust relationship to allow Monad to assume it (see AWS Authentication Methods)

For Static Credentials:

1. Create or use an IAM user with programmatic access
2. Attach the above policy
3. Generate or use existing Access Key ID and Secret Access Key

Configuration

Settings

Setting	Type	Required	Default	Description
Cluster Name	string	Yes	-	The name of the EKS cluster.
Region	string	Yes	us-east-1	The AWS region where the EKS cluster is running. Must be a valid AWS region.
Role ARN	string	No	-	IAM Role ARN to assume for accessing CloudWatch Logs. Use either this or static credentials (Access Key + Secret Key), not both.
Backfill Start Time	string	No	-	ISO 8601 formatted date (e.g., 2025-01-01T00:00:00Z) to start fetching historical audit logs from. If not specified, ingestion begins from the current time.
Ingestion Lag (seconds)	number	No	10	Absorbs CloudWatch ingestion lag so late-arriving audit events are not dropped. Minimum 5s (unset defaults to 10s), maximum 14 days (1209600 seconds). See the Ingestion Lag Time section below for how to size this in your cluster.

Secrets

Secret	Type	Required	Description
Access Key	string	Conditional	AWS Access Key ID. Required only if using static credentials instead of Role ARN.
Secret Key	string	Conditional	AWS Secret Access Key. Required only if using static credentials instead of Role ARN.

Authentication Note: Choose either Role ARN (recommended for security) or static credentials (Access Key + Secret Key). See AWS Authentication Methods for detailed setup instructions.

Details

Incremental Syncing

This connector maintains state using an incremental timestamp cursor:

• First sync: Monad begins ingesting audit logs from the time the input is created (or from the specified backfill_start_time)
• Subsequent syncs: Only logs with timestamps after the last checkpoint are processed
• Checkpointing: The cursor is updated after each batch of logs is successfully processed
• Restart behavior: If a sync is interrupted, the next run resumes from the last checkpointed timestamp

Log Group and Stream Structure

EKS publishes audit logs to:

• Log Group: /aws/eks/<cluster-name>/cluster
• Log Streams: Multiple streams within the log group, each prefixed with a timestamp

Monad queries the entire log group using FilterLogEvents, which automatically searches across all streams.

Filtering and Limitations

• No filtering: Monad retrieves all audit logs from the specified log group. AWS EKS does not allow filtering at the CloudWatch Logs API level for audit logs.
• Log retention: By default, CloudWatch Logs retains logs indefinitely. You can set a retention policy in the AWS Console to manage costs.
• Rate limiting: CloudWatch Logs API has rate limits (typically 1,000 requests per second). For clusters with high API activity, monitor CloudWatch Logs metrics to ensure limits are not exceeded.

Audit Log Format

EKS audit logs are JSON objects conforming to the Kubernetes audit event specification. Each event includes:

• verb: API action (e.g., create, get, list, delete)
• user: User identity and authentication info
• sourceIPs: Source IP address(es) of the request
• objectRef: Reference to the Kubernetes object affected
• requestObject/responseObject: Request and response payloads (if enabled in audit policy)
• annotations: Additional metadata

For details on the audit log format, see the Kubernetes Audit Event Specification.

Ingestion Lag Time

Some audit events reach CloudWatch late — after the moment they actually occurred on the cluster. For EKS this is usually a few seconds, but delays stretch into minutes during control-plane scaling, API server throttling, or CloudWatch slowdowns. If Monad has already moved past that moment, those late events would be missed.

The Ingestion Lag (seconds) tells Monad to look a little further back in time on every poll so late arrivals are still picked up. Anything Monad has already delivered is filtered out automatically, so you never see duplicates downstream.

How to choose a value:

• Leave it unset to use the default: 10 seconds. Works for most clusters under normal load.
• Raise it if your cluster experiences frequent control-plane scaling, API server throttling, or has a history of CloudWatch delivery issues.
• Allowed range: 5 seconds to 1209600 seconds (14 days).

Measuring ingestion lag in your cluster

If you want a data-driven number instead of guessing, you can measure your cluster's actual audit-log delay directly in the AWS Console.

Steps (Console → CloudWatch → Logs → Logs Analytics):

1. Select the log group /aws/eks/<cluster-name>/cluster.
2. Set the time range to 7 days.
3. Paste and run:

Code
fields @ingestionTime - @timestamp AS lag_ms, @logStream
| filter @logStream like /kube-apiserver-audit/
| stats max(lag_ms) AS max_lag_ms,
        avg(lag_ms) AS avg_lag_ms,
        pct(lag_ms, 99) AS p99_lag_ms
        BY bin(1h)
| sort @timestamp desc

Read the results:

• Look at p99_lag_ms — this is the delay 99% of your audit events land inside. Divide by 1000 and round up to get a safe value in seconds.
• If max_lag_ms regularly spikes into the minutes (typically during control-plane scaling or throttling), use that spike size instead.
• Steady, low-traffic clusters usually stay well under the default; no change needed.

Rule of thumb: start with the default. Only raise it if the query above tells you delays are larger, or if you notice missing events after a known incident. Setting it to hours or days "just in case" adds needless work without improving reliability.

Rate Limits

The connector respects CloudWatch Logs API rate limits:

Limit	Value	Notes
API Requests per second	1,000	Shared across all API operations in your account
Concurrent requests	Limited by account quota	Monad uses a single worker thread per input
Log event payload	≤ 1 MB per batch	CloudWatch automatically batches smaller events

Source: AWS CloudWatch Logs API Reference

Monitoring: Use CloudWatch Metrics to monitor:

• IncomingLogEvents — Number of log events published
• IncomingBytes — Volume of log data published

Related Articles

• Amazon EKS Control Plane Audit Logging
• Kubernetes Audit Event Specification
• AWS CloudWatch Logs FilterLogEvents API

Sync frequency

By default this input polls approximately every 10 seconds, with each sync beginning after the previous one completes. A cron schedule configured on the pipeline overrides this cadence. See Input Sync Frequency for details.