Redshift Audit Logs
Ingests Amazon Redshift database audit logs (connection and user logs) delivered to an Amazon S3 bucket. These logs help you monitor authentication, connection activity, and changes to database user definitions for security and compliance.
Sync Type: Incremental
Overview
Amazon Redshift can publish database audit logs to an S3 bucket. This connector reads those log files from S3, parses them, and emits one record per log line. It is a focused wrapper over the S3 input: you choose which audit log type to ingest, and Monad handles the bucket layout, file format, and incremental syncing for you.
How It Works
- Redshift delivers gzip-compressed, pipe-separated audit log files to your S3 bucket under
AWSLogs/{AccountID}/redshift/{Region}/{Year}/{Month}/{Day}/. - Each file name carries the log type, e.g.
..._connectionlog_2013-10-29T18:01.gz. - Based on the selected Log Type, Monad processes only the matching files and parses each line using the documented column layout for that log type.
- An incremental cursor (based on object last-modified time and key ordering) tracks progress, so only new objects are processed on subsequent runs.
Requirements
1. Enable Audit Logging to S3 on Your Redshift Cluster
Audit logging is not enabled by default. Enable it and configure S3 as the destination:
- Navigate to Amazon Redshift in the AWS Console.
- Select your cluster, then go to Properties → Database configurations → Edit audit logging.
- Turn on audit logging and choose S3 as the log export type.
- Select or create a destination bucket (and optional key prefix).
For details, see Database audit logging.
2. Grant IAM Permissions
The role or credentials used must be able to list and read objects in the audit log bucket.
Code
See AWS Authentication Methods for role-assumption vs. static-credential setup.
Configuration
Settings
| Setting | Type | Required | Default | Description |
|---|---|---|---|---|
| Bucket | string | Yes | - | Name of the S3 bucket that receives Redshift audit logs. |
| Log Type | string | Yes | Connection Log | Which Amazon Redshift audit log to ingest. See Log Types. |
| Prefix | string | No | - | Prefix of the audit log keys, up to (but not including) the date partition — e.g. AWSLogs/123456789012/redshift/us-east-1. If you set a custom S3 key prefix when enabling audit logging, include it here. |
| Region | string | No | auto-detect | The region of the S3 bucket. If left blank, the region is auto-detected. |
| Role ARN | string | No | - | IAM Role ARN to assume when reading from S3. Use either this or static credentials, not both. |
| Backfill Start Time | string | No | - | ISO 8601 date (e.g. 2025-01-01T00:00:00Z) to start fetching from. If not specified, a full sync of existing data is performed on the first run; subsequent runs are incremental. |
Secrets
| Secret | Type | Required | Description |
|---|---|---|---|
| Access Key | string | Conditional | AWS Access Key ID. Required only if using static credentials instead of Role ARN. |
| Secret Key | string | Conditional | AWS Secret Access Key. Required only if using static credentials instead of Role ARN. |
Authentication Note: Choose either Role ARN (recommended) or static credentials (Access Key + Secret Key). See AWS Authentication Methods.
Log Types
Select a single log type per input. Each type has a fixed column layout; Monad emits one JSON record per log line with these fields.
Connection Log
Logs authentication attempts, connections, and disconnections.
| Field | Description |
|---|---|
event | Connection or authentication event. |
recordtime | Time the event occurred. |
remotehost | Name or IP address of remote host. |
remoteport | Port number for remote host. |
pid | Process ID associated with the statement. |
dbname | Database name. |
username | User name. |
authmethod | Authentication method. |
duration | Duration of connection in microseconds. |
sslversion | SSL version. |
sslcipher | SSL cipher. |
mtu | Maximum transmission unit. |
sslcompression | SSL compression type. |
sslexpansion | SSL expansion type. |
iamauthguid | IAM authentication ID for the CloudTrail request. |
application_name | Initial or updated application name for a session. |
os_version | Operating system version of the client machine. |
driver_version | ODBC/JDBC driver version. |
plugin_name | Name of the plugin used to connect. |
protocol_version | Internal protocol version used by the driver. |
sessionid | Globally unique identifier for the session. |
compression | Compression algorithm in use for the connection. |
User Log
Records changes to database user definitions (create, drop, rename, alter).
| Field | Description |
|---|---|
userid | ID of the user affected by the change. |
username | User name of the user affected by the change. |
oldusername | For a rename, the original user name; otherwise empty. |
action | Action that occurred. |
usecreatedb | If true (1), the user has create-database permissions. |
usesuper | If true (1), the user is a superuser. |
usecatupd | If true (1), the user can update system catalogs. |
valuntil | Password expiration date. |
pid | Process ID. |
xid | Transaction ID. |
recordtime | Time in UTC that the query started. |
Details
Incremental Syncing
- First sync: processes existing objects under the prefix (or from the specified backfill start time).
- Subsequent syncs: only objects newer than the last checkpoint are processed, tracked by last-modified time and lexicographic key ordering.
- Restart behavior: on interruption, the next run resumes from the day prefix of the last checkpointed timestamp. You will not lose records, though some objects on the in-progress page may be reprocessed after a catastrophic failure.
Bucket Structure
Redshift organizes audit logs as:
Code
Set Prefix to the path up to the region (e.g. AWSLogs/123456789012/redshift/us-east-1), prepending any custom key prefix you configured. The date partition ({Year}/{Month}/{Day}) is handled automatically.
Log Format
Audit log files are gzip-compressed, headerless, pipe-separated values. Monad applies the column layout for the selected log type, so no header row is expected in the files.