Skip to main content

Amazon SQS

Publishes data to specified Amazon SQS queues. Supports both Standard and FIFO queue types.

Overview​

The Amazon SQS output connector for Monad enables you to publish data directly to Amazon SQS queues, supporting both Standard and FIFO queue types. It is designed to ensure efficient, secure, and reliable delivery of your data to SQS, making it easy to integrate Monad with AWS-native workflows and event-driven architectures.

Requirements​

To configure SQS as an output destination for Monad, complete the following steps:

  1. Create an IAM Role:
    Create an IAM role that allows Monad to assume it and access your SQS queue. You need to set up a trust relationship that permits Monad to assume the role. For more details, refer to the AWS IAM role creation guide.

    Copy and paste the trust relationship below:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "",
                "Effect": "Allow",
                "Principal": {
                    "AWS": [
                        "arn:aws:iam::339712996529:role/monad-app"
                    ]
                },
                "Action": [
                  "sts:AssumeRole",
                  "sts:TagSession"
                ]
            }
        ]
    }

  2. Set IAM Role Permissions:
    The role must have permissions to perform SQS actions including sqs:SendMessage.

    You can either set the permissions via UI or copy the permission policy below:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "",
                "Effect": "Allow",
                "Action": [
                    "sqs:SendMessage",
                ],
                "Resource": "arn:aws:sqs:{region}:{account-id}:{queue-name}"
            }
        ]
    }

    Note: Be sure to update the Resource field in the policy below to match your actual SQS queue ARN. Replace {region}, {account-id}, and {queue-name} with your AWS region, account ID, and queue name, respectively.

  3. Alternative: Using AWS CLI:
    You can also create the required IAM role and permissions using AWS CLI commands:

    # Set your variables
    ROLE_NAME="MonadSQSOutputRole"
    QUEUE_NAME="your-queue-name"
    AWS_REGION="your-region"
    AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

    # Create the IAM role with trust policy
    aws iam create-role \
        --role-name "$ROLE_NAME" \
        --assume-role-policy-document '{
          "Version": "2012-10-17",
          "Statement": [
            {
              "Effect": "Allow",
              "Principal": {
                "AWS": ["arn:aws:iam::339712996529:role/monad-app"]
              },
              "Action": ["sts:AssumeRole", "sts:TagSession"]
            }
          ]
        }'

    # Create the IAM policy for SQS access
    aws iam create-policy \
        --policy-name "${ROLE_NAME}-policy" \
        --policy-document '{
          "Version": "2012-10-17",
          "Statement": [
            {
              "Effect": "Allow",
              "Action": [
                "sqs:SendMessage"
              ],
              "Resource": "arn:aws:sqs:'${AWS_REGION}':'${AWS_ACCOUNT_ID}':'${QUEUE_NAME}'"
            }
          ]
        }'

    # Attach the policy to the role
    aws iam attach-role-policy \
        --role-name "$ROLE_NAME" \
        --policy-arn "arn:aws:iam::${AWS_ACCOUNT_ID}:policy/${ROLE_NAME}-policy"

    After running these commands, note the role ARN which will be in the format: arn:aws:iam::<AWS_ACCOUNT_ID>:role/<ROLE_NAME>

Configuration​

The following configuration defines the input parameters. Each field's specifications, such as type, requirements, and descriptions, are detailed below.

Settings​

SettingTypeRequiredDescription
AWS IAM Role ARNstringYesThe Amazon Resource Name (ARN) of the IAM role to assume which grants access to the SQS queue.
Queue URLstringYesThe complete URL of the SQS queue where messages will be sent.
Queue TypestringYesThe type of SQS queue. Options are "standard" or "fifo".
Message Group IDstringConditionalRequired for FIFO queues only. Groups related messages together to ensure ordering within the group.

Secrets​

None.

Queue Type Options​

  1. Standard Queue ("standard"):

    • Description: Provides at-least-once delivery and best-effort ordering with nearly unlimited throughput.
    • Use case: Ideal for applications that can handle duplicate messages and don't require strict ordering.
    • Characteristics: High throughput, at-least-once delivery, best-effort ordering.

  2. FIFO Queue ("fifo"):

    • Description: Provides exactly-once processing and strict message ordering within message groups.
    • Use case: Required when message order is critical and duplicate processing must be avoided.
    • Characteristics: Exactly-once delivery, strict ordering, requires Message Group ID.
    • Note: Queue name must end with .fifo suffix.
    • Deduplication: If messages with identical content are sent within the deduplication window (5 minutes), only one message will be ingested.

Troubleshooting​

Common Issues and Solutions:

  1. Messages not appearing in queue:

    • Verify the Queue URL is correct and accessible
    • Check that the IAM role has proper permissions for the specified queue
    • Ensure the AWS region matches the queue's region

  2. Permission denied errors:

    • Confirm the IAM role ARN is correct and the role exists
    • Verify the trust relationship allows Monad to assume the role
    • Check that required SQS permissions are attached to the role

  3. FIFO queue issues:

    • Ensure Message Group ID is provided for FIFO queues
    • Verify the queue name ends with .fifo suffix
    • Note that identical message content within the deduplication window (5 minutes) will result in only one message being ingested due to SQS deduplication