amazon-sqs - SQS Event Bridge Once Per Minute

Question

We are looking at Event Bridge to give us a scheduled task added to our SQS once per minute.

We are looking at Event Bridge to make it happen. So far it properly puts messages into the queue, but we are trying to schedule it for once per minute and noticing that the queue only gets messages once per five minutes sometimes six minutes.

The metrics seem to state invocation is happening; however, the queue isn't receiving them in the time frame specified.

Considerations

• SQS FIFO Queue - Deduplication
• Constant JSON String

The "duh" of not seeing messages at prescribed interval is because of this in the AWS documentation:

The token used for deduplication of sent messages. If a message with a particular message deduplication ID is sent successfully, any messages sent with the same message deduplication ID are accepted successfully but aren’t delivered during the 5-minute deduplication interval

Open to suggestions and will be looking for a workaround.

Update

I tried using Input Transformer to fix by adding the time as uniquely changing item in the queue message; however, still not getting below 5 minutes.

Variable Input

{"addedOn":"$.time"}

Message

{"AddedOn":<addedOn>}

Answer 1

The queue polling built into SQS just wasn't polling for my updated count of greater than 10 messages. Once I deleted out the old messages the timing was correct and it was updating 1/min.

The answer is if you are going to use a constant string it'll have to be for scheduled jobs that are greater than 5 minutes.

Adding info here despite redundancy from question for linked Google searches:

The token used for deduplication of sent messages. If a message with a particular message deduplication ID is sent successfully, any messages sent with the same message deduplication ID are accepted successfully but aren’t delivered during the 5-minute deduplication interval

Despite the messages being a unique event once per minute the Constant (JSON text) not being unique still saw it as a duplicate to remove.

To solve I switched to Input transformer

Example event and what you other fields you can add as variables:

{
 "version": "0",
 "id": "7bf73129-1428-4cd3-a780-95db273d1602",
 "detail-type": "EC2 Instance State-change Notification",
 "source": "aws.ec2",
 "account": "123456789012",
 "time": "2015-11-11T21:29:54Z",
 "region": "us-east-1",
 "resources": [
 "arn:aws:ec2:us-east-1:123456789012:instance/i-abcd1111"
 ],
 "detail": {
 "instance-id": "i-0123456789",
 "state": "RUNNING"
 }
}

I needed a unique variable so time was an obvious choice.

Input for Input Transformer

Input Path:

{"addedOn":"$.time"}

Template:

{"AddedOn":<addedOn>}

Documentation

Also, found that moving over to not using FIFO queues is a potential solution if that's an easy option for future SQS developers as well.