Transfer · To Tasks · to_aws_sqs

Purpose

Uploads data to the Amazon Simple Queue Service (SQS).

Limitation

Only supports database as a source in the extract part of the configuration.

Method of use

Before you can upload data to Amazon SQS you have to give Transfer access. Follow the steps below:

  1. Have the Amazon Cloud administrator create an IAM account (if you don't have one already)
  2. Hand him the message queue you want to access
  3. Make sure to activate `Programmatic access`
  4. Make sure the following policies are added:
    1. `AmazonSQSFullAccess` - Minimum required access.
  5. Send Access Key ID and Secret Access Key back to Onesecondbefore staff. They will add it to Transfer.
  6. You should now be able to download data cloud objects from Amazon S3

Configuration

Example usage

load:
    conn_id: aws_s3
    mapping:
        QueueUrl: QueueUrl
        MessageBody: MessageBody
        DelaySeconds: DelaySeconds
        MessageAttributes: MessageSystemAttributes
        MessageDeduplicationId: MessageDeduplicationId
        MessageGroupId: MessageGroupId

Properties

propertytypemandatorydescription
conn_idstringnoConnection string as handed to you by the Onesecondbefore team. Default is `aws_s3`
mappingobjectyesContains the mappings from the dataset to the service. See below for mapping options.

Mappings

mappingtype (in dataset)mandatorydescription
QueueUrlstringyesContains the field name in the dataset that contains the `QueueUrl` of the message queue.
MessageBodyobjectyesContains the message body.
DelaySecondsintegernoContains the message body.
MessageAttributesobjectnoContains the message attributes.
MessageDeduplicationIdstringnoContains the message deduplication ID.
MessageGroupIdstringnoContains the message group ID.

Details

itemdescription
APIAmazon SQS REST API
To typeData upload