Amazon MQ

Amazon MQ Message Queue

Synopsis

Creates a target that publishes to an Amazon MQ broker, of either engine: RabbitMQ or ActiveMQ. It is the RabbitMQ target with Amazon MQ's rules on top: every key of that target keeps its meaning, and the keys on this page are added or constrained.

What it adds to the RabbitMQ target:

• engine is required, and decides the protocol: activemq speaks AMQP 1.0 alone.
• TLS is always on: url is amqps://, on port 5671 unless it names another.
• url names one broker, or the two brokers of an active/standby pair.
• On the RabbitMQ engine, the target can sign in with an IAM token or an OAuth 2.0 token, renewed before it expires, or with a client certificate.

The device that consumes from Amazon MQ is Amazon MQ.

Schema

- name: <string>
  description: <string>
  type: amazonmq
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    engine: <string>
    url: <string>
    protocol: <string>
    exchange: <string>
    routing_key: <string>
    queue: <string>
    bind_queue: <boolean>
    binding_key: <string>
    authentication: <string>
    username: <string>
    password: <string>
    region: <string>
    iam_audience: <string>
    key: <string>
    secret: <string>
    session: <string>
    role_arn: <string>
    external_id: <string>
    token_url: <string>
    client_id: <string>
    client_secret: <string>
    scope: <string>
    audience: <string>
    tls:
      verify: <boolean>
      server_name: <string>
      ca_name: <string>
      cert_name: <string>
      key_name: <string>
      passphrase: <string>
      min_tls_version: <string>
      max_tls_version: <string>

Every other key of the RabbitMQ target (the message properties, batching, timeout, the retry settings with disable_retries, field_format, scheduling and debug.*) applies with the same meaning.

Configuration

The following fields are used to define the target:

Field	Required	Default	Description
name	Y		Target name
description	N	-	Optional description
type	Y		Must be amazonmq
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Connection

Field	Required	Default	Description
engine	Y	-	The engine of the Amazon MQ broker: rabbitmq or activemq
url	Y	-	One amqps:// URL, or two separated by a comma (active/standby), tried in turn. Port 5671 unless the URL names another. The path names the virtual host. Any other scheme is refused
protocol	N	0-9-1 on rabbitmq, 1.0 on activemq	On rabbitmq: 0-9-1 or 1.0. On activemq: always 1.0, and 0-9-1 is a configuration error

exchange, routing_key and queue route as on the RabbitMQ target: either exchange or queue must be set. With both set, the queue is declared and bound to the exchange only when bind_queue is true. bind_queue and binding_key work as described under Queue binding, on the rabbitmq engine under 0-9-1 only. Under 1.0, which the activemq engine always speaks, both are refused.

Authentication

Field	Required	Default	Description
authentication	N	plain	plain, or on the rabbitmq engine only: iam, oauth2 or external. iam, oauth2 and external on activemq are refused, and the error says why
username	Y for plain	-	The user. Left out, the user in url is used. With neither, the target is refused
password	N	-	The user's password, or the one in url

authentication	Signs in with
plain	username and password, or the user in url
iam	A token of IAM's, through STS's GetWebIdentityToken (IAM outbound federation). See IAM
oauth2	A token of an OAuth 2.0 identity provider, through the client credentials grant. See OAuth 2.0
external	The TLS client certificate, which is then required: set tls.cert_name and tls.key_name. The broker must be set up for it. See Prerequisites

Under iam and oauth2, the token is the password and the username is empty (SASL PLAIN).

IAM

Used under authentication: iam.

Field	Required	Default	Description
region	Y	-	The AWS region whose STS issues the token
iam_audience	Y	-	The audience of the token, as the broker's IAM settings name it
key	N	-	AWS access key ID. Without key, secret and session, the default chain is used: the instance profile, IRSA, or the environment
secret	N	-	AWS secret access key
session	N	-	AWS session token, for temporary credentials
role_arn	N	-	A role assumed through sts:AssumeRole first. The token is then asked for as that role
external_id	N	-	The external ID sent with the assumed role. Needs role_arn

OAuth 2.0

Used under authentication: oauth2.

Field	Required	Default	Description
token_url	Y	-	The identity provider's token endpoint. https:// only
client_id	Y	-	The client of the client credentials grant
client_secret	Y	-	The client's secret. Takes a $secret{…} reference
scope	N	-	Scopes, separated by spaces
audience	N	-	Sent as the audience form parameter

TLS Configuration

TLS is always on: tls.status: false is a configuration error, and a tls.status left out is on.

TLS material is resolved through the shared client builder, so these keys mean the same thing on every target that uses it. They are nested under a tls: block.

Field	Required	Default	Description
tls.verify	N	true	Verify the server certificate. On by default — set it to false only to accept an untrusted certificate, and only where you control the network path
tls.server_name	N	-	SNI hostname override for the handshake. Use it when the certificate's name does not match the address you connect to
tls.ca_name	N	-	CA bundle used to verify the server certificate. When unset the host trust store is used; when set it replaces the host trust store rather than adding to it
tls.cert_name	N*	-	Client certificate, for mutual TLS
tls.key_name	N*	-	Client private key, for mutual TLS
tls.passphrase	N	-	Passphrase for an encrypted client private key
tls.min_tls_version	N	tls1.2	Lowest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)
tls.max_tls_version	N	tls1.3	Highest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)

* cert_name and key_name are individually optional but must be supplied together — a certificate without its key, or a key without its certificate, is a configuration error.

Details

Token Renewal

Under iam and oauth2, the token is renewed at the next send once 80% of its lifetime has passed. The lifetime is read from the token answer's expiry, or from the JWT's exp claim.

Protocol	Renewal
AMQP 0-9-1	update-secret on the shared connection
AMQP 1.0	PUT /auth/tokens on RabbitMQ 4's management node, on each producer

If the broker refuses the in-place update, the target is opened again with a fresh token. If no token can be obtained, that is an authentication error. Logs and errors never carry the token, the password, the client secret or the user in url.

Prerequisites

1. An Amazon MQ broker, of the RabbitMQ or the ActiveMQ engine
2. Network connectivity to the broker on its AMQPS port
3. For iam: Amazon MQ for RabbitMQ 3.13, or 4.2 and later, with OAuth 2.0 / IAM configured on the broker, taking the token's audience (iam_audience), and IAM outbound federation enabled on the AWS account. Neither is configured by the target. When the broker refuses an iam or oauth2 login, the error names these requirements
4. For oauth2: the broker's OAuth 2.0 settings taking the identity provider's tokens, their audience and their scopes
5. For external: the target signs in with SASL EXTERNAL. It presents the client certificate and sends no username, and the broker reads the user from the certificate. Setting tls.cert_name and tls.key_name is not enough on its own. The broker must also be set up for certificate sign-in, and none of this is configured by the target:
   • Amazon MQ for RabbitMQ 4 or later. Amazon MQ offers SSL certificate authentication on version 4 and later only, so a RabbitMQ 3.13 broker refuses external
   • The broker's configuration offers the EXTERNAL mechanism (auth_mechanisms.1 = EXTERNAL) and names the certificate field the username is read from (ssl_cert_login_from, for example common_name)
   • The broker trusts the certificate authority that signed the client certificate: aws.arns.ssl_options.cacertfile names the CA certificate's ARN, and aws.arns.assume_role_arn the role the broker reads it with. That role is the broker's, not the target's
   • A broker user with the name read from the certificate, with permissions on the virtual host the url names

IAM Permissions

Only authentication: iam calls AWS. plain, oauth2 and external need no IAM permission.

The identity the token is asked for as needs sts:GetWebIdentityToken. That identity is the static key, the default chain's, or role_arn when it is set:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AmazonMQWebIdentityToken",
      "Effect": "Allow",
      "Action": "sts:GetWebIdentityToken",
      "Resource": "*"
    }
  ]
}

With role_arn, the calling identity also needs:

{
  "Sid": "AmazonMQAssumeRole",
  "Effect": "Allow",
  "Action": "sts:AssumeRole",
  "Resource": "arn:aws:iam::ACCOUNT_ID:role/ROLE_NAME"
}

Examples

ActiveMQ Active/Standby

Publishing to an Amazon MQ for ActiveMQ active/standby broker over AMQP 1.0, with a username and a password:

targets:
  - name: orders_activemq
    type: amazonmq
    properties:
      engine: activemq
      url: "amqps://b-1234-1.mq.eu-west-1.amazonaws.com:5671,amqps://b-1234-2.mq.eu-west-1.amazonaws.com:5671"
      queue: "orders"
      username: "producer"
      password: "${AMAZONMQ_PASSWORD}"

RabbitMQ with IAM

Publishing to an Amazon MQ for RabbitMQ broker with an IAM token, asked for with static AWS credentials:

targets:
  - name: orders_rabbitmq_iam
    type: amazonmq
    properties:
      engine: rabbitmq
      url: "amqps://b-5678.mq.eu-west-1.amazonaws.com"
      queue: "orders"
      authentication: iam
      region: "eu-west-1"
      iam_audience: "rabbitmq"
      key: "${AWS_ACCESS_KEY_ID}"
      secret: "${AWS_SECRET_ACCESS_KEY}"