Azure Service Bus
Synopsis
Creates a target that sends processed messages to Azure Service Bus queues or topics with support for multiple authentication methods, batch processing, and advanced message properties. Provides reliable message delivery to Azure Service Bus for decoupled application architectures and event-driven systems.
Schema
- name: <string>
description: <string>
type: azservicebus
pipelines: <pipeline[]>
status: <boolean>
properties:
client_connection_string: <string>
tenant_id: <string>
client_id: <string>
client_secret: <string>
namespace: <string>
queue: <string>
topic: <string>
session_id: <string>
message_id: <string>
correlation_id: <string>
content_type: <string>
time_to_live: <numeric>
max_events: <numeric>
timeout: <numeric>
field_format: <string>
debug:
status: <boolean>
dont_send_logs: <boolean>
Configuration
The following fields are used to define the target:
| Field | Required | Default | Description |
|---|---|---|---|
name | Y | Target name | |
description | N | - | Optional description |
type | Y | Must be azservicebus | |
pipelines | N | - | Optional post-processor pipelines |
status | N | true | Enable/disable the target |
Connection
Azure Service Bus target supports two authentication methods:
Method 1: Connection String Authentication
| Field | Required | Default | Description |
|---|---|---|---|
client_connection_string | Y* | Service Bus connection string (required if not using method 2) | |
queue | N** | Queue name to send messages to | |
topic | N** | Topic name to send messages to |
Method 2: Service Principal Authentication
| Field | Required | Default | Description |
|---|---|---|---|
tenant_id | Y* | Azure tenant ID (required if not using connection string) | |
client_id | Y* | Azure service principal client ID | |
client_secret | Y* | Azure service principal client secret | |
namespace | Y* | Service Bus namespace name only, without the .servicebus.windows.net suffix, which Director appends (required if not using connection string) | |
queue | N** | Queue name to send messages to | |
topic | N** | Topic name to send messages to |
* = Conditionally required (see authentication methods above)
** = Either queue or topic must be specified, but not both
Method 3: Managed Identity
Managed Identity Authentication
Azure targets support Managed Identity authentication for credential-free access when Director is deployed on Azure infrastructure.
How it works: When tenant_id, client_id, and client_secret are omitted from the configuration, the target automatically uses Azure's DefaultAzureCredential, which attempts authentication in the following order:
- Environment variables (
AZURE_TENANT_ID,AZURE_CLIENT_ID,AZURE_CLIENT_SECRET) - Workload Identity (for Kubernetes deployments)
- Managed Identity (system-assigned or user-assigned)
- Azure CLI credentials
- Azure PowerShell credentials
Deployment requirement: Director must run on Azure infrastructure that supports Managed Identity:
- Azure Virtual Machines
- Azure App Service
- Azure Container Instances
- Azure Kubernetes Service (AKS)
- Azure Functions
Required permissions: The Managed Identity must be granted the appropriate Azure RBAC roles documented in each target's IAM Permissions section.
Managed Identity eliminates credential management overhead and is the recommended authentication method for Azure-hosted Director deployments.
Message Properties
| Field | Required | Default | Description |
|---|---|---|---|
session_id | N | - | Session ID for session-enabled queues/topics |
message_id | N | - | Custom message ID |
correlation_id | N | - | Correlation ID for request-response patterns |
content_type | N | application/json | Message content type |
time_to_live | N | - | Message time-to-live in seconds |
Performance
| Field | Required | Default | Description |
|---|---|---|---|
max_events | N | 1000 | Maximum number of messages per batch |
field_format | N | - | Data normalization format. See applicable Normalization section |
timeout | N | 30 | Seconds bounding a single batch send |
Scheduling
See Scheduling and Pool Behavior for interval and cron fields shared by all targets.
Debug Options
| Field | Required | Default | Description |
|---|---|---|---|
debug.status | N | false | Enable debug logging |
debug.dont_send_logs | N | false | Process logs but don't send to target (testing) |
Details
The Azure Service Bus target sends processed messages to Azure Service Bus queues or topics for reliable message delivery in distributed systems. It supports automatic batching for optimal performance, advanced message properties for complex workflows, and multiple authentication methods for flexible deployment scenarios.
Messages are sent in batches to improve throughput and reduce network overhead. The target handles connection pooling and automatic reconnection on network failures.
IAM Permissions
When using Service Principal or Managed Identity authentication, the following Azure role is required:
| Azure Role | Scope | Purpose |
|---|---|---|
Azure Service Bus Data Sender | Service Bus Namespace, Queue, or Topic | Send message batches to queue or topic |
When using connection string authentication, Azure RBAC roles are not applicable -- the connection string must include Send permission.
Queues vs Topics
Queues
- Point-to-point messaging
- Single consumer per message
- FIFO ordering (optional)
- Ideal for task distribution and load leveling
Topics
- Publish-subscribe messaging
- Multiple subscribers per message
- Message filtering with subscriptions
- Ideal for event broadcasting and fan-out scenarios
Session Support
Session-enabled queues and topics provide FIFO guarantees for messages with the same session ID. Use the session_id property to enable session-based processing for ordered message delivery.
Message Properties
The target supports setting custom message properties:
- Message ID: Unique identifier for tracking and deduplication
- Correlation ID: Links related messages in request-response patterns
- Content Type: MIME type of message body
- Time to Live: Automatic message expiration
Batch Processing
Messages are accumulated in memory and sent in batches when the batch size limit is reached or during finalization. The maximum batch size is configurable via the max_events parameter.
Dead Letter Queue
Azure Service Bus automatically moves messages to the dead letter queue when they exceed max delivery count or expire. Configure these settings in the Azure portal or via infrastructure as code.
At-Least-Once Delivery
Service Bus guarantees at-least-once delivery. Messages may be delivered more than once in case of network failures or processing errors. Design your message handlers to be idempotent.
Examples
The following are commonly used configuration types.
Basic with Connection String
Creating a basic Service Bus queue target with connection string... | |
Target sends JSON messages to Service Bus queue in batches... | |
Service Principal Authentication
Using service principal authentication for secure access... | |
Managed Identity
Using Managed Identity for credential-free authentication on Azure infrastructure... | |
Session-Enabled Queue
Using sessions for ordered message processing... | |
Request-Response Pattern
Using correlation ID for request-response messaging... | |
Message Expiration
Setting time-to-live for automatic message expiration... | |
Topic with Subscriptions
Publishing messages to a topic for multiple subscribers... | |
Pipeline Processing
Applying post-processing pipelines before sending... | |
Field Normalization
Using field normalization for standard format... | |
High-Volume Configuration
Optimizing for high-volume message sending... | |
Troubleshooting
This section covers the errors you are most likely to see with the azservicebus target, what causes each one, and how to fix it.
Where to look:
- Director logs. Target errors are tagged with the target name and carry
"Section":"SenderPool". The part afterReason:or after the last colon is the actual cause. - The target's connection status in the web interface. For this target the status only reflects startup. Configuration and credential-resolution problems appear there as
connection failed for <target name>: <reason>. Anything that needs the network, which includes every authentication and permission problem, does not. The status stays atconnection ready for <target name>while the log fills with errors, so for this target read the log rather than the status.
See Target Delivery Errors for how Director logs and retries target failures.
Which permission is missing?
Match the error against this table first. Which right is missing depends on the authentication method.
| Error text | Missing role or right | Scope |
|---|---|---|
failed to create message batch: (unauthorized): rpc: failed, status code 401 ... | A current SharedAccessKey for the policy named in SharedAccessKeyName. The key is wrong, was rotated, or the signature has expired | The |
failed to create message batch: (unauthorized): *Error{Condition: amqp:unauthorized-access ...}, connection string mode | A shared access policy carrying the Send claim for the configured queue or topic | The namespace, or that same queue or topic. A string containing EntityPath= grants rights on that one entity only |
The same amqp:unauthorized-access error with tenant_id, client_id and client_secret set, or with Managed Identity | Azure Service Bus Data Sender | The Service Bus namespace, or the specific queue or topic |
ClientSecretCredential authentication failed. ... AADSTS7000215 (or AADSTS7000222, AADSTS700016) | A valid client secret value for an app registered in tenant_id. No Service Bus right is reached yet | The app registration in Microsoft Entra ID |
DefaultAzureCredential: failed to acquire a token. or ManagedIdentityCredential authentication failed. | A managed identity assigned to the resource Director runs on | The Azure virtual machine, container or cluster hosting Director |
No management rights (Manage or Listen) are needed. The queue or topic must already exist and be active. The target never creates it.
The status says connected but "ThreadSafeInit on reinit failed" repeats every few seconds
[Error] [director] [target-<target id>] [basic_servicebus_queue] Sender worker 1 ThreadSafeInit on reinit failed for "basic_servicebus_queue": failed to create message batch: <reason>
[Information] [director] [target-<target id>] [basic_servicebus_queue] Target "basic_servicebus_queue" reinitialized successfully (generation 7).
Cause: the target contacts Azure for the first time when it sends, not when it starts. Startup only builds the client from the configuration, so a wrong key, a missing role, a mistyped namespace and a missing queue all pass it, and the status shows connected. The first send is the first real test. It opens the connection, and every authentication, DNS and entity error surfaces there as failed to create message batch: followed by the cause. Each failure reinitializes the target, which logs reinitialized successfully with a higher generation number, and the next delivery fails the same way.
Fix: read the reason after failed to create message batch: and match it against the entries below. A climbing generation counter next to a connected status is always this pattern.
Data waits in the Director queue and is redelivered about every 5 seconds. It is kept for queue.limit seconds, 48 hours by default, and with the In-memory storage tier it does not survive a Director restart.
"failed to create message batch: (unauthorized)"
Sender worker 2 ThreadSafeInit on reinit failed for "basic_servicebus_queue": failed to create message batch: (unauthorized): rpc: failed, status code 401 and description: ...
Sender worker 1 ThreadSafeInit on reinit failed for "basic_servicebus_queue": failed to create message batch: (unauthorized): *Error{Condition: amqp:unauthorized-access, Description: ..., Info: map[]}
Cause: two forms, and the second half of the line tells them apart.
rpc: failed, status code 401means the token itself was refused. In connection string mode theSharedAccessKeyis wrong, was rotated, or the shared access signature has expired. The description typically mentions an invalid signature.amqp:unauthorized-accessmeans the token was accepted but the identity may not send to this entity. The description typically states that aSendclaim is required. In connection string mode the policy lacks Send, or the string is scoped throughEntityPath=to a different entity than the configuredqueueortopic. With a service principal or Managed Identity theAzure Service Bus Data Senderrole is missing.
Fix: for a connection string, open the namespace in the Azure Portal, then Endpoint=sb://. For a service principal or Managed Identity, open the namespace or the queue or topic, then Azure Service Bus Data Sender to the identity. Director retries on its own, so no restart is needed once the right is in place.
Data waits and is redelivered about every 5 seconds. The connection status stays green throughout.
"ClientSecretCredential authentication failed" with an AADSTS code
Sender worker 1 ThreadSafeInit on reinit failed for "basic_servicebus_queue": failed to create message batch: ClientSecretCredential authentication failed. POST https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/v2.0/token ... RESPONSE 401: 401 Unauthorized ... AADSTS7000215: Invalid client secret provided. ...
Cause: the service principal credentials are wrong, so the token request fails before any Service Bus permission is checked. The code names the field:
| Code | Meaning | Check |
|---|---|---|
AADSTS7000215 | Invalid client secret | The secret ID was pasted instead of the secret value, or the secret was rotated |
AADSTS7000222 | Client secret expired | Create a new secret on the app registration and update client_secret |
AADSTS700016 | Application not found in the directory | client_id is wrong, or tenant_id is a different tenant than the one the app registration lives in |
AADSTS90002 | Tenant not found | tenant_id is not a valid tenant ID |
If the reason is failed to create credentials: invalid tenantID instead, the target failed at startup and the status does show connection failed. tenant_id then holds a character other than letters, digits, dots and hyphens, typically a stray quote or brace. Use the tenant GUID.
Data waits and is redelivered about every 5 seconds until the credentials are corrected.
"DefaultAzureCredential: failed to acquire a token" or "ManagedIdentityCredential authentication failed"
Cause: client_connection_string is empty and tenant_id, client_id and client_secret are all empty, so Director authenticates as a Managed Identity. When Director does not run on an Azure resource that has one, the token request fails and the reason lists every credential the chain attempted, typically ending in a managed identity timeout. A reason containing the requested identity isn't assigned to this resource means a user-assigned identity is configured but not attached to the resource.
Filling in only one or two of the three service principal fields does not fall back to Managed Identity. That configuration is rejected at validation with tenant_id, client_id, and client_secret must all be provided together for service principal authentication, and nothing is sent until it is corrected. An empty ${VAR} behind one of the fields is a common trigger.
Fix: fill in all three service principal fields, switch to a connection string, or run Director on an Azure resource with a managed identity and assign Azure Service Bus Data Sender to that identity.
Data waits and is redelivered about every 5 seconds.
"failed to resolve connection string" or "failed to create service bus client from connection string"
Failed to reinitialize target "basic_servicebus_queue" (attempt 3). Reason: failed to resolve connection string: credential: env variable "SERVICEBUS_CONNECTION_STRING" is not set
Failed to reinitialize target "basic_servicebus_queue" (attempt 1). Reason: failed to create service bus client from connection string: key "SharedAccessKeyName" must not be empty
Cause: both fail at startup, so here the status does show connection failed. failed to resolve connection string means a ${VAR} or $secret{...} reference could not be read. The variable is not set in the environment of the Director service, or the secret store is misnamed, which reads credential: store "vault" not found in configuration. The same applies to failed to resolve tenant ID, failed to resolve client ID and failed to resolve client secret. failed to create service bus client from connection string means the string was read but is not complete: a missing prefix gives key "Endpoint" must not be empty, and a truncated paste gives failed parsing connection string due to unmatched key value separated by '='.
Fix: set the variable for the account the Director service runs under, or correct the secret store name, then paste the full string in the form Endpoint=sb://mynamespace.servicebus.windows.net/;SharedAccessKeyName=...;SharedAccessKey=.... If the variable exists but resolves to an empty value, the reason becomes namespace is required when not using connection string.
Startup is retried with a growing backoff and the attempt counter climbs until the cause is fixed. Data waits in the queue.
"no such host" with the namespace suffix repeated twice
Cause: namespace was given as the full host name. Director appends .servicebus.windows.net itself, so the lookup goes to a name carrying the suffix twice. The reason starts with (connlost): and typically ends in lookup mynamespace.servicebus.windows.net.servicebus.windows.net: no such host. The same error with a single suffix is a typo in namespace, or DNS on the Director host.
Fix: set namespace to the short name, mynamespace, not mynamespace.servicebus.windows.net. A namespace in a sovereign cloud, on a .chinacloudapi.cn or .usgovcloudapi.net host, cannot be addressed through namespace at all. Use client_connection_string for it, since the host is then taken from the string's Endpoint= value.
Data waits and is redelivered about every 5 seconds.
"amqp:not-found" or "com.microsoft:entity-disabled" while creating the message batch
Sender worker 1 ThreadSafeInit on reinit failed for "basic_servicebus_queue": failed to create message batch: *Error{Condition: amqp:not-found, Description: ..., Info: map[]}
Cause: the namespace was reached and the credentials were accepted, but the queue or topic in the configuration does not exist there. The description typically names the entity that could not be found. The usual reasons are a misspelled name, a deleted entity, an entity that lives in another namespace, and a topic name placed in queue or the reverse. com.microsoft:entity-disabled means the entity exists but is not active.
Fix: compare queue or topic against the entity list of the namespace in the Azure Portal, and correct the name or create the entity. For a disabled entity set its
Data waits and is redelivered about every 5 seconds.
"the message is too large"
Sender worker 3 execute() failed for basic_servicebus_queue: record rejected by target: the message is too large
Sender worker 3 deterministic failure for basic_servicebus_queue after 4 attempts — dropping (giving up): record rejected by target: the message is too large
Cause: a single record exceeds the largest message the namespace accepts. See your tier's message size limit in the Azure documentation. Director fills a batch until it holds max_events messages or the link's size limit is reached. A record that does not fit into a batch already holding messages is normal: that batch is sent and the record opens the next one, with no error. Only a record that does not fit into an empty batch is rejected. A few bytes of framing per message count toward the limit.
Fix: shrink the record in a pipeline before it reaches the target, or move the namespace to a tier with a larger message size. Lowering max_events does not help, because the limit applies to one record. Note that max_events: 0 is read as unset and becomes 1000.
The rejected record is dropped after 4 deliveries, as the second line shows. The other records in the payload continue to be delivered.
"failed to send message batch: context deadline exceeded" or "com.microsoft:server-busy"
Sender worker 4 execute() failed for basic_servicebus_queue: target broken: failed to finalize target cache: failed to send message batch: context deadline exceeded
Cause: the batch was built but the namespace did not acknowledge it within timeout seconds, 30 by default. The usual reasons are a congested or half-open connection, or a Standard tier namespace that is throttling. Throttling also surfaces directly as *Error{Condition: com.microsoft:server-busy, Description: ..., Info: map[]}, whose description typically says the entity is being throttled. amqp:resource-limit-exceeded in the same position means the queue or topic is full because nothing is consuming it.
Fix: raise timeout when sends are slow but succeed. For throttling, check the namespace's throttled requests metric, lower the send rate, spread the load across entities, or move to Premium. For a full entity, drain or purge it, or raise its maximum size.
The batch is redelivered about every 5 seconds until it is accepted. Nothing is dropped.
"amqp:not-allowed" when sending to a session-enabled entity
Cause: the queue or subscription requires sessions but session_id is not configured, so the service refuses the batch. The description typically states that the SessionId was not set. The reverse case is harmless: a session_id sent to an entity without sessions is ignored, though the ordering it implies is then not guaranteed.
Fix: set session_id to a fixed string, as in the Session-Enabled Queue example, or turn sessions off on the entity.
The verdict is deterministic, but the batch is not dropped. It is redelivered about every 5 seconds and the target stalls until the configuration or the entity changes.
"i/o timeout" or "tls: failed to verify certificate" while connecting
Cause: Director cannot open the AMQP connection. This target speaks AMQP over TLS on TCP port 5671 only. There is no WebSocket or port 443 mode, and the connection is dialled directly: HTTPS_PROXY applies to the Microsoft Entra token request, never to the namespace connection, so egress that is only possible through a proxy cannot work. A (connlost): reason that typically ends in i/o timeout means port 5671 is blocked, or the namespace firewall excludes the Director host. A reason that typically contains x509: certificate signed by unknown authority means a TLS-intercepting proxy, or missing root certificates on the host.
Fix: allow outbound TCP 5671 from the Director host to the namespace host, and add its egress IP address or virtual network to the namespace's network rules. The host is <namespace>.servicebus.windows.net when namespace is set, and the host of Endpoint=sb://... when a connection string is used, so a sovereign cloud or a custom domain takes the host from the connection string. For TLS interception, exempt that same host or install the intercepting CA in the operating system trust store. This target has no custom CA option.
An isolated (connlost): failure during a send, such as a forced link detach after a quiet period, is redelivered on the next attempt and usually heals by itself. Otherwise data waits and is redelivered about every 5 seconds.
"ValidateConfig failed" for this target
| Reason | Fix |
|---|---|
either queue or topic must be specified | Set queue or topic |
cannot specify both queue and topic | Remove one of them |
tenant_id, client_id, and client_secret must all be provided together for service principal authentication | Fill in all three, or clear all three to use Managed Identity |
namespace is required when not using connection string authentication for azservicebus target | Set namespace when no connection string is configured |
max_events must be greater than 0, got -5 | Use a positive integer |
Nothing is sent while the configuration is rejected. The check repeats about every 30 seconds until the configuration changes.
The target is healthy but nothing arrives in the queue or topic
Check these in order.
-
The connection was never tested. A connected status does not mean a send succeeded. Search the log for
ThreadSafeInit on reinit failedand follow the first entry above. -
debug.dont_send_logsis enabled. Events are processed but never buffered or transmitted, and the sent counter does not move. The only notice is a debug-level line at startup, and only whendebug.statusis enabled as well. Remove the flag after testing. -
A fixed
message_idmeets duplicate detection.message_idgives every message the same ID. On an entity with duplicate detection enabled, Service Bus keeps the first message of each detection window and discards the rest, while Director counts every record as sent. Leavemessage_idempty, and a unique ID is generated per message. -
time_to_liveis too short. Messages are accepted, then expire before a consumer reads them, and are dead-lettered or discarded according to the entity settings. A value above the entity's own default is adjusted down without notice. Raise or removetime_to_live, and check the dead-letter queue. -
The topic has no subscription. Messages published to a topic without subscriptions are accepted and discarded by the service. Create the subscription before pointing the target at the topic.