Azure Event Hubs

Microsoft Azure Real-time Messaging

Synopsis

Creates a target that sends processed messages to Azure Event Hubs with support for multiple authentication methods, batch processing, and automatic retry mechanisms. Provides high-throughput event streaming to Azure Event Hubs for real-time analytics and downstream processing.

Schema

- name: <string>
  description: <string>
  type: eventhubs
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    client_connection_string: <string>
    tenant_id: <string>
    client_id: <string>
    client_secret: <string>
    namespace: <string>
    event_hub: <string>
    partition:
      id: <string>
      key: <string>
    field_format: <string>
    max_bytes: <numeric>
    max_events: <numeric>
    timeout: <numeric>
    tls:
      status: <boolean>
      cert_name: <string>
      key_name: <string>
      insecure_skip_verify: <boolean>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Field	Required	Default	Description
name	Y		Target name
description	N	-	Optional description
type	Y		Must be eventhubs
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Connection

EventHubs target supports two authentication methods:

Method 1: Connection String Authentication

Field	Required	Default	Description
client_connection_string	Y*		Event Hubs connection string (required if not using method 2)
event_hub	Y		Event hub name to send messages to

Method 2: Service Principal Authentication

Field	Required	Default	Description
tenant_id	Y*		Azure tenant ID (required if not using connection string)
client_id	Y*		Azure service principal client ID
client_secret	Y*		Azure service principal client secret
namespace	Y*		Event Hubs namespace as a fully qualified host, e.g. mynamespace.servicebus.windows.net (required if not using connection string)
event_hub	Y		Event hub name to send messages to

* = Conditionally required (see authentication methods above)

Method 3: Managed Identity

Managed Identity Authentication

Azure targets support Managed Identity authentication for credential-free access when Director is deployed on Azure infrastructure.

How it works: When tenant_id, client_id, and client_secret are omitted from the configuration, the target automatically uses Azure's DefaultAzureCredential, which attempts authentication in the following order:

1. Environment variables (AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET)
2. Workload Identity (for Kubernetes deployments)
3. Managed Identity (system-assigned or user-assigned)
4. Azure CLI credentials
5. Azure PowerShell credentials

Deployment requirement: Director must run on Azure infrastructure that supports Managed Identity:

• Azure Virtual Machines
• Azure App Service
• Azure Container Instances
• Azure Kubernetes Service (AKS)
• Azure Functions

Required permissions: The Managed Identity must be granted the appropriate Azure RBAC roles documented in each target's IAM Permissions section.

note

Managed Identity eliminates credential management overhead and is the recommended authentication method for Azure-hosted Director deployments.

Partition Configuration

Field	Required	Default	Description
partition.id	N*	-	Specific partition ID to send messages to
partition.key	N*	-	Partition key for message routing

* = Mutually exclusive - use either partition.id OR partition.key, not both

Message Configuration

Field	Required	Default	Description
field_format	N	-	Data normalization format. See applicable Normalization section

Performance

Field	Required	Default	Description
max_bytes	N	0	Maximum batch size in bytes. 0 uses the maximum message size the Event Hubs service allows for the connection, so it does not mean unlimited
max_events	N	1000	Maximum number of events per batch
timeout	N	30	Connection timeout in seconds

TLS

Field	Required	Default	Description
tls.status	N	false	Enable TLS encryption
tls.cert_name	N*	-	TLS certificate
tls.key_name	N*	-	TLS private key
tls.insecure_skip_verify	N	false	Skip TLS certificate verification (NOT recommended for production)

* = Conditionally required (only when tls.status: true)

note

TLS material fields (cert_name, key_name, ca_name, client_ca_name) accept any of the following:

• File name — resolved relative to the service root directory. Nested paths such as certs/prod/server.pem are supported.
• Absolute path — honored only if it resolves inside the service root. Any path that escapes the root is refused.
• Inline PEM content — used verbatim when the value contains -----BEGIN.
• Environment variable — ${ENV_VAR}.
• Vault reference — $secret{id=...} or $secret{store=...,ref=...}.

warning

Setting insecure_skip_verify: true disables certificate validation and should only be used for testing/development environments.

Scheduling

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging
debug.dont_send_logs	N	false	Process logs but don't send to target (testing)

Details

The EventHubs target sends processed messages to Azure Event Hubs for real-time event streaming and analytics. It supports automatic batching for optimal performance and multiple authentication methods for flexible deployment scenarios.

Messages are sent with automatic partition distribution unless a specific partition ID or key is provided. The target handles connection pooling and automatic reconnection on network failures.

IAM Permissions

When using Service Principal or Managed Identity authentication, the following Azure role is required:

Azure Role	Scope	Purpose
Azure Event Hubs Data Sender	Event Hubs Namespace or Event Hub	Send event data batches to Event Hubs

note

When using connection string authentication, Azure RBAC roles are not applicable -- the connection string must include Send permission.

Partition Management

You can control message routing to Event Hubs partitions using two mutually exclusive options:

• partition.id: Routes all messages to a specific partition by ID (0-based index)
• partition.key: Uses a partition key for consistent hashing across partitions

If neither is specified, Event Hubs automatically distributes messages across available partitions using round-robin distribution.

Examples

The following are commonly used configuration types.

Basic with Connection String

Creating a basic EventHubs target with connection string...	- name: basic_eventhubs_target type: eventhubs properties: client_connection_string: "Endpoint=sb://mynamespace.servicebus.windows.net/;SharedAccessKeyName=mykey;SharedAccessKey=myvalue" event_hub: "processed-logs" max_events: 100
Target sends messages to Event Hubs in batches...	{ "timestamp": "2024-01-15T10:30:00Z", "host": "server01", "message": "User authentication successful", "severity": "info" }

Service Principal Authentication

Using service principal authentication for secure access...

- name: sp_eventhubs_target type: eventhubs properties: tenant_id: "12345678-1234-1234-1234-123456789012" client_id: "87654321-4321-4321-4321-210987654321" client_secret: "${AZURE_CLIENT_SECRET}" namespace: "production-namespace.servicebus.windows.net" event_hub: "security-events" max_events: 250

Managed Identity

Using Managed Identity for credential-free authentication on Azure infrastructure...

- name: managed_identity_eventhubs type: eventhubs properties: namespace: "production-namespace.servicebus.windows.net" event_hub: "security-events" max_events: 250

High-Throughput Configuration

Optimizing for high-volume message sending...

- name: high_volume_target type: eventhubs properties: client_connection_string: "${EVENTHUBS_CONNECTION_STRING}" event_hub: "high-volume-events" max_events: 500 max_bytes: 1048576

Partition Key Configuration

Using partition key for consistent message routing...

- name: partitioned_key_target type: eventhubs properties: tenant_id: "${AZURE_TENANT_ID}" client_id: "${AZURE_CLIENT_ID}" client_secret: "${AZURE_CLIENT_SECRET}" namespace: "analytics-namespace.servicebus.windows.net" event_hub: "partitioned-logs" partition: key: "source_system" max_events: 200

Partition ID Configuration

Sending all messages to a specific partition...

- name: partitioned_id_target type: eventhubs properties: client_connection_string: "${EVENTHUBS_CONNECTION_STRING}" event_hub: "specific-partition" partition: id: "0" max_events: 150

Secure Connection with TLS

Implementing TLS encryption for secure transmission...

- name: secure_eventhubs_target type: eventhubs properties: client_connection_string: "${EVENTHUBS_CONNECTION_STRING}" event_hub: "secure-events" max_events: 150 tls: status: true cert_name: "eventhubs.crt" key_name: "eventhubs.key"

Pipeline Processing

Applying post-processing pipelines before sending...

- name: pipeline_eventhubs_target type: eventhubs pipelines: - format_timestamp - add_metadata - validate_schema properties: client_connection_string: "${EVENTHUBS_CONNECTION_STRING}" event_hub: "processed-events" max_events: 100

Troubleshooting

This section covers the errors you are most likely to see with the eventhubs target, what causes each one, and how to fix it. See Target Delivery Errors for how Director logs and retries target failures.

Where to look:

• Director logs. Target errors are tagged with the target name and carry "Section":"SenderPool". The part after Reason: or after the last colon is the actual cause.
• The target's connection status in the web interface. It shows the same reason as the log line, prefixed with connection failed for <target name>:.

Which permission is missing?

Match the error you see against this table first. The right that is missing depends on the authentication method.

Error text	Missing role or right	Scope
(unauthorized): rpc: failed, status code 401 and description: ... at startup, or (unauthorized): *Error{Condition: amqp:unauthorized-access, ...} on flush, with a service principal or Managed Identity	Azure Event Hubs Data Sender	The Event Hubs namespace, or the single event hub the target sends to
The same (unauthorized) text with client_connection_string	A shared access policy holding the Send claim, and its current key	The namespace Shared access policies page. A policy created on the event hub itself yields a connection string with EntityPath, which this target does not accept
ClientSecretCredential authentication failed. ... AADSTS7000215, AADSTS7000222, or AADSTS700016	A valid client secret value, or a client_id and tenant_id that belong to the same app registration. Not a role problem	The app registration in Microsoft Entra ID
DefaultAzureCredential: failed to acquire a token.	A managed identity on the host Director runs on, holding Azure Event Hubs Data Sender	The Azure VM, container, or AKS workload identity

Assign the role under Access control (IAM) > Add role assignment on the namespace or the event hub. Startup also reads the event hub's properties. If startup keeps failing with (unauthorized) after the role is in place and has had time to propagate, test with Azure Event Hubs Data Owner on the event hub, or add the Listen claim to the shared access policy, to rule out a send-only right being refused for that read.

note

Azure RBAC changes can take several minutes to propagate. Director retries automatically, so no restart is needed once the role is assigned.

"connection string contains an EntityPath. eventHub must be an empty string"

[Error] [director] [target-<target id>] [basic_eventhubs_target] Failed to reinitialize target "basic_eventhubs_target" (attempt 7). Reason: connection string contains an EntityPath. eventHub must be an empty string

Cause: client_connection_string was copied from the shared access policies of the event hub rather than of the namespace, so it ends in ;EntityPath=myeventhub. The target always names the hub through event_hub, and a connection string that names it as well is refused. The two cannot be combined.

Fix: open the Event Hubs namespace in the Azure Portal, then Shared access policies, and copy the connection string of a policy that has the Send claim. It has the form Endpoint=sb://mynamespace.servicebus.windows.net/;SharedAccessKeyName=mykey;SharedAccessKey=... with no EntityPath. Keep event_hub set. You can also delete the ;EntityPath=... segment from the string you already have. Retried until fixed, and nothing is sent until the string is changed.

A related reason means the string is incomplete rather than hub-scoped: key "Endpoint" must not be empty, key "SharedAccessKeyName" must not be empty, key "SharedAccessKey" or "SharedAccessSignature" cannot both be empty, failed parsing connection string due to an incorrectly formatted Endpoint value, or failed parsing connection string due to unmatched key value separated by '='. That points to a truncated paste, a ${VAR} that resolved to an empty value, or quotes swallowed by YAML. Re-copy the whole string and quote it in YAML.

"dial tcp: lookup mynamespace: no such host"

[Error] [director] [target-<target id>] [managed_identity_eventhubs] Failed to reinitialize target "managed_identity_eventhubs" (attempt 3). Reason: dial tcp: lookup mynamespace: no such host

Cause: namespace is used exactly as written. Nothing appends .servicebus.windows.net, so a bare namespace name is not a resolvable host. The same error appears for a typo in the host, a DNS outage, or a private endpoint whose name does not resolve from the Director host.

Fix: set namespace to the fully qualified host, for example mynamespace.servicebus.windows.net, or to your private-link or custom domain. With client_connection_string the host comes from the Endpoint=sb://... part and namespace is ignored. Verify resolution from the Director host. Retried until fixed.

"(unauthorized)" with status code 401 or amqp:unauthorized-access

[Error] [director] [target-<target id>] [sp_eventhubs_target] Failed to reinitialize target "sp_eventhubs_target" (attempt 2). Reason: (unauthorized): rpc: failed, status code 401 and description: <service description>
[Error] [director] [target-<target id>] [sp_eventhubs_target] Sender worker 1 Finalize failed on flush for target "sp_eventhubs_target": (unauthorized): *Error{Condition: amqp:unauthorized-access, Description: <service description>, Info: map[]}

Cause: the identity authenticated, but Event Hubs refused the operation. The service description typically states that the Send claim is required, or reports an invalid signature. With a service principal or Managed Identity, the identity has no Azure Event Hubs Data Sender role on the namespace or hub, or the assignment has not propagated yet. With a connection string, the key was regenerated, the shared access signature expired, or the policy lacks the Send claim. The flush-time form appears when a key or role is removed while the target runs, or when the client secret expired so the token could not be refreshed.

Fix: assign the role, or re-copy the current key, as described in the table above. Retried until fixed, and data stays queued meanwhile.

"ClientSecretCredential authentication failed" with an AADSTS code

[Error] [director] [target-<target id>] [sp_eventhubs_target] Failed to reinitialize target "sp_eventhubs_target" (attempt 4). Reason: ClientSecretCredential authentication failed. POST https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/v2.0/token
RESPONSE 401: 401 Unauthorized
{ "error": "invalid_client", "error_description": "AADSTS7000215: Invalid client secret provided. ..." }

Cause: the service principal credentials are wrong, so the token request fails before Event Hubs is contacted. The AADSTS code tells you which field:

Code	Meaning	Check
AADSTS7000215	Invalid client secret	The secret ID was pasted instead of the secret value, or the secret was rotated
AADSTS7000222	Client secret expired	Create a new secret on the app registration and update client_secret
AADSTS700016	Application not found in the directory	client_id is wrong, or tenant_id is a different tenant than the one the app registration lives in
AADSTS90002	Tenant not found	tenant_id is not a valid tenant ID

If the reason is invalid tenantID. You can locate your tenantID by following the instructions listed here: ..., then tenant_id contains characters that are not allowed, for example a URL, spaces, or braces left over from a failed substitution. Use the tenant GUID or the mytenant.onmicrosoft.com form. The reason spans several lines, so a log shipper that splits on newlines shows only the first one. Retried until fixed.

"DefaultAzureCredential: failed to acquire a token"

[Error] [director] [target-<target id>] [managed_identity_eventhubs] Failed to reinitialize target "managed_identity_eventhubs" (attempt 5). Reason: DefaultAzureCredential: failed to acquire a token.
Attempted credentials:
	EnvironmentCredential: missing environment variable AZURE_TENANT_ID
	ManagedIdentityCredential: <reason>

Cause: the target is using Managed Identity. That is the intended mode when tenant_id, client_id, and client_secret are all absent. It is also the fallback when one of the three is present but resolves to an empty value, because Director then treats the service principal as not configured. The ManagedIdentityCredential line carries the reason: no identity is enabled on the host, the identity is not assigned to this resource, or the instance metadata endpoint at 169.254.169.254 did not answer.

Fix: enable a system-assigned or user-assigned identity on the VM or container Director runs on, and assign it Azure Event Hubs Data Sender. Managed Identity token requests bypass the configured HTTP proxy, which is expected. If you meant to use a service principal, fill all three fields. Setting only one or two is rejected at configuration time. Retried until fixed.

"failed to resolve client secret" and other "failed to resolve ..." reasons

[Error] [director] [target-<target id>] [sp_eventhubs_target] Failed to reinitialize target "sp_eventhubs_target" (attempt 2). Reason: failed to resolve client secret: credential: env variable "AZURE_CLIENT_SECRET" is not set

Cause: a credential field uses a ${VAR}, $env{...}, $path{...}, or $secret{...} reference that could not be resolved. The text after credential: says why. env variable "..." is not set means the variable is not visible to the Director service. store "..." not found in configuration means the secret store is not defined in the system configuration. path reference "..." names no path means the $path{} value is empty. The same wrappers appear as failed to resolve connection string, failed to resolve tenant ID, failed to resolve client ID, failed to resolve TLS certificate, and failed to resolve TLS private key. The TLS fields are resolved even when tls.status is false, so an unset variable in tls.cert_name blocks startup although client certificates are off.

Fix: export the variable for the Director service, fix the secret store, or remove an unused tls.cert_name and tls.key_name. Retried until fixed.

"rpc: failed, status code 404" or amqp:not-found

Cause: the event hub named in event_hub does not exist in that namespace. The service description typically reports that the messaging entity could not be found, and names the path it looked for, such as sb://mynamespace.servicebus.windows.net/myeventhub. The hub name is misspelled, the hub was deleted, or the connection string or namespace points at a different namespace than you intended. The target never creates event hubs.

Fix: correct event_hub, or create the hub in that namespace. If the hub exists but its status is disabled in the portal, the reason is *Error{Condition: com.microsoft:entity-disabled, ...} instead, and re-enabling the entity clears it. Retried until fixed.

"i/o timeout", "connection refused", or certificate errors on port 5671

[Error] [director] [target-<target id>] [basic_eventhubs_target] Failed to reinitialize target "basic_eventhubs_target" (attempt 12). Reason: dial tcp <ip>:5671: i/o timeout

Cause: Director speaks AMQP over TLS to the namespace on TCP port 5671, and that port is blocked. There is no fallback to WebSockets or port 443, so an egress policy that only allows HTTPS fails this way. AMQP traffic does not use HTTP_PROXY or HTTPS_PROXY either. Token traffic does: a service principal contacts login.microsoftonline.com on port 443 through the proxy, and Managed Identity contacts the instance metadata endpoint directly.

Fix: allow outbound TCP 5671 from the Director host to mynamespace.servicebus.windows.net, and 443 to login.microsoftonline.com for service principals. Retried until fixed.

A reason such as x509: certificate signed by unknown authority or tls: failed to verify certificate typically means a TLS-intercepting proxy or firewall sits on port 5671. Install its root CA into the host trust store. The connection is always encrypted, so tls.status: true does not switch TLS on. It only presents a client certificate from tls.cert_name and tls.key_name, which Event Hubs does not require, and it is the only setting that makes tls.insecure_skip_verify apply. invalid tls configuration or tls: private key does not match public key at startup means that pair is missing, unreadable, or mismatched. Set tls.status: false unless you need to present a client certificate.

"record rejected by target: the EventData could not be added because it is too large for the batch"

[Error] [director] [target-<target id>] [high_volume_target] Sender worker 3 deterministic failure for high_volume_target after 4 attempts — dropping (giving up): record rejected by target: the EventData could not be added because it is too large for the batch

Cause: a single record does not fit in an empty batch. The ceiling is max_bytes when it is set, and otherwise the maximum message size the Event Hubs service reports for the link. Size is counted after AMQP encoding, which adds a few bytes per event plus the batch envelope, so a record slightly under max_bytes can still be too large. Multi-megabyte events, such as large EDR or base64 payloads, are the usual trigger.

Fix: raise max_bytes up to the service limit, trim the record in a pipeline, or route large records to another target. This is the only error on this target that drops data. The record is dropped after 4 deliveries and the drop is logged. Every other record in the same payload is delivered normally.

"maximum message size for batch was set to N bytes, which is larger than the maximum size allowed by link"

[Error] [director] [target-<target id>] [high_volume_target] Sender worker 0 ThreadSafeInit on reinit failed for "high_volume_target": maximum message size for batch was set to 33554432 bytes, which is larger than the maximum size allowed by link (1048576)

Cause: max_bytes is above the maximum message size the service allows on this namespace. The number in parentheses is the limit the service reported.

Fix: lower max_bytes to at most that number, or remove it so the service maximum is used. The error occurs when each worker opens its send link rather than at startup, so the connection status can still look healthy. Nothing is sent until the value is changed.

"context deadline exceeded" on flush, or com.microsoft:server-busy

[Error] [director] [target-<target id>] [high_volume_target] Sender worker 0 Finalize failed on flush for target "high_volume_target": context deadline exceeded

Cause: the batch send did not complete within timeout seconds (default 30). Throttling is the usual reason. When ingress exceeds the namespace's throughput units the service answers com.microsoft:server-busy, and the send is retried inside the same deadline until it runs out. A slow or lossy network, or a very large batch, has the same effect. When the retries are exhausted before the deadline, the reason is *Error{Condition: com.microsoft:server-busy, ...} instead.

Fix: add throughput units or enable auto-inflate on the namespace, lower max_bytes or max_events, or raise timeout. Retried until fixed. The failed batch is discarded from memory, its payloads stay queued, and they are redelivered after the target reinitializes.

"(connlost)" errors, and duplicate events after them

[Error] [director] [target-<target id>] [basic_eventhubs_target] Sender worker 2 Finalize failed on flush for target "basic_eventhubs_target": (connlost): *Error{Condition: amqp:connection:forced, Description: <service description>, Info: map[]}

Cause: the AMQP connection dropped and could not be re-established during the send. Common reasons are the service closing an idle connection after a quiet period, Azure maintenance, or an idle timeout on a proxy or firewall. Other forms are (connlost): amqp: connection closed and (connlost): read tcp ...: connection reset by peer.

Fix: none is usually needed. The target reinitializes and the batch is redelivered. If it happens often, check idle timeouts between Director and Azure. When the connection drops right after a batch was accepted, that batch is still reported as failed and redelivered, so the receiver sees those events twice. Deduplicate downstream on a record ID if duplicates matter. GlobalDispose ... returned: target not initialized and ThreadSafeDispose failed lines around a reinitialization are cleanup noise, not separate faults.

"ValidateConfig failed for target ..."

The configuration was rejected before any connection was attempted. Nothing is sent, and the check is repeated until the configuration changes.

Reason	Fix
event_hub is required for eventhubs target	Set event_hub. It is required in every authentication method, including connection strings
namespace is required when not using connection string authentication for eventhubs target	Set namespace to the fully qualified host, or use client_connection_string
tenant_id, client_id, and client_secret must all be provided together for service principal authentication	Fill all three, or clear all three to use Managed Identity
partition.id and partition.key cannot both be set for eventhubs target	Keep one of the two
max_events must be greater than 0, got 0 or max_bytes must be zero or greater, got -1	Remove the field, or set max_events to 1 or more and max_bytes to 0 or more

The target is healthy but nothing arrives in Event Hubs

Check these in order.

1. debug.dont_send_logs is enabled. Records are processed and acknowledged but never sent, and the target looks healthy. Turn it off.
2. A worker cannot open its send link. Startup only probes the event hub. Each worker then attaches its own link, and failures there are logged as Sender worker N ThreadSafeInit on reinit failed for "..." while the connection status can still look healthy. *Error{Condition: com.microsoft:argument-out-of-range, ...} means partition.id names a partition that does not exist. Partition IDs run from 0 to the partition count minus one. The maximum message size for batch ... error above is the other cause. Data waits until the worker succeeds.
3. Records arrive but are not normalized. If field_format names a format that does not exist, records are delivered unchanged and nothing is logged. Use a supported name from the Normalization section.
4. Records arrive on unexpected partitions. partition.id: "" and partition.key: "" are treated as unset, and events are distributed round-robin without a warning. Remove the key or give it a value. Two further settings are corrected silently: timeout: 0 or a negative value becomes 30 seconds, and tls.insecure_skip_verify: true is ignored unless tls.status is true.