Azure Blob Storage

Microsoft Azure Long Term Storage

Synopsis

Creates a target that writes log messages to Azure Blob Storage with support for various file formats, authentication methods, and retry mechanisms. Inherits file format capabilities from the base file target.

Schema

- name: <string>
  description: <string>
  type: azblob
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    account: <string>
    tenant_id: <string>
    client_id: <string>
    client_secret: <string>
    connection_string: <string>
    endpoint: <string>
    container: <string>
    name: <string>
    format: <string>
    extension: <string>
    compression: <string>
    schema: <string>
    field_format: <string>
    timeout: <numeric>
    max_size: <numeric>
    batch_size: <numeric>
    containers:
      - container: <string>
        name: <string>
        format: <string>
        compression: <string>
        extension: <string>
        schema: <string>
    function_app: <string>
    function_token: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Field	Required	Default	Description
name	Y		Target name
description	N	-	Optional description
type	Y		Must be azblob
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Azure

Field	Required	Default	Description
account	N*		Azure storage account name
tenant_id	N*	-	Azure tenant ID (required unless using managed identity or function app)
client_id	N*	-	Azure client ID (required unless using managed identity or function app)
client_secret	N*	-	Azure client secret (required unless using managed identity or function app)
connection_string	N*	-	Azure Storage connection string. Alternative to the service principal; carries the account key and any custom endpoint (e.g., Azurite). When set, the service-principal/AAD flow is skipped
endpoint	N*	-	Overrides the default https://{account}.blob.core.windows.net/ endpoint for the AAD flow
container	N**	-	Default container name, acting as the catch-all when containers is also specified. It has no default — an unset value is passed through empty

* = Conditionally required. See authentication methods below. account is required only when neither connection_string nor endpoint is provided.

** = Required if you want a catch-all container for unmatched events, or if not using the containers array.

Connection

Field	Required	Default	Description
timeout	N	30	Connection timeout in seconds
max_size	N	0	Maximum file size in bytes before uploading
batch_size	N	100000	Maximum number of messages per file

note

When max_size is reached, the current file is uploaded to blob storage and a new file is created. Setting max_size to 0 does not disable rotation — an explicit 0 is replaced by the 32 MB default (MustInt64), so there is no unlimited setting. Raise the value instead.

Function App (Optional)

Field	Required	Default	Description
function_app	N	-	Director Proxy base URL used to upload blobs, for example https://my-function-app.azurewebsites.net. It must start with https:// and carry no path, because Director appends the upload path
function_token	N	-	Director Proxy Access Token

If function_app is specified, the target will use the Function App to upload blobs instead of direct Azure Blob Storage SDK. This is useful for scenarios where direct access to storage is restricted.

note

function_app and function_token are not configurable in the GUI creation wizard. Set them directly in the target YAML file.

Files

The following fields can be used for files:

Field	Required	Default	Description
containers	N*	-	Array of container configurations for file distribution
containers.container	Y	-	Container name
containers.name	Y	-	Blob name template
containers.format	N	"json"	Output format: json, multijson, avro, parquet
containers.compression	N	"zstd"	Compression algorithm. See Compression below
containers.extension	N	Matches format	File extension override
containers.schema	N**	-	Schema reference: a Library name, built-in name, file path, or inline definition (required for Avro and Parquet formats)
name	N	"vmetric.{{.Timestamp}}.{{.Extension}}"	Default blob name template (used with container for catch-all)
format	N	"json"	Default output format (used with container for catch-all)
extension	N	Matches format	Default file extension (used with container for catch-all)
compression	N	"zstd"	Default compression algorithm (used with container for catch-all)
schema	N	-	Default schema reference (Library name, built-in name, file path, or inline definition; used with container for catch-all)
field_format	N	-	Data normalization format. See applicable Normalization section

* = Either container or containers must be specified.

** = Conditionally required for Avro and Parquet formats when using containers.

Scheduling

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging
debug.dont_send_logs	N	false	Process logs but don't send to target (testing)

Details

The Azure Blob Storage target supports writing to multiple containers with various file formats and schemas. The target provides cloud storage integration with comprehensive file format support.

Authentication Methods

The target supports multiple authentication methods:

Service Principal Authentication: Use tenant_id, client_id, and client_secret for explicit credential-based authentication. Optionally set endpoint to override the default https://{account}.blob.core.windows.net/ URL.

Connection String Authentication: Set connection_string to authenticate with the account key embedded in the string. The connection string also carries any custom endpoint (e.g., Azurite), so the service-principal/AAD flow is skipped entirely and tenant_id/client_id/client_secret/account are not required.

Function App Authentication: When function_app is specified, authentication is handled through the Function App token, and direct storage credentials are not required.

Managed Identity Authentication

Azure targets support Managed Identity authentication for credential-free access when Director is deployed on Azure infrastructure.

How it works: When tenant_id, client_id, and client_secret are omitted from the configuration, the target automatically uses Azure's DefaultAzureCredential, which attempts authentication in the following order:

1. Environment variables (AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET)
2. Workload Identity (for Kubernetes deployments)
3. Managed Identity (system-assigned or user-assigned)
4. Azure CLI credentials
5. Azure PowerShell credentials

Deployment requirement: Director must run on Azure infrastructure that supports Managed Identity:

• Azure Virtual Machines
• Azure App Service
• Azure Container Instances
• Azure Kubernetes Service (AKS)
• Azure Functions

Required permissions: The Managed Identity must be granted the appropriate Azure RBAC roles documented in each target's IAM Permissions section.

note

Managed Identity eliminates credential management overhead and is the recommended authentication method for Azure-hosted Director deployments.

IAM Permissions

When using Service Principal or Managed Identity authentication, the following Azure role is required:

Azure Role	Scope	Purpose
Storage Blob Data Contributor	Storage Account or Container	Write blobs to containers

note

When using connection string authentication, Azure RBAC roles are not applicable -- access is governed by the storage account key embedded in the connection string.

Container Routing

The target supports flexible container routing through pipeline configuration or explicit container settings:

Configuration-based routing: Define multiple containers in the target configuration, each with its own format, compression, and schema settings. Logs are routed to specific containers based on configuration.

Pipeline-based routing: Use the container field in pipeline processors to dynamically route logs to different containers at runtime. This enables conditional routing based on log content, source, or other attributes.

Catch-all routing: When a log doesn't match any specific container configuration or when no container field is set in the pipeline, logs are routed to the catch-all container (configured via the container field in target properties).

Routing priority:

1. Pipeline container field (highest priority)
2. Configured containers in containers array (if container name matches)
3. Default container field (catch-all, lowest priority)

This multi-level routing enables flexible data distribution strategies, such as routing different log types to different containers based on content analysis, source system, severity level, or any other runtime decision.

File Formats

Format	Description
json	Each log entry is written as a separate JSON line (JSONL format)
multijson	All log entries are written as a single JSON array
avro	Apache Avro format with schema and compression support
parquet	Apache Parquet columnar format with schema

Compression

Some formats support built-in compression to reduce storage costs and transfer times. When supported, compression is applied at the file/block level before upload.

Format	Default	Compression Codecs
JSON	-	Not supported
MultiJSON	-	Not supported
Avro	zstd	deflate, snappy, zstd
Parquet	zstd	gzip, snappy, zstd, brotli, lz4

File Management

Files are rotated based on size (max_size parameter) or event count (batch_size parameter), whichever limit is reached first. Template variables in blob names enable dynamic file naming for time-based partitioning.

Schema Support

The target supports the following built-in schema templates for structured data formats:

• Syslog - Standard schema for Syslog messages
• CommonSecurityLog - Schema compatible with Common Security Log Format (CSL)

You can also reference custom schema files by name (without the .json extension). The system will search for schema files in:

1. User schema directory: <user-path>/schemas/
2. Package schema directory: <package-path>/schemas/

Schema files are searched recursively in these directories, and filename matching is case-insensitive.

Templates

The following template variables can be used in blob names:

Variable	Description	Example
{{.Year}}	Current year	2024
{{.Month}}	Current month	01
{{.Day}}	Current day	15
{{.Timestamp}}	Current timestamp in nanoseconds	1703688533123456789
{{.Format}}	File format	json
{{.Extension}}	File extension	json
{{.Compression}}	Compression type	zstd
{{.TargetName}}	Target name	my_logs
{{.TargetType}}	Target type	azblob
{{.Table}}	Container name	logs
{{.Thread}}	Writer thread index. The sender also appends this automatically when two threads would otherwise produce the same path, so an explicit token is only needed to control WHERE it lands	3
{{.ServiceRoot}}	The service root directory	/opt/vmetric

Multiple Containers

Single target can write to multiple Azure Blob Storage containers with different configurations, enabling data distribution strategies (e.g., raw data to one container, processed data to another).

note

Files with no messages (i.e. with counter=0) are automatically skipped during upload.

Examples

The following upload configurations are available.

Basic Configuration

The minimum configuration for a JSON blob storage:

targets:
  - name: basic_blob
    type: azblob
    properties:
      account: "mystorageaccount"
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      container: "logs"

Managed Identity

Using Managed Identity for credential-free authentication on Azure infrastructure:

targets:
  - name: managed_identity_blob
    type: azblob
    properties:
      account: "mystorageaccount"
      container: "logs"
      format: "json"

Pipeline-Based Routing

Dynamic container routing using pipeline processors to analyze log content and route to appropriate containers:

targets:
  - name: smart_routing_blob
    type: azblob
    pipelines:
      - dynamic_routing
    properties:
      account: "mystorageaccount"
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      containers:
        - container: "security-events"
          name: "security_{{.Year}}_{{.Month}}_{{.Day}}_{{.Timestamp}}.parquet"
          format: "parquet"
          schema: "CommonSecurityLog"
          compression: "zstd"
        - container: "application-events"
          name: "app_{{.Year}}_{{.Month}}_{{.Day}}_{{.Timestamp}}.json"
          format: "json"
        - container: "system-events"
          name: "system_{{.Year}}_{{.Month}}_{{.Day}}_{{.Timestamp}}.avro"
          format: "avro"
          schema: "Syslog"
          compression: "snappy"
      container: "other-events"
      name: "other_{{.Timestamp}}.json"
      format: "json"

pipelines:
  - name: dynamic_routing
    processors:
      - set:
          field: "_vmetric.container"
          value: "security-events"
          if: "ctx.event_type == 'security'"
      - set:
          field: "_vmetric.container"
          value: "application-events"
          if: "ctx.event_type == 'application'"
      - set:
          field: "_vmetric.container"
          value: "system-events"
          if: "ctx.event_type == 'system'"

Multiple Containers with Catch-All

Configuration with multiple target containers and a catch-all default container:

targets:
  - name: multi_container_blob
    type: azblob
    properties:
      account: "mystorageaccount"
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      containers:
        - container: "security-logs"
          name: "security_{{.Year}}_{{.Month}}_{{.Day}}_{{.Timestamp}}.parquet"
          format: "parquet"
          schema: "CommonSecurityLog"
          compression: "zstd"
        - container: "system-logs"
          name: "system_{{.Year}}_{{.Month}}_{{.Day}}_{{.Timestamp}}.json"
          format: "json"
        - container: "application-logs"
          name: "app_{{.Year}}_{{.Month}}_{{.Day}}_{{.Timestamp}}.avro"
          format: "avro"
          schema: "Syslog"
          compression: "snappy"
      container: "general-logs"
      name: "general_{{.Timestamp}}.json"
      format: "json"
      schema: "Syslog"

Parquet Format

Configuration for daily partitioned Parquet files:

targets:
  - name: parquet_blob
    type: azblob
    properties:
      account: "mystorageaccount"
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      container: "logs"
      format: "parquet"
      compression: "zstd"
      name: "logs/year={{.Year}}/month={{.Month}}/day={{.Day}}/data_{{.Timestamp}}.parquet"
      schema: "Syslog"
      max_size: 536870912  # 512MB

Avro with Custom Schema

Configuration for Avro format with a custom schema file:

targets:
  - name: avro_blob
    type: azblob
    properties:
      account: "mystorageaccount"
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      container: "logs"
      format: "avro"
      compression: "snappy"
      name: "logs_{{.Year}}_{{.Month}}_{{.Day}}_{{.Timestamp}}.avro"
      schema: "MyCustomSchema"

Function App Upload

Configuration using Azure Function App for uploading:

targets:
  - name: function_blob
    type: azblob
    properties:
      account: "mystorageaccount"
      function_app: "https://my-function-app.azurewebsites.net"
      function_token: "your-function-token"
      container: "logs"

Debug Configuration

Configuration with debugging enabled:

targets:
  - name: debug_blob
    type: azblob
    properties:
      account: "mystorageaccount"
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      container: "logs"
      debug:
        status: true
        dont_send_logs: true

High Volume with Batching

Configuration optimized for high-volume ingestion:

targets:
  - name: high_volume_blob
    type: azblob
    properties:
      account: "mystorageaccount"
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      container: "logs"
      format: "parquet"
      compression: "zstd"
      batch_size: 50000
      max_size: 536870912  # 512MB
      timeout: 60

Troubleshooting

This section covers the errors you are most likely to see with the azblob target, what causes each one, and how to fix it. See Target Delivery Errors for how Director logs and retries target failures.

Where to look:

• Director logs. Target errors are tagged with the target name and carry "Section":"SenderPool". The part after Reason: or after the last colon is the actual cause.
• The target's connection status in the web interface. It shows the same reason as the log line, prefixed with connection failed for <target name>:.

Two facts about this target matter for every entry below. Building the storage client does not contact Azure, so a wrong credential or a missing role is reported on the first upload, not at startup, and the connection status can show connected while every upload fails. The target also never creates containers. Every container named in container or containers must already exist.

Which permission is missing?

Match the error you see against this table first. Which identity needs the role depends on the authentication method: the application in client_id for a service principal, the host's managed identity when any of tenant_id, client_id, or client_secret is empty, and the Director Proxy's own managed identity when function_app is set.

Error text	Missing role	Scope
RESPONSE 403 ... ERROR CODE: AuthorizationPermissionMismatch	Storage Blob Data Contributor	The storage account, or only the container the target writes to. Owner or Contributor alone is not enough, because neither carries blob write permission
function returned status: 500 Internal Server Error, body: Failed to process log file: Cannot upload file. Reason: ...	Storage Blob Data Contributor for the Director Proxy managed identity	The storage account or the container
DefaultAzureCredential: failed to acquire a token	A managed identity on the Director host, holding Storage Blob Data Contributor	The Azure resource Director runs on, and the storage account
ClientSecretCredential authentication failed ... AADSTS...	None. The credential itself is wrong. See the next entry	Tenant
RESPONSE 403 ... ERROR CODE: AuthenticationFailed or InvalidAuthenticationInfo	None. The account key or SAS in connection_string is wrong, rotated, or expired	Storage account
RESPONSE 403 ... ERROR CODE: AuthorizationFailure	None. The storage account firewall or network rules block the Director host	The storage account's Networking settings

note

Azure RBAC changes typically take several minutes to propagate. Director retries automatically, so no restart is needed once the role is assigned.

"ClientSecretCredential authentication failed" with an AADSTS code

Sender worker 0 execute() failed for basic_blob: target broken: failed to finalize target cache: ClientSecretCredential authentication failed. POST https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/v2.0/token
RESPONSE 401 Unauthorized
{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided. ..."}
To troubleshoot, visit https://aka.ms/azsdk/go/identity/troubleshoot#client-secret

Cause: the service principal credentials are wrong. The token request fails before any permission is checked, and it appears on the first upload rather than at startup. The AADSTS code tells you which field:

Code	Meaning	Check
AADSTS7000215	Invalid client secret	The secret ID was pasted instead of the secret value, or the secret was rotated
AADSTS7000222	Client secret expired	Create a new secret on the app registration and update client_secret
AADSTS700016	Application not found in the directory	client_id is wrong, or tenant_id is a different tenant than the one the app registration lives in
AADSTS90002	Tenant not found	tenant_id is not a valid tenant ID

Fix: correct the field and save the target. The failed batch stays in the queue and is redelivered once the token request succeeds. Nothing is lost.

"DefaultAzureCredential: failed to acquire a token" or ManagedIdentityCredential errors

Sender worker 0 execute() failed for managed_identity_blob: target broken: failed to finalize target cache: DefaultAzureCredential: failed to acquire a token.
Attempted credentials:
	EnvironmentCredential: missing environment variable AZURE_TENANT_ID
	ManagedIdentityCredential: managed identity timed out. See https://aka.ms/azsdk/go/identity/troubleshoot#dac for more information
	AzureCLICredential: Azure CLI not found on path

Cause: the target is using Managed Identity and no identity is available on the host. This is the expected path for the Managed Identity example. It is also what happens by accident when any one of tenant_id, client_id, or client_secret is empty. When one of the three is missing, even if the other two are filled in, Director treats the service principal as not configured and falls back to Managed Identity. A typical trigger is a client_secret whose $secret{...} reference resolves to an empty value.

managed identity timed out means the Azure instance metadata endpoint did not answer. Those token requests go to the metadata endpoint directly and bypass any HTTPS_PROXY setting, so a host firewall that blocks that address also produces this timeout. the requested identity isn't assigned to this resource means Azure answered, but no managed identity is assigned to the resource Director runs on.

Fix: either fill in all three service principal fields, or enable a managed identity on the Azure resource Director runs on and assign it Storage Blob Data Contributor on the storage account. Retried until fixed.

"failed to resolve client secret" and other credential reference errors

Failed to reinitialize target "basic_blob" (attempt 7). Reason: failed to resolve client secret: credential: env variable "AZ_BLOB_SECRET" is not set

Cause: a credential property holds a ${ENV} or $secret{...} reference that could not be resolved. The first part names the property: failed to resolve connection string, tenant ID, client ID, client secret, or function token. The part after credential: says why:

Text	Meaning
env variable "..." is not set	The variable is not exported to the Director service user. A variable set in your own shell is not visible to the service
failed to parse token "...": not a valid secret token	The $secret{...} syntax is malformed
store "..." not found in configuration	The store name in $secret{...} does not match a configured credential store
provider type "..." is not registered, or no credentials configured	The store's provider type is misspelled, or no credential store is defined at all

Fix: export the variable for the service, or correct the secret reference or the store definition. Nothing is sent while this persists, retried until fixed.

"RESPONSE 403" with ERROR CODE AuthorizationPermissionMismatch

Sender worker 1 execute() failed for basic_blob: target broken: failed to finalize target cache: PUT https://mystorageaccount.blob.core.windows.net/mycontainer/vmetric.1757590123456789000.json
RESPONSE 403: 403 ...
ERROR CODE: AuthorizationPermissionMismatch

Cause: the token was accepted, but the identity has no blob write permission on the container. Typically the application or managed identity was given Owner or Contributor on the storage account. Those roles manage the account and do not include writing blobs. The same code appears with a connection_string whose SAS lacks Create or Write.

Fix: in the Azure Portal open the storage account, or the single container if you want to limit the scope, then Access control (IAM) > Add role assignment, and assign Storage Blob Data Contributor to the application whose ID is in client_id, or to the managed identity. For a SAS, regenerate it with Create and Write. The batch is kept and redelivered until the role is in place. Nothing is lost.

"RESPONSE 404" with ERROR CODE ContainerNotFound

Sender worker 1 execute() failed for basic_blob: target broken: failed to finalize target cache: PUT https://mystorageaccount.blob.core.windows.net/mycontainer/vmetric.1757590123456789000.json
RESPONSE 404: 404 ...
ERROR CODE: ContainerNotFound

Cause: the container in the PUT URL does not exist in the storage account. The target never creates containers, so a misspelled container, or a containers entry whose container was never created, ends here.

Fix: create the container in the storage account under Data storage > Containers, or correct the name. Azure container names must be lower-case. Retried until fixed.

"RESPONSE 403" with ERROR CODE AuthenticationFailed, InvalidAuthenticationInfo, or AuthorizationFailure

AuthenticationFailed or InvalidAuthenticationInfo: the connection_string carries an account key that was rotated, or a SAS that has expired. Copy the current value from the storage account's Access keys page. If the key is current, check the host clock. A large clock skew invalidates the request signature.

AuthorizationFailure: the storage account's firewall, virtual network rules, or private endpoint reject the Director host's address. Allow the egress address under the storage account's Networking settings, or route through the private endpoint. Both cases are retried until fixed.

"Failed to reinitialize target ... (attempt N)" with a configuration reason

Failed to reinitialize target "parquet_blob" (attempt 2). Reason: invalid container configuration: schema is required for parquet format

Cause: the target could not start, and Director retries with a growing attempt counter. For this target the reason is almost always a configuration mistake:

Reason	Fix
azure blob target: missing account name — set 'account', 'endpoint', or 'connection_string'	Set account to the storage account name, not its URL
connection string needs either AccountName or BlobEndpoint, ... either AccountKey or SharedAccessSignature, or connection string is either blank or malformed. ...	Only part of the string was pasted, or the value is not key=value pairs separated by semicolons. Copy the full Connection string value from the Access keys page. A SAS URL is not a connection string
no valid containers found, invalid container configuration: container is required, or name is required	Neither container nor containers is set, or a containers entry is missing its container or its name
invalid container configuration: schema is required for parquet format, or for avro format	Set schema for that container, or at the top level for the catch-all
schema file not found for: MyCustomSchema	The file is not under the user or package schemas/ directory. The match is case-insensitive and takes no .json extension
failed to create Avro encoder: ..., reported as ThreadSafeInit on reinit failed	The Avro schema was found but is not encodable. Fix the schema

Nothing is sent while the target is failing. Incoming data waits in the Director queue and is delivered once the target starts. Fix the reason shown and the next attempt picks it up. Restarting Director is not required.

"no such host", "context deadline exceeded", "connection refused", or TLS errors

Sender worker 2 execute() failed for basic_blob: target broken: failed to finalize target cache: Put "https://mystorageaccount.blob.core.windows.net/mycontainer/vmetric.1757590123456789000.json": dial tcp: lookup mystorageaccount.blob.core.windows.net: no such host
Sender worker 0 Finalize failed on flush for target "parquet_blob": Put "https://mystorageaccount.blob.core.windows.net/mycontainer/vmetric.1757590123456789000.parquet": context deadline exceeded

Cause: Director could not complete the HTTPS request to the storage endpoint.

• no such host: account is misspelled, or endpoint names a host that does not resolve. The hostname in the URL is the value to check.
• context deadline exceeded: the upload did not finish within timeout seconds, 30 by default. The timeout covers one whole upload, including the SDK's own retries, so a 32 MB blob on a slow link can exceed it. Raise timeout, or lower max_size.
• connection refused: nothing listens at endpoint, which is typical for a stopped emulator or proxy.
• tls: failed to verify certificate: x509: certificate signed by unknown authority: a TLS-intercepting proxy is in the path. The target has no TLS options of its own and trusts the OS certificate store, so install the proxy's CA certificate at the OS level.
• authorized requests are not permitted for non-TLS protected (https) endpoints: endpoint is an http:// URL while the target authenticates with a service principal or a managed identity. Use https://, or authenticate with connection_string when you need plain HTTP.

Fix: allow outbound HTTPS (443) from the Director host to your storage endpoint, and to login.microsoftonline.com for service principal authentication. If you use a proxy, set HTTPS_PROXY for the Director service. Retried until fixed.

"RESPONSE 400", "RESPONSE 409", or "RESPONSE 503" from the storage account

Error code	Cause	Fix
InvalidResourceName, OutOfRangeInput, or InvalidUri with 400	The container name has upper-case letters or underscores, or the blob name is invalid. A name template with a syntax error is not reported on its own. Its unexpanded text becomes the blob name, and the template characters then fail here	Use lower-case container names. Check name against the Templates table
BlobImmutableDueToPolicy with 409	The container has an immutability policy, and name produces the same blob name on every flush, so the next upload tries to replace a locked blob	Add {{.Timestamp}} to name so every upload creates a new blob
ContainerBeingDeleted or ContainerDisabled with 409, or AccountIsDisabled with 403	The container or the account is being removed, or is disabled	Re-enable it, or point the target at another container
ServerBusy with 503, or OperationTimedOut and InternalError with 500	The storage account is throttling ingress	Spread the load over more containers or accounts, and raise timeout. The SDK already retries these up to 3 times inside the timeout window before the error is logged

warning

Azure rejects the 400 and 409 cases on every attempt, but Director still retries them and reinitializes the target each time. The batch is never dropped, so the error repeats until the configuration is corrected.

"failed to send log record: file holder not found"

Sender worker 3 execute() failed for smart_routing_blob: target broken: failed to send log record: file holder not found

Cause: a pipeline set _vmetric.container to a value that matches none of the containers entries, and the target has no catch-all container. The comparison is exact and case-sensitive. The same wrapper with an Avro or Parquet message after failed to send log record: means one record does not fit the schema, usually a missing non-nullable column or a wrong type.

Fix: add a catch-all container, or make the pipeline value match a configured container name exactly. For an encoding error, make the schema fields nullable, or normalize the record with field_format. Until then that record is retried every few seconds and the target reinitializes on every attempt, so everything queued behind it on that worker stalls.

Only the last batch of the day is in the container

Cause: the blob name template contains no {{.Timestamp}} and no {{.Thread}}. A template such as app_{{.Year}}_{{.Month}}_{{.Day}}.json expands to the same blob name on every flush, and each upload replaces the previous blob. Only the final batch of the period survives. Nothing is logged, and the delivery counters and the connection status report success, because every upload did succeed.

Fix: always include {{.Timestamp}} in name, for example app_{{.Year}}_{{.Month}}_{{.Day}}_{{.Timestamp}}.json. The default template vmetric.{{.Timestamp}}.{{.Extension}} is safe. Batches already overwritten cannot be recovered.

The target is healthy but nothing arrives in the container

Check these in order.

1. debug.dont_send_logs is enabled. In that mode events are processed but never uploaded. With debug.status: true, startup logs Log sending is disabled for this target once, naming the target. With debug.status: false nothing is logged at all. Remove the flag. The Debug Configuration example on this page has it set.

2. The blob is overwritten on every flush. See the entry above. Look for a single blob whose modified time keeps changing.

3. Every upload is failing. Building the storage client does not contact Azure, so the connection status stays connected while uploads fail. Search the Director log for execute() failed or Finalize failed on flush with the target name.

4. The data landed in another container. A pipeline _vmetric.container value routes the event to that container, and unmatched events go to the catch-all container.

5. The blob is there, but its content is not what you expect. format and compression are matched case-sensitively. A value such as Parquet falls back to JSON lines with that literal extension, and the schema check is skipped. A misspelled compression is ignored and the blob is written uncompressed, while {{.Compression}} in name still renders the requested value. Use the lower-case names listed under File Formats and Compression.

Errors when using Director Proxy

When the target uploads through Director Proxy, Director only talks to the proxy, and the proxy's managed identity writes the blob. In YAML this mode is the function_app and function_token pair. Their values are the Proxy URL and the Access Token of the proxy entry, shown on its Configuration tab in the web interface. For this target the pair is set in the target YAML, as noted under Function App above. Storage Blob Data Contributor must be granted to the proxy's identity, not to Director, and account must still be set, because the proxy needs the storage account name. See VirtualMetric Director Proxy for the deployment and Proxy for managing proxy entries.

Error text	Cause	Fix
function returned status: 400 Bad Request, body: Function token is not valid	function_token does not match the token configured on the Director Proxy	Copy the proxy's Access Token from its Configuration tab into function_token
function returned status: 500 Internal Server Error, body: Failed to process log file: Cannot upload file. Reason: ...	The Director Proxy's managed identity lacks Storage Blob Data Contributor, or the container does not exist	Assign the role to the proxy's identity on the storage account or the container, and create the container. The text after Reason: is the storage error described earlier on this page
function returned status: 400 Bad Request, body: Please provide a Account Name via X-Account-Name	account is empty	Set account. It is required in this mode too
function returned status: 400 Bad Request, body: Data size exceeds the maximum allowed size of 104857600 bytes, or body: Failed to read log file: http: request body too large	One blob is larger than the 100 MB the Director Proxy accepts	Keep max_size well below 100 MB. The 512 MB value in the Parquet and High Volume examples applies to direct upload only. An oversized blob is retried until max_size is lowered
failed to create HTTP request: parse "...": ...	function_app is not a valid URL, for example it contains a space	Set function_app to the proxy's Proxy URL. It must start with https://, and an http:// prefix is not supported. Give the base address only, without a path. Director appends the upload path itself
HTTP request failed: Post "...": dial tcp: lookup ...: no such host, ...: context deadline exceeded, or failed to read response body: ...	Director cannot reach the Director Proxy, the upload took longer than timeout, or the connection dropped mid-response	Check DNS and firewall rules from the Director host, confirm function_app is the address of the deployed Director Proxy, and raise timeout for large blobs
HTTP request failed: Post "...": context canceled	Another property of the target failed to initialize, for example a $secret{...} reference that could not be resolved. That failure is not reported separately in this mode	Fix the credential and connection properties even though uploads go through Director Proxy
A 401, 404, or 503 status with a body not listed above	Typically the Azure Functions host rather than the Director Proxy itself, for example a wrong URL path, a stopped app, or a cold start	Confirm the Director Proxy is running and the Proxy URL is correct. A cold start clears on the next retry