Azure Data Explorer

Microsoft Azure Observability

Synopsis

Creates an Azure Data Explorer (Kusto) target that ingests data directly into Azure Data Explorer tables. Supports multiple tables, custom schemas, and various file formats.

Schema

- name: <string>
  description: <string>
  type: azdx
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    tenant_id: <string>
    client_id: <string>
    client_secret: <string>
    function_app: <string>
    function_token: <string>
    endpoint: <string>
    database: <string>
    table: <string>
    schema: <string>
    format: <string>
    compression: <string>
    flush_immediately: <boolean>
    timeout: <numeric>
    batch_size: <numeric>
    max_size: <numeric>
    field_format: <string>
    tables:
      - table: <string>
        schema: <string>
        format: <string>
        compression: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Field	Required	Default	Description
name	Y	-	Target name
description	N	-	Optional description
type	Y	-	Must be azdx
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Azure

Field	Required	Default	Description
tenant_id	N(1)	-	Azure tenant ID (required for direct authentication)
client_id	N(1)	-	Azure client ID (required for direct authentication)
client_secret	N(1)	-	Azure client secret (required for direct authentication)
function_app	N(1)	-	Director Proxy endpoint URL (required for proxy forwarding)
function_token	N(1)	-	Director Proxy authentication token (required with function_app)
endpoint	Y	-	Azure Data Explorer cluster endpoint
database	Y	-	Target database name
table	N(2)	-	Default/fallback table name (catch-all for unmatched events)
schema	N(2)	-	Table schema for the default/fallback table: a Library name, built-in name, or inline definition
format	N	parquet	Data format. See Formats below
compression	N	zstd	Compression algorithm. See Compression below

(1) = Conditionally required. Use either direct authentication (tenant_id, client_id, client_secret) OR Director Proxy forwarding (function_app, function_token), OR omit all credentials to use Managed Identity.

(2) = Required if you want a catch-all table for unmatched events, or if not using the tables array

Managed Identity Authentication

Azure targets support Managed Identity authentication for credential-free access when Director is deployed on Azure infrastructure.

How it works: When tenant_id, client_id, and client_secret are omitted from the configuration, the target automatically uses Azure's DefaultAzureCredential, which attempts authentication in the following order:

1. Environment variables (AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET)
2. Workload Identity (for Kubernetes deployments)
3. Managed Identity (system-assigned or user-assigned)
4. Azure CLI credentials
5. Azure PowerShell credentials

Deployment requirement: Director must run on Azure infrastructure that supports Managed Identity:

• Azure Virtual Machines
• Azure App Service
• Azure Container Instances
• Azure Kubernetes Service (AKS)
• Azure Functions

Required permissions: The Managed Identity must be granted the appropriate Azure RBAC roles documented in each target's IAM Permissions section.

note

Managed Identity eliminates credential management overhead and is the recommended authentication method for Azure-hosted Director deployments.

Ingestion Options

Field	Required	Default	Description
flush_immediately	N	true	Send data to ADX without waiting for batch completion
timeout	N	30	Connection timeout in seconds
batch_size	N	100000	Maximum number of messages per batch
max_size	N	32MB	Maximum batch size in bytes. An explicit 0 is replaced by the 32 MB default, so it does not mean unlimited
field_format	N	-	Data normalization format. See applicable Normalization section

Multiple Tables

You can define multiple tables to ingest data into:

targets:
  - name: azdx_multiple_tables
    type: azdx
    properties:
      tables:
        - table: "security_logs"
          schema: "<schema definition>"
        - table: "system_logs"
          schema: "<schema definition>"

Scheduling

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging
debug.dont_send_logs	N	false	Process logs but don't send to target (testing)

Details

The Azure Data Explorer target supports ingesting data into multiple tables with different schemas. When using the SystemS3 field in your logs, the value will be used to route the message to the appropriate table.

Table Routing and Catch-All Behavior

warning

At most ten tables are allowed. An eleventh fails the target with maximum of 10 tables allowed. The catch-all table at the root level counts toward the limit, so ten entries under tables plus a catch-all fails.

Each entry requires its own table key — an entry without one fails with invalid table configuration: table is required. For avro and parquet an entry also requires a schema.

The target uses a routing system to direct events to the appropriate table:

1. Explicit Table Matching: If an event has a SystemS3 field, the target looks for a table defined in the tables array with a matching name
2. Catch-All Table: If no matching table is found (or if SystemS3 is not set), the event is routed to the default table specified at the root level

The table and schema properties at the root level serve as a catch-all mechanism. This is particularly useful for automation scenarios where systems may look for specific tables with specific schemas. If no matching table is found in the tables array, these events will fall back to the default table instead of being dropped.

Example routing logic:

Event with SystemS3="security_logs" → routes to "security_logs" table if defined
Event with SystemS3="unknown_table" → routes to default table if configured
Event without SystemS3 → routes to default table if configured

Data is buffered locally until batch_size or max_size is reached, or when an explicit flush is triggered.

Tables are not discovered from the database. Every target must configure either the catch-all table or at least one entry under tables, otherwise it fails with at least one table must be configured (either 'table' or 'tables'). Each configured table must exist in the database, and its name must match the Kusto table name exactly, including case.

Permissions

Azure Data Explorer uses its own database-level RBAC (not Azure IAM roles). The following ADX database roles apply:

ADX Database Role	Scope	Purpose
Database Ingestor	ADX Database	Required. Ingest data via managed ingestion. Table Ingestor on each configured table works as well
Database Viewer	ADX Database	Optional. Read table names and schemas (.show tables, .show table schema) when checking the configuration against the database

Formats

Format	Description
json	Each log entry is written as a separate JSON line (JSONL format)
multijson	All log entries are written as a single JSON array
avro	Apache Avro format with schema
parquet	Apache Parquet columnar format with schema (default)

Compression

Data can use the following compression algorithms:

Format	Default	Compression Codecs
JSON	-	Not supported
MultiJSON	-	Not supported
Avro	zstd	deflate, snappy, zstd
Parquet	zstd	gzip, snappy, zstd, brotli, lz4

warning

Consider cluster capacity when setting batch sizes and timeouts.

Examples

Basic

The minimum required configuration for Parquet ingestion:

targets:
  - name: basic_adx
    type: azdx
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "11111111-1111-1111-1111-111111111111"
      client_secret: "your-client-secret"
      endpoint: "https://cluster.region.kusto.windows.net"
      database: "logs"
      table: "system_events"
      schema: "TimeGenerated:datetime,Computer:string,Message:string"

Managed Identity

Using Managed Identity for credential-free authentication on Azure infrastructure:

targets:
  - name: managed_identity_adx
    type: azdx
    properties:
      endpoint: "https://cluster.region.kusto.windows.net"
      database: "logs"
      table: "system_events"
      schema: "TimeGenerated:datetime,Computer:string,Message:string"

Multiple Tables with Catch-All

Configuration with multiple target tables and a catch-all default table:

targets:
  - name: multi_table_adx
    type: azdx
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "11111111-1111-1111-1111-111111111111"
      client_secret: "your-client-secret"
      endpoint: "https://cluster.region.kusto.windows.net"
      database: "logs"
      format: "parquet"
      compression: "zstd"
      # Catch-all table for unmatched events
      table: "general_logs"
      schema: "TimeGenerated:datetime,Message:string,Source:string"
      tables:
        - table: "security_events"
          schema: "TimeGenerated:datetime,Computer:string,EventID:int,Message:string"
          format: "parquet"
          compression: "zstd"
        - table: "system_events"
          schema: "TimeGenerated:datetime,Computer:string,EventID:int,Message:string"
          format: "avro"
          compression: "snappy"
        - table: "application_events"
          schema: "TimeGenerated:datetime,Computer:string,EventID:int,Message:string"
          format: "parquet"
          compression: "brotli"

In this example, events with SystemS3 set to "security_events", "system_events", or "application_events" will route to their respective tables. All other events will route to the "general_logs" catch-all table.

High-Volume

Configuration optimized for high-volume ingestion:

targets:
  - name: high_volume_adx
    type: azdx
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "11111111-1111-1111-1111-111111111111"
      client_secret: "your-client-secret"
      endpoint: "https://cluster.region.kusto.windows.net"
      database: "logs"
      table: "system_events"
      schema: "TimeGenerated:datetime,Computer:string,Message:string"
      format: "parquet"
      batch_size: 50000
      max_size: 67108864
      timeout: 60
      flush_immediately: false

max_size must stay below 104857600. A larger batch is rejected when the target forwards through Director Proxy, and that batch is dropped.

With Debugging

Configuration with debug options:

targets:
  - name: debug_adx
    type: azdx
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "11111111-1111-1111-1111-111111111111"
      client_secret: "your-client-secret"
      endpoint: "https://cluster.region.kusto.windows.net"
      database: "logs"
      table: "system_events"
      schema: "TimeGenerated:datetime,Computer:string,Message:string"
      debug:
        status: true
        dont_send_logs: true  # Test mode that doesn't actually upload

Normalized

Using field normalization before ingestion:

targets:
  - name: normalized_adx
    type: azdx
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "11111111-1111-1111-1111-111111111111"
      client_secret: "your-client-secret"
      endpoint: "https://cluster.region.kusto.windows.net"
      database: "logs"
      table: "system_events"
      schema: "TimeGenerated:datetime,Computer:string,Message:string"
      field_format: "asim"

Automation-Friendly Configuration

Configuration designed for automation tools that expect specific table names:

targets:
  - name: automation_adx
    type: azdx
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "11111111-1111-1111-1111-111111111111"
      client_secret: "your-client-secret"
      endpoint: "https://cluster.region.kusto.windows.net"
      database: "logs"
      # Catch-all ensures automation-generated events always have a destination
      table: "automated_events"
      schema: "Timestamp:datetime,EventType:string,Data:string,Source:string"
      tables:
        - table: "monitoring_alerts"
          schema: "Timestamp:datetime,AlertLevel:string,Message:string"
        - table: "deployment_logs"
          schema: "Timestamp:datetime,Service:string,Version:string,Status:string"

In this configuration, automation tools looking for "monitoring_alerts" or "deployment_logs" will find their specific tables with the expected schemas. Any other automated events will be captured in the "automated_events" catch-all table, ensuring no data is lost.

Troubleshooting

This section covers the errors you are most likely to see with the azdx target, what causes each one, and how to fix it.

Where to look:

• Director logs. Target errors are tagged with the target name and carry "Section":"SenderPool". The part after Reason: or after the last colon is the actual cause.
• The target's connection status in the web interface. It shows the same reason as the log line, prefixed with connection failed for <target name>:.

See Target Delivery Errors for how Director logs and retries target failures.

warning

Credentials and database permissions are not checked when the target starts. A wrong secret or a missing Database Ingestor role first appears when the first batch is flushed. Until then the connection status shows connected. If data does not arrive, look for execute() failed and Finalize failed on flush lines, not only for Failed to reinitialize.

Which permission is missing?

Match the error you see against this table first. Azure Data Explorer uses its own database roles, not Azure IAM role assignments.

Error text	Missing role	Scope
error from Kusto endpoint, ...(403 Forbidden): in an execute() failed or Finalize failed on flush line	Database Ingestor, or Table Ingestor on every table the target writes to	The ADX database named in database
problem getting authorization context from Kusto via Mgmt: ...(403 Forbidden)	Database Ingestor	The same database. This form appears when a batch is larger than 4 MB and goes through queued ingestion
function returned status: 500 Internal Server Error, body: Failed to process log file: ...	Database Ingestor for the Director Proxy identity	The ADX database. Grant it to the proxy's managed identity, not to Director
Error while getting token : ClientSecretCredential authentication failed. ... AADSTS...	No role is missing. The service principal credentials are wrong	See the AADSTS entry below
DefaultAzureCredential failed to acquire a token	No role is missing. Managed Identity was used and is not available	See the Managed Identity entry below

Grant the role with a management command on the database, replacing the placeholders with the application (client) ID and the tenant ID of the identity Director uses:

.add database mydatabase ingestors ('aadapp=<client id>;<tenant id>')

The role can also be assigned from the database's Permissions page in the Azure Portal. In Managed Identity mode grant it to the identity of the Director host. In Director Proxy mode grant it to the proxy's identity. Director retries automatically, so no restart is needed once the role is in place.

"at least one table must be configured (either 'table' or 'tables')"

[Error] [director] [target-<target id>] [basic_adx] ValidateConfig failed for target "basic_adx": at least one table must be configured (either 'table' or 'tables')

Cause: the target has neither a root table nor a tables array. Tables are not discovered from the database. One of the two is required.

Fix: add a catch-all table with its schema, or at least one entry under tables. A configuration with only endpoint and database is rejected before the target starts.

Nothing is sent until the configuration is corrected. The check is repeated about every 30 seconds until it changes.

"invalid catch-all table configuration: schema is required for parquet format"

Failed to reinitialize target "basic_adx" (attempt 6). Reason: invalid catch-all table configuration: schema is required for parquet format

Cause: the default format is parquet, and Parquet and Avro files need a schema. The root table is set but the root schema is missing. The same rule applies to every entry under tables, where the message reads invalid table configuration: schema is required for parquet format. If an entry overrides format with avro or parquet while the root format is json or multijson, the message is schema not provided.

Fix: add a schema next to the table. It can be a Library name, a built-in name, or an inline definition such as TimeGenerated:datetime,Message:string. Set format: json instead if you do not want a schema.

The target does not start until the schema is added. Retries back off to once per minute and the attempt counter keeps growing. Data waits in the Director queue and is delivered once the target initializes.

"unsupported data format"

Cause: format is not one of json, multijson, avro, parquet. The check is case-sensitive, so JSON, Parquet, jsonl, and csv are all rejected.

Fix: use one of the four values in lowercase. The target does not start until it is corrected, and data waits in the queue.

"maximum of 10 tables allowed"

Cause: the target defines more than ten tables. The catch-all table counts toward the limit, so ten entries under tables plus a catch-all is eleven and fails.

Fix: keep the total at ten or fewer, for example nine tables entries plus the catch-all, or split the tables across two targets that write to the same database. The related message invalid table configuration: table is required means a tables entry has a schema but no table key.

"endpoint is required", "invalid endpoint URL for Azure Data Explorer", or "database is required"

Failed to reinitialize target "basic_adx" (attempt 2). Reason: invalid endpoint URL for Azure Data Explorer

Cause: endpoint or database is empty, or endpoint is not a full https:// URL with a hostname. An http:// URL and a bare hostname such as mycluster.westeurope.kusto.windows.net are both rejected.

Fix: set endpoint to the full cluster URI, for example https://mycluster.westeurope.kusto.windows.net. The query URI and the https://ingest-... data ingestion URI are both accepted. Set database to the database name.

These errors are deterministic. The target does not start until they are corrected.

"Error while getting token" with an AADSTS code, or "ClientSecretCredential authentication failed"

Sender worker 1 Finalize failed on flush for target "basic_adx": Op(...): Kind(KInternal): Error while getting token : ClientSecretCredential authentication failed. POST https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/v2.0/token ... RESPONSE 401 Unauthorized ... "AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID ..."

Cause: the service principal credentials are wrong. The target initialized without error because no token is requested at startup. The first flush requests one and fails. Expect an execute() failed or Finalize failed on flush line about every 5 seconds while the connection status still shows connected. The AADSTS code tells you which field:

Code	Meaning	Check
AADSTS7000215	Invalid client secret	The secret ID was pasted instead of the secret value, or the secret was rotated
AADSTS7000222	Client secret expired	Create a new secret on the app registration and update client_secret
AADSTS700016	Application not found in the directory	client_id is wrong, or tenant_id is a different tenant than the one the app registration lives in
AADSTS90002	Tenant not found	tenant_id is not a valid tenant ID

Fix: correct the field the code points to. A tenant_id that is neither a GUID nor a valid domain name is the one credential error caught at startup, as Reason: invalid tenantID.

No data is lost while this persists, within the Director queue limits. Failed payloads are redelivered until the credentials work.

"failed to resolve client secret", "failed to resolve tenant ID", or "failed to resolve function token"

Failed to reinitialize target "basic_adx" (attempt 3). Reason: failed to resolve client secret: credential: store "mystore" not found in configuration

Cause: the field uses a ${VAR} or $secret{store=...,ref=...} reference that could not be resolved. The inner text names the reason: env variable "..." is not set, store "..." not found in configuration, no credentials configured, or an error returned by the vault provider.

Fix: export the variable for the Director service, or correct the store name and the reference in the $secret{...} token. The service principal fields are resolved even in Director Proxy mode, so an unresolvable client_secret blocks a proxy-forwarding target too.

The target does not start until the reference resolves.

"DefaultAzureCredential failed to acquire a token" or ManagedIdentityCredential errors

Cause: Director authenticated with Managed Identity. That is the intended mode when tenant_id, client_id, and client_secret are all omitted and Director runs on an Azure resource with a managed identity. It is also the silent fallback when only one or two of the three are set, for example when a key is misspelled as client-secret, or a $secret{...} reference resolves to an empty value. Nothing is logged about the fallback, so a half-configured service principal looks like a Managed Identity problem.

Fix: either fill in all three fields with the exact key names tenant_id, client_id, and client_secret, or run Director on an Azure resource with a managed identity and grant that identity Database Ingestor on the database. Managed Identity token requests go to the Azure instance metadata endpoint directly and bypass any configured proxy, which is expected.

Like the AADSTS errors above, this appears at the first flush, not at startup.

"error from Kusto endpoint" with 403 Forbidden

Sender worker 2 execute() failed for natsobj:/vm-pipeline-payloads/sender.basic_adx.<id>: Op(...): Kind(KHTTPError): error from Kusto endpoint, ...(403 Forbidden):

Cause: a token was issued, so the credentials are right, but the identity has no ingest permission on the database. The JSON body that follows typically carries the code Forbidden. When the batch is larger than 4 MB the same missing role appears as problem getting authorization context from Kusto via Mgmt: ...(403 Forbidden), because that path first asks the cluster for an ingestion token.

Fix: grant Database Ingestor on the database to the identity in use, with the .add database command shown above. Table Ingestor on each table the target writes to also works. In Director Proxy mode the identity is the proxy's and the error arrives as a 500 from the proxy instead.

Payloads are redelivered until the role is in place. Each failure also reinitializes the target, so reinitialized successfully lines between the errors do not mean the problem is solved.

"error from Kusto endpoint" with 400 Bad Request

Cause: the cluster rejected the file. Typical reasons, in order of likelihood:

1. Wrong table or database name. Kusto names are case-sensitive. mytable and MyTable are different tables, and a tables entry must match the Kusto table name exactly.
2. The file bytes do not match the declared format. This happens when a tables entry overrides format with a value different from the root format. The file is written in the entry's format, but the cluster is told the root format. Use one format per target.
3. Streaming ingestion is not enabled. Batches up to 4 MB are sent through streaming ingestion first. If the database or table has no streaming ingestion policy, the cluster typically answers 400 and the batch is not retried through queued ingestion.

Fix: correct the item that applies, or enable the streaming ingestion policy on the database or table. The rejected payload is redelivered until it is accepted, so a permanent mismatch keeps the same payload cycling every few seconds and reinitializing the target.

"hostname is currently not trusted"

Sender worker 1 Finalize failed on flush for target "basic_adx": could not validate endpoint: Op(): Kind(KClientArgs): Can't communicate with 'adx.example.internal' as this hostname is currently not trusted; please see https://aka.ms/kustotrustedendpoints.

Cause: the hostname in endpoint is not a known Azure Data Explorer domain, so the client refuses to send credentials to it. A custom DNS alias for the cluster and a typo in the domain both produce this.

Fix: use the cluster's own hostname, https://<cluster>.<region>.kusto.windows.net, or the equivalent sovereign-cloud hostname. Retried until fixed.

"target broken: file holder not found"

Sender worker 3 execute() failed for natsobj:/vm-pipeline-payloads/sender.basic_adx.<id>: target broken: file holder not found

Cause: a record carries a SystemS3 value that matches none of the tables entries, and no catch-all table is configured. The match is exact and case-sensitive, so Security_Events does not route to security_events.

Fix: add a root table with its schema as a catch-all, add the missing table to tables, or correct the value the pipeline writes to SystemS3.

This payload is redelivered about every 5 seconds until the configuration changes, and each attempt reinitializes the target. Other payloads keep flowing, so the symptom is a steady stream of these lines while most data still arrives.

"context deadline exceeded"

Sender worker 2 Finalize failed on flush for target "basic_adx": context deadline exceeded

Cause: uploading one table's batch took longer than timeout seconds, 30 by default. The limit applies to each batch upload, so large batches on a slow link are the usual trigger. In Director Proxy mode the same limit surfaces as HTTP request failed: ... context deadline exceeded (Client.Timeout exceeded while awaiting headers).

Fix: raise timeout, lower max_size so that each batch is smaller, or both. Retried until it succeeds.

"no such host", "i/o timeout", "could not upload file to any container", or TLS errors

Sender worker 1 execute() failed for natsobj:/vm-pipeline-payloads/sender.basic_adx.<id>: Op(...): Kind(KHTTPError): ... dial tcp: lookup mycluster.westeurope.kusto.windows.net: no such host

Cause: Director cannot reach the cluster or one of the services it depends on. no such host usually means a wrong cluster or region in endpoint. i/o timeout means the port is blocked. x509: certificate signed by unknown authority means TLS is intercepted. could not upload file to any container, could not upload file to any queue, and failed to fetch ingestion resources mean the cluster is reachable but its staging storage is not.

Fix: allow outbound HTTPS (443) from the Director host to:

• https://<cluster>.<region>.kusto.windows.net, the query endpoint, used for streaming ingestion
• https://ingest-<cluster>.<region>.kusto.windows.net, the data ingestion endpoint, used for queued ingestion
• the cluster's staging storage accounts under *.blob.core.windows.net and *.queue.core.windows.net. Their names are assigned by the cluster and are not configurable. Every batch larger than 4 MB is uploaded there, so with the default max_size of 32 MB most full batches take this path
• login.microsoftonline.com, for service principal authentication

If you use a proxy, set HTTPS_PROXY for the Director service. Only the system trust store is used and this target has no custom CA option, so a TLS-intercepting proxy needs its CA trusted on the Director host. All of these are retried until the connection succeeds.

The target is healthy but nothing arrives in Azure Data Explorer

Check these in order.

1. The batch was queued and failed inside the cluster. Batches larger than 4 MB, and smaller batches whose streaming attempt failed with a transient error, go through queued ingestion. Queued ingestion is accepted as soon as the file reaches the cluster's staging storage. Director counts the records as delivered at that point and logs no error, but the ingestion runs inside the cluster minutes later. A schema or column mismatch, a table name in the wrong case, a missing table-level permission, or a deleted table all fail there with no Director log line. Run .show ingestion failures on the cluster to see them, and enable the cluster's failed-ingestion diagnostic log for ongoing monitoring. While testing, keep batches under 4 MB with a small batch_size so streaming ingestion is used and the cluster reports the problem in the flush error.

2. A tables entry overrides format. The file is written in the entry's format but declared to the cluster in the root format. The mismatch is reported in the flush error when streaming ingestion is used, and fails silently when queued ingestion is used. Use one target per format.

3. debug.dont_send_logs is enabled. Records are processed but never buffered or uploaded, and the delivered counters do not move. The notice Log sending is disabled for this target (basic_adx). Logs will be processed by the pipeline but will not be sent to the target. is logged only when debug.status is also true. With debug.status: false the target is completely silent. Remove dont_send_logs.

4. The cluster is still batching. flush_immediately is the Kusto ingestion property that bypasses the cluster's own batching policy, not the Director flush setting. With flush_immediately: false the cluster applies that policy before the data becomes queryable, so allow for the delay.

5. The records went to another table. A record with a SystemS3 value is routed to the tables entry with that exact name, and any other record goes to the catch-all table. Query the catch-all table when records are missing from the table you expected.

Errors when using Director Proxy

When the target forwards through Director Proxy, Director only talks to the proxy, and the proxy's managed identity does the Azure Data Explorer work. In YAML this mode is the function_app and function_token pair. In the web interface it is the proxy entry selected in the target wizard, whose Proxy URL and Access Token are edited on the proxy's Configuration tab. Database Ingestor must be granted to the proxy's identity, not to Director. The service principal fields are ignored in this mode. See VirtualMetric Director Proxy for the deployment and Proxy for managing proxy entries.

Director appends its own path to the proxy address, so Proxy URL is the base address of the deployed proxy, for example https://myproxy.example.net. Errors returned by the proxy are logged as function returned status: <status>, body: <body>.

Error text	Cause	Fix
function returned status: 400 Bad Request, body: Function token is not valid	The token Director presents does not match the one the Director Proxy expects. A rotated token, and a proxy that requires a token while function_token is empty, both produce this	Update the proxy's Access Token on its Configuration tab, or function_token in YAML, to the token configured on the Director Proxy. Retried until fixed
function returned status: 500 Internal Server Error, body: Failed to process log file: ...	The Director Proxy reached the cluster and was refused. Most often its managed identity lacks Database Ingestor. The same 500 also covers a wrong table or database name, a format mismatch, and a disabled streaming ingestion policy	Grant Database Ingestor on the database to the proxy's identity. If that is already in place, read the error text after Failed to process log file: and check the proxy's own logs. Retried until fixed
record rejected by target: function returned status: 400 Bad Request, body: Data size exceeds the maximum allowed size of 104857600 bytes, followed by deterministic failure ... after 4 attempts — dropping (giving up)	One table's batch file is larger than the 100 MB the Director Proxy accepts per request. This happens when max_size is raised above 104857600	Keep max_size well under 100 MB. The default of 32 MB is safe. This batch is dropped after 4 deliveries and its data is lost, so correct max_size before enabling proxy forwarding
function returned status: 401 Unauthorized or 404 Not Found	Typically the Function App hosting the Director Proxy rejected the request before it reached the proxy itself, because of an authentication requirement on the app or a Proxy URL that is not the proxy's base address	Confirm Proxy URL (function_app in YAML) is the deployed proxy's https:// address with no path after the host, and review the app's authentication settings. Retried until fixed
HTTP request failed: ...	Director cannot reach the Director Proxy. The text that follows names the reason: no such host for DNS, connection refused or i/o timeout for a firewall, x509: certificate signed by unknown authority for TLS interception, proxyconnect tcp for an HTTP proxy, or context deadline exceeded when the proxy took longer than timeout seconds	Check DNS and firewall rules from the Director host, and the HTTPS_PROXY setting. Trust the intercepting CA on the host. For the timeout, raise timeout or lower max_size. Retried until fixed
failed to create HTTP request: ...	function_app contains characters that do not form a valid URL, such as a space	Correct Proxy URL or function_app. This is not caught at startup, so it appears at the first flush and is retried until fixed