NATS

NATS Message Queue

Synopsis

Creates a target that publishes log messages to NATS JetStream subjects with support for batching, authentication, TLS encryption, and automatic retry logic. NATS is a high-performance messaging system for cloud-native applications.

Schema

- name: <string>
  description: <string>
  type: nats
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    url: <string>
    subject: <string>
    username: <string>
    password: <string>
    timeout: <integer>
    batch_size: <integer>
    max_retries: <integer>
    retry_delay: <integer>
    field_format: <string>
    tls:
      status: <boolean>
      verify: <boolean>
      cert_name: <string>
      key_name: <string>
      min_tls_version: <string>
      max_tls_version: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Field	Required	Default	Description
name	Y		Target name
description	N	-	Optional description
type	Y		Must be nats
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Connection

Field	Required	Default	Description
url	Y	-	NATS server URL (e.g., nats://localhost:4222)
subject	Y	-	NATS subject name for message publishing
username	N	-	NATS username for authentication
password	N	-	NATS password for authentication
timeout	N	30	Connection timeout in seconds

Batch Configuration

Field	Required	Default	Description
batch_size	N	1000	Number of messages to batch before publishing (minimum 1)
max_retries	N	3	Maximum retry attempts for failed publish operations
retry_delay	N	1	Delay between retry attempts in seconds

Processing

Field	Required	Default	Description
field_format	N	-	Data normalization format. See applicable Normalization section

TLS Configuration

Field	Required	Default	Description
tls.status	N	false	Enable TLS encryption

TLS material is resolved through the shared client builder, so these keys mean the same thing on every target that uses it. They are nested under a tls: block.

Field	Required	Default	Description
tls.verify	N	true	Verify the server certificate. On by default — set it to false only to accept an untrusted certificate, and only where you control the network path
tls.server_name	N	-	SNI hostname override for the handshake. Use it when the certificate's name does not match the address you connect to
tls.ca_name	N	-	CA bundle used to verify the server certificate. When unset the host trust store is used; when set it replaces the host trust store rather than adding to it
tls.cert_name	N*	-	Client certificate, for mutual TLS
tls.key_name	N*	-	Client private key, for mutual TLS
tls.passphrase	N	-	Passphrase for an encrypted client private key
tls.min_tls_version	N	tls1.2	Lowest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)
tls.max_tls_version	N	tls1.3	Highest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)

* cert_name and key_name are individually optional but must be supplied together — a certificate without its key, or a key without its certificate, is a configuration error.

Scheduling

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging
debug.dont_send_logs	N	false	Process logs but don't send to target (testing)

Details

The NATS target uses NATS JetStream to publish log messages with guaranteed delivery semantics. Messages are accumulated in batches and published when the batch size is reached. Each message is published with automatic retry logic for handling transient failures.

The target maintains a persistent connection to the NATS server with automatic reconnection capabilities. Connection options include configurable timeouts, maximum reconnect attempts, and authentication.

NATS JetStream provides stream-based messaging with persistence, acknowledgment, and replay capabilities, making it suitable for reliable log delivery in distributed systems.

Prerequisites

1. A running NATS server with JetStream enabled
2. Network connectivity to the NATS server
3. Valid authentication credentials if the server requires authentication
4. TLS certificates if using encrypted connections

note

The NATS target requires JetStream to be enabled on the NATS server. Core NATS messaging without JetStream is not supported.

warning

Both tls.cert_name and tls.key_name must be provided together when using client certificate authentication. Providing only one will result in a configuration error.

Examples

Basic

Minimum configuration for publishing to NATS JetStream:

targets:
  - name: basic_nats
    type: nats
    properties:
      url: "nats://localhost:4222"
      subject: "logs.system"

With Authentication

Configuration with username/password authentication:

targets:
  - name: authenticated_nats
    type: nats
    properties:
      url: "nats://nats.example.com:4222"
      subject: "logs.application"
      username: "logger"
      password: "secure_password"

With TLS

Configuration with TLS encryption and certificate verification:

targets:
  - name: secure_nats
    type: nats
    properties:
      url: "nats://nats.example.com:4222"
      subject: "logs.secure"
      username: "logger"
      password: "secure_password"
      tls:
        status: true
        verify: true
        min_tls_version: "tls1.2"

With Client Certificate

Configuration using mutual TLS with client certificates:

targets:
  - name: mtls_nats
    type: nats
    properties:
      url: "nats://nats.example.com:4222"
      subject: "logs.secure"
      tls:
        status: true
        verify: true
        cert_name: "client.pem"
        key_name: "client-key.pem"
        min_tls_version: "tls1.2"

Custom Batch Size

Configuration with custom batching and retry settings:

targets:
  - name: batch_nats
    type: nats
    properties:
      url: "nats://nats.example.com:4222"
      subject: "logs.highvolume"
      batch_size: 5000
      max_retries: 5
      retry_delay: 2
      timeout: 60

With Normalization

Configuration using field normalization:

targets:
  - name: normalized_nats
    type: nats
    properties:
      url: "nats://nats.example.com:4222"
      subject: "logs.ecs"
      field_format: "ecs"

With Pipeline

Using a pipeline for additional log processing:

targets:
  - name: pipeline_nats
    type: nats
    pipelines:
      - enrich_logs
    properties:
      url: "nats://nats.example.com:4222"
      subject: "logs.enriched"

Troubleshooting

This section covers the errors you are most likely to see with the nats target, what causes each one, and how to fix it. Everything here applies to the synadia target as well, because Synadia Cloud is reached over the same connection and the same JetStream publish path.

Where to look:

• Director logs. Target errors are tagged with the target name and carry "Section":"SenderPool". The part after Reason: or after the last colon is the actual cause.
• The target's connection status in the web interface. It shows the same reason as the log line, prefixed with connection failed for <target name>:. A connected status only proves that the server accepted the login. The subject, the stream behind it and your publish permission are first tested when the target flushes, so read the log as well.

See Target Delivery Errors for how Director logs and retries target failures.

Which permission or setting is missing?

Match the error against this table first. These are NATS authorization rules on the server or in your Synadia Cloud account, not settings in Director.

Error text	What is missing	Where to fix it
nats: Authorization Violation	A user the server accepts, together with its password	The server's authorization block, or the user list of your Synadia Cloud account
nats: no response from stream	A JetStream stream whose subjects cover subject, on a server or account where JetStream is enabled	The stream configuration
nats: timeout	Usually publish permission on subject for the user that connected	The permissions of that user
nats: maximum payload exceeded	Room within the server's maximum payload size for one record	Server configuration
nats: message size exceeds maximum allowed	Room within the stream's maximum message size	The stream configuration
nats: maximum bytes exceeded, nats: maximum messages exceeded	Room within the stream's storage limits	The stream configuration, or the account quota

The target only publishes. It never creates, lists or inspects streams, so no management right is needed, and a stream that does not exist is never created for you.

warning

Only a username and a password are sent to the server. NKey seeds, user JWTs and credentials files are not accepted in username or password. A credential of that kind pasted into password is sent as a plain password and refused. A token can be carried in url instead, as nats://TOKEN@nats.example.com:4222. A credential carried that way is not echoed back: the destination recorded in the target's statistics has it removed, and reads nats://nats.example.com:4222/<subject>.

"nats: Authorization Violation"

[Error] [director] [target-<target id>] [basic_nats] Failed to reinitialize target "basic_nats" (attempt 6). Reason: failed to connect to NATS: nats: Authorization Violation

Cause: the server refused the login. Four configurations produce it.

1. username and password are both set, but the pair is wrong or the user no longer exists.
2. Only one of the two is set. Nothing is then sent as a credential and the connection is attempted anonymously, which a server that requires authentication refuses.
3. An NKey seed or a user JWT was placed in password. Only user name and password authentication exists, so the seed or the token is sent as a plain password and the server rejects it.
4. A ${VAR} reference in username or password is not set in the environment of the Director service, so its literal text is sent as the value.

Fix: set both username and password to a pair the server accepts, or put a token in url as nats://TOKEN@nats.example.com:4222 — the token is stripped from the destination shown in the target's statistics. For Synadia Cloud, create a user with a password in the account and use those two values. Do not paste an NKey seed or a JWT into either field.

A reason that instead typically says the user or the account authentication has expired means the credential was valid but has timed out. Renew it in the account. A reason that typically says the maximum number of active account connections has been exceeded means the account has no free connection slot left. Each target holds one connection, so count your targets against the account limit.

Nothing is lost. Data waits in the Director queue while the connection is retried with a growing backoff, and the attempt counter climbs until the cause is fixed. No restart is needed.

"nats: no response from stream"

Sender worker 2 execute() failed for basic_nats/<payload id>: target broken: failed to finalize target cache: published 0/1000 messages, 1000 failed, last error: nats: no response from stream

Cause: the publish left Director but no JetStream stream answered it. Either no stream's subject list covers subject, or JetStream is not enabled on the server or for the account. Nothing on the server side answers in either case, so no explanation comes back with the error.

Fix: create a stream whose subjects include subject, or add the subject to an existing stream. Wildcards count, so a stream on logs.> covers logs.application. Check the subject for a typo, and check that you are pointed at the server that holds the stream. On Synadia Cloud, confirm that JetStream is enabled for the account. On a self-hosted server, confirm that JetStream is turned on and that the account is allowed to use it.

The batch is retried until the stream exists. Nothing is dropped.

"nats: timeout"

Sender worker 0 execute() failed for basic_nats/<payload id>: target broken: failed to finalize target cache: published 0/1000 messages, 1000 failed, last error: nats: timeout

Cause: the publish was sent but no acknowledgement arrived within five seconds. Two very different situations look identical here.

• The user is not allowed to publish to the subject. The server refuses the message and never acknowledges it, so Director only sees the wait run out. The real reason is recorded in the NATS server log as a permissions violation naming the subject. This is the more common of the two.
• The stream did not answer in time. It exists and the permission is in place, but the stream or its cluster is overloaded, is electing a new leader, or the connection is re-establishing underneath.

Fix: check the publish permission first. Grant publish on subject to the user in username, then watch whether the next flush succeeds. If the permission is already there, check the stream and the cluster in the NATS server log or in the Synadia Cloud console.

The batch is retried until it is accepted. Nothing is dropped.

A flush can occupy a worker for a long time before anything is logged

Symptom: after a configuration change, nothing arrives on the stream, the queue grows, the connection status stays green, and no error appears in the log for an hour or more. Then a single published 0/1000 messages line shows up.

Cause: every record in the batch is published and retried on its own, and the combined error is only logged once the whole batch has been tried. When the cause is the same for every record, a subject no stream covers or a denied publish permission, that wait is paid a thousand times over. With the default batch_size of 1000, a missing stream takes about an hour and a half, and a denied publish permission several hours, because each record waits five seconds for an acknowledgement that never comes.

Fix: while you are diagnosing, set batch_size to a small value such as 10. The same error then appears within seconds, and you can put the value back once the cause is fixed. Confirm the stream and the publish permission before you point a busy route at a new subject.

A Director shutdown or a configuration reload interrupts the flush and logs Finalize failed on flush with a count of the records it did not reach. That line is expected during a reload and means nothing is wrong.

"failed to connect to NATS" and the server is not reached

Failed to reinitialize target "basic_nats" (attempt 3). Reason: failed to connect to NATS: nats: no servers available for connection

Cause: the connection never opened. The reason names which step failed.

• nats: no servers available for connection means the host was reached but nothing accepted the connection on that port. The server is down, or the port is wrong. When url carries no port, 4222 is used.
• A reason that typically ends in a dial timeout means the packets are dropped rather than refused, which is usually a firewall or a routing problem. Raising timeout helps only when the server is merely slow to answer.
• A reason that typically says the host name could not be found means url is misspelled, or the name does not resolve from the Director host. An unset ${VAR} inside url is passed through literally and fails here too, typically as a parse error naming the malformed address.
• nats: protocol exception, INFO not received means the port answers but does not speak the NATS client protocol. The monitoring port and an HTTP proxy both look like this. Point url at the client port.

Fix: confirm the host and port in url, that the server is running, and that outbound TCP to that port is allowed from the Director host. Data waits in the queue while the connection is retried with a growing backoff.

"nats: secure connection not available"

Cause: tls.status is true but the server does not offer TLS on that port.

Fix: enable TLS on the server, or set tls.status: false for a plain connection.

The opposite case is silent. With tls.status: false against a server that requires TLS, the connection is upgraded anyway using the operating system trust store, and tls.verify, tls.ca_name, tls.server_name and the version range are all ignored. Set tls.status: true whenever the server uses TLS, so the settings you configured are the ones in force.

"nats: tls error" when the certificate is not trusted

Cause: the TLS handshake failed. The wording after nats: tls error: comes from the TLS library and names the reason.

• A reason that typically says the certificate was signed by an unknown authority means the server presents a certificate from a private certificate authority that the Director host does not trust.
• A reason that typically says the certificate is valid for one name but not the one you used means url points at an IP address or an alias that is not in the certificate.
• A reason that typically says a certificate is required, or that the certificate was rejected, means the server asks for client certificates and none was sent, or the one sent is not trusted there.

Fix: for a private certificate authority, set tls.status: true and put the CA certificate in tls.ca_name. For a name mismatch, set tls.server_name to the host name on the certificate. For a server that demands client certificates, set tls.cert_name and tls.key_name. Turning tls.verify off removes the check entirely and suits testing only.

The connection is retried with a growing backoff. Data waits in the queue.

"failed to load TLS certificate"

Cause: the certificate material could not be read or assembled, before the connection was even attempted. The rest of the reason names the field.

Reason contains	Meaning
could not be resolved (env/vault token, inline PEM, or a path under the service root)	The file is missing, or it sits outside the service root. Certificate material may be a path under the service root, inline PEM text, ${ENV} or $secret{...}
contains no valid PEM certificate(s)	The file is not PEM. Convert it
is not inline PEM and does not resolve to a path under the service root	The client certificate or key path is outside the service root. Move the files under it
read certificate, read key	The file exists but the Director service cannot read it. Fix the file permissions
wrong passphrase?	tls.passphrase is wrong or missing for an encrypted key
env variable ... is not set	A ${VAR} used in a TLS field is not set for the Director service
min TLS version ... is greater than max TLS version	tls.min_tls_version is above tls.max_tls_version

Fix: correct the field named in the reason. Note that an unrecognized version string, such as TLSv1.2, is replaced by the default without a message, so use the documented tls1.0 to tls1.3 spelling.

Nothing is sent while this persists, and the attempt counter climbs until the material loads.

"nats: maximum payload exceeded" or "nats: message size exceeds maximum allowed"

Sender worker 1 Finalize failed on flush for target "basic_nats": published 999/1000 messages, 1 failed, last error: nats: maximum payload exceeded

Cause: one record is larger than a limit. nats: maximum payload exceeded is the server's maximum payload for a single message, one mebibyte by default. nats: message size exceeds maximum allowed is the stream's own maximum message size. Records are never split.

Fix: shrink the record in a pipeline before it reaches the target, raise the server's maximum payload, or raise the stream's maximum message size. nats: maximum bytes exceeded and nats: maximum messages exceeded in the same position mean the stream is full and set to reject new messages, so raise its limits or add a consumer that drains it.

The oversized record is retried and never dropped, so the flush cannot complete while it is present. As the example line shows, the other 999 records of that batch were published, and they are published again on every redelivery.

"nats: connection closed"

Cause: the connection dropped while a flush was in progress and did not come back. The client re-establishes it max_retries times, waiting retry_delay seconds between attempts, and closes the connection when those are used up. With the defaults, a three second interruption is enough. Revoked credentials end the same way, because the new connection is rejected.

Fix: find the server-side cause, a restart, a network interruption or a revoked user. Raising max_retries and retry_delay widens the window the client tolerates, at the cost of longer publish retries, because both settings govern publishing as well.

Director re-establishes the connection on the next attempt by itself, and the batch is redelivered.

"nats: invalid subject"

Cause: subject contains a space, a tab or a line break. This is not caught when the configuration is saved, so it fails at every publish instead.

Fix: remove the whitespace. Every attempt fails instantly until you do, and nothing reaches the stream.

"ValidateConfig failed" for this target

Reason	Fix
url is required for nats target	Set url, for example nats://nats.example.com:4222
subject is required for nats target	Set subject to a subject a stream covers
both tls.cert_name and tls.key_name must be provided together or omitted together	Provide both, or neither
batch_size must be greater than 0	Remove the setting to use the default of 1000, or set a positive value. 0 is read as unset

Nothing is sent while the configuration is rejected, and the check repeats until the configuration changes.

Records appear more than once on the stream

Cause: a flush that fails partway is redelivered whole. The records it had already published are published again, because Director sets no message ID on them, so the stream's duplicate window cannot recognize them. A single oversized record is enough to repeat the rest of its batch on every attempt.

Fix: clear the failing error quickly, and expect consumers to tolerate duplicates. A smaller batch_size reduces how many records repeat per failure.

The target is healthy but nothing arrives on the stream

Check these in order.

1. debug.dont_send_logs is enabled. Records are processed and then discarded before they are buffered. Nothing is published, the target reports healthy, and no counter moves. Nothing is logged about it either, unless debug.status is enabled as well. Remove the flag after testing.

2. The flush has not finished yet. A large batch against a missing stream or a denied publish permission can occupy the worker for hours before a single line is logged. See the entry above and lower batch_size while you check.

3. The stream is discarding the oldest messages. A stream at its message or byte limit that is set to discard old messages accepts every publish and quietly evicts earlier ones. Director counts each record as delivered. Add a consumer, raise the limits, or switch the stream to rejecting new messages so the condition becomes visible as nats: maximum bytes exceeded.

4. Only one of username and password is set. The connection is then anonymous. On a server that allows anonymous connections it succeeds and looks healthy, but the anonymous user usually has no publish permission, which surfaces later as nats: timeout.