Syslog

Forward

Synopsis

Creates a target that forwards log messages to a remote syslog server. Supports multiple transport protocols, message templating, and TLS encryption.

note

The default message template is <{{.Priority}}> {{.Timestamp}} {{.AppName}}[{{.PID}}]: {{.Message}}, which is neither strictly RFC 3164 nor RFC 5424. Set template to emit a specific format. The SIEM-specific syslog targets default to an RFC 3164 header instead.

tip

For details of the format, see Appendix.

Schema

- name: <string>
  description: <string>
  type: syslog
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    type: <string>
    address: <string>
    port: <numeric>
    field_format: <string>
    template: <string>
    application_name: <string>
    priority: <numeric>
    tls:
      status: <boolean>
      verify: <boolean>
      cert_name: <string>
      key_name: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Field	Required	Default	Description
name	Y		Target name
description	N	-	Optional description
type	Y		Must be syslog
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Connection

Field	Required	Default	Description
type	N	"udp"	Transport protocol: udp, tcp, or tls
address	N	"127.0.0.1"	Remote server address
port	N	514	Remote server port

note

In the GUI, the Port field is prefilled with 514 and switches to 6514 — the IANA syslog-tls port — as soon as type is set to tls or TLS Encryption is enabled. It switches back if TLS is turned off. A port you type yourself is never changed, and leaving it blank keeps the server-side default. This runs in the create wizard only: a saved target keeps its stored port when its transport changes. The prefill is a console convenience — the configuration default remains 514 regardless of transport.

Messages

Field	Required	Default	Description
template	N	"<{{.Priority}}> {{.Timestamp}} {{.AppName}}[{{.PID}}]: {{.Message}}"	Message template
application_name	N	"vmetric"	Application name in the messages
priority	N	14	Syslog priority value
field_format	N	-	Data normalization format. See applicable Normalization section

TLS

Field	Required	Default	Description
tls.status	N	false	Enable TLS encryption

TLS material is resolved through the shared client builder, so these keys mean the same thing on every target that uses it. They are nested under a tls: block.

Field	Required	Default	Description
tls.verify	N	true	Verify the server certificate. On by default — set it to false only to accept an untrusted certificate, and only where you control the network path
tls.server_name	N	-	SNI hostname override for the handshake. Use it when the certificate's name does not match the address you connect to
tls.ca_name	N	-	CA bundle used to verify the server certificate. When unset the host trust store is used; when set it replaces the host trust store rather than adding to it
tls.cert_name	N*	-	Client certificate, for mutual TLS
tls.key_name	N*	-	Client private key, for mutual TLS
tls.passphrase	N	-	Passphrase for an encrypted client private key
tls.min_tls_version	N	tls1.2	Lowest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)
tls.max_tls_version	N	tls1.3	Highest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)

* cert_name and key_name are individually optional but must be supplied together — a certificate without its key, or a key without its certificate, is a configuration error.

warning

When using TLS, ensure that the certificate files are accessible and have the appropriate permissions.

Templates

The following variables can be used in the message template:

Variable	Description	Example
{{.Year}}	Current year	2024
{{.Month}}	Current month	01
{{.Day}}	Current day	15
{{.Priority}}	Syslog priority	14
{{.Timestamp}}	RFC3339 timestamp	2024-01-15T14:30:00Z
{{.AppName}}	Application name	vmetric
{{.PID}}	Process ID	1234
{{.Message}}	Log message	User login successful

note

Invalid templates will fall back to sending the raw message content without formatting.

Scheduling

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging
debug.dont_send_logs	N	false	Process logs but don't send to target (testing)

Field Normalization

Field normalization helps standardize log data before sending it to the syslog server, ensuring consistent data formats:

• ecs - Elastic Common Schema
• cim - Common Information Model
• asim - Advanced Security Information Model

Examples

Basic UDP

The minimum configuration for a UDP syslog:

targets:
  - name: udp_syslog
    type: syslog
    properties:
      address: "192.168.1.100"

Secure TLS

Configuration for a TLS-encrypted syslog:

targets:
  - name: secure_syslog
    type: syslog
    properties:
      type: "tls"
      address: "logs.example.com"
      port: 6514
      tls:
        status: true
        verify: true
        cert_name: "client-cert.pem"
        key_name: "client-key.pem"

Template

Configuration with a custom message template:

targets:
  - name: custom_syslog
    type: syslog
    properties:
      address: "192.168.1.100"
      template: "{{.Timestamp}} [{{.AppName}}] severity={{.Priority}} {{.Message}}"
      application_name: "myapp"
      priority: 13

With Field Normalization

Using field normalization for standard format:

targets:
  - name: normalized_syslog
    type: syslog
    properties:
      address: "192.168.1.100"
      field_format: "cim"

Troubleshooting

This section covers the errors you are most likely to see with the syslog target, what causes each one, and how to fix it.

Where to look:

• Director logs. Target errors are tagged with the target name and carry "Section":"SenderPool". The part after Reason:, or after the last colon, is the actual cause.
• The target's connection status in the web interface. It shows the same reason as the log line, prefixed with connection failed for <target name>:.

See Target Delivery Errors for how Director logs and retries target failures.

What the receiver needs

Check these first. The transport you choose decides which ones apply.

Prerequisite	When it applies	Error when it is missing
A receiver listening on address and port over the configured transport	tcp and tls	connection refused at the end of a dial tcp ... line
A network path from the Director host to address and port	always	i/o timeout after 30 seconds. With udp, no error at all
DNS resolution of the host in address, from the Director host	address is a hostname	no such host at the end of a dial tcp: lookup ... line
Room for one connection per sender worker, plus one idle connection	tcp and tls	failed to create per-worker syslog client: followed by the dial error
The receiver's CA in the trust store of the Director host	type: tls while tls.status stays false	tls: failed to verify certificate: x509: certificate signed by unknown authority
A readable client certificate and key under the service root	every time tls.status: true, without exception	client certificate: read certificate ...
A readable CA bundle under the service root	tls.status: true with tls.ca_name set	ca_name: ... could not be resolved
The passphrase for an encrypted client key	the client key is encrypted	client certificate: decrypt PKCS#8 private key (wrong passphrase?):
A client certificate issued by a CA the receiver trusts	the receiver demands mutual TLS	typically remote error: tls: bad certificate

warning

tls.status: true always loads a client certificate. tls.cert_name and tls.key_name default to cert.pem and key.pem in the service root, and neither may be left empty. Turning on tls.status without a certificate pair of your own is the most common way to break this target. For one-way TLS against the host trust store, set type: "tls" and leave tls.status at false. In that mode tls.verify, tls.ca_name and tls.server_name are ignored, so an untrusted CA can only be cured by installing it on the Director host.

"client certificate: read certificate ..." right after enabling TLS

[Error] [director] [target-<target id>] [secure_syslog] Failed to reinitialize target "secure_syslog" (attempt 1). Reason: client certificate: read certificate "<service root>/cert.pem": open <service root>/cert.pem: no such file or directory

Cause: tls.status: true is set and the client certificate pair cannot be read. A client certificate is loaded every time tls.status is on. When you do not name one, the defaults cert.pem and key.pem are looked for in the service root, and they are usually not there. You see the same error when the file you named is missing, sits outside the service root, or cannot be read by the account the Director service runs under.

Fix: choose one of two modes.

• One-way TLS. Set type: "tls" and leave tls.status at false. The receiver's certificate is checked against the host trust store, and no client certificate is loaded.
• Mutual TLS. Keep tls.status: true, and set tls.cert_name and tls.key_name to a certificate and key that exist under the service root. Paths are resolved relative to the service root. An absolute path outside it is refused with is not inline PEM and does not resolve to a path under the service root.

Nothing is sent while this lasts. The target never starts, incoming data waits in the Director queue, and delivery resumes on the next attempt once the files are readable.

"connection refused" when the target dials the receiver

[Error] [director] [target-<target id>] [custom_syslog] Failed to reinitialize target "custom_syslog" (attempt 3). Reason: dial tcp syslog.example.com:514: connect: connection refused

Cause: nothing is accepting connections on that port. The receiver is stopped, listens on a different port, or is bound to an address that is not reachable from the Director host. Only tcp and tls report this. A udp target never contacts the receiver while starting, so it stays silent.

Fix: confirm the receiver is running and listening on the value in port, and that the firewall between the two hosts allows the connection. Receivers commonly listen on 514 for plain TCP and 6514 for TLS. A receiver bound to 127.0.0.1 accepts nothing from another host. The wording after the last colon comes from the operating system, and on Windows it typically reads connectex: No connection could be made because the target machine actively refused it.

No data is lost. Retries back off from 5 seconds to once a minute and continue until the receiver answers, so the attempt counter climbs for as long as the cause persists. Queued data is delivered afterwards.

"no such host" for the configured address

[Error] [director] [target-<target id>] [custom_syslog] Failed to reinitialize target "custom_syslog" (attempt 12). Reason: dial tcp: lookup syslog.example.com: no such host

Cause: the hostname in address does not resolve from the Director host. Usually a typo, a name that only exists in an internal zone the host does not query, or a missing search domain. The name is resolved on every connection attempt, so udp fails here too.

Fix: resolve the name from the Director host itself, then correct address or use the IP address instead. The wording after the last colon comes from the operating system, and on Windows it typically reads getaddrinfow: No such host is known.

No data is lost. The target retries until the name resolves.

"i/o timeout" while connecting or while sending

[Error] [director] [target-<target id>] [custom_syslog] Failed to reinitialize target "custom_syslog" (attempt 4). Reason: dial tcp syslog.example.com:514: i/o timeout

Cause: traffic to the receiver is discarded rather than refused, which is what a firewall that drops packets looks like. Connecting, including the TLS handshake, is allowed 30 seconds. Sending one event is allowed 30 seconds of its own, and a receiver that has stopped reading produces the same wording on a write tcp ... line.

Fix: open the path from the Director host to the receiver's address and port for the transport you configured. A timeout while connecting points at the network. A timeout while sending points at a receiver that still accepts the connection but no longer reads from it, so check the receiver's own queues and disk.

No data is lost either way. A connect timeout leaves everything queued. A send timeout fails that payload, which is redelivered about every 5 seconds.

"target broken: write tcp ..." after the receiver restarts

[Error] [director] [target-<target id>] [custom_syslog] Sender worker 3 execute() failed for custom_syslog: target broken: write tcp syslog.example.com:514: write: broken pipe

Cause: the open connection is gone. The receiver restarted, an idle timeout on the receiver or on a stateful firewall closed the session, or the path dropped. Only tcp and tls produce this.

Fix: none, if the receiver is back. Director closes the sockets and reconnects every worker on its own. If the reconnect then fails, read the new reason and use the entries above. Repeated drops on a quiet link usually mean a session timeout somewhere in the path, so raise that timeout or keep a low-rate event flowing. The wording after the last colon comes from the operating system, and on Windows it typically reads wsasend: An existing connection was forcibly closed by the remote host.

The failed payload is not lost. It is redelivered about every 5 seconds. Events already handed to the socket when the receiver went away are typically lost without an error, because the local write is accepted before the break is known, so expect a small gap around every receiver restart.

Events run together on one line at the receiver

Symptom: a tcp or tls receiver shows long lines holding several events, or rejects them as unparseable, while Director reports no error at all.

Cause: nothing is inserted between events on a stream transport. Each rendered message is written exactly as the template produced it. Unless that message ends in a newline, the next event starts on the same line. Receivers that split on newlines see one very long line. Receivers that expect a length prefix cannot parse it at all. udp is unaffected, because each event is its own datagram.

Fix: end template with a newline. Use a double-quoted YAML scalar, so that \n is read as an escape:

template: "<{{.Priority}}> {{.Timestamp}} {{.AppName}}[{{.PID}}]: {{.Message}}\n"

warning

An event longer than 2048 bytes is cut to fit, and the trailing newline is part of what is cut. Keep events well under 2048 bytes, or the framing disappears on exactly the events that need it most.

Nothing is lost on the way out. What happens next is up to the receiver. Some store the joined line, others discard it.

The target looks healthy but nothing reaches the receiver

Cause and fix: work through this list in order.

1. The transport is udp. This is the default. A datagram is handed to the operating system and is never acknowledged, so a stopped receiver, a closed port, a dropped packet and a wrong address all look exactly like a successful delivery. Confirm with a packet capture on the receiver, and use tcp or tls wherever delivery matters.
2. debug.dont_send_logs is on. Events are processed and then discarded. No counter moves and no error is logged. With debug.status also on, startup logs Log sending is disabled for this target. Remove the setting.
3. The receiver is bound to another address. A receiver listening only on 127.0.0.1, or on a single interface, accepts nothing from another host. With tcp and tls you see a refused connection. With udp you see nothing.
4. Events are running together. See the entry above. A receiver that frames on newlines may discard the joined line.
5. The device runs on the direct fast path. A device whose route leads to this one target, with no extra processing, is driven straight from the listener. If the receiver is unreachable when an inbound connection opens, records on that connection are discarded until the source reconnects, and the logs show ThreadSafeInit failed for with the target name. Set disable_direct: true at service level to send every device through the queue instead. See Service Configuration.

Cases 2 and 5 discard data on purpose. Case 1 loses it silently in the network. Cases 3 and 4 leave it for the receiver to reject.

The data arrives but does not look right

Symptom	Cause	Fix
Events end in ... and are exactly 2048 bytes long	An event longer than 2048 bytes, header included, is cut to fit. This target has no message size setting	Shorten the event upstream in a pipeline
Multi-line events arrive on one line, with spaces where the line breaks were	Carriage returns and line feeds inside a message are replaced with spaces before sending, so that a message cannot forge a second record	Expected behavior. Reformat multi-line content upstream if the receiver needs it
Events carry no syslog header, and no error is logged	template could not be parsed or rendered, so the message is sent on its own	Correct template. An unbalanced {{ and an unknown field are the usual causes
The application name stays vmetric although you set it	The field is application_name. A key the target does not recognize is ignored, and the default stays in place	Use application_name
field_format is set, but some records are not normalized	A record that cannot be mapped is forwarded unchanged rather than failed	Check the mapping against the source fields with the pipeline debugger
The sent counter is far below the number of events the receiver got	This target counts one event per delivered payload, not one per event	Judge delivery from the receiver, not from this counter

None of these lose data. The events arrive, but not in the shape you expect.

"x509: certificate signed by unknown authority" and other handshake errors

[Error] [director] [target-<target id>] [secure_syslog] Failed to reinitialize target "secure_syslog" (attempt 2). Reason: tls: failed to verify certificate: x509: certificate signed by unknown authority

Cause and fix: the text after x509: names the check that failed.

Error text	Cause	Fix
certificate signed by unknown authority	The receiver's certificate comes from a CA the Director host does not trust	Install the CA on the Director host, or set tls.status: true with tls.ca_name pointing at the CA bundle
cannot validate certificate for ... because it doesn't contain any IP SANs	address holds an IP address, and the certificate carries only DNS names	Put the DNS name in address, or set tls.server_name to a name on the certificate
certificate is valid for ..., not ...	The name in address is not on the receiver's certificate	Use a name the certificate carries, or set tls.server_name
certificate has expired or is not yet valid	The certificate is outside its validity window, or the two clocks disagree	Renew the certificate, and check the clock on both hosts
tls: first record does not look like a TLS handshake	TLS is pointed at a plaintext port, commonly 514	Use the receiver's TLS port, commonly 6514, or switch to type: "tcp"

When the receiver rejects your client certificate, the refusal comes back from the receiver and typically reads remote error: tls: bad certificate or remote error: tls: unknown certificate authority. Issue the client certificate from a CA the receiver trusts. A receiver that requires a client certificate while none is configured typically answers remote error: tls: certificate required, or closes the connection with EOF.

No data is lost during any of these. The target retries until the handshake succeeds.

"unknown connection type" and other rejected settings

[Error] [director] [target-<target id>] [custom_syslog] ValidateConfig failed for target "custom_syslog": unknown connection type

Cause and fix: the target refuses to start until the setting is corrected.

Error text	Cause	Fix
unknown connection type	type is not exactly udp, tcp or tls. The value is case-sensitive, so TCP is rejected	Use the lowercase form
invalid address	address is present but empty	Set it, or remove the key to accept the default 127.0.0.1
port must be between 1 and 65535, got 0	port is outside the valid range	Use the receiver's listening port
invalid tls configuration	tls.status: true with an empty tls.cert_name or tls.key_name	Supply both, or use type: "tls" with tls.status at false
min TLS version ... is greater than max TLS version ...	tls.min_tls_version is above tls.max_tls_version	Correct the range. Accepted values run from tls1.0 to tls1.3, and a value outside that set falls back to the default without a warning
credential: env variable ... is not set	A ${VAR} or $secret{...} reference in a tls.* field could not be resolved	Define the variable for the account the Director service runs under, or correct the secret store name

Nothing is sent while the configuration is rejected, and nothing is lost. The configuration is re-read about every 30 seconds, so correcting it is enough. No restart is needed. One setting is accepted but still surprises: tls.status: true makes the transport TLS over TCP even when type is udp, so set type: "tls" and use the receiver's TLS port.