Elastic Security

SIEM Platform Observability

Synopsis

The Elastic Security target sends security telemetry to Elastic Security (formerly Elastic SIEM) using the Elasticsearch Bulk API with security-optimized index patterns and ECS normalization. This target extends standard Elasticsearch configuration with security-specific index management and detection rule integration.

Schema

- name: <string>
  description: <string>
  type: elasticsecurity
  properties:
    version: <integer>
    index: <string>
    endpoints:
      - endpoint: <string>
        username: <string>
        password: <string>
    max_payload_size_kb: <integer>
    batch_size: <integer>
    timeout: <integer>
    use_compression: <boolean>
    write_action: <string>
    field_format: <string>
    pipeline: <string>
    filter_path: <string>
    insecure_skip_verify: <boolean>
    ca_name: <string>
    server_name: <string>
    cert_name: <string>
    key_name: <string>
    passphrase: <string>
    min_tls_version: <string>
    max_tls_version: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Base Target Fields

Field	Required	Default	Description
name	Y	-	Unique identifier for this target
description	N	-	Human-readable description
type	Y	-	Must be elasticsecurity
pipelines	N	-	Pipeline names to apply before sending
status	N	true	Enable/disable the target

Elastic Security Connection

Field	Required	Default	Description
version	N	auto	Elasticsearch major version (7, 8, or auto for automatic detection)
index	Y	-	Default security index name (e.g., logs-security.events-default)
endpoints	Y	-	Array of Elasticsearch endpoint configurations
endpoints[].endpoint	Y	-	Elasticsearch URL (e.g., https://elastic.example.com:9200)
endpoints[].username	N	-	Basic authentication username
endpoints[].password	N	-	Basic authentication password

Bulk API Configuration

Field	Required	Default	Description
max_payload_size_kb	N	4096	Maximum bulk request size in KB
batch_size	N	10000	Maximum number of events per batch
timeout	N	30	Connection timeout in seconds
use_compression	N	true	Enable GZIP compression
write_action	N	create	Bulk API action (index, create)

Security Platform Configuration

Field	Required	Default	Description
field_format	N	-	Data normalization format. See applicable Normalization section
pipeline	N	-	Elasticsearch ingest pipeline name
filter_path	N	errors,items.*.error,items.*._index,items.*.status	Response filter path

TLS

TLS is engaged by the endpoint URL scheme: an https:// endpoint enables it, an http:// endpoint does not. There is no status field, and these fields are top-level — they are not nested under a tls: block.

Field	Required	Default	Description
insecure_skip_verify	N	false	Skip server certificate verification. Use only for testing.
ca_name	N	-	CA bundle used to verify the server certificate. When unset, the host trust store is used.
server_name	N	-	SNI hostname override for the TLS handshake
cert_name	N*	-	Client certificate for mutual TLS
key_name	N*	-	Client private key for mutual TLS
passphrase	N	-	Passphrase for an encrypted private key
min_tls_version	N	tls1.2	Minimum accepted TLS version (tls1.0, tls1.1, tls1.2, tls1.3)
max_tls_version	N	tls1.3	Maximum accepted TLS version (tls1.0, tls1.1, tls1.2, tls1.3)

* = Mutual TLS requires both cert_name and key_name. Setting only one is a configuration error.

note

TLS material fields (cert_name, key_name, ca_name, client_ca_name) accept any of the following:

• File name — resolved relative to the service root directory. Nested paths such as certs/prod/server.pem are supported.
• Absolute path — honored only if it resolves inside the service root. Any path that escapes the root is refused.
• Inline PEM content — used verbatim when the value contains -----BEGIN.
• Environment variable — ${ENV_VAR}.
• Vault reference — $secret{id=...} or $secret{store=...,ref=...}.

Scheduling

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging for this target
debug.dont_send_logs	N	false	Log events without sending to Elastic Security

Details

Security Index Patterns

Recommended Index Patterns:

• logs-security.events-*: Security events and alerts
• logs-security.auth-*: Authentication and authorization logs
• logs-security.network-*: Network traffic and firewall logs
• logs-security.endpoint-*: Endpoint security events
• logs-security.threat-*: Threat intelligence indicators

Index Naming Convention:

• Format: logs-{dataset}-{namespace}
• Dataset: Security data category
• Namespace: Environment or team identifier (e.g., default, production)

Data Stream Compatibility

Use index patterns compatible with Elastic data streams for automatic index lifecycle management and rollover.

ECS Normalization

Elastic Common Schema (ECS):

• Apply field_format: ECS for automatic field normalization
• Ensures compatibility with Elastic Security detection rules
• Provides consistent field naming across data sources
• Enables pre-built dashboards and visualizations

Critical ECS Fields:

• @timestamp: Event timestamp (ISO 8601)
• event.category: Event categorization (authentication, network, process, file)
• event.type: Event type (start, end, info, error)
• source.ip / destination.ip: Network addresses
• user.name: Username for authentication events
• process.name: Process information for endpoint events

Detection Rule Integration

Rule Compatibility:

• Elastic Security detection rules require ECS-normalized data
• Custom rules can reference specific index patterns
• Pre-built rules work automatically with standard ECS fields

Index Configuration:

• Configure indices in Elastic Security � Management � Index Patterns
• Create index patterns matching your configured target indices
• Enable index patterns in Detection Rules

Performance Optimization

Batch Configuration:

• Larger batches reduce API calls and improve throughput
• Balance batch size against memory usage and latency
• Default batch_size: 10000 suitable for most workloads

Compression:

• GZIP compression enabled by default
• Reduces network bandwidth by 70-90%
• Minimal CPU overhead for compression/decompression

Load Balancing:

• Configure multiple endpoints for high availability
• Events distributed randomly across endpoints
• Automatic failover to available endpoints

Index Mapping

Ensure Elasticsearch index mappings match ECS field types. Incorrect mappings cause ingestion failures. Use index templates for automatic mapping configuration.

Examples

Basic Security Events

Sending security events to Elastic Security with ECS normalization...

targets: - name: elastic-security-events type: elasticsecurity properties: index: logs-security.events-default field_format: ECS endpoints: - endpoint: https://elastic.example.com:9200 username: "${ELASTIC_USERNAME}" password: "${ELASTIC_PASSWORD}"

Authentication Logs

Forwarding authentication logs to dedicated security index...

targets: - name: elastic-auth-logs type: elasticsecurity properties: index: logs-security.auth-production field_format: ECS use_compression: true endpoints: - endpoint: https://elastic.example.com:9200 username: "${ELASTIC_USERNAME}" password: "${ELASTIC_PASSWORD}"

High-Availability Configuration

Multiple Elasticsearch endpoints for high availability and load balancing...

targets: - name: elastic-security-ha type: elasticsecurity properties: index: logs-security.events-default field_format: ECS batch_size: 10000 max_payload_size_kb: 4096 use_compression: true endpoints: - endpoint: https://elastic-node1.example.com:9200 username: "${ELASTIC_USERNAME}" password: "${ELASTIC_PASSWORD}" - endpoint: https://elastic-node2.example.com:9200 username: "${ELASTIC_USERNAME}" password: "${ELASTIC_PASSWORD}" - endpoint: https://elastic-node3.example.com:9200 username: "${ELASTIC_USERNAME}" password: "${ELASTIC_PASSWORD}"

With Ingest Pipeline

Using Elasticsearch ingest pipeline for additional processing...

targets: - name: elastic-security-pipeline type: elasticsecurity properties: index: logs-security.network-default field_format: ECS pipeline: security-enrichment endpoints: - endpoint: https://elastic.example.com:9200 username: "${ELASTIC_USERNAME}" password: "${ELASTIC_PASSWORD}"

Threat Intelligence

Forwarding threat intelligence indicators to Elastic Security...

targets: - name: elastic-threat-intel type: elasticsecurity properties: index: logs-security.threat-default field_format: ECS write_action: create endpoints: - endpoint: https://elastic.example.com:9200 username: "${ELASTIC_USERNAME}" password: "${ELASTIC_PASSWORD}"

Endpoint Security Events

Sending endpoint detection events to Elastic Security...

targets: - name: elastic-endpoint-security type: elasticsecurity properties: index: logs-security.endpoint-default field_format: ECS batch_size: 5000 use_compression: true endpoints: - endpoint: https://elastic.example.com:9200 username: "${ELASTIC_USERNAME}" password: "${ELASTIC_PASSWORD}"

Production Configuration

Production-ready Elastic Security configuration with HA, compression, and ECS normalization...

targets: - name: elastic-security-production type: elasticsecurity properties: index: logs-security.events-production field_format: ECS batch_size: 10000 max_payload_size_kb: 4096 timeout: 30 use_compression: true write_action: create insecure_skip_verify: false endpoints: - endpoint: https://elastic-node1.example.com:9200 username: "${ELASTIC_USERNAME}" password: "${ELASTIC_PASSWORD}" - endpoint: https://elastic-node2.example.com:9200 username: "${ELASTIC_USERNAME}" password: "${ELASTIC_PASSWORD}"