Concat

Parse Elastic Compatible

Synopsis

Concatenates values from multiple source fields into a single target field using a specified separator.

Schema

- concat:
    sources: <string[]>
    target: <ident>
    separator: <string>
    description: <text>
    if: <script>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>

Configuration

The following fields are used to define the processor:

Field	Required	Default	Description
sources	Y		Array of field names to concatenate
target	Y		Field to store the concatenated result
separator	N	" " (space)	String used to join the source values
description	N	-	Explanatory notes
if	N	-	Condition to run
ignore_failure	N	false	See Handling Failures
ignore_missing	N	false	Skip missing fields instead of failing
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
tag	N	-	Identifier

Details

The processor reads values from the specified source fields, converts them to strings, and joins them using the separator. Empty values are excluded from the concatenation. If ignore_missing is true, missing fields are skipped; otherwise, the processor fails if any source field is missing.

All source values are converted to strings using Go's standard string conversion, making the processor compatible with various data types including numbers, booleans, and complex objects.

note

The order of values in the concatenated result follows the order specified in the sources array. Rearranging the source fields will change the output order.

tip

When concatenating sensitive data or creating identifiers, consider the implications of the separator choice. Some separators may cause issues in downstream systems or create ambiguous results if the source data contains the separator character.

caution

Complex objects (maps, arrays) are converted to their string representation, which may not be human-readable. Consider using specific field paths or preprocessing the data before concatenation.

For more complex string manipulation, consider using the set processor and the so-called mustache syntax, i.e. {{var}}.

Examples

Basic Usage

Concatenate first and last name...	{ "first_name": "John", "last_name": "Doe" } - concat: sources: [first_name, last_name] target: full_name
into a single field:	{ "first_name": "John", "last_name": "Doe", "full_name": "John Doe" }

Custom Separator

Use a custom separator...	{ "city": "New York", "state": "NY", "zip": "10001" } - concat: sources: [city, state, zip] target: address separator: ", "
to format the output:	{ "city": "New York", "state": "NY", "zip": "10001", "address": "New York, NY, 10001" }

Multiple Field Types

Concatenate different data types...	{ "user_id": 12345, "department": "Engineering", "active": true } - concat: sources: [user_id, department, active] target: user_info separator: "|"
which are automatically converted to strings:	{ "user_id": 12345, "department": "Engineering", "active": true, "user_info": "12345|Engineering|true" }

Handling Missing Fields

With ignore_missing: true...	{ "first_name": "Jane", "title": "Dr." } - concat: sources: [title, first_name, middle_name, last_name] target: display_name separator: " " ignore_missing: true
missing fields are skipped:	{ "first_name": "Jane", "title": "Dr.", "display_name": "Dr. Jane" }

Creating URLs

Build URLs from components...	{ "protocol": "https", "domain": "api.example.com", "path": "/v1/users" } - concat: sources: [protocol, domain, path] target: api_url separator: "://"
Note: This creates an invalid URL format	{ "protocol": "https", "domain": "api.example.com", "path": "/v1/users", "api_url": "https://api.example.com:///v1/users" }

Better URL Building

For proper URL building, concatenate in parts...	{ "base_url": "https://api.example.com", "endpoint": "/v1/users", "query": "limit=10" } - concat: sources: [base_url, endpoint] target: full_url separator: "" - concat: sources: [full_url, query] target: api_request separator: "?"
to create well-formed URLs:	{ "base_url": "https://api.example.com", "endpoint": "/v1/users", "query": "limit=10", "full_url": "https://api.example.com/v1/users", "api_request": "https://api.example.com/v1/users?limit=10" }

Log Message Formatting

Create structured log messages...	{ "timestamp": "2024-01-15T10:30:00Z", "level": "ERROR", "service": "auth-service", "message": "Login failed" } - concat: sources: [timestamp, level, service, message] target: formatted_log separator: " | "
with consistent formatting:	{ "timestamp": "2024-01-15T10:30:00Z", "level": "ERROR", "service": "auth-service", "message": "Login failed", "formatted_log": "2024-01-15T10:30:00Z | ERROR | auth-service | Login failed" }

Empty Separator

Use an empty separator...	{ "prefix": "USER", "id": "001", "suffix": "ADMIN" } - concat: sources: [prefix, id, suffix] target: user_code separator: ""
to join values without any delimiter:	{ "prefix": "USER", "id": "001", "suffix": "ADMIN", "user_code": "USER001ADMIN" }