Remove

Mutate Elastic Compatible

Synopsis

Removes one or more fields from the document. Can handle both single fields and arrays of fields to remove.

Schema

- remove:
    field: <ident|ident[]>
    schema: <string>
    schema_type: <string>
    requirement_filter: <string>
    root_field: <ident>
    description: <text>
    disabled: <boolean>
    if: <script>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>

Configuration

The following fields are used to define the processor:

Field	Required	Default	Description
field	Y*		Field name(s) to remove. Accepts a single string, an array of strings, or glob patterns (*, ?, [])
schema	N		Schema reference path. When set, the field list is loaded from the schema and field is overwritten
schema_type	N	parquet	Schema file format. Accepted values: avro, parquet
requirement_filter	N	all	Filters which schema requirement levels are loaded from the schema
root_field	N		When set, the remove operation runs inside this nested map instead of at the top level of the log entry
description	N		Explanatory note
disabled	N	false	Disable the processor without removing it from the pipeline
if	N		Condition to run
ignore_failure	N	false	See Handling Failures
ignore_missing	N	false	Continue processing if the source field does not exist
on_failure	N		See Handling Failures
on_success	N		See Handling Success
tag	N		Identifier

* = Required unless schema is set.

Details

The processor supports template variables in field names and can remove nested fields using dot notation.

When schema is set, the list of fields to remove is loaded from the referenced schema file and the field value is overwritten with the schema's field list. The schema_type field selects the schema format (avro or parquet, defaulting to parquet); requirement_filter controls which requirement levels from the schema are included, defaulting to all. When schema is set, ignore_missing is automatically enabled regardless of the configured value.

When root_field is set, the remove operation runs inside the specified nested map instead of at the top level of the log entry. Field names in field are resolved as keys within that nested map, not as top-level paths.

Field values support glob patterns: * matches any sequence of characters, ? matches any single character, and [...] matches a character set. Patterns are evaluated using filepath.Match semantics and apply to immediate key names within the target scope (top level or root_field map).

note

When multiple fields are specified, the processor attempts to remove all of them. If ignore_missing is set to true, execution continues for the remaining fields even if a field removal fails.

warning

Be careful when removing nested fields as it might affect the structure of your document. Always ensure parent fields exist before attempting to remove child fields.

Examples

Single Field

Remove a single field from the document...	{ "user": { "id": "12345", "password": "secret", "email": "user@example.com" } } - remove: field: user.password
to avoid disclosing sensitive data:	{ "user": { "id": "12345", "email": "user@example.com" } }

Multiple Fields

Remove multiple fields at once...	{ "debug": true, "temp_data": ["a", "b", "c"], "result": "success", "message": "Operation completed" } - remove: field: - debug - temp_data
cleaning up temporary fields:	{ "result": "success", "message": "Operation completed" }

Templates

Remove fields using template variables...	{ "type": "user_data", "user_data_temp": "temporary", "user_data_cache": "cached" } - remove: field: - "{{{type}}}_temp" - "{{{type}}}_cache"
with dynamic field names:	{ "type": "user_data" }

Error Handling

Handle missing fields gracefully...	{ "field1": "value1" } - remove: field: - field1 - field2 ignore_missing: true on_failure: - append: field: tags value: removal_incomplete
with appropriate errors:	{ "tags": ["removal_incomplete"] }