Redact

Security Elastic Compatible

Synopsis

Uses Grok patterns to identify and redact sensitive information in text fields. Matches are replaced with configurable prefix/suffix markers.

Schema

redact:
  - field: <ident>
  - patterns: <string[]>
  - description: <text>
  - if: <script>
  - ignore_failure: <boolean>
  - ignore_missing: <boolean>
  - on_failure: <processor[]>
  - on_success: <processor[]>
  - pattern_definitions: <map[string]string>
  - prefix: <string>
  - skip_if_unlicensed: <boolean>
  - suffix: <string>
  - tag: <string>

Configuration

Field	Required	Default	Description
field	Y	-	Field containing text to be redacted
patterns	Y	-	List of Grok patterns to match and redact
description	N	-	Explanatory note
if	N	-	Condition to run
ignore_failure	N	false	See Handling Failures
ignore_missing	N	false	If true, continue silently if field doesn't exist
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
pattern_definitions	N	-	Map of custom Grok pattern definitions
prefix	N	<	Prefix for redacted text
suffix	N	>	Suffix for redacted text
skip_if_unlicensed	N	false	Skip processing if license doesn't support redaction
tag	N	-	Identifier

Details

This processor is particularly useful for protecting Personally Identifiable Information (PII) by replacing sensitive data like email addresses, IP addresses, credit card numbers, and other patterns with redacted placeholders.

note

The redact processor uses Grok pattern matching, which is case-sensitive by default. For case-insensitive matching, modify your patterns accordingly.

warning

Applying many patterns to large volumes of text may impact performance. Consider limiting pattern complexity and the number of patterns in such cases.

Examples

Emails

Starting with a document containing an email address...	{ "message": "Contact us at support@example.com for help" } redact: - field: message - patterns: - "%{EMAILADDRESS:email}"
the email is replaced with a redacted marker:	{ "message": "Contact us at <email> for help" }

Multiple Patterns

Redact multiple types of sensitive data...	{ "log": "User test@company.com from IP 192.168.1.1" } redact: - field: log - patterns: - "%{IP:client}" - "%{EMAILADDRESS:email}" - prefix: "[" - suffix: "]"
with custom markers:	{ "log": "User [email] from IP [client]" }

Custom

Define custom patterns for specific formats...	{ "data": "Customer ID: ABC-12345-XY" } redact: - field: data - patterns: - "%{CUSTOMER_ID:id}" - pattern_definitions: CUSTOMER_ID: "[A-Z]{3}-\d{5}-[A-Z]{2}"
matching specialized formats:	{ "data": "Customer ID: <id>" }

Error Handling

In case of a missing field...	redact: - field: nonexistent_field - patterns: - "%{IP:client}" - ignore_missing: true - on_failure: - append: field: tags value: redaction_failed
display message:	{ "tags": ["redaction_failed"] }