Skip to main content

Encrypt

Security Elastic Compatible

Synopsis​

Encrypts string values using AES encryption with optional compression. Supports both AES-256-GCM (Galois/Counter Mode) and AES-256-CFB (Cipher Feedback) modes of operation.

Schema​

encrypt:
  - algorithm: <string>
  - encryption_key: <string>
  - field: <ident>
  - iv_field: <ident>
  - description: <text>
  - if: <script>
  - ignore_failure: <boolean>
  - ignore_missing: <boolean>
  - on_failure: <processor[]>
  - on_success: <processor[]>
  - tag: <string>
  - with_compression: <boolean>

Configuration​

FieldRequiredDefaultDescription
algorithmNAES-256-GCMEncryption algorithm. Valid values: AES-256-GCM or AES-256-CFB
encryption_keyY-32-byte key for encryption
fieldY-Field containing the value to encrypt
iv_fieldY-Field to store the initialization vector
descriptionN-Explanatory note
ifN-Conditional expression
ignore_failureNfalseContinue if encryption fails
ignore_missingNfalseContinue if source field is missing
on_failureN-Processors to run on failure
on_successN-Processors to run on success
tagN-Identifier for logging
with_compressionNfalseCompress data before encryption

Details​

The processor encrypts string values using AES encryption, with support for two encryption modes:

Encryption Modes​

  • AES-256-GCM (Recommended):
    • Provides confidentiality and authenticity
    • More secure against tampering
    • Slightly slower performance
  • AES-256-CFB:
    • Provides confidentiality
    • Faster performance
    • No built-in authentication

Compression​

When with_compression is enabled:

  • Data is compressed before encryption
  • Reduces encrypted data size
  • Most effective for text-based data
  • Minimal impact on already compressed content

Security Recommendations​

Key Management​

  • Use cryptographically secure random number generators
  • Implement regular key rotation (every 90 days)
  • Store keys securely using:
    • Key Management Services (KMS)
    • Hardware Security Modules (HSM)
  • Follow principle of least privilege

Initialization Vector (IV) Management​

  • Generate using cryptographically secure random number generator
  • Never reuse IVs with the same key
  • Recommended IV sizes:
    • GCM mode: 12 bytes (96 bits)
    • CFB mode: 16 bytes (128 bits)

Secure Storage​

  • Store encrypted data and IVs separately from encryption keys
  • Use filesystem-level encryption
  • Implement secure backup procedures
  • Use authenticated encryption (GCM mode)

Examples​

Basic Encryption​

Encrypt a sensitive field using GCM mode...

{
  "password": "mysecret123"
}
encrypt:
  - field: password
  - iv_field: password_iv
  - encryption_key: "${ENCRYPTION_KEY}"
  - algorithm: AES-256-GCM

creates encrypted output:

{
  "password": "KZh/JR2baS2MkZpseKZYoBN2tQ==",
  "password_iv": "F+e8YorshrvFiFTC"
}

Compression with Encryption​

Compress and encrypt large text data...

{
  "log_data": "2024-01-01 DEBUG User logged in from IP 192.168.1.1\n2024-01-01 DEBUG User logged in from IP 192.168.1.1\n..."
}
encrypt:
  - field: log_data
  - iv_field: log_iv
  - encryption_key: "${ENCRYPTION_KEY}"
  - algorithm: AES-256-GCM
  - with_compression: true

reduces encrypted data size:

{
  "log_data": "Yh8dR2S2kZpsMkYoBN2tQ==",
  "log_iv": "K+r8YorvFiFTC"
}

Error Handling​

Handle missing or invalid fields...

{
  "other_field": "value"
}
encrypt:
  - field: missing_field
  - iv_field: missing_iv
  - encryption_key: "${ENCRYPTION_KEY}"
  - ignore_missing: true
  - on_failure:
      - set:
          field: error_status
          value: "encryption_failed"

continues processing with error handling:

{
  "other_field": "value",
  "error_status": "encryption_failed"
}
note
  • Encryption keys are cached for performance
  • Compression is most effective for text-based data
  • Use environment variables or secure key management for encryption keys
warning
  • Use exactly 32-byte encryption keys
  • Never reuse initialization vectors
  • Protect encryption keys rigorously
  • Be aware of performance overhead