_file-formats
File Formats
| Format | Description |
|---|---|
json | Each log entry is written as a separate JSON line (JSONL format) |
multijson | All log entries are written as a single JSON array |
avro | Apache Avro format with schema |
parquet | Apache Parquet columnar format with schema |
Compression
Some formats support built-in compression to reduce storage costs and transfer times. When supported, compression is applied at the file/block level before upload.
| Format | Default | Compression Codecs |
|---|---|---|
| JSON | - | Not supported |
| MultiJSON | - | Not supported |
| Avro | zstd | deflate, snappy, zstd |
| Parquet | zstd | gzip, snappy, zstd, brotli, lz4 |
File Management
Files are rotated based on size (max_size parameter) or event count (batch_size parameter), whichever limit is reached first. Template variables in file names enable dynamic file naming for time-based partitioning.