Library: Event Split Rules

Synopsis

An event split rule set is an ordered list of rules, kept in the Library on the Event Split Rules tab. Each rule splits one incoming message into separate logs. For example, an Azure Event Hubs message {"records":[...]} becomes one log per record.

A device names the rule sets it uses in its event_split_rules property. The device tries the rules in order and uses the first one that matches. A message that no rule matches is handled as it would be without rules. How a device applies its rule sets, which device types take them, and what each rule type does are covered under Devices: Event Split Rules.

Splitting changes a device's log counts. One message with 35 records counts as 35 logs, and every later count, the device's In count, routes and targets included, counts the separate logs.

Requirements

• Edition - The tab shows when the edition has the library.event_split_rule feature, which every edition has today, Free included.
• Permissions - Rule sets use the Library permissions described under Roles:

To do this	You need
See the tab, a rule set's drawer and its page, and use the preview	Library read (LIBRARY_READ)
Create a rule set, or clone one	Library create (LIBRARY_CREATE)
Change a rule set, reset it to platform, or restore it	Library edit (LIBRARY_EDIT)
Hide a built-in rule set, or delete one of your own	Library delete (LIBRARY_DELETE)

A rule set's Activity logs tab also needs the audit log read permission (AUDIT_READ).

Accessing Event Split Rules

To open the rule sets:

1. Click the hamburger menu in the top left corner.
2. Select Content Management > Library.
3. Open the Event Split Rules tab.

The Rule Sets Table

The table lists each rule set with the following columns:

• Name - The rule set's name. Click it to open the drawer.
• Description - The rule set's description.
• Tags - The rule set's tags.
• Origin - Built-in or Custom. A built-in rule set you changed also shows Modified, and one you hid shows Hidden.
• Used by - How many devices use the rule set. The drawer and the rule set's Used by tab name them.

Above the table, the search field filters by name, the Origin filter narrows the list to All, Built-in, Custom or Hidden rule sets, and the Create rule set button opens the create wizard.

Clicking a rule set's name opens a drawer with its overview, its rules and the devices that use it. The drawer offers Manage and Clone, with Reset to platform on a built-in rule set you changed and Restore on a hidden one.

The row menu depends on the rule set:

Rule set	Row menu
A built-in rule set	Manage rule set, Clone rule set, Reset to platform (only after you changed the rule set), Hide rule set
A rule set of your own	Manage rule set, Clone rule set, Delete rule set
A hidden rule set	See rule set details, Restore rule set

A user without Library edit sees See rule set details instead of Manage rule set. Each item appears only for users with its permission.

A rule set that devices use cannot be deleted or hidden. Delete rule set and Hide rule set then open Rule set is in use instead, which lists the devices: remove the rule set from them first, then try again.

To act on several rule sets at once, see Library > Bulk Actions.

Built-in Rule Sets

The platform ships these rule sets. Each has one rule.

Rule set	Key	What it splits
Azure records	azure_records	The batches Azure sends to Event Hubs, {"records":[...]}, into one log per record. Azure Diagnostic Settings, Activity Log, Entra ID and Defender XDR all send them.
AWS CloudTrail	aws_cloudtrail	AWS CloudTrail log files, {"Records":[...]}, into one log per event.
JSON array	json_array	A message that is a JSON array, into one log per element.
JSON lines	json_lines	A message of JSON objects, one after another, into one log per object.
Lines	lines	A message into one log per line. A line that starts with a space or a tab stays with the line before it.
Do not split	do_not_split	Nothing: each message stays whole, as one log.
CSV with a header	csv_header	A CSV table into one JSON log per row, named by the first row. It has no condition, so it matches every message and every file the rules are checked against: list it after the rule sets that look for a format.
IIS / W3C extended logs	w3c_extended	An IIS or W3C extended log file into one JSON log per line, named by its #Fields line. A value of - is left out.
Zeek logs	zeek_logs	A Zeek log file into one JSON log per line, named by its #fields line. Dots in the field names become underscores, so id.orig_h is id_orig_h. A value of - is left out.

Creating a Rule Set

Click Create rule set and complete three steps.

General settings

• Name - Required. Between 3 and 64 characters.
• Description (Optional)
• Tags (Optional) - Labels that help you find the rule set in the list, separated by commas.
• Bytes to inspect - How much of the start of each message a rule's condition can read, from 16 to 4096 bytes. Defaults to 256.

Rules

A rule set holds 1 to 50 rules. Add rule adds one, named Rule 1, Rule 2 and so on, with JSON array as its type. Each rule has:

• Rule name - Required, and unique within the rule set.
• Rule type - Do not split (none), Lines (lines), Separator (separator), Regex (regex), JSON lines (jsonl), JSON array (json_array), CSV (csv) or File header (header). See Rule Types for what each does.
• Condition (Optional) - Which messages and files the rule applies to, written as a pipeline if: condition. It can test message (the first bytes of the message or file, as many as Bytes to inspect allows), source (the event hub, topic, queue, subscription or URL path, or a file's path) and device.type and device.name. Empty means every message and file. See Conditions.
• The type's own settings:
  • Separator takes Separator, the fixed text that marks where one log ends and the next begins.
  • Regex takes Pattern, a regular expression that matches where a log starts or ends, and Pattern role: Start marker begins a new log at each match, Terminator ends a log at each match.
  • JSON array takes Array field (Optional), the field that holds the array, such as records or data.items (empty means the whole message is the array), and Fields to copy (Optional) and Fields to remove (Optional). See Rule Types for how copied fields are named and limited.
  • CSV makes each row of a CSV table a JSON log, named by the first row. It takes Delimiter (Optional), a comma by default (enter \t for a tab), Quote character (Optional) and Escape character (Optional), both a double quote by default. Each is one character, and neither the quote nor the escape character may be the delimiter. See CSV.
  • File header makes each line of a table with header lines a JSON log, as in IIS, W3C and Zeek logs. It takes Header line pattern (Optional) (^# by default), Fields line pattern (Optional) (^#[Ff]ields[:]?\s+(.*) by default, with one group around the list of names), Delimiter pattern (Optional) (\s by default; Zeek logs need \t), Null value (Optional), a value to leave out such as -, and Clean field names, which turns each character of a column name that is not a letter or a digit into _. See File Header.
• Maximum log size (Optional) - From 1024 to 10485760 bytes, and 10485760 (10 MB) by default. A longer log is cut into pieces of this size, and a cut JSON log is no longer valid JSON.
• Rule status - Enabled or Disabled. A new rule is enabled. A disabled rule is skipped.

Under Rule order, the device tries the rules from the top and uses the first one that matches. Drag a rule by its handle to move it. With the keyboard, Space lifts the rule, the arrow keys move it, and Space drops it.

Under the rules, the Preview tries them on a message before you save.

Review and create

Review the two sections, each with Change to return to it, then click Create rule set.

The Rule Set's Key

A device's event_split_rules list names rule sets by their key. The platform makes a rule set's key from its name when you create it: the lowercase letters and digits, joined with hyphens, such as defender-xdr-batches. The key never changes, even when you rename the rule set. Built-in rule sets have keys with underscores, such as azure_records, so the two never collide.

Cloning a Rule Set

Clone rule set opens the wizard with a copy of the rule set's settings and rules. The copy is always a rule set of your own, with its own key.

Preview

The preview shows the logs a device would send on, before you save. It sits under the rules in the wizard's Rules step, and on the rule set's Rules tab.

In - the message to try the rules on:

• Text - The Message, typed or pasted, or loaded with Upload file. The whole text is one message, up to 4 MB.
• Library sample - A sample. Each of its first 100 records is one message.

Three more fields fill in what a condition can test: Source (Optional), Device type (Any device type by default) and Device name (Optional).

Out - the logs. A summary line counts them, such as Messages in: 3. Logs out: 105. Each message then shows its logs and the rule that split it (Split by Records.), or says that no rule matched. When a rule matches but cannot split the message, the preview says why. The preview stops at 1,000 logs. A device splits every message in full, within its limits: a message that a rule would split into more logs than they allow stays whole, as one log.

Rule Set Details

A rule set's page has four tabs:

• General settings - The general settings, with Manage general settings, and Rule set information: Key, Origin, Created and Last updated.
• Rules - The rules in order, with Manage rules to edit them, and the Preview.
• Used by - The devices that use the rule set. Click a name to open the device.
• Activity logs - A searchable record of actions performed on the rule set.

The header shows the origin tags, and an Actions menu: Clone rule set, Reset to platform (after a change), and Hide rule set or Delete rule set.

A built-in rule set's page says This rule set comes with the platform. You can change it or hide it. Reset to platform undoes your changes.

A hidden rule set's page is read-only and says Devices cannot use a hidden rule set. Restore it from the Actions menu to use it again. Its Actions menu offers Restore rule set alone.

Changing, Hiding and Restoring Rule Sets

• A changed rule set reaches a running device at its next message, with no restart. Changing the device's own event_split_rules list does restart it.
• A built-in rule set that you change shows Modified. Reset to platform replaces your changes with the platform's current version.
• Hide rule set takes a built-in rule set out of the list, and devices can no longer use it. The Hidden filter lists it. Restore rule set brings it back in the platform's current version.
• Delete rule set removes a rule set of your own for good.
• Neither hide nor delete works on a rule set that devices use. See The Rule Sets Table.
• Reset and restore fail when a rule set of your own already has the same name. The message says A custom rule set already uses this name. Rename that rule set, then try again.