Library: Samples

Standard and Enterprise

Synopsis

A sample is a reusable piece of log data kept in the Library, alongside lookup tables, schemas and grok patterns. Wherever the platform needs realistic records to work with — a device producing synthetic traffic, a pipeline being debugged, a prediction being run — it draws them from a sample instead of asking you to paste the same log lines in again.

A sample comes in one of two kinds:

• A template describes what a record looks like, using {{name}} slots that are filled in afresh for every record generated from it. One template produces an endless stream of plausible, varied records.
• A set of records is real log data, stored exactly as written, one record per line. Nothing is generated or substituted.

Samples ship with the platform and can also be created by you. Either way, they are named, reusable content you manage like any other Library asset.

note

Library samples are not the same thing as Datasets and Profiles, which govern which Windows and Linux telemetry your agents collect. These two features share no configuration.

Accessing Samples

To open the samples list:

1. Click the hamburger menu in the top left corner.
2. Select Content Management > Library.
3. Open the Samples tab.

Samples share the Library's common interaction model — the detail drawer, the Assigned resources count, cloning, and an Activity logs trail on every sample. Those are described under Library.

Sample Types

Sample type is chosen when the sample is created and cannot be changed afterwards. It decides both what the sample holds and where the platform offers it.

Type	Holds	Offered to
Template	Template lines with placeholders, rendered into records on demand	Device simulation
Records	Real records, one per line, served as written	The pipeline debugger and prediction

The split follows what each consumer needs. A simulated device may run for days, and records with the same address and the same user name on every line would not resemble a real stream, so it needs a template whose values are redrawn per record. A pipeline debugger run needs the opposite: the same input every time, so that two runs can be compared.

There is nothing to configure beyond the type. Where a sample is offered follows from it, and there is no control to change that.

Device Type Scoping

Device types limits which devices may use the sample. Only the types listed here are offered it — so a syslog device is never shown a sample of S3 object records, and an S3 device is never shown Cisco ASA syslog lines.

Leaving the field empty means any device type that supports simulation, which is the right choice for a generic shape such as JSON application events. The list shows Any for such a sample.

The scope is enforced everywhere, not only in the dropdown. A device configuration naming a sample outside its scope is refused when you save it, with simulation: sample "aws_cloudtrail" is not available for syslog devices.

Which device types can be listed is fixed by which ones support simulation — 59 of them, as described under Device Simulation.

Built-in and Custom Samples

Samples carry an origin, shown in the list as a Built-in tag and filtered with the Origin control.

Built-in samples ship with the platform. Eleven are provided, all of them templates, and all offered to device simulation:

Sample	Vendor	Format	Device types
Cisco ASA firewall	Cisco	Syslog	syslog
Palo Alto Networks traffic log	Palo Alto Networks	CSV	syslog
FortiGate forward traffic	Fortinet	Syslog	syslog
Check Point firewall (CEF)	Check Point	CEF	syslog
Linux authentication (sshd, sudo)	Linux	Syslog	syslog
nginx access log	nginx	Text	Any
AWS VPC Flow Logs	Amazon Web Services	Text	Any
Windows Security events (JSON)	Microsoft	JSON	awss3
AWS CloudTrail	Amazon Web Services	JSON	awss3
Zeek conn.log (JSON)	Zeek	JSON	awss3
Okta System Log	Okta	JSON	awss3

Your organization holds its own copy of each one. They are read-only — opening one shows This sample is read-only, and neither its settings nor its content can be edited, nor can it be deleted. To base something of your own on a built-in sample, use Clone sample: cloning opens the create flow pre-filled with the built-in's values, and what you save from it is an ordinary sample of yours.

Because the copies are per organization, a built-in sample that gains a new version from the platform reaches every organization that has not diverged from it, and nothing you do to your own samples is visible to anyone else.

Note what the table above implies for a device of some other type. A kafka or mssql device is offered only the two unscoped built-ins, because the rest are scoped to syslog or awss3. For anything closer to what that source really produces, clone a built-in and widen its Device types, or write a sample of your own.

Custom samples are the ones you create, by cloning or from scratch. They can be edited and deleted, and are filtered with Origin > Custom.

The Samples Table

The table lists each sample with the following columns:

• Name - The sample's name. Click to open the detail drawer.
• Type - Template or Records.
• Vendor - The product or vendor the logs imitate.
• Format - Syslog, CEF, JSON, CSV or Text.
• Device types - The types the sample is offered to, or Any.
• Assigned resources - Count of resources currently using the sample. Click the count to list them.

Above the table, the Search field filters by name, and three dropdowns narrow the list: Origin (All, Built-in, Custom), Type (All, Template, Records) and Device Type. The Create new sample button opens the create flow.

The row action menu offers Manage sample and Clone sample, with Delete sample added for your own samples. A sample that is assigned to one or more resources cannot be deleted: a blocking modal headed Sample cannot be deleted lists them, and they have to be pointed elsewhere first.

Creating a Sample

Click Create new sample and complete three steps.

General settings

• Sample name - Required. Between 3 and 64 characters.
• Description - Optional.
• Vendor - Optional, at most 64 characters. The product or vendor the logs imitate.
• Format - Required. The format the logs use.
• Device type - Optional. Only these device types offer the sample; leave it empty for any simulation-capable type.
• Sample type - Required. Template or Records, as described above.

Configuration

What this step shows follows the type chosen in the previous one.

For a Template, enter the template in the Template field or upload a file. One record per line, with placeholders written as {{name}}, and a Placeholders editor below for the typed slots the template fills in. A bare type such as {{ip}} or {{timestamp}} needs no entry at all. The types and their options are in Simulation Placeholders.

For a template, Generate preview renders a few records with the same generator the Director runs, so you can check the output before the sample is saved.

For Records, paste the records into the Records field or upload a file. Every non-empty line is one record, stored exactly as written; blank lines are dropped, and there is no rendering or substitution.

Review and create

Review the summary of the previous steps, returning to any step to make changes, then create the sample.

Limits

Limit	Value
Name	3 to 64 characters
Vendor	At most 64 characters
Template	At most 256 KiB
Records	At most 4 MiB
Placeholders per sample	200

A template is additionally bound by the per-template limits in Simulation Placeholders — 64 placeholders in play, 1000 lines, and 64 KiB per line — which are the tighter of the two.

Sample Details

Clicking a sample's name opens a detail drawer summarizing it, with Manage sample and Clone sample as quick actions. The detail page has four tabs:

• Sample overview - Name, Description, Vendor, Format, Sample type, Device types, and the created and last-updated timestamps. Editable fields are saved from this tab.
• Configuration - The Template and its Placeholders, with the same preview available, or the Records, which have none. A template with no declared placeholders reads No placeholders defined. Bare types such as {{ip}} or {{timestamp}} need none.
• Assigned resources - The devices and pipelines currently using this sample.
• Activity logs - A searchable record of actions performed on the sample.

On a built-in sample every editable control is replaced by the read-only notice described above.

Using a Sample

In device simulation. A template sample appears in the Sample dropdown of a device's Simulation section, for the device types it is scoped to. The sample supplies both the template and its default placeholders; the device may override any placeholder by name without changing the sample. See Device Simulation.

In the pipeline debugger. A records sample is one of the inputs a debugger run can be given, so a pipeline can be exercised against a known set of records that does not change between runs. The debugger's Library samples list shows only the samples offered to it, beside the samples that came with the pipeline's Content Hub pack. See Debugging.

In prediction. The Prediction tab reads a records sample the same way, to estimate how much each processor reduces or enriches the data before the pipeline is applied.

A sample reaches a Director only when one of its devices selects it, so a large Library adds nothing to what a Director carries.

Editing a sample that a device already uses restarts that device, because a simulated device prepares its template once when it starts. The change takes effect on the restart rather than in place.