Content Hub: Overview

The Content Hub is a central repository of pipeline templates covering security devices, network equipment, and enterprise applications. Each template carries the parsing, transformation, and enrichment logic for one source type, so a common device can be onboarded without writing that logic by hand.

What Are Templates?

A template — labeled a content pack throughout the Content Hub interface — is a pre-configured pipeline definition holding all the processing logic for one source type. It contains field extraction rules, normalization mappings, enrichment configurations, and output formatting, and is ready to deploy without further configuration.

Templates are complete pipeline packages. They define how raw log data is parsed, which fields are extracted, how values are normalized to standard schemas, and what enrichments are applied. When installed, a template becomes a fully functional pipeline within DataStream.

Why Use Templates?

Templates remove the need to build a pipeline from scratch. Rather than configuring parsers, field mappings, and transformation rules for each data source by hand, you install a template and begin processing immediately.

Building a pipeline for a complex source means writing and testing every parsing rule yourself. Installing a template is a single action.

Each template is developed by VirtualMetric's engineering team, validated against real log samples, and updated as vendors change their formats. Installing one therefore gives every deployment the same parsing behavior for that source, rather than a per-site interpretation of it.

For organizations processing data from multiple vendors and platforms, templates normalize every source the same way. Data from Palo Alto firewalls, Cisco switches, and Azure services all map to consistent field names and schemas.

Template Library

The Template Library contains pipeline configurations for common data sources and use cases.

Development and Maintenance

All templates in the Content Hub are developed and maintained by VirtualMetric's engineering team.

Each is tested across multiple device firmware versions, using real log samples and edge cases, and updated as vendors add features or change log formats.

Templates normalize to the security schemas DataStream supports — ASIM, OCSF, ECS, CIM, and UDM — keeping field mapping consistent across vendors. That normalization is what carries data into SIEM and analytics platforms unchanged, and what makes a source usable for regulatory and audit reporting.

Coverage spans:

• Security vendors such as Palo Alto, Cisco, Fortinet, and Check Point
• Network infrastructure including switches, routers, and load balancers
• Cloud platforms — Azure, AWS, and Google Cloud
• Enterprise applications and databases

Template Categories

Security device templates cover:

• Firewall logs, including threat detection fields
• Intrusion detection and prevention system events
• Endpoint protection and antivirus solutions
• Network access control and authentication systems

Infrastructure templates cover:

• Network device logs and SNMP data
• Server operating system events
• Application performance and error logs
• Database audit and transaction logs

Cloud platform templates cover:

• Cloud service activity and audit logs
• Container and orchestration platform monitoring
• Serverless function execution and error tracking
• Cloud security and compliance events

Interface Navigation

The Content Hub interface enables browsing, searching, and evaluating templates before installation. Templates can be filtered by device type, vendor, and target platform to quickly find relevant configurations.

Content Discovery

The Content Hub interface provides multiple ways to discover relevant templates through search and filtering capabilities.

A global search field with the placeholder Search Content Packs matches template names and descriptions, with results updating as you type. The result count is shown as "Viewing N content packs".

Available filter categories include:

• Device type (single-select): Filter by device category (firewall, router, server, etc.)
• Target (multi-select): Filter by device product family (for example, Cisco Firepower, FortiGate, AWS WAF)
• Device vendor (multi-select): Filter by manufacturer (Cisco, Palo Alto, Microsoft, etc.)

Filter selections display visual feedback with highlighting and count indicators. Dynamic count updates show template matches for each filter option. Clear filter removal options maintain visual selection states.

Template Cards

Each template card displays essential information for evaluation: the pack name, description, and tags for device type, vendor, and family.

Visual indicators help identify status at a glance. A blue Installed badge appears on already-installed packs (uninstalled packs show no status badge), and packs with pending updates show a circular-arrow icon with an "Update available" tooltip.

Template Details

Clicking on any template card opens detailed documentation across four specialized views.

General Overview Tab

Template metadata covers:

• Template description and use case documentation
• Supported device models and firmware versions
• Prerequisites and dependency requirements
• Installation and configuration guidance

Technical specifications cover:

• Supported log formats and parsing capabilities
• Output schema and field mapping
• Performance characteristics and resource requirements
• The integration compatibility matrix

Processor documentation lists every processing component the template uses, with configuration examples, links to the processor reference pages, and customization recommendations.

Pipeline Overview Tab

The pipeline architecture view shows data processing flow visually, with parent and child pipeline relationships and each processing stage's dependencies.

The read-only pipeline view shows the full YAML configuration with syntax highlighting, structure visualization, and inline processing-logic comments. Child pipeline cross-references allow exploration of the whole structure.

Log Transformation Overview Tab

Sample data processing walks a real log sample through the template step by step, ending in the output format with field mapping examples.

The before-and-after comparison places the raw log alongside the processed output with normalized fields, followed by schema compliance validation results and processing statistics.

License Details Tab

Licensing information covers:

• The full license text for that specific template
• Elastic License 2.0 terms and conditions
• Copyright notices and attribution requirements
• Usage restrictions and compliance guidelines

The read-only display renders the license in a code editor, with terms specific to the template version and a reference to the general licensing framework documentation.

Template Installation Process

Installing a template creates a local copy that can be customized for specific organizational requirements. The installation process handles dependencies automatically and provides immediate access to the installed pipeline.

Template Actions

Template detail pages provide different actions based on installation status.

For uninstalled templates, the Install content pack button is available in the header. Clicking triggers a dependency check and installation process.

note

Template installation requires the pipeline-create permission. Users without this permission will not see the install button.

For installed templates, an Installed badge (blue tag) is displayed next to the template name. The Actions menu provides management operations:

• Review update - Opens the update review interface (shown when an update is pending)
• Manage dependencies - Updates optional and required dependencies (only shown if dependencies exist)
• See installed pipeline - Navigates to the installed pipeline detail page
• See installed route / Install route - Shown when the content pack includes a route configuration; opens the installed advanced route or the route install modal

Installation Workflow

Before installation, users can thoroughly evaluate templates through the detail view tabs. Complete pipeline logic review is available without installation. Sample data transformation testing allows verification of expected output. License terms review and resource requirement estimation help inform the installation decision.

Dependency Management

When installing a template with dependencies, a modal appears for dependency selection.

Required dependencies include essential processing components that must be installed for the template to function. They appear as a non-interactive list and are installed automatically. Already-installed entries show the dependency name followed by (installed).

info

Required dependencies are mandatory. The template will not function correctly without them.

Optional dependencies include enhanced processing features and advanced transformations. These appear as user-selectable checkboxes. Already-installed dependencies are marked (installed) with a disabled checkbox.

Dependency list features include Show More and Show Less buttons for lists exceeding 5 items. Clickable dependency names open dependency details. Visual distinction differentiates installed from available dependencies.

An installation progress indicator displays during processing.

For installed templates with dependencies, use the Manage dependencies action to install additional optional dependencies or view currently installed dependencies.

Installation Completion

Upon successful template installation, a success toast message confirms the installation. The user is automatically redirected to the installed pipeline detail page at the /pipelines/{id}/general-overview route. This provides immediate access to pipeline configuration and customization.

After installation, the template remains visible in Content Hub with an Installed badge. The template detail page provides access to the installed pipeline via the See installed pipeline action. Full pipeline editing capabilities are available in the Pipeline management section.

Post-Installation Management

After installation, templates become editable pipelines within DataStream. Users can customize processing logic, modify field mappings, and integrate templates with existing workflows.

Template Integration

Once installed, templates become fully integrated into the DataStream platform.

Customization covers:

• Full editing access to installed pipeline configurations
• Custom field mapping and transformation rule modification
• Integration with existing processing workflows
• Child pipeline creation and management

Tracking covers:

• Installation status indicators throughout the Content Hub interface
• Version tracking, with update notifications when a new version is published
• Usage statistics and performance monitoring
• Change history and configuration audit trails

Template Updates

When Content Hub templates are updated by VirtualMetric, installed pipelines can receive these updates while preserving custom modifications. A merge workflow enables selective update acceptance while retaining organization-specific changes.

tip

Template updates preserve your customizations. The merge workflow lets you review changes and decide which updates to accept while keeping organization-specific modifications intact.

Update Notification System

Installed templates with available updates display visual indicators throughout the platform.

In the Content Hub listing, a renew icon (circular arrow) is displayed on cards for packs with pending updates. Hovering over the icon shows an "Update available" tooltip. Clicking the card navigates to the content pack detail view.

In the content pack detail view, a warning alert banner is displayed at the top of the page content. The banner shows a "Pipeline update available" title and explains that the template source has updates. The Review update action button provides access to the update review interface. The Review update option is also available in the Actions menu.

Review Update Interface

Navigate to update review through multiple entry points:

• Click Review update action button on warning alert banner
• Select Review update from the Actions menu
• Click renew icon on a Content Hub card (redirects to detail, then review)

The review update page is structured with two panels.

The left panel contains the pipeline tree with a resizable sidebar (adjustable width 256-600px). The main pipeline appears as the root with an update indicator if applicable. Child pipelines are nested with individual update indicators.

Renew icons are displayed on tree items with available updates. Clicking tree items displays specific pipeline changes. The selected pipeline is highlighted in the tree.

The right panel contains the merge editor with a side-by-side diff view showing original vs updated YAML comparison. Syntax highlighting provides YAML color coding for readability. Line-by-line changes show visual indicators for additions, deletions, and modifications.

The original content (left) is read-only, while the modified content (right) is editable.

Manual editing allows modification of the updated version before accepting. The editor respects the user's light/dark theme preference.

Update Application Workflow

To review and apply changes:

1. Select Pipeline - Click a pipeline or child pipeline in the tree navigation to view changes for that specific component. Navigate between multiple pipelines with updates as needed.

2. Examine Differences - Review the side-by-side comparison of changes. Identify new features, modifications, and removals. Understand the impact of template updates before proceeding.

3. Customize Updates - Edit the right-side (modified) content as needed to merge custom configurations with template updates. Preserve organization-specific modifications. Adjust updated content before application.

4. Accept Changes - Click the Accept Changes button to apply updates. A success notification confirms the update application. The pipeline refreshes with updated content. The process repeats for additional pipelines with updates; an Accept All Changes button applies all pending updates at once.

When all pipeline updates are applied, a blue info notification banner confirms "All updates installed" and indicates no pending updates. A button allows navigation back to the pipeline detail view. Renew icons are removed from overview cards.

Update Scenarios

For individual pipeline updates, the main pipeline has an update while child pipelines remain unchanged. The update only affects root pipeline content. Child pipelines remain on their current version.

For child pipeline updates, one or more child pipelines have updates while the main pipeline may or may not have updates. Each child pipeline is reviewed and updated independently.

For complete template updates, the main pipeline and all child pipelines have updates. Navigate through the tree reviewing each component. Apply updates systematically across the entire template structure.

For partial updates, accept some pipeline updates and skip others. The Review Update interface remains accessible to return later and apply remaining updates. This provides flexibility for staged rollout of template changes.