Content Hub: Overview

The Content Hub provides a centralized repository of professionally-developed pipeline templates designed to accelerate DataStream deployment and standardize data processing workflows. These templates contain pre-built parsing, transformation, and enrichment logic for popular security devices, network equipment, and enterprise applications.

Template Library

Professional Template Development

All templates in the Content Hub are developed and maintained by VirtualMetric's engineering team:

Quality Assurance:

• Rigorous testing across multiple device firmware versions
• Validation against real-world log samples and edge cases
• Performance optimization for high-volume data processing
• Regular updates to support new device features and log formats

Industry Standards Compliance:

• Support for multiple security schemas (ASIM, OCSF, ECS, CIM, UDM)
• Consistent field mapping and normalization across vendors
• Integration with popular SIEM and analytics platforms
• Compliance with regulatory and audit requirements

Comprehensive Coverage:

• Support for major security vendors (Palo Alto, Cisco, Fortinet, Check Point)
• Network infrastructure devices (switches, routers, load balancers)
• Cloud platform integrations (Azure, AWS, Google Cloud)
• Enterprise applications and databases

Template Categories

Security Device Templates:

• Firewall logs with advanced threat detection parsing
• Intrusion detection and prevention system events
• Endpoint protection and antivirus solutions
• Network access control and authentication systems

Infrastructure Templates:

• Network device logs and SNMP data processing
• Server operating system event collection
• Application performance and error log analysis
• Database audit and transaction log processing

Cloud Platform Templates:

• Cloud service activity and audit log processing
• Container and orchestration platform monitoring
• Serverless function execution and error tracking
• Cloud security and compliance event analysis

Interface Navigation

Content Discovery

Search and Filtering: The Content Hub interface provides multiple ways to discover relevant templates:

Search Functionality:

• Global search field for template name and description matching
• Real-time search results with highlighting of matching terms
• Search history and suggested queries for common use cases

Filter Categories:

• Device Type (single-select): Filter by device category (firewall, router, server, etc.)
• Target (multi-select): Filter by supported destination platforms (Sentinel, Splunk, etc.)
• Device Vendor (multi-select): Filter by manufacturer (Cisco, Palo Alto, Microsoft, etc.)

Filter Behavior:

• Visual feedback with highlighting and count indicators for filter selections
• Dynamic count updates showing template matches for each filter option
• Clear filter removal options with visual selection states

Template Cards

Information Display: Each template card provides essential information for evaluation:

Template Overview:

• Template name and version information
• Supported device types and vendor compatibility
• Target platform integration capabilities
• Installation status indicator (Available/Installed)

Visual Indicators:

• Installed badge: Blue status tag displayed for already-installed templates
• Icon representations for device types and target platforms
• Template complexity indicators (Simple/Advanced/Enterprise)
• Last update timestamps and version history

Template Details

Comprehensive Template Information: Clicking on any template card opens detailed documentation across four specialized views:

General Overview Tab

Template Metadata:

• Complete template description and use case documentation
• Supported device models and firmware versions
• Prerequisites and dependency requirements
• Installation and configuration guidance

Technical Specifications:

• Supported log formats and parsing capabilities
• Output schema and field mapping documentation
• Performance characteristics and resource requirements
• Integration compatibility matrix

Processor Documentation:

• Complete list of processing components used in template
• Individual processor documentation with configuration examples
• Links to detailed processor reference documentation
• Best practice recommendations for customization

Pipeline Overview Tab

Pipeline Architecture:

• Visual representation of data processing flow
• Parent and child pipeline relationships
• Processing stage documentation and dependencies

Read-Only Pipeline View:

• Complete YAML configuration display
• Syntax highlighting and structure visualization
• Processing logic explanation and comments
• Child pipeline navigation and cross-references

Log Transformation Overview Tab

Sample Data Processing:

• Real-world log samples showing input data format
• Step-by-step transformation process demonstration
• Final output format with field mapping examples

Before and After Comparison:

• Raw log data in original format
• Processed output showing normalized fields
• Schema compliance validation results
• Performance metrics and processing statistics

License Details Tab

Template Licensing Information:

• Complete license text for the specific template
• Elastic License 2.0 terms and conditions
• Copyright notices and attribution requirements
• Usage restrictions and compliance guidelines

Read-Only Display:

• Full license text displayed in code editor format
• Detailed terms specific to this template version
• Reference to general licensing framework documentation

Template Installation Process

Template Actions

Template detail pages provide different actions based on installation status:

Uninstalled Templates:

• Install Template button available in header
• Clicking triggers dependency check and installation process
• Requires PIPELINE_CREATE permission

Installed Templates:

• Installed badge displayed next to template name (blue tag)
• Actions menu provides management operations:
  • Manage Dependencies - Update optional and required dependencies (only shown if dependencies exist)
  • See Installed Pipeline - Navigate to the installed pipeline detail page

Installation Workflow

Template Evaluation: Before installation, users can thoroughly evaluate templates through the detail view tabs:

Preview Capabilities:

• Complete pipeline logic review without installation
• Sample data transformation testing
• License terms review
• Resource requirement estimation

Dependency Management

Pipeline Dependencies Modal: When installing a template with dependencies, a modal appears for dependency selection:

Dependency Categories:

Required Dependencies:

• Essential processing components that must be installed
• Core libraries and shared processing modules
• Schema definitions and validation rules
• Checkbox selection to confirm installation
• Already installed dependencies shown with "Installed" indicator (disabled checkbox)

Optional Dependencies:

• Enhanced processing features and advanced transformations
• Integration modules for specific target platforms
• Performance optimization components
• User-selectable checkboxes for installation
• Already installed dependencies indicated and disabled

Dependency List Features:

• Show More/Show Less buttons for lists exceeding 5 items
• Clickable dependency names to view dependency details
• Visual distinction between installed and available dependencies
• Installation progress indicator during processing

Managing Dependencies: For installed templates with dependencies, use Manage Dependencies action to:

• Install additional optional dependencies
• View currently installed required and optional dependencies
• Update dependency selections for the installed template

Installation Completion

Installation Success: Upon successful template installation:

Success Notification:

• Success toast message confirms template installation
• Notification displays template installation confirmation

Automatic Redirection:

• User automatically redirected to installed pipeline detail page
• Navigation to /pipelines/{id}/general-overview route
• Immediate access to pipeline configuration and customization

Post-Installation Access:

• Template remains visible in Content Hub with Installed badge
• Template detail page provides access to installed pipeline via See Installed Pipeline action
• Full pipeline editing capabilities in Pipeline management section

Post-Installation Management

Template Integration

Pipeline Integration: Once installed, templates become fully integrated into the DataStream platform:

Customization Capabilities:

• Full editing access to installed pipeline configurations
• Custom field mapping and transformation rule modification
• Integration with existing processing workflows
• Child pipeline creation and management

Status Tracking:

• Installation status indicators throughout Content Hub interface
• Version tracking and update notification system
• Usage statistics and performance monitoring
• Change history and configuration audit trails

Template Updates

When Content Hub templates are updated by VirtualMetric, installed pipelines can receive these updates while preserving custom modifications through a merge workflow.

Update Notification System

Update Availability Indicators: Installed templates with available updates display visual indicators throughout the platform:

Pipeline Overview:

• Renew Icon - Circular arrow icon displayed on pipeline cards
• Update Tooltip - Hover over icon shows "Update available" message
• Card Click - Clicking card navigates to pipeline detail view

Pipeline Detail View:

• Warning Alert Banner - Yellow alert displayed at top of page content
• Alert Title - "Update available" notification
• Alert Subtitle - Explanation that template source has updates
• Review Update Button - Action button to access update review interface
• Actions Menu - "Review Update" option added to pipeline actions menu

Review Update Interface

Accessing Update Review: Navigate to update review through multiple entry points:

• Click Review Update action button on warning alert banner
• Select Review Update from pipeline Actions menu
• Click Renew icon on pipeline overview card (redirects to detail, then review)

Review Update Page Structure:

Left Panel - Pipeline Tree:

• Resizable Sidebar - Adjustable width navigation panel (292-600px)
• Main Pipeline - Root pipeline with update indicator if applicable
• Child Pipelines - Nested pipelines with individual update indicators
• Renew Icons - Displayed on tree items with available updates
• Navigation - Click tree items to view specific pipeline changes
• Selection Highlight - Selected pipeline highlighted in tree

Right Panel - Merge Editor:

• Side-by-Side Diff View - Original vs updated YAML comparison
• Syntax Highlighting - YAML syntax coloring for readability
• Line-by-Line Changes - Visual indicators showing additions, deletions, modifications
• Original (Left) - Read-only current pipeline content
• Modified (Right) - Editable updated template content
• Manual Editing - Ability to modify updated version before accepting
• Theme Support - Respects user's light/dark theme preference

Update Application Workflow

Reviewing Changes:

1. Select Pipeline

   • Click pipeline or child pipeline in tree navigation
   • View changes for specific pipeline component
   • Navigate between multiple pipelines with updates

2. Examine Differences

   • Review side-by-side comparison of changes
   • Identify new features, modifications, and removals
   • Understand impact of template updates

3. Customize Updates

   • Edit right-side (modified) content as needed
   • Merge custom configurations with template updates
   • Preserve organization-specific modifications
   • Adjust updated content before application

4. Accept Changes

   • Click Accept Changes button to apply updates
   • Success notification confirms update application
   • Pipeline refreshes with updated content
   • Process repeats for additional pipelines with updates

Update Completion: When all pipeline updates are applied:

• Info Notification - Blue banner confirms "All updates installed"
• Completion Message - Indicates no pending updates
• Return Action - Button to navigate back to pipeline detail view
• Status Update - Renew icons removed from overview cards

Update Scenarios

Individual Pipeline Updates:

• Main pipeline has update, child pipelines unchanged
• Update only affects root pipeline content
• Child pipelines remain on current version

Child Pipeline Updates:

• One or more child pipelines have updates
• Main pipeline may or may not have updates
• Each child pipeline reviewed and updated independently

Complete Template Updates:

• Main pipeline and all child pipelines have updates
• Navigate through tree reviewing each component
• Apply updates systematically across entire template structure

Partial Updates:

• Accept some pipeline updates, skip others
• Review Update interface remains accessible
• Return later to apply remaining updates
• Flexibility to staged rollout of template changes

Best Practices

Template Selection

Evaluation Criteria:

• Device compatibility and log format support
• Target platform integration requirements
• Performance and scalability characteristics
• Maintenance and update lifecycle

Implementation Planning:

• Staging environment testing and validation
• Production deployment scheduling and rollback planning
• Team training and documentation requirements
• Long-term maintenance and customization strategies

Customization Guidelines

Modification Approach:

• Start with template defaults and incrementally customize
• Document all modifications for maintenance and updates
• Test customizations thoroughly before production deployment
• Maintain compatibility with template update processes

The Content Hub accelerates DataStream deployment by providing professionally-developed, tested, and maintained pipeline templates that can be quickly installed and customized for specific organizational requirements.