Skip to main content

Components

VirtualMetric DataStream is a telemetry pipeline solution that simplifies data collection, processing, and routing for Microsoft Sentinel and other Azure data services. At its core, DataStream uses pipelines to process, enrich, and direct data flows to their optimal destinations. It is composed of the following components:

VirtualMetric Director​

VirtualMetric Director™ is a tool designed for listening on various data sources, extracting and transforming them, and routing them to destinations. This powerful component acts as the central nervous system of your pipeline, orchestrating the flow across your entire infrastructure.

Director provides a unified interface for managing multiple sources and destinations, enabling seamless data collection, transformation, and distribution. Its architecture is built to handle enterprise-scale data volumes at high levels of performance and reliability.

Key capabilities include:

  • Source Management - support for multiple protocols (TCP, UDP, HTTP), file system monitoring, database change tracking, API integration, custom source implementations
  • Data Transformation - real-time data processing, rormat conversion, field extraction, data enrichment, custom transformation rules
  • Intelligent Routing - dynamic destination selection, load balancing, failover handling, priority-based routing, conditional routing
  • Monitoring and Control - real-time pipeline visibility, performance metrics, health monitoring, alert management, configuration validation

Director's flexible architecture allows it to adapt to changing requirements. Whether you're collecting logs from applications or monitoring system metrics, or gathering security events, Director provides the necessary tools to help you ensure efficient data handling and delivery.

VirtualMetric Agent​

VirtualMetric Agent™ is a lightweight, high-performance data collection component designed to efficiently gather telemetry data from various sources while maintaining minimal system impact. This versatile agent serves as the first point of contact in the telemetry pipeline, ensuring reliable data collection and initial processing.

The Agent is engineered with efficiency and reliability in mind, offering robust data collection capabilities without compromising system performance or stability.

Key features include:

  • Efficient Collection - low resource utilization, minimal CPU and memory footprint, optimized disk I/O, configurable collection intervals, adaptive rate limiting
  • Reliable Processing - local buffering, crash recovery, data persistence, automatic reconnection, error handling
  • Flexible Integration - multiple source types support, custom collector plugins, format adaptation, protocol conversion, destination selection
  • Advanced Monitoring - self-diagnostics, performance metrics and health status reporting, resource usage tracking, alert generation

Agent's architecture ensures seamless data collection and transmission while providing robust monitoring and management capabilities. Its modular design allows for easy extension and customization to meet specific organizational needs. It can operate both independently and as part of a larger telemetry infrastructure, making it suitable for various deployment scenarios, from single-server installations to large-scale distributed environments.