Devices: Overview
VirtualMetric Director uses devices as an abstraction layer to collect data from various sources and manage them for telemetry. As such, they decouple data sources from Director's pipelines.
Each device type provides specific configuration options detailed in their respective chapters.
Device Categories
Devices come in various categories, depending on their intended purpose of use. The main distinction, however, is OS monitoring versus protocol listening.
Operating System Monitoring
For monitoring operating systems, Director uses a unified agent-based approach with two types of deployment:
Managed (Traditional): The agent is installed and managed by system administrators. This provides persistent installation on the target system. Local data is buffered in the emergence of network issues. Director supports Windows, Linux, MacOS, Solaris, and AIX.
Auto-managed (Agentless): The agent is automatically deployed and managed, no manual installation is required. Auto-managed agents provide local data buffering, network resilience, and performance optimization. This deployment type is self-healing since the agent is automatically redeployed if the process terminates. Also, it supports remote credential management. Deployment is done using WinRM for Windows, and SSH for Linux, MacOS, Solaris, and AIX.
Both approaches provide local data processing, store-and-forward capability against connectivity issues, real-time metrics and events, and native OS monitoring. The key difference is deployment and lifecycle management, not functionality.
Protocol Listeners
Data is received from listeners using various network protocols with two types of collection:
Push-based: Devices like Syslog (via UDP/TCP), SNMP Traps, HTTP/HTTPS, TCP/UDP, and eStreamer send event data.
Pull-based: Data is fetched from Kafka topics, Microsoft Sentinel, REST APIs, database queries, and other custom integration types.
Data Processing
Devices enable handling and processing of data through the following means:
Pipelines: These provide the capability to parse and enrich messages with custom rules. Data is transformed—through field mapping and type conversion— and events are correlated and aggregated, custom fields are extracted, tagged, filtered, and routed based on message content. For certain conditions, alerts are generated. Additionally, there is templating support for output formatting, and built-in and custom functions.
Queues: These provide persistent message storage, configurable batch processing capability, automatic checkpoint recovery, and rotation and cleanup ability.
Performance
Devices can improve telemetry performance through their multi-worker architecture, dynamic scaling ability, socket reuse optimization, buffer management, and resource monitoring capability.
Security
For security, devices offer the following:
Authentication: Basic authentication, API keys, HMAC signing, and client certificates.
Encryption: TLS/SSL, SNMPv3 privacy, and custom encryption.
In addition, they also provide access control, and audit logging.
Always implement the appropriate security controls when deploying devices in production environments.
Deployment
As stated above, devices offer various deployment options for different purposes.
Standalone
With this approach, sources are connected directly to Director, bypassing any intermediate mechanism.
High Availability
Here, sources are first connected to a load balancer—which distributes the work to multiple processes—after which they are streamed to either a Director cluster:
-or- sources are connected to a cluster with VIP:
Common Uses
Devices are commonly used for the following:
Infrastructure monitoring: Provides system performance metrics, event logs, resource utilization, and service availability information.
Security operations: Enables security event monitoring, threat detection, compliance monitoring, and provides audit trails.
Application telemetry: Provides application logs and performance metrics, and enables error tracking and user activity monitoring.
Network monitoring: Provides network device logs and SNMP data, and enables traffic analysis and connection tracking
Configuration
Director offers flexible organization options for device configuration files: they all have a .yaml or .yml extension, and reside inside the config directory whose structure is like so:
Director discovers all the configuration files recursively, regardless of the subfolder structure.