Skip to main content
Version: 1.2.0

Deployment: On Local

This guide walks you through deploying DataStream on a local server or workstation. Local deployment is ideal for development, testing, or small-scale production environments where you need full control over the system resources.

Prerequisites​

Before starting a local deployment, ensure your system meets these requirements:

CategoryRequirements
Hardware
  • CPU: 2+ cores
  • RAM: 4GB minimum
  • Storage: 20GB available disk space
Software
  • OS: Linux (Ubuntu 20.04+, RHEL 8+), Windows Server 2019+, macOS
  • .NET Runtime: 6.0 or later

Deployment Options​

DataStream offers a straightforward installation method for local deployment.

Native Installation​

Install DataStream directly on the host operating system.

Installation Steps​

  1. Download the installer package -

    Invoke-WebRequest -Uri "https://download.datastream.example.com/latest/datastream-installer.exe" -OutFile "datastream-installer.exe"

  2. Make the installer executable (Linux/macOS only) -

    # Not needed for Windows

  3. Run the installer -

    .\datastream-installer.exe

  4. Configure the service -

    The installer will create a configuration directory at C:\ProgramData\DataStream

    notepad C:\ProgramData\DataStream\config.yaml

  5. Start the service -

    Start-Service DataStream
    Set-Service DataStream -StartupType Automatic

Configuration​

After deployment, configure DataStream to meet your specific requirements:

  1. Define devices in the configuration directory -

    Example Syslog devices configuration:

    devices:
        - id: 1
          name: local_syslog
          type: syslog
          properties:
            port: 514

  2. Add processors and pipelines as needed -

    pipelines:
      - name: basic_processing
        processors:
          - grok:
            field: message
            patterns:
              - "%{COMMONAPACHELOG}"

  3. Configure targets to forward processed data -

    targets:
      - name: local_file
        type: file
        properties:
          path: /var/log/datastream/processed.log

Verification​

After deployment, verify that DataStream is running correctly:

  1. Check the service status -

    Get-Service DataStream

  2. Review log output -

    Get-Content -Path "C:\ProgramData\DataStream\logs\service.log" -Tail 50 -Wait

  3. Test a device -

    Using PowerShell to send a UDP message to port 514:

    $Message = [Text.Encoding]::ASCII.GetBytes("Test message for DataStream")
    $UdpClient = New-Object System.Net.Sockets.UdpClient
    $UdpClient.Connect("127.0.0.1", 514)
    $UdpClient.Send($Message, $Message.Length)

Resource Allocation​

For local deployments, adjust resource allocations based on your workload:

ScaleVolume (events/sec)Recommendation
Small< 100Default configuration
Medium= 100-1000Allocate 2-4 dedicated cores, 8GB RAM
Large> 1000Consider distributing across multiple instances

Maintenance​

Regular maintenance tasks for local deployments:

  1. Update DataStream when new versions are released -

    • Download the latest installer:

      Invoke-WebRequest -Uri "https://download.datastream.example.com/latest/datastream-installer.exe" -OutFile "datastream-installer.exe"

    • Stop the service:

      Stop-Service DataStream

    • Run the installer to update:

      .\datastream-installer.exe

    • Start the service:

      Start-Service DataStream

  2. Backup configuration files regularly:

    Compress-Archive -Path "C:\ProgramData\DataStream" -DestinationPath "C:\Backups\datastream-config-backup.zip"

  3. Monitor disk usage to prevent storage issues:

    Get-ChildItem -Path "C:\ProgramData\DataStream\logs" -Recurse | Measure-Object -Property Length -Sum

Troubleshooting​

Common issues with local deployments:

IssuePotential Resolution
Service fails to start
  • Check system resources and configuration file syntax
  • Verify .NET Runtime is installed correctly
No data collected
  • Verify network settings and ports
  • Check firewall settings to ensure required ports are open
High CPU/memory usage
  • Adjust batch size and processing settings
  • Increase system resources if consistently high

For detailed logs, check:

Get-Content -Path "C:\ProgramData\DataStream\logs\service.log"