Skip to main content

oVirt / Red Hat Virtualization Datasets

oVirt / Red Hat Virtualization datasets define what an oVirt / Red Hat Virtualization device reads from an oVirt engine or a Red Hat Virtualization Manager through its REST API (dataset type ovirt). They attach to devices of type ovirt, and in Datasets & Profiles the category is labelled oVirt / Red Hat Virtualization. To create and assign datasets through the web interface, see Management. For the dataset and profile model, see Overview.

Collector Types​

There are seventeen datasets, one per data model, each with its own definition id, event.dataset (ovirt.<dataset>), interval, and pipeline, so it can be searched, routed, and processed on its own. Nine are inventories of what the engine holds, read hourly; two read the hosts' and machines' status, and five their statistics, every minute; the seventeenth is the engine's event log.

Dataset (console title)DatasetDefinitionDefinition idKindDefault intervalMinimum
oVirt Hostshostovirt_host_collector9100inventory3,600 s60 s
oVirt Host Processorshost_processorovirt_host_processor_collector9101inventory3,600 s60 s
oVirt Host NICshost_network_adapterovirt_host_network_adapter_collector9102inventory3,600 s60 s
oVirt Usersuserovirt_user_collector9103inventory3,600 s60 s
oVirt Storage Domainsstorage_domainovirt_storage_domain_collector9104inventory3,600 s60 s
oVirt Virtual Machinesvmovirt_vm_collector9105inventory3,600 s60 s
oVirt VM NICsvm_network_adapterovirt_vm_network_adapter_collector9106inventory3,600 s60 s
oVirt VM Snapshotsvm_snapshotovirt_vm_snapshot_collector9107inventory3,600 s60 s
oVirt VM Disksvm_diskovirt_vm_disk_collector9108inventory3,600 s60 s
oVirt Host Statushost_statusovirt_host_status_collector9150metric60 s10 s
oVirt Host Statisticshost_statsovirt_host_stats_collector9151metric60 s15 s
oVirt Host NIC Statisticshost_network_adapter_statsovirt_host_network_adapter_stats_collector9152metric60 s15 s
oVirt VM Statusvm_statusovirt_vm_status_collector9153metric60 s10 s
oVirt VM Statisticsvm_statsovirt_vm_stats_collector9154metric60 s15 s
oVirt VM Disk Statisticsvm_disk_statsovirt_vm_disk_stats_collector9155metric60 s15 s
oVirt VM NIC Statisticsvm_network_adapter_statsovirt_vm_network_adapter_stats_collector9156metric60 s15 s
oVirt Eventseventovirt_event_collector44log60 s10 s

Every dataset takes these properties:

FieldRequiredTypeDefaultDescription
intervalNintper datasetSeconds between readings. At least the dataset's minimum above.
pipeline_nameNstringthe device'sThe preprocessing pipeline the dataset's records enter.
initial_lookbackNintthe device'sevent only. Seconds before its first reading where the dataset starts when it has no checkpoint. 0 starts at the first reading.

Wizard fields​

Every oVirt / Red Hat Virtualization dataset's Dataset configuration step says how the dataset is read and has an Interval (Optional) field, in seconds: left empty, the dataset is read at its default; otherwise at least its minimum.

Rules​

  • Intervals. An interval under the dataset's minimum is left out with a warning in the device's log, and the rest of the device runs. The inventories are read hourly by default, the status, statistics, and events every minute. A statistics dataset is read no more often than every 15 seconds, because the engine collects the statistics from its hosts every 15 seconds: a shorter interval would only read again what the last reading wrote.
  • One of each. A dataset type is read once per device, however many datasets of it are assigned. The first decides its settings; the others are left out with a warning.
  • Pipelines. A dataset's records enter the pipeline the dataset selects, otherwise the device's. Two datasets on one device can reach two pipelines.
  • Large engines. A dataset that asks after every host or machine one by one sends at most eight requests at once. On an engine with thousands of machines, give the per-machine datasets — the machines' parts and statistics — a longer interval.

Inventories​

Inventory datasets write every item at every reading. From a dataset's second reading on, each record carries event.action:

event.actionevent.typeMeaning
addedcreationNot in the last delivered reading
changedchangeIts fields differ from the last delivered reading's
unchangedinfoAs it was
removeddeletionGone: written once, as it was last seen, dated at the reading
  • The baseline is the last delivered reading. A reading that fails, or whose records could not all be written, never becomes the one the next compares with, so no change goes unreported and nothing is marked removed because a request failed.
  • Unread is not removed. A host whose statistics or interfaces the engine could not answer for is left out of the host reading and kept as the last reading held it; so is a running machine whose reported devices could not be read. The parts of a host or a machine whose parts could not be read, and the processors of a host whose record says nothing of its CPU's topology yet, are kept the same way.
  • What moves while an item stays the same is written but not compared, such as a host's status, a machine's power state, or a storage domain's space. Each table below says which fields these are.
  • Migrations. A machine the engine migrated to another host of its cluster did not change; one moved to another cluster did, and so did a host.
  • The first reading after a start marks nothing: the baseline is kept in memory.

Fields sit under ovirt.<dataset>.

Hosts​

DatasetOne record perFields
hostHost (hypervisor) the engine manageshost.name, host.id (its hardware UUID), host.ip, host.mac (every interface's), host.os.name/version; ovirt.host.id, name, address, description, comment, type (rhel, ovirt_node), status, status_detail, external_status, os.type/version, vdsm_version, libvirt_version, manufacturer, model, family, hardware_version, serial_number, uuid, cpu.model/type/mhz/sockets/cores_per_socket/cores/threads_per_core/threads (the counts over the whole host), memory.size, boot_time, spm.status/priority (its data center's storage pool manager role), kdump_status, selinux_mode, ksm, transparent_hugepages, numa_supported, update_available, reinstallation_required, protocol, port (VDSM's), iscsi_initiator. Not compared: status, status_detail, external_status, spm.status, update_available.
host_processorProcessor socket of every hostovirt.host_processor.index, description (the CPU's model), type (the engine's CPU type), vendor (intel, amd), mhz, cores, threads
host_network_adapterNetwork interface of every host: physical, bond, or VLANovirt.host_network_adapter.id, name, description, mac, ip, netmask, gateway, boot_protocol, ipv6, ipv6_gateway, ipv6_boot_protocol, mtu, speed, status (up, down), bridged, custom_configuration, base_interface and vlan (a VLAN's), network.id, ad_aggregator_id, bond.mode/mode_name/options (mode=4 miimon=100)/slaves/active_slave (by name). Not compared: status, speed, bond.active_slave, ad_aggregator_id.

Users and Storage Domains​

DatasetOne record perFields
userUser the engine knowsuser.id, user.name (the login: the principal, or the user name less its directory), user.full_name, user.email, user.domain (the directory it logs in from: internal-authz, an LDAP profile's); ovirt.user.id, name, last_name, user_name, principal, email, department, description, namespace, domain_entry_id, domain.id/name
storage_domainStorage domainovirt.storage_domain.id, name, description, comment, type (data, iso, export, image), storage_format, status, external_status, master, backup, block_size, wipe_after_delete, discard_after_delete, supports_discard, critical_space_action_blocker (GB), warning_low_space_indicator (percent), available, used, committed, used_pct, storage.type/address/path/port/nfs_version/mount_options/vfs_type/volume_group, data_centers (id and name of each, when it is attached to more than one). Not compared: status, external_status, master, available, used, committed, used_pct.

Virtual Machines​

DatasetOne record perFields
vmVirtual machineovirt.vm.id, name, description, comment, status, stop_reason, type (server, desktop, high_performance), origin, created, start_time (while it runs) or stop_time, stateless, delete_protected, run_once, next_run_configuration_exists, os_type, bios_type, placement_affinity, high_availability.enabled/priority, cpu.architecture/sockets/cores_per_socket/threads_per_core/count/shares, memory.size/guaranteed/max/ballooning/over_commit_pct/transparent_hugepages, template.id, original_template.id, instance_type.id, and while it runs what its guest agent reports: guest.hostname, guest.os.family/distribution/codename/architecture/version/kernel, guest.timezone.name/utc_offset, guest.ip; related.hosts, related.ip. Not compared: status, stop_reason, start_time, stop_time, run_once, next_run_configuration_exists, guest.
vm_network_adapterNetwork interface of every machineovirt.vm_network_adapter.id, name, description, comment, interface (virtio, e1000, rtl8139), mac, plugged, linked, synced, vnic_profile.id, guest.ip (the addresses the guest agent reports of the interface, matched by MAC address). Not compared: guest, synced.
vm_snapshotSnapshot of every machine, its active state among themovirt.vm_snapshot.id, description (the name the engine shows it by), type (regular, active, stateless, preview), status, date, persist_memory_state. Not compared: status.
vm_diskDisk of every machineovirt.vm_disk.id, name, alias, description, comment, format, content_type, storage_type (image, lun, managed_block_storage), status, sparse, shareable, provisioned_size, actual_size, total_size, wipe_after_delete, propagate_errors, qcow_version, backup, storage_domain.id/name, disk_profile.id, and its attachment's interface, bootable, active, read_only, pass_discard, uses_scsi_reservation, logical_name (the guest's name for it). Not compared: status, actual_size, total_size, logical_name.

Status​

host_status and vm_status write one record per host or machine at every reading, dated at the reading, from their own records. status is 1 when the host is up, or when the machine is up or migrating, otherwise 0, with why in reason.

DatasetFields
host_statusovirt.host_status.status (1 up, 0 otherwise), reason (why it is 0), state (the engine's: up, maintenance, non_responsive, …), status_detail, external_status, spm, vms.active/migrating/total (the machines it runs, as the engine counts them)
vm_statusovirt.vm_status.status (1 up or migrating, 0 otherwise), reason, state (up, down, paused, migrating, …), start_time and run_once while it runs, stop_time and stop_reason when it does not

Statistics​

The statistics datasets read the engine's current value of each statistic, which the engine collects from its hosts about every 15 seconds. The answer carries no time of its own, so each record is dated at its reading.

  • Only what the engine collects. The host datasets read the hosts the engine monitors (up, maintenance, preparing_for_maintenance); a host that does not respond, is down, or is being installed has no statistics the engine collected since, and is not asked for them. The machine datasets read the machines that run (up, powering_up, powering_down, migrating, reboot_in_progress, paused, saving_state, restoring_state).
  • No zero for what the engine does not give. A figure the engine does not report is left out, never written as zero, and so is one it reports as zero where zero means it has no value: a host's total memory, and the used and free memory derived from it; a host's CPU user, system, and idle parts when all three are zero; a machine's used memory and uptime; the memory figures a machine's guest agent reports, when no agent reports them; a disk's rates when it is not an image disk; a migration's progress while the machine does not migrate. A figure in another unit than its field names, or that is negative or not a number, is left out rather than written wrongly.
  • Units. Percentages are fractions of 1, latencies are milliseconds, and sizes are bytes. Counters (errors, data, disk operations) are the engine's running totals, under names that say so.
  • Sums. A host's network rates are its interfaces' added up, written only when every interface's could be read.
  • Left out, and said so. A host, machine, disk, or interface whose statistics the engine could not answer for is left out of the reading, and the device's log says how many and why; the others are written.
DatasetOne record perFields
host_statsMonitored hostovirt.host_stats.memory.total.bytes, memory.used.bytes/pct, memory.free.bytes/pct, memory.shared.bytes, swap.total/free/used.bytes, ksm.cpu.pct, cpu.user/system/idle.pct, cpu.load.5m (the load average itself), cpu.sockets/cores/threads, hugepages.<size>kb.free/total (hugepages.2048kb.free), network.received/transmitted.bytes_per_sec/bits_per_sec (every interface's added up)
host_network_adapter_statsNetwork interface of a monitored hostovirt.host_network_adapter_stats.id, name, received/transmitted.bytes_per_sec, bits_per_sec, bytes and errors (totals since the interface came up)
vm_statsRunning machineovirt.vm_stats.memory.installed.bytes, memory.used.bytes/pct, memory.free/buffered/cached/unused.bytes (what its guest agent reports), cpu.guest/hypervisor/total.pct, network.usage.pct, uptime (seconds), migration.progress.pct (while it migrates)
vm_disk_statsDisk of a running machineovirt.vm_disk_stats.id, name, read/write.bytes_per_sec (an image disk's), read/write.ops (totals since the machine started, oVirt 4.4 and later), read/write/flush.latency.ms, provisioned_size, actual_size, total_size
vm_network_adapter_statsNetwork interface of a running machineovirt.vm_network_adapter_stats.id, name, mac, received/transmitted.bytes_per_sec, bits_per_sec, bytes and errors

Events​

event reads the engine's event log. The engine numbers its events as it logs them, and the dataset writes every event numbered after its checkpoint, in the order the engine numbered them, dated when it happened. An event is tagged with the objects it names: host.name is the host it names, and its cluster and data center are its own, else the host's or the machine's.

The engine keeps its events for 30 days by default (AuditLogAgingThreshold).

DatasetOne record perFields
eventEventmessage (the event's description), event.id (its index), event.code, event.provider (its origin: oVirt, or the external system that logged it), log.level (its severity: normal, warning, error, alert), user.id; ovirt.event.index, code, severity, origin, correlation_id, custom_data, custom_id (an external event's own), flood_rate, log_on_host, and host, vm, cluster, data_center, storage_domain, template, user (id, and name where the discovery knows it): the objects it names

The dataset resumes from where the last delivered reading stopped:

  • Across a restart or a failover. The checkpoint, the last event's index, is kept in the Director cluster's replicated store, per device, so a restart, or another node after a failover, resumes from it.
  • Written after delivery. A reading's checkpoint is stored only once its events are delivered. A crash before then reads those events again: at least once, never lost. event.id, the event's index, is the key to recognize a repeat by.
  • Late events. The engine numbers an event when it creates it and saves it when its transaction commits, so an event can appear after one numbered after it. A reading asks from 100 below the checkpoint, and an event that appears late is written, once.
  • The first reading of a dataset with no checkpoint reads the newest 1,000 events and writes those of the last initial_lookback seconds: the dataset's own, else the device's, else a day. When all 1,000 are within the lookback, the device's log says so: older events of the lookback are not read.

Examples​

Reading the virtual machines' statistics every 30 seconds into their own pipeline, the hosts' status every minute, and the events from an hour back...

devices:
  - id: 512300150
    name: engine01
    type: ovirt
    status: true
    properties:
      address: "engine.example.com"
      username: "svc-virtualmetric@internal"
      password: "$secret{store=corp-vault,ref=ovirt/engine}"
    definitions:
      - name: ovirt_vm_stats_collector
        status: true
        properties:
          interval: 30
          pipeline_name: "ovirt-vm-stats"
      - name: ovirt_host_status_collector
        status: true
      - name: ovirt_event_collector
        status: true
        properties:
          initial_lookback: 3600
      - name: ovirt_host_collector
        status: true