Skip to main content

Sangfor HCI Datasets

Sangfor HCI datasets define what a Sangfor HCI device reads from a Sangfor HCI management platform (dataset type sangfor). They attach to devices of type sangfor, and in Datasets & Profiles the category is labelled Sangfor HCI. To create and assign datasets through the web interface, see Management. For the dataset and profile model, see Overview.

Collector Types​

There are 13 datasets, one per data model, each with its own definition id, event.dataset, interval, and pipeline, so it can be searched, routed, and processed on its own. Each dataset's event.dataset is sangfor.<dataset>.

Dataset (console title)DatasetDefinitionDefinition idKindDefault intervalMinimum
Sangfor Hostshostsangfor_host_collector10100inventory3,600 s60 s
Sangfor Host GPUshost_gpusangfor_host_gpu_collector10101inventory3,600 s60 s
Sangfor Host Memoryhost_memorysangfor_host_memory_collector10102inventory3,600 s60 s
Sangfor Host Processorshost_processorsangfor_host_processor_collector10103inventory3,600 s60 s
Sangfor Virtual Machinesvmsangfor_vm_collector10104inventory3,600 s60 s
Sangfor VM Disksvm_disksangfor_vm_disk_collector10105inventory3,600 s60 s
Sangfor VM Hardwarevm_hardwaresangfor_vm_hardware_collector10106inventory3,600 s60 s
Sangfor VM Network Adaptersvm_network_adaptersangfor_vm_network_adapter_collector10107inventory3,600 s60 s
Sangfor Datastoresdatastoresangfor_datastore_collector10108inventory3,600 s60 s
Sangfor Host CPU Usagehost_processor_statssangfor_host_processor_stats_collector10150metric60 s20 s
Sangfor Host Memory Usagehost_memory_statssangfor_host_memory_stats_collector10151metric60 s20 s
Sangfor Host GPU Usagehost_gpu_statssangfor_host_gpu_stats_collector10152metric60 s20 s
Sangfor VM Usagevm_statssangfor_vm_stats_collector10153metric60 s20 s

There is no event dataset. The platform's alarms are counted on each host and machine (alarm_count, warning), which the inventories write.

Every dataset takes these properties:

FieldRequiredTypeDefaultDescription
intervalNintper datasetSeconds between readings. At least the dataset's minimum above.
pipeline_nameNstringthe device'sThe preprocessing pipeline the dataset's records enter.

Wizard fields​

Every Sangfor HCI dataset's Dataset configuration step says how the dataset is read and has an Interval (Optional) field, in seconds: left empty, the dataset is read at its default; otherwise at least its minimum.

Rules​

  • Intervals. An interval under the dataset's minimum is left out with a warning in the device's log, and the rest of the device runs. Inventories are read hourly by default, because what the platform holds changes rarely. The use datasets may be read every 20 seconds at most.
  • One of each. A dataset type is read once per device, however many datasets of it are assigned. The first decides its settings; the others are left out with a warning.
  • Every reading reads afresh. The platform answers everything a host has, and how it is doing, in its hosts list, and the same of a virtual machine in its machines list. Every dataset reads its list again at every reading.
  • Pipelines. A dataset's records enter the pipeline the dataset selects, otherwise the device's. Two datasets on one device can reach two pipelines.

Inventories​

Inventory datasets write every item at every reading. From a dataset's second reading on, each record carries event.action:

event.actionevent.typeMeaning
addedcreationNot in the last delivered reading
changedchangeIts fields differ from the last delivered reading's
unchangedinfoAs it was
removeddeletionGone: written once, as it was last seen, dated now
  • The baseline is the last delivered reading. A reading that fails, or whose records could not all be written, never becomes the one the next compares with, so nothing is marked removed because a request failed.
  • What moves while an item stays the same is written but not compared, such as a host's status, a machine's power state, or a datastore's use. Each table below says which fields these are.
  • Ids, not names. An item is compared by its own fields and by the id of the availability zone it sits in, never by a name another object gives it. Renaming an availability zone marks nothing; moving a host, a machine, or a datastore to another zone marks it changed. A machine the platform moves to another host did not change: its record is tagged with the host it is on now.
  • The first reading after a start marks nothing: the baseline is kept in memory.

Sizes and memory are bytes (the platform counts them in MB, read as MiB). The platform's own ratios (ratio, allocation_ratio, memory_ratio) are written as it gives them, to a thousandth: the API does not document their scale.

Hosts​

DatasetOne record perFields
hostHost (hypervisor)host.name, host.ip; sangfor.host.id, name, ip, type, description, status (not compared), alarm_count (not compared), cpu.model/sockets/cores_per_socket/cores/total_mhz, memory.size, storage.size, gpu.count
host_gpuHost that has GPUssangfor.host_gpu.count, memory.size
host_memoryHostsangfor.host_memory.size, reserved (what the platform keeps back), allocation_max (as the platform gives it; not compared)
host_processorProcessor socketsangfor.host_processor.index, model, vendor (intel, amd, hygon), cores (the socket's)

The host dataset also refreshes the discovery that tags every dataset's records with its availability zone and host.

Virtual Machines​

DatasetOne record perFields
vmVirtual machinesangfor.vm.id, name, description, status, power_state, stopped, type, vmtype, instant_vm, encrypted, expire_time, location, backup_file_count, uptime, shutdown_duration, has_gpu, vtool_installed, os.type/name/installed, image, project, user, storage, dh.id/name, group.id/name/id_path/name_path, protection.*, in_protection, network_type, ips, floating_ip.*, alarm_count, warning.*, advanced.*; related.hosts, related.ip. Not compared: status, power_state, stopped, uptime, shutdown_duration, backup_file_count, alarm_count, warning, ips, in_protection, floating_ip.bind_status.
vm_diskDisksangfor.vm_disk.id, storage.id/name/tag_id, storage_file, physical_disk_type, size, eagerly_scrub, preallocate
vm_hardwareVirtual machinesangfor.vm_hardware.cpu.sockets/cores_per_socket/cores/mhz/total_mhz, memory.size, storage.size/used/used_pct/ratio/file_size (the use not compared), gpu.count/memory_total
vm_network_adapterNetwork interface, known by its vif id, else its port id, else its MAC addresssangfor.vm_network_adapter.vif_id, name, device_id, port_id, mac, model, ip, ipv6, network_type, custom_gateway, connected, subnet.id/name/cidr/gateway, vpc.id/name. Not compared: connected, ip, ipv6.

A machine's own name and its addresses are its vm record's related.hosts and related.ip, which is how a search finds it from the machine's own logs. host.name on a machine's records is always the host it runs on.

Datastores​

DatasetOne record perFields
datastoreDatastoresangfor.datastore.id, name, description, type, status, target, storage_tag_id, size, used, used_pct, ratio, read/write.bytes_per_sec/max_bytes_per_sec. Not compared: status, the use, and the throughput.

A datastore is tagged with its availability zone and names no host.

Use​

The use datasets read the platform's current figures from its hosts and machines lists, one list reading per dataset reading. The lists carry no time of their own, so each record is dated at its reading.

  • Only what is running. A host the platform says is down, and a machine that does not run, have no current figures and no record. The GPU datasets read the hosts and machines that have GPUs.
  • No zero for what the platform does not give. A figure the platform leaves out, or writes as null, as text that is not a number, or as a negative, is left out rather than written as zero. A host or machine that gives none of the dataset's figures has no record. A zero the platform writes is its own value, and is written.
  • Derived fractions. The part used (used.pct) is the platform's own used and total figures divided, a fraction of 1, written only when it gives both and the total is not zero.
  • Units. Memory in bytes, CPU in MHz, disk traffic in operations and bytes a second, network traffic in bits a second; the platform's ratios as it gives them.
DatasetOne record perFields
host_processor_statsRunning hostsangfor.host_processor_stats.used.mhz/pct (of the host's total MHz), ratio, allocation_ratio, allocation_max
host_memory_statsRunning hostsangfor.host_memory_stats.used.bytes/pct, ratio, allocation_ratio, allocation_max
host_gpu_statsRunning host that has GPUssangfor.host_gpu_stats.used (GPUs in use), ratio, memory.used.bytes/pct, memory.ratio
vm_statsRunning virtual machinesangfor.vm_stats.cpu.used.mhz/pct, cpu.ratio, memory.used.bytes/pct, memory.ratio, disk.read/write.iops/bytes_per_sec, network.read/write.bits_per_sec; while it has a GPU, gpu.ratio/memory_used/memory_ratio

Examples​

Reading a platform's hosts and virtual machines hourly, and the hosts' CPU use every 20 seconds...

- id: 512300160
  name: hci01
  type: sangfor
  status: true
  properties:
    address: "hci.example.com"
    username: "svc-virtualmetric"
    password: "$secret{store=corp-vault,ref=sangfor/hci}"
  definitions:
    - name: sangfor_host_collector
      status: true
    - name: sangfor_vm_collector
      status: true
    - name: sangfor_host_processor_stats_collector
      status: true
      properties:
        interval: 20