VMware vSphere (vCenter / ESXi)

Pull

Synopsis

Director polls one vCenter Server, or one standalone ESXi host, through the vSphere API (the SOAP endpoint at /sdk), and writes what it reads as ECS documents: the ESXi hosts and what they have, how they are doing and their performance counters, the virtual machines and what they have, how they are doing and their performance counters, the objects a vCenter holds beside its hosts (datastores, distributed switches and port groups, resource pools, folders), and the events and tasks the server logs. Nothing is installed on vCenter or on the hosts.

What the device reads is decided by the datasets and profiles assigned to it, as for Windows, Linux, and SNMP devices. Each data model is a dataset of its own, 41 of them — see VMware vSphere Datasets.

One device type for both. You do not say which of the two the address is: the server says so when the device logs in. A vCenter device reads every ESXi host the vCenter manages, through the one device and one session; a standalone ESXi device reads the host itself. Point a device at a vCenter rather than at each of its hosts. A host a vCenter manages can also be read on its own as a standalone device; its records then name no vCenter.

Nothing it finds is registered as a device. The hosts, clusters, datacenters, and virtual machines the device discovers are records of its datasets, tagged with where they sit (see Tags), never devices of their own.

The device dials the server and binds no port, so any number of VMware vSphere devices may use port 443. One Director polls each device: two Directors never read the same device or open two sessions for it. Two devices that name the same vCenter are two devices, each with its own session and its own records.

note

The VMware vSphere device needs Director 2.2.3 or later. With no Director or cluster at that version selected, the wizard shows VMware vSphere (vCenter / ESXi) devices are unavailable on directors or clusters below version 2.2.3 and the device cannot be created.

Account

Sign in as a vCenter SSO user (for example svc-virtualmetric@vsphere.local) or an ESXi user. The Read-only role on the root of the inventory is enough: the device changes nothing.

Schema

- id: <numeric>
  name: <string>
  description: <string>
  type: vmware
  tags: <string[]>
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    address: <string>
    username: <string>
    password: <string>
    port: <numeric>
    timeout: <numeric>
    initial_lookback: <numeric>
    tls:
      status: <boolean>
      insecure_skip_verify: <boolean>
      ca_name: <string>
      server_name: <string>
      min_tls_version: <string>
      max_tls_version: <string>
  definitions: <definition[]>

Configuration

The following fields are used to define the device:

Device

Field	Required	Default	Description
id	Y	-	Unique numeric identifier
name	Y	-	Device name
description	N	-	Optional description
type	Y	-	Must be vmware
tags	N	-	Optional tags
pipelines	N	-	Optional pre-processor pipelines
status	N	true	Enable/disable the device

Connection

Field	Required	Default	Description
address	Y	-	Host name or IP address of the vCenter Server or the ESXi host. A wildcard address, a URL, or host:port is refused.
port	N	443, or 80 without TLS	The vSphere API's port
username	Y	-	A vCenter SSO user or an ESXi user (resolvable via ${ENV_VAR} or $secret{...})
password	Y	-	The user's password (resolvable via ${ENV_VAR} or $secret{...})

The username and password take plain text, ${ENV_VAR}, or $secret{...} ($secret{id=...} or $secret{store=...,ref=...}). They are resolved at every login, so a password rotated in the vault is used from the next login on. The configuration keeps the reference, and a changed reference restarts the device.

TLS

Field	Required	Default	Description
tls.status	N	true	The API is called over HTTPS. false calls it over plain HTTP, for a server behind a proxy that terminates TLS.
tls.insecure_skip_verify	N	false	true accepts any certificate. Pin the CA instead where you can.
tls.ca_name	N	the system trust store	A CA to verify the server's certificate against: for a vCenter, its VMCA root certificate, which the vCenter's landing page offers for download
tls.server_name	N	address	The name the certificate is verified for, when it is not address
tls.min_tls_version	N	tls1.2	tls1.0, tls1.1, tls1.2, or tls1.3
tls.max_tls_version	N	-	No cap

note

TLS material fields (cert_name, key_name, ca_name, client_ca_name) accept any of the following:

• File name — resolved relative to the service root directory. Nested paths such as certs/prod/server.pem are supported.
• Absolute path — honored only if it resolves inside the service root. Any path that escapes the root is refused.
• Inline PEM content — used verbatim when the value contains -----BEGIN.
• Environment variable — ${ENV_VAR}.
• Vault reference — $secret{id=...} or $secret{store=...,ref=...}.

warning

With tls.status set to false, the username and password are sent in cleartext at every login. A proxy that terminates TLS does not protect the hop between the Director and the proxy, so use plain HTTP only where that path is protected on its own.

Datasets

The device's definitions are its datasets, each named after its type, vmware_<dataset>_collector. In the console they are the datasets and profiles assigned to the device. For the dataset types, their intervals, and their fields, see VMware vSphere Datasets.

Nothing assigned is allowed: the device logs in at start, reports its connection state, and collects nothing.

Advanced Configuration

Field	Required	Default	Description
timeout	N	60	Seconds each request may take, the login included, from 1 to 300
initial_lookback	N	86400	Seconds before its first reading where an event or task dataset with no checkpoint starts. 0 starts at the first reading. A dataset's own initial_lookback wins.

Connection States

• A configuration error — no address, no username, a reference that does not resolve, a CA that cannot be read — stops the device at start with Configuration error. The message names the property but never its value.
• A refused login sets Authentication error and stops the device, with the server's own reason. The Director starts it again later.
• A server that does not answer sets Connection error; one whose certificate does not verify, Certificate error.

The connection state follows the server, whichever dataset's reading shows it: the next reading that succeeds sets Connected again. A fault the server answers one call with — a permission the user lacks, an object that went away — fails that dataset's reading and is logged, and does not change the connection state.

Session

• One session per device. The device logs in at start and runs every call in that session. Every dataset shares it.
• Renewal. A session expires after 30 minutes without a request (a vCenter's default) and when the server restarts. A call the server answers NotAuthenticated logs in again once and is run once more; one renewal serves every dataset. A renewed session that is refused again fails that reading with Authentication error.
• Logout. The session is logged out when the device stops: a vCenter caps its sessions, and one a stopped device left behind would count until it expired.
• User agent. The device identifies itself as VirtualMetric Director <version>, which the server shows in its sessions and login events.

When Things Are Read

• Every dataset on its own interval, all of them at once when the device starts. An hourly inventory of a thousand hosts does not hold back the status, events, and tasks read every minute.
• A failed reading is read again after 60 seconds when its interval is longer, so an hourly inventory is not an hour late after a minute's outage.
• What goes wrong in a dataset is its own. A fault, records that cannot be written, or a panic costs that dataset's reading alone; the other datasets go on.
• Failover. A device that moves to another node resumes its events and tasks from their checkpoints and discovers the inventory again.

vCenter or Standalone ESXi

	vCenter Server	Standalone ESXi host
Hosts, their status, parts, and performance counters	Every ESXi host the vCenter manages	The host itself
Events	The vCenter's events, every host's and virtual machine's	The host's own events
Tasks	The task history, and the tasks running now	The host's recent tasks: it keeps no task history
Datastores, resource pools, folders	Every one the vCenter holds	The host's own
Distributed switches and port groups	Every one	None: a standalone host has none
Virtual machines	Every virtual machine, templates included	The machines the host runs
observer.product	vCenter Server	ESXi
vmware.datacenter.name, vmware.cluster.name	The datacenter and cluster the object is in	None

Tags

Every record carries, as searchable fields, what it came through and where its object sits:

Field	Holds
observer.name	The device's name in the console: the vCenter, or the standalone ESXi host, the record came through
observer.product	vCenter Server or ESXi, as the server said at login
observer.hostname or observer.ip	The device's address
observer.vendor, observer.version	VMware, and the server's version (8.0.2)
host.name	The ESXi host the record is about: the host itself, the host a part belongs to or a counter was sampled on, the host a virtual machine runs on, or the host an event names
vmware.datacenter.name	The datacenter the object is in, on a vCenter
vmware.cluster.name	The cluster the object is in, where it is in one
vmware.vm.id, vmware.vm.name	The virtual machine a record is about, on every record of a machine's, of its parts, and of how it is doing

The datacenter, cluster, and hosts come from the device's discovery of the vSphere inventory, which it refreshes with every reading of the host inventory, or hourly when no host inventory dataset is assigned. The discovery is kept in memory only, and nothing is registered from it.

A virtual machine is not a host.* of its own: host.name is always the ESXi host, so a search by host finds the host, its parts, the machines on it, and their events and tasks. To reach a machine's records from its own logs, the vm record carries its name and, while VMware Tools runs, its guest's hostname under related.hosts and its guest's addresses under related.ip.

event.dataset: "vmware.host" and observer.name: "vc01"
event.dataset: "vmware.host_status" and vmware.cluster.name: "Production"
event.dataset: "vmware.task" and host.name: "esx07.example.com"
vmware.vm.id: "vm-1234" or vmware.event.vm.id: "vm-1234" or vmware.task.entity.id: "vm-1234"

Records

Every record is one ECS 8.11 document:

• event.module is vmware, and event.dataset is vmware.<dataset> — for example vmware.host, vmware.vm_stats, or vmware.event.
• event.kind is state for an inventory item, metric for how a host or a machine is doing, or event for an event or a task update.
• event.category is host, with network beside it for the network datasets.
• Fields ECS has no name for sit under vmware.<dataset>. Times are dates, sizes are bytes, and percentages are fractions of 1.

Inventory records carry event.action from a dataset's second reading on: added, changed, unchanged, or removed. A removed item is reported once, as it was last seen.

Each record also carries its dataset's own definition id. For the fields of each dataset, see VMware vSphere Datasets.

Console

In the console, go to Fleet Management > Devices > VMware vSphere (vCenter / ESXi) and click Add new device. The wizard's steps are:

1. General settings, as on every device.
2. Connection settings: Address (the vCenter or ESXi host), Port, Username, and Password; under TLS settings, TLS encryption, Skip certificate verification, CA certificate (Optional), Server name (Optional), Minimum TLS version (Optional), and Maximum TLS version (Optional). Switching TLS off and on moves the port between 80 and 443, unless you typed one.
3. Advanced settings: Timeout and Initial lookback, both in seconds.
4. The datasets or profile the device reads. A device receives datasets or a profile, never both.
5. Review and create.

Simulation

The VMware vSphere device is not simulated. What it writes is decided by its datasets, 41 record shapes under 41 definition ids, and one generic sample cannot stand in for them. A simulation block on a VMware vSphere device is refused on save.

Not Included

VMware Cloud Director is a device of its own. The console thumbnail of a virtual machine is not read: taking it would send each running machine a key press, which a device with the Read-only role neither takes nor may take.

Examples

vCenter Server

Polling a vCenter with its password in a vault, reading the hosts hourly, their status every minute, and the events and tasks...

devices: - id: 512300130 name: vc01 type: vmware status: true properties: address: "vcsa.example.com" username: "svc-virtualmetric@vsphere.local" password: "$secret{store=corp-vault,ref=vmware/vcenter}" definitions: - name: vmware_host_collector - name: vmware_host_status_collector - name: vmware_event_collector - name: vmware_task_collector - name: vmware_vm_collector - name: vmware_vm_status_collector

Standalone ESXi Host with a Pinned CA

Polling one ESXi host, verifying its certificate against a CA of your own, and starting the events a week back...

devices: - id: 512300131 name: esx07 type: vmware status: true properties: address: "esx07.example.com" username: "${ESXI_USER}" password: "${ESXI_PASSWORD}" initial_lookback: 604800 tls: ca_name: "esxi-ca.pem" definitions: - name: vmware_host_collector - name: vmware_host_stats_collector - name: vmware_event_collector