VMware Cloud Director

Pull

Synopsis

Director polls one VMware Cloud Director (vCloud Director until 10.1) through its API, both the legacy API at /api and the Cloud API at /cloudapi, and writes what it reads as ECS documents: the cloud's organizations and their users, the vCenter Servers attached to it and their ESXi hosts, its organization VDCs, vApps, and virtual machines, and its audit trail. Nothing is installed on the cloud's cells.

What the device reads is decided by the datasets and profiles assigned to it, as for Windows, Linux, and SNMP devices. Each data model is a dataset of its own — see VMware Cloud Director Datasets.

The Cloud Director is the device. It is a device of its own, beside the vCenter Servers and ESXi hosts a VMware vSphere device reads. One device polls one Cloud Director.

Nothing it finds is registered as a device. The organizations, VDCs, vApps, and vCenter Servers the device discovers are records of its datasets, and they tag the records of the objects inside them (see Tags). None of them becomes a device of its own.

The device dials the Cloud Director and binds no port, so any number of VMware Cloud Director devices may use port 443. One Director polls each device: two Directors never read the same audit trail.

note

The VMware Cloud Director device needs Director 2.2.3 or later. With no Director or cluster at that version selected, the wizard shows VMware Cloud Director devices are unavailable on directors or clusters below version 2.2.3 and the device cannot be created.

Supported Versions

VMware Cloud Director 10.0 to 10.6.1, that is, API versions 33.0 to 39.1. Every login first asks the server which API versions it serves, and speaks the newest of them in that range. A server that serves only newer versions is spoken to in the oldest of those, and a pre-release version is never chosen. A server older than 10.0 is refused at start with Initialization error, naming the newest version it serves.

Account

The device logs in to the organization named by organization, as one of that organization's users. Which organization it is decides what the device can read:

	`organization: System` (a System administrator)	Any other organization (its administrator)
Organizations, their users, and VDCs	Every organization's	The organization's own
vApps and virtual machines	Every one	The organization's own
The ESXi host and vCenter Server of a virtual machine	Written	Not written
vCenter Servers and their ESXi hosts	Read	Nothing
Audit trail	Every organization's events	The organization's own events
`observer.version`	The Cloud Director's version	Left out

For the provider's view, log in to System as a user with the read-only rights of the System Administrator role.
For one tenant's view, log in to the tenant's organization as a user with the read-only rights of its Organization Administrator role.

The device changes nothing. A dataset the account may not read keeps the device Connected, and the device's message names each such dataset, and why, until that dataset reads again:

vCloud poller connected to vcd01. vcloud_vcenter_collector reads nothing: only a System administrator may read it, through the provider's API, and the device logs in to organization Acme

A System organization user whose custom role does not grant the provider's API is refused the vCenter Servers and the ESXi hosts by the server, and the message says so, with the server's answer.

Schema

- id: <numeric>
  name: <string>
  description: <string>
  type: vcloud
  tags: <string[]>
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    address: <string>
    organization: <string>
    username: <string>
    password: <string>
    port: <numeric>
    timeout: <numeric>
    initial_lookback: <numeric>
    tls:
      status: <boolean>
      insecure_skip_verify: <boolean>
      ca_name: <string>
      server_name: <string>
      min_tls_version: <string>
      max_tls_version: <string>
  definitions: <definition[]>

Configuration

The following fields are used to define the device:

Device

Field	Required	Default	Description
`id`	Y	-	Unique numeric identifier
`name`	Y	-	Device name
`description`	N	-	Optional description
`type`	Y	-	Must be `vcloud`
`tags`	N	-	Optional tags
`pipelines`	N	-	Optional pre-processor pipelines
`status`	N	`true`	Enable/disable the device

Connection

Field	Required	Default	Description
`address`	Y	-	Host name or IP address of a Cloud Director cell, or of the load balancer in front of the cells. A wildcard address, a URL, or `host:port` is refused.
`port`	N	`443`, or `80` without TLS	The API's port
`organization`	N	`System`	The organization the device logs in to. See Account.
`username`	Y	-	A user of that organization (resolvable via `${ENV_VAR}` or `$secret{...}`)
`password`	Y	-	The user's password (resolvable via `${ENV_VAR}` or `$secret{...}`)

The username and password take plain text, ${ENV_VAR}, or $secret{...} ($secret{id=...} or $secret{store=...,ref=...}). They are resolved at every login, so a password rotated in the vault is used from the next login on. The configuration keeps the reference, and a changed reference restarts the device.

TLS

Field	Required	Default	Description
`tls.status`	N	`true`	The API is called over HTTPS. `false` calls it over plain HTTP, for an endpoint behind a proxy that terminates TLS.
`tls.insecure_skip_verify`	N	`false`	`true` accepts any certificate. Pin the CA instead where you can.
`tls.ca_name`	N	the system trust store	A CA to verify the server's certificate against
`tls.server_name`	N	`address`	The name the certificate is verified for, when it is not `address`
`tls.min_tls_version`	N	`tls1.2`	`tls1.0`, `tls1.1`, `tls1.2`, or `tls1.3`
`tls.max_tls_version`	N	-	No cap

note

TLS material fields (cert_name, key_name, ca_name, client_ca_name) accept any of the following:

File name — resolved relative to the service root directory. Nested paths such as certs/prod/server.pem are supported.
Absolute path — honored only if it resolves inside the service root. Any path that escapes the root is refused.
Inline PEM content — used verbatim when the value contains -----BEGIN.
Environment variable — ${ENV_VAR}.
Vault reference — $secret{id=...} or $secret{store=...,ref=...}.

warning

With tls.status set to false, the login's credentials and every session token are sent in cleartext. Use plain HTTP only where the network between the Director and the proxy that terminates TLS is trusted.

Datasets

The device's definitions are its datasets, each named after its type, vcloud_<dataset>_collector. The ESXi hosts are the exception: they are the VMware vSphere device's host dataset, vmware_host_collector. In the console they are the datasets and profiles assigned to the device. For the dataset types, their intervals, and their fields, see VMware Cloud Director Datasets.

A device with nothing assigned is allowed: it logs in at start, reports its connection state, and collects nothing.

Advanced Configuration

Field	Required	Default	Description
`timeout`	N	`60`	Seconds each request may take, the login included, from `1` to `300`
`initial_lookback`	N	`86400`	Seconds before its first reading where the event dataset starts when it has no checkpoint. `0` starts at the first reading. The dataset's own `initial_lookback` wins.

Connection States

A configuration error — no address, no username, an organization with @ or : in it, a reference that does not resolve, a CA that cannot be read — stops the device at start with Configuration error. The message names the property but never its value.
A refused login sets Authentication error and stops the device, with the server's own reason. The Director starts it again later.
A server that does not answer sets Connection error; one whose certificate does not verify, Certificate error.
A server older than 10.0 sets Initialization error.

The connection state follows the server: the device is Connected once it logs in and whenever a reading succeeds. A request the server refuses the account (403), or a record that went away between a list and its reading (404), fails that record or that dataset's reading, not the device.

Session

Login. The device logs in through the Cloud API's sessions endpoint, as user@organization. Every request after it, to both APIs, carries the session's access token. Every dataset shares the one session.
Renewal. A session expires after the server's idle timeout, thirty minutes by default, and when a cell restarts. A request the server answers 401 logs in again once and is sent once more; one renewal serves every dataset. A renewed session that is refused again fails that reading with Authentication error.
Deadlines. Every request has the device's timeout as its deadline, and stopping the device abandons the requests in flight, the login included.
Logout. The session is logged out when the device stops.
Only the configured server is called. The links in the API's answers name the cells' public address, which need not be the one the Director reaches. The device builds every request itself on the address it was configured with, and never follows a redirect, so the token is never sent to another host.
User agent. The device identifies itself as VirtualMetric Director <version>.

When Things Are Read

Every dataset on its own interval, all of them at once when the device starts. The hourly inventories do not hold back the audit trail, which is read every minute.
A failed reading is read again after 60 seconds when its interval is longer. A reading the server refused the account waits for its interval, since it would be refused again.
What goes wrong in a dataset is its own. A refused request, records that cannot be written, or a panic costs that dataset's reading alone. The dataset keeps its checkpoint or its baseline, and the other datasets go on.

Field	On	Holds
`observer.vendor`, `observer.product`	Every record	`VMware`, `Cloud Director`
`observer.name`	Every record	The device's name in the console: the Cloud Director the record came through
`observer.hostname` or `observer.ip`	Every record	The device's `address`
`observer.version`	Every record of a System administrator's device	The Cloud Director's version (`10.5.1`)
`vcloud.organization.id`, `.name`	Users, VDCs, vApps, virtual machines, events	The organization the object is in, or the one an event happened in
`vcloud.vdc.id`, `.name`	vApps, virtual machines	The VDC the object is in
`vcloud.vapp.id`, `.name`	Virtual machines	The vApp the machine is in
`vcloud.vcenter.id`, `.name`	VDCs, virtual machines	The vCenter Server behind it, on a System administrator's device
`host.name`	Virtual machines, ESXi hosts	The ESXi host a machine runs on, or the host itself, on a System administrator's device

Records

Every record is one ECS 8.11 document:

event.module is vcloud, and event.dataset is vcloud.<dataset>. The ESXi hosts are written as the VMware vSphere device writes them: event.module vmware, event.dataset vmware.host.
event.kind is state for an inventory item, and event for an audit event.
event.category is iam for organizations and users, and host for the others.
Fields ECS has no name for sit under vcloud.<dataset>. Sizes are bytes, CPU is MHz, times are dates in UTC, and a field the server leaves empty is left out.

Inventory records carry event.action from a dataset's second reading on: added, changed, unchanged, or removed.

Each record also carries its dataset's own definition id. For the fields of each dataset, see VMware Cloud Director Datasets.

Console

In the console, go to Fleet Management > Devices > VMware Cloud Director and click Add new device. The wizard's steps are:

General settings, as on every device.
Connection settings: Address, Port, Organization, Username, and Password; then TLS encryption, and while it is on, Skip certificate verification, CA certificate (Optional), Server name (Optional), Minimum TLS version (Optional), and Maximum TLS version (Optional). Organization starts as System.
Advanced settings: Timeout and Initial lookback, both in seconds.
Data collection: the datasets or profile the device reads. A device receives datasets or a profile, never both.
Review and create.

Simulation

The VMware Cloud Director device is not simulated. What it writes is decided by its datasets, eight record shapes under eight definition ids, and one generic sample cannot stand in for them. A simulation block on a VMware Cloud Director device is refused on save. See Simulation.

Examples

Provider View

Logging in to the System organization with the password in a vault, reading every organization, the vCenter Servers and their ESXi hosts, and the audit trail...

devices:
  - id: 512300140
    name: vcd01
    type: vcloud
    status: true
    properties:
      address: "vcloud.example.com"
      organization: "System"
      username: "svc-virtualmetric"
      password: "$secret{store=corp-vault,ref=vcloud/provider}"
    definitions:
      - name: vcloud_organization_collector
      - name: vcloud_organization_user_collector
      - name: vcloud_vcenter_collector
      - name: vcloud_vdc_collector
      - name: vcloud_vapp_collector
      - name: vcloud_vm_collector
      - name: vmware_host_collector
      - name: vcloud_event_collector

One Tenant with a Pinned CA

Logging in to one tenant's organization, verifying the certificate against a CA of your own, and starting the audit trail a week back...

devices:
  - id: 512300141
    name: vcd-acme
    type: vcloud
    status: true
    properties:
      address: "vcloud.example.com"
      organization: "Acme"
      username: "${VCLOUD_USER}"
      password: "${VCLOUD_PASSWORD}"
      initial_lookback: 604800
      tls:
        ca_name: "vcloud-ca.pem"
    definitions:
      - name: vcloud_vdc_collector
      - name: vcloud_vapp_collector
      - name: vcloud_vm_collector
      - name: vcloud_event_collector

Synopsis​

Supported Versions​

Account​

Schema​

Configuration​

Device​

Connection​

TLS​

Datasets​

Advanced Configuration​

Connection States​

Session​

When Things Are Read​

Tags​

Records​

Console​

Simulation​

Examples​

Provider View​

One Tenant with a Pinned CA​