Commvault

Pull

Synopsis

Director polls one Commvault CommServe through its Web Console's REST API (/webconsole/api), logging in with a username and a password, and writes what it reads as ECS documents: the CommServe's clients and alert rules, and the backup and restore jobs and the events of its clients. Nothing is installed on the CommServe.

What the device reads is decided by the datasets and profiles assigned to it, as for Windows, Linux, and SNMP devices. Each data model is a dataset of its own, four of them — see Commvault Datasets.

The CommServe is the device. One device polls one CommServe, that is, one CommCell. A second CommCell is a second device.

The device dials the Web Console and binds no port, so any number of Commvault devices may use port 443. One Director node polls each device: the device takes part in the cluster's ownership lease, so two Directors never read the same CommServe, and a device that moves to another node resumes from its checkpoints.

note

The Commvault device needs Director 2.2.3 or later. With no Director or cluster at that version selected, the wizard shows Commvault devices are unavailable on directors or clusters below version 2.2.3 and the device cannot be created.

The device does not run in simulation. Each of its datasets writes records of its own shape, so one generated sample cannot stand in for them, and a simulation block on a Commvault device is refused on save. See Simulation.

Account

Sign in as a CommCell user with the view permissions the datasets need: clients, jobs, events, and alerts. A client rule that names client groups also needs the permission to view client groups (see Filters). The device reads; it changes nothing.

A request the CommServe refuses on its own — a permission the user lacks, answered 403 or with an error in the body — fails that dataset's reading and is logged. It does not change the device's connection state.

Schema

- id: <numeric>
  name: <string>
  description: <string>
  type: commvault
  tags: <string[]>
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    address: <string>
    username: <string>
    password: <string>
    port: <numeric>
    timeout: <numeric>
    initial_lookback: <numeric>
    tls:
      status: <boolean>
      insecure_skip_verify: <boolean>
      ca_name: <string>
      server_name: <string>
      min_tls_version: <string>
      max_tls_version: <string>
  definitions: <definition[]>

Configuration

The following fields are used to define the device:

Device

Field	Required	Default	Description
`id`	Y	-	Unique numeric identifier
`name`	Y	-	Device name
`description`	N	-	Optional description
`type`	Y	-	Must be `commvault`
`tags`	N	-	Optional tags
`pipelines`	N	-	Optional pre-processor pipelines
`status`	N	`true`	Enable/disable the device

Connection

Field	Required	Default	Description
`address`	Y	-	Host name or IP address of the Web Console that serves the REST API. A wildcard address, a URL, or `host:port` is refused.
`port`	N	`443`, or `80` without TLS	The Web Console's port
`username`	Y	-	A CommCell user (resolvable via `${ENV_VAR}` or `$secret{...}`)
`password`	Y	-	The user's password (resolvable via `${ENV_VAR}` or `$secret{...}`)

The username and password take plain text, ${ENV_VAR}, or $secret{...} ($secret{id=...} or $secret{store=...,ref=...}). They are resolved at every login, so a password rotated in the vault is used from the next login on.

TLS

Field	Required	Default	Description
`tls.status`	N	`true`	The API is called over HTTPS. `false` calls it over plain HTTP, for a Web Console that has no certificate.
`tls.insecure_skip_verify`	N	`false`	`true` accepts any certificate. Pin the CA instead where you can.
`tls.ca_name`	N	the system trust store	A CA to verify the Web Console's certificate against
`tls.server_name`	N	`address`	The name the certificate is verified for, when it is not `address`
`tls.min_tls_version`	N	`tls1.2`	`tls1.0`, `tls1.1`, `tls1.2`, or `tls1.3`
`tls.max_tls_version`	N	-	No cap

note

TLS material fields (cert_name, key_name, ca_name, client_ca_name) accept any of the following:

File name — resolved relative to the service root directory. Nested paths such as certs/prod/server.pem are supported.
Absolute path — honored only if it resolves inside the service root. Any path that escapes the root is refused.
Inline PEM content — used verbatim when the value contains -----BEGIN.
Environment variable — ${ENV_VAR}.
Vault reference — $secret{id=...} or $secret{store=...,ref=...}.

warning

With tls.status set to false, the login and every token are sent in cleartext. The password travels base64-encoded, which is an encoding rather than encryption, and the token that comes back is sent with every request. Use plain HTTP only where the path between the Director and the Web Console is trusted.

Datasets

The device's definitions are its datasets, each named after its type, commvault_<dataset>_collector. In the console they are the datasets and profiles assigned to the device. For the dataset types, their intervals, their filters, and their fields, see Commvault Datasets.

A device with nothing assigned is allowed: it logs in at start, reports its connection state, and collects nothing.

Advanced Configuration

Field	Required	Default	Description
`timeout`	N	`30`	Seconds each request may take, the login's included, from `1` to `300`
`initial_lookback`	N	`86400`	Seconds before its first reading where a job or event dataset with no checkpoint starts. `0` starts at the first reading. A dataset's own `initial_lookback` wins over the device's.

Connection States

A configuration error — no address, no username, a reference that does not resolve, a CA that cannot be read — stops the device at start with Configuration error. The message names the property but never its value.
A refused login sets Authentication error and stops the device, with the API's own reason in the device's log. The Director starts it again later.
A Web Console that does not answer, or one answering 502, 503, or 504 for a CommServe behind it that is not answering, sets Connection error; one whose certificate does not verify, Certificate error.

The connection state follows the CommServe, whichever dataset's reading shows it: the next reading that succeeds sets Connected again.

Session

Login. The device logs in with POST /Login, sending the password base64-encoded, as the API takes it. Every request after it carries the token the login returned. Every dataset shares the one token.
Renewal. The token expires after thirty minutes without a request, and when the Web Console restarts. A request the CommServe answers 401 logs in again once and is sent once more; one renewal serves every dataset. A renewed token that is refused again fails that reading with Authentication error.
Deadlines. Every request has the device's timeout as its deadline, and stopping the device abandons the requests in flight.

When Things Are Read

Every dataset on its own interval, all of them at once when the device starts. An hourly client inventory does not hold back the jobs and events read every minute.
A failed reading is read again after 60 seconds when its interval is longer, so an hourly inventory is not an hour late after a minute's outage.
What goes wrong in a dataset is its own. A refused request, records that cannot be written, or a panic costs that dataset's reading alone. The dataset keeps its checkpoint or its baseline, and the other datasets go on.

Checkpoints

The jobs and events datasets resume from where the last delivered reading stopped:

Across a restart or a failover. The checkpoint is kept in the Director cluster's replicated store, per device and dataset, so a restart, or another node after a failover, resumes from it.
Written after delivery. A reading's checkpoint is stored only once its records are delivered. A crash before then reads those records again at the next start: at least once, never lost.
The first reading of a dataset with no checkpoint goes back as far as the initial lookback, a day unless the dataset or the device says otherwise.
What a filter leaves out still moves the checkpoint on. A job update or an event the client rule leaves out is not written, but the checkpoint moves past it.

Records

Every record is one ECS 8.11 document:

event.module is commvault, and event.dataset is commvault.<dataset> — commvault.client, commvault.alert_rule, commvault.job, or commvault.event.
event.kind is state for a client or an alert rule, and event for a job update or an event.
The CommServe is named under observer.*: observer.vendor is Commvault, observer.product is CommServe, observer.name is the device's name in the console, and observer.hostname or observer.ip is the device's address.
The client a record is about is named under host.*.
Fields ECS has no name for sit under commvault.<dataset>. Times the API gives in seconds are written as dates, and sizes are bytes.

Client and alert rule records carry event.action from a dataset's second reading on: added, changed, unchanged, or removed. For the fields of each dataset, see Commvault Datasets.

event.dataset: "commvault.job" and event.outcome: "failure"
event.dataset: "commvault.event" and host.name: "sql01"
observer.name: "cs01"

Console

In the console, go to Fleet Management > Devices > Commvault and click Add new device. The wizard's steps are:

General settings, as on every device.
Connection settings: Address, Port, Username, and Password; under TLS settings, TLS encryption, Skip certificate verification, CA certificate (Optional), Server name (Optional), Minimum TLS version (Optional), and Maximum TLS version (Optional). Switching TLS off and on moves the port between 80 and 443, unless you typed one.
Advanced settings: Timeout and Initial lookback.
The datasets or profile the device reads. A device receives datasets or a profile, never both.
Review and create.

Examples

CommServe with Its Password in a Vault

Polling a CommServe with its password in a vault, reading its jobs and events every minute and its clients and alert rules hourly...

devices:
  - id: 512300118
    name: cs01
    type: commvault
    status: true
    properties:
      address: "commserve.example.com"
      username: "svc-virtualmetric"
      password: "$secret{store=corp-vault,ref=backup/commvault}"
    definitions:
      - name: commvault_job_collector
      - name: commvault_event_collector
      - name: commvault_client_collector
      - name: commvault_alert_rule_collector

Self-Signed Certificate with a Pinned CA

Verifying the Web Console's certificate against its own CA, with a longer timeout and a week of history on the first reading...

devices:
  - id: 512300119
    name: cs02
    type: commvault
    status: true
    properties:
      address: "192.0.2.50"
      username: "${COMMVAULT_USER}"
      password: "${COMMVAULT_PASSWORD}"
      timeout: 120
      initial_lookback: 604800
      tls:
        ca_name: "commvault-ca.pem"
        server_name: "commserve.example.com"
    definitions:
      - name: commvault_job_collector
      - name: commvault_event_collector

Synopsis​

Account​

Schema​

Configuration​

Device​

Connection​

TLS​

Datasets​

Advanced Configuration​

Connection States​

Session​

When Things Are Read​

Checkpoints​

Records​

Console​

Examples​

CommServe with Its Password in a Vault​

Self-Signed Certificate with a Pinned CA​