Skip to main content

Deploy: Overview

Deploy runs deployment templates: text files, written in Bicep, that describe DataStream resources. One template can create or update directors, clusters, devices, targets, quick and advanced routes, pipelines, Content Hub packs and routes, library items, datasets, profiles and vault secrets. Before it runs, a deployment previews exactly what it will change.

A template deploys from the console or through the REST API with an API token, and any existing resource can be exported as a template, to deploy again elsewhere or to change first.

How a Deployment Works​

A few rules explain everything else on this page.

  • A resource is identified by its name. Deploying looks each resource up by the name the template gives it. When it does not exist, it is created. When it exists, its properties are compared with the template's, and it is updated only if they differ.
  • Nothing is ever deleted. A resource the template does not mention is never touched. Removing a resource from a template and deploying it again leaves the resource where it is.
  • One deployment runs per tenant at a time. Starting a second while one is running is refused with Another deployment is running.
  • A deployment runs one resource at a time, in dependency order, and stops at the first failure. The resources after it are skipped, and the resources already written stay written. Once the cause is fixed, deploy again: what already matches reports no change.
  • A deployment runs for at most 30 minutes. Past that, the rest is skipped and the deployment fails.
  • A secret must come through a @secure() parameter. A template that writes a password, a client secret, a token or a vault secret's value any other way is refused, because the value would be stored with the deployment and shown to everyone who can read it. A secure parameter's value is never shown, logged, audited or stored: it appears as ***.
  • Each pipeline change lands as its own commit. A pipeline and each of its child pipelines are separate commits, merged one after the other, the parent first.

The template language itself, every resource type's properties and worked examples are in Templates.

Opening Deploy​

Click Deploy in the side navigation, right after Search.

The page needs the Deployment read permission. See Permissions.

The Deployments List​

The list shows the tenant's deployments, newest first, with these columns:

ColumnShows
NameThe deployment's name. Opens its detail page.
StatusRunning, Succeeded, Failed, Canceled or Interrupted.
ResourcesHow many resources the template declares.
Started byThe user, or the API token, that started it.
StartedWhen it started.
DurationHow long it ran.

Search deployments matches the name and who started it, and the Status filter narrows the list. Click New deployment in the toolbar to write a new one. With no deployments yet, the empty state offers Start from a sample.

New Deployment​

New deployment opens a three-step wizard.

Template​

Write the template in Bicep in the editor, click Upload to open a .bicep file (at most 1 MiB), or pick one of the Samples:

  • Syslog device on UDP 514
  • Syslog to Microsoft Sentinel
  • Pipeline with staging and sync
  • Serverless director on Azure
  • Grok pattern and SNMP vendor

The template is checked as you type: problems are underlined in the editor and listed under Problems in the template, and the resources it declares are summarized beside it. Download saves the template to a file. A template with errors cannot go past this step.

Parameters​

A form generated from the template's param declarations:

  • A field whose parameter has a default is optional and filled in already.
  • A @secure() parameter is a password field.
  • A parameter with @allowed([...]) is a dropdown.
  • A parameter with @metadata({ picker: 'director' }) offers a list of the tenant's existing directors. The same works for cluster, device, target, pipeline, advancedRoute, dataset, profile and vaultSecret. The value is still the resource's name.

Deployment name is optional. Left empty, the deployment is named after the time it starts.

Review​

A preview of what the deployment will do, writing nothing. Each resource is listed as one of:

ChangeMeaning
CreateIt does not exist and will be created.
ModifyIt exists and differs. Each changed property is listed with its current value (Now) and its new one (After deploy).
No changeIt exists and already matches.
ReadAn existing resource: looked up, never written.
ActionAn action that runs on every deployment, such as a pipeline staging sync.
IgnoreIts condition is false, so it is skipped.
UnknownThe preview cannot tell, for instance because its name depends on another resource.

If you are missing a permission the deployment needs, the review lists it under You are missing permissions this deployment needs, and the deployment cannot start. A change a resource cannot take, such as a director's mode or a device's type, is reported here too.

Click Deploy to start it. The detail page opens.

Deployment Detail​

The detail page shows the deployment as it runs, and afterwards.

  • Overview: the deployment details (status, started, completed, duration, started by, source, resources); the Operations table, one row per resource with its operation and status, updated live while the deployment runs; any error; and the template's Outputs.
  • Template: the template as it was deployed, with Download.
  • Parameters: the values it ran with. Secure values show as hidden.

Actions offers:

  • Redeploy opens the wizard with this deployment's template.
  • Cancel stops a running deployment before its next resource. The resource in progress finishes, and what was already deployed stays as it is.

A deployment that stopped reporting progress, for instance because the server running it restarted, is marked Interrupted. The resource it was running may or may not have been applied: check it, then redeploy.

A New Director's API Key and Install Script​

When a deployment creates a director, its API key and install script appear once, at the top of the detail page, under Copy these values now. Only the person who started the deployment sees them, and only for 15 minutes after it ends. They are never stored, so copy them then. If they are lost, the director's API key can be regenerated from its page.

Restricted Details​

Anyone with the Deployment read permission sees the tenant's deployments, but a resource of a type you cannot read shows as restricted: its name and changes are hidden. A deployment with any such resource also hides its template, parameters and outputs, and names the read permissions you would need.

Show Template​

The Create wizards of most resources offer Show template on their review step. It opens the Template and API request drawer, whose Template tab shows the resource you are about to create as a Bicep template, with Open in Deploy, Download and Copy. A secret you typed never lands in it: it becomes a @secure() parameter to supply when you deploy.

The drawer's other tab, API request, is covered in REST API Access.

Export Template​

Export template is in the Actions menu of the detail pages of devices, agents, targets, advanced routes, pipelines, directors, clusters, datasets and profiles. It opens the resource as a Bicep template, with Open in Deploy, Download and Copy.

An exported template:

  • declares a parameter for the resource's name, defaulting to its current name;
  • turns every director, pipeline, secret or other resource it refers to into an existing resource with its own name parameter, so it deploys to another tenant where the ids differ;
  • declares each secret as an empty @secure() parameter, which keeps the stored value;
  • outputs the resource's id.

Deployed back unchanged, an export changes nothing. Change a value, for instance a syslog device's UDP 514 to TCP 1514, and deploy it, and the resource is updated in place.

Export template needs the Deployment read permission and the resource's own read permission. Open in Deploy also needs the Deployment create permission.

Permissions​

Deploy has its own permission scope, Deployment, with two permission sets:

Permission setGrantsBuilt-in roles
Deployment AdminDEPLOYMENT_READ, DEPLOYMENT_CREATEOwner, Admin, Contributor
Deployment ViewerDEPLOYMENT_READUser

DEPLOYMENT_READ opens the list, the detail pages, the template checks and previews, Show template and Export template. DEPLOYMENT_CREATE runs, redeploys and cancels deployments.

A deployment also needs, for every resource it writes, that resource's own read, create and update permissions, the same as the console asks for, and read permission for every existing resource it looks up. These are checked before anything is written: if any is missing, the whole deployment is refused and nothing happens. See Roles.

Audit​

Every deployment is recorded in Audit Logs with the object type Deployment:

  • deployment started, succeeded, failed, canceled and interrupted;
  • cancel requested, naming who asked;
  • template exported.

Each resource the deployment writes also keeps its own audit row, such as a device created or a target updated. No row carries the template's text, a secure parameter's value or a new director's API key.

Deploying Through the API​

The same deployments run through the public REST API with an API token, so a template can be deployed from a CI pipeline or a script. See Deployments API.