Juniper JTI

Pull

Synopsis

Director subscribes to the sensors of one Junos device over Juniper's gRPC telemetry interface, JTI (the OpenConfigTelemetry service), as Telegraf's jti_openconfig_telemetry input does. It dials the device, signs in through its Login service, holds a subscription for every dataset, and writes what Junos streams as ECS records: interface counters, line card CPU and memory, the routing engine's state, BGP and LSPs, and whatever else the device's sensors expose. Nothing is installed on the Junos device.

One device, one Junos device. Monitoring forty routers takes forty devices. A Junos release that serves gNMI can be read by the gNMI device too; the Juniper JTI device is for the releases and the setups that stream over JTI. Junos's native sensors, which a line card exports over UDP, are not part of this device.

What to subscribe to is declared by datasets. A dataset is one group of sensors: the sensor paths (/interfaces/, /junos/system/linecard/cpu/memory/), how often Junos samples them, and the table their values are written to. A device runs any number of datasets, a subscription each, all of them on its one connection.

The device dials its Junos device and binds no port, so Juniper JTI devices never collide on one. One Director owns each device: two would subscribe twice, write every value twice, and share one client ID.

note

The Juniper JTI device needs Director 2.3.1 or later. With no Director or cluster at that version selected, the wizard shows Juniper JTI devices are unavailable on directors or clusters below version 2.3.1 and the device cannot be created.

Schema

- id: <numeric>
  name: <string>
  description: <string>
  type: juniper_jti
  tags: <string[]>
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    address: <string>
    port: <numeric>
    username: <string>
    password: <string>
    client_id: <string>
    tls:
      status: <boolean>
      server_name: <string>
      ca_name: <string>
      cert_name: <string>
      key_name: <string>
      passphrase: <string>
      insecure_skip_verify: <boolean>
      min_tls_version: <string>
      max_tls_version: <string>
    redial: <numeric>
    keepalive_time: <numeric>
    keepalive_timeout: <numeric>
    max_msg_size: <string|numeric>
    timestamp_source: <string>
  definitions: <definition[]>

Configuration

The following fields are used to define the device:

Device

Field	Required	Default	Description
`id`	Y	-	Unique numeric identifier
`name`	Y	-	Device name
`description`	N	-	Optional description
`type`	Y	-	Must be `juniper_jti`
`tags`	N	-	Optional tags
`pipelines`	N	-	Optional pre-processor pipelines
`status`	N	`true`	Enable/disable the device

Connection

Field	Required	Default	Description
`address`	Y	-	Host name or IP address of the Junos device, without a port. A wildcard address (`0.0.0.0`, `::`) is refused, and so is a host and a port written as one. An IPv6 address may carry a zone (`fe80::1%eth0`).
`port`	N	`32767`	The port Junos serves its gRPC services on, from `1` to `65535`. See On the Junos Device.
`username`	N	-	Signs the device in, with `password` and `client_id` (see Signing In). Plain text, `${ENV_VAR}`, or `$secret{...}`. With none, the device signs in with nothing.
`password`	N	-	Plain text, `${ENV_VAR}`, or `$secret{...}`. A password with no username is refused.
`client_id`	N	`virtualmetric-<the Director's host name>`	Printable text, at most 128 characters. Junos keeps a client's subscriptions by its ID, so two collectors that share one take each other's: the default carries the Director's host name for that reason.
`tls`	N	off	How the connection is secured. See TLS.

Subscriptions

Field	Required	Default	Description
`redial`	N	`10`	Seconds the device waits before it subscribes again after a subscription failed or ended, from `1` to `3600`.
`keepalive_time`	N	`600`	Seconds a connection may be quiet before gRPC pings the Junos device, from `0` to `86400`; `0` turns the pings off. It decides how soon a connection whose far end is gone is noticed (see Lost Connections). A gRPC server refuses pings sent more often than it allows: keep it above five minutes.
`keepalive_timeout`	N	`20`	Seconds gRPC waits for the ping's answer before it gives the connection up, from `0` to `86400`.
`max_msg_size`	N	`4MB`	The largest message taken from the Junos device: a number of bytes, or a size such as `16MB`, from 4 KiB to 1 GiB. 4 MiB is gRPC's own default.
`timestamp_source`	N	`data`	What a record is dated by: `data`, the time Junos stamped its batch with, or `collection`, the time the batch arrived (see Time).

A configuration error, such as no address, a value out of its range, a value that is not a number or a switch, a TLS version that does not exist, or a certificate without its key, stops the device at start with the connection state Configuration error, and the message names the field. A switch is true or false, or text that says so; anything else is refused, never read as false.

Connection

On the Junos Device

JTI is served by the Junos device's gRPC extension service. With TLS:

set system services extension-service request-response grpc ssl port 32767

Or, for a device whose tls.status is false, in clear text:

set system services extension-service request-response grpc clear-text port 32767

Signing In

Before each subscription, a device with a username calls the Junos device's Login service with the username, the password, and the client ID, and then sends the three as the username, password, and clientid metadata of the subscription.

A refused sign-in fails the subscription with the state Authentication error.
A Login that does not answer within ten seconds fails the subscription with Connection error.
A Junos device with no Login service, one that answers Unimplemented, is subscribed to all the same.
The credentials are read at every subscription, so a password rotated in the vault is sent from the next one on, and restarts nothing.
Credentials that cannot be read, such as a variable that is not set or a secret store out of reach, fail the subscription with Authentication error, and nothing is sent. The message names the field, never its value.
gRPC sends metadata of printable ASCII alone, so a username or a password with any other character is refused, saying so.

warning

A username sent over a connection TLS does not secure is sent in cleartext. The device says so in its log when it starts, and runs. Send credentials without TLS only where the network between Director and the Junos device is trusted.

TLS

TLS is on when tls.status is true. The Junos device's certificate is then checked, against tls.ca_name or the system's trust store, unless tls.insecure_skip_verify is on.

Field	Required	Default	Description
`status`	N	`false`	Secures the connection
`server_name`	N	the address	The name the certificate is checked against
`ca_name`	N	the system's trust store	The authority of the Junos device's certificate: PEM inline, a file under the Director's root, `${ENV_VAR}`, or `$secret{...}`
`cert_name`, `key_name`	N	-	A client certificate and its key, for a Junos device that identifies its clients by one. Both or neither.
`passphrase`	N	-	For an encrypted key
`insecure_skip_verify`	N	`false`	Skips the check of the Junos device's certificate
`min_tls_version`, `max_tls_version`	N	Go's	`tls1.0`, `tls1.1`, `tls1.2`, or `tls1.3`; another value is refused

With tls.status off, the other tls fields are not used, and a device that names a CA or a certificate says so in its log when it starts.

The TLS material is read when the device starts, and material that cannot be read then stops it with Configuration error. The Director reads it again whenever it compares the device's configuration. Material that changed where it is kept restarts the device. Material that cannot be read at such a check, with the settings unchanged, restarts nothing: the device goes on with what it read, and says so in its log every ten minutes for as long as it lasts.

Lost Connections

A subscription that fails or ends is asked for again after redial seconds, for as long as the device runs, and the device's other subscriptions go on. This covers a Junos device that cannot be reached, refuses the sign-in or the subscription, or ends the stream, and a connection that is lost.

A connection whose far end is gone without a word, such as a router that rebooted or a NAT entry that expired, is noticed by gRPC's pings. With nothing from the Junos device for keepalive_time seconds, it is pinged, and the connection is given up when no answer comes within keepalive_timeout. With the pings off, only the operating system's TCP keepalive notices a dead connection, two hours of silence later.

Datasets

The device's definitions are its datasets. In the console they are the datasets and profiles assigned to the device in its Data collection; see Datasets and Profiles. In a configuration file each dataset is a definition named juniper_jti_sensor_collector, the device's one dataset type:

Dataset	Definition	Definition ID	Kind	One record per
`sensor`	`juniper_jti_sensor_collector`	`21150`	metric	batch and list entry

Any number of datasets, at most 128, a subscription each, all on the one connection.
Pipelines. A dataset's records enter the pre-processing pipeline the dataset selects, or the device's when it selects none.
Nothing assigned is allowed. The device then checks every minute that its Junos device answers, and writes nothing, so its connection state follows the Junos device. It signs in through the Login service when it has a username, and opens a TCP connection to the port when it has none. The check is given ten seconds.
A dataset that cannot be read is left out, with a warning in the device's log that names it: no table, a sensor without its leading slash, or a value out of its range. The rest of the device runs. With every dataset assigned to it left out, the device has nothing to subscribe to, and stops with Configuration error.

Dataset Properties

Field	Required	Default	Description
`table`	Y	-	Names the record's values: they sit under `juniper_jti.<table>`. Letters, digits, and underscores, at most 64, starting with a letter or an underscore, and not `sensor`, which is taken by what the record is.
`sensors`	Y	-	The sensor paths, as Junos takes them, each beginning with a slash: `/interfaces/`, `/junos/system/linecard/cpu/memory/`. A list, or text with one path per line. At most 64, of at most 1024 characters each; a path given twice is subscribed to once.
`interval`	N	`10`	Seconds between samples, from `1` to `86400`. Junos takes it in milliseconds, as each sensor's sample frequency.
`pipeline_name`	N	the device's	The pre-processing pipeline the dataset's records enter

From a Batch to Records

Junos streams a sensor as batches: the values of one component, each a key and a typed value. A key is a path, written whole or below the __prefix__ key Junos sends before it:

__prefix__ = /interfaces/interface[name='ge-0/0/0']/
state/counters/in-octets = 81236752
state/counters/out-octets = 41986720

A Record per List Entry

The values of a batch are grouped by the keys of the list entries on their paths: the values of ge-0/0/0 are one record, those of ge-0/0/1 another. The keys are the record's labels, so an interface's name is juniper_jti.sensor.labels.name. A key whose name an entry above already took is named after its element too: a protocol's name under a network instance's is protocol_name.

Field Names

A value is named after its path below the dataset's sensor. The elements of the sensor path are not repeated in every field, and the rest nest under the table, with an underscore for every hyphen and dot. With the sensor /interfaces/ and the table interfaces, in-octets above is juniper_jti.interfaces.interface.state.counters.in_octets. A path under none of the dataset's sensors, such as one Junos's own sensors stream (/junos/system/linecard/...), keeps all of its elements.

Values keep their type: a 64-bit counter is a number, a string a string, and bytes are base64. A NaN or an infinity is left out, since a record is JSON. When a value and an object would take one place, both are kept, the value in the object under _value.

Junos's own keys about the batch (__timestamp__, __junos_re_stream_creation_timestamp__, __junos_re_payload_get_timestamp__) go under juniper_jti.sensor.meta, named without their underscores.

Deletions

A path Junos says was deleted is a record of its own, a deletion event (see Records).

Time

A record is dated at its batch's own timestamp, in milliseconds. A batch dated before 2000 or after 2200, such as one with no time at all or one sent in seconds, is dated when it arrived. With timestamp_source: collection, every record is dated when its batch arrived.

Records

Every record is one ECS 8.11 document. These fields are on every record:

Field	Value
`@timestamp`	The batch's time (see Time)
`ecs.version`	`8.11.0`
`event.module`	`juniper_jti`
`event.dataset`	`juniper_jti.sensor`
`observer.name`	The device's name in the console
`observer.hostname`	The Director's host name
`observer.vendor`, `observer.product`	`VirtualMetric`, `Juniper JTI`
`server.address`, `server.port`	The Junos device's address and port, as configured
`server.ip`	The address, when it is an IP address
`network.protocol`, `network.transport`	`grpc`, `tcp`
`juniper_jti.sensor.table`	The dataset's table
`juniper_jti.sensor.path`	The sensor that produced the batch, as Junos names it (`sensor_1000:/interfaces/:/interfaces/:PFE`)
`juniper_jti.sensor.system_id`	The device that sent it, as Junos names it: its host name and the line card or routing engine
`juniper_jti.sensor.component_id`, `juniper_jti.sensor.sub_component_id`	The component that produced it
`juniper_jti.sensor.sequence`	The batch's sequence number
`juniper_jti.sensor.labels.<name>`	The keys of the list entries on the path (see A Record per List Entry)
`juniper_jti.sensor.meta.<name>`	Junos's own keys about the batch

A record of values, what a batch sent for one list entry, also has:

Field	Value
`event.kind`	`metric`
`event.category`, `event.type`	`host`, `info`
`juniper_jti.<table>.<field>`	The values (see Field Names)

A deletion has instead:

Field	Value
`event.kind`	`event`
`event.category`, `event.type`	`configuration`, `deletion`
`event.action`	`delete`
`juniper_jti.sensor.deleted`	The path deleted, below the sensor, without its keys, which are its labels

Device State

What happened	State
Junos answered every subscription, or, with no dataset, the check	Connected
The Junos device cannot be reached, the call ran out of time, or the stream was ended or lost	Connection error
The TLS handshake failed: a certificate that does not verify, or a Junos device that does not speak TLS	Certificate error
The `Login` service refused the sign-in, Junos refused the credentials (`Unauthenticated`, `PermissionDenied`), or the credentials cannot be read	Authentication error
Junos refused the subscription (`InvalidArgument`, `Unimplemented`, `NotFound`, `FailedPrecondition`, `OutOfRange`), such as for a sensor it does not have or with JTI not on; or a message was larger than `max_msg_size`	Configuration error
Any other status	Error

With several datasets, the state is the first failure among them, and Connected once none fails. A failure is logged once for as long as it lasts, with what to check, and its end is logged. A batch that cannot be read, or records that could not be written, do not change the device's state: Junos answered.

Configuration Changes

Any change of the device, or of a dataset assigned to it, restarts the device, which signs in and subscribes anew. So does an edit that no longer validates, which the restart then reports as Configuration error. Credentials are compared as they are configured, which is a secret's name and not its value, so a secret rotated in its store is sent from the next subscription on and restarts nothing. TLS material is compared as described in TLS.

note

The Juniper JTI device is not simulated. A dataset's records are the values of the sensors an operator groups, a shape a generated sample cannot stand in for.

Differences from Telegraf

The device is Telegraf's jti_openconfig_telemetry input as a device. Where it differs:

Telegraf	Juniper JTI device
`servers`, several devices to one plugin	One device per Junos device, `address` and `port`
`sensors`, entries of an optional reporting rate, an optional name, and the paths	A dataset per entry: `interval`, `table`, and `sensors`
`sample_frequency`, 1000 ms by default	`interval`, whole seconds, 10 by default
`retry_delay`, 1000 ms; `0` retries nothing	`redial`, 10 seconds by default, as the gNMI device names it; a subscription is always asked for again
`client_id`, `telegraf`	`client_id`, the Director's host name added to the default
`timestamp_source`, `collection` by default	`timestamp_source`, `data` by default: the batch's own time, or the time it arrived when that is not a time
Tags named after the whole path of a key (`/interfaces/interface/@name`), and `str_as_tags`	Labels named after the key (`name`); strings stay values
Fields named after the whole path (`/interfaces/interface/state/counters/in-octets`)	Nested fields below the sensor (`interface.state.counters.in_octets`)
The `device` tag	`server.address` and `server.ip`
`enable_tls`	`tls.status`
`keep_alive_period`, off by default	`keepalive_time` and `keepalive_timeout`, as the gNMI device names them; on by default, every 600 seconds of silence
Deleted paths not written	A deletion event per deleted path

Console

In the console, go to Fleet Management > Devices > Juniper JTI, listed under Protocols beside gNMI, and click Add new device. The wizard's steps are:

General settings, as on every device.
Connection settings:
- Address and Port.
- Authentication: Username (Optional) and Password (Optional), and Client ID (Optional), which shows only once a username is set. Left empty, the Director uses virtualmetric- followed by its host name.
- TLS settings: the TLS encryption switch. Enabled, it shows Skip certificate verification, the CA certificate, Server name (Optional), the minimum and maximum TLS version, and a Client certificate section with Certificate (Optional), Private key (Optional), and Passphrase (Optional).
While a username is set and TLS encryption is disabled, the step shows the warning The credentials are sent in cleartext.
Advanced settings: Redial delay, Maximum message size (Optional), Keepalive time, Keepalive timeout, and Timestamp source (Junos timestamp or Arrival time).
Data collection: the Juniper JTI datasets the device subscribes to.
Review and create.

The device's page has the same tabs, and Data collection.

A Juniper JTI dataset is created in Fleet Management > Datasets as the type Juniper JTI Sensors; see Management. Its settings are Table name, Sensors (one path per line), the sample interval, and the pre-processing pipeline.

Examples

Juniper MX, Interfaces and Line Cards

Subscribing to a Juniper MX's interfaces every 10 seconds, and to its line cards' CPU and memory every minute in one subscription, over TLS with a client ID of its own...

- id: 612400401
  name: mx-01
  description: Juniper MX edge router, interfaces and line cards
  type: juniper_jti
  status: true
  properties:
    address: "192.0.2.41"
    port: 32767
    username: telemetry
    password: ${JTI_PASSWORD}
    client_id: vm-mx-01
    tls:
      status: true
      ca_name: ops-ca.pem
      server_name: mx-01.example.com
    max_msg_size: 16MB
  definitions:
    - name: juniper_jti_sensor_collector
      status: true
      properties:
        table: interfaces
        sensors:
          - /interfaces/
        interval: 10
    - name: juniper_jti_sensor_collector
      status: true
      properties:
        table: linecard
        sensors:
          - /junos/system/linecard/cpu/memory/
          - /junos/system/linecard/npu/memory/
        interval: 60

Synopsis​

Schema​

Configuration​

Device​

Connection​

Subscriptions​

Connection​

On the Junos Device​

Signing In​

TLS​

Lost Connections​

Datasets​

Dataset Properties​

From a Batch to Records​

A Record per List Entry​

Field Names​

Deletions​

Time​

Records​

Device State​

Configuration Changes​

Differences from Telegraf​

Console​

Examples​

Juniper MX, Interfaces and Line Cards​

Synopsis

Schema

Configuration

Device

Connection

Subscriptions

Connection

On the Junos Device

Signing In

TLS

Lost Connections

Datasets

Dataset Properties

From a Batch to Records

A Record per List Entry

Field Names

Deletions

Time

Records

Device State

Configuration Changes

Differences from Telegraf

Console

Examples

Juniper MX, Interfaces and Line Cards