OPC UA
Synopsis
Director collects the values of the nodes of one OPC UA server: a PLC, a protocol gateway, a SCADA or historian, an edge server, or anything else a plant exposes through OPC UA. Nothing is installed on the server or beside it: Director connects as any OPC UA client would, over a session it secures as the server's endpoints allow, and writes what it collected as ECS records.
One device, one server, one session. Collecting from forty servers takes forty devices. The device holds one session to its server and keeps it; its reads and its subscriptions share it. A session that is lost is opened again (see Reconnects).
What to collect is declared by datasets. A server's address space is its own, and only its operator knows which nodes matter. So the device's datasets are your own, each a table of named nodes, of one of two types:
- A node dataset (
opcua_node_collector) reads the nodes on an interval, and writes one record of all of them for each reading. - A subscription dataset (
opcua_subscription_collector) has the server push the nodes as they change, and writes one record for each change.
They are one device because they are one server: a plant server is read on an interval for what is slow and steady, and subscribed to for alarms and states that must not wait for the next reading. Both run over the same session, and a device can have any number of either. Events, history, methods, browsing, and writing are not part of the device, and neither is any transport other than opc.tcp.
The device dials the server and binds no port, so any number of OPC UA devices can name port 4840, and none of them blocks a listener. One Director owns each device.
The OPC UA device needs Director 2.3.0 or later. With no Director or cluster at that version selected, the wizard shows OPC UA devices are unavailable on directors or clusters below version 2.3.0 and the device cannot be created.
Schema
- id: <numeric>
name: <string>
description: <string>
type: opcua
tags: <string[]>
pipelines: <pipeline[]>
status: <boolean>
properties:
address: <string>
port: <numeric>
path: <string>
security_policy: <string>
security_mode: <string>
connect_timeout: <numeric>
request_timeout: <numeric>
session_timeout: <numeric>
locales: <string[]>
authentication:
method: <string>
username: <string>
password: <string>
tls:
cert_name: <string>
key_name: <string>
ca_name: <string>
workarounds:
additional_valid_status_codes: <string[]>
use_unregistered_reads: <boolean>
monitored_items_batch_size: <numeric>
definitions: <definition[]>
Configuration
The following fields are used to define the device:
Device
| Field | Required | Default | Description |
|---|---|---|---|
id | Y | - | Unique numeric identifier |
name | Y | - | Device name |
description | N | - | Optional description |
type | Y | - | Must be opcua |
tags | N | - | Optional tags |
pipelines | N | - | Optional pre-processor pipelines |
status | N | true | Enable/disable the device |
Connection
| Field | Required | Default | Description |
|---|---|---|---|
address | Y | - | Host name or IP address of the server. An IPv6 address can be written as it is or in brackets. A wildcard address (0.0.0.0, ::) is refused, and so is a port or a path written into the address: each is its own field. |
port | N | 4840 | The server's port, from 1 to 65535 |
path | N | - | The endpoint's path, such as /OPCUA/SimulationServer. A leading slash is added when it is missing. Letters, digits, and the characters a URL path can hold. |
security_policy | N | auto | auto, None, Basic128Rsa15, Basic256, Basic256Sha256, Aes128_Sha256_RsaOaep, or Aes256_Sha256_RsaPss, in any case, or a policy's URI. See Endpoint Selection. |
security_mode | N | auto | auto, None, Sign, or SignAndEncrypt. See Endpoint Selection. |
connect_timeout | N | 5 | Seconds the discovery of the server's endpoints, the connect, and the reading of the server's limits wait, from 1 to 300 |
request_timeout | N | 10 | Seconds each request waits for its answer, from 1 to 300 |
session_timeout | N | 1200 | Seconds the server keeps a session that has no activity, from 1 to 86400. The server may shorten it. |
locales | N | - | The languages the server should use for text, in order of preference, such as [en, de]. At most 8. |
Authentication and Certificates
| Field | Required | Default | Description |
|---|---|---|---|
authentication.method | N | Anonymous | Anonymous, UserName, or Certificate, in any case |
authentication.username | N* | - | The user (resolvable via ${ENV_VAR} or $secret{...}) |
authentication.password | N | - | Its password (resolvable via ${ENV_VAR} or $secret{...}). A server may accept a user with none. |
tls.cert_name, tls.key_name | N* | - | The client's application instance certificate and its private key: PEM given inline, or the name of a file under the Director's root. Both or neither. See The Client Certificate. |
tls.ca_name | N | - | The server's certificate: PEM inline, or a file under the Director's root. The channel is encrypted for it instead of the certificate the server's endpoint presents. See What Is Authenticated. |
* UserName requires authentication.username. Certificate requires tls.cert_name and tls.key_name: the certificate is the user.
A device that fails these checks, or any bound on this page, does not start, and its state is Configuration error; the message names the field.
Credentials are read when the device connects, not when its configuration is read: at the first connect, and at every one after, from where they are kept. A connection opened again after a password was rotated in the secret store uses the new one, and a secret store that cannot be reached when the configuration is read does not stop the device. A variable that is not set, or a secret that cannot be read, is an error of the connection, with the state Authentication error, and never a login with an empty password. The device compares its configuration for changes as it holds it (a secret's name, not its value), so a value that changed restarts nothing.
Workarounds
workarounds is an object of settings for servers that break the specification. Each is off by default.
| Field | Required | Default | Description |
|---|---|---|---|
workarounds.additional_valid_status_codes | N | - | Status codes of another severity (Uncertain, Bad) that count as a good value, for servers that report values they mean to be read with one of them, such as ["0x40900000"] (Uncertain_LastUsableValue). Hexadecimal or decimal, at most 64. Every status of the good severity (Good, and the ones that say more of a good value, such as GoodOverload and GoodClamped) counts without being listed. |
workarounds.use_unregistered_reads | N | false | Read the nodes as they are, without registering them first, for servers that do not support registering |
workarounds.monitored_items_batch_size | N | 0 | Create the monitored items of a subscription in requests of this many, from 0 to 100000, for servers that drop the connection when a request exceeds their message size. 0 sends them in one request. |
Connection
Endpoint Selection
The device asks the server for its endpoints at the URL it is configured with, and picks the one the configuration asks for. The candidates are the endpoints that:
- speak UA-TCP (
opc.tcp); the HTTPS and WebSocket endpoints a server lists beside them are not candidates; - match
security_policyandsecurity_mode(automatches all); and - accept the kind of user the device identifies as (anonymous, a user name, or a certificate).
Of those, the one with the highest security level is chosen. An endpoint that would take the connection but not the user cannot hold a session, so it is not chosen where another is.
A channel is secured by a policy and a mode together, or by neither, so the two cannot contradict each other:
Nonefor one with the other left atautoisNonefor both, and secures nothing.Nonefor one and a real value for the other, such assecurity_policy: Nonewithsecurity_mode: SignAndEncrypt, is a Configuration error. Taking either as the intended meaning would connect to a channel that was not asked for.
When no endpoint fits, the log says what was asked for and what the server offers, which is what it takes to change one into the other:
the server has no endpoint for security policy Basic256Sha256, security mode SignAndEncrypt and
Anonymous identity; it offers None/None (Anonymous); Basic256Sha256/Sign (Anonymous, UserName)
The Client Certificate
A channel that secures something is signed with an application instance certificate of the client, which the server has to trust. Give the device one in tls.cert_name and tls.key_name: an RSA key and its certificate. OPC UA requires the certificate to name the application by a URI in its subject alternative name, and a server compares it with the application URI the client presents. The device presents itself as the application its certificate names, so the two cannot differ.
Without a certificate, the device makes a self-signed one when it first meets an endpoint that needs it: RSA 2048, valid for a year, naming the application urn:virtualmetric:opcua:client, with the Director's host name. It lives only as long as the device runs, and is made again 30 days before it expires, at the next connect, on a device that runs that long. The server has to be told to trust it again after every start and every renewal, which is why the log warns that the device has no certificate of its own. A Certificate user cannot use a generated certificate, since the server has to know the user.
A server that does not trust the client's certificate often says nothing: it closes the connection as the secure channel opens. On an endpoint that secures something, the device reads that as a Certificate error, and its log says what to check: add the certificate to the server's trusted certificates, and check that its subject alternative name holds the application URI.
The Password
A UserName user's password is encrypted with the security policy of the endpoint's user token policy, or with the channel's when the token policy names none. A policy of None protects nothing: sent to an endpoint of no security whose token policy does not encrypt it either, the password crosses the network as it is.
The device does not send a password in the clear unless it is told to. The list of endpoints is discovered over a channel that secures nothing, so a server could otherwise make an unencrypted endpoint the best choice. An endpoint whose user token policy does not encrypt the password is not chosen for a UserName user unless security_policy or security_mode names None. When the server offers no other, the connection fails with an error that says so, and what to change. When None is named, the log says once, at connect, that the password reaches the server as it is.
An encrypted password is encrypted for the certificate the answering server presents in its CreateSession response. What that certificate proves is covered next.
What Is Authenticated
The device does not check the server's certificate: not against a list of trusted certificates or an issuing authority, and not its validity, host name, or application URI. It encrypts for the certificate the list of endpoints names, and that list is discovered over a channel that secures nothing. Anyone between the server and the device can answer the discovery with an endpoint carrying a certificate of their own; the channel is then secured with it, and a password is encrypted for it.
tls.ca_name narrows this:
- It secures only a channel of mode
SignorSignAndEncrypt. The device encrypts its part of the channel's keys for that certificate, so only the holder of its private key can derive the keys and answer over the channel. The connection fails when whoever answers does not hold that key, whatever certificate the endpoint list names. - It does nothing for a channel of mode
None, whichsecurity_mode: autocan choose when it is the best the list offers (for aUserNameuser, one whose user token policy encrypts the password). - Without
tls.ca_name, or on a channel of modeNone, a server that is not the right one can take the connection and the password.
Only security_mode Sign or SignAndEncrypt, so that no endpoint of no security is chosen, together with tls.ca_name keeps a device from connecting to a server that is not its own. Short of that, a password is kept from being read on the network, not from being sent to the wrong server.
One Session
The device connects at its start and keeps the session until something goes wrong. Reads and subscriptions share it, and requests run at the same time, as OPC UA allows. At connect, the device also reads:
- the server's namespaces, which turn a node written by the URI of its namespace (
nsu=urn:plant;s=Temperature) into the index the server knows it by; and - the server's operation limits, the most nodes a read or a registration can carry, by which the device splits its requests. A server that does not say has no limit the device knows of.
A server that refuses its namespaces with a status that says they cannot be read (the user may not read them, they are not readable, or they are not there) is still read by the indexes of its nodes. The log says so and why, as a warning, once for as long as it lasts, and nodes named by the URI of their namespace are left out until the next connect. Any other failure to read them (a timeout, a server with no resources for the request) fails the connect, which is retried.
Reconnects
A connection that fails or is lost is opened again by the device. The wait before each attempt is one second at first, and doubles up to thirty seconds while the server cannot be reached; a connection that lasted a minute starts the wait again from one second. Every attempt resolves the credentials, discovers and selects the endpoint, resolves every dataset's nodes against the namespaces the server has now, registers the nodes, and creates the subscriptions again.
A connection ends when:
- the library reports it gone: the server closed it, or the network dropped it;
- a request fails with a status that says the session or its channel is gone at the server (an invalid or closed session or channel, a server that halted or shut down, no communication). A request the server refuses with a status of its own, such as a service it does not support or a node it does not know, fails that request alone, and the connection stays;
- three rounds of readings in a row go unanswered (a timeout, or an error of the transport or of the secure channel). A round is the datasets that fall due together, however many they are, and it is unanswered when the server answered none of them. A reading the server refuses with a status of its own for the request, such as a response too large for it or too many nodes for one read, is an answer: it fails that dataset alone (see Reading);
- on a device that subscribes, two liveness checks in a row go unanswered. Every 30 seconds the device reads the server's state and checks that the server still has its subscriptions. A subscription the server says it no longer has ends the connection at once; a server that does not support the check is taken to have them;
- a subscription ends at the server: its status changes because it ran out of keep-alives or was moved to another session.
A dataset the server will not take is left out, and the rest of the device runs. A node that does not resolve is left out of its table; a dataset none of whose nodes resolve is left out until the next connect, and so is a subscription the server will not create, or one of whose nodes it will not monitor. Each is logged once for as long as it lasts.
Stopping
A stop ends what is in flight at once, closes the session (waiting at most two seconds for a server that does not answer, whatever the request timeout is), and flushes the queues; only then is the device stopped. A restarted device therefore never has its predecessor still writing beside it. This includes a connect that a silent server holds open: the stop does not wait for it, however long connect_timeout is. A flush that takes longer than 15 seconds is logged as a warning, and the device waits for it all the same.
Datasets
The device's definitions are its datasets. In the console they are the datasets and profiles assigned to the device in its
| Dataset | Definition | Definition ID | Kind | One record per |
|---|---|---|---|---|
node | opcua_node_collector | 15150 | metric | reading of the table's nodes (one record of all of them) |
subscription | opcua_subscription_collector | 15151 | metric | change of one node of the table |
- Any number of datasets of each type. Each dataset is a table of its own, so a device collects as many as are assigned: for example, the readings of a press, of the drive beside it, and the alarms of both. Two datasets can name the same table; their values then sit under one name, and a node named alike in both should have one type.
- Pipelines. A dataset's records enter the pre-processing pipeline the dataset selects, or the device's when it selects none, so two datasets on one device can reach two pipelines.
- Live capture names the dataset each record came from.
- Nothing assigned is allowed. The device then only checks, every minute, that its server accepts a connection (closing it at once), so its connection state stays current, and writes nothing.
- A dataset that cannot be read is left out, with a warning in the device's log that names it and says why. The rest of the device runs.
Dataset Properties
| Field | Applies to | Required | Default | Description |
|---|---|---|---|---|
table | both | Y | - | The name the dataset's values sit under in a record (see Records): letters, digits, and underscores, at most 64, not starting with a digit. table and endpoint are reserved. |
interval | node | N | 60 | Seconds between readings, 1 or more |
subscription_interval | subscription | N | 1000 | How often the server publishes what changed, in milliseconds, from 10 to 3600000 (an hour). A whole number, or a duration such as 250ms or 2s. The server may revise it. |
timestamp | both | N | gather | The time a record is dated at: gather (when Director received it), server (the time the server stamped the value with), or source (the time the value's source did) |
data_type | both | N | false | Write the OPC UA data type of each value beside it, as <node>_type |
labels | both | N | - | An object of names and values written on every record as ECS labels. At most 32; a name is letters, digits, underscores, and hyphens, and a value at most 256 characters. A numeric value is written as it was typed (1000000, not 1e+06). |
pipeline_name | both | N | the device's | The pre-processing pipeline the dataset's records enter |
Timestamps. A subscription's record, which holds one node, is dated at that node's time of the kind asked for. A reading of several nodes is dated at the newest of their times that has one. A value that carries no time of that kind, or the OPC UA epoch (1601) that a server which stamps nothing sends, is dated when it was received.
Nodes
A dataset's inputs are its nodes, at most 2048. Each needs a name, what its value is called in a record (letters, digits, and underscores, at most 64, unique in the table), and the node's identity, written whole or in parts, not both:
| Field | Description |
|---|---|
id | The node ID as OPC UA writes it: ns=2;s=Temperature, ns=3;i=1042, i=2258 (namespace 0), ns=1;g=<guid>, or ns=1;b=<base64>. A bare name is refused, since it would be a string in namespace 0. |
id by namespace URI | nsu=http://plant.example/;s=Temperature. The URI is looked up in the server's namespaces at every connect, so a server that restarts with another numbering is still read correctly. |
namespace, identifier_type, identifier | The parts: the namespace's index (0 when left out); s (string), i (numeric), g (GUID), or b (opaque); and the identifier. |
namespace_uri, identifier_type, identifier | The same, with the namespace given by its URI |
Monitoring
A subscription dataset's nodes take monitoring_params, all of it optional. A node dataset ignores it.
| Field | Default | Description |
|---|---|---|
sampling_interval | 0 | How often the server checks the node, in milliseconds or a duration, up to an hour. 0 is as fast as the server can. |
queue_size | 10 | How many changes the server keeps for the node between two publishes, from 1 to 100000 |
discard_oldest | true | Which change the server drops when its queue is full: the oldest (true) or the newest (false) |
data_change_filter.trigger | StatusValue | What counts as a change: Status, StatusValue (the server's default), or StatusValueTimestamp |
data_change_filter.deadband_type | None | None, Absolute (a change smaller than the amount is not notified), or Percent (of the node's engineering range) |
data_change_filter.deadband_value | - | Required for a deadband: the amount, not negative, and at most 100 for Percent |
A filter that asks for what the server does anyway (StatusValue and no deadband) is not sent.
Reading
A node dataset is read as soon as it is due: at the start, and then every interval, counted from the start of one reading to the start of the next. A reading that takes longer than its interval delays the next, which starts 100 ms after it ends. Datasets that fall due together are read one after the other over the same session.
- One request for a dataset's nodes, split by the server's
MaxNodesPerReadlimit when it has one. The nodes are registered with the server first, once per connection, in requests the size itsMaxNodesPerRegisterNodeslimit allows; a server reads a registered node faster. Registered node IDs are used for reads only. A server that will not register is read without it, with a warning, anduse_unregistered_readsskips the registration. Only a node's value is read, with both of its timestamps, and the server may answer from a value up to two seconds old. - Good values. A node's value counts when its status is of the good severity: every status whose top two bits are clear, which is Good and the statuses that say more of a good value (such as
GoodOverloadandGoodClamped), or one ofadditional_valid_status_codes. A node with any other status (Uncertain, Bad), with no value, or whose data type cannot be written to a record (a structure) is left out of the record. This is logged once for as long as it lasts, and again when the node is written again. A record none of whose nodes has a good value is not written. The server having answered is what shows it is reachable, so the state stays Connected while nodes are the trouble. - A reading that fails writes no record, and is logged once for as long as the reason lasts, with a recovery logged after. A reading the server refuses with a status for the request itself, such as a response too large for it or too many nodes for one read, is its answer: it fails that dataset alone, and the state stays Connected however often the dataset falls due. A reading the server does not answer (a timeout, or an error of the transport or of the secure channel) counts toward a reconnect: three rounds in a row in which the server answered none of the datasets due, or a failure whose status says the session is gone, end the connection (see Reconnects).
Subscriptions
A subscription dataset creates one OPC UA subscription at the server, publishing every subscription_interval, and one monitored item for each node, watching the node's value with the node's monitoring (see Monitoring). Each change the server notifies is one record. The server may revise the intervals to what it can do.
- After a reconnect the subscriptions are created again, and the server publishes the nodes' current values first, so a record follows a reconnect for each node the server has a value for.
- A notification of a node that has no good value writes no record, as in a reading, and is logged once.
- A publish request the server refuses with a status of its own (it has no resources for it, for example) is logged, and does not end the subscriptions: the next publish request is sent a second later, and the changes the server kept meanwhile follow as soon as it takes one. A run of refusals is logged once, however long it lasts, until the server takes a publish request again.
- A subscription whose status changes is over at the server, and ends the connection.
- Pipelines for a subscription's records are built when the first record is written and again every minute, so a change to a pipeline reaches the device within a minute.
Notifications are handled on their own, so a slow reading cannot hold up the server's publishing.
Records
Every record is one ECS 8.11 document with the following fields:
| Field | Value |
|---|---|
@timestamp | The record's time, as the dataset's timestamp sets |
ecs.version | 8.11.0 |
event.module | opcua |
event.dataset | opcua.node or opcua.subscription |
event.kind | metric |
event.category, event.type | host, info |
observer.name | The device's name in the console |
observer.hostname | The Director's host name |
observer.vendor, observer.product | VirtualMetric, OPC UA |
server.address, server.port | The address the device is configured with, and its port |
server.ip | The address, when it is an IP address |
network.protocol, network.transport | opcua, tcp |
labels.<name> | The dataset's labels, when it has any |
opcua.<node|subscription>.table | The dataset's table |
opcua.<node|subscription>.endpoint | The endpoint's URL, without a credential |
opcua.<node|subscription>.<table>.<node> | The value of each node that was written |
opcua.<node|subscription>.<table>.<node>_type | The node's OPC UA data type, when the dataset has data_type on |
The values sit under the table's name, so two tables can name a node alike with different types without one mapping conflicting with the other. With data_type on, no node can be named like another node's type field (speed and speed_type): the dataset is left out.
Values
| OPC UA type | Written as |
|---|---|
| Boolean | true or false |
| SByte, Byte, Int16, UInt16, Int32, UInt32, Int64, UInt64 | A number |
| Float | A number, as the decimal it was typed as: 0.1, not the nearest double |
| Double | A number. NaN and infinities are left out, since JSON has neither, and this is logged. |
| String, XmlElement | Text |
| DateTime | RFC 3339, in UTC |
| Guid, NodeId, ExpandedNodeId | Their text; a GUID in lower case |
| ByteString | Base64 |
| LocalizedText, QualifiedName | The text, and the name without its namespace |
| StatusCode | Its number |
| An array of any of these | An array of them, with the type written as, for example, Double[]. A multi-dimensional array is written flat, row by row. An array of more than 1024 elements in all is left out, and this is logged once. |
| ExtensionObject (a structure), DataValue, Variant, DiagnosticInfo | Not written: the node is left out of the record, and this is logged once |
Device State
The connection state follows the server:
| State | When |
|---|---|
| Connected | The server answered: a reading was made, or refused with a status of the server's own, or the subscriptions were created. Nodes without a good value do not change it. |
| Connection error | The server cannot be reached; a reading went unanswered and no other dataset was answered; the connection was lost; or none of the datasets could be set up (the device closes the connection and connects again to try again) |
| Certificate error | The server refused the client's certificate: untrusted, not yet valid, a host name or URI that does not match, or revoked. The log says to add it to the server's trusted certificates. |
| Authentication error | The server refused the user, or the credentials could not be read: an environment variable that is not set, or a secret that cannot be fetched |
| Configuration error | The configuration cannot run (see Configuration) |
A dataset that fails while another is read is not an error of the connection, and neither is a dataset the server refuses with a status of its own, a record that could not be written, a failure inside one dataset, or a stop.
The OPC UA device is not simulated. A dataset's records are the values of your own nodes, a shape a generated sample cannot stand in for.
Console
In the console, go to
General settings , as on every device.Connection settings :Address ,Port , andEndpoint path (Optional) , then three sections.Security :Security policy andSecurity mode , bothAuto by default (see Endpoint Selection), andServer certificate (Optional) , which istls.ca_name(see What Is Authenticated).Authentication :Authentication method , one ofAnonymous ,Username and password , orCertificate .Username and password showsUsername andPassword (Optional) .Client certificate :Certificate (Optional) andPrivate key (Optional) , the client's application instance certificate and its key, both or neither (see The Client Certificate). TheCertificate method requires both.
Advanced settings , in two sections.Session :Connect timeout ,Request timeout ,Session timeout , andLocales (Optional) .Workarounds :Additional valid status codes (Optional) ,Monitored items batch size (Optional) , andUnregistered reads (see Workarounds).
Data collection : the OPC UA datasets the device collects.Review and create .
With
The device's page has the same tabs, and
An OPC UA dataset is created in opcua_node_collector) or opcua_subscription_collector); see Management. Both have
Each node has a
Examples
Open Server, Read and Subscribed
Reading four nodes of an open server every 10 seconds, with their data types, and subscribing to two others, one with a deadband... | |
Secured Server with a User
Connecting signed and encrypted, with a user whose password comes from the environment, the client's own certificate, and the server's certificate pinned so that only it can answer. Nodes are named by the URI of their namespace, which survives a server restart that renumbers namespaces... | |
Reachability Only
With no dataset assigned, the device only checks every minute that the server accepts a connection, and writes nothing... | |