Kubernetes Datasets
Kubernetes datasets define what a Kubernetes device reads from a Kubernetes cluster's API server (dataset type kubernetes). They attach to devices of type kubernetes, and in
Collector Types
There are fourteen datasets, one per data model, each with its own definition id, event.dataset (kubernetes.<dataset>), interval, and pipeline, so it can be searched, routed, and processed on its own. Thirteen are inventories of what the cluster holds, read hourly; the fourteenth is the cluster's events, read every minute.
| Dataset (console title) | Dataset | Definition | Definition id | Kind | Default interval | Minimum |
|---|---|---|---|---|---|---|
| Kubernetes CSI Drivers | csi_driver | kubernetes_csi_driver_collector | 23100 | inventory | 3,600 s | 60 s |
| Kubernetes Namespaces | namespace | kubernetes_namespace_collector | 23101 | inventory | 3,600 s | 60 s |
| Kubernetes Nodes | node | kubernetes_node_collector | 23103 | inventory | 3,600 s | 60 s |
| Kubernetes Persistent Volume Claims | persistent_volume_claim | kubernetes_persistent_volume_claim_collector | 23104 | inventory | 3,600 s | 60 s |
| Kubernetes Pods | pod | kubernetes_pod_collector | 23105 | inventory | 3,600 s | 60 s |
| Kubernetes Service Accounts | service_account | kubernetes_service_account_collector | 23107 | inventory | 3,600 s | 60 s |
| Kubernetes Services | service | kubernetes_service_collector | 23108 | inventory | 3,600 s | 60 s |
| Kubernetes Deployments | deployment | kubernetes_deployment_collector | 23110 | inventory | 3,600 s | 60 s |
| Kubernetes DaemonSets | daemon_set | kubernetes_daemon_set_collector | 23111 | inventory | 3,600 s | 60 s |
| Kubernetes StatefulSets | stateful_set | kubernetes_stateful_set_collector | 23112 | inventory | 3,600 s | 60 s |
| Kubernetes Ingresses | ingress | kubernetes_ingress_collector | 23113 | inventory | 3,600 s | 60 s |
| Kubernetes Role Bindings | role_binding | kubernetes_role_binding_collector | 23114 | inventory | 3,600 s | 60 s |
| Kubernetes Cluster Role Bindings | cluster_role_binding | kubernetes_cluster_role_binding_collector | 23115 | inventory | 3,600 s | 60 s |
| Kubernetes Events | event | kubernetes_event_collector | 43 | log | 60 s | 10 s |
Each dataset's definition id sits at its Red Hat OpenShift twin's offset: kubernetes.node (23103) is openshift.node (11103).
There is no metric dataset. The nodes', pods', and containers' usage and status are read on every node by the Linux agent's Kubernetes Metrics dataset, and the pods' logs by its Kubernetes Pod Logs dataset. Assign those to the Linux devices on the cluster's nodes for usage and logs.
The account decides what reads. A dataset the device's account may not list is refused and reads nothing, while the device stays Connected and names it. The built-in view cluster role is refused the nodes, the CSI drivers, the role bindings, and the cluster role bindings. For the role that reads all fourteen, see the Account section of the Kubernetes device.
Every dataset takes these properties:
| Field | Required | Type | Default | Description |
|---|---|---|---|---|
interval | N | int | per dataset | Seconds between readings. At least the dataset's minimum above. |
pipeline_name | N | string | the device's | The preprocessing pipeline the dataset's records enter. |
initial_lookback | N | int | the device's | event only. Seconds before its first reading where the dataset starts when it has no checkpoint. 0 starts at the first reading. |
Wizard fields
Every Kubernetes dataset's
Rules
- Intervals. An interval under the dataset's minimum is left out with a warning in the device's log, and the rest of the device runs. The inventories are read hourly by default, the events every minute.
- One of each. A dataset type is read once per device, however many datasets of it are assigned. The first decides its settings; the others are left out with a warning.
- Pipelines. A dataset's records enter the pipeline the dataset selects, otherwise the device's. Two datasets on one device can reach two pipelines.
- Large clusters. An inventory writes every object of its kind at every reading, changed or not. On a cluster of thousands of pods, give the pod dataset the interval its changes need, not a shorter one.
- Namespaces and labels. The device's
namespacesandlabel_selectornarrow the namespaced datasets. Nodes, namespaces, CSI drivers, cluster role bindings, and events are not narrowed by the label selector.
Inventories
Inventory datasets write every object at every reading. From a dataset's second reading on, each record carries event.action:
event.action | event.type | Meaning |
|---|---|---|
added | creation | Not in the last delivered reading |
changed | change | Its fields differ from the last delivered reading's |
unchanged | info | As it was |
removed | deletion | Gone: written once, as it was last seen, dated at the reading |
- Objects are matched by their uid. An object deleted and made again under the same name is a new one.
- The baseline is the last delivered reading. A reading that fails, or whose records could not all be written, never becomes the one the next compares with. A list is read whole or not at all, so nothing is marked removed because a request failed.
- What moves while an object stays the same is written but not compared, such as a node's readiness, a pod's phase and restarts, or a workload's status. Each table below says which fields these are. A new image, a cordoned node, a namespace held to another Pod Security level, a claim bound to its volume, a pod bound to a node, a scaled workload, and a binding given another subject are changes.
- Where an object sits is compared by the names its own record gives, such as its namespace and its node, never by what the discovery found. A node moved to another zone, or a failed discovery, marks nothing.
- The first reading after a start marks nothing: the baseline is kept in memory.
Every object's own record has its id (its uid), name, and created under kubernetes.<dataset>.
Nodes, Namespaces, and Pods
| Dataset | One record per | Fields |
|---|---|---|
node | Node | host.name, host.hostname, host.id (its system UUID), host.ip, host.architecture, host.os.full/kernel/type, cloud.availability_zone/region/machine.type; kubernetes.node.roles (control-plane, worker, from its node-role.kubernetes.io/* labels), unschedulable, provider_id (aws:///eu-west-1a/i-…, azure:///…, gce://…), pod_cidrs, taints, os.image/kernel/type, architecture, kubelet_version, kube_proxy_version, container_runtime, machine_id, system_uuid, boot_id, kubelet_port, zone, region, instance_type, capacity and allocatable (cpu, memory, pods, ephemeral_storage, hugepages.<size>), addresses, conditions, ready. Not compared: ready, conditions, boot_id. |
namespace | Namespace, or each named one | kubernetes.namespace.phase (Active, Terminating), pod_security.enforce/audit/warn (the Pod Security Admission levels its labels set: privileged, baseline, restricted; only enforce rejects a pod that violates its level, while audit and warn record or warn of it and admit the pod) and enforce_version/audit_version/warn_version. Not compared: phase. |
pod | Pod of every namespace | kubernetes.pod.generate_name, node, owner.kind/name/id, service_account, restart_policy, priority, priority_class, scheduler, host_network, qos_class, containers (name, image, init, requests and limits in cores and bytes), phase, reason, ready, start_time, host_ip, ip, restart_count, container_statuses (name, ready, started, restart_count, state, reason, exit_code, image_id, init); related.ip. Not compared: phase, reason, ready, start_time, host_ip, ip, restart_count, container_statuses. |
Workloads
| Dataset | One record per | Fields |
|---|---|---|
deployment | Deployment of every namespace | kubernetes.deployment.generation, replicas, strategy.type/max_surge/max_unavailable (a number or a share, 25%), min_ready_seconds, revision_history_limit, progress_deadline_seconds, paused, revision; its pods' service_account, host_network, host_pid, host_ipc, node_selector (key=value), containers (as a pod's); status.replicas/updated_replicas/ready_replicas/available_replicas/unavailable_replicas/observed_generation, conditions (Available, Progressing). Not compared: status, generation, conditions, revision. |
daemon_set | DaemonSet of every namespace | kubernetes.daemon_set.generation, update_strategy, min_ready_seconds, revision_history_limit, its pods' fields (as a Deployment's), status.desired_number_scheduled/current_number_scheduled/updated_number_scheduled/number_misscheduled/number_ready/number_available/number_unavailable/observed_generation. Not compared: status, generation. |
stateful_set | StatefulSet of every namespace | kubernetes.stateful_set.generation, replicas, service_name, pod_management_policy, update_strategy (partition), min_ready_seconds, revision_history_limit, claim_retention.when_deleted/when_scaled, volume_claim_templates (name, storage_class, access_modes, requested), its pods' fields, status.replicas/ready_replicas/current_replicas/updated_replicas/available_replicas/current_revision/update_revision/observed_generation. Not compared: status, generation. |
The API leaves out a Deployment's status count when it is zero. Once the Deployment has reported its status, a count it left out is written as 0.
Services, Ingresses, and Accounts
| Dataset | One record per | Fields |
|---|---|---|
service | Service of every namespace | kubernetes.service.type, cluster_ip, headless, external_ip, external_name, load_balancer_ip, session_affinity, ip_families, ip_family_policy, internal_traffic_policy, external_traffic_policy, ports (name, protocol, app_protocol, port, target_port, node_port), load_balancer.ip/hostname; related.ip, related.hosts |
ingress | Ingress of every namespace | kubernetes.ingress.ingress_class (ingressClassName, else the kubernetes.io/ingress.class annotation), default_backend, rules (host, paths: path, path_type, backend.service.name/port, a number or a name, or backend.resource.api_group/kind/name), tls (hosts, secret_name: the secret by name, never its content), load_balancer.ip/hostname; related.hosts, related.ip |
service_account | Service account of every namespace | user.id, user.name (the user its token authenticates as, system:serviceaccount:<namespace>:<name>); kubernetes.service_account.image_pull_secrets (by name), secrets (how many), automount_token. No secret's content is read. |
Role Bindings
| Dataset | One record per | Fields |
|---|---|---|
role_binding | Role binding of every namespace | kubernetes.role_binding.role_ref.kind/name/api_group (the Role or ClusterRole it grants), subjects (kind: User, Group, ServiceAccount; name, namespace, api_group); related.user: its users and service accounts as the API server authenticates them (system:serviceaccount:<namespace>:<name>) |
cluster_role_binding | Cluster role binding | kubernetes.cluster_role_binding.role_ref and subjects, as a role binding's; related.user |
What a role grants, its rules, is the role's, which no dataset reads.
Storage
| Dataset | One record per | Fields |
|---|---|---|
persistent_volume_claim | Persistent volume claim of every namespace | kubernetes.persistent_volume_claim.storage_class, volume_mode, volume_name (the volume it is bound to), access_modes, requested (the storage it asks for), capacity (the storage it has, once bound), phase (Pending, Bound, Lost). Not compared: phase. |
csi_driver | CSI driver | kubernetes.csi_driver.attach_required, pod_info_on_mount, fs_group_policy, volume_lifecycle_modes, storage_capacity, requires_republish, selinux_mount |
Events
event reads every event of every namespace it reads. An event that recurs is one object whose count and last time move on, and each count is an occurrence of its own. A reading writes each event's latest occurrence when its checkpoint does not hold it, once, in the order they happened, dated at the event's last time (its lastTimestamp, else its series' last observed time, else its eventTime, else its first time, else when the API made it). An event with none of these is left out.
- Recurrences between readings. When an event recurred more than once since the last reading, one record is written for its newest count, and its
kubernetes.event.countsays how many times it has happened. The occurrences in between are not written as records of their own, since the API gives no time for each of them. - Identifying an occurrence.
event.id(<uid>/<count>) names the occurrence. An occurrence can be written again after a crash (see below), andevent.idis the key to recognize it by.
The API server keeps an event an hour after it last happened by default (its --event-ttl); managed clusters keep their own.
| Dataset | One record per | Fields |
|---|---|---|
event | Occurrence of an event | message, event.id (<uid>/<count>: the occurrence), event.reason, event.action, event.provider (the component that reported it: kubelet, default-scheduler, deployment-controller), log.level (its type, normal or warning); kubernetes.event.id, name, count, type, reason, action, first_seen, last_seen, event_time, created, reporting_component, reporting_instance, source.component/host, involved_object and related (kind, name, namespace, uid, api_version, field_path) |
The dataset resumes from where the last delivered reading stopped:
- Across a restart or a failover. The checkpoint is kept in the Director cluster's replicated store, per device, so a restart, or another node after a failover, resumes from it.
- Written after delivery. A reading's checkpoint is stored only once its events are delivered. A crash before then writes those events again at the next start: at least once, never lost.
- The first reading of a dataset with no checkpoint goes back as far as the initial lookback, a day unless the dataset or the device says otherwise, so it reads every event the cluster still holds. An outage longer than the cluster keeps its events loses the ones it expired.
Examples
Reading the pods every 15 minutes into their own pipeline, the role bindings for who may do what, and the events from an hour back... | |