Cato Networks Datasets

Cato Networks datasets define what a Cato Networks device reads from Cato's GraphQL API (dataset type cato_networks). They attach to devices of type cato_networks, and in Datasets & Profiles the category is labelled Cato Networks. To create and assign datasets through the web interface, see Management. For the dataset and profile model, see Overview.

Collector Types

There are seven datasets, each with its own definition id, interval, checkpoint, and pipeline. Each is a tile in the wizard's Cato Networks category.

Dataset (console title)	Definition	Definition id	Kind	Reads
Cato Networks Events	cato_events_collector	29100	log	Every event of the dataset's accounts, from the queue marker the last poll stopped at. The first poll takes the marker and returns nothing; each later poll drains the queue 3,000 events a call. Cato keeps a marker for three days: a device off for longer loses the events in between.
Cato Networks Audit Trail	cato_audit_collector	29101	log	What administrators changed, for the time since the last poll, 60 seconds behind now (the last 24 hours on its first poll).
Cato Networks XDR Stories	cato_xdr_stories_collector	29102	log	The XDR stories updated since the last poll (the last 24 hours on its first poll), the oldest update first, 100 a page. A story is read again each time it changes. Read per account.
Cato Networks Site Metrics	cato_site_metrics_collector	29103	metric	A record per site, with its links, for the time since the last poll (the last 5 minutes on its first poll). Read per account.
Cato Networks Application Usage	cato_application_usage_collector	29104	metric	A row per value of the dimensions the dataset groups by, with its traffic, upstream, downstream, and flows, for the time since the last poll (the last 15 minutes on its first poll), 1,000 rows a page. Read per account.
Cato Networks Sites and Sockets	cato_sites_collector	29105	inventory	Each site as it is at the poll: connectivity, PoP, location, HA and degraded state, and its Sockets. Read per account.
Cato Networks Connected Users	cato_connected_users_collector	29106	inventory	The remote users connected at the poll: their Client's version and OS, their PoP, the address they connect from, and how long they have been connected. Read per account.

A device reads each dataset type once. The console refuses a second dataset of a type the device already reads, whether assigned on its own or through a profile (DATASET_TYPE_ALREADY_ASSIGNED, 409): it would read the same feed with the same key again. To send different accounts to different pipelines, filter on the account in the pipeline.

Dataset Properties

Each property left empty, or outside its range, keeps the connector's own value.

Field	Console label	Default	Range	Description
account_ids	Account IDs	-		Required. The accounts the dataset reads: one ID, several separated by commas or line breaks, or a list. A dataset with none is left out, and the Director logs why.
interval	Interval (Optional)	per type	per type, up to 86400	Seconds between polls. A value outside the range is taken as the nearest end. See Intervals.
pipeline	Pre-processing pipeline (Optional)	the device's		The dataset's own pre-processing pipeline.
ingestion_delay	Ingestion delay	60	0–86400	Seconds behind now that a poll's window ends. The windowed datasets only: audit, XDR stories, site metrics, and application usage. The events feed reads a queue, and the sites and connected users are read as they are.
query_window	First poll window	audit and XDR stories 1440, site metrics 5, application usage 15	1–43200	Minutes back a windowed dataset's first poll reads.
rate_limit_per_minute	Requests per minute	events 90, audit 4.8, site metrics 12, sites and connected users 15, others 60	1 and up	A whole number. It can only lower the connector's rate: the default keeps under Cato's limit for the query (see Rate Limits).
dimensions	Group by	application_name, category		Application usage only. See Application Usage Dimensions.
retry_count	Maximum retries	3	1–10	Times a failed request is sent again.
timeout	Request timeout	60	1–300	Seconds a request may take.

Intervals

Dataset	Default	Minimum
Events, audit	60 s	10 s
XDR stories, site metrics	300 s	60 s
Application usage	900 s	300 s
Sites	3,600 s	300 s
Connected users	900 s	300 s

Application Usage Dimensions

dimensions names what each application usage row is a sum over, as a list or as names separated by commas: application_name, application_type, application_risk_level, category, is_cloud_app, is_sanctioned_app, new_app, src_site_name, src_country, src_is_site_or_vpn, dest_site_name, dest_country, dest_is_site_or_vpn, dest_domain, tld, user_name, vpn_user_email, department, device_os_type, device_type, client_class, pop_name, egress_pop_name, ISP_name, traffic_direction, connection_origin, network_rule, action, or qos_priority. Names Cato does not group by are dropped; with none left, the default applies.

warning

Each dimension multiplies the rows. An account of about 400 remote users produced about 1,500 rows in 15 minutes grouped by application and category, and about 37,000 with src_site_name added, which counts each remote user as a site.

Read Per Account

The events and audit feeds ask for every account of their dataset in one call. Every other Cato query takes one account a call, so those five datasets are read one connector per account: a dataset naming three accounts runs three connectors, each named after its account (CatoXdrStories-1234), each with its own checkpoint, and each writing its account into its records as account_id. The console shows one dataset; the Director's device page lists the connectors.

These datasets need a Director of 2.3.1 or later, the first release that reads a dataset per account:

Director release	What the platform sends
2.3.1 or later	Every dataset, every account
Known to be older than 2.3.1	The events and audit datasets only. The five datasets read per account are left out, even when they name one account, and the platform's log names each one left out.
Not known to the platform	Every dataset

Checkpoints

Checkpoints are kept in the Director cluster's replicated store, per device, per dataset, and per connector: the events feed's marker, the end of the audit feed's last window, and one per account for a dataset read per account.

• A dataset assigned in place of another starts from its own first poll, and an account added to a dataset starts from its first poll's window.
• Editing, assigning, or removing a dataset restarts the device's collector, and each connector resumes from its checkpoint.
• Moving from a ccf device configured for Cato to a cato_networks device starts the feeds afresh: the two share no checkpoints. An existing ccf-based Cato device keeps working unchanged.

Records

Each record is written to the custom log table with its dataset's definition id, as Cato returns it. A pipeline normalizes it.

Dataset	Record
Events, audit, application usage	One Cato fieldsMap, a flat object of strings. Cato writes application usage's numbers as text too.
XDR stories	A story, with its incident nested
Site metrics	A site, with its metrics and its interfaces, each with theirs. The metrics sum the time since the last poll; the record carries no from or to.
Sites	A site, with its info, haStatus, and devices (its Sockets)
Connected users	A remote user, with remoteIPInfo and info

The records of a dataset read per account each carry account_id, as the events and audit feeds' fieldsMap already do; a record that has its own keeps it.