Appendix

ASIM

The Advanced Security Information Model is a layer between the data and the user to configure what and how to ingest data from a source and to route it to a destination.

For details, see this article.

Available ASIM tables:

ASimAuditEventLogs ASimAuthenticationEventLogs ASimDhcpEventLogs ASimDnsActivityLogs ASimFileEventLogs ASimNetworkSessionLogs ASimProcessEventLogs ASimRegistryEventLogs ASimUserManagementActivityLogs

CIM

ECS

Grok Patterns

The grok patterns used by Director's selection process:

Category	Patterns
General	DATA GREEDYDATA NOTSPACE SPACE WORD
Numeric	BASE10NUM INT NUMBER
Networking	HOSTNAME IP IPV4 IPV6 MAC
Data and Time	DATESTAMP DATESTAMP_RFC822 TIMESTAMP_ISO8601
File System	FILENAME PATH
HTTP	HTTPDATE HTTPDERRORLOG HTTPDUSER
System	SYSLOGBASE SYSLOGHOST SYSLOGTIMESTAMP
Other	EMAILADDRESS URIPARAM URIPATH UUID

Metadata Tags

The metadata tags used by Director's ingestion process:

Field	Subfields
_ingest	on_failure_processor_tag on_failure_processor_type
_temp	observer.mac
destination	bytes domain ip nat.port port user.domain name
email	from.address to.address
event	category kind original outcome type
source	bytes ip user.domain group.name id xlatesrc
observer	product type vendor
related	hash ip
vendor	__nsons __p_dport __pos _conf _ingest _temp _temp_ action administrator bcc cc client_inbound_bytes client_inbound_packets client_ip client_outbound_bytes client_outbound_bytes client_outbound_packets contextnum destination_dns_hostname device_name device_type dst dst_machine_name dst_user_dn dst_user_name endpoint_ip file_size from hll_key ifname lastupdatetime mac_source_address operation operation_number origin_ip product product received_bytes received_bytes s_port s_port segment_time sent_byte sequencenum server_inbound_bytes server_inbound_packets server_outbound_bytes server_outbound_bytes server_outbound_interface server_outbound_packets server_outbound_packets service severity src src_machine_name src_user_group src_user_name subs_exp syslog5424_ts tags time to uid uid xlatedport xlatedst xlatedst xlatesport xlatesport xlatesrc xlatesrc

Parquet Files

The parquet file was designed by Apache as a column-based format. Unlike row-based formats like CSV, it stores the records in columns.

Row-based storage First row contains field names:

id,name,last_name,age
1,John,Buck,35
2,Jane,Doe,27
3,Joe,Dane,42

Column-based Storage First column contains field names:

id:1,2,3
name:John,Jane,Joe
last_name:Buck,Doe,Dane
age:35, 27,42

Parquet files use dictionary encoding, compression and bit packing, and run-length encoding, features which make it more efficient compared to row-based formats.

PEM Files

A Privacy Enhanced Mail file is a container format often used to store cryptographic keys, certificates, and other data. It is a base64-encoded file that starts with a header and ends with a footer, e.g.:

-----BEGIN CERTIFICATE-----
MIIH/TCCBeWgAwIBAgIQaBYE3/M08XHYCnNVmcFBcjANBgkqhkiG9w0BAQsFADBy
MQswCQYDVQQGEwJVUzEOMAwGA1UECAwFVGV4YXMxEDAOBgNVBAcMB0hvdXN0b24x
ETAPBgNVBAoMCFNTTCBDb3JwMS4wLAYDVQQDDCVTU0wuY29tIEVWIFNTTCBJbnRl
cm1lZGlhdGUgQ0EgUlNBIFIzMB4XDTIwMDQwMTAwNTgzM1oXDTIxMDcxNjAwNTgz
M1owgb0xCzAJBgNVBAYTAlVTMQ4wDAYDVQQIDAVUZXhhczEQMA4GA1UEBwwHSG91
...
-----END CERTIFICATE-----

where CERTIFICATE can also be PRIVATE KEY or RSA KEY. These blocks communicate what is encoded in the file.

This is a text-based format, and the base64-encoded data can be uppercase and lowercase letters, digits, '+', and '/'.

A PEM file can contain multiple blocks of such data. These are generally used to encode for example RSA keys for SSH connections, certificates used for SSL encryption and the keys associated with them, etc.

A PEM file for certificates can specify

• the end-user certificate assigned to a domain name by a CA (Certificate Authority)
• up to 4 intermedidate certificates
• a root certificate self-signed by the CA

These are generally issued by the SSL provider to be used in a web service.

PEM files are also used for SSH. Typical use on the command line is:

ssh -i keyfile.pem root@host

where the -i parameter specifies the PEM file to use to create a secure SSH connection host.

Syslog

SYSLOG is a standard protocol used for system logging in computer networks. Using this protocol, devices and applications send log messages to central servers that store them for monitoring and analysis.

Syslog has numeric severity levels, indicating their importance. The lower the value, the more critical the event. The levels are used for:

• Prioritization of critical issues
• Filtering of log messages
• Automation of specific types of events
• Compliance with regulatory standards

Level	Severity	Description
0	Emergency	System unusable
1	Alert	Immediate action required
2	Critical	Condition critical
3	Error	Errors exist
4	Warning	Warnings exist
5	Notice	Significant condition
6	Info	Info messages
7	Debug	Debug messages

---