Skip to main content

Federated Search: The Search Page

The Search page is the console for federated search. You pick the sources, write a KQL query, and the page reads those sources at query time: nothing is ingested first. The rows that come back are loaded into a query engine running in your browser, and the full query runs there.

Opening the Search Page​

Click Search in the side navigation, right after Home.

The page needs the Live data read permission (LIVEDATA_READ), the same permission that gates live capture, since a search reads the same devices' streams and agents' event logs. Among the built-in roles that is Contributor, Admin and Owner; the User role cannot open Search. See Roles.

The Schema picker additionally needs Pipelines read. Without it the picker is not shown, and queries run against the source schema.

This release searches three kinds of source:

SourceTableHow it is read
A syslog deviceSyslogCaptured live from the device's stream for the capture window. A capture looks forward only: nothing from before the run is returned, and a quiet device returns nothing.
A Windows Security Events datasetSecurityEventEach agent of the dataset queries its local Security event log over the time range, newest first. Only matching events are sent.
A Windows File Logs datasetFileLogEach agent reads the files the dataset collects on it, newest first, over the time range. Only matching lines are sent.

FileLog has TimeGenerated (the line's own time, or the time it was read for a line with no date), Computer, FilePath, RawData (the line; a multi-line record joined with line breaks) and DeviceName.

SecurityEvent and Syslog use the Microsoft Sentinel schema, so Sentinel hunting queries against them run as written, as long as they avoid the functions the in-browser engine does not have (see What the In-Browser Engine Cannot Run). FileLog is VirtualMetric's own table.

If the tenant has none of these sources, the page shows Nothing to search yet. Add a syslog device, a Windows Security Events dataset or a Windows File Logs dataset under Fleet Management to search it here.

File Datasets​

  • A search never names a path to read. Each agent reads only the files its own assignment of that dataset collects, and a FilePath filter can only narrow them.
  • A search does not disturb collection. It reads the files with the collector's own reader but its own read state, so the collector's saved positions and schedule are untouched.
  • Search results go only to the browser that asked, never to routes, pipelines or targets.

The Page​

From top to bottom:

  1. A breadcrumb (Home / Search / Results), then the search's name. Click it to rename; the name is kept in the page's link and names exported files.

  2. The action bar on the right, as icons with tooltips, then the query language, KQL:

    • Read the sources again, Examples (starter queries for SecurityEvent, Syslog and FileLog) and Recent queries;
    • Show/Hide Chart, Show every line and Show/Hide Row Details;
    • Full Screen, Export (Export as CSV / Export as JSON) and Show SQL.

    Read the sources again forces a fresh read; see Refining a Query. Recent queries lists the last queries run in this browser.

  3. Sources, Schema and Rows per source:

    • Sources: a multi-select of the tenant's searchable devices and datasets.
    • Schema: Source schema queries the raw table. Choosing one of the tenant's pipelines instead sends the events through that pipeline first, and the query then runs against the table the pipeline produces. See Pipelines.
    • Rows per source: 1,000, 5,000 (the default) or 10,000. A live capture keeps at most 5,000 rows, so a device is read at 5,000 even when 10,000 is chosen.

  4. Query (KQL): an editor with syntax highlighting and completion of table and column names, taken from the chosen sources' schemas. The query is checked as you type and mistakes are underlined in place. Press Ctrl+Enter to search.

  5. The filter row:

    • Filters on the left opens the filter panel (see The Filter Panel).

    • For datasets, on the right: the 15m, 60m, 6h, 7d and 30d shortcuts, and a picker:

      • Presets: Last 15 minutes, Last 1 hour (the default), Last 6 hours, Last 12 hours, Last 24 hours, Last 3 days, Last 7 days, Last 30 days.
      • Date & Time Range: a start and an end, at most 31 days apart, starting in the past.

    • For devices, Capture replaces the range: 10s, 15s (the default), 30s or 45s.

    • Then Search, which becomes Stop while a search runs.

  6. The run summary, described under Results.

  7. The chart: events over time as bars.

    • Drag across the bars to zoom the result to that stretch; double-click, or click Reset zoom, to zoom back out.
    • Under the bars: FROM and TO (the span read), STATUS, ELAPSED TIME, RESULTS and LOAD (three dots, fewer lit for a lighter search).

  8. The results grid:

    • A caret on a multi-line row shows all its lines.

    • Drag a column header's edge to resize the column. Right-click a header for Autosize This Column / Autosize All Columns.

    • Select text in a cell and right-click it:

      • Copy Selected Text;
      • Add selected text as CONTAINS / Add selected text as NOT CONTAINS adds a filter rule without searching;
      • Search selected text as CONTAINS / Search selected text as NOT CONTAINS adds the rule and searches at once.

    • Without a selection, the menu offers Copy followed by the column's name (for example Copy Account). More rows load as you scroll.

  9. The row's details, opened with Show/Hide Row Details in the action bar: the clicked row as JSON Format or Pretty Format, beside the grid. Before a row is clicked it reads Select a row to see it here.

Results​

  • Per-source summary: which machines answered and with how many events, and what was Filtered at the source (see How a Run Works). A machine that did not answer is named with the reason, for example not connected or did not answer in time.
  • Generated SQL: a drawer, opened from Show SQL in the action bar, showing the SQL the query was translated to and the predicates each source applied before sending rows.

Zooming the chart works on the rows already in the browser. A filter runs after the query and, like a refined query, answers from the browser when the loaded rows cover it (see Refining a Query).

The Filter Panel​

  • Rules are Field, operator and Value, and the operators depend on the field:

    • text: equal, not equal, begins with, doesn't begin with, ends with, doesn't end with, contains, doesn't contain, is one of, is none of;
    • numbers and times: equal, not equal, greater, less, greater or equal, less or equal, between, not between;
    • true/false fields: equal, not equal.

  • Rules combine with AND or OR. Add Group nests a group with its own AND/OR; each rule and group can be cloned or removed.

  • Advanced query writes the filter as a KQL condition instead, for example Account contains 'admin' and EventID == 4625. Switching it on carries the rules over as text.

  • Apply Filters searches with the filter. Revert returns to the filter in force, and Delete All clears the panel.

  • An incomplete rule is outlined and not applied. You have unsaved changes shows until the panel matches what was applied.

  • The Filters button counts the rules. Its × clears them and searches again. The panel can be pinned to stay open while reading results.

  • How it applies: the filter runs after the query, as | where <condition>, on the query's results.

How a Run Works​

The steps below explain why a first run takes seconds and a refinement takes milliseconds.

  1. The query is translated from KQL to SQL on the server. The same translation is what checks the query while you type.

  2. Only the sources the query needs are read. The page reads only the sources whose tables the query uses. If no selected source feeds a table the query names, the page says so rather than running.

  3. Leading predicates are applied at the source. Predicates at the start of the query are pushed to the sources, and the summary lists them under Filtered at the source:

    • TimeGenerated bounds narrow the time range.
    • EventID == or EventID in (...) becomes part of the agent's event log query, for up to 15 event IDs. A query naming more reads every event in the range and filters them in the browser instead.
    • Computer == picks which agents are asked.
    • FilePath == keeps a file dataset's read to those files.

    Pushed predicates are only ever a narrowing filter: the full query still runs in the browser over what the sources return.

  4. The query runs in the browser. The rows are loaded into a query engine in your browser, and the full translated query runs there.

Refining a Query​

A refined query over the same sources and range answers from the browser in milliseconds, without reading the sources again, when the rows already loaded cover it: its range and its source-side filters (TimeGenerated, EventID, Computer, FilePath) are the same as the last read's or narrower. The summary marks such a result From the previous run.

A refinement the loaded rows do not cover, such as a query for EventID == 4625 after a read filtered to EventID == 4624, reads that source again on its own. So does any refinement after a read that stopped at its row limit, since such a read holds only the newest rows of its range.

Click Read the sources again to force a fresh read, for example to pick up events that arrived since the last run.

Limits​

LimitValue
Capture window10–45 s in the console (the API accepts 5–60 s)
Rows per sourceup to 10,000 for a dataset, 5,000 for a live capture
Agents asked per dataset runat most 20; filter on Computer to choose which
One agent's readstops after 30 s or 32 MiB
Time rangeat most 31 days

Partial Results​

An agent reads newest first, so when it stops at its time or size limit, what is missing is always the oldest part of the range. The summary names each such agent, for example DC02 · stopped at the time limit, and adds a note that the oldest part of the range is missing. STATUS under the chart reads Partial result. Narrowing the range or adding filters reads the rest.

A source that returns the full row limit shows Stopped at N rows.

When a dataset has more than 20 agents, only the first 20 are asked and the page says so. Add a Computer filter to choose which ones.

Requirements​

  • Windows Security Events datasets need a director and a Windows agent built with this release. An older agent does not answer the query, and it shows in the summary as did not answer in time.
  • Windows File Logs datasets need a director and a Windows agent built with this release. An older agent shows as did not answer in time, or with an event log channel is required.
  • Syslog devices work with existing directors.

What the In-Browser Engine Cannot Run​

The engine in the browser has DuckDB's core functions only. KQL that needs JSON or dynamic handling, such as parse_json, bag_unpack, or dynamic property access like d.Field, is refused with a message instead of running.

This is a restriction of the Search page. The DuckDB column of the KQL Support Matrix describes the full server-side DuckDB, which has those functions.

Self-Hosted Deployments Behind a Reverse Proxy​

The Search page runs its engine as WebAssembly. A reverse proxy that sets its own Content-Security-Policy must allow:

  • 'wasm-unsafe-eval' in script-src, which permits compiling WebAssembly and nothing else;
  • blob: in connect-src, which is how the engine's module reaches its worker.

Without both, the page reports that its query engine could not start.