Tool reference: read-only tools, continued

Read-only tools, continued

overflow_detect

Page-wide overflow scan: the silent UI-breakage primitive. Generalises inspect’s per-element overflow check into a typed multi-detector pass: walks the DOM, applies four overflow-shape detectors, returns one finding per offending element. The bugs this catches are precisely the ones a screenshot looks “fine” for (clipped pixel doesn’t shout) and find() doesn’t surface (“the element rendered but its content was lost”):

overflow_detect({ session?, scope?, types?, limit? }) returns { ok, scope, findings: [{ selector, bbox: {x,y,w,h} | null, type, evidence }], truncated, warnings, tokensEstimate }.

Detector types (default = all four; opt out via types:[…]):

Type	Condition	Evidence	Why it matters
layout	scrollWidth/Height > clientWidth/Height AND overflow:auto|scroll on the relevant axis	{ scrollWidth, clientWidth, scrollHeight, clientHeight, overflowX, overflowY }	Content overflows the padding box; scrollbar IS provided. Subtler than clipped: recoverable, but often unintended.
clipped	same dimensional check, but overflow:hidden|clip on the relevant axis	same shape as layout	The high-value finding: content invisible with no scrollbar. “The button got cut off.”
text-ellipsis	text-overflow:ellipsis AND scrollWidth > clientWidth	{ scrollWidth, clientWidth, visibleText, fullText }	Truncated labels. visibleText is a best-effort prefix (offsetWidth-bounded heuristic); the agent reads fullText for the truth.
viewport-horizontal	documentElement.scrollWidth > clientWidth	{ documentScrollWidth, viewportWidth, overrunPx, widestDescendantSelector?, widestDescendantWidth? }	The “horizontal scrollbar on body” mobile-layout bug. Singleton finding: selector "html", evidence carries the overrun amount + the widest overrunning descendant when cheaply identifiable.

EPSILON = 1 CSS px tolerates sub-pixel rounding noise. Without it, pages that scale fonts or run on a fractional devicePixelRatio routinely trip false positives by ≤0.5 px.

Inputs:

• scope?: "viewport" | "document": "document" (default) walks every element; "viewport" skips elements fully off-screen (cheaper on very large pages).
• types?: ("layout" | "clipped" | "text-ellipsis" | "viewport-horizontal")[]: default = all four. Empty array is treated as default (an empty filter that silently matches nothing would be a usage error with no signal); unknown values are dropped silently.
• limit?: number: cap on findings returned (default 50, max 500). Findings past the cap are dropped and truncated:true is set. Prevents huge result sets on very broken pages.

Selector synthesis tiers (per finding’s selector field):

1. [data-testid="..."] if present.
2. [role="..."][aria-label="..."] (both stable).
3. nth-of-type CSS path bounded at 5 levels.
4. tag.classes (up to 3 class names).

Capped at 200 chars; longer falls through to tag only with evidence.selectorTruncated:true so the agent can see why the selector is a bare tag.

Bounded walk: MAX_ELEMENTS_SCANNED = 10000. When the cap is hit the result carries warnings:["scan stopped at MAX_ELEMENTS_SCANNED (10000) — re-run with scope:viewport for a narrower pass"], so an agent that runs against a huge page knows to narrow down.

Typical use:

• Post-render layout sanity sweep: call after a navigation/render to surface any clipped controls before the agent starts driving them.
• Mobile responsive checks: drive set_viewport({ width: 375 }) first, then overflow_detect to catch horizontal-scrollbar regressions.
• “The button I clicked got truncated” diagnosis, combined with find() / inspect(): overflow_detect finds the offenders; inspect reads the full computed-style context for any one element.
• CI sanity gate: fail the build when truncated:false AND findings.length > 0 for clipped type (cheap regression catch).

Read-only (capability read). Distinct from inspect (which targets one element + reads styles+box) and find (ranking). On a clean page returns { ok:true, findings:[], truncated:false, warnings:[] }.

Example.

set_viewport({ width: 375, height: 812 })          // mobile breakpoint first
overflow_detect({ types: ["clipped", "viewport-horizontal"] })
// → { "ok": true, "scope": "document",
//     "findings": [{ "selector": "[data-testid=\"summary-card\"]",
//                    "bbox": { "x": 16, "y": 380, "w": 344, "h": 120 }, "type": "clipped",
//                    "evidence": { "scrollHeight": 188, "clientHeight": 120, "overflowY": "hidden", … } }],
//     "truncated": false, "warnings": [], "tokensEstimate": 138 }

generate_locator

Convert a session-internal eN ref (from snapshot() / find() / plan()) into a Playwright-string locator expression an adopter can paste verbatim into a .spec.ts. The bridge between agent-driven exploration and a deterministic regression suite. find() already returns a richer selectorHint + stability + actionable predicate, but the in-process ref is browxai-internal; this tool emits the real Playwright expression a human reading a .spec.ts would expect to see.

Inputs: { ref: string, session?: string }

Output: JSON

{
  "ok": true,
  "playwright": "page.getByRole('button', { name: 'Save' })",
  "stability": "high",
  "components": [
    { "kind": "role", "value": "button", "name": "Save" },
    { "kind": "text", "value": "Save" },
  ],
  "tokensEstimate": 28,
}

Or, when the ref isn’t in this session’s registry (structured failure, no throw):

{
  "ok": false,
  "failure": {
    "kind": "ref-not-found",
    "ref": "e42",
    "hint": "ref \"e42\" is not in this session's registry. Call snapshot() or find() first…",
  },
  "tokensEstimate": 41,
}

Tier mapping (same five-tier preference order find() uses; the emitted expression mirrors how browxai itself would resolve the ref at action time):

Ref shape	Emitted expression	stability
data-testid (default attr)	page.getByTestId('save-btn')	high
Custom test attribute (data-cy, data-type, …)	page.locator('[data-cy="submit-form"]')	high
role + accessible name	page.getByRole('button', { name: 'Save' })	high
Stable structural CSS path (semantic anchor / #id / [data-*])	page.locator('main > table > tbody > tr:nth-child(4)')	medium
Purely positional CSS path (chains of :nth-child under generic tags)	page.locator('div > div:nth-child(2) > div')	low
Role only (no name, no path)	page.getByRole('button')	low

stability semantics are the same as find()’s: high = “uniquely identifies this element via a stable signal” (testid or role+name); medium = “stable structural / stable text on a stable role”; low = “positional or role-only, likely to drift on the next render.” Both labels reflect per-snapshot uniqueness; long-term deploy stability is still the adopter’s call on top.

components is the structured breakdown of the parts that built the string: { kind: "testid"|"role"|"text"|"css", value, name?, attribute? }. Adopters who want to compose their own locator (chain .filter(), combine two kinds, scope into a parent) can read this directly without re-parsing the emitted string.

Quoting / escaping. The emitted expression is paste-safe: single-quoted JS string literals, single-quotes and backslashes inside accessible names / testIds are escaped (page.getByRole('button', { name: 'O\'Brien' })). For non-default test attributes the attribute-CSS form uses double-quoted JSON-escaped values inside the single-quoted outer string.

Secrets masking. Emitted strings + component values pass through the per-session secret registry on egress, the same posture as find().selectorHint. A registered real-value rendered into a name / testId gets substituted with its alias before the JSON ships.

Read-only (capability read); no new capability gate. In the batch whitelist, so you can compose find, then generate_locator, then record the string somewhere durable in one batch.

point_probe({ coords, crop?, session? })

Read-only: what is actually under a viewport coordinate. point_probe({ coords:{x,y} }) returns { ok, point, stack:[…], scrollContainer, clickableAncestor, cropBase64? }. The coordinate-target verifier for canvas / virtualised-timeline / painted UIs where the target isn’t a clean accessible element and find()/inspect can’t address it.

• stack: the full document.elementsFromPoint(x,y) top-down (capped 8); stack[0] is what a real click({coords}) would hit. Each layer carries tag/id/testId/role/name/classes + computed pointerEvents/visibility/display/zIndex/cursor + bbox, enough to prove “this point hits the audio segment, not the video layer above it” and to see why (pointer-events:none passthrough, z-index ordering).
• scrollContainer / clickableAncestor: nearest scrollable ancestor and nearest semantically-clickable ancestor of the top element (what a click here would actually activate).
• crop:true adds a small bounded PNG (base64) around the point; off by default (token-cheap). No agent JS. Capability read. Pairs with click({coords}): probe first, then drive.
• On failure the result is structured for triage: { ok:false, point, url, error } (the coordinate + page URL, not a bare error).

Example.

point_probe({ coords: { x: 512, y: 380 } })
// → { "ok": true, "point": { "x": 512, "y": 380 },
//     "stack": [{ "tag": "canvas", "testId": "timeline", "role": null,
//                 "pointerEvents": "auto", "zIndex": "10", "cursor": "pointer", "bbox": {…} }, …],
//     "scrollContainer": { "tag": "div", "classes": ["timeline-scroll"] },
//     "clickableAncestor": { "tag": "div", "testId": "clip-4" } }
// stack[0] is what click({coords}) would hit — probe first, then drive.

ws_read

Session-wide ring of recent WebSocket / Server-Sent-Events frames (cap 500; HTTP is network_read, this is the realtime channel). ws_read({ session?, limit?, urlPattern? }) returns { total, frames: [{ url, dir: "sent"|"recv", kind: "ws"|"sse", opcode?, event?, payload, truncated?, ts }] }. Payloads truncated (~2000 chars). The verification primitive for realtime correctness: chat / multiplayer / collaborative-editing / live-dashboard broadcasts, where the frame stream is the only ground truth. Per-action frames also land in ActionResult.network.wsFrames (frames that arrived during that action’s window), so you can assert a click produced the expected broadcast without polling ws_read separately. Capability: read.

Example.

ws_read({ urlPattern: "rt.example.com", limit: 10 })
// → { "total": 3, "frames": [{ "url": "wss://rt.example.com/socket", "dir": "recv",
//      "kind": "ws", "payload": "{\"type\":\"presence\",\"users\":4}", "ts": 1765540264118 }, …] }

Interactive WebSocket: ws_send / ws_intercept / ws_unintercept

The read-only WS view is ws_read; this family is the mutation half: send a frame on a live page-side socket, or rewrite/drop INBOUND frames before app handlers see them. Sibling of the HTTP route family on the realtime channel. All three sit under capability action.

A page-side wrapper on window.WebSocket is installed eagerly at session creation (Page.addInitScript) so a socket constructed during initial document parse is captured. Each new WebSocket(...) is assigned a stable per-session wsId (ws-1, ws-2, …) you can discover via eval_js JSON.stringify(window.__browxWs.list()), which returns [{wsId, url, readyState}].

ws_send({ wsId, message, session? })

Push a payload onto an OPEN socket. Calls the real (unwrapped) WebSocket.prototype.send, so app-level message listeners do NOT observe a fake event; only the server sees the outbound frame. Returns { ok:true, wsId, url, bytes } on success; { ok:false, error } if the id is unknown or the socket isn’t OPEN. Binary frames are not in MVP; send as text.

ws_intercept({ pattern, response, session? })

Install a route-handler for INBOUND frames. pattern is a glob (the route family’s intent: * = single segment, ** = any) matched against socket.url at frame time. Three response modes:

• "drop": silently discard the frame before app handlers run.
• "echo": mirror the inbound payload back to the server (the app still receives the original locally).
• { data: "<string>" }: replace the inbound payload with data; app handlers see the replacement.

Re-adding the same pattern replaces the prior entry (no duplication). The interceptor evaluates on every matching frame until removed.

ws_unintercept({ pattern?, session? })

Remove one interceptor (by exact pattern), or every interceptor this session installed when no pattern is given.

Example (family).

ws_send({ wsId: "ws-1", message: "{\"type\":\"ping\"}" })
// → { "ok": true, "wsId": "ws-1", "url": "wss://rt.example.com/socket", "bytes": 15 }

ws_intercept({ pattern: "wss://rt.example.com/**", response: { data: "{\"type\":\"noop\"}" } })
ws_unintercept({})   // remove every interceptor this session installed

Caveats. The wrapper installs at session creation; if you swap a session out via the BYOB rebuild path, both the wrapper AND any active interceptors are lost (a fresh wrapper installs on the new context, but the registry is empty). Same with full session close. There is no equivalent of network_emulate’s “applies cross-context”; the wrapper is per-context by construction.

Workers visibility: workers_list / worker_message_send / worker_messages_read / sw_intercept_fetch

Web Workers + Service Workers are otherwise invisible to the surface: network_read shows page fetches but never sees a Service Worker that responds from its cache; the postMessage IPC between page and workers is off-grid entirely. This family makes both observable and mutable.

Two completely different transport stories under one façade:

• Web Workers. A page-side wrapper of window.Worker is installed eagerly at session creation (Page.addInitScript, same posture as the WS family), so a worker constructed during initial document parse is captured. Each new Worker(...) gets a stable per-session id ww-1, ww-2, …. The wrapper mirrors every message-from-worker into a 500-entry ring (4 KiB payload cap, oldest evicted first); worker_message_send calls the real (unwrapped) Worker.prototype.postMessage so the worker’s onmessage sees a real event, not a synthetic one.
• Service Workers. SWs are independent targets the page-side wrapper can’t reach. Listing and sending use Playwright’s own handles: context.serviceWorkers() returns a Worker per live SW, and Worker.evaluate runs in the SW global. Playwright exposes these on Chromium-based engines only. The ids are sw-1, sw-2, … keyed by script URL, so the id of a SW stays the same when it stops and restarts, and a listing and a message from the same script agree on it. Playwright drops a SW when its target goes away, so a stopped SW cannot be messaged. workers_list still shows the sw-N id of every SW this session has seen running, with state: "stopped". A script URL that only a page message named is not listed, because the page can name any URL it likes. Messages a SW posts back to the page (client.postMessage) are recorded from the page’s navigator.serviceWorker message events by the same init script, in a second 500-entry / 4 KiB ring, and surface in worker_messages_read under the sw-N id of the posting script’s URL. That capture is page-side, so it works on any engine the page-side wrapper installs on. worker_message_send to an sw-N dispatches a synthetic MessageEvent into the SW global with Worker.evaluate, so self.onmessage and message listeners fire. The event has no source, so a worker that replies should use clients.matchAll({ includeUncontrolled: true }) and client.postMessage; the reply then comes back through worker_messages_read. sw_intercept_fetch stays on CDP: it attaches to SW targets with ServiceWorker.enable + Target.setAutoAttach({autoAttach:true, waitForDebuggerOnStart:false, flatten:true}) and arms Fetch.enable, so requests the SW’s fetch handler chose to intercept are paused, and the canned response is returned.

workers_list({ type?, session? })

Enumerate the workers in this session. type filters: "web" / "service" / "all" (default). Returns [{ workerId, type, url, state? }]. On an engine that cannot reach service workers (anything but Chromium-based), type: "service" returns an error naming the engine, and "all" lists the Web Workers only. A Service Worker has state: "running" while Playwright holds it and state: "stopped" once it is gone, and keeps its id. A Service Worker on an origin BROWX_BLOCKED_ORIGINS blocks, or that BROWX_ALLOWED_ORIGINS does not list, is left out, and so are its messages. Capability: read.

worker_message_send({ workerId, message, session? })

postMessage to a worker: ww-N for Web Workers, sw-N for Service Workers. message is a string; structured-clone / MessagePort transfer is not in MVP. A Service Worker receives a synthetic MessageEvent (no source). Sending to an sw-N returns ok: false with an error naming the engine when the engine cannot reach service workers, and when no live service worker has that id. It also returns ok: false with an origin-blocked error, and sends nothing, when the service worker is on an origin the origin policy does not allow. Capability: action.

worker_messages_read({ workerId?, session? })

Drain buffered messages FROM workers since the last read. Returns [{ workerId, data, at }]. Omit workerId to drain ALL workers; pass one to drain that worker only. Service Worker messages are the ones the SW sends to this page; they use the same shape, with a sw-N workerId. A SW that messaged the page and has since stopped still has its sw-N id, which workers_list shows with state: "stopped" when the session saw it running; sending to it fails. Each call drains (removes) what it returned; re-reads see only what arrived since. Message data is reported by the page and is untrusted, like any page text. Capability: read.

sw_intercept_fetch({ pattern, response, session? })

Register a fetch interceptor for Service-Worker-handled requests. pattern is a glob matched against the intercepted request URL (same shape as route / ws_intercept: * = single path segment, ** = any). response is { status?, body?, contentType?, headers? } (defaults 200, empty body, application/json). Fires only when the SW’s fetch handler runs, i.e. when the SW chose to intercept the request, which cleanly separates SW-mediated traffic from page-direct traffic. Re-add of the same pattern replaces. sw_unintercept_fetch({ pattern?, session? }) removes one entry or all of them. Capability: action.

Example (family).

workers_list({})
// → [{ "workerId": "sw-1", "type": "service", "url": "https://app.example.com/sw.js", "state": "running" },
//    { "workerId": "ww-1", "type": "web", "url": "https://app.example.com/search-worker.js" }]

worker_message_send({ workerId: "ww-1", message: "{\"cmd\":\"reindex\"}" })
worker_messages_read({ workerId: "ww-1" })
// → [{ "workerId": "ww-1", "data": "{\"done\":true,\"indexed\":1284}", "at": 1765540264201 }]

sw_intercept_fetch({ pattern: "**/api/profile", response: { status: 200, body: "{\"name\":\"Ada\"}" } })

Caveats. Per-context by construction; lost on session close or BYOB rebuild (a fresh wrapper installs on the new context; the registry is empty). Web Worker listings carry only the scriptURL captured at construction. Chromium does not expose it via any public API post-hoc. MessagePort transfer is not in MVP. The CDP path for child-session sends relies on flatten-mode routing; SW message round-trips are best-effort under that boundary.

ActionResult.network.mutations

Action windows that include a write-shaped request (POST / PUT / PATCH / DELETE with a 2xx response) get a bounded mutations array on top of summary / requests:

"mutations": [
  { "method": "POST", "urlPattern": "https://api.example.com/v1/records",
    "status": 200, "ok": true, "durationMs": 142,
    "responseShape": ["id", "date", "type", "task"] }
]

• urlPattern strips the query string and replaces id-shaped path segments (numeric / UUID / long hex) with :id: stable per logical endpoint, no record-id leak.
• responseShape is the top-level keys only of the parsed JSON response (or [].key for an array-of-objects response). No values, no nested keys. Capped at 20 entries.
• responseShape is omitted for non-JSON bodies, oversized bodies (>256 KB), and binary responses.
• Confirms “the click caused one successful mutation that wrote back keys X/Y/Z” without exposing actual data. Pair with element.container.changed to validate the visible state matches.

Full response-body inspection is intentionally not exposed here; that would broaden the leak surface and bloat agent context. A future dedicated tool (under a higher-risk capability) can expose full bodies opt-in for the rare debugging case.

Inputs: { limit?: number (default 50, max 500) }

Output: JSON { summary, requests }.

Example.

network_read({ limit: 50 })
// → { "summary": { "total": 14, "byType": { "xhr": 9, "document": 1, "other": 4 }, "failed": 0 },
//     "requests": [{ "method": "GET", "url": "https://api.example.com/v1/records?…",
//                    "status": 200, "type": "Fetch", "ms": 88 }, …] }
// URLs are redacted at egress: query strings / fragments show as `?…` / `#…`.

eval_js

Run a JavaScript expression in the page’s main frame. The escape hatch when no other tool covers your case (typically: trigger a page-side function the app exposes, e.g. window.__siteDocs.capture()). Use sparingly.

⚠ eval_js element.click() does NOT fire framework click handlers. A programmatic .click() (or dispatched synthetic event) here is not a trusted/synthetic-equivalent event, so Vue @click / React synthetic / custom-element listeners never run. The app does nothing and you’ll wrongly conclude the feature is broken. This is a recurring, expensive false negative. Use the click tool for any click you’re testing; reserve eval_js for reading state or calling app-exposed functions. The server emits a soft warning on the result when it detects .click() in the expression.

Inputs: { expr: string, returnType?: "json" | "void" (default "json") }. The return value must be JSON-serializable for "json" mode; "void" is fire-and-forget.

Output: JSON { ok: true, value } / { ok: true, returnType: "void" } / { ok: false, error }.

Trust boundary: the call originates from the (trusted) agent, but the return value is page-controlled, so treat it as untrusted just like snapshot text.

Gating: off by default, since the eval capability isn’t in DEFAULT_CAPABILITIES. Set BROWX_CAPABILITIES=read,navigation,action,human,eval to enable; the server logs a loud warning at startup.

Example (the legitimate case, calling an app-exposed function):

eval_js({ expr: "window.__app.flushQueue()", returnType: "void" })
// → { "ok": true, "returnType": "void" }

eval_js({ expr: "window.__app?.version" })
// → { "ok": true, "value": "3.42.1" }   // page-controlled — treat as untrusted data

find_feedback

Tell browxai which candidate was the right answer to a prior find(query). Subsequent finds whose query overlaps the token set will boost candidates matching this winner’s identity (testId, or role+name). Session-scoped, in-memory, capped at 100 entries with LRU eviction. The learning is intentionally simple: a “don’t re-do that mistake” signal, not an ML model.

Inputs: { query: string, ref: string }, where query is the query you previously passed to find() (or a paraphrase; token overlap is what matters), and ref is the ref the agent ended up acting on.

Output: JSON { ok, recorded: { query, identity }, memorySize }.

Example.

find_feedback({ query: "the save button in the toolbar", ref: "e42" })
// → { "ok": true, "recorded": { "query": "the save button in the toolbar",
//      "identity": { "testId": "save-btn" } }, "memorySize": 3 }

Recording tools

start_recording({ flowName, replay? }) / end_recording() / record_annotate({ copy, arrow?, target?, stepId?, label?, phase? }).

Recorded actions become a draft flow-file YAML (site-docs-flavoured): a locators block + steps with selectorHints transcribed from the action target. Use during calibration to cut hand-writing the YAML; review the locators (entries flagged stability: medium|low deserve a second look) and add prerequisites/assertions before committing.

Optional session-replay artifact. Pass start_recording({ flowName, replay: { tier?, sizeCap?, eventCap?, redaction?, maskSelectors?, dir? } }) to also emit a .browx archive when end_recording runs. tier is "actions" | "replay" (default, DOM stream + network/WS metadata) | "reexecutable" (adds response bodies). Requires the off-by-default replay capability; without it the argument is refused with a structured capability-denied shape. The archive is as sensitive as the session it recorded, registered secrets are masked at capture time before anything reaches disk, but the DOM, request/response metadata, and console output are otherwise the session’s real content. Store under $BROWX_WORKSPACE and treat it accordingly. Truncation (size-cap / event-cap / backpressure) is reported on manifest.truncated; backpressure is distinct and does not stop capture. end_recording returns replay: { path, absolutePath, bytes, events, counts, entries, truncated? } when a writer was engaged; export_session_report links the most-recent completed archive under replayArtifact. record_annotate({ label }) tags the current annotation with an acceptance-criterion id, the offline player’s coverage view groups spans by this label.

Reads are recorded too. extract, find, snapshot and eval_js dispatch no action, but a session whose purpose is to read something is exactly the session worth replaying, so they land in the trace as a second step kind. A read step carries read: <tool> instead of action: <type>, plus what the call asked for: schema + scope for extract, query for find, scope for snapshot, expr for eval_js. The locator a find resolved joins the locators: block like any action target. The returned page data is not recorded; the trace holds the request, never the response.

steps:
  - id: find-2
    read: find
    query: "the reply button"
    target: $button_reply
  - id: extract-3
    read: extract
    schema: { "type": "object", "properties": { "sender": { "type": "string" } } }
    scope: "e12"

End-recording output: { name, yaml, stepCount }. The YAML draft is the deliverable.