Tool reference: advanced tools

Advanced tools: gestures, route mocking, compound observers

These tools were formerly an off-by-default experimental lane; as of v0.1.0 they are promoted into the stable surface under their natural capabilities. Pointer gestures and route mocking are action; the compound act-and-observe tools and region screenshots are read; named-region bind/resolve and profile snapshot/restore are human coordination, all in the default capability set. The one exception is poll_eval: it evaluates page JS, so it sits under the off-by-default eval capability. They cover the heavier media-editor / race-condition QA workflows.

Pointer gestures: drag / double_click / mouse_down / mouse_move / mouse_up / mouse_wheel

For timeline scrub/trim, drag-reorder, sliders, lasso: interactions click/hover can’t express.

• drag({ from, to, steps?, preflight?, session? }): press at from, move to to over steps intermediate points (default 12, clamped 1 to 100), release. from/to are each {ref}|{selector}|{coords} (element targets resolve to box centre). Returns { ok, from, to, steps }. preflight: true instead probes the from point and returns { ok, preflight: { point, hit, resizeRisk } } without dragging. hit is the point_probe stack, resizeRisk is true when a press-point layer has a *-resize cursor. Check it before dragging a narrow item so you grab its body, not a resize handle (to is not required when preflight:true).
• double_click({ target, session? }): double-click a {ref}|{selector}|{coords} target.
• mouse_down / mouse_move / mouse_up({ coords?, session? }): low-level mouse for custom gestures. mouse_move requires coords; mouse_down/mouse_up move there first when coords is given, else act at the current pointer position.
• mouse_wheel({ coords, deltaX?, deltaY?, session? }): coordinate-space wheel event dispatched via CDP at coords (viewport CSS px) regardless of the current pointer position. For canvas, virtualised lists, and map tiles that listen for wheel and ignore scroll’s element-level path. deltaX/deltaY are CSS px (DOM WheelEvent convention: positive deltaY scrolls content up); at least one must be non-zero. Returns { ok, coords, deltaX, deltaY }.

Example (drag with a preflight check first).

drag({ from: { coords: { x: 180, y: 300 } }, preflight: true })
// → { "ok": true, "preflight": { "point": {…}, "hit": {…}, "resizeRisk": true } }
//   resizeRisk → the press point sits on a *-resize cursor; nudge inward first.

drag({ from: { ref: "e33" }, to: { coords: { x: 720, y: 240 } }, steps: 24 })
// → { "ok": true, "from": {…}, "to": {…}, "steps": 24 }

Touch + multi-touch gestures: touch_start / touch_move / touch_end / gesture_pinch / gesture_swipe

A separate dispatch pipeline from mouse_*. Mobile-default apps, canvas / map / drawing widgets, and pull-to-refresh / swipeable list UIs wire touchstart / touchmove / touchend handlers that the mouse pipeline does NOT reach. CDP Input.dispatchTouchEvent is the touch sibling of dispatchMouseEvent; touch and mouse stay net-additive, and neither aliases the other.

Touch does NOT auto-fire mouse events. Browsers MAY synthesize mousedown/mouseup/click from a touchend on touch-aware pages, but that’s app-policy (governed by the page’s touch-action CSS and preventDefault choices in its handlers), not a browxai guarantee. An agent that needs both pipelines must dispatch both explicitly (e.g. touch_start + mouse_down).

• touch_start({ coords, identifier?, session? }): dispatch a touchstart at coords (viewport CSS px). identifier (default 1) maps to DOM TouchEvent.changedTouches[].identifier; use distinct ids per finger when fanning out multi-touch by hand. Returns { ok, action:"start", coords, identifier, tokensEstimate }.
• touch_move({ coords, identifier?, session? }): touchmove update. Same shape as touch_start.
• touch_end({ coords?, identifier?, session? }): touchend. coords is optional: omit to dispatch an empty touchPoints[] (the spec’s “all fingers up” form, which is what gesture_pinch / gesture_swipe use internally); supply coords + identifier to lift a specific finger. Returns { ok, action:"end", coords?, identifier, tokensEstimate }.
• gesture_pinch({ coords, scale, steps?, startOffset?, session? }): two-finger pinch in/out centred on coords. Two touch points start at coords ± startOffset (default 40 CSS px) and converge or diverge linearly so the final separation is startOffset × scale. scale < 1 is pinch-in (zoom out); scale > 1 is pinch-out (zoom in). steps (default 12, clamped 1 to 100) intermediate touchMove dispatches. Linear interpolation is deliberate: pinch handlers read inter-frame deltas; velocity-detecting curves can misfire fling heuristics on libraries like Hammer.js, linear is the safe default. Returns { ok, coords, scale, steps, startOffset, endOffset, tokensEstimate }.
• gesture_swipe({ from, to, durationMs?, steps?, identifier?, session? }): single-finger swipe from from to to. Distinct from drag (which uses the mouse pipeline). durationMs (default 200: fast flick; 500+ reads as deliberate scroll) is split across steps (default 16, clamped 1 to 200) touchMove dispatches. Smoothed with an ease-out curve (1 - (1 - t)²), matching the natural deceleration most fling-detect heuristics expect (Hammer.js, native scroll inertia, react-spring physics). Returns { ok, from, to, steps, durationMs, tokensEstimate }.

Example.

gesture_swipe({ from: { x: 200, y: 600 }, to: { x: 200, y: 200 }, durationMs: 250 })
// → { "ok": true, "from": {…}, "to": {…}, "steps": 16, "durationMs": 250 }

gesture_pinch({ coords: { x: 512, y: 400 }, scale: 2, steps: 20 })   // pinch-out = zoom in
// → { "ok": true, "coords": {…}, "scale": 2, "steps": 20, "startOffset": 40, "endOffset": 80 }

Per-engine posture. All five run on chromium and android. On firefox, webkit and safari they return a structured refusal, { ok:false, error, engine, hint, tokensEstimate }, because the touch pipeline dispatches through CDP Input.dispatchTouchEvent and those engines expose no equivalent. The refusal comes from the action substrate, so an engine that can dispatch touch by other means answers instead of being refused on a CDP question; the error line is unchanged from earlier versions.

Multi-touch fan-out by hand: for gestures the canned compounds don’t cover (e.g. three-finger rotate), dispatch a sequence of touch_start / touch_move / touch_end calls with distinct identifier values per finger. The CDP touch pipeline maintains active touchpoint state across dispatches as long as the identifiers stay consistent. Note that Chromium fires a separate DOM touchstart / touchend for each finger added or lifted (where a browser could send one event with multiple changedTouches), even when you batch multiple points into one CDP dispatch.

Network route mocking: route / route_queue / unroute

Drive Playwright request interception for race-condition QA, per-session (discarded with the session).

• route({ urlPattern, method?, status?, body?, contentType?, delayMs?, session? }): fulfil every request matching urlPattern (Playwright glob) with one canned response; non-matching method falls through to the real network.
• route_queue({ urlPattern, method?, responses:[{status?,body?,contentType?,delayMs?}], session? }): fulfil successive matches from responses[] (one per request, in order); once exhausted, matches hit the real network. Each response has its own delayMs: give response #1 a long delay and #2 a short one to make backend responses arrive out of request order (the exact “response order ≠ request order” failure class).
• unroute({ urlPattern?, method?, session? }): remove one route, or (no urlPattern) every route this session registered.

Example.

// Force the error path:
route({ urlPattern: "**/api/records*", method: "GET",
        status: 500, body: "{\"error\":\"boom\"}", contentType: "application/json" })

// Reproduce out-of-order responses (response #2 lands before #1):
route_queue({ urlPattern: "**/api/save", responses: [
  { status: 200, body: "{\"rev\":1}", delayMs: 1200 },
  { status: 200, body: "{\"rev\":2}", delayMs: 50 },
] })

unroute({})   // drop every route this session registered

Network + CPU emulation: network_emulate / cpu_emulate

Throttle the session’s network conditions and the renderer CPU. For flaky-mobile / offline / “works on M3, breaks on Chromebook” repros against a real backend, without a real lab device. Both are per-session, both persist across navigation (re-applied on main-frame framenavigated in case a renderer swap drops the CDP override), both compose with route_queue: a route’s delayMs stacks ON TOP of latencyMs.

• network_emulate({ offline?, latencyMs?, downloadBps?, uploadBps?, packetLoss?, session? }): wraps CDP Network.emulateNetworkConditions. offline:true wins over latency / bps. downloadBps / uploadBps are bytes/sec (0 / unset = unthrottled). packetLoss is a 0..1 hint (most Chromium builds ignore it). Empty input (or {offline:false} with nothing else set) resets to no throttle. Returns { ok, applied:{offline, latencyMs, downloadBps, uploadBps, packetLoss?}, reset, warning?, tokensEstimate }.
• cpu_emulate({ throttleRate?, session? }): wraps CDP Emulation.setCPUThrottlingRate. throttleRate: 1 = no throttle (and is the reset path); 2 = 2× slowdown; 4 to 6 = mid-to-low-end mobile. Independent of network_emulate; call both for a full low-end-device repro. Returns { ok, applied:{throttleRate}, reset, warning?, tokensEstimate }.

Example.

network_emulate({ latencyMs: 400, downloadBps: 187500 })   // ~1.5 Mbps down + 400 ms RTT
cpu_emulate({ throttleRate: 4 })                           // mid-tier mobile CPU
// reset both:
network_emulate({})
cpu_emulate({ throttleRate: 1 })

Composition: route_queue({ urlPattern:"**/api/*", responses:[{delayMs:400, body:"…"}] }) + network_emulate({ latencyMs:200 }) means the matched request waits ~200 ms of emulated link latency before the route handler’s 400 ms delay fires, then fulfils. The two delays stack.

BYOB / attached Chrome: the override applies to the attached browser’s page and stays in effect after browxai detaches, until the human resets DevTools’ Network / Performance panels or closes the page. Both tools surface warning on the result in attached session mode so the operator knows to reset.

Clock control: clock

Drive the page’s virtual clock deterministically, for date-sensitive flows (renewal dates, “today” filters, scheduling, expiry edges) where rewinding Date.now() to a known instant beats matching test data to wall time. Wraps CDP Emulation.setVirtualTimePolicy. Per-session; persists across navigation (re-applied on main-frame framenavigated in case a renderer swap drops the policy). Independent of network_emulate / cpu_emulate; compose freely with any combination.

• clock({ mode: "freeze", atIso?, session? }): pause virtual time at atIso (or wall-clock now if omitted). CDP policy: pauseIfNetworkFetchesPending (network keeps running so the page can still load assets; the JS clock is held).
• clock({ mode: "advance", byMs?|atIso?, session? }): jump the clock by byMs (relative, max 1 year) or to absolute atIso (exactly one of the two), then re-pin. Subsequent advances accumulate from the cached anchor, not wall-clock.
• clock({ mode: "release", session? }): resume real time.

Returns { ok, applied:{ mode, nowIso, paused }, warning?, tokensEstimate }.

Example.

clock({ mode: "freeze", atIso: "2026-12-31T23:59:00Z" })   // test the year-end expiry edge
clock({ mode: "advance", byMs: 120000 })                    // jump 2 minutes, stay pinned
clock({ mode: "release" })                                  // back to real time — always do this on BYOB

BYOB / attached Chrome: the virtual-time policy stays in effect on the attached browser until released (mode:"release"), reloaded, or the page is closed. A page that displays a wall-clock-looking time which has actually been frozen is a debugging trap; the result surfaces a warning in attached session mode.

Deterministic Math.random: seed_random

Override the page’s Math.random with a Mulberry32 PRNG seeded from a caller-supplied integer, for flake repros where unseeded randomness drives id generation, dice / card / A-B picks, or jittered retry timing. Injected via Playwright addInitScript so every new document in the session (including subsequent navigations) bootstraps the same override; the current page’s main realm is re-seeded immediately so the effect is visible without navigating. Per-session; persists across navigation (re-applied on main-frame framenavigated, mirroring network_emulate / clock).

• seed_random({ seed, session? }): seed is a non-negative integer in [0, 2^32 - 1] (the Mulberry32 state domain; 0 is valid). Returns { ok, applied:{seed}, warning?, tokensEstimate }. Re-calling with a different seed swaps the active seed on both the current realm and any future document bootstrap. Example: seed_random({ seed: 1337 }) before re-running a flake repro makes every Math.random-driven branch take the same path run after run.

MVP scope: only Math.random is overridden. crypto.randomUUID / crypto.getRandomValues are NOT touched: web-crypto is a much bigger deterministic-stub surface and is left to a future tool. Workers (Web / Service) are out of scope; the init script runs in document realms only.

BYOB / attached Chrome: the override is installed on the attached browser’s BrowserContext and stays in effect for as long as the context lives, even after browxai detaches; surfaced as a warning in attached session mode.

HAR record / replay: start_har / stop_har + open_session({har}) / open_session({hars})

Full-session reproducibility: capture every request the page made into a HAR (HTTP Archive) file, then later replay a session against that archive instead of the live network. Two recording entrypoints + one replay entrypoint:

• start_har({ path?, mode?, content?, urlFilter?, session? }): begin HAR recording on a live session via context.routeFromHAR(path, {update:true}). From the next request onward every page network event is logged into an in-memory HAR. path is a workspace-rooted .har path (a path outside $BROWX_WORKSPACE, or a .zip path, is rejected); default is <workspace>/har/<session-id>-<ISO>.har. mode = "full" (default, full HAR with sizes/timing/cookies) or "minimal" (just enough for routeFromHAR to replay). content = "omit" (default: no bodies), "embed" (bodies inlined) or "attach" (sidecar files). embed and attach write request and response bodies to disk, so they also need the off-by-default network-body capability; without it the call is refused with requiredCapability: "network-body". urlFilter narrows to matching requests. Returns { ok, session, path, mode, content, replacedPrior, finalizesOn:"close_session", hint, tokensEstimate }. Re-calling start_har while a recorder is active transparently stops the prior one and swaps targets (replacedPrior:true). Capability action.
• stop_har({ session? }): remove the HAR recording route so further requests aren’t logged. Returns { ok, session, wasActive, path?, finalized:false, nativeRecord, har?, inlineBytes?, hint, tokensEstimate }. If the file is already on disk and under ~256 KB, it’s also inlined on the result. Capability action.
• open_session({ har: { path?, mode?, content?, urlFilter? } }): wire HAR at context creation via Playwright’s native recordHar option (the blessed primitive when you know up-front you want a HAR for the whole session). content defaults to "omit" and needs network-body for embed / attach, as on start_har. Honoured on persistent + incognito; ignored on attached (consumer’s Chrome is not-owned; a runtime start_har is the BYOB path). Once wired this way, start_har refuses, and stop_har reports the constraint and a no-op (the native primitive can’t be toggled off mid-session). stop_har will return nativeRecord:true here.
• open_session({ hars: ["a.har", "b.har", …] }): REPLAY one or more HAR files against the new session. Each file is wired with context.routeFromHAR(file, {notFound:"fallback"}) immediately post-create: requests in the archive are served from it, anything missing falls through to live network. Workspace-rooted paths only; a missing file errors (no silent fallback on a typo). Compose multiple HARs to layer fixtures.

Finalize timing: Playwright writes the .har file on context.close(). There is no public mid-session flush. The canonical flow is start_har, then drive the page, then stop_har (optional), then close_session, then read the .har from disk. Both start_har and open_session({har}) honour this; every result carries finalizesOn:"close_session" so the constraint is visible to the agent.

Credential headers and secrets on close: when the session closes and the file is written, browxai rewrites it: the Cookie, Set-Cookie and Authorization header values (and the cookies arrays Playwright parses from them) are emptied with the names kept, and with secrets on, registered values become <NAME> in header values, URLs, query values, request bodies and text response bodies, including text sidecar files beside a .har. Without network-body, request bodies are emptied as well: postData.text is removed and postData.params is set to [], because Playwright writes urlencoded form values there even with content:"omit". With network-body on, request bodies are kept (masked) as before. Base64 bodies are left as written. A .zip HAR path is refused by start_har and open_session({har}); use a .har path. Every file a session recorded to (a second start_har adds one) is scrubbed on close. Replaying a scrubbed HAR serves responses without those header values.

Re-recording within a session: stop_har then start_har again with a fresh path works cleanly; on the runtime path the prior recorder is transparently flushed before the new one wires. On the native (open_session({har})) path the recorder is locked to the session’s lifetime; close + reopen the session to swap.

Inline cap: stop_har inlines the .har on the result when the file exists and is ≤ ~256 KB; otherwise the caller reads it from path after close_session.

Example (record once, replay forever). Replay serves the bodies the archive holds, so record with content: "embed" (needs network-body). A HAR recorded with the default omit replays empty bodies.

start_har({ path: "har/checkout.har", content: "embed", urlFilter: "**/api/**" })
// …drive the checkout flow…
stop_har({})
close_session({ session: "default" })   // finalizes the file on disk

// Later — replay the same flow against the archive, no live backend needed:
open_session({ session: "replay", hars: ["har/checkout.har"] })

Video recording: open_session({recordVideo}) / stop_video / get_video

Record every page in the session as a .webm via Playwright’s native recordVideo context option. The same shape as the native HAR path (open_session({har})): video is wired at context creation and finalized when the context closes. Playwright does NOT expose a runtime start or a mid-context flush, so the tool surface is the symmetric stop + read pair, with no start call. Capability file-io (sibling to upload_file / download_get).

• open_session({ recordVideo: { path?, size? }, … }): wire video at context creation via Playwright’s native recordVideo option. path is workspace-rooted (a path outside $BROWX_WORKSPACE is rejected); default is <workspace>/videos/<session-id>-<ISO>.webm. size is {width, height} (Playwright’s option; defaults to viewport scaled to fit 800x800). Honoured on persistent + incognito (we own the context); refused on attached with a structured error (the consumer’s Chrome is not-owned, so we don’t wire context-creation primitives on it). Returns a video: { path, size?, finalizesOn:"close_session" } field on the open_session result.
• stop_video({ session? }): signal that the recording should be finalized. The.webm is written to disk only when the session closes (close_session). Playwright provides no mid-context flush on the native recordVideo primitive. This call marks the recorder as pendingFinalize:true and returns the reserved target path; the actual file appears on disk after close_session. Returns { ok, session, wasActive, path?, pendingFinalize, finalized:false, finalizesOn:"close_session", hint, tokensEstimate }. Returns a structured error on attached sessions or when no recorder is active. Capability file-io.
• get_video({ format?, session? }): read the finalized video off disk. format:"path" (default) returns the absolute path + on-disk size. format:"bytes" additionally inlines as base64 when the file is under ~1 MiB; larger files return path + tooLargeToInline:true so the caller reads them off disk. Returns { ok, session, path, bytes, format, videoBase64?, tooLargeToInline?, hint, tokensEstimate }. Returns a structured error when the file isn’t yet on disk (the get-before-close_session case, pointing the caller at close_session), on attached sessions, or when no recorder was wired. Capability file-io.

Finalize timing: the canonical flow is open_session({recordVideo}), then drive the session, then stop_video (optional, signals intent), then close_session, then get_video. Playwright finalizes the .webm on context.close() (which close_session triggers); the registry’s teardown then calls page.video().saveAs(targetPath) for a deterministic output filename. The finalizesOn:"close_session" field on every result envelope makes the constraint visible.

No runtime start: Playwright’s recordVideo is a context-creation primitive; there is no public mid-context start. To swap target paths in one session: close_session, then open_session again with the new recordVideo.path.

BYOB / attached Chrome: open_session({recordVideo}) is refused on attached sessions with a hard error. The consumer’s Chrome is not-owned; we don’t wire context-creation primitives on it. Open a managed persistent or incognito session with {recordVideo:{...}} to record.

Inline cap: get_video({format:"bytes"}) inlines as base64 when the file is ≤ ~1 MiB; larger files return tooLargeToInline:true and the caller reads from path.

Example.

open_session({ session: "rec", mode: "incognito", recordVideo: { path: "videos/run.webm" } })
// …drive the flow…
stop_video({ session: "rec" })          // optional — marks pendingFinalize
close_session({ session: "rec" })       // Playwright writes the .webm here
get_video({ session: "rec", format: "path" })
// → { "ok": true, "path": "/…/.browxai/videos/run.webm", "bytes": 2914308, "format": "path" }

Performance tracing: perf_start / perf_stop / perf_insights

“This click took 4s. Why?” has no diagnostic surface in the read-only tools: a screenshot/snapshot/network slice shows what happened and nothing about why it was slow. These three tools wrap CDP Tracing.start / Tracing.end to produce a chromium-format trace file (the same shape DevTools’ Performance panel and chrome://tracing consume), then extract structured insights from it. Per-session; one trace lifecycle at a time. All three are under capability action (perf_stop writes a file).

• perf_start({ categories?, session? }): arm a CDP trace on this session. Omit categories for the DevTools-Performance-equivalent default (devtools.timeline, loading, blink.user_timing, frame, latency). Idempotent restart: calling perf_start while a trace is already running cleanly stops the in-flight one (events discarded) and starts fresh, so an agent that lost track of state always recovers by calling again. Returns { ok, running:true, categories, restarted, warning?, tokensEstimate }.
• perf_stop({ path?, session? }): stop the in-flight trace and flush events to a workspace-rooted JSON file. Default path: <workspace>/perf-traces/<sessionId>-<ts>.json (path-traversal rejected; path must resolve under $BROWX_WORKSPACE). Safe to call any number of times: if no trace is running, returns notRunning:true instead of an error. Returns { ok, path, bytes, eventCount, categories, durationMs, summary:{ longTaskCount, layoutShiftCount, renderBlockingCount, lcpCandidateCount }, hint, warning?, tokensEstimate }. The summary is the one-glance answer; perf_insights is the detailed read.
• perf_insights({ tracePath, session? }): read a written trace JSON and return structured insights: longTasks (≥50 ms blocking work, sorted longest-first, top-50), layoutShifts (per-shift score + hadRecentInput), renderBlocking (CSS / sync-JS critical-path resources with duration), lcpCandidates (final candidate = effective LCP), navigation (FP / FCP / DCL / load milestones relative to navigationStart), plus totals aggregates. tracePath is workspace-rooted; rejected if it escapes $BROWX_WORKSPACE. Same JSON format the DevTools Performance panel consumes, so bring-your-own trace works too. Returns { ok, tracePath, eventCount, metadata, insights, tokensEstimate }.

Composition: typical “diagnose a slow click” sequence as a single batch: perf_start, then the action (click / fill / etc.), then perf_stop, then perf_insights({tracePath}). All four are batch-allowed; the per-call capability gates still apply.

Example.

perf_start({})
click({ ref: "e42" })          // the slow interaction
perf_stop({})
// → { "ok": true, "path": "/…/perf-traces/default-1765540264.json", "eventCount": 18211,
//     "summary": { "longTaskCount": 3, "layoutShiftCount": 1,
//                  "renderBlockingCount": 0, "lcpCandidateCount": 2 }, … }
perf_insights({ tracePath: "perf-traces/default-1765540264.json" })
// → { "ok": true, "insights": { "longTasks": [{ "durationMs": 412, … }], "layoutShifts": […],
//     "navigation": { "fcpMs": 840, … }, "totals": {…} } }

BYOB / attached Chrome: perf_stop is required to release the trace buffer on the human’s Chrome. close_session also cleans up on its way out (best-effort), and perf_stop surfaces a warning in attached mode so the operator sees that the buffer was released.

V8 heap snapshots: heap_snapshot / heap_retainers

“This page slowly leaks memory. What’s still holding the old DOM tree alive?” has no diagnostic surface in the read-only tools either: a snapshot shows what’s on the page now, not what’s still retained from a previous state. These two tools wrap CDP HeapProfiler.takeHeapSnapshot to produce a V8 .heapsnapshot (the format chrome://inspect’s Memory panel consumes on drag-and-drop) and run a structured retainer query against it. One-shot (a heap snapshot is a point-in-time capture, not a recording window; no start/stop pair). Both are under capability action (heap_snapshot writes a file; heap_retainers is kept under the same capability so a memory-diagnosis batch doesn’t have to juggle two grants).

• heap_snapshot({ path?, session? }): take a V8 heap snapshot on this session’s target. Default file path: <workspace>/heap-snapshots/<sessionId>-<ts>.heapsnapshot (path-traversal rejected; path must resolve under $BROWX_WORKSPACE). Snapshots are heavy (tens to hundreds of MiB on a real page); don’t take them in a tight loop. Returns { ok, path, bytes, hint, warning?, tokensEstimate }. Drag-and-drop the file onto chrome://inspect’s Memory panel for the full interactive view.
• heap_retainers({ snapshotPath, query:{ name?, type?, nameMatch? }, session? }): parse a written snapshot and report top retainers (sorted by retainer self-size desc, capped at 50) of nodes matching the query. query.name defaults to exact string match against the node’s V8 string-table name; use nameMatch:"substring" for containment. query.type filters by V8 node-type ("closure", "object", "hidden", …). At least one of name / type is required; a match-everything query is never the right answer. snapshotPath is workspace-rooted; rejected if it escapes $BROWX_WORKSPACE. Pure file read + in-process parse, no CDP touch, so it works against snapshots saved by heap_snapshot OR exported from DevTools. Returns { ok, snapshotPath, summary:{nodeCount,edgeCount,stringCount,totalSelfSize}, matchCount, retainers:[{ retainerName, retainerType, retainerSelfSize, edgesToMatches, sampleHeldNodes:[] }], sampleMatches:[], warnings?, tokensEstimate }.

Composition: typical “find the leak” sequence as a single batch: trigger the suspect interaction (click / fill / …), then heap_snapshot, then heap_retainers({ snapshotPath, query:{ name:"MyClass" } }). Both are batch-allowed; the per-call capability gates still apply.

Example.

heap_snapshot({})
// → { "ok": true, "path": "/…/heap-snapshots/default-1765540264.heapsnapshot", "bytes": 48211230 }
heap_retainers({ snapshotPath: "heap-snapshots/default-1765540264.heapsnapshot",
                 query: { name: "RecordStore", nameMatch: "substring" } })
// → { "ok": true, "matchCount": 12,
//     "retainers": [{ "retainerName": "recordCache", "retainerType": "object",
//                     "retainerSelfSize": 1048576, "edgesToMatches": 12, "sampleHeldNodes": […] }], … }

BYOB / attached Chrome: a snapshot captured against the human’s Chrome is written to $BROWX_WORKSPACE as usual. heap_snapshot surfaces a warning in attached mode so the operator sees that the capture ran on the human’s session.

Performance audit: perf_audit / coverage_start / coverage_stop / layout_thrash_trace / memory_diff

browxai’s perf surface goes beyond measurement (perf_start / perf_stop / perf_insights) to actionable. The four tools below give an agent a structured audit with remediation hints, dead-code coverage reports, focused layout-thrash diagnosis, and a pure-function heap-snapshot diff.

• perf_audit({ session?, categories?, durationMs?, format? }): the headline tool. Records a CDP trace + JS/CSS precise coverage + network response metadata for durationMs (default 5000, max 30000), then runs 8 pluggable category analysers against the assembled context and composes a report. Returns { ok, summary:{score, topIssues:[{category, severity, title}]}, byCategory:{[cat]:{issues[], remediations[]}}, evidence:{tracePath, coveragePath?}, durationMs, categoriesRun, warnings, tokensEstimate }. Categories (default = all): render-blocking (resources blocking first paint), unused-code (scripts/stylesheets with <30% usage), oversize-images (>500KB), layout-thrashing (>5 forced sync layouts in window), long-tasks (>50ms main-thread blockers), leak-suspects (>10% retainer growth, fed by memory_diff data on the context), cache-opportunities (static assets with missing/short Cache-Control), font-loading (fonts loaded >200ms after document start). format (default "summary") caps each category to 3 issues + 3 remediations AND enforces a 2000-token body budget: over-budget low/medium severity entries are dropped + a warnings[] entry surfaces it. "full" is unbounded. Score = 100 − sum(severity-weight × issue-count) floored at 0 (high=10, medium=4, low=1). Evidence files (workspace-rooted): the trace under <workspace>/perf/<sessionId>-audit-<ts>.json + a coverage JSON alongside; both load in DevTools’ Performance / Coverage panels. The category set is internally pluggable: adding a category = adding a registry entry in src/page/perf-audit.ts; the public surface doesn’t change. Capability read.
• coverage_start({ session? }): arm precise JS + CSS coverage tracking on this session, wrapping CDP Profiler.startPreciseCoverage (per-script byte-level use counts) + CSS.startRuleUsageTracking (per-stylesheet rule-level use counts) in lockstep. Idempotent restart: calling coverage_start while a tracker is already running cleanly stops the in-flight one (results discarded) and starts fresh. Returns { ok, running:true, startedAt, restarted, warning?, hint, tokensEstimate }. Capability action.
• coverage_stop({ session? }): stop both trackers and return the parsed report. Returns { ok, jsCoverage:[{url, totalBytes, usedBytes, usagePercent, deadRanges?}], cssCoverage:[{url, totalBytes, usedBytes, usedRules, totalRules, usagePercent, deadRules?}], durationMs, tokensEstimate }. JS coverage semantics: V8’s detailed coverage emits ranges per function; a count:0 root range = the whole function is dead, a count:1 root with count:0 sub-blocks = dead conditional branches. We follow the same algorithm DevTools’ Coverage panel uses. usagePercent is the headline metric the agent reads: <30 indicates substantial dead code (the audit’s unused-code analyser flags it). deadRanges / deadRules are top-50 byte ranges per file. Safe to call any number of times: if no tracker is running, returns notRunning:true and no error. Pure parsing past the CDP fetches; no file is written, and the caller decides whether to persist. Capability read (non-mutating composition past the CDP stop). perf_audit calls both internally; use these directly only for raw reports or longer windows.
• layout_thrash_trace({ session?, durationMs? }): focused CDP trace just for forced synchronous layouts + LayoutShift + Recalc Style events, aggregated by originating call-stack. Returns { ok, forcedLayoutsCount, layoutShiftsCount, eventsByOrigin:[{originatingStack, count, totalDurationMs}], tracePath, durationMs, warning?, tokensEstimate }. originatingStack reads from the trace’s stackTrace field on each event (chromium populates it when DevTools is attached); "<anonymous>" when no stack is available. tracePath is a workspace-rooted JSON file under <workspace>/perf/<sessionId>-layout-thrash-<ts>.json, loadable in DevTools’ Performance panel for the full visual. Capped at the top 50 origins, sorted by count desc. durationMs default 5000, max 30000. Capability read.
• memory_diff({ beforePath, afterPath, session? }): pure-function consumer of two .heapsnapshot files (the format heap_snapshot writes / DevTools exports). No browser interaction. Groups nodes by ${type}:${name}, sums self_size per group, reports per-group deltas. Returns { ok, retainerGrowth:[{node, type, sizeBefore, sizeAfter, deltaBytes, deltaPercent}], summary:{totalGrowth, top3Growers:[{node, deltaBytes, deltaPercent}]}, tokensEstimate }. Noise filter: groups whose |deltaBytes| < 1024 are dropped, since sub-KB noise is rampant in V8 heaps and crowds the actionable signal. deltaPercent is a number (or the string "+inf" when sizeBefore:0). Sorted by deltaBytes desc, capped at 100 rows. Both paths are workspace-rooted; rejected if they escape $BROWX_WORKSPACE. Capability read.

Example.

perf_audit({ durationMs: 8000 })
// → { "ok": true,
//     "summary": { "score": 72, "topIssues": [
//        { "category": "render-blocking", "severity": "high", "title": "2 stylesheets block first paint" },
//        { "category": "unused-code", "severity": "medium", "title": "vendor.js is 78% unused" }] },
//     "byCategory": { "render-blocking": { "issues": […], "remediations": […] }, … },
//     "evidence": { "tracePath": "/…/perf/default-audit-1765540264.json", "coveragePath": "/…" } }

Composition: typical “what should I fix on this page?” sequence as a single batch: navigate, then perf_audit({format:"summary"}). The summary stays small enough for one round-trip; the agent can perf_audit({format:"full"}) follow-up if a category needs the detail. For leak-detection: heap_snapshot (before suspect interaction), then drive the action, then heap_snapshot (after), then memory_diff({beforePath, afterPath}), then perf_audit with memoryDiff flowing through the context (the audit’s leak-suspects category consumes it). All five new tools are batch-allowed; per-call capability gates still apply.

BYOB / attached Chrome: perf_audit / coverage_stop / layout_thrash_trace release any in-flight trace + coverage state on the human’s Chrome before returning. close_session also cleans up on its way out (best-effort). Each surfaces a warning in attached mode so the operator sees that buffers were released.

act_and_diff({ action, scope?, session? })

Run one action and report the DOM changes it caused within a scope, for selection-heavy UIs where “which clip/row became selected” shows only as class / aria-* / data-* / inline-style changes, invisible to snapshot/find/text_search. Captures a structural DOM map before, dispatches the inner action, captures after, diffs. action is {tool,args} from the batch whitelist (inner tool’s capability + deadline still apply). Returns { action: <inner result>, diff: { changed:[{ path, tag, testId, classDelta:{added,removed}, styleDelta, attrDelta }], added, removed, counts } }. scope (CSS selector, default document.body) must exist before and after the action.

Example.

act_and_diff({ action: { tool: "click", args: { ref: "e21" } }, scope: ".timeline" })
// → { "action": {…},
//     "diff": { "changed": [{ "path": "div.timeline > div:nth-child(4)", "tag": "div",
//                "testId": "clip-4", "classDelta": { "added": ["selected"], "removed": [] },
//                "attrDelta": { "aria-selected": ["false", "true"] } }],
//               "added": 0, "removed": 0, "counts": { "changed": 1 } } }

act_and_wait_for_network({ action, match, timeoutMs? })

Run one action and wait for a specific network response. Async SPAs fire follow-up requests after the action-result window, so ActionResult.network misses them. The waiter is armed before the action dispatches (no race). match = urlPattern (case-insensitive substring) / method / status, at least one required. Returns { action: <inner result>, network: { matched, method?, url?, status? } } (url redacted, same as network_read). timeoutMs = max wait (default 10000).

Example.

act_and_wait_for_network({
  action: { tool: "click", args: { ref: "e42" } },
  match: { urlPattern: "/api/orders", method: "POST" },
  timeoutMs: 8000,
})
// → { "action": {…}, "network": { "matched": true, "method": "POST",
//     "url": "https://api.example.com/api/orders", "status": 201 } }

poll_eval({ expr, intervalMs?, timeoutMs?, session? })

Repeatedly evaluate a JS expression until it returns truthy or timeoutMs elapses, for waiting on async job completion / store updates without ad-hoc in-page loops (a long in-page promise would trip the anti-wedge deadline). Returns { ok, truthy, value, polls, elapsedMs, timedOut }. The value is page-controlled and untrusted, like eval_js. Requires the off-by-default eval capability. intervalMs default 250 (min 50); timeoutMs default 5000. Prefer wait_for({text}) when the condition has any visible-DOM signal; that path needs no capability.

Example.

poll_eval({ expr: "window.__jobs?.pending === 0", intervalMs: 500, timeoutMs: 10000 })
// → { "ok": true, "truthy": true, "value": true, "polls": 6, "elapsedMs": 2750, "timedOut": false }

Visual regions + cross-session + session report

• screenshot_region({ box, session? }): PNG of an arbitrary viewport rectangle (not an element), for virtualised timelines / canvas / unlabelled positioned regions.
• screenshot_marks({ candidates, label?, session? }): composed PNG with numbered bounding boxes painted over the supplied candidates: the set-of-marks primitive multimodal agents reach for when they want to ground a vision read against a small palette of stable refs (“click 2” instead of estimating a coordinate). Each candidate is either a bare {ref} (its box comes from the ref table, resolved the way a click resolves it; a ref the table has never seen is named in warnings) OR a full find() candidate row passed through ({ref, role, name, testId, bbox}: fast path, no extra tree walk). label is "index" (default), then paints 1..N array positions, "ref", then paints the existing eN, "role", then paints the role for visual grounding. The numbering scheme shares the existing name_ref / eN namespace, with no parallel ID space, so the result’s mapping[index] === ref and an agent can address either way (click({ref: mapping[2]})). Candidates with bbox:null (clipped / off-screen) are kept in marks with painted:false so the mapping stays complete. Image-library choice: a transient in-page DOM overlay drawn over the viewport, screenshot taken, overlay removed: dependency-free (browxai has no Node-side image library in dependencies) and running in the same coordinate space find().evidence.bbox reports., then { marks:[{index, ref, role?, name?, testId?, bbox, painted}], mapping:{"1":"eN", …}, warnings } + the PNG.
• name_region({ name, box, session? }) / region({ name, session? }): bind a viewport rectangle to a mnemonic and resolve it back to { box, center }; pass center to click({coords}) to act on the same media segment without coordinate drift across a sub-agent’s select, copy and re-check. Example: name_region({ name: "clip_4", box: { x: 220, y: 410, width: 80, height: 32 } }), then region({ name: "clip_4" }), then { box, center: { x: 260, y: 426 } }, then click({ coords: center }).
• cross_session_sample({ action, actionSession, sampleSession, metric, durationMs, … }): drive an action in one session and trace a metric in another over the same window, in one call, for realtime-propagation assertions (“an action in session A should reflect in session B”). Returns { action, sample }.
• export_session_report({ note?, session? }): bundle a session’s QA evidence (url, redacted like network_read URLs: no query string, fragment or userinfo; console errors, recent network summary, named regions, live sessions, note) into one JSON object for multi-agent QA review. Returned, not written to disk. When a start_recording({replay}) writer completed on this session, the report links the resulting archive under replayArtifact: { path, absolutePath }. The archive carries real page content and is as sensitive as the session was, so treat it accordingly.
• session_metrics({ session? }) returns a per-session cumulative tool-call rollup: callsByTool, durationMsByTool, errorsByTool, tokensEstimateSum, capabilityDenials, sessionStartedAt, sessionDurationMs. Pair with export_session_report for a full audit pass: that one is QA evidence (what the page looked like / what fired); this one is dispatch evidence (what the agent ran, how expensive it got, what got denied). Read-only: it piggybacks on the per-call envelope data the server already has; no new instrumentation, no per-call disk writes. Capability denials (gate-blocked calls) are counted as a session-wide scalar: the denial shape is a property of the capability config, not the tool, and the count alone is the actionable signal. Per-tool errors count ok:false results that were NOT denials. Note: this is dispatch-level rollup; for an rrweb-based session-replay artifact (a la Browserbase) use start_recording({replay}) / end_recording behind the off-by-default replay capability. export_session_report covers the JSON-evidence half of the pairing and links the artifact under replayArtifact when one exists.

export_playwright_script({ path?, session? })

Lower a session’s recorded action trace into a runnable @playwright/test spec file, adjacent to export_session_report (QA evidence) and end_recording (the site-docs flow-file YAML); this one emits TypeScript a code-as-action consumer can run as the seed for a skill-compilation loop. Each recorded step lowers to ONE Playwright call using the BEST stable selectorHint captured at the time of the call (tier-1 attribute gives page.locator(...), tier-2 role+name gives getByRole({ name }), role-only / tier-5 gives getByRole() with a // TODO: fragile selector comment above the line so the consumer SEES the brittle spots). Coords-mode actions are not recorded by the action window, so the export never has to lower a non-replayable target, by construction.

Recorded reads lower too, so a read-shaped flow exports as a function instead of a macro:

Recorded read	Lowers to	Counted
extract	const <name> = { field: await <locator>.innerText(), … }	handled when every field had a selector to lower from
find	const <name> = <locator>;, the name the YAML locators: block uses	handled when the find resolved a candidate
eval_js	const <name> = await page.evaluate("<recorded expression>");	handled
snapshot	a comment naming the scope	unhandled: a serialised a11y tree is not a script step

Every const an extract or eval_js declares is logged at the end of the test body (console.log(JSON.stringify({ … }, null, 2))); that log is the exported flow’s output.

extract fields lower from the recorded schema’s x-browx-source.selector (attr, then getAttribute, value / prop:"value", then inputValue, otherwise trimmed visible text). A field that resolved through the implicit name-as-query rule has no selector to lower, so it emits null with a TODO and the step counts as unhandled. A scope that was a ref gets a TODO too, since refs are session-local, so the fields read page-wide until you narrow the root locator.

eval_js sits behind the off-by-default eval capability in browxai; Playwright runs the exported page.evaluate ungated, so the generated header states the provenance. The header also names the one-time npx playwright install chromium a fresh @playwright/test install needs.

Requires an active recording. Call start_recording({flowName}) first, drive the flow with the usual action and read tools, then call this. Export is inspect-style: it does NOT end the recording (use end_recording separately for the YAML flow-file).

With path, ALSO writes the source to a workspace-rooted .spec.ts file (path-traversal rejected; must resolve under $BROWX_WORKSPACE).

Capability read. Returns { ok, name, source, stats: { steps, handled, unhandled, fragile }, path?, bytes?, tokensEstimate }.

Example. After start_recording({flowName:"login"}) + a navigate + fill({ref:"e1",value:"alice"}) + click({ref:"e2"}) against a Sign-in button discovered via tier-2 role+name, calling export_playwright_script({path:"scripts/login.spec.ts"}) writes a file shaped like:

import { test, expect } from "@playwright/test";

void expect;

test("login", async ({ page }) => {
  await page.goto("https://app.example.com/login");
  await page.locator('[data-testid="username"]').fill("alice");
  await page.getByRole("button", { name: "Sign in" }).click();
});

Managed profile inventory: profile_status

Answers which profiles exist, how big they are, when they were last written, and which are open right now. A workspace that has accumulated dozens of profile directories has no TTL and no collection; modifiedAt is the signal you age them by.

profile_status({ profile? }): omit profile for the whole inventory; pass a name to scope to one ("default" is the default session’s <workspace>/profile dir). An unknown name returns an empty list; a missing profiles root returns an empty inventory. Capability read. Read-only: it opens nothing and touches no page.

Shape.

{
  "ok": true,
  "profilesRoot": "<workspace>/profiles",
  "count": 2,
  "profiles": [
    {
      "name": "job-search",
      "path": "<workspace>/profiles/job-search",
      "bytes": 48219043,
      "files": 1204,
      "modifiedAt": "2026-07-02T09:14:51.000Z",
    },
    {
      "name": "job-search-live",
      "path": "<workspace>/profiles/job-search-live",
      "bytes": 61044210,
      "files": 1533,
      "modifiedAt": "2026-09-09T14:52:07.000Z",
      "live": {
        "sessions": ["agent-a"],
        "cookieDomains": [".google.com", "mail.google.com"],
        "observedAt": "2026-09-09T14:52:09.102Z",
      },
      "savedAuthState": {
        "name": "job-search-live",
        "path": "<workspace>/.auth-states/job-search-live.json",
        "savedAt": "2026-08-30T11:02:00.000Z",
        "cookieDomains": [".google.com"],
        "originsWithLocalStorage": ["https://mail.google.com"],
      },
    },
  ],
  "warnings": [],
  "tokensEstimate": 214,
}

live is present only while a session is running out of that directory. savedAuthState is present only when an auth_save slot carries the same name. scanTruncated: true on a row means the directory walk hit its entry budget, so that row’s bytes / files / modifiedAt are lower bounds.

What the origin data is and is not. This tool does not report whether a profile is logged in anywhere, and no field should be read that way. Chromium encrypts cookie values with an OS-provided key (Keychain on macOS, DPAPI on Windows), so a closed profile’s authentication state is unreadable without launching the browser. What you get instead, each labelled by where it came from:

Field	What it means	When it was true
live.cookieDomains	Domains the open browser context holds cookies for. Cookies exist; they may not authenticate.	The instant of this call
savedAuthState.*	The contents of <workspace>/.auth-states/<name>.json. The link to the profile is the matching name only; browxai does not record which profile a slot was captured from.	savedAt, possibly expired
modifiedAt	Newest mtime under the profile directory.	On disk now

To learn a profile’s real auth state, open a session on it and navigate.

Example (pick the right profile, then collect the rest).

profile_status({})
// → job-search  last written 2026-07-02, no live session
//   job-search-live  live under session "agent-a", cookies for .google.com
open_session({ session: "mail", mode: "persistent", profile: "job-search-live" })

Profile snapshot / restore: profile_snapshot / profile_restore

Checkpoint and reset a persistent session’s profile directory for repeatable destructive authenticated-SPA tests.

• profile_snapshot({ snapshot, profile? }): copy the profile dir into <workspace>/profile-snapshots/<snapshot>. profile defaults to "default".
• profile_restore({ snapshot, profile? }): copy a named snapshot back over the profile dir. Refuses a snapshot profile_snapshot did not write, or whose files changed after it was taken: each snapshot carries a manifest with a digest of its files, MACed with a workspace key. Snapshots from earlier versions have no manifest and must be taken again. profile-snapshots/ is write-protected against every other tool.
• All sessions must be closed first (close_sessions({all:true})): copying a profile dir while Chromium has it open corrupts it; both tools refuse while any session is live. Names are letters/digits/._- only (no path traversal).

Example (repeatable destructive test).

close_sessions({ all: true })
profile_snapshot({ snapshot: "clean-login" })
// …run the destructive flow…
close_sessions({ all: true })
profile_restore({ snapshot: "clean-login" })   // back to the known-good state