Docs / Clients & integrations

Clients & Integrations

Two kinds of surface: integrations (how an agent drives the browser) and control surfaces (how a human manages accounts, permissions, billing, and approves vault requests).

Integrations (agent-facing)

All drive the same engine; exact tool and verb availability differs by transport.

The full MCP tool surface (built) lives in cmd/gb-mcp/tools.go and is no longer hand-duplicated here — a hand-maintained list drifted stale exactly once already (this section under-listed the real tool set, caught while building P2-210's capability-manifest conformance check). docs/conformance/compatibility-matrix.md is the current, generated, per-verb picture instead; regenerate it with go run ./internal/conformance/gen after any tool change.

Structured action responses are bounded before they are written to the node connection: snapshot/read/extract/extract_all/evaluate/cookies are capped at 1 MiB of final JSON, screenshot/mark at 32 MiB, and PDF at 64 MiB. An oversized response returns HTTP 422 with code: "action_output_too_large", the verb, and limit_bytes; it is never silently truncated. Heavy outputs also use one node-wide fail-fast slot, so a stalled or oversized request cannot queue an unbounded set of concurrent screenshot/PDF/cookie operations. Raw CDP remains the separate frame-preserving, policy-filtered tunnel with its 256 MiB frame backstop.

On a hosted session, screenshot and pdf accept an opt-in as_artifact:true (P1-114): instead of the normal inline png_base64/ pdf_base64 response, the control plane persists the result through the same durable artifact store recordings use and returns {artifact_id, kind, size, checksum, content_type, download_url} — useful when the caller doesn't want a large payload in the act response itself. The default (omitted or false) is unchanged. platform.Artifact now carries a kind ("recording" for every pre-existing row, "screenshot", or "pdf") and a checksum (SHA-256 hex); GET /api/artifacts and GET/DELETE /api/artifacts/{id} already worked generically across kinds and needed no changes.

record_stop becomes successful only after every requested media/DOM producer has stopped and the bundle is assembled. A teardown or assembly failure retains the spool for a retry and records degradation evidence instead of advertising an incomplete bundle as successful. Stop first pauses ordinary WAL fanout at the capture boundary. If teardown degrades, its marker and WAL detach share one terminal publication boundary, so later actions or watcher events cannot enter a retained spool. Both normal and degraded final drains and fsyncs use a five-second cleanup context independent of caller cancellation; a timeout retains the spool for the next record_stop, and no bundle manifest is exposed before that finish succeeds.

The node also enforces transient recording budgets: the WAL defaults to 256 MiB and video to 768 MiB. WAL overflow emits recording_degraded with phase:"wal_capacity" when the reserved terminal tail fits, then closes the subscriber. Video uses ffmpeg -fs and emits phase:"video_capacity" when the final file reaches its limit. In either case the node serializes browser teardown after the writer/encoder has exited and retains the bounded partial evidence as READY; it does not continue a browser session without its recording producer.

Control surfaces (human-facing)

Same backend (control-plane API); four front-ends.

Web console — https://app.getagentbrowser.com/app

• Sign up / log in, workspace & member management.
• API keys (create, reveal once, list, and revoke).
• Vault: register username/password/TOTP credentials with exact origin allowlists and optional approval; review the audit log.
• Playground: open, drive, screenshot, and close the console's current hosted session; see hosted and self-hosted render-node capacity.
• Live session viewer: share a read-only SSE event feed for a recording session; persisted recording video is available for download after the session closes.
• Approvals: approve/deny pending vault requests from the overview.
• Recordings & automation: download/delete capture bundles; configure signed webhooks and repeating screenshot/PDF/read/extract jobs.
• Usage & billing: workspace usage and free-tier state; checkout and billing portal links when billing is configured.
• Spend (admin/owner): create/edit/delete consequence-based spend policies (merchant/category/account match, per-action limits, hard-deny), and approve/deny agent spend that a policy paused — mirrors the existing vault-approval UX pattern.
• SSO / domain claims / SCIM / service accounts (admin/owner-only, nav links hidden for non-admins): configure OIDC SSO for the workspace, claim + DNS-verify owned domains with an optional auto-join role, mint/rotate the SCIM provisioning bearer token, and create/disable/delete scoped non-human service-account API keys.
• Workflows: author, version, and run reusable JSON-defined workflow templates (the same action verbs and typed parameter/output-schema shape the API accepts) directly from the console, with run history.
• Identities: create/rotate/delete managed reusable logins that bind a vault credential to an origin allowlist, MFA policy, and concurrency policy; view health status and last-login.
• Legal hold (Settings, admin/owner-only): place/clear a workspace-wide hold that blocks account/workspace deletion and is skipped by retention sweeps.
• Tech: the console HTML/JavaScript and its cookie-authenticated API are served by gb-server; Cloudflare routes the public hostname to that control plane.

Electron desktop app

• A packaged native shell around the hosted web console, with safe external-link handling and menu shortcuts for the console and marketing site.
• The “Run a local node…” item is an informational source-build guide. The app does not bundle, install, start, stop, or monitor gb-noded.
• Self-hosting currently requires an operator to build gb-noded from this repository and configure its unique node credential. One-click node management remains roadmap work and is not part of the downloadable installers.

iOS & Android apps — ✅ built & proven live

• Approve/deny vault requests on the go. Both apps sign in via the token API (POST /api/auth/login → bearer), poll /api/pending, and Approve/Deny. iOS keeps the bearer in a device-bound Keychain item; Android encrypts it with a non-exportable Android Keystore key. Both synchronously invalidate local access with a durable logout marker, revoke only their presented session through POST /api/auth/logout, then physically delete the stored bearer. iOS also unregisters and locally disables APNs delivery on logout; registering the same physical token to a new account atomically transfers its server ownership.
• iOS (apps/ios/): SwiftUI approvals plus read-only vault metadata and usage; built with xcodegen + xcodebuild for the iPhone simulator on the Mac. Verified live — the Approvals screen showed a real pending request from the control plane and approving it unblocked the agent's login.
• Android (apps/android/): dependency-free Kotlin (framework Views + org.json); sign-in, approval resolution, read-only vault metadata (VaultActivity) and usage (UsageActivity), session control (sign out everywhere), and FCM push registration/receipt (PushManager/AgentBrowserMessagingService), matching iOS feature-for-feature except billing (neither app has a billing screen). gradle :app:assembleDebug → app-debug.apk. Verified live in a real hardware- accelerated emulator (GCP resolute-parity-484218-g1 project, agentbrowser-android-emulator) — real login, real Approvals/Vault/Usage API round trips, real push-permission dialog confirming FCM registration.
• Next: production APNs credentials (blocked on Eric providing Apple Developer Program access — GoodQA AgentBrowser #39; the client-side APNs code itself is already built and verified), a full cross-device FCM send-and-receive round trip, and mobile billing views (neither app has one, matching parity by omission).
• Build toolchain runs on an internal, tailnet-only build host; deployment specifics are documented internally, not published here.

Shared API contract

The control plane has three explicit authentication classes. Human clients use a session cookie or the bearer token returned by POST /api/auth/login; passing {"signup":true} to that route creates an account. Agent clients use a scoped gbk_ workspace API key. Render nodes use the unique credential configured for that exact node identity and endpoint.

The browser-session stream is one route with three transports; it is not a separate, unqualified WebSocket endpoint. Node registration is also distinct from session allocation: clients never create an “instance” through an /api/instances route.