jevbrowse: Grounded Browser Automation for AI Agents (SDK + MCP)

AI products · Creator, sole engineer · prototype

A browser runtime that lets any agent or program drive a real browser by intent, without reading raw accessibility trees or generating text for every click.

The detail

Browser agents are slow because a text model reads the whole page for each action, and they fail quietly because nothing proves an action worked. jevbrowse splits the job. The outer agent decides what to do. The runtime takes one indexed snapshot of the live page, asks a small decision-only model where each intent lands among candidates the code supplies, and executes through Playwright on the real DOM node.

Every call returns typed evidence: page state, whether the page changed, decision probabilities and timing. An ambiguous target comes back as ambiguous with its alternatives, so the caller chooses instead of the tool guessing. When an intent has exactly one unambiguous match, a local heuristic answers with no network call.

Four surfaces share one runtime: a Python SDK, an MCP server over stdio and streamable HTTP, a CLI and an HTTP JSON API. A pytest plugin turns verified flows into re-runnable tests with an on-disk decision cache.

Python, Playwright, Pydantic, MCP, FastAPI, pytest.

What I did

  • Split browser use into two roles: the outer agent decides what to do, the runtime decides where it lands and whether it worked
  • Built one-snapshot observation and one decision request per intent, with a local exact-match short-circuit
  • Returned typed evidence for every call: page state, change detection, decision probabilities and timing
  • Shipped four surfaces on one runtime: a Python SDK, an MCP server, a CLI and an HTTP API
  • Added file upload, embedded application-form frames, captcha detection and a human-in-the-loop handoff
  • Wrote a pytest plugin so verified browser flows become fast, re-runnable tests, plus 423 tests of the runtime itself

Stack

Python, Playwright, MCP, Browser Automation, AI Agent Development, FastAPI, Pydantic, pytest, API Design

Case study: https://www.siddhvasudev.com/projects/jevbrowse-grounded-browser-automation-for-ai