Browser Automation Agent
An agentic browser-automation prototype that turns natural-language requests into structured search and browser steps, executes them in local Chrome, and replans from newly observed page state.
Role
End-to-end (agent orchestration, OpenAI integration, browser tooling, validation, and chat UI)
Timeline
November 2025; revisited July 2026
Problem
Multi-step web tasks rarely follow a fixed sequence: target sites must be discovered, pages change, elements move, and an interaction can fail after a plan has already been generated.
Approach
Built a hierarchical phase-and-step executor around OpenAI's Responses API and Vercel AI SDK. Zod-validated plans coordinate grounded web search and Stagehand browser actions, while accessibility-tree feedback, fresh-state validation, retries, and backtracking help the agent recover as pages change.
Results
- Implemented structured orchestration across search, navigation, action, observation, extraction, and phase-completion steps.
- Added fresh-state planning, selector self-healing, retry batches, phase backtracking, and whole-plan regeneration.
- Built a custom chat interface that streams high-level execution status without exposing private model reasoning.
Impact
- Demonstrates direct OpenAI Responses API integration through Vercel AI SDK and custom agent orchestration.
- Grounds URL discovery in web-search evidence and browser interaction in accessibility-tree state.
- Separates planning, execution, validation, and browser tooling into typed modules.
Demo
Gallery
