AgentUX

Problem

Formal usability testing is slow and expensive. Recruitment, scripts, and lab time put it out of reach for small teams.

That was concrete for us. We were designing a site for a sports nonprofit, sent surveys, and planned tests. Nobody responded. We tested on friends and roommates instead, and knew we were not hearing from real users.

AgentUX exists so a team can run persona-based tests on any website from a Chrome side panel, in minutes, without a research budget.

What we built

I built AgentUX with Manjusri Gobiraj, Alice Lan, and Khang Nguyen at DiamondHacks 2026. We took 1st Place in the Wildcard Track among 400+ competitors.

The product is a Manifest V3 side panel talking to a FastAPI backend. The backend runs a seven-step pipeline: summarize the page, generate tasks, assign them to personas, execute in parallel on Browser Use cloud agents, parse confusion from the traces, suggest CSS and JS fixes, then apply only what a human approves.

Personas include first-time visitors, elderly users, and custom ones you write. Ten or more can run at once. Live logs stream over WebSocket, color-coded by persona, with embedded live browser sessions so you can watch them get stuck.

How it works

Gemini turns a page summary into tasks you can edit or delete before anything runs. Each persona then navigates on its own.

The scoring engine reads agent output for hesitation, backtracks, retries, errors, frustration, and misclicks, then rolls those signals into a confusion heatmap and usability, accessibility, and clarity scores.

Suggested fixes are screenshot-checked with Playwright before they show up in Results. Approved edits save per domain in Chrome storage and re-inject on reload. You can also copy a structured prompt of every issue into an IDE.

Challenges

Parallel cloud agents are useless if the panel goes dark. We had to keep REST, WebSockets, and the extension message relay in one progress model so setup, live feed, and results stayed one run.

Agent traces are prose, not click events. Scoring had to extract element hints and signal types from language, then aggregate them across personas without pretending the log was a perfect instrumentation feed.

LLM CSS and JS can look right and still break the page. Playwright before and after shots, plus human approval, were the only way we were willing to persist a fix.

All of that had to ship in a 24-hour hackathon. We were still debugging while standing in line for food.

Impact

On known issues, the parallel personas found 85%+. Different personas failed in different places, which is the point of running them together instead of one scripted walkthrough.

The intended users are small businesses, nonprofits, and students who cannot staff a research pipeline. The landing site is agentux.dev.

What I learned

A usability score is only as good as the signals you can actually extract. Keyword traces are a start. They are not a substitute for real session replay.

Generated UI patches need a visual check and a person in the loop before they become durable. Shipping apply without approve would have been the wrong demo.

The product idea was the constraint. If testing requires a lab, most teams will skip it. The architecture had to fit in a side panel.