AI APP IDEA #1
Autonomous QA Agent for Cymetica — Browser-Based Bug Discovery Pipeline
0
Score
+0
Upvotes
-0
Downvotes
0
Comments
Progress
1
Submitted
2
Reviewing
3
Approved
4
Building
5
Shipped
Description
Architecture for an autonomous QA agent built specifically for Cymetica, designed to discover bugs through real browser interaction.
Stack: C#, Microsoft Agent Framework 1.0, Playwright for .NET, SQLite, JSONL. Runs locally, fully independent of Cymetica infrastructure.
Microsoft Agent Framework is the production-ready successor that unifies Semantic Kernel and AutoGen into a single framework — combining enterprise-grade stability with AutoGen's multi-agent orchestration patterns. Built-in support for human-in-the-loop checkpoints, session-based state management, and tool integration via MCP.
Pipeline Architecture:
1. Platform Context — System prompt gives the agent a full briefing on Cymetica sections (Arena, Terminal, Fund Dashboard, Analytics), critical functions, and what correct behavior looks like. Without this the agent can't distinguish a bug from a loading state.
2. Scope Config — JSON file defining which sections to test, their URLs, and priority. User defines what gets tested, the agent decides how to explore each section.
3. Autonomous Bug Discovery — Agent receives one section at a time, decomposes it into a test plan, and explores the UI the way a human tester would. Each iteration: thinks, runs a tool, gets the result, decides what to do next. Runs until it signals DONE or hits a step limit.
4. Tool Layer — Playwright for .NET handles full browser automation: navigation, screenshots, DOM traversal, click/type interactions, UI state validation. Agent operates purely from what's visible in the browser with no backend API access. Structured logging of every action, selector used, and result.
5. Self-Healing — Selector fails -> XPath fallback -> screenshot + DOM analysis -> agent finds alternative itself. Three consecutive failures -> section flagged as unstable, agent moves on.
6. Memory — Episodic log (JSONL): which selectors failed, which pages are consistently flaky, what patterns repeat across sessions. Structured DB (SQLite): every candidate bug with severity, section, steps to reproduce, and screenshot.
7. Human-in-the-Loop Review Gate — Before anything surfaces publicly, a human reviews each candidate bug: confirms it's real, sets severity, adds reproduction context. Microsoft Agent Framework has this built in natively.
8. Bug Leaderboard Submission — After approval the agent automatically submits the bug to cymetica.com/leaderboard/bugs.
9. Regression Test Generation + Fix Verification — After each confirmed bug, agent auto-generates a Playwright script that reproduces the exact failing scenario. Script runs again after fix deployment and verifies the bug is resolved.
Full cycle: Agent explores section -> finds candidate bug -> human review -> approved -> submitted to leaderboard -> regression script auto-generated -> fix deployed -> agent re-runs script -> verified closed
Discussion (0)
No comments yet. Be the first to share your thoughts.