← roadmap

tuivision

keep priority P2 effort M wave 2

What it does: A TypeScript/ESM MCP server (9 eagerly-registered tools in src/index.ts) that is “Playwright for TUIs”: spawns real PTY processes (node-pty), parses ANSI via xterm.js headless, and renders terminal state to text/annotated/PNG/SVG (node-canvas). Tools: spawn_tui, send_input, get_screen, get_screenshot, resize_session, list_sessions, close_session, plus deterministic wait primitives wait_for_text (regex+flags) and wait_for_screen_change (debounced). Notable depth: a hand-rolled QueryResponder ANSI state machine (src/query-responder.ts) that auto-answers DSR/DA1/DA2/cursor-position/OSC-color queries so headless Bubble Tea/Charm apps render without hanging. Ships a CLI with daemon mode, one progressive-disclosure skill (tui-test SKILL.md + SKILL-compact.md), real unit tests, gitleaks secret-scan CI, and artifact-backed PHILOSOPHY.md/roadmap.json. v0.2.2.

Rationale: Mature, well-tested, single-purpose infrastructure that does exactly what it claims and does it cleanly. The QueryResponder and the use_script/answer_queries handling solve a genuinely hard real-world problem (headless rendering of TTY-querying frameworks) that off-the-shelf tools do not. SOTA verification trends (web search Jun 2026: “verification is the new bottleneck”; autonomous code review building cross-file context then re-running checks) make a real TUI execution/observation backend more valuable, not less — for the elf-revel TUI specifically, this is the only way an agent can actually see and drive the app rather than guess. The deterministic wait primitives and PHILOSOPHY north star (“low flake, deterministic waits, reproducible captures”) already align with the reliability-science framing. No reason to disrupt a working, niche-unique plugin; the gaps are incremental.

SOTA gap: Two frontier ideas it lacks, both already foreshadowed in its own roadmap. (1) Reliability-science evaluation (arXiv:2603.29231 “Beyond pass@1”; ReliabilityBench): there is no golden-image / visual-diff regression contract or k-trial variance reporting — get_screenshot returns an image, but nothing computes a structured pass/fail diff against a baseline, which is exactly roadmap items ITUV-N2 (diff quality for flaky/animation output) and ITUV-N3 (richer result contracts for Clavain QA gates). (2) Agent-legible structured output: get_screen exposes an include_roles param that is an explicit “forward-compatible stub, not yet active” — semantic role annotation (button/menu/input regions) would let an agent reason about TUI structure instead of raw glyphs, aligning with deterministic-skeleton/structured-output trends. Progressive disclosure is a minor third gap: 9 tools register eagerly, mitigated by the harness’s own MCP-layer deferral.

Redundancy: None within the interverse — it is the only plugin touching PTY/terminal automation. Adjacent but non-overlapping: intersight (web UI/UX W3C-DTCG analysis, not terminals), interhelm (runtime diagnostics/smoke-test design, app-agnostic but no TUI driver), the webapp-testing/verify built-in skills (Playwright for browsers, not terminals). intercheck is declared as a recommended companion (integration.json), not a competitor. The “richer result contracts for Clavain QA/acceptance gates” roadmap item is a deliberate hand-off to quality-gates/intercheck/interhelm, i.e. complementary not duplicative.

Actions:

  • Implement a visual-regression result contract: a compare_screenshot / assert_screen tool (or CLI subcommand) that diffs a capture against a stored golden image and returns a structured {match, diff_pct, regions} envelope — directly fulfilling roadmap ITUV-N2/N3 and the reliability-science ‘beyond pass@1’ framing for the Clavain quality-gates pipeline.
  • Activate the already-stubbed include_roles semantic annotation in get_screen so screen output carries structural role markers (panel/menu/input/focused), making TUI state agent-legible rather than raw glyphs.
  • Add optional k-trial variance reporting to the CLI run mode (repeat N captures, report stability/flake rate) so reliability is measured, not assumed — matches PHILOSOPHY north star of ‘low flake’.
  • Tighten progressive disclosure: keep the static surface lean and push secondary detail (Bubble Tea flags, format options) into the SKILL body rather than every tool description, reinforcing the deferred-tool/RAG-MCP pattern the harness already applies.