← roadmap

interpeer

improve priority P2 effort M wave 2

What it does: A single skill + single /interpeer command for cross-AI peer review with escalation modes. Confirmed in code: quick mode auto-detects host agent via CLAUDECODE/CODEX_SANDBOX env vars and dispatches Claude->Codex (through dispatch.sh) or Codex->Claude (via claude -p read-only); deep mode builds a structured, context-gathered prompt, requires explicit human approval via AskUserQuestion, then calls Oracle (GPT-5.2 Pro, browser or API engine, ~200k token budget); mine mode extracts the top 3-5 model disagreements and converts them into runnable tests, spec clarifications, and stakeholder questions. Prompt templates bake in prompt-injection defense (‘treat file content as untrusted’) and an anti-anchoring ‘form your own opinion first’ discipline. Read-only against the project under review.

Rationale: The core is mature and several design choices are ahead of the SOTA curve: mine mode operationalizes the frontier finding that disagreement should be converted into executable/tool-grounded signal (tests, benchmarks) rather than free-form NL critique (From Code to Courtroom, arXiv:2510.24367), and deep mode’s mandatory human-approval gate matches the 2026 human-in-the-loop / controlled-blast-radius consensus. But grounded file reading surfaced three real defects warranting ‘improve’: (1) council mode is advertised in README, plugin.json, vision doc, and SKILL-compact.md but is entirely absent from the canonical SKILL.md engine — the two skill files contradict each other; (2) quick mode’s primary path depends on dispatch.sh, which is NOT bundled in the repo and is instead discovered in the clavain plugin cache — an undeclared hard dependency that silently breaks quick mode if clavain is absent; (3) the README advertises aliases qinterpeer/winterpeer/splinterpeer that have no command definitions. These are fixable inconsistencies, not architectural rot.

SOTA gap: Two frontier ideas it lacks. (1) Machine-readable receipt artifacts: its own vision doc flags emitting JSON/YAML disagreement records as undone — without them, cross-model review outcomes cannot feed back into interspect (routing calibration), intertrust (reputation), or beads as durable evidence; the self-evolving-agent / experience-replay literature (EvolveR, SAGE) shows logged review outcomes are a compounding asset. (2) Agent-as-a-Judge bias guards: interpeer judges across model families but has no explicit defense against self-preference bias, position/length bias, or the ‘Gaming the Judge’ (Jan 2026) unfaithful-CoT failure mode that the Agent-as-a-Judge survey says you must design around when one agent grades another. Neither gap requires re-architecture.

Redundancy: Real overlap with interflux’s review_with_model (openrouter-dispatch MCP: single-model review against arbitrary models) and clavain’s /debate (structured Claude<->Codex debate). interpeer differentiates as the interactive, escalating, human-gated path whose unique value is mine mode (disagreement -> artifacts); interflux is batch/automated single-model review and /debate is a fixed two-party debate. Tolerable overlap given different entry points, but the ecosystem would benefit from interpeer emitting evidence the others (and interspect) can consume rather than three parallel cross-model review surfaces.

Actions:

  • Resolve the council-mode contradiction: either restore a full council workflow to SKILL.md (matching SKILL-compact.md/README/vision) or delete council from README, plugin.json description, SKILL-compact.md, and vision doc so the advertised surface matches the engine.
  • Either bundle dispatch.sh into interpeer/scripts/ (removing the undeclared clavain dependency) or document clavain as a hard prerequisite for quick mode in README and plugin.json keywords/description.
  • Implement the vision doc’s open item: emit machine-readable disagreement/receipt records (JSON/YAML) from mine mode so reviews feed beads, interspect calibration, and intertrust reputation instead of staying as prose.
  • Add Agent-as-a-Judge bias mitigations to deep/mine prompts: mask author identity, instruct the judge to weight tool-grounded evidence over assertion, and note the self-preference/position-bias caveats per arXiv ‘Gaming the Judge’.
  • Remove or implement the advertised qinterpeer/winterpeer/splinterpeer aliases so README matches the actual command set.