intertest

What it does: A pure-discipline (methodology, not tooling) plugin: 3 skills, 0 commands/agents/hooks/MCP per CLAUDE.md. (1) systematic-debugging — a 4-phase root-cause-before-fix loop (read errors, reproduce, pattern-compare, single-hypothesis test, fix at source) with an “after 3 failed fixes, question the architecture” escalation, plus supporting technique files (root-cause-tracing.md, defense-in-depth.md, condition-based-waiting.md). (2) test-driven-development — strict RED-GREEN-REFACTOR with an iron law (no production code without a failing test first; delete code written before its test) and a testing-anti-patterns.md. (3) verification-before-completion — an evidence-gate: no completion/success claim without freshly running the proving command in-message, with a claim-to-required-evidence table. Each skill ships a SKILL-compact.md (progressive disclosure). It is the canonical, marketplace-published home for disciplines extracted from Clavain; Clavain’s agent-rig.json depends on it via intertest@interagency-marketplace and its /tdd, /repro-first-debugging, /verify commands orchestrate these skills.

Rationale: Per its actual files this is a mature, focused, correctly-scoped plugin and the 2026 frontier validates rather than challenges its thesis: “Skills beat prompts” + “verification is the new bottleneck” + “structured tool-grounded feedback beats LLM-judge prose” are all direct external confirmation of what these three skills encode. It already does progressive disclosure (SKILL-compact.md per skill), has structural tests (tests/structural enforcing 3 skills + frontmatter), and is cleanly extracted so Clavain and other rigs consume it via the marketplace. That argues against modernize/deprecate. But it is not pure “keep”: there is a concrete correctness bug (stale clavain:-prefixed cross-references that resolve to nothing post-extraction) plus one narrow frontier gap (single-shot verification vs. k-trial reliability). Both are small, additive fixes that raise quality without changing the plugin’s identity — hence “improve.”

SOTA gap: verification-before-completion encodes single-shot success (“run the command once, see 0 failures, then claim”) as sufficient evidence. 2026 reliability-science evaluation (Beyond pass@1, arXiv:2603.29231; ReliabilityBench, arXiv:2601.06112) argues a single passing run is NOT evidence of reliability — for stochastic/flaky behavior you need k-trial consistency, robustness to perturbed inputs, and graceful degradation under tool/infra failure. The skill has no notion of variance or re-run-N-times for non-deterministic checks. (Note: its core thesis — evidence/tool-grounded gates over “should work” prose — is strongly validated by ReVeal arXiv:2506.11442 and “From Code to Courtroom” arXiv:2510.24367, which show structured executable feedback beats free-form LLM-judge critique; the gap is narrow and additive, not a rethink.)

Redundancy: Intentional, well-managed layering rather than harmful duplication. intertest holds the canonical discipline content; Clavain exposes thin orchestrating commands over it (clavain:tdd, clavain:repro-first-debugging, clavain:verify, clavain:quality-gates, clavain:code-review-discipline) and declares intertest as a marketplace dependency in agent-rig.json — this matches the SOTA “thin skill/command orchestrating canonical capability” pattern. verification-before-completion’s evidence-gate overlaps conceptually with clavain:verify and intercheck:quality, but at a different altitude (universal methodology vs. project-specific runners). One real defect, not redundancy: systematic-debugging/SKILL.md cross-references clavain:test-driven-development and clavain:verification-before-completion (lines 179, 287-288) — stale names left over from the Clavain extraction; the skills now live under intertest:, so those invocations point at non-existent skill ids.

Actions:

• Fix stale cross-references in skills/systematic-debugging/SKILL.md (lines 179, 287, 288): change clavain:test-driven-development and clavain:verification-before-completion to intertest:test-driven-development and intertest:verification-before-completion so Phase 4 Step 1 points at skills that actually exist. Add a structural test asserting no clavain:-prefixed skill ids appear in intertest’s own skill bodies.
• Add a ‘reliability vs single-shot’ section to verification-before-completion/SKILL.md (and its compact): for non-deterministic/flaky checks, require re-running the proving command N times (e.g. 3-5) and reporting consistency, not just one green run — citing the pass@1 reliability-science framing (k-trial consistency, robustness under perturbation/infra failure). Keep single-run sufficient for deterministic checks (type-check, build).
• Strengthen the ‘Bug fixed’ and ‘Regression test works’ evidence rows to explicitly require the RED-GREEN-revert cycle as the standard (it currently lives only in the patterns block) — grounding self-correction in executable signals per ReVeal/Code-to-Courtroom findings that tool-grounded feedback, not prose, drives reliable fixes.
• Optional housekeeping: PHILOSOPHY.md still reports ‘Brainstorms analyzed: 0 / Plans analyzed: 0 / inferred’ boilerplate — either populate it from the real docs/intertest-vision.md or trim it so the plugin’s own canonical reference isn’t placeholder text.