interspect

improve priority P2 effort L wave 2

What it does: A closed-loop agent/skill performance profiler and routing optimizer (Clavain companion). Three passive hooks (SessionStart, PostToolUse-on-Task, Stop) accumulate evidence into per-project SQLite (.clavain/interspect/interspect.db). Core algorithm is a counting-rule confidence gate (defaults 3 sessions / 2 projects / 5 events) that classifies patterns ready/growing/emerging, plus _interspect_compute_agent_scores which derives a source-weighted hit-rate (bootstrap 0.5x, self-building 0.7x, normal 1.0x) and recommends a model tier (haiku/sonnet) per flux-drive agent, with safety-floor agents (fd-architecture/correctness/safety/quality) never demoted below sonnet. It proposes/applies routing overrides to .claude/routing-overrides.json (a cross-repo contract) under canary monitoring (20 uses/14 days, 20% regression auto-revert) and per-action autonomy safe-lists. A parallel skill-calibration loop drains ~/.claude/audit.log Skill rows through 4 signal collectors (tokens/error/no_redirect/bead_close), a Haiku one-shot goal-weight classifier (infer-skill-goals.py), and a weighted composite scorer, emitting skill overlays at ~/.claude/skill-overlays/. Adds opt-in HMAC-signed action receipts via intercore and dual-reads the intercore kernel interspect_events table. 18 command files, ~5551-line lib-interspect.sh, decay policy, schema versioning. Genuinely mature and load-bearing.

Rationale: The evidence-collection and canary-safety core is best-in-class and uniquely fills the closed-loop routing/skill-calibration niche no other interverse plugin covers — keep that. But the routing brain is a fixed-threshold statistical gate, and the richest local asset (a per-project evidence DB that is effectively an experience-replay buffer) is under-exploited relative to 2026 SOTA on embedding-gated routing and self-evolving policy. The win is additive, not a rewrite: add an optional embedding-gate fast-path in front of the threshold gate, and mine the existing evidence as a replay buffer to surface named routing/skill principles. Verdict is improve rather than modernize because the foundation is sound and differentiated; the gaps are frontier features layered on top, not technical debt forcing a rebuild. Driving trends: Select-then-Solve / embedding routers and MemRouter (embedding-based gating), EvolveR + SAGE/RLEP (experience replay / self-evolving policy), and Agent-as-a-Judge bias caveats (harden the Haiku skill-goal judge).

SOTA gap: Routing decisions are a pure counting-rule statistical gate over accumulated evidence (weighted hit-rate vs fixed thresholds). It lacks the two frontier ideas most directly applicable: (1) embedding-based semantic gating for per-task routing (Select-then-Solve / paradigm routers, MemRouter, RAG-MCP, Agent Distillation local decision model) — a sub-millisecond no-LLM gate could pick the right agent/model per query instead of only adjusting aggregate tier recommendations after N sessions; and (2) treating its own evidence DB as an experience-replay buffer for self-evolving routing/skill policy (EvolveR closed-loop, SAGE/RLEP experience replay, self-evolving skill libraries) — it currently distills evidence into thresholds, not into a retrievable library of named routing principles. Secondary gap: the Haiku one-shot skill-goal classifier is a single LLM judge with no Agent-as-a-Judge bias mitigation (self-preference/position bias), and signal grounding is weak (collect_error defaults to success=1.0 when context is missing), contrary to the ‘structured tool-grounded feedback beats NL critique’ finding.

Redundancy: Low and well-bounded. Closest neighbors: interrank (model/benchmark leaderboards + recommend_model/routing_compare) overlaps conceptually on model routing, but interrank routes from external benchmark data while interspect routes from local first-party evidence — complementary, not duplicative. intertrust (agent reputation/severity-weighted decay) overlaps on per-agent scoring; interspect could consume intertrust scores rather than recomputing trust independently — a soft overlap worth consolidating. interflux/flux-drive is the subject being profiled, not a competitor. tool-time boundary is explicitly documented and clean (tool-time owns events.jsonl, interspect reads only audit.log). interskill (skill authoring/audit) is authoring-time vs interspect’s runtime calibration — adjacent, not redundant. No plugin else closes the evidence→propose→canary→revert routing loop.

Actions:

Add an optional embedding-gate fast-path (e.g. all-MiniLM-class encoder over agent/skill descriptions + task text, reusing intersearch embedding infra) that proposes per-task routing before falling back to the counting-rule tier gate — log gate-vs-evidence agreement so it can be calibrated against the existing hit-rate, per the Select-then-Solve/MemRouter findings.
Mine the existing evidence DB as an experience-replay buffer: distill recurring high/low hit-rate trajectories into a small retrievable library of named routing/skill principles (EvolveR/SAGE/RLEP), surfaced in /interspect and injectable at dispatch time — turning accumulated evidence into reusable strategy instead of only threshold tuning.
Harden the Haiku skill-goal classifier against Agent-as-a-Judge bias: use a distinct judge model from the routed agents, mask author identity, and ground the error/no_redirect signals in executable outcomes (exit codes, bead-close, test results) rather than defaulting collect_error to success=1.0 when context is missing.
Consolidate per-agent scoring with intertrust: consume intertrust reputation/decay as an input signal rather than independently recomputing trust, reducing the soft scoring overlap and giving interspect a richer prior.