interlore

improve priority P3 effort M wave 3

What it does: A pure-prose Claude Code skill plugin (1 skill observe, 3 commands scan/review/status, 0 agents/hooks/MCP servers). The observe SKILL.md drives the LLM to glob decision artifacts (docs/brainstorms, docs/prds, docs/plans, .claude/flux-drive-output/fd-*.md), read PHILOSOPHY.md as a baseline, two-pass-extract decision markers via Grep (**Decision:**, ‘chose X over Y’, policy language, Alignment/Conflict enrichment lines), cluster decisions by tradeoff axis, classify each cluster (established=3+ unique bead-deduped decisions over 2+ weeks / emerging=2 / nascent=1), diff against PHILOSOPHY.md sections (conforming/drift/emerging), and write structured YAML proposals to .interlore/proposals.yaml. Review mode presents each pending proposal with its evidence chain and uses AskUserQuestion to accept (append proposed_text to PHILOSOPHY.md), reject (record reason + add axis to rejected_patterns to suppress re-proposal), or defer. Cardinal bet: propose, never auto-apply — humans own philosophy; generation is explicitly delegated to interdoc.

Rationale: The architecture is genuinely sound and well-disciplined: tight scope, an explicit detect/generate boundary with interdoc, propose-never-apply (the human-in-the-loop checkpoint SOTA repeatedly validates as beating fire-and-forget autonomy), git-auditable YAML state over an opaque auto-updating store (matches the ‘human-auditable, version-controlled memory’ line against silent memory pollution), union-merge evidence and bead-ID dedup. So this is not a deprecate or a ground-up modernize. But two concrete problems hold it back. (1) Mechanism depth: the plugin’s stated premise is detecting LATENT, SEMANTIC design patterns, yet its actual detection is lexical Grep over hardcoded markers plus ‘fuzzy match on axis description’ delegated to the LLM with no structure. That is precisely the flat-keyword approach the 2025-26 frontier replaced with embedding-based semantic gating. (2) It appears unproven: no .interlore/ state or proposals.yaml exists anywhere under ~/projects, version is still 0.1.0, and the last substantive commit was April — despite abundant input substrate (dozens of PHILOSOPHY.md + docs/brainstorms across Sylveste). It compiles and has structural tests but shows no signs of having been run on the live corpus. Improve = wire it to the ecosystem’s existing semantic infra and validate it for real, not rewrite it.

SOTA gap: Embedding-based semantic gating for pattern clustering (RAG-MCP, arXiv: >50% fewer tokens and tool-selection accuracy tripling 13.6%->43% by retrieving over a semantic index instead of keyword-matching; Select-then-Solve embedding routers; GraphRAG entity-relation clustering). interlore clusters ‘tradeoff axes’ by lexical Grep + ad-hoc LLM fuzzy matching — exactly the flat-vector/keyword approach the frontier superseded for connected/global questions. The ecosystem ALREADY has the substrate it should reuse: intersearch embedding_index/embedding_query and interknow’s qmd MCP (semantic retrieval with provenance). Detecting recurring design tradeoffs across a corpus of decisions is an embedding-similarity / graph-clustering problem, and grounding axis clustering in embeddings would directly serve the plugin’s own ‘signal quality over coverage’ north star. Secondary gap: detection critiques are unstructured LLM judgment with no reliability framing (no k-trial variance / Agent-as-a-Judge bias guards) — minor here because a human gates every proposal.

Redundancy: Partial and conceptual, not duplicative. interlore is the DETECT half of a detect->generate pipeline; interdoc (recursive AGENTS.md generator with Oracle critique) is the generate half and the explicitly-declared boundary partner — no functional overlap, complementary. interknow (durable pattern repository with provenance, temporal decay, qmd semantic retrieval) is the closest sibling: both ‘extract recurring patterns with evidence from project activity,’ but interknow targets engineering lessons/patterns into a searchable repo while interlore targets design-philosophy tradeoffs into PHILOSOPHY.md proposals — distinct artifacts, though interlore should consume interknow/intersearch embedding infra rather than its own Grep heuristics. intermem (graduates stable auto-memory facts to AGENTS.md/CLAUDE.md via stability detection + interactive approval) shares the propose-then-promote-to-doc lifecycle pattern but operates on a different source (auto-memory facts vs decision artifacts) and target (AGENTS.md vs PHILOSOPHY.md). The philosophy-drift niche is legitimately unclaimed; redundancy risk is the substrate (semantic search) it reimplements lexically, not the output.

Actions:

Prove it on the live corpus first: run /interlore:scan against Sylveste’s actual docs/brainstorms + PHILOSOPHY.md (dozens of artifacts exist) and inspect the generated .interlore/proposals.yaml. Until that smoke test passes, the plugin is unvalidated scaffold — this is the gating action before any enhancement.
Replace lexical axis clustering with embedding-based semantic gating (driven by RAG-MCP / Select-then-Solve SOTA): after Grep extracts candidate decision excerpts, embed them via the existing intersearch embedding_index/embedding_query (or interknow’s qmd) and cluster tradeoff axes by vector similarity instead of LLM ‘fuzzy match on axis description.’ This directly serves the plugin’s signal-quality north star and reuses ecosystem infra rather than reinventing it.
Bump beyond 0.1.0 and update README/AGENTS once a real run exists; add at least one behavioral/golden test that scans a fixture corpus and asserts proposals.yaml shape, not just the current structural tests (file existence + plugin name). Structural-only tests are why the plugin can be unproven yet ‘passing.’
Add a lightweight reliability guard for proposals: when classifying drift vs conforming, run the classification at low temperature across a couple of trials and only surface a proposal if the verdict is stable — cheap variance check aligned with the pass@1->reliability-science shift, keeping false-positive philosophy proposals out of the human’s review queue.