← roadmap

interject

improve priority P2 effort M wave 2

What it does: Ambient discovery engine. Pluggable source adapters (arXiv, HN, GitHub, Anthropic docs/changelog, plus Exa/Tavily/Brave/PubMed/Semantic Scholar/SearXNG) fetch recent items; engine.py scores each via cosine similarity of a local sentence-transformers embedding (nomic-embed-text-v1.5/768d from the shared intersearch lib) against a learned profile vector, adjusted by keyword/source/recency weights and a capability-gap bonus. Confidence tiers (high/medium/low/discard) drive outputs.py: high -> bead(P2)+briefing+brainstorm, medium -> bead(P4,pending_triage)+briefing, all tiers -> submit to an ic kernel record. Closed-loop learning: promote/dismiss nudge the profile vector (weighted moving average), and adapt_thresholds + source-weight conversion-rate feedback (feedback.py) tune the surface over time. Exposes a 10-tool stdio MCP server and a SessionStart hook that surfaces score>=0.8 items via direct sqlite. SQLite v2 schema; 131 briefing docs already generated under docs/research/. Actively maintained (v0.1.14, recent commits).

Rationale: The core is sound and matches several 2026 best-practices: it already embodies the embedding-based semantic gate (Select-then-Solve / RAG-MCP-style cheap vector routing) by deciding relevance with a no-LLM cosine gate, and it separates persistent knowledge (SQLite + git-tracked briefings) from the context window. It fills a genuine, differentiated niche the rest of the roster does not: UNPROMPTED, scheduled, learned-profile scanning of the outside world. But two concrete claims in the README/PHILOSOPHY are not realized in code, and that is where the value sits. (1) The headline promise — ‘not just here is a new paper but here is how it relates to the embedding infra you are building in intersearch’ — is unimplemented: _write_briefing/_write_brainstorm emit a templated ‘scored X against interest profile’ line; awareness.py and gaps.py compute project/bead topics but feed them only into scoring, never into an LLM-grounded rationale tying a discovery to a specific repo/bead. The ‘why it matters’ is the differentiator and it is currently a stub. (2) Retrieval is static: _handle_search and session_context load up to 200 discoveries and rank by raw cosine in Python — no agentic/multi-hop retrieval and no temporal/graph memory of how discoveries connect. These are improve-grade fixes on a healthy plugin, not a rewrite.

SOTA gap: Two frontier ideas are missing. (a) Experience-driven distillation into reusable principles (EvolveR arXiv:2510.16079, ExpeL): interject learns a single drifting profile VECTOR from promote/dismiss, not a retrievable library of named ‘why I cared / why I dismissed’ principles — so it cannot explain or transfer its taste, and the learning signal is a low-information dot product. (b) Agentic + graph-structured retrieval (A-RAG arXiv:2602.03442, GraphRAG): briefing generation and the interject_search/session_context tools are flat top-k cosine; an LLM-grounded relevance step that retrieves the user’s actual open beads/plan docs (via the existing awareness.py) and writes the project-specific rationale would make the README’s central promise real. Lesser gaps: 10 MCP tools are eagerly registered rather than progressive-disclosure/deferred (RAG-MCP / Tool Search Tool, ~85% token cut) and the recency model is a crude 7-day linear decay.

Redundancy: Meaningful overlap, but interject’s ambient/scheduled/unprompted scan + learned profile remains distinct. Overlaps: interdeep (deep research / content extraction / research orchestration) and interflux (multi-agent research, bundles its own Exa search) — both do ON-DEMAND research where interject does ambient; the promote pipeline should hand the high-tier item TO interdeep/interflux rather than re-implementing research. intersearch — already a hard dependency (interject re-exports its embedding client); keep the dependency, do not duplicate the embedding/index layer. interseed (idea garden capture/refine/graduate) and interpath/clavain brainstorm — interject’s promote->bead->brainstorm-doc path duplicates idea-capture/graduation; promoting a discovery should feed interseed/brainstorm rather than write its own brainstorm template. No overlap on the core scanning loop itself.

Actions:

  • Make ‘why it matters’ real: add an LLM-grounded relevance step that, for high/medium tiers, retrieves the user’s actual open beads + recent plan/brainstorm topics (awareness.py already collects these) and writes the project-specific rationale into the briefing/brainstorm, replacing the templated ‘scored X against interest profile’ boilerplate in outputs.py.
  • Hand off instead of duplicating: on interject_promote of a high-tier item, delegate deep research to interdeep/interflux and idea-graduation to interseed/clavain brainstorm rather than emitting interject’s own brainstorm template — shrinks redundancy and improves output quality.
  • Add an experience-distillation layer (EvolveR/ExpeL pattern): on promote/dismiss, store a short natural-language ‘why’ note keyed to the discovery embedding, and retrieve the nearest such notes at scan time to bias scoring — gives explainable, transferable taste beyond the single drifting profile vector.
  • Reduce MCP context cost: gate the 10 tools behind progressive disclosure / defer_loading (Tool Search Tool / RAG-MCP) so only inbox+status load eagerly and scan/promote/search/profile load on demand.