← roadmap

interdeep

improve priority P2 effort M wave 2

What it does: Per its code: interdeep is a Python FastMCP server exposing 4 stateless tools — extract_content (trafilatura fetch+clean), extract_batch (asyncio.gather with a concurrency semaphore), compile_report (deterministic markdown assembly via reports/markdown.py), research_status (probes trafilatura/Playwright availability + reads version from plugin.json). Extraction (extraction/hybrid.py) is trafilatura-first with a Playwright fallback triggered only when output < 200 chars; favor_recall=True, includes links/tables/formatting; never raises (returns method=‘failed’). It also ships a ‘deep-research’ SKILL.md defining a static 5-phase protocol (Orient/Search/Extract/Synthesize/Persist), 3 subagents (research-planner=haiku query decomposition, source-evaluator=haiku credibility/relevance gate, report-compiler=sonnet report assembly), and a /research command with quick|balanced|deep depth modes. Explicit design stance: it does NOT own search (companions interject/interflux provide it), MCP tools are stateless, ‘host-agent-as-brain’ keeps orchestration intelligence in SKILL.md, every component degrades gracefully standalone.

Rationale: The extraction backbone is genuinely unique in the roster and well-built: trafilatura-first + Playwright-fallback MCP tools, certifi/system-CA patch for Cloudflare intermediates, batch concurrency control, fail-soft results. Nothing else in interverse provides clean web content extraction, so that core is worth keeping. But the orchestration half is the weak, redundant part. The SKILL.md is a static, hand-authored linear pipeline — precisely the ‘static templates’ the 2026 workflow-optimization surveys (arXiv:2603.22386 From Static Templates to Dynamic Runtime Graphs; arXiv:2505.19591 Evolving Orchestration) say is being superseded by per-task DAGs. The Search→Extract loop is single-pass, not agentic: SOTA agentic RAG (A-RAG arXiv:2602.03442, GraphSearch arXiv:2509.22009, RELOOP arXiv:2510.20505) decides when/what to retrieve mid-reasoning and does multi-hop; interdeep does one search pass then extracts. Deep research specifically benefits from batched parallel tool calls per iteration (‘Scaling Parallel Tool Calling for Efficient Deep Research’, Feb 2026; LAMaS) — interdeep’s protocol is accidentally serial across phases. The subagent split correctly embodies orchestrator-worker context isolation, but report-compiler dumps a full report back into the lead context instead of returning a 1-2k condensed summary per Anthropic’s context-engineering playbook (compaction + sub-agent isolation, +29-39% lifts). Verdict is ‘improve’ not ‘modernize/deprecate’: the extraction niche is real and defensible; the fix is to sharpen scope toward extraction and bring the orchestration up to agentic/parallel retrieval — not a ground-up rewrite.

SOTA gap: Agentic, parallel, runtime-graph research orchestration. The plugin lacks (1) agent-controlled multi-hop retrieval — retrieval folded into the reasoning loop (A-RAG/GraphSearch/RELOOP) instead of a fixed Search→Extract→Synthesize line; (2) parallel-tool-call batching per iteration, which the Feb-2026 deep-research parallelism results show improves quality monotonically (LAMaS critical-path DAG); (3) structured, tool-grounded verification of findings — source-evaluator and the report stage rely on free-form NL judgment, which arXiv:2510.24367 and ReVeal show fails to drive reliable improvement vs. executable/grounded signals; (4) condensed sub-agent return contracts (1-2k summaries) per Anthropic’s write/select/compress/isolate context-engineering framework.

Redundancy: Significant on the orchestration half, none on extraction. The host harness already ships a ‘deep-research’ skill (fan-out web search + adversarial claim verification + cited report) that is strictly more advanced than interdeep’s static 5-phase protocol. interflux provides flux-research / flux-drive —mode=research (multi-agent research with triage, parallel dispatch, source-attributed synthesis) and bundles exa web_search/web_fetch MCP tools. intersynth is the dedicated parallel-findings synthesis engine. interject owns search/discovery; interknow owns storage. So interdeep’s orchestration, search-delegation, and synthesis layers overlap deep-research + interflux + intersynth + interject. Its non-redundant, defensible core is solely the trafilatura/Playwright content-extraction MCP tools (extract_content/extract_batch) — which those plugins lack and which interflux’s exa web_fetch only partially covers.

Actions:

  • Reposition to the extraction niche: make extract_content/extract_batch/compile_report the headline, and have interdeep’s SKILL.md/command defer the orchestration loop to the host deep-research skill and interflux flux-research rather than duplicating a static 5-phase pipeline (resolves the redundancy with deep-research + interflux + intersynth).
  • Modernize orchestration to agentic+parallel: replace the linear Search→Extract→Synthesize protocol with an iterative loop that batches parallel search+extract calls per round and decides when to retrieve more based on coverage gaps (cite arXiv:2602.03442 A-RAG, Feb-2026 parallel-tool-calling-for-deep-research, LAMaS DAG).
  • Add structured, grounded verification: have source-evaluator emit machine-checkable evidence (claim→source span, corroboration count) and compile_report flag uncorroborated claims, replacing free-form NL credibility judgments (cite arXiv:2510.24367; ReVeal arXiv:2506.11442).
  • Tighten sub-agent context contract: have report-compiler / source-evaluator return condensed 1-2k-token summaries to the lead rather than full reports/content, per Anthropic context-engineering (compaction + sub-agent isolation).