← roadmap

tool-time

keep priority P2 effort M wave 2

What it does: Tool-usage analytics for Claude Code / Codex CLI / OpenClaw. Hooks (PostToolUse ’*’, SessionStart, SessionEnd, plus a PreToolUse:Task output-redirect) append one JSONL event per tool call to ~/.claude/tool-time/events.jsonl. summarize.py does pure threshold-free 7-day per-project aggregation -> stats.json (per-tool calls/errors/rejections, edit-without-read, skills, MCP servers, installed plugins). analyze.py does 90-day deep analytics -> analysis.json (session classification, bigrams/trigrams, retry patterns, weekly trends, time-of-day patterns, cross-source comparison, project breakdown) with tool-name normalization across the three sources. The /tool-time skill reads both files, flags problems (error/rejection rates, edit-without-read, Bash dominance, low diversity, retry loops), reviews CLAUDE.md/AGENTS.md, and proposes concrete edits; optionally pulls cass cross-agent data and recommends playbooks.com skills. backfill.py/parsers.py parse Codex+OpenClaw transcripts (no hooks). community/ is a Cloudflare Worker (Hono) + D1 with opt-in, strict-allow-list anonymized submission and a public dashboard (tool-time.org), incl. GDPR delete. emit-interspect-evidence.sh bridges stats.json into interspect’s evidence DB (idempotent, threshold-configurable).</purpose_confirmed> Low / complementary, not redundant. interstat does token-efficiency benchmarking + session analytics and intersearch/cass does cross-agent session analytics — tool-time’s distinct niche is per-tool call/error/rejection telemetry via hooks plus multi-source (Codex/OpenClaw) transcript parsing and a public community-comparison dashboard, which neither replicates. It is the upstream producer of evidence for interspect (routing/calibration) via emit-interspect-evidence.sh, and feeds interspect/interstat rather than overlapping them. One soft overlap: the SKILL’s CLAUDE.md/AGENTS.md gap review touches interscribe/interdoc territory, and its cass call duplicates intersearch’s analytics surface — worth keeping those as thin delegations rather than parallel implementations.

Rationale: Mature, well-tested (test_summarize/test_upload/test_analyze + structural + shell tests), privacy-conscious (strict allow-list, GDPR delete, gitleaks waivers), and architecturally clean — the deliberate split of a threshold-free data pipeline (summarize/analyze) from an opinionated agent skill keeps the data honest and recommendations adaptable. It already embodies several SOTA-validated patterns: hook-driven deterministic instrumentation (harness-engineering-as-discipline), multi-harness portability (Claude Code + Codex + OpenClaw via transcript parsers, with cross-source tool-name normalization), and it is the canonical telemetry producer feeding interspect’s routing/calibration (self-improvement-as-memory-write). The roadmap shows recent active hygiene (removed a redundant PreToolUse binding, consolidated wildcard hooks). No frontier idea is being violated; the gaps are additive enhancements, not corrections — hence keep, not modernize.

SOTA gap: Token/cost accounting. The dominant 2026 finding (‘the token bill comes due’ — token economics as the #1 enterprise pain; cost observability and context-rot as named failure modes) is exactly what a tool-usage analytics layer should now measure, but tool-time counts calls/errors/rejections only — it has no per-tool or per-session token or cost dimension, so it cannot surface the highest-ROI signal (which tools/chains burn the most tokens, where context-rot snowballs, cost-per-session trends). It is well-positioned to add this (the event schema and analyze.py trend machinery already exist) and it would sharpen the interspect evidence it emits. Secondary, optional gap: the data is an ideal replay buffer for an EvolveR/experience-replay self-improvement loop, but that consumption belongs in interspect/interlab, not here.

Actions:

  • Add a token/cost dimension: capture token usage from PostToolUse tool_result metadata into the event schema, aggregate cost-per-tool and cost-per-session in summarize.py/analyze.py, and surface ‘highest-token tools/chains’ + ‘cost-per-session trend’ in the /tool-time skill and dashboard — directly addressing the 2026 token-economics finding.
  • Add a context-rot signal to analyze.py: detect re-read-after-failure and stale-tool-output loops (the snowballing pattern cited as the root cost driver) as a first-class anti-pattern alongside the existing retry_patterns, and emit it as interspect evidence.
  • Tighten the SKILL’s external surface: make the cass cross-agent call and the CLAUDE.md/AGENTS.md gap-review delegate to intersearch and interscribe/interdoc respectively rather than re-implementing, to avoid soft overlap.
  • Verify the playbooks.com and tool-time.org/workers.dev endpoints are still live and the community schema (migrations 001-004) matches current upload.py allow-list; stale external dependencies are the main rot risk for an otherwise solid plugin.