Vibe Documentation — Question-Tree Traceability Check

Scope and Method

The documentation under src/docs/ was generated by Socratic Code-Theory Recovery Phase 2 from QUESTION_TREE.adoc (repository root). Every claim in that documentation should trace back to a leaf of the Question Tree: an [ANSWERED] leaf (carrying file:line evidence) or an [OPEN] leaf (a question the code cannot answer).

This check traces each claim in:

  • the arc42 chapters (arc42/chapters/01–12);

  • the four ADRs (adrs/adr-001–004);

  • the PRD (spec/prd-vibe.adoc) and the specification (spec/use-cases-vibe.adoc).

The src/docs/reports/ folder is excluded — it holds inspection artifacts, not Phase-2 documentation.

It reports two defect classes:

  • Class A — a claim that traces to no Question-Tree leaf.

  • Class B — a claim that traces to an [OPEN] leaf but is stated as fact, without being marked as a gap, a deferral, or a (team answer).

A claim that traces to an [OPEN] leaf and is marked deferred is correct handling and is not a defect; the closing section lists those to show the trace was exhaustive.

Note
Re-run after defect disposition

This is a re-run of the check. The two Class B findings of the previous run — the asserted top-five quality-goal selection and the missing use-case-catalog completeness caveat — have been corrected in the documentation and are no longer defects; they now appear in OPEN Leaves That Are Correctly Marked. The four Class A findings are unchanged: each asserts a figure or set the documentation backs with file:line evidence but that no Question-Tree leaf recovered, so closing them belongs in Phase 1 (the tree), not Phase 2 (the documentation).
Class Count

A — claims tracing to no leaf

4

B — claims on an OPEN leaf, not marked as a gap

0

Total

4

The 14 [OPEN] leaves of the Question Tree are: Q1.2.2, Q1.4, Q1.5.2, Q1.6, Q2.2.UC.completeness, Q2.5.2, Q2.6.BR.intent, Q3.4.2, Q3.8.1, Q3.9.ADR.HexagonalPorts, Q3.9.ADR.LLMBackendAbstraction, Q3.9.ADR.MiddlewarePipeline, Q3.9.ADR.SessionsAsFiles, Q4.9.

Class A — Claims That Trace to No Leaf

T-01 (Major) — The "context warning at 50 % of the threshold" has no leaf

10_quality_requirements.adoc (Auto-Compaction scenario) claims: "a warning is injected at 50 % of that threshold". The same context-warning middleware is referenced in adrs/adr-003-middleware-pipeline.adoc line 16 ("a context warning").

No Question-Tree leaf records a context-warning middleware or the 50 % figure:

  • Q3.6.SCN.Compaction describes only the COMPACT action — "When context_tokens crosses the model’s auto_compact_threshold, AutoCompactMiddleware returns COMPACT" — and its evidence stops at middleware.py:81-96.

  • Q3.10 records only "auto-compaction at the model’s token threshold".

  • Q2.2.PUC.InteractiveChat.Extensions mentions auto-compaction, no warning.

The 50 % warning is a claim introduced in Phase 2 with no recovered leaf behind it. The documentation cites middleware.py:81-126, which does contain ContextWarningMiddleware; the gap is in the Question Tree, not the documentation evidence.

T-02 (Minor) — Four Chapter 2 constraints have no leaf

Q3.2 is the only Question-Tree leaf for constraints. Its answer enumerates: Python >= 3.12, uv-managed, strict pyright + ruff, no relative imports, no # type: ignore, UNIX target, Apache-2.0.

02_architecture_constraints.adoc asserts four further constraints that no leaf — Q3.2 or any other — records:

  • "`asyncio` is the orchestration runtime". The Question Tree records asyncio only as a performance mechanism (Q4.2: "Concurrent tool execution via asyncio tasks"), never as a constraint.

  • _-prefixed private modules and models.py placement (the "Hexagonal module conventions" row). Q3.4.1 recovered the *_port.py suffix only; the private-module and models.py conventions appear in no leaf.

  • "Pydantic for all external data". Q2.3.SUC.ToolInterface records Pydantic only for tool argument models, not as a whole-system constraint on all external data.

  • "Conventional Commits, no force-push". No leaf recovered any git-workflow convention.

Each constraint carries an AGENTS.md file:line citation, so the evidence resolves; the gap is that Q3.2 did not recover them.

T-03 (Minor) — The PyPI "cached 24 h" figure has no leaf

03_context_and_scope.adoc states the PyPI version lookup is "cached 24 h". The fact of update checks traces to Q5.3 (operational risks, evidence update.py:82), but no leaf records the 24-hour cache duration. The figure is a Phase-2 addition; update.py does define UPDATE_CACHE_TTL_SECONDS = 24 * 60 * 60, so the documentation claim is code-true — the gap is again in the tree.

T-04 (Minor, by design) — The ADR Pugh Matrices have no leaf

Each of the four ADRs carries a Pugh Matrix — a baseline option, a third alternative, and -1/0/+1 scores per criterion. The Question Tree contains no comparative trade-off analysis: Q3.9.ADR.* are all [OPEN] and Q3.9.index records only that "no ADR register exists". The chosen baselines (e.g. "Direct coupling to concrete classes"), the third alternatives (e.g. "Framework DI container", "Embedded SQLite database") and the non-? scores are Phase-2 synthesis with no leaf behind them.

This is recorded for completeness, not as a process error: the matrices are a mandated Phase-2 synthesis artifact, and their ? cells are correctly marked as needing team judgment. Only the asserted (non-?) scores lack a traceable origin.

Class B — Claims on an OPEN Leaf, Not Marked as a Gap

None. The two findings of the previous run are now corrected — see OPEN Leaves That Are Correctly Marked.

OPEN Leaves That Are Correctly Marked

For completeness, the following [OPEN] leaves are claimed in the documentation and correctly marked as gaps — no defect:

OPEN leaf Where the gap is marked

Q1.2.2, Q1.6 (segment priority)

PRD "Target Users" note + Open Questions table; Ch1 "Open issues" NOTE.

Q1.4 (why Vibe was built)

PRD Problem Statement WARNING; Ch1 "Open issues" NOTE.

Q1.5.2 (business KPI)

PRD Success Criteria WARNING; Ch1 "Open issues" NOTE.

Q2.2.UC.completeness (use-case catalog completeness)

Spec use-case-catalog NOTE ("recovered from code …​ whether it is the intended complete set …​ deferred"); PRD Open Questions table.

Q2.5.2 (Gherkin acceptance criteria)

Spec Acceptance Criteria preamble ("deferred"); PRD Open Questions.

Q2.6.BR.intent (rules vs. choices)

Spec Cross-cutting Business Rules WARNING; PRD Open Questions.

Q3.4.2 (why the strategies were chosen)

Ch4 preamble ("deferred …​ Accepted (inferred)"); Ch9 NOTE.

Q3.8.1 (STRIDE threat model)

Ch8 Threat Model WARNING ("candidate threats …​ deferred"); Ch11 R-007.

Q3.9.ADR.* (the four ADR rationales)

Each ADR Status — "Accepted (inferred)" — and Ch9 NOTE.

Q4.9 (quality-goal selection and ranking)

Ch1 Quality Goals preamble now marks both the selection of the top goals and their ranking as inferred and deferred; Ch1 "Open issues" NOTE; Ch4 and Ch10 deferral notes; PRD Open Questions.

Conclusion

The documentation traces cleanly for the large majority of its claims, and it now handles all 14 [OPEN] leaves correctly — the two previously-asserted claims (the top-five quality-goal selection and the use-case-catalog completeness) have been re-marked as gaps.

Four claims (T-01 (Major) — The "context warning at 50 % of the threshold" has no leaf–T-04 (Minor, by design) — The ADR Pugh Matrices have no leaf) still trace to no Question-Tree leaf — a 50 % warning threshold, four Chapter 2 constraints, a 24-hour cache, the ADR trade-off scores. Each is backed by file:line evidence in the documentation and, for T-01 and T-03, verified against the current source; what is missing is the corresponding Phase-1 leaf. Closing them is a Question-Tree completion task, not a documentation defect, and is left for the next Phase-1 pass.

Findings are recorded for disposition.