The Socratic Code Theory Recovery method inverts the direct approach (Experiment 1a), which generates documentation first and flags uncertainties as an afterthought. This leads to "confidence inflation" — the LLM writes plausible-sounding rationale instead of admitting ignorance. A question-first approach that recursively decomposes high-level questions until each leaf is either answered from code or marked as unanswerable should produce more honest Open Questions and a clearer separation of known vs. unknown.

84% of questions were answered with code evidence. 16% remained open.

The tree uses Semantic Anchors as decomposition guides:

This confirms that Semantic Anchors work not just as prompt compression (Experiment 1a finding) but also as decomposition heuristics. The terms carry enough structure to guide a MECE breakdown.

Strong MECE: Q-2 (Specification) and Q-3 (Architecture) — no overlap, full coverage.

Weak MECE: Q-5 (Risks) — ad-hoc categories with overlaps (missing docs vs. missing schema, edge-case fragility vs. operational risks). A risk taxonomy like STRIDE would improve this branch.

Overlap: Q-3.10 (quality requirements in arc42) and Q-4 (quality goals per ISO 25010) cover similar ground from different angles. Not a defect per se, but the boundary is unclear.

The Direct approach produced 33 Open Questions, but 2 were false opens (already answerable from code) and several "Medium Confidence" items were really guesses dressed as analysis. The Socratic approach produced 26 Open Questions with zero false opens.

The key difference is structural: in the Socratic approach, the LLM must explicitly decide at each question "can I answer this?" before proceeding. In the Direct approach, the LLM writes documentation first and retrospectively considers what it might not know.

Example — ADR rationale:

The Socratic approach categorized only 38% of Open Questions (vs. 100% in Direct) and assigned Ask roles to only 27% (vs. 52%). This is a prompt issue — the template was followed less consistently during the synthesis phase. The Question Tree leaves have better metadata than the synthesized OPEN_QUESTIONS.adoc.

Direct wins 7 chapters on content volume and detail. Socratic wins 3 chapters (9, 10, 11) — the chapters where honesty about unknowns matters most.

The Direct approach produces more ADRs (6 vs. 3) but presents them without caveats. The Socratic approach produces fewer but explicitly marks each as "reverse-engineered from code." For an architect reading these, the Socratic ADRs are safer: you know what you’re getting.