Reasoning Consensus: Structural Ensembling of LLM Reasoning via Weighted DAG Aggregation

원문 초록

Large Language Models (LLMs) reason in free-form chain-of-thought (CoT) without revealing which steps were well supported and which alternatives were rejected. In high-stakes domains, users then cannot judge if an answer rests on sound reasoning. Majority voting and multi-agent debate improve answers using model agreement, but do not expose agreement on individual steps. Reasoning Consensus (RC) extracts a typed reasoning graph from each CoT, merges equivalent steps across graphs from diverse models, and weights every step by how many chains attest to it. RC reveals this step-wise support and the varied justifications behind each answer. On 7 benchmarks across law, science, medicine, narrative reasoning and logic, RC matches majority voting and debate in accuracy (69.4% vs. 69.0-70.3%). An LLM judge prefers RC's best-supported graph to other graphs for the same answer. In a user study, participants read RC graphs faster than debate transcripts or vote tallies, and 14 of 18 preferred them for future use.