Beyond Self-Reflection: An Execute-Distill-Verify Framework for Agentic Experience Learning

원문 초록

Experience learning lets large language model agents reuse lessons through external memory without updating model parameters. However, reflection can endorse execution errors and store them as future guidance, a failure mode we call the Self-Confirmation Trap. We propose EDV, an Execute-Distill-Verify framework for offline experience construction. Heterogeneous agents independently execute each task; a third-party distiller compares their trajectories and environmental feedback to extract candidate lessons; and the executors separately review each candidate, retaining only unanimously approved lessons. Evaluation uses a single solver retrieving from the frozen memory bank. Experiments on $\tau^2$-bench, Mind2Web, and MMTB show improved revisited-task performance, with additional evidence of within-domain transfer to held-out tasks on $\tau^2$-bench. On an annotated Retail subset, EDV reduces the Self-Confirmation Trap Rate from 11.0% with RB-MaTTS to 5.0%, indicating less retention of erroneous guidance.