On Emergent Capabilities and Model Merging

원문 초록

Model merging combines separately fine-tuned checkpoints by arithmetic on their weights, and is now a routine way to assemble capabilities cheaply. We ask what it does to emergent capabilities: behaviors that arise from fine-tuning but were never an explicit training target. Studying two independent testbeds (activation oracles and emergent-misaligned models) across three model families, we find that the answer is threefold. First, merging preserves an emergent capability that both parents carry: merging two misaligned checkpoints retains most of their broad misalignment across the whole mixing range. Second, merging cannot create an emergent capability that is superadditive in its parents: no weighted merge of two single-task oracles reaches the jointly-trained oracle's auditing ability. Third, when only one parent carries the capability, merging dilutes it faster than the trained capability that accompanies it: the gap is significant in most settings.