Profile-aware diagnostics of discarded parameter-Hessian mass

원문 초록

Evaluating block-diagonal curvature models involves two questions: how much curvature they omit, and what a specific correction changes at a finite step. We measure the squared off-block fraction of the parameter Hessian against references for group sizes and the observed within-group curvature profile. In CIFAR-100 ResNet-18 and a skip-free counterpart, similar initial discarded fractions accompany different profiles. At saved trained ResNet-18 weights, changing from stage to layer groups moves estimated off-block mass from below to above the profile envelope; an approximate conditional screen resolves this comparison in all six seed/input cells, while some skip free comparisons remain unresolved. Separately, in saved ResNet-50/Stanford Cars states, we compare candidates formed from off-group Hessian action, the remaining model residual and their combination through the same Hessian-informed map. At unit scale off-group input alone raises native minibatch loss relative to the base preconditioner candidate in 17 of 18 state/geometry comparisons, yet the combined candidate has lower loss than the remainder-only candidate in nine of those cases; preferences vary with scale and correction-norm matching. The profiles help assess block models and alternative groupings, while same-state candidate comparisons test a particular correction without equating structural mass with training gain.