Training-Time Batch Normalization Reshapes Local Partition Geometry in Piecewise-Affine Networks
원문 초록
The effect of batch normalization (BN) on the function realized during training is less understood than its effect on optimization. We study training-time BN in continuous piecewise-affine (CPA) networks through the affine regions they induce in local $\ell_\infty$ windows around the data. Conditioned on a mini-batch, every switching hyperplane of a BN layer with learned parameters and a general CPA activation is a translate of a hyperplane through the batch centroid, at an offset given by a standardized quantity times the batch spread, independent of the raw bias. We prove an exact formula, valid under general position, for the number of regions in a window in terms of window cuts and interior intersections. For a BN layer it gives a lower bound on the local count in terms of the standardized offsets; the bound depends on the batch only through its spread and not on the biases or the batch location. Without BN the best such bound is $1$: for any weights and batch, some biases leave the window as a single region. We prove that the formula holds in deeper layers, inside parent affine regions of any rank. By the exact formula, BN increases the expected local count whenever it increases the expected numbers of window cuts and of interior intersections, which we examine empirically. Offset distributions of networks trained on synthetic data, MNIST, CIFAR-10 and TinyImageNet show higher window-cut rates under BN, and exact enumeration in two dimensions attributes most of the additional BN regions to interior intersections.
제출 이력
확인된 버전만 표시합니다. 전체 제출 이력은 arXiv 원문에서 확인하세요.
- arXiv:2605.04946v3제출 시각 미확인
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
공통 태그로 연결된 공개 논문이 아직 없습니다.
토론
이 논문과 연결된 포럼 글이 아직 없습니다.