Oversmoothing as Representation Degeneracy in Neural Sheaf Diffusion

원문 초록

Neural Sheaf Diffusion (NSD) learns a task-adapted sheaf geometry for node classification. We connect the study of these learned geometries with Geometric Invariant Theory (GIT) to analyse their representation-theoretic structure and develop mechanisms for geometric control during training. Identifying cellular sheaves with representations of the incidence quiver connects direct-sum decompositions to the harmonic space and motivates the study of trivial or low-complexity summands as potential degeneration modes. We make this perspective concrete by focusing on the trivial representation: on an embedded copy, the sheaf Dirichlet energy reduces to weighted scalar graph energy. The GIT stability framework provides a criterion for excluding such copies within the semistable class. Using its moment-map characterisation of polystability, we construct differentiable surrogates that encourage the corresponding balance conditions. We identify an equal-stalk obstruction to assigning a destabilising weight to the trivial representation and show how nonuniform stalk dimensions remove it. Controlled energy-minimisation experiments reproduce trivial-component formation, while native feature-conditioned NSD experiments on heterophilic graphs show how moment-map regularisation changes the balance and Dirichlet-energy trajectories of learned geometries during training, alongside their classification behaviour.