Lattice-Based Pruning of Recurrent Neural Networks via Poset Modeling: A Corrected Theory and a Matched-Sparsity Evaluation

원문 초록

Magnitude pruning of recurrent neural networks (RNNs) ignores how information flows between neurons. An earlier version proposed to model RNN dependencies as a partially ordered set (poset) and to retain the meet-irreducible elements of its dependency lattice. By Birkhoff's representation theorem, the meet-irreducible ideals of a finite poset are in bijection with its elements, so retaining them retains everything; the critical-set criterion used earlier selects precisely the elements comparable to all others, of which a dense recurrent layer has none. Condensing a thresholded dependency graph into strongly connected components yields a well-defined dependency order, a graded order score and an acyclic pruning rule; for Cohen-Grossberg networks and LSTMs, acyclic pruning makes local stability decompose over modules. For a 128-unit LSTM pruned at matched sparsity on row-wise and chunked sequential MNIST (three seeds, equal training budgets), the original rules remove every recurrent weight or equal random column pruning; the order score is no better than degree, PageRank or norm; trained weights show little directional bias; and imposing acyclicity from initialisation costs 2.1 and 10.5 percentage points at 50% and 90% sparsity. In the networks studied, order structure must be imposed rather than discovered.