We study Bayesian inference of persistence from early observations of a heterogeneous susceptible--infected--susceptible (SIS) reaction--diffusion model with standard incidence. We show that the leading prevalence dynamics depend on the unknown coefficients only through the difference between the transmission and recovery rates and the infected diffusivity. The same pair determines the principal eigenvalue whose sign governs persistence, so early prevalence can inform the threshold even when the individual rates remain uncertain. We prove global weak well-posedness and quantitative parameter stability for the SIS system and use the resulting observation-map estimates to establish Bayesian well-posedness. For an initial infection of size $\varepsilon$, we derive a uniform second-order expansion that describes how the separate rates enter prevalence. Incidence depends explicitly on transmission at leading order. When the noise standard deviation is proportional to $\varepsilon$, we prove $O(\varepsilon)$ convergence in Hellinger distance of the nonlinear posteriors to their low-prevalence limits for both prevalence alone and joint prevalence and incidence data. We bound the principal eigenvalue using the growth rate of the infected $L^2$ norm between two times. This rate gives a lower bound for the full nonlinear system, and the upper bound separates spatial transients from nonlinear saturation. Uniform spectral estimates and bounds on infection size and measurement error give sufficient conditions for persistence and extinction. We also bound the error in posterior persistence probabilities under forward and eigenvalue approximation, accounting for posterior mass near the threshold. Numerical examples illustrate threshold inference and the predicted \(O(\varepsilon)\) convergence of the nonlinear posteriors to their low-prevalence limits in Hellinger distance.
We propose a deep backward multistep high-order regression (MHDBR) method for semilinear parabolic partial differential equations through their backward stochastic differential equation representations. The method combines multistep time discretization with regression targets formed by conditional expectations. Under explicit regularity, starting-value, forward-approximation and network-moment assumptions, we prove an error bound separating time discretization, conditional Monte Carlo sampling and neural-network approximation. The analysis assumes exact population minimization; finite-training-sample generalization and optimization errors are not included. For generators independent of the gradient variable, suitably constrained GroupSort networks and sufficiently large conditional sample sizes give order $J$ in the stated nodal $L^2$ error norm. The required network sizes depend on dimension, so this result does not establish dimension-independent complexity. We present a reproducible time-discretization test satisfying the analytical assumptions and three broader neural-network benchmarks, whose relation to the assumptions is discussed explicitly.
Gaussian basis functions provide an efficient and flexible alternative to spline activations in Kolmogorov-Arnold networks (KANs), but their performance is sensitive to the Gaussian scale parameter \epsilon. We introduce the partition-of-unity Gaussian KAN (PU-GKAN), a kernel-based normalization framework that transfers the Shepard partition-of-unity principle to trainable KAN edge functions. Each Gaussian edge basis is normalized by its local sum over fixed centers, preserving the standard edge-based architecture and essentially the same trainable coefficient structure as GKAN. The resulting feature map reproduces constants exactly at the edge level and admits an explicit normalized finite-feature kernel interpretation. Numerical experiments show that PU-GKAN reduces sensitivity to \epsilon, improves validation accuracy for most smooth and moderately non-smooth targets, and provides more stable training behavior. These benefits persist across variations in sample size and number of centers, higher-dimensional architectures, and Matern RBF bases. PU-GKAN also improves physics-informed neural networks for Helmholtz and wave equations, with validation-error reductions of up to 90%. These results establish partition-of-unity normalization as a mathematically grounded and broadly applicable framework for improving the reliability and accuracy of radial basis function KANs.
Neural operators have achieved strong performance in learning solution operators of partial differential equations (PDEs), but their inherently continuous representations struggle to capture discontinuities and sharp transitions. Existing approaches typically approximate such features within continuous function spaces, often requiring increased model capacity and high-resolution data. In this work, we propose Cut-DeepONet, a two-stage training framework that explicitly models discontinuities while reducing learning complexity. Our approach reformulates the problem via a lifting strategy, partitioning the domain into smooth subregions while representing discontinuities as boundaries in a higher-dimensional space. This separation aligns the operator learning task with the inductive bias of neural networks and avoids directly approximating discontinuities. An additional network predicts input-dependent discontinuity locations for unseen inputs, which are then used to guide the neural operator in generating smooth components within each region. Experiments on benchmark PDEs show that Cut-DeepONet outperforms state-of-the-art methods, even when trained on low-resolution datasets. The method excels on problems with discontinuities and sharp transitions, while using fewer trainable parameters. Our results highlight the benefits of changing the representation of operator learning rather than increasing model complexity.
Adversarial training of binary classifiers can be reformulated as regularized risk minimization involving a nonlocal total variation. Building on this perspective, we establish a characterization of the subdifferential of this total variation using duality techniques. To achieve this, we derive a dual representation of the nonlocal total variation and a related integration-by-parts formula, involving a nonlocal gradient and divergence. We provide such duality statements both in the space of continuous functions vanishing at infinity on proper metric spaces and for the space of essentially bounded functions on separable metric spaces. Furthermore, under some additional conditions we provide characterizations of the subdifferential in these settings.
Phase-field models are widely used to simulate microstructure evolution but their numerical solution remains computationally demanding. Neural operator surrogates provide a promising alternative, yet existing approaches are data-driven, requiring large datasets and exhibiting limited robustness when extrapolating to unseen configurations. We introduce DeepRitzSplit, an unsupervised neural operator framework for gradient-flow systems based on a variational formulation of energy-stable splitting schemes. It learns the time evolution semigroup associated with a convex-concave free-energy decomposition by minimizing the underlying variational problem at each time step. It is designed to inherit the dissipative structure of the underlying gradient flow without requiring trajectory datasets. We further introduce a Reaction-Diffusion Neural Operator (RDNO) architecture motivated by splitting schemes, with separate reaction and diffusion components to improve training efficiency and interpretability. We assess the approach on the Allen-Cahn equation and apply it to a phase-field model of anisotropic dendritic growth during solidification. In the Allen-Cahn benchmark, DeepRitzSplit shows better generalization than the supervised baseline on the out-of-distribution configurations tested, using 180 training configurations. In dendritic growth, the method captures the overall phase morphology and dendrite tip velocity well, while the tip radius is more sensitive to the training strategy. Refinement based on the residual of the splitting scheme markedly improves this second-order morphological quantity. The trained operators retain good predictive capabilities on configurations substantially different from the training set. These results demonstrate the potential of variationally trained neural operators as physically informed surrogates for accelerated microstructure simulations.
We present a computational framework that combines machine-learning-based discovery with rigorous computer-assisted numerical certification and formal verification for strongly nonlinear partial differential equations. We demonstrate the approach for periodic gravity-capillary traveling water waves. Physics-informed neural networks are used as a search mechanism, with the wave profile, wave speed and, in continuation experiments, surface tension learned directly from the governing equations without training data. Through a staged training and refinement procedure, the neural networks achieve near-machine-precision residuals of the governing equations, even in strongly nonlinear regimes. A single formulation recovers high-amplitude Stokes waves, Crapper waves and Wilton ripples. Continuation from a high-amplitude Wilton ripple in surface tension reveals new high-amplitude candidate solutions. We freeze one candidate as an explicit finite representation and rigorously certify a uniform bound on its governing-equation residual using interval arithmetic, accounting for floating-point roundoff, Fourier truncation, aliasing and grid-to-continuum errors. This independently validates the neural-network accuracy. We formalize the certification argument in Lean, which machine-checks the deductions from certified numerical bounds to the residual theorem. Both the interval certification and formalization were developed with substantial assistance from a large language model, while the validity of the result remains independent of the language model: the numerical bounds are rigorously enclosed by interval arithmetic and the subsequent mathematical deductions are checked by the Lean kernel. Together, these steps provide a pipeline from neural discovery to rigorous numerical certification and machine-checked verification with auditable mathematical guarantees.
Modeling the nonlinear acoustic wave propagation in biological tissues needs accuracy, especially in ultrasound imaging and therapeutic applications. The 1D Westervelt equation, \( p_{xx} - p_{tt} + \gamma\, p_{ttt} + \eta\,(p^{2})_{tt}=0, \) could be used to describe a finite-amplitude ultrasound propagation in these tissues. This equation accounts for nonlinearity, \(\eta\), and attenuation, \(\gamma\). To solve this equation, we observe that traditional solvers may struggle to do so correctly. This is due to the difficulties arising from the nonlinear nature of the equation. We solve the equation by defining three synthetic cases - linear (\(\eta=0.001\), \(\gamma=10^{-6}\)), shock-dominated (\(\eta=0.02\), \(\gamma=10^{-6}\)), and dissipation-dominated (\(\eta=0.01\), \(\gamma=5\times10^{-4}\)) - and using Physics-Informed Neural Networks (PINNs) to showcase the solution. A sinusoidal activation function is used to capture the oscillatory behavior of the problem. The results of our study, with PDE residual converging to at least \(10^{-6}\), boundary condition loss to \(10^{-5}\), and the PINN loss converging to \(10^{-5}\) for each case at \(30,000\) epochs, show that \(\gamma\) scales the rate of amplitude decay, while \(\eta\) governs frequency-dependent phase speeds.
In this paper, we establish a universal approximation theorem for solution operators of semilinear stochastic heat equations in spatial dimension \(d\ge2\), extending deep neural operator approximation results to SPDEs with distribution-valued Gaussian forcing. A smooth Fourier cutoff reduces the forcing to finitely many measurements, while Besov and Schauder estimates control the resulting solution error. The finite-dimensional solution map is then approximated by ReLU branch networks together with grid-based trunk functions, yielding explicit bounds on the numbers of forcing measurements and trunk functions and approximation in every finite stochastic moment. For linear equations, the reduction coincides with truncation of the first Wiener chaos. Lastly, as an application, we prove a high-probability local-in-time approximation theorem for stochastic Navier--Stokes equations, with finite measurements of both the initial condition and forcing and an explicit sensing rate.
In this paper, we derive an explicit solution of the stationary prediction with expert advice PDE for five experts. The formula is given in three regions. In the first two regions, it is the four-expert solution plus a single integral with an elementary positive density. In the third region, it is a finite sum of hyperbolic products whose coefficients are determined by one scalar quadrature. Our formula establishes that the rank ordered adversary $(1,0,1,0,0)$ achieves the Hamiltonian maximum in the PDE everywhere in the ordered sector, while the COMB strategy $(1,0,1,0,1)$ is optimal only on a lower dimensional subset of the sector (where $x_1=x_2$ and $x_3=x_4$). The verification of the Hamiltonian inequalities is a tedious task, part of which is completed with a computer assisted proof. The verification reduces to 21 scalar inequalities, which we prove using 147 exact rational Bernstein polynomial certificates. The exact certificates and their independent arithmetic checks are included in a supplement to this paper, and a Lean 4 formalization machine-checks the verification and both main theorems, apart from the viscosity characterization.
We develop a quantitative statistical theory of transformers in the large-context regime by adopting the abstraction of contextual flow maps (CFMs): dynamical systems that evolve a distinguished token in the presence of a contextual measure across a stack of attention blocks. Within this framework, the finite-context model approximates an idealized infinite-context system in which the contextual measure is replaced by its underlying population, so that the context length $n$ becomes a statistical resource. Exploiting the McKean--Vlasov structure of the dynamics and the classical machinery of propagation of chaos, we establish a forward bound controlling the deviation between the finite- and infinite-context CFMs uniformly along depth, and a backward bound controlling the deviation between the corresponding training trajectories uniformly across iterations of online gradient descent. Both bounds achieve the optimal Wasserstein rate $n^{-1/d}$ for general CFMs and parametric rate $n^{-1/2}$ for a restricted class of CFMs that includes transformers as a special case. The analysis rests on a new Eulerian adjoint formulation of the loss gradient and stability estimates for the resulting forward--adjoint system, both of which may be of independent interest.
Operator learning for partial differential equations (PDEs) aims to learn solution operators on infinite-dimensional function spaces from finite-resolution data. In this setting, it is important for the learned model to be discretization-invariant, or resolution-robust, and to reflect PDE-specific structure. It is therefore natural to ask how such structure should be encoded in the model architecture, hypothesis class, or learning procedure. In this paper, we study operator learning for solution operators of nonlinear parabolic PDEs based on Duhamel--Picard iteration. We formulate Picard iteration as an abstract state-transition model and present a theoretical framework for Picard-type operator learning. We derive implementation-agnostic generalization error bounds that separate the implementation error from the estimation error associated with the abstract state-transition model induced by Picard iteration. A key consequence is that increasing the Picard depth reduces the Picard truncation error without causing an unbounded growth of the entropy-based estimation error. We also extend the analysis to long-time prediction by rolling out the same learned local model over successive time blocks. Finally, we illustrate the theory for nonlinear heat equations on the torus using a Picard-type Fourier neural operator as a concrete implementation.
We study stationary solutions of McKean-Vlasov equations on the circle. We establish an exact equivalence between solutions of the stationary McKean-Vlasov equation and an infinite-dimensional quadratic system of equations over Fourier coefficients, allowing explicit characterization of stationary states in a sequence space rather than a function space. This framework provides a transparent description of local bifurcations, their periodicity and resonance structures, while accommodating singular potentials. We derive analytic expressions for the emergence, form and shape (supercritical, critical, subcritical or transcritical) of bifurcations involving possibly multiple Fourier modes and connect them with discontinuous phase transitions. We also characterize, under suitable assumptions, the detailed structure of the stationary bifurcating solutions through leading small-amplitude coefficients for arbitrarily many fixed Fourier modes. At the global level, we establish regularity and concavity properties of the free energy landscape, proving existence, compactness, and coexistence of globally minimizing stationary measures, and identifying discontinuous phase transitions with points of non-differentiability of the minimum free energy map. We apply the theory to the Noisy Mean-Field Transformer model, where we show how changing the inverse temperature $\beta$ affects the geometry of the infinitely many bifurcations from the uniform measure. We also discuss how increasing $\beta$ motivates the study of approximate multi-mode stationary profiles as candidates for a separate metastability analysis; persistence times require additional dynamical estimates. Moreover, we determine the exact threshold $\beta_0\simeq2.447$ separating continuous and discontinuous phase transitions in this model. The same threshold marks the change of the first bifurcation from supercritical to subcritical.
We establish mathematical foundations for optimal spatiotemporal vector fields governing continuous score-guided probability flows over bounded submanifolds $M \subset SE(3)$ on the Special Euclidean group $SE(3) \cong \mathbb{R}^3 \times SO(3)$. Modeling dynamic density evolution via a score-perturbed Lie-algebraic continuity equation under zero-flux boundary conditions, we resolve the non-local score drift singularity $s(g,t) = \nabla_{SE(3)} \log \rho(g,t)$ as $\rho \rightarrow 0^+$ via weighted parabolic flux dissipation bounds. Using left-invariant exponential group mollifiers, we derive boundary-preserving DiPerna--Lions commutator estimates tracking structure constants $\{c_{ij}^k\}$. Second, interpreting the optimality system as a 7D space-time elliptic operator $L_{7D} = -\Delta_{SE(3)} - \partial_t^2$ with zero-order corner compatibility on $\partial M \times \{0,T\}$, we establish Agmon--Douglis--Nirenberg $W_p^2(Q_T)$ shift estimates tracking Lie algebra constants, boundary geometry $\|\partial M\|_{C^2}$, and Ricci curvature $\|Ric_{SE(3)}\|_{L^\infty}$, yielding a full $C^\infty(\overline{Q}_T)$ Schauder bootstrap expansion. Third, we extend this system to non-cooperative Markov Mean-Field Games (MFGs) on $SE(3)$, establishing score-guided Nash equilibria via Schauder--Tychonoff fixed points. Crucially, the constant diagonal Ricci curvature $Ric_{SE(3)} = \mathrm{diag}(0,0,0,1/2,1/2,1/2)$ from $SO(3)$ induces a sharp Bochner--Weitzenb\"ock spectral gap boost $\lambda_1(-\Delta_{SE(3)}) \ge \lambda_1(-\Delta_{\mathbb{R}^3}) + 1/2 > 0$. This positive rotational curvature enhances the Lie-algebraic Poincar\'e coercivity constant, demonstrating that intrinsic group curvature stabilizes score-guided flows against dispersion and expands the critical Nash uniqueness horizon $T_{SE(3)}^*$ beyond flat Euclidean limits.
Transformers with self-attention modules as their core components have become an integral architecture in modern large language and foundation models. In this paper, we study the evolution of tokens in deep encoder-only transformers at inference time which is described in the large-token limit by a mean-field continuity equation. Leveraging ideas from the convergence analysis of interacting multi-particle systems, with particles corresponding to tokens, we prove that the token distribution rapidly concentrates onto the push-forward of the initial distribution under a projection map induced by the key, query, and value matrices, and remains metastable for moderate times. Specifically, we show that the Wasserstein distance of the two distributions scales like $\sqrt{{\log(\beta+1)}/{\beta}}\exp(Ct)+\exp(-ct)$ in terms of the temperature parameter $\beta^{-1}\to 0$ and inference time $t\geq 0$. For the proof, we establish Lyapunov-type estimates for the zero-temperature equation, identify its limit as $t\to\infty$, and employ a stability estimate in Wasserstein space together with a quantitative Laplace principle to couple the two equations. Our result implies that for time scales of order $\log\beta$, which correspond to the early-phase behavior most relevant for practical finite-depth models, the token distribution concentrates at the identified limiting distribution. Numerical experiments confirm this and, beyond that, complement our theory by showing that for finite $\beta$ and large $t$ the dynamics enter a different terminal phase, dominated by the spectrum of the value matrix.