PyTorchKR Papers

최신 논문

총 524,201편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2605.12309v4cs.CV

G$^2$TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models

The development of separate-encoder Unified multimodal models (UMMs) comes with a rapidly growing inference cost due to dense visual token processing. In this paper, we focus on understanding-side visual token reduction for improving the efficiency of separate-encoder UMMs. While this topic has been widely studied for MLLMs, existing methods typically rely on attention scores, text-image similarity and so on, implicitly assuming that the final objective is discriminative reasoning. This assumption does not hold for UMMs, where understanding-side visual tokens must also preserve the model's capabilities for editing images. We propose G$^2$TR, a generation-guided visual token reduction framework for separate-encoder UMMs. Our key insight is that the generation branch provides a task-agnostic signal for identifying understanding-side visual tokens that are not only semantically relevant but also important for latent-space image reconstruction and generation. G$^2$TR estimates token importance from consistency with VAE latent, performs balanced token selection, and merges redundant tokens into retained representatives to reduce information loss. The method is training-free, plug-and-play, and applied only after the understanding encoding stage, making it compatible with existing UMM inference pipelines. Experiments on image understanding and editing benchmarks show that G$^2$TR substantially reduces visual tokens and prefill computation by 1.94x while maintaining both reasoning accuracy and editing quality, outperforming baselines on almost all benchmarks. Code is at: https://github.com/lijunxian111/G2TR.

arXiv:2605.12154v2cs.AI

MM-OptBench: A Solver-Grounded Benchmark for Multimodal Optimization Modeling

Optimization modeling translates real decision-making problems into mathematical optimization models and solver-executable implementations. Although language models are increasingly used to generate optimization formulations and solver code, existing benchmarks for this capability are almost entirely text-only. This omits many optimization-modeling tasks that arise in operational practice, where requirements are described in text but essential instance information is often conveyed through visual artifacts such as tables, graphs, maps, schedules, and planning dashboards. We introduce multimodal optimization modeling, a benchmark setting where models must construct both a mathematical formulation and executable solver code from a text-and-visual problem specification. To evaluate this setting, we develop a solver-grounded framework that generates structured optimization instances, verifies each instance with an exact solver, and builds both model-facing inputs and hidden reference files from the same verified source. We instantiate it as MM-OptBench, a benchmark of 780 solver-verified instances spanning 6 optimization families, 26 subcategories, and 3 structural difficulty levels. We evaluate 9 multimodal large language models (MLLMs), including 6 frontier general-purpose and 3 math-specialized models, with aggregate, family-level, difficulty-level, and failure-mode analyses. The task remains far from solved: the best two models reach 52.1% and 51.3% pass@1, while on average across the six general-purpose MLLMs, pass@1 is 43.4% on easy and 15.9% on hard instances. All three math-specialized MLLMs solve 0/780 instances. Post-hoc extraction diagnostics and oracle-reading evaluation reveal difficulties in both multimodal grounding and downstream solver-artifact generation. MM-OptBench provides a concrete testbed for solver-grounded, decision-oriented multimodal intelligence.

arXiv:2605.11967v2cs.CV

H2G: Hierarchy-Aware Hyperbolic Grouping for 3D Scenes

Hierarchical 3D grouping aims to recover scene groups across multiple granularities, from fine object parts to complete objects, without relying on semantic labels or a fixed vocabulary. The main challenge is to transform 2D foundation-model cues into coherent hierarchy supervision and embed that hierarchy in a 3D representation. We propose H2G, a hyperbolic affinity field for hierarchical 3D grouping. Our method derives semantically organized tree supervision by interpreting foundation-model affinities through Dasgupta's objective for similarity-based hierarchical clustering. This supervision is distilled into a single Lorentz hyperbolic feature field, whose geometry is well suited for tree-like branching structures. A hierarchy-aware objective aligns the field with fine-level assignments, coarse object structure, compact feature clusters, and LCA (Lowest Common Ancestor) ordering. This formulation represents multiple grouping levels in one feature space, enabling semantic hierarchical grouping grounded in 2D foundation-model knowledge. Code is available at https://github.com/Rhqo/H2G.

arXiv:2605.10379v3cs.CL

Not All Proofs Are Equal: Evaluating LLM Proof Quality Beyond Correctness

Large language models (LLMs) have become capable mathematical problem-solvers, often producing correct proofs for challenging problems. However, correctness alone is not sufficient: mathematical proofs should also be clear, insightful, contain transferable knowledge, and be generally useful to the reader. While proof quality is subjective and depends on the reader and context, many of its components can be made concrete. In this work, we identify such components and introduce ProofRank, a benchmark curated from challenging competitions that measures several scalable proxies of proof quality: (i) conciseness - quantifying whether proofs avoid unnecessary steps; (ii) computational ease - the extent to which a proof relies on tedious calculations; (iii) cognitive simplicity - how accessible the used proof techniques are; (iv) diversity - how varied a model's proofs for a single problem are; and (v) adaptivity - how well a model can follow a specified technique. We find proof quality varies substantially across models along dimensions that correctness largely fails to capture. Every proof-level proxy can be steered through prompting, but often at a moderate correctness cost. ProofRank thus captures important aspects of mathematical proofs beyond correctness, and indicates the need for models that produce proofs which are not only correct but also valuable.

arXiv:2605.09995v2cs.CL

Annotations Mitigate Post-Training Mode Collapse

Post-training (via supervised fine-tuning) improves instruction-following, but often induces semantic mode collapse by biasing models toward low-entropy fine-tuning data at the expense of the high-entropy pretraining distribution. Crucially, we find this trade-off worsens with scale. To close this semantic diversity gap, we propose annotation-anchored training, a principled method that enables models to adopt the preference-following behaviors of post-training without sacrificing the inherent diversity of pretraining. Our approach is simple: we pretrain on documents paired with semantic annotations, inducing a rich annotation distribution that reflects the full breadth of pretraining data, and we preserve this distribution during post-training. This lets us sample diverse annotations at inference time and use them as anchors to guide generation, effectively transferring pretraining's semantic richness into post-trained models. We find that models trained with annotation-anchored training can attain $6 \times$ less diversity collapse than models trained with SFT, and improve with scale.

arXiv:2605.08698v3cs.CV

Supersampling Stable Diffusion and Beyond: Training-Free Neural Network Scaling via Signal Interpolation

This work presents a training-free approach for interpolating neural networks using signal interpolation methods (i.e., linear, bilinear, bicubic). We discovered this phenomenon while attempting to interpolate the convolution layers of Stable-Diffusion-1.5 for beyond-training-resolution (BTR) image generation. As we know, for generating BTR images using DDPMs (Denoising Diffusion Probabilistic Models), there are mainly two approaches, namely, 1) patch-wise diffusion and 2) convolution-kernel dilation. Patch-wise diffusion cannot solve the object-duplication problem efficiently, while the kernel dilation approach may be harder to fine-tune later. Thus, we proposed convolution-kernel interpolation of the Stable Diffusion 1.5 model for $4\times$ BTR image generation. We discover and show mathematically that signal interpolation methods can correctly scale convolution kernels when multiplied by a flat-attenuation factor, while achieving competitive performance against the state of the art. We then applied the same methodology to non-DDPM deep neural network architectures, adapting them for BTR image classification, achieving a worst-case performance drop of $\sim 7\%$ on the ImageNet-1k validation set. Again, we subsample the penultimate fully connected layer of VGG-16 using our method, reducing up to $\sim 9\%$ parameters with $\sim 5\%$ loss in performance. Finally, we also discuss how we can reduce the memory footprint of training neural networks to $4\times$ or greater.

arXiv:2605.08505v2cs.LG

Scaling Limits of Long-Context Transformers

We study the long-context limit of softmax self-attention with a fixed query and a random context of $n$ i.i.d. keys on the sphere with the intrinsic dimension of the key distribution held fixed, viewing the inverse temperature $\beta_n$ as the scaling parameter that decides whether attention degenerates into uniform averaging or collapses onto the single closest key. We show that the critical scale at which selectivity emerges is determined by the local exponent of the distance-to-query distribution near zero rather than by global features of the context, and scales like $\beta_n^\ast \asymp n^{2/(d-1)}$ for uniform keys on $\mathbb{S}^{d-1}$. Furthermore, we characterize the limiting laws of the ordered attention weights and of the attention output across all regimes of $\beta_n$: a subcritical regime in which the output reduces to a local average around $q$ with explicit deterministic bias and Gaussian fluctuations; a critical regime in which a finite collection of nearest keys retains macroscopic mass without single-key collapse; and a supercritical regime in which all mass concentrates on the closest key. Of notable interest is the subcritical case with identity value matrix where the attention map approximately implements a backward heat equation.

arXiv:2605.08453v2cs.LG

Sink vs. diagonal patterns as mechanisms for attention switch and oversmoothing prevention

This paper studies the role of sinks and diagonal patterns as attention switch and anti-oversmoothing mechanisms. We analyze geometric conditions under which sinks can be represented, showing a necessary alignment between the embedding of the sink and all other embeddings. Next, we refine the current understanding of the role of sinks in oversmoothing prevention: we specify the conditions under which dense attention is expected to smooth more than sparse attention, and empirically verify that such conditions are often satisfied in practice. We further prove an equivalence between sinks and hard attention switch, in which the output of the attention is identically 0. Finally, we relax the hard attention switch by allowing token self-communication: we provide a quantitative comparison of the costs of representing sinks vs.\ diagonal patterns, showing why sinks are favored in pretrained transformers. The introduction and analysis of diagonal patterns and the generalization of the attention switch close the gap between what oversmoothing prevention requires and what sinks provide, while also establishing when and why attention layers act like MLPs if token communication is not necessary.

arXiv:2605.08332v3quant-ph

Optimal FALQON for Quantum Approximate Optimization via Layer-wise Parameter Tuning

Feedback-based adaptive quantum optimization (FALQON) is a promising approach for combinatorial optimization on noisy intermediate-scale quantum devices, but prior studies report that standard fixed-parameter schedules can require relatively deep circuits to reach strong solution quality. We study Optimal FALQON, a layer-wise formulation that treats the time-step $\delta_k$ and scaling parameter $w_k$ as classical decision variables optimized per layer. The benchmark uses all 94 non-isomorphic 3-regular graphs with 12 vertices under 8192-shot simulation and compares standard FALQON, Optimal FALQON, and QAOA/QAOA-MA Powell runs warm-started from FALQON-derived schedules. Across approximation ratio, success probability, evaluation-normalized efficiency, and layer-normalized efficiency, Optimal FALQON substantially improves on standard FALQON, and Optimal-FALQON warm starts substantially improve on standard-FALQON warm starts.

arXiv:2605.07695v2cs.CV

OphEdit: Training-Free Text-Guided Editing of Ophthalmic Surgical Videos

High-fidelity surgical video generation holds great promise for medical training and AI development, yet precise, controllable editing of such videos remains a formidable challenge. Modifying surgical attributes such as instrument, tissue interactions or procedural phases is particularly difficult due to the strict anatomical and temporal constraints of ophthalmic procedures. We introduce OphEdit, the first training-free framework for text-guided editing of ophthalmic surgical videos. Our approach uses a deterministic second-order ODE inversion pipeline to capture attention Value tensors from the source video and selectively injects them into the conditional branch of classifier-free guidance during denoising, preserving the intricate anatomical geometry of the eye while mapping text-driven semantic modifications onto the video. Clinical evaluation by an ophthalmologist shows that OphEdit handles complex transformations, such as instrument swaps and procedural variations, with superior structural fidelity and temporal consistency compared to adapted general-domain video editors, offering a scalable route to diverse surgical training data. The code and prompts can be accessed at https://github.com/ophedit/OphEdit

arXiv:2605.07147v3cs.LO

MathlibPR: Pull Request Merge-Outcome Benchmark for Formal Mathematical Libraries

The ecosystem of Lean and Mathlib has become the de facto standard for large language model (LLM) assisted formal reasoning with remarkable successes in recent years. Those successes, however, only consume Mathlib as an essential dependency but do not directly contribute to it. In the meantime, the growth of Mathlib has recently been bottlenecked by the review process, which requires human reviewers to judge whether proposed pull requests (PRs) follow the Mathlib's conventions and are worth integrating as part of a shared mathematical infrastructure. This leads to our central question: can LLMs help review Mathlib PRs? To this end, we introduce MathlibPR, a benchmark built from real Mathlib4 PR histories. We further propose a staged evaluation protocol and use it to evaluate both LLM models (e.g., DeepSeek, Qwen, Goedel, and Kimina) and LLM agents (e.g., Codex and Claude Code). Surprisingly, both LLM models and LLM agents struggle to distinguish merge-ready PRs from build-passing PRs that were revised or never merged. By turning Mathlib PR histories into a supervised signal, MathlibPR provides a step toward reviewer assistants and reward models that could help evaluate PRs and steer LLMs toward producing merge-ready Mathlib contributions.

arXiv:2605.06874v2cs.LG

On the Divergence of Differential Temporal Difference Learning without Local Clocks

Learning rate is a critical component of reinforcement learning (RL). This work uses global and local clocks to distinguish two types of learning rates. The former is of the standard form $\alpha_t$ that depends only on the time step $t$ (i.e., a global clock). The latter is of the form $\alpha_{\nu(S_t, t)}$, where $\nu(s, t)$ counts the number of visits to state $s$ until time $t$ (i.e., a local clock). In discounted RL, an RL algorithm that is convergent with a local clock is always also convergent with a global clock, and vice versa. We are not aware of any counterexample. The key contribution of this work is to show that this nice correspondence breaks down in average-reward RL. Specifically, we construct a counterexample showing that although differential temporal difference (TD) learning is convergent with a local clock, it can diverge with a global clock. We empirically observe this divergence in our experiments and also prove that the associated ODE of differential TD with a global clock is unstable. This counterexample closes the open problem in Wan et al. [2021], Blaser et al. [2026].

arXiv:2605.06046v4cs.LG

Requests of a Feather Must Flock Together: Batch Size vs. Prefix Homogeneity in LLM Inference

Auto-regressive token generation in large language models is memory-bound because it requires "attending to" key and value tensors (KV cache) of all previous tokens. Prior work aims to improve the efficiency of this decode process by batching multiple requests together, and maximizing batch size subject to GPU memory constraints. The key observation of our work is that with prefix-sharing workloads, smaller, prefix-homogeneous batches - where all requests share a common prefix - can achieve higher decode throughput than larger, heterogeneous batches, due to better spatial and temporal locality during KV cache accesses. However, current prefix-aware schedulers maximize prefix reuse within a batch only to reduce KV cache memory footprint, but do not stop batch formation at smaller homogeneous batches that could have performed better. Further, we show that shared prefix detection in existing schedulers relies on radix-tree traversals, incurring substantial CPU overhead that is often comparable to GPU execution time. This paper presents Feather, a prefix-aware scheduler that uses reinforcement learning (RL) to learn the optimal tradeoff between batch size and prefix homogeneity. We also introduce Chunked Hash Tree (CHT), a lightweight data structure that enables fast prefix detection and efficient request selection for the RL scheduler, avoiding expensive tree traversals. We integrate Feather into vLLM and SGLang, and our evaluation shows that Feather achieves 2-10$\times$ higher end-to-end throughput as compared to existing schedulers by reducing the total number of KV cache accesses, while doing no worse than the status quo when there is insufficient prefix sharing.

arXiv:2605.05179v3cs.LG

Estimating the expected output of wide random MLPs more efficiently than sampling

By far the most common way to estimate an expected loss in machine learning is to draw samples, compute the loss on each one, and take the empirical average. However, sampling is not necessarily optimal. Given an MLP at initialization, we show how to estimate its expected output over Gaussian inputs without running samples through the network at all. Instead, we produce approximate representations of the distributions of activations at each layer, leveraging tools such as cumulants and Hermite expansions. We show both theoretically and empirically that for sufficiently wide networks, our estimator achieves a target mean squared error using substantially fewer FLOPs than Monte Carlo sampling. We find moreover that our methods perform particularly well at estimating the probabilities of rare events, and additionally demonstrate how they can be used for model training. Together, these findings suggest a path to producing models with a greatly reduced probability of catastrophic tail risks.

arXiv:2605.03830v2cs.CV

Identity-Consistent Multi-Pose Generation of Contactless Fingerprints

Contactless fingerprint recognition has gained increasing attention due to its advantages in hygiene and acquisition flexibility. However, the absence of physical contact constraints introduces severe nonlinear geometric distortions caused by free finger poses in 3D space, resulting in a substantial cross-modal domain gap between contactless and conventional contact-based fingerprints. Existing solutions largely rely on explicit geometric correction or image enhancement, which are fragile under extreme pose variations. In this paper, we propose Identity-Consistent Multi-Pose Generation of Contactless Fingerprints (IMPOSE), a physics-inspired framework that synthesizes identity-preserving, multi-pose contactless fingerprint samples to empower recognition models. IMPOSE consists of three stages: (1) rolled fingerprint identity generation via latent diffusion with discrete codebook representations, (2) cross-modal translation from rolled to contactless modality guided by Sauvola-based local adaptive binarization as an identity anchor, and (3) physics-based multi-pose simulation through 3D finger model texture mapping and projection. The generated samples maintain strict identity consistency at the ridge topology level and spatial alignment with standard fingerprint coordinate space. Extensive experiments on the UWA and PolyU CL2CB databases demonstrate that fine-tuning fixed-length dense descriptors (FDD) with IMPOSE-synthesized data achieves state-of-the-art cross-modal matching, reducing EER to 8.74% on UWA and 2.26% on PolyU CL2CB. Synthetic data also yields consistent gains across mainstream representations including DeepPrint and AFRNet, and the hybrid strategy combining synthetic and real data achieves the best overall results. The code and generated samples are available at https://github.com/Yu-Yy/IMPOSE.

arXiv:2604.27613v2cs.LG

AMGenC: Generating Charge Balanced Amorphous Materials

Amorphous (disordered) materials require simulation cells with at least hundreds to thousands of atoms to accurately capture their diverse local atomic environments, compared to crystalline materials that can be described by unit cells containing a few to hundreds of atoms. To advance the design of amorphous materials with desired properties and facilitate the exploration of their vast design space, generative inverse design has emerged as a promising approach. It aims to directly output materials with properties closely aligned with the desired ones using probabilistic generative models conditioned on desired properties, which can be more resource efficient than the traditional trial-and-error approach. However, due to the inherent stochasticity of probabilistic generative models, when element assignments are unconstrained, a large portion of generated materials may be charge unbalanced, and existing methods either fix the composition in advance or discard charge unbalanced samples after generation. In this work, we propose AMGenC, a new generative inverse design method for amorphous materials that can guarantee the generation of formally charge balanced samples, with minimal additional computational overhead and without sacrificing inverse design accuracy. AMGenC achieves this through an element noise that gives the generation process a starting point centered around charge balance, and the combination of a per-step soft projection and a final discrete projection for steering the elements toward exact charge balance throughout the generation. We perform extensive experiments on two amorphous materials datasets. Experimental results provide evidence that AMGenC achieves its design goal.

arXiv:2604.26707v2cs.CV

CurEvo: Curriculum-Guided Self-Evolution for Video Understanding

Recent self-evolution frameworks for video understanding have demonstrated the potential for autonomous improvement from unlabeled videos. However, existing approaches often suffer from weakly controlled optimization and capability progression because they lack structured guidance for generating, evaluating, and reusing supervision throughout the iterative learning process. We present CurEvo, a curriculum-guided self-evolution framework that organizes this loop as a competence-aware progression. Curriculum-Guided Self-Evolution maintains a competence-aware curriculum state and cumulative training pool across iterations. Within this loop, Multi-Dimensional Question Generation supplies complementary supervision, from directly verifiable structural questions for perception to type-conditioned questions for recognition and understanding. Type-Adaptive Evaluation separately assesses question quality and answer reliability with criteria matched to each type's evidence; its scores determine which pairs enter the pool and how the next curriculum state is updated. Across seven backbones and four VideoQA benchmarks, CurEvo consistently improves benchmark accuracy and benchmark semantic score. For Qwen2.5-VL(7B), accuracy rises from 67.84% to 69.91% on ActivityNet-QA and from 76.94% to 79.61% on NExT-QA. CurEvo thus establishes self-generated supervision as a stable route to autonomous video understanding without task-specific human QA annotations during self-evolution.

arXiv:2604.25076v2cs.LG

Zero Shot Coordination for Sparse Reward Tasks with Diverse Reward Shapings

Many Multi-Agent Reinforcement Learning (MARL) agents fail to adapt properly to cooperating with agents trained with the same objectives but different seeds, algorithms, or other training differences. This is the problem of Zero-Shot Coordination (ZSC), which focuses on training agents to cooperate well with unknown agents. ZSC has been studied for a variety of tabular cases and simple games such as Hanabi, achieving excellent results. However, existing solutions to ZSC only consider identical rewards for your trained agents and all future partners. This is not realistic for the trained agents, as they do not consider the problem of cooperating with agents that have identical sparse objectives but shape the rewards for those objectives in different manner. To address this issue, we show how to train an ensemble of methods using randomized reward shapings chosen using 4 selection algorithms. Experiments done on the Overcooked environment demonstrate consistent improvements of 62.2%-119.2% in sparse reward over baseline ZSC algorithms when playing with agents that have identical sparse rewards but different reward shapings.

arXiv:2604.24749v5cs.LG

The Optimal Sample Complexity of Multiclass and List Learning

While the optimal sample complexity of binary classification in terms of the VC dimension is well-established, determining the optimal sample complexity of multiclass classification had remained open. The appropriate complexity parameter for multiclass classification is the DS dimension, and despite significant efforts, a gap of $\sqrt{\text{DS}}$ had persisted between the upper and lower bounds on sample complexity. Recent work by Hanneke et al. (2026) shows a novel algebraic characterization of multiclass hypothesis classes in terms of their DS dimension. Building up on this, we show that the maximum hypergraph density of any multiclass hypothesis class is upper-bounded by its DS dimension. This proves a longstanding conjecture of Daniely and Shalev-Shwartz (2014). As a consequence, we determine the optimal dependence of the sample complexity on the DS dimension for multiclass as well as list learning.

arXiv:2604.23952v3stat.ML

Score-Based Modeling of Effective Langevin Dynamics with State-Dependent Mobility

In a previous work [L. T. Giorgini, Phys. Rev. E 114, L012102 (2026)], we introduced stochastic reduced-order models that reproduce by construction the stationary density and short-time correlations of the observed variables, using a constant mobility matrix and the score, the gradient of the logarithm of this density. Here we generalize the framework to state-dependent mobilities, calibrated so that the model also reproduces the finite-lag correlation functions of chosen target observables. The calibration uses exact finite-lag forms of the generalized fluctuation-dissipation theorem, which express time derivatives of lagged correlations as averages over observed pairs of states and are linear in the mobility. When the mobility is expanded on a basis of known fields, its coefficients follow from a least-squares problem involving only the stationary score, and the observables and basis are selected by data-driven diagnostics. Otherwise, a neural-network mobility is calibrated through the conditional score of the finite-time transition density. We validate the approach analytically on a Cox-Ingersoll-Ross process and numerically on a two-dimensional nonequilibrium diffusion with multiplicative noise and a 36-dimensional soft-spin Landau-Lifshitz chain. Both methods reproduce the stationary statistics and the target correlations accurately. Because the identities can be integrated between consecutive lags, so that no time derivatives need to be estimated, the recovered drift and diffusion are much less sensitive to coarse sampling than Kramers-Moyal estimates, which require short sampling intervals. Calibration requires no simulation of the model and applies to high-dimensional systems.