Reinforcement Learning with Verifiable Rewards (RLVR) has substantially improved mathematical reasoning, yet progress on hard prompts often stalls: a prompt can remain unsolved for many training steps before abruptly transitioning to solved--a discrete "zero-to-one" jump whose underlying dynamics and steerability remain poorly understood. We investigate two questions: what drives this transition, and can it be steered? Through event-based analysis measuring the time-to-first-positive (TTFP), we find that negative rollouts preceding successful transitions consistently retain higher early-stage generation entropy, distinguishing near-solvable failures from dead ends. This signal motivates NAPO (Negative-Augmented Policy Optimization), a method integrated with Group Relative Policy Optimization (GRPO) that ranks negatives in unsolved groups via an early-branching driver and assigns graded advantages, injecting informative gradients precisely where standard group-relative updates vanish. Under fixed rollout budgets, NAPO accelerates early solution discovery and improves Pass@256 by 10.0 points on AIME24 and 6.6 points on AIME2025, demonstrating that zero-to-one transitions are diagnosable and steerable through driver-guided negative learning. Our code is available at https://github.com/jameszhu-pixel-time/Z2O.
Multimodal discrete diffusion language models (dLLMs) generate responses through iterative mask prediction and offer a promising alternative to autoregressive large vision-language models. However, they still suffer from hallucination, producing outputs that are plausible under language priors but insufficiently grounded in the image. We argue that hallucination in multimodal dLLMs is not only a token prediction problem, but also a premature commitment problem: during iterative denoising, some masked positions are unmasked too early because their confidence is driven more by language priors than by visual evidence, and these early commitments can bias later generation. To address this issue, we propose Visual Gain Ordering (VIGOR), a training-free and plug-and-play inference strategy that reorders masked positions according to counterfactual visual evidence. At each denoising step, VIGOR compares the confidence of each candidate token under the original image-conditioned pass and an additional visual-ablation pass, and prioritizes positions whose predictions are more strongly supported by the image. Experiments on LLaDA-V and Lumina-DiMOO show that VIGOR consistently reduces hallucination and improves multimodal reasoning performance across benchmarks. Code is available at https://github.com/THU-BPM/Vigor.
Counterfactual explanations (CE) explain model decisions by identifying input modifications that lead to different predictions. Most existing methods operate at the instance level. Distributional Counterfactual Explanations (DCE) extend this setting by optimizing an optimal transport objective that balances proximity to a factual input distribution and alignment to a target output distribution, with statistical certification via chance constrained bounds. However, DCE relies on gradient based optimization, while many real-world tabular pipelines are dominated by non-differentiable models. We propose DISCOVER, a model-agnostic solver for distributional counterfactual explanations. DISCOVER preserves the original DCE objective and certification while replacing gradient descent with a budgeted propose-and-select search paradigm. It exploits a sample-wise decomposition of the transport objective to compute per-row impact scores and enforce a top-k intervention budget, focusing edits on the most influential samples. To guide candidate generation without predictor gradients, DISCOVER introduces an OT-guided cone sampling primitive driven by input-side transport geometry. Experiments on multiple tabular datasets demonstrate strong joint alignment of input and output distributions, extending distributional counterfactual reasoning to modern black box learning pipelines. A code repository is available at: https://github.com/VALHALLA9/Discover.
프론티어 대규모 언어 모델(LLM)의 안전성(safety) 평가는 유해 생성을 분석 대상이라기보다 공격 결과로 주로 다뤄 왔다. 그 결과 모델 오동작 시 생성되는 유해 출력에 대해서는 알려진 바가 거의 없으며, 이는 부분적으로 프론티어 LLM 오동작에 대한 대규모·고품질 수집물을 구하기 어렵기 때문이다. 이 공백을 메우기 위해 우리는 HarmProfile을 제안한다. HarmProfile은 다양한 유해 카테고리와 모델(model) 패밀리에 걸친 모델 오동작을 수집하고, 그로부터 도출된 유해 출력 분포를 모델 수준 리스크 프로파일로 정의하는 콘텐츠 중심 벤치마크 데이터셋(dataset)이다. 전제는 발화 코퍼스로부터 언어적 행동을 특성화할 수 있듯이, 모델 리스크는 안전성 실패의 내용·심각도·변이로부터 특성화할 수 있다는 것이다. HarmProfile은 13개 모델 패밀리에 걸친 23개 프론티어 LLM으로부터 8만 개 이상의 검증된 아티팩트를 포함하며, 15개 유해 카테고리와 57개 하위 카테고리로 구성된다. 이 코퍼스를 이용해 우리는 프론티어 LLM이 대규모로 유해 콘텐츠를 안정적으로 생성하면서도 서로 다른 리스크 프로파일을 보인다는 것을 발견한다. 유해성과 다양성 모두 모델 역량과 함께 증가하며, 이는 프론티어 LLM이 안전해 보일 수 있으나 정렬(alignment) 표면 아래에 점점 더 위험한 지식을 품고 있을 수 있음을 시사한다. 소스 코드는 https://github.com/fresh-ma/HarmProfile 에서 이용할 수 있다.
대규모 언어 모델(LLM)의 테스트-타임 스케일링(test-time scaling, TTS) 전략은 주로 희소한 결과 보상(outcome reward)을 사용한 강화학습 또는 정적 프로세스 보상 모델(PRM)이 안내하는 검색 기반 방법에 크게 의존한다. 그러나 결과 기반 강화학습은 학습 불안정성과 샘플 비효율성을 자주 겪으며, 정적 PRM은 값비싼 단계별 감독이 필요하고 분포 이동(distributional shift)으로 인한 보상 해킹(reward hacking)에 취약하다. 본 논문에서는 적대적 역강화학습(Adversarial Inverse Reinforcement Learning, AIRL)과 Group Relative Policy Optimization(GRPO)을 통합한 프레임워크 AIRL-S를 제안한다. 참조 궤적(reference trajectories)로부터 직접 밀집(dense)한 단계별 보상 모델을 추론함으로써 AIRL-S는 라벨이 붙은 프로세스 데이터에 대한 의존을 없애고, 동일한 학습된 PRM을 학습 신호와 검색 기반 TTS의 검증자(verifier)로 모두 사용한다. 수학, 과학, 코드 생성 분야 8개 벤치마크에 걸친 광범위한 평가에서 우리의 정책 모델(policy model)은 기본 모델 대비 평균 성능을 9% 향상시켜 GPT-4o에 필적한다. 또한 AIRL과 GRPO 목적 함수가 서로를 어떻게 보완하는지, 학습된 PRM이 생성기와 검색 알고리즘 간에 어떻게 전이(transfer)되는지 분석하여, 복잡한 추론 작업에서 테스트-타임 연산을 스케일링하는 견고하고 비용 효율적인 방법론을 확립한다.
대규모 언어 모델(LLM)이 장문 생성에 점점 더 많이 쓰이면서, 장문 출력을 신뢰할 수 있게 평가하는 일은 중요한 과제가 되었다. LLM-as-a-judge는 인간 평가에 대한 확장 가능한 대안을 제공하지만, 장문 출력 평가에서의 신뢰성은 아직 충분히 검토되지 않았다. 기존 메타 평가(meta-evaluation) 벤치마크는 주로 단문(short-form) 출력에 초점을 맞춘다. 단문 평가와 비교하면, 장문 평가는 단순히 출력 길이의 문제가 아니며, 종종 판정자(judge)가 전체 구성, 과제 관련 범위와 깊이, 섹션 간 일관성, 시나리오별 품질 기준에 대한 더 복잡한 문서 수준 평가를 해야 한다. 본 연구에서는 다양한 실제 시나리오와 판정 프로토콜에 걸친 장문 출력에 대한 LLM 판정자 평가를 위한 포괄적 벤치마크인 LongJudgeBench를 제안한다. 여러 베이스 모델과 판정 설정을 아우르는 폭넓은 LLM 판정자들을 체계적으로 평가하였다. 결과는 상당한 신뢰성 격차를 보여준다. 현재 LLM 판정자는 시나리오 간 불안정하며, rubric(평가 기준)이나 참조(reference)는 도움이 되지만 항상 충분하지는 않다. LongJudgeBench가 더 견고하고 문맥 인지적이며 사람과 정렬(alignment)된 LLM-as-a-judge 방법에 대한 후속 연구를 지원하기를 바란다. 코드는 https://github.com/cjj826/LongJudgeBench 에서 이용할 수 있다.
온-정책(on-policy) 증류(OPD)는 언어 모델 사후 학습을 위한 조밀한 토큰별 감독 신호를 제공하지만, 그 효과는 교사(teacher) 모델의 품질로 인해 제한됩니다. 외부 교사 모델은 분포 불일치를 겪으며, 특권적 조건화(privileged conditioning)를 사용하는 자기 증류는 문맥 내 학습(ICL) 능력에 의해 한계가 있습니다. 본 논문에서는 모델 자체의 RLVR 학습 궤적으로부터 직접 합성 교사를 구축하는 RISE(Recursive Improvement via Self-Extrapolating Policy Distillation)를 제안합니다. 매개변수 공간 또는 출력 로짓 공간에서 현재 체크포인트와 후행 앵커 사이의 변위를 외삽함으로써, RISE는 어떠한 외부 모델이나 특권적 조건화 없이도 희소한 결과 유도 매개변수 업데이트를 조밀한 토큰 수준의 목표로 변환합니다. RISE는 상호 보완적인 루프에서 RLVR과 OPD를 결합합니다. 결과 보상은 외삽을 올바른 추론 방향으로 유도하고(grounding), 외삽된 교사는 토큰 수준의 결정을 정제합니다. 또한 학생 모델이 개선됨에 따라 교사 모델이 매 반복마다 갱신되므로, 증류는 일회성 압축 단계가 아닌 재귀적 개선 메커니즘이 됩니다. 수학적 추론, 다중 영역 STEM, 코드 생성 및 멀티턴 에이전트 태스크를 포괄하는 실험을 통해, RISE가 모든 환경에서 RLVR 전용 학습 및 온-정책 자기 증류를 능가함을 보입니다.
최근 대규모 언어 모델(LLM)은 사람이 읽을 수 있는 추론 생성 능력에 힘입어 의사결정 에이전트로서 교통 신호 제어(TSC)에 도입되고 있습니다. 그러나 기존의 LLM 기반 TSC 기법들은 최종 결과만을 기반으로 최적화하며 타당한 추론 단계와 결함이 있는 추론 단계를 구분하지 못합니다. 이로 인해 유용한 단계와 오해의 소지가 있는 단계가 함께 업데이트되어 모델이 효과적인 추론을 학습하는 데 지장을 줍니다. 이러한 한계를 극복하기 위해, 본 논문에서는 신호 의사결정을 검증 가능한 의미론적 단계로 분해하는 LLM 기반 프레임워크인 ProcessLight를 제안합니다. 나아가 ProcessLight를 바탕으로 단계 수준의 신용 할당(credit assignment)을 통해 구조화된 추론 과정을 최적화하는 새로운 강화학습 프레임워크인 STeP-PO(Step-wise Traffic Process Policy Optimization)를 개발합니다. 구체적으로 STeP-PO는 단계 품질 점수(step quality score)를 사용하여 국소적 추론 품질을 평가하고, 단계 중요도(step importance)를 통해 각 단계가 최종 행동에 미치는 영향을 측정한 뒤 의미론적 단계 트리 구조상에서 단계 수준의 이점(advantage)을 할당합니다. 이렇게 도출된 단계 수준의 이점은 추론 토큰으로 전파되어 결과 기반 보상만을 넘어서는 미세한 정책 최적화를 가능하게 합니다. 여러 실제 데이터셋을 활용한 광범위한 실험을 통해 제안 방법의 우수성을 입증합니다. 코드는 https://github.com/wenzhaoabc/processlight 에서 확인할 수 있습니다.
Facial image manipulation has achieved great progress in recent years. However, previous methods either operate on a predefined set of face attributes or leave users little freedom to interactively manipulate images. To overcome these drawbacks, we propose a novel framework termed MaskGAN, enabling diverse and interactive face manipulation. Our key insight is that semantic masks serve as a suitable intermediate representation for flexible face manipulation with fidelity preservation. MaskGAN has two main components: 1) Dense Mapping Network (DMN) and 2) Editing Behavior Simulated Training (EBST). Specifically, DMN learns style mapping between a free-form user modified mask and a target image, enabling diverse generation results. EBST models the user editing behavior on the source mask, making the overall framework more robust to various manipulated inputs. Specifically, it introduces dual-editing consistency as the auxiliary supervision signal. To facilitate extensive studies, we construct a large-scale high-resolution face dataset with fine-grained mask annotations named CelebAMask-HQ. MaskGAN is comprehensively evaluated on two challenging tasks: attribute transfer and style copy, demonstrating superior performance over other state-of-the-art methods. The code, models, and dataset are available at https://github.com/switchablenorms/CelebAMask-HQ.
We propose a framework to continuously learn object-centric representations for visual learning and understanding. Existing object-centric representations either rely on supervisions that individualize objects in the scene, or perform unsupervised disentanglement that can hardly deal with complex scenes in the real world. To mitigate the annotation burden and relax the constraints on the statistical complexity of the data, our method leverages interactions to effectively sample diverse variations of an object and the corresponding training signals while learning the object-centric representations. Throughout learning, objects are streamed one by one in random order with unknown identities, and are associated with latent codes that can synthesize discriminative weights for each object through a convolutional hypernetwork. Moreover, re-identification of learned objects and forgetting prevention are employed to make the learning process efficient and robust. We perform an extensive study of the key features of the proposed framework and analyze the characteristics of the learned representations. Furthermore, we demonstrate the capability of the proposed framework in learning representations that can improve label efficiency in downstream tasks. Our code and trained models are made publicly available at: https://github.com/pptrick/Object-Pursuit.
We propose a probabilistic shape completion method extended to the continuous geometry of large-scale 3D scenes. Real-world scans of 3D scenes suffer from a considerable amount of missing data cluttered with unsegmented objects. The problem of shape completion is inherently ill-posed, and high-quality result requires scalable solutions that consider multiple possible outcomes. We employ the Generative Cellular Automata that learns the multi-modal distribution and transform the formulation to process large-scale continuous geometry. The local continuous shape is incrementally generated as a sparse voxel embedding, which contains the latent code for each occupied cell. We formally derive that our training objective for the sparse voxel embedding maximizes the variational lower bound of the complete shape distribution and therefore our progressive generation constitutes a valid generative model. Experiments show that our model successfully generates diverse plausible scenes faithful to the input, especially when the input suffers from a significant amount of missing data. We also demonstrate that our approach outperforms deterministic models even in less ambiguous cases with a small amount of missing data, which infers that probabilistic formulation is crucial for high-quality geometry completion on input scans exhibiting any levels of completeness.
Video snapshot compressive imaging (SCI) captures multiple sequential video frames by a single measurement using the idea of computational imaging. The underlying principle is to modulate high-speed frames through different masks and these modulated frames are summed to a single measurement captured by a low-speed 2D sensor (dubbed optical encoder); following this, algorithms are employed to reconstruct the desired high-speed frames (dubbed software decoder) if needed. In this paper, we consider the reconstruction algorithm in video SCI, i.e., recovering a series of video frames from a compressed measurement. Specifically, we propose a Spatial-Temporal transFormer (STFormer) to exploit the correlation in both spatial and temporal domains. STFormer network is composed of a token generation block, a video reconstruction block, and these two blocks are connected by a series of STFormer blocks. Each STFormer block consists of a spatial self-attention branch, a temporal self-attention branch and the outputs of these two branches are integrated by a fusion network. Extensive results on both simulated and real data demonstrate the state-of-the-art performance of STFormer. The code and models are publicly available at https://github.com/ucaswangls/STFormer.git
The artistic style within a painting is the means of expression, which includes not only the painting material, colors, and brushstrokes, but also the high-level attributes including semantic elements, object shapes, etc. Previous arbitrary example-guided artistic image generation methods often fail to control shape changes or convey elements. The pre-trained text-to-image synthesis diffusion probabilistic models have achieved remarkable quality, but it often requires extensive textual descriptions to accurately portray attributes of a particular painting. We believe that the uniqueness of an artwork lies precisely in the fact that it cannot be adequately explained with normal language. Our key idea is to learn artistic style directly from a single painting and then guide the synthesis without providing complex textual descriptions. Specifically, we assume style as a learnable textual description of a painting. We propose an inversion-based style transfer method (InST), which can efficiently and accurately learn the key information of an image, thus capturing and transferring the artistic style of a painting. We demonstrate the quality and efficiency of our method on numerous paintings of various artists and styles. Code and models are available at https://github.com/zyxElsa/InST.
In this work, we are dedicated to text-guided image generation and propose a novel framework, i.e., CLIP2GAN, by leveraging CLIP model and StyleGAN. The key idea of our CLIP2GAN is to bridge the output feature embedding space of CLIP and the input latent space of StyleGAN, which is realized by introducing a mapping network. In the training stage, we encode an image with CLIP and map the output feature to a latent code, which is further used to reconstruct the image. In this way, the mapping network is optimized in a self-supervised learning way. In the inference stage, since CLIP can embed both image and text into a shared feature embedding space, we replace CLIP image encoder in the training architecture with CLIP text encoder, while keeping the following mapping network as well as StyleGAN model. As a result, we can flexibly input a text description to generate an image. Moreover, by simply adding mapped text features of an attribute to a mapped CLIP image feature, we can effectively edit the attribute to the image. Extensive experiments demonstrate the superior performance of our proposed CLIP2GAN compared to previous methods.
The increasing privacy concerns on personal private text data promote the development of federated learning (FL) in recent years. However, the existing studies on applying FL in NLP are not suitable to coordinate participants with heterogeneous or private learning objectives. In this study, we further broaden the application scope of FL in NLP by proposing an Assign-Then-Contrast (denoted as ATC) framework, which enables clients with heterogeneous NLP tasks to construct an FL course and learn useful knowledge from each other. Specifically, the clients are suggested to first perform local training with the unified tasks assigned by the server rather than using their own learning objectives, which is called the Assign training stage. After that, in the Contrast training stage, clients train with different local learning objectives and exchange knowledge with other clients who contribute consistent and useful model updates. We conduct extensive experiments on six widely-used datasets covering both Natural Language Understanding (NLU) and Natural Language Generation (NLG) tasks, and the proposed ATC framework achieves significant improvements compared with various baseline methods. The source code is available at \url{https://github.com/alibaba/FederatedScope/tree/master/federatedscope/nlp/hetero_tasks}.
With the increase of the adoption of blockchain technology in providing decentralized solutions to various problems, smart contracts have become more popular to the point that billions of US Dollars are currently exchanged every day through such technology. Meanwhile, various vulnerabilities in smart contracts have been exploited by attackers to steal cryptocurrencies worth millions of dollars. The automatic detection of smart contract vulnerabilities therefore is an essential research problem. Existing solutions to this problem particularly rely on human experts to define features or different rules to detect vulnerabilities. However, this often causes many vulnerabilities to be ignored, and they are inefficient in detecting new vulnerabilities. In this study, to overcome such challenges, we propose a framework to automatically detect vulnerabilities in smart contracts on the blockchain. More specifically, first, we utilize novel feature vector generation techniques from bytecode of smart contract since the source code of smart contracts are rarely available in public. Next, the collected vectors are fed into our novel metric learning-based deep neural network(DNN) to get the detection result. We conduct comprehensive experiments on large-scale benchmarks, and the quantitative results demonstrate the effectiveness and efficiency of our approach.
Score-based Generative Models (SGMs) have demonstrated exceptional synthesis outcomes across various tasks. However, the current design landscape of the forward diffusion process remains largely untapped and often relies on physical heuristics or simplifying assumptions. Utilizing insights from the development of scalable Bayesian posterior samplers, we present a complete recipe for formulating forward processes in SGMs, ensuring convergence to the desired target distribution. Our approach reveals that several existing SGMs can be seen as specific manifestations of our framework. Building upon this method, we introduce Phase Space Langevin Diffusion (PSLD), which relies on score-based modeling within an augmented space enriched by auxiliary variables akin to physical phase space. Empirical results exhibit the superior sample quality and improved speed-quality trade-off of PSLD compared to various competing approaches on established image synthesis benchmarks. Remarkably, PSLD achieves sample quality akin to state-of-the-art SGMs (FID: 2.10 for unconditional CIFAR-10 generation). Lastly, we demonstrate the applicability of PSLD in conditional synthesis using pre-trained score networks, offering an appealing alternative as an SGM backbone for future advancements. Code and model checkpoints can be accessed at \url{https://github.com/mandt-lab/PSLD}.
This work focuses on training a single visual relationship detector predicting over the union of label spaces from multiple datasets. Merging labels spanning different datasets could be challenging due to inconsistent taxonomies. The issue is exacerbated in visual relationship detection when second-order visual semantics are introduced between pairs of objects. To address this challenge, we propose UniVRD, a novel bottom-up method for Unified Visual Relationship Detection by leveraging vision and language models (VLMs). VLMs provide well-aligned image and text embeddings, where similar relationships are optimized to be close to each other for semantic unification. Our bottom-up design enables the model to enjoy the benefit of training with both object detection and visual relationship datasets. Empirical results on both human-object interaction detection and scene-graph generation demonstrate the competitive performance of our model. UniVRD achieves 38.07 mAP on HICO-DET, outperforming the current best bottom-up HOI detector by 14.26 mAP. More importantly, we show that our unified detector performs as well as dataset-specific models in mAP, and achieves further improvements when we scale up the model. Our code will be made publicly available on GitHub.
This paper explores interactive facial image editing via dialogue and introduces the ChatEdit benchmark dataset for evaluating image editing and conversation abilities in this context. ChatEdit is constructed from the CelebA-HQ dataset, incorporating annotated multi-turn dialogues corresponding to user edit requests on the images. The dataset is challenging, as it requires the system to dynamically track user requests, edit images, and generate appropriate responses. Accordingly, we propose three benchmark tasks: (i) user edit request tracking, (ii) image editing, and (iii) response generation. We present a novel baseline framework that integrates a dialogue module for both tracking user requests and generating responses and an image editing module for image editing. Unlike previous approaches, our framework directly tracks user edit requests from the entire dialogue history up to the current turn and modifies the original image rather than adjusting the previous turn's output, thereby reducing error accumulation and preventing attribute forgetfulness. Extensive experiments on the ChatEdit dataset underline our framework's superior performance against prior models, while also highlighting potential room for further research. We will release the code and data publicly to facilitate advancements in complex interactive facial image editing.
Language model probing is often used to test specific capabilities of models. However, conclusions from such studies may be limited when the probing benchmarks are small and lack statistical power. In this work, we introduce new, larger datasets for negation (NEG-1500-SIMP) and role reversal (ROLE-1500) inspired by psycholinguistic studies. We dramatically extend existing NEG-136 and ROLE-88 benchmarks using GPT3, increasing their size from 18 and 44 sentence pairs to 750 each. We also create another version of extended negation dataset (NEG-1500-SIMP-TEMP), created using template-based generation. It consists of 770 sentence pairs. We evaluate 22 models on the extended datasets, seeing model performance dip 20-57% compared to the original smaller benchmarks. We observe high levels of negation sensitivity in models like BERT and ALBERT demonstrating that previous findings might have been skewed due to smaller test sets. Finally, we observe that while GPT3 has generated all the examples in ROLE-1500 is only able to solve 24.6% of them during probing. The datasets and code are available on $\href{https://github.com/text-machine-lab/extending_psycholinguistic_dataset}{Github}$.