PyTorchKR Papers

“Code Generation”

총 10,140편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2406.04662공식 코드 · Papers with Code

Evaluating and Mitigating IP Infringement in Visual Generative AI

The popularity of visual generative AI models like DALL-E 3, Stable Diffusion XL, Stable Video Diffusion, and Sora has been increasing. Through extensive evaluation, we discovered that the state-of-the-art visual generative models can generate content that bears a striking resemblance to characters protected by intellectual property rights held by major entertainment companies (such as Sony, Marvel, and Nintendo), which raises potential legal concerns. This happens when the input prompt contains the character's name or even just descriptive details about their characteristics. To mitigate such IP infringement problems, we also propose a defense method against it. In detail, we develop a revised generation paradigm that can identify potentially infringing generated content and prevent IP infringement by utilizing guidance techniques during the diffusion process. It has the capability to recognize generated content that may be infringing on intellectual property rights, and mitigate such infringement by employing guidance methods throughout the diffusion process without retrain or fine-tune the pretrained models. Experiments on well-known character IPs like Spider-Man, Iron Man, and Superman demonstrate the effectiveness of the proposed defense method. Our data and code can be found at https://github.com/ZhentingWang/GAI_IP_Infringement.

arXiv:2407.08136공식 코드 · Papers with Code

EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions

The area of portrait image animation, propelled by audio input, has witnessed notable progress in the generation of lifelike and dynamic portraits. Conventional methods are limited to utilizing either audios or facial key points to drive images into videos, while they can yield satisfactory results, certain issues exist. For instance, methods driven solely by audios can be unstable at times due to the relatively weaker audio signal, while methods driven exclusively by facial key points, although more stable in driving, can result in unnatural outcomes due to the excessive control of key point information. In addressing the previously mentioned challenges, in this paper, we introduce a novel approach which we named EchoMimic. EchoMimic is concurrently trained using both audios and facial landmarks. Through the implementation of a novel training strategy, EchoMimic is capable of generating portrait videos not only by audios and facial landmarks individually, but also by a combination of both audios and selected facial landmarks. EchoMimic has been comprehensively compared with alternative algorithms across various public datasets and our collected dataset, showcasing superior performance in both quantitative and qualitative evaluations. Additional visualization and access to the source code can be located on the EchoMimic project page.

arXiv:2407.08931공식 코드 · Papers with Code

Global-Local Collaborative Inference with LLM for Lidar-Based Open-Vocabulary Detection

Open-Vocabulary Detection (OVD) is the task of detecting all interesting objects in a given scene without predefined object classes. Extensive work has been done to deal with the OVD for 2D RGB images, but the exploration of 3D OVD is still limited. Intuitively, lidar point clouds provide 3D information, both object level and scene level, to generate trustful detection results. However, previous lidar-based OVD methods only focus on the usage of object-level features, ignoring the essence of scene-level information. In this paper, we propose a Global-Local Collaborative Scheme (GLIS) for the lidar-based OVD task, which contains a local branch to generate object-level detection result and a global branch to obtain scene-level global feature. With the global-local information, a Large Language Model (LLM) is applied for chain-of-thought inference, and the detection result can be refined accordingly. We further propose Reflected Pseudo Labels Generation (RPLG) to generate high-quality pseudo labels for supervision and Background-Aware Object Localization (BAOL) to select precise object proposals. Extensive experiments on ScanNetV2 and SUN RGB-D demonstrate the superiority of our methods. Code is released at https://github.com/GradiusTwinbee/GLIS.

arXiv:2408.13909

LowCLIP: Adapting the CLIP Model Architecture for Low-Resource Languages in Multimodal Image Retrieval Task

This research explores the development of multimodal vision-language models for image retrieval in low-resource languages, specifically Azerbaijani. Existing vision-language models primarily support high-resource languages, and fine-tuning them remains computationally demanding. To address challenges in vision-language retrieval for low-resource languages, we integrated the CLIP model architecture and employed several techniques to balance computational efficiency with performance. These techniques include synthetic data generation through machine translation, image augmentation, and further training the attention mechanisms of transformer-based models with domain-specific data. We integrated Multilingual BERT as a text encoder with image encoders like ResNet50, EfficientNet0, Vision Transformer (ViT), and Tiny Swin Transformer. Our study found that models like EfficientNet0 and Tiny Swin Transformer perform best on the datasets they were trained on, such as COCO, Flickr30k, and Flickr8k. Augmentation techniques boosted EfficientNet0 MAP on Flickr30k from 0.84 to 0.87 and ResNet50 MAP on MSCOCO from 0.70 to 0.80, contributing to a new state of the art in vision-language retrieval. We share our configurations and results to support further research. Code and pre-trained models are available at https://github.com/aliasgerovs/azclip.

arXiv:2408.15548공식 코드 · Papers with Code

ConsistencyTrack: A Robust Multi-Object Tracker with a Generation Strategy of Consistency Model

Multi-object tracking (MOT) is a critical technology in computer vision, designed to detect multiple targets in video sequences and assign each target a unique ID per frame. Existed MOT methods excel at accurately tracking multiple objects in real-time across various scenarios. However, these methods still face challenges such as poor noise resistance and frequent ID switches. In this research, we propose a novel ConsistencyTrack, joint detection and tracking(JDT) framework that formulates detection and association as a denoising diffusion process on perturbed bounding boxes. This progressive denoising strategy significantly improves the model's noise resistance. During the training phase, paired object boxes within two adjacent frames are diffused from ground-truth boxes to a random distribution, and then the model learns to detect and track by reversing this process. In inference, the model refines randomly generated boxes into detection and tracking results through minimal denoising steps. ConsistencyTrack also introduces an innovative target association strategy to address target occlusion. Experiments on the MOT17 and DanceTrack datasets demonstrate that ConsistencyTrack outperforms other compared methods, especially better than DiffusionTrack in inference speed and other performance metrics. Our code is available at https://github.com/Tankowa/ConsistencyTrack.

arXiv:2409.18164공식 코드 · Papers with Code

Data-Prep-Kit: getting your data ready for LLM application development

Data preparation is the first and a very important step towards any Large Language Model (LLM) development. This paper introduces an easy-to-use, extensible, and scale-flexible open-source data preparation toolkit called Data Prep Kit (DPK). DPK is architected and designed to enable users to scale their data preparation to their needs. With DPK they can prepare data on a local machine or effortlessly scale to run on a cluster with thousands of CPU Cores. DPK comes with a highly scalable, yet extensible set of modules that transform natural language and code data. If the user needs additional transforms, they can be easily developed using extensive DPK support for transform creation. These modules can be used independently or pipelined to perform a series of operations. In this paper, we describe DPK architecture and show its performance from a small scale to a very large number of CPUs. The modules from DPK have been used for the preparation of Granite Models [1] [2]. We believe DPK is a valuable contribution to the AI community to easily prepare data to enhance the performance of their LLM models or to fine-tune models with Retrieval-Augmented Generation (RAG).

arXiv:2409.19572공식 코드 · Papers with Code

Mitigating the Negative Impact of Over-association for Conversational Query Production

Conversational query generation aims at producing search queries from dialogue histories, which are then used to retrieve relevant knowledge from a search engine to help knowledge-based dialogue systems. Trained to maximize the likelihood of gold queries, previous models suffer from the data hunger issue, and they tend to both drop important concepts from dialogue histories and generate irrelevant concepts at inference time. We attribute these issues to the over-association phenomenon where a large number of gold queries are indirectly related to the dialogue topics, because annotators may unconsciously perform reasoning with their background knowledge when generating these gold queries. We carefully analyze the negative effects of this phenomenon on pretrained Seq2seq query producers and then propose effective instance-level weighting strategies for training to mitigate these issues from multiple perspectives. Experiments on two benchmarks, Wizard-of-Internet and DuSinc, show that our strategies effectively alleviate the negative effects and lead to significant performance gains (2%-5% across automatic metrics and human evaluation). Further analysis shows that our model selects better concepts from dialogue histories and is 10 times more data efficient than the baseline. The code is available at https://github.com/DeepLearnXMU/QG-OverAsso.

arXiv:2410.23836

Stereo-Talker: Audio-driven 3D Human Synthesis with Prior-Guided Mixture-of-Experts

This paper introduces Stereo-Talker, a novel one-shot audio-driven human video synthesis system that generates 3D talking videos with precise lip synchronization, expressive body gestures, temporally consistent photo-realistic quality, and continuous viewpoint control. The process follows a two-stage approach. In the first stage, the system maps audio input to high-fidelity motion sequences, encompassing upper-body gestures and facial expressions. To enrich motion diversity and authenticity, large language model (LLM) priors are integrated with text-aligned semantic audio features, leveraging LLMs' cross-modal generalization power to enhance motion quality. In the second stage, we improve diffusion-based video generation models by incorporating a prior-guided Mixture-of-Experts (MoE) mechanism: a view-guided MoE focuses on view-specific attributes, while a mask-guided MoE enhances region-based rendering stability. Additionally, a mask prediction module is devised to derive human masks from motion data, enhancing the stability and accuracy of masks and enabling mask guiding during inference. We also introduce a comprehensive human video dataset with 2,203 identities, covering diverse body gestures and detailed annotations, facilitating broad generalization. The code, data, and pre-trained models will be released for research purposes.

arXiv:2411.13852공식 코드 · Papers with Code

Dealing with Synthetic Data Contamination in Online Continual Learning

Image generation has shown remarkable results in generating high-fidelity realistic images, in particular with the advancement of diffusion-based models. However, the prevalence of AI-generated images may have side effects for the machine learning community that are not clearly identified. Meanwhile, the success of deep learning in computer vision is driven by the massive dataset collected on the Internet. The extensive quantity of synthetic data being added to the Internet would become an obstacle for future researchers to collect "clean" datasets without AI-generated content. Prior research has shown that using datasets contaminated by synthetic images may result in performance degradation when used for training. In this paper, we investigate the potential impact of contaminated datasets on Online Continual Learning (CL) research. We experimentally show that contaminated datasets might hinder the training of existing online CL methods. Also, we propose Entropy Selection with Real-synthetic similarity Maximization (ESRM), a method to alleviate the performance deterioration caused by synthetic images when training online CL models. Experiments show that our method can significantly alleviate performance deterioration, especially when the contamination is severe. For reproducibility, the source code of our work is available at https://github.com/maorong-wang/ESRM.

arXiv:2411.16740공식 코드 · Papers with Code

Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documents

Large multimodal models (LMMs) have achieved impressive progress in vision-language understanding, yet they face limitations in real-world applications requiring complex reasoning over a large number of images. Existing benchmarks for multi-image question-answering are limited in scope, each question is paired with only up to 30 images, which does not fully capture the demands of large-scale retrieval tasks encountered in the real-world usages. To reduce these gaps, we introduce two document haystack benchmarks, dubbed DocHaystack and InfoHaystack, designed to evaluate LMM performance on large-scale visual document retrieval and understanding. Additionally, we propose V-RAG, a novel, vision-centric retrieval-augmented generation (RAG) framework that leverages a suite of multimodal vision encoders, each optimized for specific strengths, and a dedicated question-document relevance module. V-RAG sets a new standard, with a 9% and 11% improvement in Recall@1 on the challenging DocHaystack-1000 and InfoHaystack-1000 benchmarks, respectively, compared to the previous best baseline models. Additionally, integrating V-RAG with LMMs enables them to efficiently operate across thousands of images, yielding significant improvements on our DocHaystack and InfoHaystack benchmarks. Our code and datasets are available at https://github.com/Vision-CAIR/dochaystacks

arXiv:2502.07203

Playmate: Flexible Control of Portrait Animation via 3D-Implicit Space Guided Diffusion

Recent diffusion-based talking face generation models have demonstrated impressive potential in synthesizing videos that accurately match a speech audio clip with a given reference identity. However, existing approaches still encounter significant challenges due to uncontrollable factors, such as inaccurate lip-sync, inappropriate head posture and the lack of fine-grained control over facial expressions. In order to introduce more face-guided conditions beyond speech audio clips, a novel two-stage training framework Playmate is proposed to generate more lifelike facial expressions and talking faces. In the first stage, we introduce a decoupled implicit 3D representation along with a meticulously designed motion-decoupled module to facilitate more accurate attribute disentanglement and generate expressive talking videos directly from audio cues. Then, in the second stage, we introduce an emotion-control module to encode emotion control information into the latent space, enabling fine-grained control over emotions and thereby achieving the ability to generate talking videos with desired emotion. Extensive experiments demonstrate that Playmate outperforms existing state-of-the-art methods in terms of video quality and lip-synchronization, and improves flexibility in controlling emotion and head pose. The code will be available at https://playmate111.github.io.

arXiv:2502.18195공식 코드 · Papers with Code

Multi-Perspective Data Augmentation for Few-shot Object Detection

Recent few-shot object detection (FSOD) methods have focused on augmenting synthetic samples for novel classes, show promising results to the rise of diffusion models. However, the diversity of such datasets is often limited in representativeness because they lack awareness of typical and hard samples, especially in the context of foreground and background relationships. To tackle this issue, we propose a Multi-Perspective Data Augmentation (MPAD) framework. In terms of foreground-foreground relationships, we propose in-context learning for object synthesis (ICOS) with bounding box adjustments to enhance the detail and spatial information of synthetic samples. Inspired by the large margin principle, support samples play a vital role in defining class boundaries. Therefore, we design a Harmonic Prompt Aggregation Scheduler (HPAS) to mix prompt embeddings at each time step of the generation process in diffusion models, producing hard novel samples. For foreground-background relationships, we introduce a Background Proposal method (BAP) to sample typical and hard backgrounds. Extensive experiments on multiple FSOD benchmarks demonstrate the effectiveness of our approach. Our framework significantly outperforms traditional methods, achieving an average increase of $17.5\%$ in nAP50 over the baseline on PASCAL VOC. Code is available at https://github.com/nvakhoa/MPAD.

arXiv:2503.00466

Bring Your Own Grasp Generator: Leveraging Robot Grasp Generation for Prosthetic Grasping

One of the most important research challenges in upper-limb prosthetics is enhancing the user-prosthesis communication to closely resemble the experience of a natural limb. As prosthetic devices become more complex, users often struggle to control the additional degrees of freedom. In this context, leveraging shared-autonomy principles can significantly improve the usability of these systems. In this paper, we present a novel eye-in-hand prosthetic grasping system that follows these principles. Our system initiates the approach-to-grasp action based on user's command and automatically configures the DoFs of a prosthetic hand. First, it reconstructs the 3D geometry of the target object without the need of a depth camera. Then, it tracks the hand motion during the approach-to-grasp action and finally selects a candidate grasp configuration according to user's intentions. We deploy our system on the Hannes prosthetic hand and test it on able-bodied subjects and amputees to validate its effectiveness. We compare it with a multi-DoF prosthetic control baseline and find that our method enables faster grasps, while simplifying the user experience. Code and demo videos are available online at https://hsp-iit.github.io/byogg/.

arXiv:2505.05216

Normalize Everything: A Preconditioned Magnitude-Preserving Architecture for Diffusion-Based Speech Enhancement

This paper presents a new framework for diffusion-based speech enhancement. Our method employs a Schroedinger bridge to transform the noisy speech distribution into the clean speech distribution. To stabilize and improve training, we employ time-dependent scalings of the inputs and outputs of the network, known as preconditioning. We consider two skip connection configurations, which either include or omit the current process state in the denoiser's output, enabling the network to predict either environmental noise or clean speech. Each approach leads to improved performance on different speech enhancement metrics. To maintain stable magnitude levels and balance during training, we use a magnitude-preserving network architecture that normalizes all activations and network weights to unit length. Additionally, we propose learning the contribution of the noisy input within each network block for effective input conditioning. After training, we apply a method to approximate different exponential moving average (EMA) profiles and investigate their effects on the speech enhancement performance. In contrast to image generation tasks, where longer EMA lengths often enhance mode coverage, we observe that shorter EMA lengths consistently lead to better performance on standard speech enhancement metrics. Code, audio examples, and checkpoints are available online.

arXiv:2505.18521v2cs.CV공식 코드 · Papers with Code

개선된 Immiscible Diffusion: 혼화성 감소를 통한 확산 학습 가속화

Improved Immiscible Diffusion: Accelerate Diffusion Training by Reducing Its Miscibility

확산 모델의 상당한 학습 비용은 배포를 저해한다. Immiscible Diffusion은 최근 선형 할당을 통해 노이즈 공간에서 확산 궤적 혼합을 줄이면 디노이징을 단순화하여 학습을 가속화할 수 있음을 보였다. 높은 배치 크기와 고차원 환경에서 비효율적인 선형 할당을 넘어 불혼화성 확산을 확장하기 위해, 우리는 이 개념을 임의의 층에서 임의의 구현 방식으로 적용 가능한 보다 광범위한 혼화성 감소 개념으로 정제한다. 구체적으로, 우리는 불혼화성 확산에 관한 디노이징 과정의 전단사(bijective) 성질을 실험적으로 입증하여 생성 다양성이 보존됨을 확인한다. 또한 단계별 분석을 통해 불혼화성이 어떻게 디노이징을 용이하게 하고 효율성을 향상시키는지 보인다. 선형 할당을 넘어, 혼화성을 줄이기 위한 K-최근접 이웃(KNN) 노이즈 선택 및 이미지 스케일링을 포함하는 구현 방법들을 제안하며, 비조건부/조건부 생성, 이미지 편집, 로보틱스 계획을 포함한 다양한 모델 및 태스크에서 최대 4배 이상의 학습 속도 향상을 달성한다. 나아가, 불혼화성에 대한 분석은 최적 수송(OT)이 확산 학습을 향상시키는 방식에 대한 새로운 관점을 제공한다. 궤적 혼화성을 근본적인 병목으로 식별함으로써, 본 연구가 고효율 확산 학습에 대한 향후 연구의 잠재적으로 새로운 방향을 제시한다고 믿는다. 코드는 https://github.com/yhli123/Immiscible-Diffusion 에서 확인할 수 있다.

arXiv:2507.00875v3cs.CL

TransLaw: 홍콩 판례의 전문 번역을 시뮬레이션하는 대규모 데이터셋 및 멀티 에이전트 벤치마크

TransLaw: A Large-Scale Dataset and Multi-Agent Benchmark Simulating Professional Translation of Hong Kong Case Law

홍콩 법원 판결문을 영어에서 번체 중국어로 번역하는 것은 기본법 제8조 및 제9조에 의해 규정되어 있으나, 병렬 리소스의 부족과 법률 용어, 인용 형식, 사법적 스타일에 대한 엄격한 요구 사항으로 인해 여전히 제약이 있습니다. 우리는 홍콩 판례를 위한 첫 번째 대규모 문장 단위 정렬 병렬 말뭉치인 HKCFA Judgment 97-22를 소개하며, 이는 1997년부터 2022년까지의 344개의 전문 번역된 판결문(11,099개 문장 쌍; 210만 토큰)으로 구성되어 있습니다. 이 리소스를 기반으로, 우리는 번역을 단어 수준 표현, 문장 수준 번역, 다차원 검토로 분해하는 TransLaw라는 멀티 에이전트 프레임워크를 제안합니다. 이 프레임워크는 전문 홍콩 법률 용어 데이터베이스, 검색-증강 생성(RAG), 반복적 피드백을 통합하며, 의미적 정렬, 용어, 인용, 스타일을 포괄하는 4차원 전문가 검토를 수행합니다. 13개의 오픈 소스 및 상용 대규모 언어 모델(LLM)에 대한 벤치마킹 결과, TransLaw는 평가된 모든 모델에서 단일 에이전트 기준선을 유의미하게 상회하며 3회 이내의 반복에서 수렴함을 입증했습니다. 제안된 Legal ACS 지표를 사용하여 10명의 공인 법률 번역가가 수행한 인간 평가 결과, 법률-의미적 정확도의 향상이 확인되었으나, TransLaw는 문체적 자연스러움 측면에서는 여전히 인간 전문가에게 미치지 못하는 것으로 나타났습니다. 해당 데이터셋과 벤치마크 코드는 https://github.com/xuanxixi/TransLaw에서 확인할 수 있습니다.

arXiv:2606.02248v2cs.CL

Geometric Latent Reasoning Induces Shorter Generations in LLMs

Large language models solve complex problems by generating lengthy chains of explicit reasoning tokens. While effective, this makes reasoning expensive, length-sensitive, and constrained to (discrete) natural language. While latent reasoning offers a continuous alternative, determining useful structures for intermediate latent states is an open challenge. In this paper, we formulate latent reasoning as a geometric path-approximation problem within the model's pretrained token-embedding space. We introduce Geometric Latent Reasoning (GLR), which uses a lightweight transition head to predict iterative direction updates in embedding space. Using textual chain-of-thought traces as anchors, GLR learns to approximate discrete reasoning trajectories while permitting continuous deviations from exact token embeddings. Evaluations on mathematical reasoning benchmarks using Qwen3 models reveal an emergent phenomenon: geometric latent reasoning induces substantially shorter generations without an explicit length objective. By replacing early explicit reasoning with continuous latent steps, models often reach correct answers using substantially fewer total generation steps. These findings suggest that continuous trajectories act as compact intermediate reasoning states, exposing a new tradeoff between latent computation budget, output length, and accuracy. Our code is publicly available at: https://github.com/idiap/GLR. The project page is available at: https://glr-llm.github.io/.

arXiv:2606.17924v2cs.RO

PearlVLA: 잠재 공간에서의 점진적 체화 행동-계획 정제

PearlVLA: Progressive Embodied Action-Plan Refinement in Latent Space

현재의 비전-언어-행동(VLA, Vision-Language-Action) 모델은 효율적인 행동 생성과 명시적인 심사숙고 사이의 트레이드오프에 직면해 있습니다. 비전-언어 백본 표현에서 직접 행동을 디코딩하면 저지연 제어가 가능해지는 반면, 텍스트 기반 추론, 픽셀 수준 서브골(subgoal), 또는 디코딩된 행동에 대한 월드 모델 평가는 계획 수립을 개선할 수 있지만 상당한 지연 시간과 계산 비용을 초래합니다. 본 논문에서는 각 중간 계획의 예측된 결과로부터 얻은 피드백을 활용하여 VLM 유래 잠재 계획을 점진적으로 정제하는 VLA 프레임워크인 PearlVLA를 제안합니다. PearlVLA는 행동 정보가 없는(action-free) 비디오로 사전학습된 고정된 잠재 월드 모델(LaWM, Latent World Model)을 사용합니다. 각 정제 라운드마다 현재 계획은 연속적인 잠재 행동 코드를 생성하고, LaWM은 이에 대응하는 잠재 시각적 서브골을 예측합니다. 미래 유도형 계획 정제기는 이 서브골을 사용하여 계획을 업데이트하므로, 각 수정은 다음 LaWM 쿼리를 재구성합니다. K번의 라운드를 거친 후, 정제된 계획은 호스트 정책의 행동 인터페이스로 한 번 전달되어 행동 청크(action chunk)를 생성합니다. 나아가 동일한 정제 상태에서 이루어진 계획 편집의 더 긴 지평(longer-horizon)으로 상상된 미래에서 얻은 보상을 비교함으로써 잠재 정제를 최적화하는 인과 정제-그룹화 프로세스-보상 강화학습(Causal Refinement-Grouped Process-Reward RL)을 소개합니다. LIBERO 및 RoboCasa 벤치마크 실험을 통해 PearlVLA가 기존의 강력한 방법들에 비해 경쟁력 있는 성능을 보임을 확인했습니다.

arXiv:2608.06347v1cs.CL

RP-OPSD: 다국어 추론 전이를 위한 추론 피벗 유도 온-정책 자기 증류

RP-OPSD: Reasoning-Pivot-Guided On-Policy Self-Distillation for Multilingual Reasoning Transfer

다국어 추론 전이는 대규모 언어 모델(LLM)의 추론 능력을 고자원 언어를 넘어 확장하는 데 매우 중요합니다. 온-정책(on-policy) 자기 증류(OPSD) 및 그 변형 기법들은 학생 모델이 생성한 롤아웃에 대해 조밀한 토큰(token) 수준의 감독을 제공하는 유망한 패러다임으로 부상했으나, 그 목적 함수는 교차 언어 전이에 가장 중요한 추론 신호를 명시적으로 우선시하지 않습니다. 본 연구에서는 타깃 언어 추론이 표면 텍스트와 추론 피벗(reasoning pivot)의 생성을 모두 포함한다는 점을 규명합니다. 여기서 추론 피벗은 추론 과정을 진전시키거나 방향을 전환하고 후속 추론을 형성하는 결정들입니다. 이는 이러한 피벗을 중심으로 특권적 증류(privileged distillation)를 집중시켜야 한다는 동기를 부여합니다. 따라서 본 논문에서는 영어 참조 풀이가 있는 경우와 없는 경우의 일치된 교사 뷰 간 분포 변화(distributional shift)를 운영 프록시로 사용하여 특권적 증류 및 참조 앵커링을 유도하는 '추론 피벗 유도 온-정책 자기 증류(RP-OPSD)'를 제안합니다. 17개 언어와 다양한 난이도를 다루는 수학 추론 벤치마크 실험 결과, 본 연구의 방법은 강력한 다국어 추론 베이스라인 및 OPSD 변형 모델들을 능가하는 성능을 보였습니다. 추가 분석에 따르면, RP-OPSD는 주로 표면적 표현을 뒷받침하는 토큰에 대해서는 가중치를 낮추는 반면, 추론 제어 및 문제 조건부 상태 업데이트 토큰에 특권적 증류를 집중시키는 것으로 나타났습니다. 코드 원본은 https://github.com/NJUNLP/RP-OPSD 에서 확인할 수 있습니다.

arXiv:2608.27508v1cs.AI

WM-R1: 강화학습으로 GUI 에이전트가 추론하고 월드 모델을 활용하도록 학습시키기

WM-R1: Training GUI Agents to Reason and leverage World Models with Reinforcement Learning

강화학습(RL)으로 학습된 GUI 에이전트(agent)는 모바일 플랫폼에서 강력한 환경 학습 능력을 보여 왔다. 그러나 RL은 일반적으로 방대한 실제 환경 상호작용을 요구하여, 특히 GUI 시나리오에서 높은 자원 비용과 불안정성을 초래한다. 이를 해결하기 위해 본 논문은 실제 환경 대신 월드 모델(world model)로 모바일 GUI 에이전트를 학습시키는 최초의 강화학습 프레임워크인 WM-R1을 제안한다. 구체적으로, 월드 모델은 모든 롤아웃(rollout) 동안 상태 전이의 원천 역할을 하며, 학습 루프 내에서 실제 Android 환경을 대체한다. 또한 WM-R1은 월드 모델을 사고 과정에 직접 내장하여, 에이전트가 최종 행동을 확정하기 전에 후보 행동의 결과를 추론할 수 있도록 한다. 특히 WM-R1은 실제 환경 상호작용의 필요성을 제거하고, 월드 모델에 기반한 대규모 병렬 및 스텝 단위(step-level)의 세분화된 궤적 생성을 지원하며, 과업 성공, 궤적 효율성, 월드 모델 활용도를 함께 최적화하는 다차원 규칙 기반 보상을 도입한다. 효율적인 학습을 위해 어려운 과업 2000개로 구성된 고품질 데이터셋을 구축하였다. Android 모바일 벤치마크 실험 결과, WM-R1으로 학습된 에이전트는 GRPO만 사용한 베이스라인과 추론 시점 시뮬레이션 방법을 모두 유의미하게 능가함을 보였다. 코드는 https://github.com/genalyu/WM-R1 에서 확인할 수 있다.