근거 기반 비디오 질의응답(grounded video question answering) 및 시간적 그라운딩(temporal grounding)과 같은 비디오 추론 작업에서는 질의를 뒷받침하는 시간적 근거를 선택해야 합니다. 현재 많은 학습 설정에서는 경계 회귀(boundary regression)나 구간 생성(span generation)과 같은 국소적 목적함수를 통해 시간적 지도(temporal supervision)가 적용되는 반면, 검증(verification)은 주로 추론 시 후보 세그먼트를 재순위화하는 데 사용됩니다. 본 연구에서는 동결된 검증기(frozen verifier)가 학습을 유도할 수도 있는지 연구합니다. 제안하는 다중 에이전트 프레임워크는 학습 가능한 Grounder와 동결된 Verifier를 결합합니다. 즉, Grounder가 후보 궤적과 근거 세그먼트를 샘플링하고, Verifier는 질의 조건부 세그먼트 점수를 부여하며, 그룹 상대 정책 그래디언트(group-relative policy-gradient) 목적함수는 동일 입력 내 비교 대상보다 우수한 궤적을 선호하도록 하고, 부트스트랩된 보정 손실(bootstrapped calibration loss)은 시간적 예측이 검증기가 선호하는 구간을 향하도록 유도합니다. 소스 작업에서 학습되고 타깃 데이터셋에 대한 미세조정 없이 평가된 20억 개 매개변수 규모의 구현체는 근거 기반 질의응답, 시간적 그라운딩, 장시간 비디오 질의응답 전반에 걸쳐 제로샷으로 전이되어 근거 기반 질의응답 벤치마크에서 28.7%의 IoU(intersection-over-union) 및 25.4%의 답변-그라운딩 정확도, 시간적 그라운딩 벤치마크에서 46.1%의 IoU, 그리고 장시간 비디오 질의응답 벤치마크에서 54.1%를 달성합니다. 동일한 규모의 강력한 기준선 대비 성능 향상은 완만하지만 일관되며, IoU 및 중간 정도의 오버랩 재현율과 같은 관련성 중심 지표에서 가장 뚜렷한 개선을 보입니다. 테스트된 벤치마크 및 전이 설정 내에서, 이러한 결과는 근거 선택을 위한 학습 신호로서 동결된 검증을 사용하는 것을 뒷받침하는 동시에 엄격한 경계 정밀도는 비교적 여전히 취약함을 보여줍니다. 코드와 모델은 https://anonymous.4open.science/r/MASIRL-E50C/ 에서 이용할 수 있습니다.
셀룰러 오토마타는 단순한 함수들 간의 국소적 상호작용으로부터 복잡한 거동이 창발할 수 있는 국소 계산 패러다임이다. 이 패러다임은 생물학적 과정, 교통 시뮬레이션, 컴퓨터 네트워크 등 많은 시스템을 설명하는 데 사용되어 왔다. 게임 분야에서는 SimCity와 같은 게임에서, 또는 동굴이나 던전과 같은 공간적 콘텐츠 생성에 셀룰러 오토마타가 활용되어 왔다. 그러나 효과적인 국소 규칙을 만드는 것은 어렵고 직관적이지 않다. 셀룰러 오토마타는 진화를 통해 효과적으로 도출할 수 있지만, 여전히 해석하기 어려울 수 있다. 본 연구에서는 시스템을 Python 코드로 표현하는 프로그래밍 가능한 셀룰러 오토마타(programmable cellular automata) 개념을 제안한다. 또한 셀룰러 오토마타를 국소 함수(local functions)와 의사결정 함수(decision function)로 모듈화한다. 국소 함수는 국소 이웃(neighborhood)을 입력받아 값을 반환하고, 의사결정 함수는 국소 함수들의 출력을 받아 다음 상태의 값을 결정한다. 셀룰러 오토마타를 Python으로 작성된 모듈로 분리하면 이러한 시스템이 어떻게 동작하는지 이해하는 데 도움이 된다. 또한 전체 상태를 입력받아 그로부터 함수를 계산하는 전역 함수(global functions)를 추가하는 방안을 탐구한다. PCG Benchmark에 포함된 서로 다른 세 가지 게임에 대한 레벨 생성을 테스트하였다. 결과적으로 전역 함수는 문제를 해결하는 데 필요한 셀룰러 오토마타의 반복 횟수를 줄여 주며, 순수하게 국소 함수만으로는 일부 문제에 대한 해법을 찾을 수 없음을 보였다. 생성된 함수들을 살펴보면 서로 다른 실험에서 공통적으로 사용된 함수들을 확인할 수 있으며, 이는 생성기를 이해하는 데 도움이 될 뿐만 아니라 해당 게임들을 더 잘 이해하고 무엇이 중요한지 파악하는 데에도 도움이 된다.
Autoregressive video diffusion models have emerged as a foundation for long-horizon world rollout. However, when extended far beyond their training windows, these models suffer from progressive drift: visual quality deteriorates, motion becomes inconsistent, and object identity gradually collapses as generated frames are recursively reused as context. We study a supervision-horizon gap: short-horizon distillation leaves later rollout states without direct supervision. We show that matching short-window marginals need not determine long-horizon behavior, while longer on-policy trajectories expose delayed failures to training. This motivates temporal manifold matching at scale for long-horizon world models. Building on this insight, we introduce LongForcing, a distillation approach that scales teacher supervision to long student self-rollouts. LongForcing preserves the short-horizon teacher-forcing and causal ODE-distillation stages, while replacing the realscore teacher in the final distribution-matching stage with an extended-horizon bidirectional teacher. The long-horizon teacher supervises extended student self-rollouts using the standard distribution-matching formulation, without modifying the student architecture or increasing inference cost. We evaluate LongForcing on text-to-video generation and interactive action-conditioned world models, achieving a VBench Quality score of 85.52 and improving WorldRoamBench 60s action score by 23.9 points in first-person ablations. Qualitative stress tests further sample interactive rollouts up to 24 hours, far beyond the training horizon. These results suggest that scaling supervision along the temporal axis is a practical step toward long-horizon world models. The project website and code are at https://github.com/Fantasy-AMAP/LongForcing.
The primary motivation of Image-to-Image Transformation is to convert an image of one domain to another domain. Most of the research has been focused on the task of image transformation for a set of pre-defined domains. Very few works are reported that actually developed a common framework for image-to-image transformation for different domains. With the introduction of Generative Adversarial Networks (GANs) as a general framework for the image generation problem, there is a tremendous growth in the area of image-to-image transformation. Most of the research focuses over the suitable objective function for image-to-image transformation. In this paper, we propose a new Cyclic-Synthesized Generative Adversarial Networks (CSGAN) for image-to-image transformation. The proposed CSGAN uses a new objective function (loss) called Cyclic-Synthesized Loss (CS) between the synthesized image of one domain and cycled image of another domain. The performance of the proposed CSGAN is evaluated on two benchmark image-to-image transformation datasets, including CUHK Face dataset and CMP Facades dataset. The results are computed using the widely used evaluation metrics such as MSE, SSIM, PSNR, and LPIPS. The experimental results of the proposed CSGAN approach are compared with the latest state-of-the-art approaches such as GAN, Pix2Pix, DualGAN, CycleGAN and PS2GAN. The proposed CSGAN technique outperforms all the methods over CUHK dataset and exhibits the promising and comparable performance over Facades dataset in terms of both qualitative and quantitative measures. The code is available at https://github.com/KishanKancharagunta/CSGAN.
RGB-Infrared (IR) person re-identification is very challenging due to the large cross-modality variations between RGB and IR images. The key solution is to learn aligned features to the bridge RGB and IR modalities. However, due to the lack of correspondence labels between every pair of RGB and IR images, most methods try to alleviate the variations with set-level alignment by reducing the distance between the entire RGB and IR sets. However, this set-level alignment may lead to misalignment of some instances, which limits the performance for RGB-IR Re-ID. Different from existing methods, in this paper, we propose to generate cross-modality paired-images and perform both global set-level and fine-grained instance-level alignments. Our proposed method enjoys several merits. First, our method can perform set-level alignment by disentangling modality-specific and modality-invariant features. Compared with conventional methods, ours can explicitly remove the modality-specific features and the modality variation can be better reduced. Second, given cross-modality unpaired-images of a person, our method can generate cross-modality paired images from exchanged images. With them, we can directly perform instance-level alignment by minimizing distances of every pair of images. Extensive experimental results on two standard benchmarks demonstrate that the proposed model favourably against state-of-the-art methods. Especially, on SYSU-MM01 dataset, our model can achieve a gain of 9.2% and 7.7% in terms of Rank-1 and mAP. Code is available at https://github.com/wangguanan/JSIA-ReID.
Vehicle color recognition plays an important role in intelligent traffic management and criminal investigation assistance. However, the current vehicle color recognition research involves at most 13 types of colors and the recognition accuracy is low, which is difficult to meet practical applications. To this end, this paper has built a benchmark dataset (Vehicle Color-24) that includes 24 types of vehicle colors, including 10091 vehicle pictures taken from 100 hours of urban road surveillance videos. In addition, in order to solve the problem of long tail distribution in Vehicle Color-24 dataset and low recognition rate of existing methods, this paper proposes a Smooth Modulated Neural Network with Multi-layer Feature Representation (SMNN-MFR) is used for 24 types of vehicle color recognition. SMNN-MFR includes four parts: feature extraction, multi-scale feature fusion, suggestion frame generation and smooth modulation. The model is trained and verified on the Vehicle Color-24 benchmark dataset. Comprehensive experiments show that the average recognition accuracy of the algorithm in the 24 categories of color benchmark databases is 94.96%, which is 33.47% higher than the Faster RCNN network. In addition, the average accuracy rate of the model when recognizing 8 types of colors is 97.25%, and the detection accuracy of algorithms in similar databases is improved. At the same time, visualization and ablation experiments also proved the rationality of our network settings and the effectiveness of each module. The code and database are published at: https://github.com/mendy-2013.
Despite achieving promising results in a breadth of medical image segmentation tasks, deep neural networks require large training datasets with pixel-wise annotations. Obtaining these curated datasets is a cumbersome process which limits the applicability in scenarios. Mixed supervision is an appealing alternative for mitigating this obstacle. In this work, we propose a dual-branch architecture, where the upper branch (teacher) receives strong annotations, while the bottom one (student) is driven by limited supervision and guided by the upper branch. Combined with a standard cross-entropy loss over the labeled pixels, our novel formulation integrates two important terms: (i) a Shannon entropy loss defined over the less-supervised images, which encourages confident student predictions in the bottom branch; and (ii) a KL divergence term, which transfers the knowledge (i.e., predictions) of the strongly supervised branch to the less-supervised branch and guides the entropy (student-confidence) term to avoid trivial solutions. We show that the synergy between the entropy and KL divergence yields substantial improvements in performance. We also discuss an interesting link between Shannon-entropy minimization and standard pseudo-mask generation, and argue that the former should be preferred over the latter for leveraging information from unlabeled pixels. We evaluate the effectiveness of the proposed formulation through a series of quantitative and qualitative experiments using two publicly available datasets. Results demonstrate that our method significantly outperforms other strategies for semantic segmentation within a mixed-supervision framework, as well as recent semi-supervised approaches. Our code is publicly available: https://github.com/by-liu/ConfKD.
Source-free Domain Adaptation (SFDA) aims to adapt a pre-trained source model to the unlabeled target domain without accessing the well-labeled source data, which is a much more practical setting due to the data privacy, security, and transmission issues. To make up for the absence of source data, most existing methods introduced feature prototype based pseudo-labeling strategies to realize self-training model adaptation. However, feature prototypes are obtained by instance-level predictions based feature clustering, which is category-biased and tends to result in noisy labels since the visual domain gaps between source and target are usually different between categories. In addition, we found that a monocentric feature prototype may be ineffective to represent each category and introduce negative transfer, especially for those hard-transfer data. To address these issues, we propose a general class-Balanced Multicentric Dynamic prototype (BMD) strategy for the SFDA task. Specifically, for each target category, we first introduce a global inter-class balanced sampling strategy to aggregate potential representative target samples. Then, we design an intra-class multicentric clustering strategy to achieve more robust and representative prototypes generation. In contrast to existing strategies that update the pseudo label at a fixed training period, we further introduce a dynamic pseudo labeling strategy to incorporate network update information during model adaptation. Extensive experiments show that the proposed model-agnostic BMD strategy significantly improves representative SFDA methods to yield new state-of-the-art results. The code is available at https://github.com/ispc-lab/BMD.
Data diversity and volume are crucial to the success of training deep learning models, while in the medical imaging field, the difficulty and cost of data collection and annotation are especially huge. Specifically in robotic surgery, data scarcity and imbalance have heavily affected the model accuracy and limited the design and deployment of deep learning-based surgical applications such as surgical instrument segmentation. Considering this, we rethink the surgical instrument segmentation task and propose a one-to-many data generation solution that gets rid of the complicated and expensive process of data collection and annotation from robotic surgery. In our method, we only utilize a single surgical background tissue image and a few open-source instrument images as the seed images and apply multiple augmentations and blending techniques to synthesize amounts of image variations. In addition, we also introduce the chained augmentation mixing during training to further enhance the data diversities. The proposed approach is evaluated on the real datasets of the EndoVis-2018 and EndoVis-2017 surgical scene segmentation. Our empirical analysis suggests that without the high cost of data collection and annotation, we can achieve decent surgical instrument segmentation performance. Moreover, we also observe that our method can deal with novel instrument prediction in the deployment domain. We hope our inspiring results will encourage researchers to emphasize data-centric methods to overcome demanding deep learning limitations besides data shortage, such as class imbalance, domain adaptation, and incremental learning. Our code is available at https://github.com/lofrienger/Single_SurgicalScene_For_Segmentation.
With the increasing affordability and accessibility of genome sequencing data, de novo genome assembly is an important first step to a wide variety of downstream studies and analyses. Therefore, bioinformatics tools that enable the generation of high-quality genome assemblies in a computationally efficient manner are essential. Recent developments in long-read sequencing technologies have greatly benefited genome assembly work, including scaffolding, by providing long-range evidence that can aid in resolving the challenging repetitive regions of complex genomes. ntLink is a flexible and resource-efficient genome scaffolding tool that utilizes long-read sequencing data to improve upon draft genome assemblies built from any sequencing technologies, including the same long reads. Instead of using read alignments to identify candidate joins, ntLink utilizes minimizer-based mappings to infer how input sequences should be ordered and oriented into scaffolds. Recent improvements to ntLink have added important features such as overlap detection, gap-filling and in-code scaffolding iterations. Here, we present three basic protocols demonstrating how to use each of these new features to yield highly contiguous genome assemblies, while still maintaining ntLink's proven computational efficiency. Further, as we illustrate in the alternate protocols, the lightweight minimizer-based mappings that enable ntLink scaffolding can also be utilized for other downstream applications, such as misassembly detection. With its modularity and multiple modes of execution, ntLink has broad benefit to the genomics community, from genome scaffolding and beyond. ntLink is an open-source project and is freely available from https://github.com/bcgsc/ntLink.
Traffic object detection under variable illumination is challenging due to the information loss caused by the limited dynamic range of conventional frame-based cameras. To address this issue, we introduce bio-inspired event cameras and propose a novel Structure-aware Fusion Network (SFNet) that extracts sharp and complete object structures from the event stream to compensate for the lost information in images through cross-modality fusion, enabling the network to obtain illumination-robust representations for traffic object detection. Specifically, to mitigate the sparsity or blurriness issues arising from diverse motion states of traffic objects in fixed-interval event sampling methods, we propose the Reliable Structure Generation Network (RSGNet) to generate Speed Invariant Frames (SIF), ensuring the integrity and sharpness of object structures. Next, we design a novel Adaptive Feature Complement Module (AFCM) which guides the adaptive fusion of two modality features to compensate for the information loss in the images by perceiving the global lightness distribution of the images, thereby generating illumination-robust representations. Finally, considering the lack of large-scale and high-quality annotations in the existing event-based object detection datasets, we build a DSEC-Det dataset, which consists of 53 sequences with 63,931 images and more than 208,000 labels for 8 classes. Extensive experimental results demonstrate that our proposed SFNet can overcome the perceptual boundaries of conventional cameras and outperform the frame-based method by 8.0% in mAP50 and 5.9% in mAP50:95. Our code and dataset will be available at https://github.com/YN-Yang/SFNet.
Although remarkable progress has been made in image style transfer, style is just one of the components of artistic paintings. Directly transferring extracted style features to natural images often results in outputs with obvious synthetic traces. This is because key painting attributes including layout, perspective, shape, and semantics often cannot be conveyed and expressed through style transfer. Large-scale pretrained text-to-image generation models have demonstrated their capability to synthesize a vast amount of high-quality images. However, even with extensive textual descriptions, it is challenging to fully express the unique visual properties and details of paintings. Moreover, generic models often disrupt the overall artistic effect when modifying specific areas, making it more complicated to achieve a unified aesthetic in artworks. Our main novel idea is to integrate multimodal semantic information as a synthesis guide into artworks, rather than transferring style to the real world. We also aim to reduce the disruption to the harmony of artworks while simplifying the guidance conditions. Specifically, we propose an innovative multi-task unified framework called CreativeSynth, based on the diffusion model with the ability to coordinate multimodal inputs. CreativeSynth combines multimodal features with customized attention mechanisms to seamlessly integrate real-world semantic content into the art domain through Cross-Art-Attention for aesthetic maintenance and semantic fusion. We demonstrate the results of our method across a wide range of different art categories, proving that CreativeSynth bridges the gap between generative models and artistic expression. Code and results are available at https://github.com/haha-lisa/CreativeSynth.
Reinforcement learning (RL) is an effective method of finding reasoning pathways in incomplete knowledge graphs (KGs). To overcome the challenges of a large action space, a self-supervised pre-training method is proposed to warm up the policy network before the RL training stage. To alleviate the distributional mismatch issue in general self-supervised RL (SSRL), in our supervised learning (SL) stage, the agent selects actions based on the policy network and learns from generated labels; this self-generation of labels is the intuition behind the name self-supervised. With this training framework, the information density of our SL objective is increased and the agent is prevented from getting stuck with the early rewarded paths. Our self-supervised RL (SSRL) method improves the performance of RL by pairing it with the wide coverage achieved by SL during pretraining, since the breadth of the SL objective makes it infeasible to train an agent with that alone. We show that our SSRL model meets or exceeds current state-of-the-art results on all Hits@k and mean reciprocal rank (MRR) metrics on four large benchmark KG datasets. This SSRL method can be used as a plug-in for any RL architecture for a KGR task. We adopt two RL architectures, i.e., MINERVA and MultiHopKG as our baseline RL models and experimentally show that our SSRL model consistently outperforms both baselines on all of these four KG reasoning tasks. Full code for the paper available at https://github.com/owenonline/Knowledge-Graph-Reasoning-with-Self-supervised-Reinforcement-Learning.
Retrieval-Augmented Generation (RAG) has emerged as an effective paradigm for generating contextually accurate answers by integrating Large Language Models (LLMs) with retrieval mechanisms. However, in legal contexts, users frequently reference norms by their labels or nicknames (e.g., Article 5 of the Constitution or Consumer Defense Code (CDC)), rather than by their content, posing challenges for traditional RAG approaches that rely solely on semantic embeddings of text. Furthermore, legal texts themselves heavily rely on explicit cross-references (e.g., "pursuant to Article 34") that function as pointers. Both scenarios pose challenges for traditional RAG approaches that rely solely on semantic embeddings of text, often failing to retrieve the necessary referenced content. This paper introduces Poly-Vector Retrieval, a method assigning multiple distinct embeddings to each legal provision: one embedding captures the content (the full text), another captures the label (the identifier or proper name), and optionally additional embeddings capture alternative denominations. Inspired by Frege's distinction between Sense and Reference, this poly-vector retrieval approach treats labels, identifiers and reference markers as rigid designators and content embeddings as carriers of semantic substance. Experiments on the Brazilian Federal Constitution demonstrate that Poly-Vector Retrieval significantly improves retrieval accuracy for label-centric queries and potential to resolve internal and external cross-references, without compromising performance on purely semantic queries. The study discusses philosophical and practical implications of explicitly separating reference from content in vector embeddings and proposes future research directions for applying this approach to broader legal datasets and other domains characterized by explicit reference identifiers.
While the capabilities of large language models (LLMs) have progressed significantly, their use in high-stakes applications have been limited due to risks of hallucination. One key approach in reducing hallucination is retrieval-augmented generation (RAG), but even in such setups, LLMs may still hallucinate when presented with questions outside of the knowledge base. Such behavior is unacceptable in high-stake applications where LLMs are expected to abstain from answering queries it does not have sufficient context on. In this work, we present a novel methodology for systematically evaluating out-of-knowledge base (OOKB) robustness of LLMs (whether LLMs know or do not know) in the RAG setting, without the need for manual annotation of gold standard answers. We implement our methodology in knowornot, an open-source library that enables users to develop their own customized evaluation data and pipelines for OOKB robustness. knowornot comprises four main features. Firstly, it provides a unified, high-level API that streamlines the process of setting up and running robustness benchmarks. Secondly, its modular architecture emphasizes extensibility and flexibility, allowing users to easily integrate their own LLM clients and RAG settings. Thirdly, its rigorous data modeling design ensures experiment reproducibility, reliability and traceability. Lastly, it implements a comprehensive suite of tools for users to customize their pipelines. We demonstrate the utility of knowornot by developing a challenging benchmark, PolicyBench, which spans four Question-Answer (QA) chatbots on government policies, and analyze its OOKB robustness. The source code of knowornot is available https://github.com/govtech-responsibleai/KnowOrNot.
With the rapid advancement of large language models like Gemini, GPT, and others, bridging the gap between the human brain and language processing has become an important area of focus. To address this challenge, researchers have developed various models to decode EEG signals into text. However, these models still face significant performance limitations. To overcome these shortcomings, we propose a new model, R1 Translator, which aims to improve the performance of EEG-to-text decoding. The R1 Translator model combines a bidirectional LSTM encoder with a pretrained transformer-based decoder, utilizing EEG features to produce high-quality text outputs. The model processes EEG embeddings through the LSTM to capture sequential dependencies, which are then fed into the transformer decoder for effective text generation. The R1 Translator excels in ROUGE metrics, outperforming both T5 (previous research) and Brain Translator. Specifically, R1 achieves a ROUGE-1 score of 38.00% (P), which is up to 9% higher than T5 (34.89%) and 3% better than Brain (35.69%). It also leads in ROUGE-L, with a F1 score of 32.51%, outperforming T5 by 3% (29.67%) and Brain by 2% (30.38%). In terms of CER, R1 achieves a CER of 0.5795, which is 2% lower than T5 (0.5917) and 4% lower than Brain (0.6001). Additionally, R1 performs better in WER with a score of 0.7280, outperforming T5 by 4.3% (0.7610) and Brain by 3.6% (0.7553). Code is available at https://github.com/Mmurrad/EEG-To-text.
Agentic AI systems, which build on Large Language Models (LLMs) and interact with tools and memory, have rapidly advanced in capability and scope. Yet, since LLMs have been shown to struggle in multilingual settings, typically resulting in lower performance and reduced safety, agentic systems risk inheriting these limitations. This raises concerns about the global accessibility of such systems, as users interacting in languages other than English may encounter unreliable or security-critical agent behavior. Despite growing interest in evaluating agentic AI, existing benchmarks focus exclusively on English, leaving multilingual settings unexplored. To address this gap, we propose MAPS, a multilingual benchmark suite designed to evaluate agentic AI systems across diverse languages and tasks. MAPS builds on four widely used agentic benchmarks - GAIA (real-world tasks), SWE-bench (code generation), MATH (mathematical reasoning), and the Agent Security Benchmark (security). We translate each dataset into ten diverse languages, resulting in 805 unique tasks and 8,855 total language-specific instances. Our benchmark suite enables a systematic analysis of how multilingual contexts affect agent performance and robustness. Empirically, we observe consistent degradation in both performance and security when transitioning from English to other languages, with severity varying by task and correlating with the amount of translated input. Building on these findings, we provide actionable recommendations to guide agentic AI systems development and assessment under multilingual settings. This work establishes a standardized evaluation framework, encouraging future research towards equitable, reliable, and globally accessible agentic AI. MAPS benchmark suite is publicly available at https://huggingface.co/datasets/Fujitsu-FRE/MAPS
Large language models (LLMs) can face factual limitations when responding to time-sensitive queries about recent events that arise after their knowledge thresholds in the training corpus. Existing search-augmented approaches fall into two categories, each with distinct limitations: multi-agent search frameworks incur substantial computational overhead by separating search planning and response synthesis across multiple LLMs, while single-LLM tool-calling methods restrict themselves to sequential planned, single-query searches from sole search sources. We present Reasoning-Search (R-Search), a single-LLM search framework that unifies multi-step planning, multi-source search execution, and answer synthesis within one coherent inference process. Innovatively, it structure the output into four explicitly defined components, including reasoning steps that guide the search process (<think>), a natural-language directed acyclic graph that represents the search plans with respect to diverse sources (<search>), retrieved results from executing the search plans (<result>), and synthesized final answers (<answer>). To enable effective generation of these structured outputs, we propose a specialized Reinforcement Fine-Tuning (ReFT) method based on GRPO, together with a multi-component reward function that optimizes LLM's answer correctness, structural validity of the generated DAG, and adherence to the defined output format. Experimental evaluation on FinSearchBench-24, SearchExpertBench-25, and seven Q and A benchmarks demonstrates that R-Search outperforms state-of-the-art methods, while achieving substantial efficiency gains through 70% reduction in context token usage and approximately 50% decrease in execution latency. Code is available at https://github.com/wentao0429/Reasoning-search.
강화학습을 통한 LLM 사후학습은 정확도를 일관되게 향상시켜 왔으나 그 대가로 행동 범위가 축소되며, 이러한 현상은 샤프닝이라 불리고 어려운 과제에서의 진전을 정체시킬 수 있다. $\mathrm{pass@k}$와 $\mathrm{max@k}$ 같은 반복 샘플링 목적함수가 샤프닝을 완화하는 것으로 알려져 있으나, 그 메커니즘은 여전히 잘 이해되지 않고 있다. 더욱이 기존의 $\mathrm{@k}$ 최적화 접근법은 전용 알고리즘을 사용하며, 확립된 정책 최적화 방법을 $\mathrm{@}1$에서 $\mathrm{@k}$로 적응시키기 위한 일반 프레임워크는 존재하지 않는다. 본 논문은 KL 정규화된 $\mathrm{pass@k}$ 및 $\mathrm{max@k}$ 목적함수의 최적 정책을 닫힌 형태로 유도한다. 이 명시적 해는 $\mathrm{@k}$ 최적화가 프롬프트 난이도에 따라 학습 압력을 자연스럽게 재분배하여 더 어려운 문제 쪽으로 향하게 함을 보여 준다. 결정적으로, 표준 $\mathrm{@}1$ 목적함수가 프롬프트 적응형 정규화 또는 보상 변환을 통해 이와 정확히 동일한 최적 정책을 달성함을 보인다. 궁극적으로 이는 $\mathrm{@k}$ 목적함수가 쉬운 과제에서의 샤프닝을 줄이면서 어려운 과제에서의 성공을 우선시하는 이유에 대한 최적화기 독립적 설명을 제공한다. 이러한 통찰을 바탕으로 $\mathrm{@k}$ 적응형 압력 정책 최적화(KAPPO)를 소개한다. 일회성 기준 보정 이후, KAPPO는 기존 학습 절차를 달리 변경하지 않고 프롬프트 적응형 정규화 또는 보상 변환을 통해 기존 $\mathrm{@}1$ 방법을 $\mathrm{@k}$로 매끄럽게 적응시킨다. 수학적 추론과 코드 생성에서 여러 RL 방법으로 수행한 실험은 KAPPO가 샤프닝에 대한 직접적인 제어를 제공함을 보여 준다. 결정적으로, $\mathrm{@k}$ 압력이 단일 완성 학습만으로 완전히 구현될 수 있으며, 다중 완성 $\mathrm{@k}$ 최적화기의 성능을 일부의 비용으로 따라잡음을 보인다.
인간은 물리적 세계와 상호작용하기 위해 다양한 감각 양식을 자연스럽게 활용하지만, 대부분의 로봇 공학용 비전-언어-행동(VLA) 모델은 오직 RGB 관측에만 의존합니다. 이는 온도, 소리, 레이더 반응과 같이 RGB 카메라로는 추론하기 어렵거나 불가능한 물리적 특성을 인지하는 능력을 제한합니다. 본 논문에서는 새로운 센서를 로봇 조작을 위한 온디맨드 도구로 통합하는 적응형 다중 모달 센싱 VLA 모델인 MuseVLA를 제시합니다. 작업 지시와 시각적 맥락이 주어지면, MuseVLA는 먼저 호출할 센싱 양식과 주목할 대상을 선택하는 센서 토큰과 타깃 설명을 생성하며, 이는 인자를 갖춘 도구 호출과 유사합니다. 그런 다음 선택된 센서 측정값을 다중 모달 융합 및 행동 생성을 위해 이종의 판독값을 인코딩하는 통합 중간 표현인 그라운딩된 센서 이미지로 변환합니다. 이러한 설계는 센서 고유의 처리를 VLA 백본에서 분리하여 다양한 양식을 효율적으로 통합할 수 있도록 합니다. 비용이 많이 드는 다중 감각 로봇 데이터셋의 필요성을 줄이기 위해, 기존 RGB 비디오 데이터셋을 그라운딩된 센서 이미지로 증강하여 보지 못한 센서 기반 작업으로의 일반화를 가능하게 하는 데이터 합성 파이프라인을 추가로 도입합니다. 온도 기반 집기 및 놓기, 오디오 기반 물체 탐색, 레이더 보조 숨겨진 물체 회수를 포함하여 다중 모달 센싱 입력이 필요한 까다로운 다지형(dexterous) 손 조작 작업 전반에 걸쳐 실제 로봇에서 MuseVLA를 평가합니다. MuseVLA는 평균 80.6%의 성공률을 달성하여 RGB 전용 및 다중 감각 VLA 베이스라인을 크게 능가하며, 보지 못한 작업에서도 강력한 제로샷 능력을 보여줍니다. 코드, 모델 및 데이터셋은 https://github.com/microsoft/MuseVLA 에서 확인할 수 있습니다.