PyTorchKR Papers

“Code Generation”

총 10,140편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2410.14853

DFlow: Diverse Dialogue Flow Simulation with Large Language Models

Developing language model-based dialogue agents requires effective data to train models that can follow specific task logic. However, most existing data augmentation methods focus on increasing diversity in language, topics, or dialogue acts at the utterance level, largely neglecting a critical aspect of task logic diversity at the dialogue level. This paper proposes a novel data augmentation method designed to enhance the diversity of synthetic dialogues by focusing on task execution logic. Our method uses LLMs to generate decision tree-structured task plans, which enables the derivation of diverse dialogue trajectories for a given task. Each trajectory, referred to as a "dialog flow", guides the generation of a multi-turn dialogue that follows a unique trajectory. We apply this method to generate a task-oriented dialogue dataset comprising 3,886 dialogue flows across 15 different domains. We validate the effectiveness of this dataset using the next action prediction task, where models fine-tuned on our dataset outperform strong baselines, including GPT-4. Upon acceptance of this paper, we plan to release the code and data publicly.

arXiv:2410.15387공식 코드 · Papers with Code

Deep Class-guided Hashing for Multi-label Cross-modal Retrieval

Deep hashing, due to its low cost and efficient retrieval advantages, is widely valued in cross-modal retrieval. However, existing cross-modal hashing methods either explore the relationships between data points, which inevitably leads to intra-class dispersion, or explore the relationships between data points and categories while ignoring the preservation of inter-class structural relationships, resulting in the generation of suboptimal hash codes. How to maintain both intra-class aggregation and inter-class structural relationships, In response to this issue, this paper proposes a DCGH method. Specifically, we use proxy loss as the mainstay to maintain intra-class aggregation of data, combined with pairwise loss to maintain inter-class structural relationships, and on this basis, further propose a variance constraint to address the semantic bias issue caused by the combination. A large number of comparative experiments on three benchmark datasets show that the DCGH method has comparable or even better performance compared to existing cross-modal retrieval methods. The code for the implementation of our DCGH framework is available at https://github.com/donnotnormal/DCGH.

arXiv:2504.15135공식 코드 · Papers with Code

KGMEL: Knowledge Graph-Enhanced Multimodal Entity Linking

Entity linking (EL) aligns textual mentions with their corresponding entities in a knowledge base, facilitating various applications such as semantic search and question answering. Recent advances in multimodal entity linking (MEL) have shown that combining text and images can reduce ambiguity and improve alignment accuracy. However, most existing MEL methods overlook the rich structural information available in the form of knowledge-graph (KG) triples. In this paper, we propose KGMEL, a novel framework that leverages KG triples to enhance MEL. Specifically, it operates in three stages: (1) Generation: Produces high-quality triples for each mention by employing vision-language models based on its text and images. (2) Retrieval: Learns joint mention-entity representations, via contrastive learning, that integrate text, images, and (generated or KG) triples to retrieve candidate entities for each mention. (3) Reranking: Refines the KG triples of the candidate entities and employs large language models to identify the best-matching entity for the mention. Extensive experiments on benchmark datasets demonstrate that KGMEL outperforms existing methods. Our code and datasets are available at: https://github.com/juyeonnn/KGMEL.

arXiv:2505.00740

Fast2comm:Collaborative perception combined with prior knowledge

Collaborative perception has the potential to significantly enhance perceptual accuracy through the sharing of complementary information among agents. However, real-world collaborative perception faces persistent challenges, particularly in balancing perception performance and bandwidth limitations, as well as coping with localization errors. To address these challenges, we propose Fast2comm, a prior knowledge-based collaborative perception framework. Specifically, (1)we propose a prior-supervised confidence feature generation method, that effectively distinguishes foreground from background by producing highly discriminative confidence features; (2)we propose GT Bounding Box-based spatial prior feature selection strategy to ensure that only the most informative prior-knowledge features are selected and shared, thereby minimizing background noise and optimizing bandwidth efficiency while enhancing adaptability to localization inaccuracies; (3)we decouple the feature fusion strategies between model training and testing phases, enabling dynamic bandwidth adaptation. To comprehensively validate our framework, we conduct extensive experiments on both real-world and simulated datasets. The results demonstrate the superior performance of our model and highlight the necessity of the proposed methods. Our code is available at https://github.com/Zhangzhengbin-TJ/Fast2comm.

arXiv:2505.10634

Cross-Image Contrastive Decoding: Precise, Lossless Suppression of Language Priors in Large Vision-Language Models

Language priors are a major cause of hallucinations in Large Vision-Language Models (LVLMs), often leading to text that is linguistically plausible but visually inconsistent. Recent work explores contrastive decoding as a training-free solution, but these methods typically construct negative contexts from the original image, resulting in visual information loss and distorted distribution. Motivated by the observation that language priors stem from the LLM backbone and remain consistent across images, we propose Cross-Images Contrastive Decoding (CICD), a simple yet effective training-free method that uses different images to construct negative contexts. We further analyze the cross-image behavior of language priors and introduce a distinction between essential priors (supporting fluency) and detrimental priors (causing hallucinations). By selectively preserving essential priors and suppressing detrimental ones, our method reduces hallucinations while maintaining coherent and fluent language generation. Experiments on 4 benchmarks and 6 LVLMs across three model families confirm the effectiveness and generalizability of CICD, especially in image captioning, where language priors are particularly pronounced. Code will be released once accepted.

arXiv:2505.15753

Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval

Large Language Models (LLMs) are known to be vulnerable to jailbreaking attacks, wherein adversaries exploit carefully engineered prompts to induce harmful or unethical responses. Such threats have raised critical concerns about the safety and reliability of LLMs in real-world deployment. While existing defense mechanisms partially mitigate such risks, subsequent advancements in adversarial techniques have enabled novel jailbreaking methods to circumvent these protections, exposing the limitations of static defense frameworks. In this work, we explore defending against evolving jailbreaking threats through the lens of context retrieval. First, we conduct a preliminary study demonstrating that even a minimal set of safety-aligned examples against a particular jailbreak can significantly enhance robustness against this attack pattern. Building on this insight, we further leverage the retrieval-augmented generation (RAG) techniques and propose Safety Context Retrieval (SCR), a scalable and robust safeguarding paradigm for LLMs against jailbreaking. Our comprehensive experiments demonstrate how SCR achieves superior defensive performance against both established and emerging jailbreaking tactics, contributing a new paradigm to LLM safety. Our code will be available upon publication.

arXiv:2505.21019공식 코드 · Papers with Code

Cardiac Digital Twins at Scale from MRI: Open Tools and Representative Models from ~55000 UK Biobank Participants

A cardiac digital twin is a virtual replica of a patient's heart for screening, diagnosis, prognosis, risk assessment, and treatment planning of cardiovascular diseases. This requires an anatomically accurate patient-specific 3D structural representation of the heart, suitable for electro-mechanical simulations or study of disease mechanisms. However, generation of cardiac digital twins at scale is demanding and there are no public repositories of models across demographic groups. We describe an automatic open-source pipeline for creating patient-specific left and right ventricular meshes from cardiovascular magnetic resonance images, its application to a large cohort of ~55000 participants from UK Biobank, and the construction of the most comprehensive cohort of adult heart models to date, comprising 1423 representative meshes across sex (male, female), body mass index (range: 16 - 42 kg/m$^2$) and age (range: 49 - 80 years). Our code is available at https://github.com/cdttk/biv-volumetric-meshing/tree/plos2025 , and pre-trained networks, representative volumetric meshes with fibers and UVCs will be made available soon.

arXiv:2506.06683

RoboPARA: Dual-Arm Robot Planning with Parallel Allocation and Recomposition Across Tasks

Dual-arm robots play a crucial role in improving efficiency and flexibility in complex multitasking scenarios. While existing methods have achieved promising results in task planning, they often fail to fully optimize task parallelism, limiting the potential of dual-arm collaboration. To address this issue, we propose RoboPARA, a novel large language model (LLM)-driven framework for dual-arm task parallelism planning. RoboPARA employs a two-stage process: (1) Dependency Graph-based Planning Candidates Generation, which constructs directed acyclic graphs (DAGs) to model task dependencies and eliminate redundancy, and (2) Graph Re-Traversal-based Dual-Arm Parallel Planning, which optimizes DAG traversal to maximize parallelism while maintaining task coherence. In addition, we introduce the Cross-Scenario Dual-Arm Parallel Task dataset (X-DAPT dataset), the first dataset specifically designed to evaluate dual-arm task parallelism across diverse scenarios and difficulty levels. Extensive experiments on the X-DAPT dataset demonstrate that RoboPARA significantly outperforms existing methods, achieving higher efficiency and reliability, particularly in complex task combinations. The code and dataset will be released upon acceptance.

arXiv:2506.13404

A Technical Study into Small Reasoning Language Models

The ongoing evolution of language models has led to the development of large-scale architectures that demonstrate exceptional performance across a wide range of tasks. However, these models come with significant computational and energy demands, as well as potential privacy implications. In this context, Small Reasoning Language Models (SRLMs) with approximately 0.5 billion parameters present a compelling alternative due to their remarkable computational efficiency and cost effectiveness, particularly in resource-constrained environments. Despite these advantages, the limited capacity of 0.5 billion parameter models poses challenges in handling complex tasks such as mathematical reasoning and code generation. This research investigates various training strategies, including supervised fine-tuning (SFT), knowledge distillation (KD), and reinforcement learning (RL), as well as their hybrid implementations, to enhance the performance of 0.5B SRLMs. We analyze effective methodologies to bridge the performance gap between SRLMS and larger models and present insights into optimal training pipelines tailored for these smaller architectures. Through extensive experimental validation and analysis, our work aims to provide actionable recommendations for maximizing the reasoning capabilities of 0.5B models.

arXiv:2507.12507cs.LG

Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training

최근 대규모 강화학습(RL)을 활용한 추론 중심 언어모델이 수학, 코드 생성 등 복잡한 과제에서 체인 오브 쏘트(chain-of-thought) 추론과 반복 탐색을 통해 성능 향상을 이뤘습니다. 본 연구에서는 소형 언어모델에 장기 강화학습을 적용하여 다양한 추론 영역에서의 효과를 분석하였으며, 검증 가능한 보상 신호와 Group Relative Policy Optimization(GRPO) 개선, 그리고 안정적 학습과 일반화 향상을 위한 KL 정규화, 클리핑 비율, 주기적 참조 정책 초기화 등의 기법을 제안하였습니다. 제안한 방법은 수학, 코딩, 논리 퍼즐 분야에서 각각 14.7%, 13.9%, 54.8%의 성능 향상을 달성하였으며, 연구 확산을 위해 모델을 공개하였습니다.

arXiv:2606.06682v1cs.LG

그래프 정보 기반 플로우 매칭을 이용한 시공간 결측치 대체

Spatiotemporal Imputation with Graph-Informed Flow Matching

결측 데이터는 대기질 모니터링 및 도시 교통 관리와 같은 응용 분야에서 발생하는 시공간 시스템의 일반적인 문제입니다. 순환 신경망 및 그래프 신경망과 같은 전통적인 머신러닝 접근 방식은 반복적인 전파에 의존하는데, 이는 시간과 공간에 걸쳐 오류를 누적하는 경향이 있습니다. 최근 확산 기반 방법들은 오류 전파를 완화하지만, 반복적인 샘플링을 요구하며 종종 문제 비의존적인 가우시안 사전 분포에 의존하여 효율성과 효과성을 제한합니다. 이러한 제한 사항을 해결하기 위해, 우리는 시공간 결측치 대체를 위한 그래프 정보 기반 플로우 매칭 프레임워크인 GiFlow를 제안합니다. GiFlow는 관측 가능한 신호의 시공간 필터링을 통해 구성된 그래프 정보 기반 사전 분포로 전형적인 가우시안 사전 분포를 대체하여, 소스 분포를 타겟에 더 잘 정렬하고 생성 궤적을 단순화합니다. 플로우 필드는 공간 어텐션, 시간 어텐션 및 시공간 전파를 통합하는 하이브리드 벡터 필드 모델로 매개변수화되어, 공간 및 시간 의존성을 공동으로 모델링할 수 있습니다. 합성 및 실제 데이터셋에 대한 광범위한 실험을 통해 제안된 GiFlow가 시공간 결측치 대체에서 최첨단 접근 방식보다 우수함을 입증했습니다. 코드는 https://github.com/zepengzhang/GiFlow 에서 확인할 수 있습니다.

arXiv:2606.28409v1cs.AR

Evidence-Driven LLM Agent for C-to-Synthesizable-C Conversion and Verification

Software-compilable C programs routinely fail to complete the four-stage pipeline of a high-level synthesis (HLS) toolchain -- compilation, C simulation (CSim), synthesis, and C/RTL co-simulation (CoSim) -- because HLS accepts only a synthesizable subset of C (HLS-C). Yet most existing large language model (LLM) systems built for HLS code repair only cover the early pipeline stages and feed raw tool logs directly to the model, yielding brittle and hard-to-reproduce fixes. We formulate C-to-HLS-C conversion as a closed-loop generation-verification-diagnosis-repair problem on an HLS tool (Xilinx Vitis), contributing three components: an end-to-end workflow of cooperating agents closed by the four-stage verifier under strict evidence isolation; a Progressive Mismatch Localization Chain (PMLC) that localizes CSim/CoSim mismatches through log normalization, AST backward slicing, and dual-trace instrumentation; and a typed-query, two-stage evidence RAG backed by a self-evolving, family-routed repair-card pool. Experimental results show that the proposed workflow substantially outperforms all comparable state-of-the-art models.

arXiv:2608.18573v1cs.CV

PATE-Forensics: Perception-as-Tool for Explainable Deepfake Forensics with General-Purpose MLLMs

Existing explainable deepfake forensic methods typically rely on task-adapted MLLM to jointly address detection, localization, and explanation. Inspired by agent-style tool use, we instead introduce a Perception-as-Tool paradigm and instantiate it as PATE-Forensics, which architecturally decouples detection and localization from explanation generation while coupling detection and localization as tightly as possible within a forensic perception tool. The DINOv3-based tool couples a multi-granularity detection module that integrates global, patch-level, and segment-level evidence with a cue-guided localization module by spatializing the patch-level and segment-level evidence into forgery score maps that guide dense mask prediction. The original image and forensic perception outputs produced by the tool form structured forensic context for a general-purpose MLLM, which is guided by prompt constraints to generate explanations without task-specific fine-tuning. On DDL-X Track 3, PATE-Forensics achieves the best official score of 0.89, outperforming the second-ranked team by 0.19 points. Our code is available at https://github.com/yqli00000/PATE-Forensics.

arXiv:2602.06268v2cs.CL

MPIB: LLM의 의료 프롬프트 인젝션 공격 및 임상 안전성 벤치마크

MPIB: A Benchmark for Medical Prompt Injection Attacks and Clinical Safety in LLMs

대규모 언어 모델(LLM)과 검색-증강 생성(RAG) 시스템이 임상 워크플로에 점점 더 통합되고 있다. 그러나 프롬프트 인젝션 공격은 이러한 시스템을 임상적으로 안전하지 않거나 오해를 부르는 출력으로 유도할 수 있다. 우리는 임상적으로 근거 있는 과업 전반에서 직접 프롬프트 인젝션과 간접적·RAG 매개 인젝션 모두 아래의 임상 안전성을 평가하기 위한 데이터셋 및 벤치마크 모음인 Medical Prompt Injection Benchmark(MPIB)를 제안한다. MPIB는 임상적으로 근거 있는 분류체계 하에서 고심각도 임상 유해 사건을 측정하는 Clinical Harm Event Rate(CHER)를 통해 결과 수준의 위험을 강조하며, 중등도 이상과 고심각도 결과를 구분하기 위해 ASR$_2$와 함께 CHER를 보고한다. 이 벤치마크는 다단계 품질 게이트와 임상 안전성 린팅을 거쳐 구성된 9,697개의 선별 인스턴스로 이루어진다. 다양한 기준선 LLM 및 방어 설정에 걸쳐 MPIB를 평가한 결과, ASR$_2$와 CHER$_3$가 상당히 달라질 수 있으며, 관찰된 비율은 적대적 지시가 사용자 쿼리에 나타나는지 검색된 컨텍스트에 나타나는지에 따라 달라짐을 확인했다. MPIB 데이터셋과 평가 코드는 각각 Hugging Face와 GitHub에서 이용할 수 있다.

arXiv:2609.02731v1cs.CV

RVSD: 대규모 비전-언어 모델의 시각적 환각 완화를 위한 검색 기반 비전 희소 디코딩

RVSD: Retrieval Vision Sparse Decoding for Mitigating Visual Hallucinations in Large Vision-Language Models

대규모 비전-언어 모델은 비전-언어 과제에서 눈에 띄는 성공을 거두었다. 그러나 이러한 모델은 여전히 시각적 환각(Visual Hallucinations, VHs)에 취약하여, 실제 응용에서의 신뢰성을 저해한다. 기존 해결책은 일반적으로 선별된 데이터셋, 추가 학습 또는 다중 라운드 디코딩을 필요로 하며, 이로 인해 상당한 연산 오버헤드가 발생한다. 본 논문에서는 학습이 필요 없고(training-free) 플러그 앤 플레이 방식인 디코딩 프레임워크 \textbf{RVSD}(\underline{R}etrieval \underline{V}ision \underline{S}parse \underline{D}ecoding)를 제안한다. RVSD는 토큰 희소화(token sparsification)와 \textbf{의미 공간 시각 검색}(Semantic-Space Visual Retrieval, SSVR)을 단일 디코딩 과정 내에서 최초로 통합한다. RVSD에서는 중요한 시각 정보를 보존하면서 중복된 토큰을 선택적으로 희소화하는 \textbf{의미 지향 토큰 선택}(semantics-directed token selection) 전략을 도입한다. 또한 시각적 보상(visual compensation)을 공유 의미 공간 내에서 필요할 때 수행되는 교차 모달 검색 과정으로 재정식화하는 SSVR 메커니즘을 제안한다. 폭넓은 실험을 통해 RVSD가 VHs 완화에서 최고 수준(state-of-the-art)의 성능을 달성하며, 긴 컨텍스트 생성 설정에서도 견고한 억제 성능을 유지함을 보인다. 코드는 여기에서 확인할 수 있다.\footnote{https://github.com/canjie-liu/RVSD}

arXiv:2609.10629v1cs.AI

Automating Quadratic Unconstrained Binary Optimization (QUBO) Formulation Generation from Natural Language

Quadratic Unconstrained Binary Optimization (QUBO) is a central formulation for combinatorial optimization and has gained increasing attention due to its compatibility with quantum, hybrid quantum-classical, and quantum-inspired solvers. However, translating natural-language problem descriptions into correct QUBO formulations remains difficult, requiring the identification of binary variables, constraints, objective functions, penalty terms, and suitable penalty weights. This process is time-consuming and often demands substantial domain expertise. To address this challenge, we propose an end-to-end multi-agent framework that automatically generates QUBO formulations from natural-language problem descriptions, supported by structured or unstructured test cases. To evaluate its performance, We also introduce QUBOBench, a benchmark containing 100 combinatorial optimization problems across 12 application domains, curated from peer-reviewed literature, competitions, and canonical NP-hard problems. Experimental results show that our framework achieves 68% accuracy on QUBOBench, outperforming a direct single-call baseline by 22%. Further analysis identifies iterative self-repair as the most important component contributing to improved performance. The data and code are open-sourced at https://quitttcat.github.io/QuantumQUBOAgent.

arXiv:2611.06542v1cs.LG

EvoOPD: Evolving Privileged Information Strategies for Black-Box On-Policy Distillation

Proprietary language models exhibit strong reasoning and coding capabilities, but their output distributions are inaccessible, making fine-grained distillation methods that rely on teacher logits difficult to apply directly to black-box models. We observe that rich textual feedback from a black-box teacher can be treated as privileged information (PI) and converted, through a PI-conditioned privileged teacher, into dense supervision signals on the student's own trajectories. However, what PI should be acquired from the teacher to most effectively support student learning remains unclear. PI strategies that differ in where they focus, what information they request, and how much detail they provide can have substantially different effects on the current student, and their relative effectiveness can further change as training progresses. Based on this observation, we propose Evolving Privileged Information Strategies for Black-Box On-Policy Distillation (EvoOPD), which treats PI acquisition itself as an optimization problem and maintains a PI Strategy Bank that evolves with the student state. EvoOPD uses the measured repair utility of PI on the current student to adapt the focus, information type, and granularity of teacher feedback, allowing the acquired information to continually match the student's learning needs. We systematically evaluate EvoOPD on mathematical reasoning and code generation with Qwen3-1.7B, 4B, and 8B students. EvoOPD achieves the best overall performance in all six model-task settings, improving over the strongest baseline in each setting by an average of 2.14 percentage points in mathematics and 1.30 points in code. Further analysis shows that the relative effectiveness of PI acquisition strategies changes with student state, making a fixed strategy difficult to sustain as the best choice throughout distillation.

arXiv:1805.09987공식 코드 · Papers with Code

Learning from Multi-domain Artistic Images for Arbitrary Style Transfer

We propose a fast feed-forward network for arbitrary style transfer, which can generate stylized image for previously unseen content and style image pairs. Besides the traditional content and style representation based on deep features and statistics for textures, we use adversarial networks to regularize the generation of stylized images. Our adversarial network learns the intrinsic property of image styles from large-scale multi-domain artistic images. The adversarial training is challenging because both the input and output of our generator are diverse multi-domain images. We use a conditional generator that stylized content by shifting the statistics of deep features, and a conditional discriminator based on the coarse category of styles. Moreover, we propose a mask module to spatially decide the stylization level and stabilize adversarial training by avoiding mode collapse. As a side effect, our trained discriminator can be applied to rank and select representative stylized images. We qualitatively and quantitatively evaluate the proposed method, and compare with recent style transfer methods. We release our code and model at https://github.com/nightldj/behance_release.

arXiv:2005.03922공식 코드 · Papers with Code

Learning Generalized Spoof Cues for Face Anti-spoofing

Many existing face anti-spoofing (FAS) methods focus on modeling the decision boundaries for some predefined spoof types. However, the diversity of the spoof samples including the unknown ones hinders the effective decision boundary modeling and leads to weak generalization capability. In this paper, we reformulate FAS in an anomaly detection perspective and propose a residual-learning framework to learn the discriminative live-spoof differences which are defined as the spoof cues. The proposed framework consists of a spoof cue generator and an auxiliary classifier. The generator minimizes the spoof cues of live samples while imposes no explicit constraint on those of spoof samples to generalize well to unseen attacks. In this way, anomaly detection is implicitly used to guide spoof cue generation, leading to discriminative feature learning. The auxiliary classifier serves as a spoof cue amplifier and makes the spoof cues more discriminative. We conduct extensive experiments and the experimental results show the proposed method consistently outperforms the state-of-the-art methods. The code will be publicly available at https://github.com/vis-var/lgsc-for-fas.

arXiv:2212.04638

FLAG3D: A 3D Fitness Activity Dataset with Language Instruction

With the continuously thriving popularity around the world, fitness activity analytic has become an emerging research topic in computer vision. While a variety of new tasks and algorithms have been proposed recently, there are growing hunger for data resources involved in high-quality data, fine-grained labels, and diverse environments. In this paper, we present FLAG3D, a large-scale 3D fitness activity dataset with language instruction containing 180K sequences of 60 categories. FLAG3D features the following three aspects: 1) accurate and dense 3D human pose captured from advanced MoCap system to handle the complex activity and large movement, 2) detailed and professional language instruction to describe how to perform a specific activity, 3) versatile video resources from a high-tech MoCap system, rendering software, and cost-effective smartphones in natural environments. Extensive experiments and in-depth analysis show that FLAG3D contributes great research value for various challenges, such as cross-domain human action recognition, dynamic human mesh recovery, and language-guided human action generation. Our dataset and source code are publicly available at https://andytang15.github.io/FLAG3D.