PyTorchKR Papers

“Code Generation”

총 10,140편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2505.20935v4cs.CV

ISAC: 다중 인스턴스 생성을 위한 학습 불필요 인스턴스-의미론적 어텐션 제어

ISAC: Training-Free Instance-to-Semantic Attention Control for Multi-Instance Generation

최근 공개 가중치 텍스트-이미지(T2I) 확산 모델은 다중 인스턴스 프롬프트를 처리하는 데 여전히 어려움을 겪으며, 유사한 객체들 사이에서 인스턴스를 누락하거나 병합하고 의미론적 혼합이 발생하는 문제가 있다. 우리는 이러한 실패의 원인을 인스턴스 경계가 안정적으로 확립되기 이전인 초기 디노이징 단계에서 찾는다. 기존의 학습 불필요 가이던스는 대체로 크로스-어텐션이나 다른 토큰 조건부 의미론적 신호에 의해 구동된다. 이러한 가이던스는 토큰 수준에서 개념을 분리할 수 있지만, 별개의 인스턴스 영역이 이미 형성되어 있다는 것을 대체로 가정한다. 초기 디노이징 단계에서는 이러한 영역을 신뢰할 수 있게 분리해 낼 수 없으므로, 개수 오류와 의미론적 혼합 문제가 지속된다. 반면, 셀프-어텐션은 초기 디노이징 과정에서 클래스에 무관한 인스턴스 레이아웃을 드러낸다. 이러한 비대칭성을 활용하기 위해, 우리는 학습 불필요·모델 불가지론적 목적 함수인 $\textbf{ISAC}$($\textbf{I}$nstance-to-$\textbf{S}$emantic $\textbf{A}$ttention $\textbf{C}$ontrol)를 제안한다. ISAC는 미세조정이나 외부 비전 모델 없이 셀프-어텐션 레이아웃을 먼저 안정화한 뒤, 그 안에서 크로스-어텐션 의미론을 결합한다. T2I-CompBench, HRS-Bench, 그리고 새롭게 구축한 IntraCompBench 전반에 걸쳐 ISAC는 기존의 학습 불필요 방법들을 일관되게 능가한다. 나아가 ISAC는 거칠고 겹치는 바운딩 박스를 밀집된 인스턴스 마스크로 정제함으로써 레이아웃-이미지 컨트롤러의 성능을 향상시킨다. 코드 및 IntraCompBench는 https://shjo-april.github.io/ISAC에서 이용 가능하다.

arXiv:2607.20125v1cs.CV

HeadCast: 효율적인 자기회귀 비디오 생성을 위한 어텐션 헤드 캐스팅

HeadCast: Casting Attention Heads for Efficient Autoregressive Video Generation

자기회귀(AR) 비디오 확산(diffusion) 모델은 긴 영상 및 스트리밍 비디오 합성을 위한 유망한 패러다임이 되었지만, 지속적으로 증가하는 Key-Value(KV) 캐시로 인해 어텐션이 추론 비용의 지배적 요인이 되며, 특히 각 프레임이 많은 토큰을 기여하는 고해상도에서 더욱 그렇다. 기존 해결책은 거친 휴리스틱으로 캐시를 제거(evict)하여 프레임 간 깜빡임(flickering)을 유발하거나, 모델 재학습을 요구한다. 우리는 사전학습된 AR 모델의 어텐션 헤드가 안정적이고 이질적인(heterogeneous) 행동을 보인다는 관찰에 기반한, 학습 불필요(training-free), 플러그 앤 플레이 가속 프레임워크인 HeadCast를 제안한다. 짧은 워밍업(warm-up) 후, HeadCast는 최대 노이즈(maximum-noise) 단계에서 일회성 분류를 수행하여 모든 헤드를 Sink, Dummy, Spatial, Global 네 가지 원형(archetype) 중 하나로 분류하고, 단일(monolithic) KV 캐시를 헤드별 경로(pathway)로 재구성한다. 핵심적으로, 공격적인 캐시 제거가 파괴하는 장거리 시간적 일관성(long-range temporal consistency)을 보존하는 Global 헤드를 유지한다. Spatial 경로는 고정 크기 그리드에서 동작하므로 해상도가 높을수록 절감 효과가 커진다. 최신(state-of-the-art) AR 모델 전반에서 HeadCast는 720P에서 최대 1.62배, 1080P에서 1.95배 추론을 가속하며, VBench 품질을 전체 어텐션(full attention)과 동등하게 유지하고 대체로 깜빡임 없이(flicker-free) 동작한다. 코드는 https://github.com/sjlgaga/HeadCast 에서 이용 가능하다.

arXiv:2608.03653v1cs.AI

AutoSND: 자동화된 네트워크 해체 휴리스틱 탐색을 위한 실행 증거 기반 구조적 정책 프레임워크

AutoSND: From Execution Evidence to Structural Policies for Automated Network Dismantling Heuristic Discovery

네트워크 해체(network dismantling)는 복잡계의 강건성과 취약성을 분석하는 데 기초적이지만, 실용적인 휴리스틱은 효과와 계산 효율성의 균형을 맞추어야 하며 대개 연구자들에 의해 수작업으로 설계됩니다. 대규모 언어 모델(LLM)에 기반한 기존의 자동 휴리스틱 설계 방법들은 후보를 생성하고 선별할 수는 있지만, 후보의 품질이나 실행 중의 실패 상태를 후속 생성을 위한 구조적 수준의 지침으로 변환하는 데 어려움을 겪습니다. 본 논문에서는 완전한 네트워크 해체 프로그램을 위한 3단계 트리 탐색 프레임워크인 AutoSND를 제안합니다. 1단계는 단순한 휴리스틱으로부터 폭넓게 탐색하며 실행 증거를 보관합니다. 2단계는 국소적 신호, 이웃 접근, 상태 업데이트 범위에 관한 구조적 정책으로 후보 기록들을 컴파일합니다. 3단계는 이러한 정책을 조건으로 트리 탐색을 이어가며 최종적으로 품질 우선 및 속도 우선 후보인 AutoSND-Q/S를 도출합니다. 12개의 실제 네트워크 및 3개의 대규모 실제 네트워크를 대상으로 한 실험 결과, AutoSND가 더 우수한 탐색 성능과 안정성을 달성하며 더 경쟁력 있고 구조적으로 해석 가능한 네트워크 해체 프로그램을 발견함을 보여줍니다. 최종 후보들은 잔여 차수(residual degree)를 백본으로 사용하고, 유계 국소 신호로 노드 순서를 조정하며, 상태 업데이트 범위를 제한하는 해석 가능한 구조를 형성합니다. 코드는 https://github.com/MirrorNew/AutoSND 에서 확인할 수 있습니다.

arXiv:2608.28389v1cs.CR

CamoDocs: 위장 문서를 이용한 검색-증강 언어 모델 대상 포이즈닝 공격

CamoDocs: A Poisoning Attack Against Retrieval-Augmented Language Models Using Camouflaged Documents

검색-증augmented generation (RAG)은 외부 문서로 LLM을 보강하지만, 공개되거나 사용자가 편집 가능한 소스는 RAG 시스템을 데이터 포이즈닝(data poisoning)에 노출시킨다. 공격자는 악성 문서를 주입하여 출력을 목표 답변 쪽으로 유도할 수 있다. 기존 포이즈닝 공격은 종종 쿼리 포함(query inclusion)에 의존하여 검색 성능을 높이기 위해 독 문서에 목표 쿼리를 삽입하지만, 이는 어휘적 및 임베딩 공간 아티팩트를 만들어 단순 필터링으로 걸러내기 쉽다. 우리는 CamoDocs를 제안한다. 이는 적대적 문서를 정상 콘텐츠 사이에 위장하여 직접적인 쿼리 포함을 피하는 포이즈닝 공격이다. CamoDocs는 합성된 정상 및 적대적 초안을 청크로 나누고, 정상 청크의 선택된 토큰을 독 문서 임베딩을 분산시키는 dispersion token으로 교체하며, 가독성 저하를 제한하기 위해 coherence filtering을 적용한다. 7개의 RAG 방어 기법, 3개의 오픈 가중치 LLM, 3개의 벤치마크 전반에서 CamoDocs는 단순 쿼리 탐지가 이용하는 쿼리 중복 아티팩트를 피하면서 높은 평균 공격 성공률(ASR)을 달성한다. 독점 모델에 대해서도 효과적이며, GPT-5.4-mini에서 평균 ASR 61.80%, Claude-Haiku-4.5에서 55.09%를 달성한다. 마지막으로, TrustRAG와 같은 삭제 중심 클러스터링 방어는 ASR을 줄일 수 있지만, NeoQA와 같이 검색에 의존하는 벤치마크에서는 유틸리티가 크게 떨어지는 대가를 치를 때만 그렇다는 것을 보인다. 코드는 https://github.com/jaewonalive/CamoDocs 에서 이용할 수 있다.

arXiv:2610.09021v1cs.AI

모든 호출에 프론티어 모델이 필요하지는 않다: 배포된 에이전트 기반 홈 오토메이션 시스템에서 소형 언어 모델의 호출 지점별 평가

Not Every Call Needs a Frontier Model: Per-Call-Site Evaluation of Small Language Models in a Deployed Agentic Home-Automation System

에이전트(agent) 시스템은 구조적으로 다른 여러 유형의 대규모 언어 모델(LLM) 호출을 수행합니다. 시스템은 의도를 라우팅하고, 동작을 분류하며, 디바이스 레지스트리에 언어를 그라운딩(grounding)하고, 다중 에이전트 파이프라인을 계획하며, 이러한 파이프라인이 실행하는 Python 코드를 작성합니다. 이러한 호출 지점들의 난이도는 최대 한 자릿수(order of magnitude)까지 차이가 나지만, 실제로는 가장 어려운 지점에 맞춰 선택된 단일 모델(model)이 모든 호출을 처리합니다. 본 연구에서는 배포된 오픈소스 홈 오토메이션 프레임워크(Wactorz)의 5개 호출 지점에 대해 수정되지 않은 프로덕션 프롬프트와 두 개의 실제 Home Assistant 설치 환경(280개 케이스, 2,520회 채점 호출)을 사용하여 0.8B부터 호스팅된 프론티어 모델에 이르는 9개 모델을 평가합니다. 연구 결과, 모든 지점에서 성능 순위가 동일하게 유지되지 않으며, 더 큰 모델이 일률적으로 더 우수하지는 않음을 확인했습니다. 한 4B 모델은 그라운딩된 작동(grounded actuation)에서 동급 계열의 2B 모델보다 성능이 낮았습니다. 대응 표본 검정 결과, 가장 우수한 로컬 모델은 5개 지점 중 4개 지점에서 두 호스팅 모델과 통계적으로 유의미한 차이가 없었습니다. 소형 호스팅 모델(p = 0.039) 및 프론티어 모델(p = 0.002)과의 차이는 오직 코드 생성 지점에서만 나타났습니다. 또한 종합 정확도는 작동 영역에 특화된 안전성(safety) 실패를 가리는데, 소형 모델들은 정확도/거부 간의 절충(trade-off)을 비정상적인 방식으로 해결했습니다. 즉, 한 모델(Gemma4 E2B)은 해당 사이트에 존재하지 않는 디바이스에 대한 요청의 87.2%에서 작동을 수행한 반면, 다른 모델은 수신된 모든 요청을 거부했습니다. 각 지점을 최적의 로컬 모델로 라우팅하면 호출당 비용 없이 95.4% 대비 91.8%의 성능을 달성합니다. 사용자가 평가한 실제 라이브 배포에서는 2개의 생성형 지점만 호스팅할 경우 비용의 28%만으로 모든 지점을 호스팅하는 것과 동일한 결과(39/43 대비 39/43)를 보였으며, 벤치마크가 예측했던 작동 격차는 26건 중 정확히 1건에서만 나타났습니다. 벤치마크, 평가 하네스 및 모든 기록은 https://github.com/waldiez/slm-callsite-eval 에 공개되어 있습니다.

arXiv:2003.09660공식 코드 · Papers with Code

NeuCrowd: Neural Sampling Network for Representation Learning with Crowdsourced Labels

Representation learning approaches require a massive amount of discriminative training data, which is unavailable in many scenarios, such as healthcare, smart city, education, etc. In practice, people refer to crowdsourcing to get annotated labels. However, due to issues like data privacy, budget limitation, shortage of domain-specific annotators, the number of crowdsourced labels is still very limited. Moreover, because of annotators' diverse expertise, crowdsourced labels are often inconsistent. Thus, directly applying existing supervised representation learning (SRL) algorithms may easily get the overfitting problem and yield suboptimal solutions. In this paper, we propose \emph{NeuCrowd}, a unified framework for SRL from crowdsourced labels. The proposed framework (1) creates a sufficient number of high-quality \emph{n}-tuplet training samples by utilizing safety-aware sampling and robust anchor generation; and (2) automatically learns a neural sampling network that adaptively learns to select effective samples for SRL networks. The proposed framework is evaluated on both one synthetic and three real-world data sets. The results show that our approach outperforms a wide range of state-of-the-art baselines in terms of prediction accuracy and AUC. To encourage reproducible results, we make our code publicly available at \url{https://github.com/tal-ai/NeuCrowd_KAIS2021}.

arXiv:2110.01232공식 코드 · Papers with Code

Benchmarking Safety Monitors for Image Classifiers with Machine Learning

High-accurate machine learning (ML) image classifiers cannot guarantee that they will not fail at operation. Thus, their deployment in safety-critical applications such as autonomous vehicles is still an open issue. The use of fault tolerance mechanisms such as safety monitors is a promising direction to keep the system in a safe state despite errors of the ML classifier. As the prediction from the ML is the core information directly impacting safety, many works are focusing on monitoring the ML model itself. Checking the efficiency of such monitors in the context of safety-critical applications is thus a significant challenge. Therefore, this paper aims at establishing a baseline framework for benchmarking monitors for ML image classifiers. Furthermore, we propose a framework covering the entire pipeline, from data generation to evaluation. Our approach measures monitor performance with a broader set of metrics than usually proposed in the literature. Moreover, we benchmark three different monitor approaches in 79 benchmark datasets containing five categories of out-of-distribution data for image classifiers: class novelty, noise, anomalies, distributional shifts, and adversarial attacks. Our results indicate that these monitors are no more accurate than a random monitor. We also release the code of all experiments for reproducibility.

arXiv:2302.02651공식 코드 · Papers with Code

1st Place Solution for PSG competition with ECCV'22 SenseHuman Workshop

Panoptic Scene Graph (PSG) generation aims to generate scene graph representations based on panoptic segmentation instead of rigid bounding boxes. Existing PSG methods utilize one-stage paradigm which simultaneously generates scene graphs and predicts semantic segmentation masks or two-stage paradigm that first adopt an off-the-shelf panoptic segmentor, then pairwise relationship prediction between these predicted objects. One-stage approach despite having a simplified training paradigm, its segmentation results are usually under-satisfactory, while two-stage approach lacks global context and leads to low performance on relation prediction. To bridge this gap, in this paper, we propose GRNet, a Global Relation Network in two-stage paradigm, where the pre-extracted local object features and their corresponding masks are fed into a transformer with class embeddings. To handle relation ambiguity and predicate classification bias caused by long-tailed distribution, we formulate relation prediction in the second stage as a multi-class classification task with soft label. We conduct comprehensive experiments on OpenPSG dataset and achieve the state-of-art performance on the leadboard. We also show the effectiveness of our soft label strategy for long-tailed classes in ablation studies. Our code has been released in https://github.com/wangqixun/mfpsg.

arXiv:2303.12688공식 코드 · Papers with Code

Pix2Video: Video Editing using Image Diffusion

Image diffusion models, trained on massive image collections, have emerged as the most versatile image generator model in terms of quality and diversity. They support inverting real images and conditional (e.g., text) generation, making them attractive for high-quality image editing applications. We investigate how to use such pre-trained image models for text-guided video editing. The critical challenge is to achieve the target edits while still preserving the content of the source video. Our method works in two simple steps: first, we use a pre-trained structure-guided (e.g., depth) image diffusion model to perform text-guided edits on an anchor frame; then, in the key step, we progressively propagate the changes to the future frames via self-attention feature injection to adapt the core denoising step of the diffusion model. We then consolidate the changes by adjusting the latent code for the frame before continuing the process. Our approach is training-free and generalizes to a wide range of edits. We demonstrate the effectiveness of the approach by extensive experimentation and compare it against four different prior and parallel efforts (on ArXiv). We demonstrate that realistic text-guided video edits are possible, without any compute-intensive preprocessing or video-specific finetuning.

arXiv:2304.10103

eTag: Class-Incremental Learning with Embedding Distillation and Task-Oriented Generation

Class-Incremental Learning (CIL) aims to solve the neural networks' catastrophic forgetting problem, which refers to the fact that once the network updates on a new task, its performance on previously-learned tasks drops dramatically. Most successful CIL methods incrementally train a feature extractor with the aid of stored exemplars, or estimate the feature distribution with the stored prototypes. However, the stored exemplars would violate the data privacy concerns, while the stored prototypes might not reasonably be consistent with a proper feature distribution, hindering the exploration of real-world CIL applications. In this paper, we propose a method of \textit{e}mbedding distillation and \textit{Ta}sk-oriented \textit{g}eneration (\textit{eTag}) for CIL, which requires neither the exemplar nor the prototype. Instead, eTag achieves a data-free manner to train the neural networks incrementally. To prevent the feature extractor from forgetting, eTag distills the embeddings of the network's intermediate blocks. Additionally, eTag enables a generative network to produce suitable features, fitting the needs of the top incremental classifier. Experimental results confirmed that our proposed eTag considerably outperforms the state-of-the-art methods on CIFAR-100 and ImageNet-sub\footnote{Our code is available in the Supplementary Materials.

arXiv:2305.18453공식 코드 · Papers with Code

Conditional Diffusion Models for Semantic 3D Brain MRI Synthesis

Artificial intelligence (AI) in healthcare, especially in medical imaging, faces challenges due to data scarcity and privacy concerns. Addressing these, we introduce Med-DDPM, a diffusion model designed for 3D semantic brain MRI synthesis. This model effectively tackles data scarcity and privacy issues by integrating semantic conditioning. This involves the channel-wise concatenation of a conditioning image to the model input, enabling control in image generation. Med-DDPM demonstrates superior stability and performance compared to existing 3D brain imaging synthesis methods. It generates diverse, anatomically coherent images with high visual fidelity. In terms of dice score accuracy in the tumor segmentation task, Med-DDPM achieves 0.6207, close to the 0.6531 accuracy of real images, and outperforms baseline models. Combined with real images, it further increases segmentation accuracy to 0.6675, showing the potential of our proposed method for data augmentation. This model represents the first use of a diffusion model in 3D semantic brain MRI synthesis, producing high-quality images. Its semantic conditioning feature also shows potential for image anonymization in biomedical imaging, addressing data and privacy issues. We provide the code and model weights for Med-DDPM on our GitHub repository (https://github.com/mobaidoctor/med-ddpm/) to support reproducibility.

arXiv:2310.15904공식 코드 · Papers with Code

Do Stochastic Parrots have Feelings Too? Improving Neural Detection of Synthetic Text via Emotion Recognition

Recent developments in generative AI have shone a spotlight on high-performance synthetic text generation technologies. The now wide availability and ease of use of such models highlights the urgent need to provide equally powerful technologies capable of identifying synthetic text. With this in mind, we draw inspiration from psychological studies which suggest that people can be driven by emotion and encode emotion in the text they compose. We hypothesize that pretrained language models (PLMs) have an affective deficit because they lack such an emotional driver when generating text and consequently may generate synthetic text which has affective incoherence i.e. lacking the kind of emotional coherence present in human-authored text. We subsequently develop an emotionally aware detector by fine-tuning a PLM on emotion. Experiment results indicate that our emotionally-aware detector achieves improvements across a range of synthetic text generators, various sized models, datasets, and domains. Finally, we compare our emotionally-aware synthetic text detector to ChatGPT in the task of identification of its own output and show substantial gains, reinforcing the potential of emotion as a signal to identify synthetic text. Code, models, and datasets are available at https: //github.com/alanagiasi/emoPLMsynth

arXiv:2312.02317공식 코드 · Papers with Code

GNN2R: Weakly-Supervised Rationale-Providing Question Answering over Knowledge Graphs

Most current methods for multi-hop question answering (QA) over knowledge graphs (KGs) only provide final conclusive answers without explanations, such as a set of KG entities that is difficult for normal users to review and comprehend. This issue severely limits the application of KG-based QA in real-world scenarios. However, it is non-trivial to solve due to two challenges: First, annotations of reasoning chains of multi-hop questions, which could serve as supervision for explanation generation, are usually lacking. Second, it is difficult to maintain high efficiency when explicit KG triples need to be retrieved to generate explanations. In this paper, we propose a novel Graph Neural Network-based Two-Step Reasoning model (GNN2R) to solve this issue. GNN2R can provide both final answers and reasoning subgraphs as a rationale behind final answers efficiently with only weak supervision that is available through question-final answer pairs. We extensively evaluated GNN2R with detailed analyses in experiments. The results demonstrate that, in terms of effectiveness, efficiency, and quality of generated explanations, GNN2R outperforms existing state-of-the-art methods that are applicable to this task. Our code and pre-trained models are available at https://github.com/ruijie-wang-uzh/GNN2R.

arXiv:2404.00876공식 코드 · Papers with Code

MGMap: Mask-Guided Learning for Online Vectorized HD Map Construction

Currently, high-definition (HD) map construction leans towards a lightweight online generation tendency, which aims to preserve timely and reliable road scene information. However, map elements contain strong shape priors. Subtle and sparse annotations make current detection-based frameworks ambiguous in locating relevant feature scopes and cause the loss of detailed structures in prediction. To alleviate these problems, we propose MGMap, a mask-guided approach that effectively highlights the informative regions and achieves precise map element localization by introducing the learned masks. Specifically, MGMap employs learned masks based on the enhanced multi-scale BEV features from two perspectives. At the instance level, we propose the Mask-activated instance (MAI) decoder, which incorporates global instance and structural information into instance queries by the activation of instance masks. At the point level, a novel position-guided mask patch refinement (PG-MPR) module is designed to refine point locations from a finer-grained perspective, enabling the extraction of point-specific patch information. Compared to the baselines, our proposed MGMap achieves a notable improvement of around 10 mAP for different input modalities. Extensive experiments also demonstrate that our approach showcases strong robustness and generalization capabilities. Our code can be found at https://github.com/xiaolul2/MGMap.

arXiv:2407.09007공식 코드 · Papers with Code

Benchmarking Language Model Creativity: A Case Study on Code Generation

As LLMs become increasingly prevalent, it is interesting to consider how ``creative'' these models can be. From cognitive science, creativity consists of at least two key characteristics: \emph{convergent} thinking (purposefulness to achieve a given goal) and \emph{divergent} thinking (adaptability to new environments or constraints) \citep{runco2003critical}. In this work, we introduce a framework for quantifying LLM creativity that incorporates the two characteristics. This is achieved by (1) Denial Prompting pushes LLMs to come up with more creative solutions to a given problem by incrementally imposing new constraints on the previous solution, compelling LLMs to adopt new strategies, and (2) defining and computing the NeoGauge metric which examines both convergent and divergent thinking in the generated creative responses by LLMs. We apply the proposed framework on Codeforces problems, a natural data source for collecting human coding solutions. We quantify NeoGauge for various proprietary and open-source models and find that even the most creative model, GPT-4, still falls short of demonstrating human-like creativity. We also experiment with advanced reasoning strategies (MCTS, self-correction, etc.) and observe no significant improvement in creativity. As a by-product of our analysis, we release NeoCoder dataset for reproducing our results on future models.

arXiv:2307.12868공식 코드 · Papers with Code

Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometry

Despite the success of diffusion models (DMs), we still lack a thorough understanding of their latent space. To understand the latent space $\mathbf{x}_t \in \mathcal{X}$, we analyze them from a geometrical perspective. Our approach involves deriving the local latent basis within $\mathcal{X}$ by leveraging the pullback metric associated with their encoding feature maps. Remarkably, our discovered local latent basis enables image editing capabilities by moving $\mathbf{x}_t$, the latent space of DMs, along the basis vector at specific timesteps. We further analyze how the geometric structure of DMs evolves over diffusion timesteps and differs across different text conditions. This confirms the known phenomenon of coarse-to-fine generation, as well as reveals novel insights such as the discrepancy between $\mathbf{x}_t$ across timesteps, the effect of dataset complexity, and the time-varying influence of text prompts. To the best of our knowledge, this paper is the first to present image editing through $\mathbf{x}$-space traversal, editing only once at specific timestep $t$ without any additional training, and providing thorough analyses of the latent structure of DMs. The code to reproduce our experiments can be found at https://github.com/enkeejunior1/Diffusion-Pullback.

arXiv:2409.08397공식 코드 · Papers with Code

360PanT: Training-Free Text-Driven 360-Degree Panorama-to-Panorama Translation

Preserving boundary continuity in the translation of 360-degree panoramas remains a significant challenge for existing text-driven image-to-image translation methods. These methods often produce visually jarring discontinuities at the translated panorama's boundaries, disrupting the immersive experience. To address this issue, we propose 360PanT, a training-free approach to text-based 360-degree panorama-to-panorama translation with boundary continuity. Our 360PanT achieves seamless translations through two key components: boundary continuity encoding and seamless tiling translation with spatial control. Firstly, the boundary continuity encoding embeds critical boundary continuity information of the input 360-degree panorama into the noisy latent representation by constructing an extended input image. Secondly, leveraging this embedded noisy latent representation and guided by a target prompt, the seamless tiling translation with spatial control enables the generation of a translated image with identical left and right halves while adhering to the extended input's structure and semantic layout. This process ensures a final translated 360-degree panorama with seamless boundary continuity. Experimental results on both real-world and synthesized datasets demonstrate the effectiveness of our 360PanT in translating 360-degree panoramas. Code is available at \href{https://github.com/littlewhitesea/360PanT}{https://github.com/littlewhitesea/360PanT}.

arXiv:2410.00079

Interactive Speculative Planning: Enhance Agent Efficiency through Co-design of System and User Interface

Agents, as user-centric tools, are increasingly deployed for human task delegation, assisting with a broad spectrum of requests by generating thoughts, engaging with user proxies, and producing action plans. However, agents based on large language models (LLMs) often face substantial planning latency due to two primary factors: the efficiency limitations of the underlying LLMs due to their large size and high demand, and the structural complexity of the agents due to the extensive generation of intermediate thoughts to produce the final output. Given that inefficiency in service provision can undermine the value of automation for users, this paper presents a human-centered efficient agent planning method -- Interactive Speculative Planning -- aiming at enhancing the efficiency of agent planning through both system design and human-AI interaction. Our approach advocates for the co-design of the agent system and user interface, underscoring the importance of an agent system that can fluidly manage user interactions and interruptions. By integrating human interruptions as a fundamental component of the system, we not only make it more user-centric but also expedite the entire process by leveraging human-in-the-loop interactions to provide accurate intermediate steps. Code and data will be released.

arXiv:2410.02780공식 코드 · Papers with Code

Guess What I Think: Streamlined EEG-to-Image Generation with Latent Diffusion Models

Generating images from brain waves is gaining increasing attention due to its potential to advance brain-computer interface (BCI) systems by understanding how brain signals encode visual cues. Most of the literature has focused on fMRI-to-Image tasks as fMRI is characterized by high spatial resolution. However, fMRI is an expensive neuroimaging modality and does not allow for real-time BCI. On the other hand, electroencephalography (EEG) is a low-cost, non-invasive, and portable neuroimaging technique, making it an attractive option for future real-time applications. Nevertheless, EEG presents inherent challenges due to its low spatial resolution and susceptibility to noise and artifacts, which makes generating images from EEG more difficult. In this paper, we address these problems with a streamlined framework based on the ControlNet adapter for conditioning a latent diffusion model (LDM) through EEG signals. We conduct experiments and ablation studies on popular benchmarks to demonstrate that the proposed method beats other state-of-the-art models. Unlike these methods, which often require extensive preprocessing, pretraining, different losses, and captioning models, our approach is efficient and straightforward, requiring only minimal preprocessing and a few components. The code is available at https://github.com/LuigiSigillo/GWIT.

arXiv:2503.04875

Architecture for a Trustworthy Quantum Chatbot

Large language model (LLM)-based tools such as ChatGPT seem useful for classical programming assignments. The more specialized the field, the more likely they lack reliability because of the lack of data to train them. In the case of quantum computing, the quality of answers of generic chatbots is low. C4Q is a chatbot focused on quantum programs that addresses this challenge through a software architecture that integrates specialized LLMs to classify requests and specialized question answering modules with a deterministic logical engine to provide trustworthy quantum computing support. This article describes the latest version (2.0) of C4Q, which delivers several enhancements: ready-to-run Qiskit code for gate definitions and circuit operations, expanded features to solve software engineering tasks such as the travelling salesperson problem and the knapsack problem, and a feedback mechanism for iterative improvement. Extensive testing of the backend confirms the system's reliability, while empirical evaluations show that C4Q 2.0's classification LLM reaches near-perfect accuracy. The evaluation of the result consists in a comparative study with three existing chatbots highlighting C4Q 2.0's maintainability and correctness, reflecting on how software architecture decisions, such as separating deterministic logic from probabilistic text generation impact the quality of the results.