PyTorchKR Papers

“Code Generation”

총 10,140편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2409.00700

Seeing Your Speech Style: A Novel Zero-Shot Identity-Disentanglement Face-based Voice Conversion

Face-based Voice Conversion (FVC) is a novel task that leverages facial images to generate the target speaker's voice style. Previous work has two shortcomings: (1) suffering from obtaining facial embeddings that are well-aligned with the speaker's voice identity information, and (2) inadequacy in decoupling content and speaker identity information from the audio input. To address these issues, we present a novel FVC method, Identity-Disentanglement Face-based Voice Conversion (ID-FaceVC), which overcomes the above two limitations. More precisely, we propose an Identity-Aware Query-based Contrastive Learning (IAQ-CL) module to extract speaker-specific facial features, and a Mutual Information-based Dual Decoupling (MIDD) module to purify content features from audio, ensuring clear and high-quality voice conversion. Besides, unlike prior works, our method can accept either audio or text inputs, offering controllable speech generation with adjustable emotional tone and speed. Extensive experiments demonstrate that ID-FaceVC achieves state-of-the-art performance across various metrics, with qualitative and user study results confirming its effectiveness in naturalness, similarity, and diversity. Project website with audio samples and code can be found at https://id-facevc.github.io.

arXiv:2410.22366공식 코드 · Papers with Code

Unpacking SDXL Turbo: Interpreting Text-to-Image Models with Sparse Autoencoders

Sparse autoencoders (SAEs) have become a core ingredient in the reverse engineering of large-language models (LLMs). For LLMs, they have been shown to decompose intermediate representations that often are not interpretable directly into sparse sums of interpretable features, facilitating better control and subsequent analysis. However, similar analyses and approaches have been lacking for text-to-image models. We investigated the possibility of using SAEs to learn interpretable features for a few-step text-to-image diffusion models, such as SDXL Turbo. To this end, we train SAEs on the updates performed by transformer blocks within SDXL Turbo's denoising U-net. We find that their learned features are interpretable, causally influence the generation process, and reveal specialization among the blocks. In particular, we find one block that deals mainly with image composition, one that is mainly responsible for adding local details, and one for color, illumination, and style. Therefore, our work is an important first step towards better understanding the internals of generative text-to-image models like SDXL Turbo and showcases the potential of features learned by SAEs for the visual domain. Code is available at https://github.com/surkovv/sdxl-unbox

arXiv:2412.05430공식 코드 · Papers with Code

DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA

Recent advances in self-supervised models for natural language, vision, and protein sequences have inspired the development of large genomic DNA language models (DNALMs). These models aim to learn generalizable representations of diverse DNA elements, potentially enabling various genomic prediction, interpretation and design tasks. Despite their potential, existing benchmarks do not adequately assess the capabilities of DNALMs on key downstream applications involving an important class of non-coding DNA elements critical for regulating gene activity. In this study, we introduce DART-Eval, a suite of representative benchmarks specifically focused on regulatory DNA to evaluate model performance across zero-shot, probed, and fine-tuned scenarios against contemporary ab initio models as baselines. Our benchmarks target biologically meaningful downstream tasks such as functional sequence feature discovery, predicting cell-type specific regulatory activity, and counterfactual prediction of the impacts of genetic variants. We find that current DNALMs exhibit inconsistent performance and do not offer compelling gains over alternative baseline models for most tasks, while requiring significantly more computational resources. We discuss potentially promising modeling, data curation, and evaluation strategies for the next generation of DNALMs. Our code is available at https://github.com/kundajelab/DART-Eval.

arXiv:2412.12596공식 코드 · Papers with Code

OpenViewer: Openness-Aware Multi-View Learning

Multi-view learning methods leverage multiple data sources to enhance perception by mining correlations across views, typically relying on predefined categories. However, deploying these models in real-world scenarios presents two primary openness challenges. 1) Lack of Interpretability: The integration mechanisms of multi-view data in existing black-box models remain poorly explained; 2) Insufficient Generalization: Most models are not adapted to multi-view scenarios involving unknown categories. To address these challenges, we propose OpenViewer, an openness-aware multi-view learning framework with theoretical support. This framework begins with a Pseudo-Unknown Sample Generation Mechanism to efficiently simulate open multi-view environments and previously adapt to potential unknown samples. Subsequently, we introduce an Expression-Enhanced Deep Unfolding Network to intuitively promote interpretability by systematically constructing functional prior-mapping modules and effectively providing a more transparent integration mechanism for multi-view data. Additionally, we establish a Perception-Augmented Open-Set Training Regime to significantly enhance generalization by precisely boosting confidences for known categories and carefully suppressing inappropriate confidences for unknown ones. Experimental results demonstrate that OpenViewer effectively addresses openness challenges while ensuring recognition performance for both known and unknown samples. The code is released at https://github.com/dushide/OpenViewer.

arXiv:2501.18950공식 코드 · Papers with Code

Fantastic Targets for Concept Erasure in Diffusion Models and Where To Find Them

Concept erasure has emerged as a promising technique for mitigating the risk of harmful content generation in diffusion models by selectively unlearning undesirable concepts. The common principle of previous works to remove a specific concept is to map it to a fixed generic concept, such as a neutral concept or just an empty text prompt. In this paper, we demonstrate that this fixed-target strategy is suboptimal, as it fails to account for the impact of erasing one concept on the others. To address this limitation, we model the concept space as a graph and empirically analyze the effects of erasing one concept on the remaining concepts. Our analysis uncovers intriguing geometric properties of the concept space, where the influence of erasing a concept is confined to a local region. Building on this insight, we propose the Adaptive Guided Erasure (AGE) method, which \emph{dynamically} selects optimal target concepts tailored to each undesirable concept, minimizing unintended side effects. Experimental results show that AGE significantly outperforms state-of-the-art erasure methods on preserving unrelated concepts while maintaining effective erasure performance. Our code is published at {https://github.com/tuananhbui89/Adaptive-Guided-Erasure}.

arXiv:2503.15809

Controlling Avatar Diffusion with Learnable Gaussian Embedding

Recent advances in diffusion models have made significant progress in digital human generation. However, most existing models still struggle to maintain 3D consistency, temporal coherence, and motion accuracy. A key reason for these shortcomings is the limited representation ability of commonly used control signals(e.g., landmarks, depth maps, etc.). In addition, the lack of diversity in identity and pose variations in public datasets further hinders progress in this area. In this paper, we analyze the shortcomings of current control signals and introduce a novel control signal representation that is optimizable, dense, expressive, and 3D consistent. Our method embeds a learnable neural Gaussian onto a parametric head surface, which greatly enhances the consistency and expressiveness of diffusion-based head models. Regarding the dataset, we synthesize a large-scale dataset with multiple poses and identities. In addition, we use real/synthetic labels to effectively distinguish real and synthetic data, minimizing the impact of imperfections in synthetic data on the generated head images. Extensive experiments show that our model outperforms existing methods in terms of realism, expressiveness, and 3D consistency. Our code, synthetic datasets, and pre-trained models will be released in our project page: https://ustc3dv.github.io/Learn2Control/

arXiv:2503.15653공식 코드 · Papers with Code

Transport-Related Surface Detection with Machine Learning: Analyzing Temporal Trends in Madrid and Vienna

This study explores the integration of machine learning into urban aerial image analysis, with a focus on identifying infrastructure surfaces for cars and pedestrians and analyzing historical trends. It emphasizes the transition from convolutional architectures to transformer-based pre-trained models, underscoring their potential in global geospatial analysis. A workflow is presented for automatically generating geospatial datasets, enabling the creation of semantic segmentation datasets from various sources, including WMS/WMTS links, vectorial cartography, and OpenStreetMap (OSM) overpass-turbo requests. The developed code allows a fast dataset generation process for training machine learning models using openly available data without manual labelling. Using aerial imagery and vectorial data from the respective geographical offices of Madrid and Vienna, two datasets were generated for car and pedestrian surface detection. A transformer-based model was trained and evaluated for each city, demonstrating good accuracy values. The historical trend analysis involved applying the trained model to earlier images predating the availability of vectorial data 10 to 20 years, successfully identifying temporal trends in infrastructure for pedestrians and cars across different city areas. This technique is applicable for municipal governments to gather valuable data at a minimal cost.

arXiv:2503.07890

Can Generative Geospatial Diffusion Models Excel as Discriminative Geospatial Foundation Models?

Self-supervised learning (SSL) has revolutionized representation learning in Remote Sensing (RS), advancing Geospatial Foundation Models (GFMs) to leverage vast unlabeled satellite imagery for diverse downstream tasks. Currently, GFMs primarily focus on discriminative objectives, such as contrastive learning or masked image modeling, owing to their proven success in learning transferable representations. However, generative diffusion models--which demonstrate the potential to capture multi-grained semantics essential for RS tasks during image generation--remain underexplored for discriminative applications. This prompts the question: can generative diffusion models also excel and serve as GFMs with sufficient discriminative power? In this work, we answer this question with SatDiFuser, a framework that transforms a diffusion-based generative geospatial foundation model into a powerful pretraining tool for discriminative RS. By systematically analyzing multi-stage, noise-dependent diffusion features, we develop three fusion strategies to effectively leverage these diverse representations. Extensive experiments on remote sensing benchmarks show that SatDiFuser outperforms state-of-the-art GFMs, achieving gains of up to +5.7% mIoU in semantic segmentation and +7.9% F1-score in classification, demonstrating the capacity of diffusion-based generative foundation models to rival or exceed discriminative GFMs. Code will be released.

arXiv:2504.12395공식 코드 · Papers with Code

InstantCharacter: Personalize Any Characters with a Scalable Diffusion Transformer Framework

Current learning-based subject customization approaches, predominantly relying on U-Net architectures, suffer from limited generalization ability and compromised image quality. Meanwhile, optimization-based methods require subject-specific fine-tuning, which inevitably degrades textual controllability. To address these challenges, we propose InstantCharacter, a scalable framework for character customization built upon a foundation diffusion transformer. InstantCharacter demonstrates three fundamental advantages: first, it achieves open-domain personalization across diverse character appearances, poses, and styles while maintaining high-fidelity results. Second, the framework introduces a scalable adapter with stacked transformer encoders, which effectively processes open-domain character features and seamlessly interacts with the latent space of modern diffusion transformers. Third, to effectively train the framework, we construct a large-scale character dataset containing 10-million-level samples. The dataset is systematically organized into paired (multi-view character) and unpaired (text-image combinations) subsets. This dual-data structure enables simultaneous optimization of identity consistency and textual editability through distinct learning pathways. Qualitative experiments demonstrate the advanced capabilities of InstantCharacter in generating high-fidelity, text-controllable, and character-consistent images, setting a new benchmark for character-driven image generation. Our source code is available at https://github.com/Tencent/InstantCharacter.

arXiv:2506.06443공식 코드 · Papers with Code

Unlocking Chemical Insights: Superior Molecular Representations from Intermediate Encoder Layers

Pretrained molecular encoders have become indispensable in computational chemistry for tasks such as property prediction and molecular generation. However, the standard practice of relying solely on final-layer embeddings for downstream tasks may discard valuable information. In this work, we challenge this convention by conducting a comprehensive layer-wise analysis of five diverse molecular encoders across 22 ADMET property prediction tasks. Our results demonstrate that embeddings from intermediate layers consistently outperform final-layer representations. Specifically, using fixed embeddings from the optimal intermediate layers improved downstream performance by an average of 5.4%, reaching gains up to 28.6%. Furthermore, finetuning up to these intermediate layers yielded even greater average improvements of 8.5%, with performance increases as high as 40.8%, achieving new state-of-the-art results on several benchmarks. Additionally, a strong positive correlation between fixed embedding performance and finetuning outcomes supports an efficient evaluate-then-finetune approach, enabling identification of optimal layers with reduced computational cost. These findings highlight the importance of exploring the full representational depth of molecular encoders to achieve substantial performance improvements and computational efficiency. The code is made publicly available at https://github.com/luispintoc/Unlocking-Chemical-Insights.

arXiv:2607.20430v1cs.CL

LLM-INSTRUCT at UZH Shared Task 2026: 단락 수준 논증 마이닝을 위한 제약 인식 검색 및 선택적 토론

LLM-INSTRUCT at UZH Shared Task 2026: Constraint-Aware Retrieval and Selective Debate for Paragraph-Level Argument Mining

우리는 UN 및 UNESCO 결의문에서의 단락 수준 논증 마이닝을 다루는 ArgMining 2026 UZH Shared Task의 우승 시스템인 LLM-INSTRUCT를 제안한다. 이 과제는 최대 8B 파라미터의 오픈 가중치 모델만을 사용하여, 엄격한 JSON 스키마 설정 하에서 단락 유형 분류, 141개 공식 태그 중 일부의 예측, 그리고 방향성 관계 예측을 요구한다. 우리는 이 과제를 제약 기반 구조화 예측으로 프레이밍한다. 시스템은 먼저 메타데이터 인식 밀집 검색(dense retrieval)으로 후보 태그 공간을 축소한 다음, 차원별 상한이 있는 제약 디코딩(constrained decoding)을 적용하고, 불확실한 경우에만 3-에이전트 토론 분기로 에스컬레이션하며, 마지막으로 출력 스키마를 검증한다. 공식 리더보드에서 LLM-INSTRUCT는 전체 1위, F1 1위, LLM-as-a-Judge 5위를 기록했다. 개발 과정에서 구성 탐색을 통해 Task 1b Micro-F1을 35.83%에서 40.08%로 추가로 개선했으며, 내부 Task 2 점수는 4.421을 유지했다. 핵심 교훈은 간단하다: 생성 전에 결정 공간을 축소하면 정확도와 제출 견고성 모두가 향상된다. 코드와 지원 스크립트는 다음에서 공개되어 있다: https://github.com/LLM-Instruct-at-UZH-Shared-Task-2026/Method

arXiv:2607.26490v1cs.AI

EvoPINN: 물리 정보 신경망을 위한 실행 가능한 알고리즘의 에이전트 기반 발견

EvoPINN: Agentic Discovery of Executable Algorithms for Physics-Informed Neural Networks

물리 정보 신경망(PINN)은 편미분 방정식(PDE)을 해결하기 위한 강력한 패러다임으로 부상했으나, 그 성능은 신경 표현, 손실 수식화, 최적화 역학에 대한 수작업의 시행착오적 엔지니어링에 크게 의존하고 있습니다. 대규모 언어 모델(LLM)은 자동화된 설계를 위한 유망한 방안을 제공하지만, 제약 없는 코드 생성은 엄격한 과학 계산 제약 조건 하에서 종종 수학적으로 유효하지 않거나 수치적으로 불안정한 해를 낳습니다. 이러한 격차를 해소하기 위해, 본 연구에서는 PINN 개발을 노동 집약적인 수동 설계에서 엄밀하고 실행 기반(execution-grounded)의 알고리즘 발견 문제로 재구성하는 에이전트 프레임워크인 EvoPINN을 제안합니다. EvoPINN은 신경 표현을 학습 프로그램과 분리하여 모듈식 탐색 공간을 탐색하며, LLM 에이전트를 활용해 메모리 조건부 프로그램 수정을 반복적으로 제안합니다. 과학적 타당성을 보장하기 위해 모든 후보는 엄격한 구조적 검증과 계산 예산이 일치된 PDE 평가를 거칩니다. 다양한 PDE 영역(진동, 타원형, 소산성, 비선형 수송)에 걸친 광범위한 실험을 통해 EvoPINN이 베이스라인 대비 상대적 $L_{2}$ 오차를 크게 줄이는 PDE 특화 학습 알고리즘을 발견함을 입증합니다. 특히 중요한 점은, EvoPINN이 엄격한 매개변수 일치 비교에서도 성능 향상이 유지되는 새로운 아키텍처인 SLRC-PINN을 자율적으로 발명하여, 진정으로 새로운 과학 계산 메커니즘을 발견하기 위한 실행 기반 에이전트의 실현 가능성을 입증했다는 것입니다.

arXiv:2608.02833v1cs.CV

CURV: 커리큘럼 시각적 그라운딩 추론을 통한 차트 이해 능력 향상

CURV: Enhancing Chart Understanding Through Curriculum Visual Grounded Reasoning

차트 질의응답(CQA)은 멀티모달 대규모 언어 모델(MLLM)이 시각적 이해와 논리적 추론을 통합할 것을 요구하지만, 현재 모델들은 정확한 시각적 그라운딩(visual grounding)과 일관된 추론 체인을 형성하는 데 어려움을 겪고 있습니다. 외재적 사고의 연쇄(Chain-of-Thought) 프롬프트와 시각적 단서가 성능을 크게 향상시키기는 하지만, 현재 MLLM은 내재적인 시각적 그라운딩 추론 능력이 부족하여 부정확한 인식과 시각적 증거와 단절된 추론을 초래합니다. 이러한 한계를 해결하기 위해, 본 논문에서는 CQA를 각 단계마다 공간적 어텐션(attention) 집중을 통해 논리적 추론과 동적 시각적 그라운딩을 조율하는 다단계 시각적 그라운딩 추론으로 재구성함으로써 내재적 시각 추론 능력을 개발하는 커리큘럼 학습(curriculum learning) 프레임워크인 CURV를 제안합니다. 모델(model) 학습을 돕기 위해, 다양한 차트 유형과 추론 패턴에 걸쳐 확장 가능한 합성 생성을 지원하는 3단계 커리큘럼 데이터셋(dataset)인 CCQA를 추가로 소개합니다. 본 커리큘럼은 기본적인 단일 연산 추론부터 복잡한 다중 차트 구성 작업까지 체계적으로 발전합니다. 실험을 통해 CURV가 베이스라인 대비 최대 $\uparrow20.50\%$의 성능 향상을 달성하고, 실제 벤치마크(최대 $\uparrow12.30\%$) 및 도메인 외(out-of-domain) 멀티모달 추론 작업(최대 $\uparrow10.20\%$)으로의 일반화가 가능함을 보여주며, 차트 이해 능력 향상을 위해 동적 그라운딩을 갖춘 시각 추론을 내재화하는 것의 유효성을 입증합니다. 코드는 https://xhguo7.github.io/CURV/ 에서 확인할 수 있습니다.

arXiv:2609.04484v1cs.CL

생성에서의 점화 패턴: 언어 모델 생성에서의 어휘적, 의미적 및 구조적 정렬

Patterns of Priming in Production: Lexical, Semantic and Structural Alignment in Language Model Generation

본 논문은 선행하는 구조적 문맥이 문장 완성에 어떠한 영향을 미치는지 조사하며, 언어 모델(LM) 생성에서의 구조적 점화(structural priming)를 탐구합니다. 이전 연구들이 구조적 교체(structural alternation)의 이해 과정에서 점화 효과를 입증해 온 반면, 생성 시 언어 모델이 각 단계마다 수많은 가능한 후속 토큰 중에서 샘플링하는 생성 과정에서도 이러한 효과가 지속되는지는 불분명했습니다. 저자들은 여격 구문(dative construction)에 대한 일련의 통제된 문장 완성 실험을 통해 이 질문을 다룹니다. 이전 연구와 일치하게, 언어 모델은 구조적 점화에 영향을 받기 쉬우며, 특히 의미적으로 일관된 문장에서 이러한 경향이 두드러짐을 확인했습니다. 점화의 크기 측면에서는, 역빈도 효과(inverse frequency effects)와 일치하게 기준선 대비 이중 목적어 여격 구문의 상대적 증가 폭이 더 컸던 반면, 더 자주 생성되는 구문인 전치사 목적어 구문에서는 더 큰 절대적 증가를 보였습니다. 마지막으로, 저자들은 어휘-의미적 일관성에 의해 구조적 점화가 강화될 뿐만 아니라, 구조적으로 점화된 완성문이 더 높은 수준의 어휘-의미적 반복을 나타냄을 관찰했습니다. 종합하자면, 본 연구의 증거는 언어 모델에서의 구조적 점화가 다차원의 언어적 표상 수준에 걸쳐 작동하며, 통사적, 어휘적, 의미적 정렬(alignment)을 촉진하고 이에 의해 촉진된다는 관점을 뒷받침합니다. 코드: https://github.com/the-context-lab/primedproduction.

arXiv:2607.04265v2cs.RO

HALO-WA: 월드-액션 모델을 위한 하이브리드-어텐션 잠재 유도 온라인 강화학습

HALO-WA: Hybrid-Attention Latent-Guided Online Reinforcement Learning for World-Action Models

월드-액션(WA) 모델은 범용 로봇 조작을 위해 장기 호라izon 액션 청크를 생성할 수 있지만, 실세계 정밀 작업에서 보정, 인식, 접촉 동역학 오류에 취약하여 정렬이나 삽입의 마지막 수 밀리미터 구간에서 실패하는 경우가 많다. 본 논문은 WA 모델을 위한 HALO-WA를 제안한다. 이는 하이브리드-어텐션 잠 latent 유도 온라인 강화학습(RL) 프레임워크로, 경량 actor-critic 어댑터를 통해 WA 생성 과정의 잠재 특징과 액션 사전(prior)을 활용하여 실제 배포 오류에 대한 빠른 온라인 적응을 가능하게 한다. HALO-WA는 하이브리드-어텐션 구조를 도입하여, 시각적 맥락과 최종 단계 보정 요구에 조건화된 WA 잠재 표현에서 작업 관련 정보를 읽어들이면서 액션 청크의 시간적 일관성을 유지함으로써 정제된 액션 청크를 생성한다. 네 가지 실세계 정밀 조작 작업에서 HALO-WA를 검증한 결과, WA-base의 평균 성공률 26.4%에서 87.1%로 향상되었고, 가장 강한 기준선 대비 19.2%p 높았으며, 작업당 온라인 학습 45–75분만 필요했다. 재현성을 위해 RoboTwin에서 보조 시뮬레이션 실험을 추가로 수행하고, 코드는 https://github.com/YeanRoot/HALO-WA 에 공개했다.

arXiv:2605.22737v4cs.LG

증류 게임: 적응형 평가와 효율적인 방어

The Distillation Game: Adaptive Evaluations & Efficient Defenses

증류(distillation) 공격은 모델 제공자에게 배포상의 트레이드오프를 만든다: 모델을 더 유용하게 만드는 바로 그 출력이 모델을 더 쉽게 모방하게 만들 수도 있다. 우리는 효용이 제약된 교사(teacher)와 적응형 학생(student) 간의 미니맥스 게임을 통해 이 트레이드오프를 연구한다. 우리의 프레임워크는 다루기 쉬운 일방적 대응 규칙들을 도출한다: 학생이 고가치 예제의 가중치를 재조정하는 적응형 평가 규칙과, 증류에 가장 유용한 출력을 억제하는 교사 측 방어 템플릿이다. 예제 가치에 대한 저비용 프록시(proxy)로부터, 우리는 Product-of-Experts(PoE)를 도출한다. 이는 생성 과정에서 교사와 프록시 학생을 결합하는, 순전파(forward-pass)만으로 작동하는 단순한 방어 기법이다. 실험적으로, 적응형 평가는 큰 수동(passive)-적응(adaptive) 간극을 드러낸다: 최신 방어 기법들에 대해, 적응형 학생은 GSM8K와 MATH에서 수동 평가가 암시하는 것보다 상당히 더 많은 능력을 회복한다. 이러한 더 강력한 평가 하에서, 고비용 방어 기법과 PoE 사이의 겉보기 강건성(robustness) 격차는 상당히 좁아지며, 동시에 PoE는 훨씬 더 저비용을 유지하면서 더 높은 품질의 추론 과정(reasoning trace)을 보존한다. 전반적으로, 우리의 결과는 강력한 증류를 막기가 여전히 어렵다는 것을, 그리고 항증류(antidistillation) 연구의 진전은 수동적 학생이 아니라 적응형 학생을 기준으로 평가되어야 한다는 것을 시사한다. 코드는 다음 주소에서 확인할 수 있다: https://github.com/ysfalh/distillation-game.

arXiv:1808.07412공식 코드 · Papers with Code

Ithemal: Accurate, Portable and Fast Basic Block Throughput Estimation using Deep Neural Networks

Predicting the number of clock cycles a processor takes to execute a block of assembly instructions in steady state (the throughput) is important for both compiler designers and performance engineers. Building an analytical model to do so is especially complicated in modern x86-64 Complex Instruction Set Computer (CISC) machines with sophisticated processor microarchitectures in that it is tedious, error prone, and must be performed from scratch for each processor generation. In this paper we present Ithemal, the first tool which learns to predict the throughput of a set of instructions. Ithemal uses a hierarchical LSTM--based approach to predict throughput based on the opcodes and operands of instructions in a basic block. We show that Ithemal is more accurate than state-of-the-art hand-written tools currently used in compiler backends and static machine code analyzers. In particular, our model has less than half the error of state-of-the-art analytical models (LLVM's llvm-mca and Intel's IACA). Ithemal is also able to predict these throughput values just as fast as the aforementioned tools, and is easily ported across a variety of processor microarchitectures with minimal developer effort.

arXiv:1909.06236공식 코드 · Papers with Code

$ρ$-VAE: Autoregressive parametrization of the VAE encoder

We make a minimal, but very effective alteration to the VAE model. This is about a drop-in replacement for the (sample-dependent) approximate posterior to change it from the standard white Gaussian with diagonal covariance to the first-order autoregressive Gaussian. We argue that this is a more reasonable choice to adopt for natural signals like images, as it does not force the existing correlation in the data to disappear in the posterior. Moreover, it allows more freedom for the approximate posterior to match the true posterior. This allows for the repararametrization trick, as well as the KL-divergence term to still have closed-form expressions, obviating the need for its sample-based estimation. Although providing more freedom to adapt to correlated distributions, our parametrization has even less number of parameters than the diagonal covariance, as it requires only two scalars, $\rho$ and $s$, to characterize correlation and scaling, respectively. As validated by the experiments, our proposition noticeably and consistently improves the quality of image generation in a plug-and-play manner, needing no further parameter tuning, and across all setups. The code to reproduce our experiments is available at \url{https://github.com/sssohrab/rho_VAE/}.

arXiv:1912.09589공식 코드 · Papers with Code

Smart Home Appliances: Chat with Your Fridge

Current home appliances are capable to execute a limited number of voice commands such as turning devices on or off, adjusting music volume or light conditions. Recent progress in machine reasoning gives an opportunity to develop new types of conversational user interfaces for home appliances. In this paper, we apply state-of-the-art visual reasoning model and demonstrate that it is feasible to ask a smart fridge about its contents and various properties of the food with close-to-natural conversation experience. Our visual reasoning model answers user questions about existence, count, category and freshness of each product by analyzing photos made by the image sensor inside the smart fridge. Users may chat with their fridge using off-the-shelf phone messenger while being away from home, for example, when shopping in the supermarket. We generate a visually realistic synthetic dataset to train machine learning reasoning model that achieves 95% answer accuracy on test data. We present the results of initial user tests and discuss how we modify distribution of generated questions for model training based on human-in-the-loop guidance. We open source code for the whole system including dataset generation, reasoning model and demonstration scripts.

arXiv:2103.10226공식 코드 · Papers with Code

Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations

Explainability for machine learning models has gained considerable attention within the research community given the importance of deploying more reliable machine-learning systems. In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction, providing details about the model's decision-making. Current methods tend to generate trivial counterfactuals about a model's decisions, as they often suggest to exaggerate or remove the presence of the attribute being classified. For the machine learning practitioner, these types of counterfactuals offer little value, since they provide no new information about undesired model or data biases. In this work, we identify the problem of trivial counterfactual generation and we propose DiVE to alleviate it. DiVE learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss to uncover multiple valuable explanations about the model's prediction. Further, we introduce a mechanism to prevent the model from producing trivial explanations. Experiments on CelebA and Synbols demonstrate that our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods. Code is available at https://github.com/ElementAI/beyond-trivial-explanations.