PyTorchKR Papers

“Code Generation”

총 10,186편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2212.03588공식 코드 · Papers with Code

ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentation

Recently, CLIP has been applied to pixel-level zero-shot learning tasks via a two-stage scheme. The general idea is to first generate class-agnostic region proposals and then feed the cropped proposal regions to CLIP to utilize its image-level zero-shot classification capability. While effective, such a scheme requires two image encoders, one for proposal generation and one for CLIP, leading to a complicated pipeline and high computational cost. In this work, we pursue a simpler-and-efficient one-stage solution that directly extends CLIP's zero-shot prediction capability from image to pixel level. Our investigation starts with a straightforward extension as our baseline that generates semantic masks by comparing the similarity between text and patch embeddings extracted from CLIP. However, such a paradigm could heavily overfit the seen classes and fail to generalize to unseen classes. To handle this issue, we propose three simple-but-effective designs and figure out that they can significantly retain the inherent zero-shot capacity of CLIP and improve pixel-level generalization ability. Incorporating those modifications leads to an efficient zero-shot semantic segmentation system called ZegCLIP. Through extensive experiments on three public benchmarks, ZegCLIP demonstrates superior performance, outperforming the state-of-the-art methods by a large margin under both "inductive" and "transductive" zero-shot settings. In addition, compared with the two-stage method, our one-stage ZegCLIP achieves a speedup of about 5 times faster during inference. We release the code at https://github.com/ZiqinZhou66/ZegCLIP.git.

arXiv:2304.07849공식 코드 · Papers with Code

ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Human

In this paper, we present ChatPLUG, a Chinese open-domain dialogue system for digital human applications that instruction finetunes on a wide range of dialogue tasks in a unified internet-augmented format. Different from other open-domain dialogue models that focus on large-scale pre-training and scaling up model size or dialogue corpus, we aim to build a powerful and practical dialogue system for digital human with diverse skills and good multi-task generalization by internet-augmented instruction tuning. To this end, we first conduct large-scale pre-training on both common document corpus and dialogue data with curriculum learning, so as to inject various world knowledge and dialogue abilities into ChatPLUG. Then, we collect a wide range of dialogue tasks spanning diverse features of knowledge, personality, multi-turn memory, and empathy, on which we further instruction tune \modelname via unified natural language instruction templates. External knowledge from an internet search is also used during instruction finetuning for alleviating the problem of knowledge hallucinations. We show that \modelname outperforms state-of-the-art Chinese dialogue systems on both automatic and human evaluation, and demonstrates strong multi-task generalization on a variety of text understanding and generation tasks. In addition, we deploy \modelname to real-world applications such as Smart Speaker and Instant Message applications with fast inference. Our models and code will be made publicly available on ModelScope: https://modelscope.cn/models/damo/ChatPLUG-3.7B and Github: https://github.com/X-PLUG/ChatPLUG .

arXiv:2306.09970공식 코드 · Papers with Code

Continual Adaptation of Vision Transformers for Federated Learning

In this paper, we focus on the important yet understudied problem of Continual Federated Learning (CFL), where a server communicates with a set of clients to incrementally learn new concepts over time without sharing or storing any data. The complexity of this problem is compounded by challenges from both the Continual and Federated Learning perspectives. Specifically, models trained in a CFL setup suffer from catastrophic forgetting which is exacerbated by data heterogeneity across clients. Existing attempts at this problem tend to impose large overheads on clients and communication channels or require access to stored data which renders them unsuitable for real-world use due to privacy. In this paper, we attempt to tackle forgetting and heterogeneity while minimizing overhead costs and without requiring access to any stored data. We study this problem in the context of Vision Transformers and explore parameter-efficient approaches to adapt to dynamic distributions while minimizing forgetting. We achieve this by leveraging a prompting based approach (such that only prompts and classifier heads have to be communicated) and proposing a novel and lightweight generation and distillation scheme to consolidate client models at the server. We formulate this problem for image classification and establish strong baselines for comparison, conduct experiments on CIFAR-100 as well as challenging, large-scale datasets like ImageNet-R and DomainNet. Our approach outperforms both existing methods and our own baselines by as much as 7% while significantly reducing communication and client-level computation costs. Code available at https://github.com/shaunak27/hepco-fed.

arXiv:2307.14659공식 코드 · Papers with Code

LLDiffusion: Learning Degradation Representations in Diffusion Models for Low-Light Image Enhancement

Current deep learning methods for low-light image enhancement (LLIE) typically rely on pixel-wise mapping learned from paired data. However, these methods often overlook the importance of considering degradation representations, which can lead to sub-optimal outcomes. In this paper, we address this limitation by proposing a degradation-aware learning scheme for LLIE using diffusion models, which effectively integrates degradation and image priors into the diffusion process, resulting in improved image enhancement. Our proposed degradation-aware learning scheme is based on the understanding that degradation representations play a crucial role in accurately modeling and capturing the specific degradation patterns present in low-light images. To this end, First, a joint learning framework for both image generation and image enhancement is presented to learn the degradation representations. Second, to leverage the learned degradation representations, we develop a Low-Light Diffusion model (LLDiffusion) with a well-designed dynamic diffusion module. This module takes into account both the color map and the latent degradation representations to guide the diffusion process. By incorporating these conditioning factors, the proposed LLDiffusion can effectively enhance low-light images, considering both the inherent degradation patterns and the desired color fidelity. Finally, we evaluate our proposed method on several well-known benchmark datasets, including synthetic and real-world unpaired datasets. Extensive experiments on public benchmarks demonstrate that our LLDiffusion outperforms state-of-the-art LLIE methods both quantitatively and qualitatively. The source code and pre-trained models are available at https://github.com/TaoWangzj/LLDiffusion.

arXiv:2310.05572공식 코드 · Papers with Code

A Simple and Robust Framework for Cross-Modality Medical Image Segmentation applied to Vision Transformers

When it comes to clinical images, automatic segmentation has a wide variety of applications and a considerable diversity of input domains, such as different types of Magnetic Resonance Images (MRIs) and Computerized Tomography (CT) scans. This heterogeneity is a challenge for cross-modality algorithms that should equally perform independently of the input image type fed to them. Often, segmentation models are trained using a single modality, preventing generalization to other types of input data without resorting to transfer learning techniques. Furthermore, the multi-modal or cross-modality architectures proposed in the literature frequently require registered images, which are not easy to collect in clinical environments, or need additional processing steps, such as synthetic image generation. In this work, we propose a simple framework to achieve fair image segmentation of multiple modalities using a single conditional model that adapts its normalization layers based on the input type, trained with non-registered interleaved mixed data. We show that our framework outperforms other cross-modality segmentation methods, when applied to the same 3D UNet baseline model, on the Multi-Modality Whole Heart Segmentation Challenge. Furthermore, we define the Conditional Vision Transformer (C-ViT) encoder, based on the proposed cross-modality framework, and we show that it brings significant improvements to the resulting segmentation, up to 6.87\% of Dice accuracy, with respect to its baseline reference. The code to reproduce our experiments and the trained model weights are available at https://github.com/matteo-bastico/MI-Seg.

arXiv:2310.17419공식 코드 · Papers with Code

AntifakePrompt: Prompt-Tuned Vision-Language Models are Fake Image Detectors

Deep generative models can create remarkably photorealistic fake images while raising concerns about misinformation and copyright infringement, known as deepfake threats. Deepfake detection technique is developed to distinguish between real and fake images, where the existing methods typically learn classifiers in the image domain or various feature domains. However, the generalizability of deepfake detection against emerging and more advanced generative models remains challenging. In this paper, being inspired by the zero-shot advantages of Vision-Language Models (VLMs), we propose a novel approach called AntifakePrompt, using VLMs (e.g., InstructBLIP) and prompt tuning techniques to improve the deepfake detection accuracy over unseen data. We formulate deepfake detection as a visual question answering problem, and tune soft prompts for InstructBLIP to answer the real/fake information of a query image. We conduct full-spectrum experiments on datasets from a diversity of 3 held-in and 20 held-out generative models, covering modern text-to-image generation, image editing and adversarial image attacks. These testing datasets provide useful benchmarks in the realm of deepfake detection for further research. Moreover, results demonstrate that (1) the deepfake detection accuracy can be significantly and consistently improved (from 71.06% to 92.11%, in average accuracy over unseen domains) using pretrained vision-language models with prompt tuning; (2) our superior performance is at less cost of training data and trainable parameters, resulting in an effective and efficient solution for deepfake detection. Code and models can be found at https://github.com/nctu-eva-lab/AntifakePrompt.

arXiv:2311.13246공식 코드 · Papers with Code

CoachLM: Automatic Instruction Revisions Improve the Data Quality in LLM Instruction Tuning

Instruction tuning is crucial for enabling Language Learning Models (LLMs) in responding to human instructions. The quality of instruction pairs used for tuning greatly affects the performance of LLMs. However, the manual creation of high-quality instruction datasets is costly, leading to the adoption of automatic generation of instruction pairs by LLMs as a popular alternative. To ensure the high quality of LLM-generated instruction datasets, several approaches have been proposed. Nevertheless, existing methods either compromise dataset integrity by filtering a large proportion of samples, or are unsuitable for industrial applications. In this paper, instead of discarding low-quality samples, we propose CoachLM, a novel approach to enhance the quality of instruction datasets through automatic revisions on samples in the dataset. CoachLM is trained from the samples revised by human experts and significantly increases the proportion of high-quality samples in the dataset from 17.7% to 78.9%. The effectiveness of CoachLM is further assessed on various real-world instruction test sets. The results show that CoachLM improves the instruction-following capabilities of the instruction-tuned LLM by an average of 29.9%, which even surpasses larger LLMs with nearly twice the number of parameters. Furthermore, CoachLM is successfully deployed in a data management system for LLMs at Huawei, resulting in an efficiency improvement of up to 20% in the cleaning of 40k real-world instruction pairs. We release various assets of CoachLM, including the training data, code and test set (https://github.com/lunyiliu/CoachLM).

arXiv:2401.09883공식 코드 · Papers with Code

Question-Answer Cross Language Image Matching for Weakly Supervised Semantic Segmentation

Class Activation Map (CAM) has emerged as a popular tool for weakly supervised semantic segmentation (WSSS), allowing the localization of object regions in an image using only image-level labels. However, existing CAM methods suffer from under-activation of target object regions and false-activation of background regions due to the fact that a lack of detailed supervision can hinder the model's ability to understand the image as a whole. In this paper, we propose a novel Question-Answer Cross-Language-Image Matching framework for WSSS (QA-CLIMS), leveraging the vision-language foundation model to maximize the text-based understanding of images and guide the generation of activation maps. First, a series of carefully designed questions are posed to the VQA (Visual Question Answering) model with Question-Answer Prompt Engineering (QAPE) to generate a corpus of both foreground target objects and backgrounds that are adaptive to query images. We then employ contrastive learning in a Region Image Text Contrastive (RITC) network to compare the obtained foreground and background regions with the generated corpus. Our approach exploits the rich textual information from the open vocabulary as additional supervision, enabling the model to generate high-quality CAMs with a more complete object region and reduce false-activation of background regions. We conduct extensive analysis to validate the proposed method and show that our approach performs state-of-the-art on both PASCAL VOC 2012 and MS COCO datasets. Code is available at: https://github.com/CVI-SZU/QA-CLIMS

arXiv:2405.17382공식 코드 · Papers with Code

ReMoDetect: Reward Models Recognize Aligned LLM's Generations

The remarkable capabilities and easy accessibility of large language models (LLMs) have significantly increased societal risks (e.g., fake news generation), necessitating the development of LLM-generated text (LGT) detection methods for safe usage. However, detecting LGTs is challenging due to the vast number of LLMs, making it impractical to account for each LLM individually; hence, it is crucial to identify the common characteristics shared by these models. In this paper, we draw attention to a common feature of recent powerful LLMs, namely the alignment training, i.e., training LLMs to generate human-preferable texts. Our key finding is that as these aligned LLMs are trained to maximize the human preferences, they generate texts with higher estimated preferences even than human-written texts; thus, such texts are easily detected by using the reward model (i.e., an LLM trained to model human preference distribution). Based on this finding, we propose two training schemes to further improve the detection ability of the reward model, namely (i) continual preference fine-tuning to make the reward model prefer aligned LGTs even further and (ii) reward modeling of Human/LLM mixed texts (a rephrased texts from human-written texts using aligned LLMs), which serves as a median preference text corpus between LGTs and human-written texts to learn the decision boundary better. We provide an extensive evaluation by considering six text domains across twelve aligned LLMs, where our method demonstrates state-of-the-art results. Code is available at https://github.com/hyunseoklee-ai/ReMoDetect.

arXiv:2406.00501공식 코드 · Papers with Code

Diffusion-based Image Generation for In-distribution Data Augmentation in Surface Defect Detection

In this study, we show that diffusion models can be used in industrial scenarios to improve the data augmentation procedure in the context of surface defect detection. In general, defect detection classifiers are trained on ground-truth data formed by normal samples (negative data) and samples with defects (positive data), where the latter are consistently fewer than normal samples. For these reasons, state-of-the-art data augmentation procedures add synthetic defect data by superimposing artifacts to normal samples. This leads to out-of-distribution augmented data so that the classification system learns what is not a normal sample but does not know what a defect really is. We show that diffusion models overcome this situation, providing more realistic in-distribution defects so that the model can learn the defect's genuine appearance. We propose a novel approach for data augmentation that mixes out-of-distribution with in-distribution samples, which we call In&Out. The approach can deal with two data augmentation setups: i) when no defects are available (zero-shot data augmentation) and ii) when defects are available, which can be in a small number (few-shot) or a large one (full-shot). We focus the experimental part on the most challenging benchmark in the state-of-the-art, i.e., the Kolektor Surface-Defect Dataset 2, defining the new state-of-the-art classification AP score under weak supervision of .782. The code is available at https://github.com/intelligolabs/in_and_out.

arXiv:2406.02528공식 코드 · Papers with Code

Scalable MatMul-free Language Modeling

Matrix multiplication (MatMul) typically dominates the overall computational cost of large language models (LLMs). This cost only grows as LLMs scale to larger embedding dimensions and context lengths. In this work, we show that MatMul operations can be completely eliminated from LLMs while maintaining strong performance at billion-parameter scales. Our experiments show that our proposed MatMul-free models achieve performance on-par with state-of-the-art Transformers that require far more memory during inference at a scale up to at least 2.7B parameters. We investigate the scaling laws and find that the performance gap between our MatMul-free models and full precision Transformers narrows as the model size increases. We also provide a GPU-efficient implementation of this model which reduces memory usage by up to 61% over an unoptimized baseline during training. By utilizing an optimized kernel during inference, our model's memory consumption can be reduced by more than 10x compared to unoptimized models. To properly quantify the efficiency of our architecture, we build a custom hardware solution on an FPGA which exploits lightweight operations beyond what GPUs are capable of. We processed billion-parameter scale models at 13W beyond human readable throughput, moving LLMs closer to brain-like efficiency. This work not only shows how far LLMs can be stripped back while still performing effectively, but also points at the types of operations future accelerators should be optimized for in processing the next generation of lightweight LLMs. Our code implementation is available at https://github.com/ridgerchu/matmulfreellm.

arXiv:2406.07496공식 코드 · Papers with Code

TextGrad: Automatic "Differentiation" via Text

AI is undergoing a paradigm shift, with breakthroughs achieved by systems orchestrating multiple large language models (LLMs) and other complex components. As a result, developing principled and automated optimization methods for compound AI systems is one of the most important new challenges. Neural networks faced a similar challenge in its early days until backpropagation and automatic differentiation transformed the field by making optimization turn-key. Inspired by this, we introduce TextGrad, a powerful framework performing automatic ``differentiation'' via text. TextGrad backpropagates textual feedback provided by LLMs to improve individual components of a compound AI system. In our framework, LLMs provide rich, general, natural language suggestions to optimize variables in computation graphs, ranging from code snippets to molecular structures. TextGrad follows PyTorch's syntax and abstraction and is flexible and easy-to-use. It works out-of-the-box for a variety of tasks, where the users only provide the objective function without tuning components or prompts of the framework. We showcase TextGrad's effectiveness and generality across a diverse range of applications, from question answering and molecule optimization to radiotherapy treatment planning. Without modifying the framework, TextGrad improves the zero-shot accuracy of GPT-4o in Google-Proof Question Answering from $51\%$ to $55\%$, yields $20\%$ relative performance gain in optimizing LeetCode-Hard coding problem solutions, improves prompts for reasoning, designs new druglike small molecules with desirable in silico binding, and designs radiation oncology treatment plans with high specificity. TextGrad lays a foundation to accelerate the development of the next-generation of AI systems.

arXiv:2408.12561공식 코드 · Papers with Code

ssProp: Energy-Efficient Training for Convolutional Neural Networks with Scheduled Sparse Back Propagation

Recently, deep learning has made remarkable strides, especially with generative modeling, such as large language models and probabilistic diffusion models. However, training these models often involves significant computational resources, requiring billions of petaFLOPs. This high resource consumption results in substantial energy usage and a large carbon footprint, raising critical environmental concerns. Back-propagation (BP) is a major source of computational expense during training deep learning models. To advance research on energy-efficient training and allow for sparse learning on any machine and device, we propose a general, energy-efficient convolution module that can be seamlessly integrated into any deep learning architecture. Specifically, we introduce channel-wise sparsity with additional gradient selection schedulers during backward based on the assumption that BP is often dense and inefficient, which can lead to over-fitting and high computational consumption. Our experiments demonstrate that our approach reduces 40\% computations while potentially improving model performance, validated on image classification and generation tasks. This reduction can lead to significant energy savings and a lower carbon footprint during the research and development phases of large-scale AI systems. Additionally, our method mitigates over-fitting in a manner distinct from Dropout, allowing it to be combined with Dropout to further enhance model performance and reduce computational resource usage. Extensive experiments validate that our method generalizes to a variety of datasets and tasks and is compatible with a wide range of deep learning architectures and modules. Code is publicly available at https://github.com/lujiazho/ssProp.

arXiv:2505.12266공식 코드 · Papers with Code

PMQ-VE: Progressive Multi-Frame Quantization for Video Enhancement

Multi-frame video enhancement tasks aim to improve the spatial and temporal resolution and quality of video sequences by leveraging temporal information from multiple frames, which are widely used in streaming video processing, surveillance, and generation. Although numerous Transformer-based enhancement methods have achieved impressive performance, their computational and memory demands hinder deployment on edge devices. Quantization offers a practical solution by reducing the bit-width of weights and activations to improve efficiency. However, directly applying existing quantization methods to video enhancement tasks often leads to significant performance degradation and loss of fine details. This stems from two limitations: (a) inability to allocate varying representational capacity across frames, which results in suboptimal dynamic range adaptation; (b) over-reliance on full-precision teachers, which limits the learning of low-bit student models. To tackle these challenges, we propose a novel quantization method for video enhancement: Progressive Multi-Frame Quantization for Video Enhancement (PMQ-VE). This framework features a coarse-to-fine two-stage process: Backtracking-based Multi-Frame Quantization (BMFQ) and Progressive Multi-Teacher Distillation (PMTD). BMFQ utilizes a percentile-based initialization and iterative search with pruning and backtracking for robust clipping bounds. PMTD employs a progressive distillation strategy with both full-precision and multiple high-bit (INT) teachers to enhance low-bit models' capacity and quality. Extensive experiments demonstrate that our method outperforms existing approaches, achieving state-of-the-art performance across multiple tasks and benchmarks.The code will be made publicly available at: https://github.com/xiaoBIGfeng/PMQ-VE.

arXiv:2611.04811v1cs.CV

NightIQA: MLLM 기반 야간 이미지 품질 평가를 위한 종합 데이터셋 및 벤치마크

NightIQA: A Comprehensive Dataset and Benchmark for MLLM-Based Nighttime Image Quality Assessment

야간 이미지는 본질적으로 복합적인 열화에 취약하여 정확한 품질 평가가 매우 어렵다. 점수 기반 IQA 방법은 품질을 하나의 숫자로 압축하므로 야간 평가에 필요한 다차원적이고 언어에 기반한 추론이 부족하다. 멀티모달 대규모 언어 모델(MLLM)은 더 포괄적인 대안을 제공하지만, 중대한 데이터 공백에 여전히 제약을 받는다. 기존의 실제 IQA 데이터셋은 압도적으로 주간 중심이고 오래된 촬영물로 구축된 경우가 많으며, 합성 데이터셋은 야간 특유의 열화 모델링이 부족하여 실제 야간 환경과 동떨어진 아티팩트를 만들어낸다. 이 공백을 메우기 위해 우리는 MLLM 기반 야간 IQA를 위해 특화 구축된 데이터셋 및 벤치마크 모음인 NightIQA를 제시한다. NightIQA는 세 가지 핵심 구성요소로 이루어진다. 첫째, NightIQA-Real은 최신 장비로 직접 촬영한 1,769장의 고해상도 이미지로 구성된 실제 데이터셋으로, 여섯 가지 대표적 왜곡 유형과 구조화된 다차원 주석을 포함한다. 둘째, NightIQA-Bench는 MLLM 기반 야간 IQA에 맞춰 설계된 1,078개의 시각 기반 객관식 문항으로 이루어진 벤치마크로, 인지, 기술적 평가, 위치 파악, 추론에 걸친 4단계 평가 계층으로 구성된다. 셋째, NightIQA-Syn은 전역 수준과 인스턴스 수준의 열화를 함께 모델링하는, 본질적으로 확장 가능한 장면 인식 합성 파이프라인으로, 초기 공개분으로 8만 장의 고충실도 이미지와 전체 생성 코드를 제공한다. 지도 미세조정(SFT)과 그룹 상대 정책 최적화(GRPO)로 NightIQA에서 미세조정한 Qwen3-VL-8B는 오픈소스 IQA 방법을 능가하고 선도적인 비공개 모델보다 높은 전체 정확도를 달성하며, 이는 도메인 특화 데이터와 구조화된 지도의 가치를 부각한다.

arXiv:2608.22063v1cs.AI

SQL 생성에서 도구 선택으로: MCP 서버를 위한 도메인 지향 패턴

From SQL Generation to Tool Selection: A Domain-Oriented Pattern for MCP Servers

대규모 언어 모델(LLM) 기반 에이전트가 Model Context Protocol(MCP)을 통해 기업 데이터에 점점 더 많이 접근하고 있으며, 많은 MCP 데이터베이스 서버는 단일 범용 SQL 실행 도구를 노출하여 유연성을 극대화한다. 본 논문은 Domain-Oriented Tooling Pattern을 제안한다: 쿼리 시점에 SQL을 생성하는 대신, 모델은 도메인 정렬 소규모 도구 집합에서 선택하며, 이들의 매개변수화된 쿼리는 서버 측에서 스키마 탐색, 조인 및 비즈니스 규칙을 캡슐화한다. 우리는 세 가지 아키텍처 불변량을 중심으로 이 패턴을 형식화하고, SQL 합성을 의도 분류로 대체하면 일상적인 요청을 처리하는 데 필요한 모델 티어가 낮아진다는 관찰인 Model Demotion을 소개한다. 참조 구현으로, 도메인 도구가 YAML 메타데이터와 외부 매개변수화 SQL 파일로 선언적으로 정의되는 오픈소스 프레임워크 MCP Blueprint를 제시한다. 공개 재현성 벤치마크로 이 패턴을 평가하며, Sakila 데이터베이스에 대해 17개의 고객 대면 작업, 네 가지 로컬 모델(3B–8B)에서 세 가지 MCP 서버 설계—원시 SQL 실행, 최소 범용 도구 팩, 수직화된 도메인 팩—를 비교한다(609개 완료 셀; temperature 0; 셀당 3회 반복). 수직화된 팩은 풀링 평균 점수 0.939에 도달하며, 원시 SQL은 0.666, 범용 팩은 0.605이다; 가장 작은 모델은 0.583에서 0.929로 개선되어 모든 더 큰 구성과 동등하거나 그를 초과하면서 정답당 비용을 한 자릿수 줄인다. 모든 하네스 코드, 프롬프트, 정답(gold answers), 고정 팩 및 셀별 결과는 공개적으로 이용 가능하다.

arXiv:2609.00224v1cs.LG

QTEA: 희소 잔차 핵심 가중치와 열 단위 최적화를 활용한 삼진(Ternary) LLM

QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization

가중치 전용 사후 학습 양자화(post-training quantization, PTQ)는 대규모 언어 모델(LLM)을 대규모로 서비스할 때의 연산 부담을 줄여 줄 수 있다. 그러나 기존 PTQ 방법은 모델 간 일반화에 실패하는 경우가 많으며, 2비트 미만에서는 심각한 정확도 손실을 겪는다. 많은 방법이 이러한 손실을 완화하기 위해 비정형 희소성(unstructured sparsity)을 활용하지만, 그 대가로 규칙성과 GPU 친화적인 실행을 잃게 된다. 본 논문은 가중치를 삼진(ternary) 값으로 양자화하고 핵심(salient) 가중치를 잔차 오차 보상기로 사용하는 2비트 미만 PTQ 프레임워크인 QTEA를 제안한다. 하드웨어 효율성을 유지하기 위해, 잔차는 핵심 열 내에서 반정형(semi-structured) 1:4 희소성을 갖는 선택된 열에 할당된다. 또한 GPTQ 방식의 열 단위(column-by-column) 양자화에 열 단위 스케일 재조정(column-wise rescale refinement)을 추가하여, 열별 스케일과 삼진 할당을 번갈아 갱신함으로써 재구성 오차를 줄인다. 아울러 GPTQ에서 순서에 의존하는 오차 전파를 확인하고, 후반 단계의 오차 누적을 완화하기 위해 오차 감쇠(error decay)를 도입한다. Qwen3-14B에서 QTEA는 모든 가중치를 유효 가중치당 1.7비트로 압축하면서, 가장 강력한 삼진 PTQ 기준선 대비 평균 정확도를 16.7% 향상시킨다. 또한 WikiText와 C4에서 각각 1.40배와 2.61배 낮은 퍼플렉시티를 달성한다. 이러한 경향은 Llama3-8B에서도 유지되어, QTEA는 동일한 데이터셋에서 6.6%의 정확도 향상과 1.34배/1.95배 낮은 퍼플렉시티를 얻는다. 마지막으로, 룩업 테이블 기반 커널을 개발하여 FP16 기준선 대비 토큰당 생성 속도를 7.2배 향상시켰다. 코드는 https://github.com/Intelligent-Microsystems-Lab/QTEA 에서 확인할 수 있다.

arXiv:2609.04871v1cs.AI

AutoLR: 산업 추천 시스템에서 연구부터 Launch Review까지의 경로 자동화

AutoLR: Automating the Path from Research to Launch Review in Industrial Recommender Systems

산업용 추천 시스템을 개선하는 일은 아이디어에서 바로 배포로 이어지는 직선적 과정이 아니라, 반복적인 연구·엔지니어링 과정이다. \textbf{DASHEN(넷이즈 게임 커뮤니티 앱)}에서는 알고리즘 엔지니어가 일반적으로 연구 논문, 기술 보고서, 기존 프로덕션 실험에서 유망한 방향을 찾고, 해당 방법을 재현하거나 적응한 뒤 프로덕션 코드베이스에 구현하고, 학습과 오프라인 실험으로 결과 모델을 평가한다. 유망한 후보는 온라인 A/B 테스트로 넘어가고, 견고한 이득을 보인 것은 Launch Review—전체 트래픽 롤아웃을 위한 내부 관문—에 제출된다. 대규모 언어 모델(LLM)은 이 워크플로의 개별 단계를 돕을 수 있지만, 장기간·종종 며칠에 걸친 실험 주기 전반에 걸쳐 이를 신뢰성 있게 조율하는 하네스(harness) 없이는 전체 과정이 여전히 사람에 의존한다. 우리는 처음 \textbf{Auto Launch Review}로 구축되었다가 이후 상류로 확장되어 연구부터 런치까지의 자율 하네스가 된 \textbf{AutoLR}를 제안한다. AutoLR는 세 가지 시스템 메커니즘을 결합한다: 제안을 토론하고 적대적으로 검토하는 \textbf{다중 전문가 council}; 후보 방향에 제한된 시험 예산을 배분하고 Council 재순위를 사용하는 \textbf{결정론적 증거 가중 탐험–활용 선택기}; 외부 연구, 프로덕션 시스템 지식, 게임 커뮤니티·플레이어 특성·콘텐츠 상호작용 패턴 같은 DASHEN 특화 도메인 지식을 구성·패치·로그·실패·오프라인 결과에서 얻은 사후 증거와 결합하는 계층적 지식 시스템. LLM 에이전트는 의미론적 추론과 코드 생성을 수행하고, 결정론적 컨트롤러는 실행, 지표 추출, 가드레일, 영속 상태 전이에 대한 권한을 유지한다.

arXiv:2610.18352v2cs.RO

마음은 구상하고 손은 이룬다: 장기 조작을 위한 접지된 행동과 진행도 평가의 결합

Let the Mind Conceive, Let the Hand Achieve: Coupling Grounded Action and Progress Evaluation for Long-Horizon Manipulation

여러 단계의 지시를 따르는 로봇은 장면이 변하더라도 올바른 물체에 올바른 순서로 행동해야 한다. 또한 과제 진행 상황을 계속 파악하고 언제 다음 단계로 넘어갈지 판단해야 한다. 이러한 과제를 해결하기 위해 우리는 과제 계획 및 실행 관리를 행동 생성과 분리하는 에이전트 하니스 Orchestra를 제안한다. Orchestra는 완전한 멀티모달 지시를 순서가 있는 하위 과제로 분해하고, 완료된 단계를 추적하며, 실행 중 관련 물체와 목적지의 위치를 갱신한다. 또한 현재 하위 과제, 공간적 접지, 최근 관측을 담은 공유 컨텍스트를 유지한다. 행동 정책은 이 컨텍스트를 사용해 행동을 생성하고, 별도로 학습된 진행도 평가기는 언제 다음 하위 과제로 넘어갈지를 추정한다. 원자적 지시와 짧은 궤적으로 학습된 로봇이 더 긴 하위 과제 시퀀스를 실행할 수 있는지 평가하기 위해, 우리는 RMBench-Compose라는 벤치마크를 도입한다. Orchestra는 RMBench의 9개 과제에서 평균 성공률 $68.22\%$를 달성하여 Mem-0을 $26.22$%p 앞선다. RMBench-Compose에서는 평균 성공 길이(ASL) $7.35$개 하위 과제를 달성하여 HarnessVLA의 $2.46$배이다. 실제 환경의 네 가지 다단계 과제에서는 기본 $\pi_{0.5}$ 정책을 사용한 Orchestra 대비 평균 ASL을 $25.0\%$ 향상시킨다. 이러한 결과는 Orchestra가 요청된 하위 과제 순서와 지정된 물체에 대한 행동을 유지하면서 로봇이 더 긴 하위 과제 시퀀스를 수행하도록 돕는다는 것을 보여 준다. 코드와 프로젝트 페이지는 https://youpengwen.github.io/Orchestra 에서 확인할 수 있다.

arXiv:1804.06281공식 코드 · Papers with Code

Deep transfer learning in the assessment of the quality of protein models

MOTIVATION: Proteins fold into complex structures that are crucial for their biological functions. Experimental determination of protein structures is costly and therefore limited to a small fraction of all known proteins. Hence, different computational structure prediction methods are necessary for the modelling of the vast majority of all proteins. In most structure prediction pipelines, the last step is to select the best available model and to estimate its accuracy. This model quality estimation problem has been growing in importance during the last decade, and progress is believed to be important for large scale modelling of proteins. The current generation of model quality estimation programs performs well at separating incorrect and good models, but fails to consistently identify the best possible model. State-of-the-art model quality assessment methods use a combination of features that describe a model and the agreement of the model with features predicted from the protein sequence. RESULTS: We first introduce a deep neural network architecture to predict model quality using significantly fewer input features than state-of-the-art methods. Thereafter, we propose a methodology to train the deep network that leverages the comparative structure of the problem. We also show the possibility of applying transfer learning on databases of known protein structures. We demonstrate its viability by reaching state-of-the-art performance using only a reduced set of input features and a coarse description of the models. AVAILABILITY: The code will be freely available for download at github.com/ElofssonLab/ProQ4.