In recent years, advanced image editing and generation methods have rapidly evolved, making detecting and locating forged image content increasingly challenging. Most existing image forgery detection methods rely on identifying the edited traces left in the image. However, because the traces of different forgeries are distinct, these methods can identify familiar forgeries included in the training data but struggle to handle unseen ones. In response, we present an approach for Generalizable Image Forgery Localization (GIFL). Once trained, our model can detect both seen and unseen forgeries, providing a more practical and efficient solution to counter false information in the era of generative AI. Our method focuses on learning general features from the pristine content rather than traces of specific forgeries, which are relatively consistent across different types of forgeries and therefore can be used as universal features to locate unseen forgeries. Additionally, as existing image forgery datasets are still dominated by traditional hand-crafted forgeries, we construct a new dataset consisting of images edited by various popular deep generative image editing methods to further encourage research in detecting images manipulated by deep generative models. Extensive experimental results show that the proposed approach outperforms state-of-the-art methods in the detection of unseen forgeries and also demonstrates competitive results for seen forgeries. The code and dataset are available at https://github.com/ZhaoHengrun/GIFL.
Text-to-image (T2I) generation models can inadvertently produce not-safe-for-work (NSFW) content, prompting the integration of text and image safety filters. Recent advances employ large language models (LLMs) for semantic-level detection, rendering traditional token-level perturbation attacks largely ineffective. However, our evaluation shows that existing jailbreak methods are ineffective against these modern filters. We introduce GhostPrompt, the first automated jailbreak framework that combines dynamic prompt optimization with multimodal feedback. It consists of two key components: (i) Dynamic Optimization, an iterative process that guides a large language model (LLM) using feedback from text safety filters and CLIP similarity scores to generate semantically aligned adversarial prompts; and (ii) Adaptive Safety Indicator Injection, which formulates the injection of benign visual cues as a reinforcement learning problem to bypass image-level filters. GhostPrompt achieves state-of-the-art performance, increasing the ShieldLM-7B bypass rate from 12.5\% (Sneakyprompt) to 99.0\%, improving CLIP score from 0.2637 to 0.2762, and reducing the time cost by $4.2 \times$. Moreover, it generalizes to unseen filters including GPT-4.1 and successfully jailbreaks DALLE 3 to generate NSFW images in our evaluation, revealing systemic vulnerabilities in current multimodal defenses. To support further research on AI safety and red-teaming, we will release code and adversarial prompts under a controlled-access protocol.
Understanding behavior requires datasets that capture humans while carrying out complex tasks. The kitchen is an excellent environment for assessing human motor and cognitive function, as many complex actions are naturally exhibited in kitchens from chopping to cleaning. Here, we introduce the EPFL-Smart-Kitchen-30 dataset, collected in a noninvasive motion capture platform inside a kitchen environment. Nine static RGB-D cameras, inertial measurement units (IMUs) and one head-mounted HoloLens~2 headset were used to capture 3D hand, body, and eye movements. The EPFL-Smart-Kitchen-30 dataset is a multi-view action dataset with synchronized exocentric, egocentric, depth, IMUs, eye gaze, body and hand kinematics spanning 29.7 hours of 16 subjects cooking four different recipes. Action sequences were densely annotated with 33.78 action segments per minute. Leveraging this multi-modal dataset, we propose four benchmarks to advance behavior understanding and modeling through 1) a vision-language benchmark, 2) a semantic text-to-motion generation benchmark, 3) a multi-modal action recognition benchmark, 4) a pose-based action segmentation benchmark. We expect the EPFL-Smart-Kitchen-30 dataset to pave the way for better methods as well as insights to understand the nature of ecologically-valid human behavior. Code and data are available at https://github.com/amathislab/EPFL-Smart-Kitchen
멀티모달 대규모 언어 모델(MLLM)에서 놀라운 진전이 있었음에도, 지배적인 사고의 연쇄(CoT) 패러다임은 여전히 자연어 표현 공간에 국한되어 있다. 그 결과 본질적으로 과도한 언어적 오버헤드를 초래하여 정보 희석과 약한 시각적 그라운딩을 낳는다. 이 과제를 해결하기 위해, 우리는 자연어 중심 CoT에서 벗어나 추론을 기호적 연결자를 포함한 간결한 중간 상태로 구성된 압축된 구조화 트레이스로 표현하고, 시각적 관찰과 논리적 추론을 분리하는 패러다igm인 Dense Reasoning Trace(DRT)를 제안한다. 먼저 Dense Trace Initialization을 도입하여 DRT 추론 모드를 모델에 내재화함으로써 시각적 근거를 유지하면서 토큰 효율을 크게 높인다. 더 나아가 트레이스 내 논리 관계를 모델이 충실히 포착하도록 Trace-Grounded Reinforcement Learning 프레임워크를 제안한다. 이는 3관점 검증 파이프라인으로 참조 트레이스를 구축하고, 구조화된 보상을 사용하는 Trace-Grounded GRPO를 적용하여 환각을 줄이고 논리적 그라운딩을 강화한 간결한 DRT 스타일 트레이스 생성을 유도한다. 어려운 추론 벤치마크에 대한 광범위한 실험에서 DRT는 Qwen3-VL 기준선 대비 토큰 효율 5.5배 개선과 정확도 1.3%p 향상을 달성한다. 이러한 결과는 복잡한 멀티모달 추론이 장황한 자연어 트레이스를 반드시 필요로 하지 않을 수 있음을 시사하며, 차세대 MLLM을 위한 더 효율적인 경로를 연다. 코드와 데이터는 https://github.com/HIT-leaderone/DRT 에서 이용할 수 있다.
We introduce Axion 1.0, a fully open agent foundation model that combines strong and broad agentic capabilities with a remarkably simple training recipe. Axion 1.0 is trained in only three stages: two lightweight supervised fine-tuning stages followed by reinforcement learning, without dedicated mid-training, direct preference optimization (DPO), or on-policy distillation (OPD). The first SFT stage develops strong self-contained reasoning through model-aware data construction, where we synthesize challenging problems and adversarially strengthen them based on the base model's own successful solution strategies. The second stage extends these capabilities to tool-dependent reasoning through a scalable pipeline for constructing diverse and automatically verifiable tasks grounded in real-world data sources and deterministic computation. Finally, reinforcement learning further strengthens long-horizon reasoning and interaction with external tools and environments. Despite this compact training recipe, Axion 1.0 achieves strong performance across a broad suite of agentic benchmarks and is competitive with leading commercial models on several of them. Equally importantly, Axion 1.0 is open beyond model weights: we release the complete recipe, including data construction, generation, filtering, verification, prompts, and training code, enabling the entire development process to be reproduced and extended. Axion 1.0 demonstrates that strong agent foundation models can be both open and simple, offering a practical and reproducible path toward advancing open agentic intelligence.
Large Language Models (LLMs) have achieved remarkable progress across a wide range of tasks, but remain vulnerable to safety risks such as harmful content generation and jailbreak attacks. Existing safety techniques---including external guardrails, inference-time guidance, and post-training alignment---each face limitations in balancing safety, utility, and controllability. In this work, we propose UpSafe$^\circ$C, a unified framework for enhancing LLM safety through safety-aware upcycling. Our approach first identifies safety-critical layers and upcycles them into a sparse Mixture-of-Experts (MoE) structure, where the router acts as a soft guardrail that selectively activates original MLPs and added safety experts. We further introduce a two-stage SFT strategy to strengthen safety discrimination while preserving general capabilities. To enable flexible control at inference time, we introduce a safety temperature mechanism, allowing dynamic adjustment of the trade-off between safety and utility. Experiments across multiple benchmarks, base model, and model scales demonstrate that UpSafe$^\circ$C achieves robust safety improvements against harmful and jailbreak inputs, while maintaining competitive performance on general tasks. Moreover, analysis shows that safety temperature provides fine-grained inference-time control that achieves controllable trade-off between utility and safety. Our results highlight a new direction for LLM safety: moving from static alignment toward dynamic, modular, and inference-aware control. Code: https://github.com/HansSunY/UpSafe.
결정 기반 블랙박스 적대적 공격은 심각한 보안 위협을 제기하지만, 현재 방법론들은 근본적인 한계를 갖고 있다. 픽셀 단위 공격은 부자연스러운 고주파 시각적 아티팩트를 자주 유발하는 반면, 잠재 공간(latent-space) 프레임워크는 저차원 매니폴드의 제한된 탐색 공간과 내재된 재구성 결함에 의해 제약받는다. 이러한 한계를 해결하기 위해, 우리는 쿼리 효율적 결정 기반 적대적 공격을 위한 Latent Geometric Chords(LGC)와 변형인 LGC-H를 제안한다. LGC의 핵심은 압축된 의미적 매니폴드 내에서 곡률 인식 기하학적 탐색을 수행함으로써 결정 경계(decision boundary)를 탐색하는 것이다. 높은 시각적 충실도를 보장하고 차원 병목을 우회하기 위해, 우리는 Residual-based Adversarial Generation(RAG) 메커니즘을 도입한다. RAG는 의미적 교란(perturbation)을 기하학적 chord로 분리하여 원본 소스 이미지에 직접 중첩한다. RAG는 기준선 재구성 결함을 실질적으로 해결하고 허용 가능한 탐색 공간 차원을 효과적으로 두 배로 늘린다. 실험 결과는 LGC가 강건한 교차 데이터셋 전이성을 달성하고 최신 기준선들을 실질적으로 능가함을 보여준다. 특히 우리 방법 LGC는 교란 크기를 최소화하면서도 최신 수준의 시각적 충실도를 달성한다--5000회 쿼리에서 Structural Similarity Index Measure(SSIM)이 0.99를 초과하고 Learned Perceptual Image Patch Similarity(LPIPS)가 0.01 미만--엄격한 지각(perceptual) 제약 하에서도 높은 공격 성공률을 유지하며, 적대적 학습으로 훈련된 강건(robust) 모델을 성공적으로 무력화한다. 소스 코드는 https://github.com/eihmuekhine/Latent-Geometric-Chords 에서 이용할 수 있다.
RLVR 학습에서 균일 샘플링이 정책 진행에 따라 이동하는 난이도 프론티어를 추적하지 못하는 문제를 해결하기 위해, HeaPA는 힙 기반 경계 샘플링으로 중간 난이도 구간에 집중하고 온-정책 쿼리 증강으로 풀을 동적으로 확장하는 프레임워크를 제안한다. 두 가지 학습 코퍼스와 7개 벤치마크에서 HeaPA는 기존 GRPO, DAPO 파이프라인 대비 정확도를 일관되게 향상시키고 더 적은 롤아웃 연산으로 목표 성능에 도달했으며, 중간~대규모 모델 스케일에서 효과가 특히 두드러졌다.
데이터 분석을 활용해 뉴스 가치가 있는 기사를 발굴하는 실무인 데이터 저널리즘은 트렌드, 격차, 책임 규명 서사를 밝혀내는 기자와 탐사보도 저널리스트의 역량에 점점 더 크게 의존하고 있습니다. 실제로 대규모 정형 데이터셋을 탐색하는 작업은 여전히 느리고 불안정합니다. 저널리스트는 가설이 발전하는 동안 수년에 걸친 여러 데이터셋의 수백 개 변수를 탐색하고, 데이터 코딩 규칙을 이해하며, 복잡한 분석 코드를 작성해야 합니다. 비록 대규모 언어 모델(LLM)이 '자연어로 질문하고 SQL/답변을 얻는' 도구로 자주 홍보되지만, 실제 뉴스룸 워크플로에서는 스키마 불일치 및 변화, 도메인 의미와 단위의 오독, 묵시적 가정 등 반복적인 실패가 드러납니다. 본 논문에서는 대화형 상호작용, 스키마 그라운딩(schema grounding), 분석적 계획 수립, 실행 가능한 쿼리 생성을 결합하여 정형 데이터에 대한 탐색적 분석을 지원함으로써 이러한 요구를 해결하는 시스템인 DataWeave를 제시합니다. DataWeave는 LLM을 자율적인 답변 엔진으로 취급하기보다는, 가설이 바뀜에 따라 출력을 검사하고 수정하며 유도할 수 있는 대화형 파트너로 규정합니다. 복잡한 도메인 의미와 잦은 스키마 변경이 수반되는 중대한 공공 데이터셋인 미국 교육부의 종합 고등교육 데이터 시스템(IPEDS)을 전문 저널리스트들이 본 시스템을 사용해 분석한 사례 연구를 제시합니다. 또한 실제 배포 경험과 반복적인 개선 과정이 현재의 DataWeave 아키텍처와 분석 워크플로를 어떻게 형성했는지 보고합니다. 본 연구 결과는 정형 데이터 분석에서 신뢰할 수 있는 인간-LLM 협업을 위한 설계 원칙과 배포 교훈을 도출합니다.
투기적 디코딩(speculative decoding)은 대규모 언어 모델(LLM)의 자기회귀적 생성을 가속한다. 각 초안 작성 단계(drafting stage)에서 경량 드래프트 모델이 토큰을 제안하고, 타깃 모델이 이후 이를 검증한다. 드래프트 모델의 성능이 점차 향상됨에 따라, 본 연구에서는 초안 작성 단계에서 생성된 모든 토큰을 타깃 모델이 완전히 수락하는 경우가 빈번하다는 점을 발견했다. 그럼에도 불구하고 각 초안 작성 단계마다 검증이 뒤따르기 때문에, 초안 작성을 계속 진행할 수 있었던 상황에서도 불필요한 타깃 모델 순전파(forward pass)가 발생하게 된다. 적응형 초안 길이(adaptive draft length) 방식은 디코딩 과정에서 검증 전에 생성할 드래프트 토큰 수를 결정하지만, 이는 자기회귀 드래프트 모델에 대해서만 속도 향상을 가져온다. 병렬 드래프트 모델의 경우, 초안 작성을 계속하려면 아직 검증되지 않은 드래프트 토큰에 대한 타깃 모델의 은닉 상태(hidden state)가 필요하다. 본 연구에서는 검증 전에 여러 초안 작성 단계를 적응적으로 수행하는 루프 형태의 투기적 디코딩인 DLoop를 제안한다. DLoop는 드래프트 모델의 신뢰도가 유지되는 동안 초안 작성을 계속하며, 누적된 모든 드래프트 토큰을 한 번에 검증한다. 루프 인식 학습(loop-aware training)은 검증되지 않은 드래프트 토큰에 대해 드래프트 모델 자체의 은닉 상태를 노출함으로써 추가 초안 작성 단계에서도 드래프트 모델의 신뢰성을 유지하도록 한다. 드래프트 모델의 순전파를 추가로 사용하는 대신, DLoop는 검증에 필요한 타깃 모델의 순전파 횟수를 줄인다. EAGLE-3, DFlash, Domino, DSpark, 다중 토큰 예측(multi-token prediction) 모듈을 포함한 다양한 투기적 디코딩 방식 전반에 걸쳐, DLoop는 무손실 디코딩을 유지하면서 실제 소요 시간(wall-clock) 기준 속도 향상을 5%에서 41%까지 개선한다. 코드는 https://github.com/naver-ai/DLoop 에서 공개될 예정이다.
힘과 토크는 사람의 움직임을 일으키며 생체역학, 로봇공학, 물리를 반영한 애니메이션의 핵심이다. 하지만 신체 움직임을 복원하거나 생성하는 학습 기반 파이프라인에서는 흔히 빠져 있다. 일부 움직임 데이터셋에는 힘 주석이 있지만, 여기에 짝지은 움직임 시퀀스는 현대 학습 파이프라인의 매개변수형 신체 모델과 호환되지 않는다. 기존 데이터셋은 대개 역동역학으로 관절 토크를 구한다. 이 계산은 입력 운동학 데이터의 품질에 민감하지만, 보통 토크 추정치의 신뢰도를 정량화하지 않는다. 이 공백을 메우기 위해 매개변수형 신체 표현에 실측 힘을 지도 신호로 제공하고 샘플별 토크 불확실성을 담은 최초의 데이터셋 ForceBody를 소개한다. 참여자 140명의 움직임 시행 10,386건(970만 프레임, 26.9시간)에서 SKEL 신체 모델과 지면 반력, 역동역학으로 구한 관절 토크를 짝지었다. 각 토크 레이블에는 몬테카를로 샘플을 역동역학 파이프라인에 통과시켜 얻은 프레임별·관절별 불확실성 주석을 함께 제공한다. SKEL 움직임에서 지면 반력과 관절 토크를 예측하는 과제로 MLP, Conv1D, LSTM, GRU, Mamba2, 트랜스포머 등 아키텍처 여섯 개를 평가했다. 불확실성 주석의 활용 예로, 샘플별 불확실성에 따라 가중치를 부여해 트랜스포머를 미세조정했을 때 관절 토크 추정의 평균 절대 오차(MAE)가 5% 줄었다. 코드와 데이터는 https://github.com/DominoAI-Lab/ForceBody-NeurIPS-ED-2026 에 공개되어 있다.
Natural language explanations (NLEs) are a special form of data annotation in which annotators identify rationales (most significant text tokens) when assigning labels to data instances, and write out explanations for the labels in natural language based on the rationales. NLEs have been shown to capture human reasoning better, but not as beneficial for natural language inference (NLI). In this paper, we analyze two primary flaws in the way NLEs are currently used to train explanation generators for language inference tasks. We find that the explanation generators do not take into account the variability inherent in human explanation of labels, and that the current explanation generation models generate spurious explanations. To overcome these limitations, we propose a novel framework, LIREx, that incorporates both a rationale-enabled explanation generator and an instance selector to select only relevant, plausible NLEs to augment NLI models. When evaluated on the standardized SNLI data set, LIREx achieved an accuracy of 91.87%, an improvement of 0.32 over the baseline and matching the best-reported performance on the data set. It also achieves significantly better performance than previous studies when transferred to the out-of-domain MultiNLI data set. Qualitative analysis shows that LIREx generates flexible, faithful, and relevant NLEs that allow the model to be more robust to spurious explanations. The code is available at https://github.com/zhaoxy92/LIREx.
It is appealing but challenging to achieve real-time deep neural network (DNN) inference on mobile devices because even the powerful modern mobile devices are considered as ``resource-constrained'' when executing large-scale DNNs. It necessitates the sparse model inference via weight pruning, i.e., DNN weight sparsity, and it is desirable to design a new DNN weight sparsity scheme that can facilitate real-time inference on mobile devices while preserving a high sparse model accuracy. This paper designs a novel mobile inference acceleration framework GRIM that is General to both convolutional neural networks (CNNs) and recurrent neural networks (RNNs) and that achieves Real-time execution and high accuracy, leveraging fine-grained structured sparse model Inference and compiler optimizations for Mobiles. We start by proposing a new fine-grained structured sparsity scheme through the Block-based Column-Row (BCR) pruning. Based on this new fine-grained structured sparsity, our GRIM framework consists of two parts: (a) the compiler optimization and code generation for real-time mobile inference; and (b) the BCR pruning optimizations for determining pruning hyperparameters and performing weight pruning. We compare GRIM with Alibaba MNN, TVM, TensorFlow-Lite, a sparse implementation based on CSR, PatDNN, and ESE (a representative FPGA inference acceleration framework for RNNs), and achieve up to 14.08x speedup.
Recently, a series of algorithms have been explored for GAN compression, which aims to reduce tremendous computational overhead and memory usages when deploying GANs on resource-constrained edge devices. However, most of the existing GAN compression work only focuses on how to compress the generator, while fails to take the discriminator into account. In this work, we revisit the role of discriminator in GAN compression and design a novel generator-discriminator cooperative compression scheme for GAN compression, termed GCC. Within GCC, a selective activation discriminator automatically selects and activates convolutional channels according to a local capacity constraint and a global coordination constraint, which help maintain the Nash equilibrium with the lightweight generator during the adversarial training and avoid mode collapse. The original generator and discriminator are also optimized from scratch, to play as a teacher model to progressively refine the pruned generator and the selective activation discriminator. A novel online collaborative distillation scheme is designed to take full advantage of the intermediate feature of the teacher generator and discriminator to further boost the performance of the lightweight generator. Extensive experiments on various GAN-based generation tasks demonstrate the effectiveness and generalization of GCC. Among them, GCC contributes to reducing 80% computational costs while maintains comparable performance in image translation tasks. Our code and models are available at https://github.com/SJLeo/GCC.
Generating images that fit a given text description using machine learning has improved greatly with the release of technologies such as the CLIP image-text encoder model; however, current methods lack artistic control of the style of image to be generated. We present an approach for generating styled drawings for a given text description where a user can specify a desired drawing style using a sample image. Inspired by a theory in art that style and content are generally inseparable during the creative process, we propose a coupled approach, known here as StyleCLIPDraw, whereby the drawing is generated by optimizing for style and content simultaneously throughout the process as opposed to applying style transfer after creating content in a sequence. Based on human evaluation, the styles of images generated by StyleCLIPDraw are strongly preferred to those by the sequential approach. Although the quality of content generation degrades for certain styles, overall considering both content \textit{and} style, StyleCLIPDraw is found far more preferred, indicating the importance of style, look, and feel of machine generated images to people as well as indicating that style is coupled in the drawing process itself. Our code (https://github.com/pschaldenbrand/StyleCLIPDraw), a demonstration (https://replicate.com/pschaldenbrand/style-clip-draw), and style evaluation data (https://www.kaggle.com/pittsburghskeet/drawings-with-style-evaluation-styleclipdraw) are publicly available.
Most existing unsupervised person re-identification (Re-ID) methods use clustering to generate pseudo labels for model training. Unfortunately, clustering sometimes mixes different true identities together or splits the same identity into two or more sub clusters. Training on these noisy clusters substantially hampers the Re-ID accuracy. Due to the limited samples in each identity, we suppose there may lack some underlying information to well reveal the accurate clusters. To discover these information, we propose an Implicit Sample Extension (\OurWholeMethod) method to generate what we call support samples around the cluster boundaries. Specifically, we generate support samples from actual samples and their neighbouring clusters in the embedding space through a progressive linear interpolation (PLI) strategy. PLI controls the generation with two critical factors, i.e., 1) the direction from the actual sample towards its K-nearest clusters and 2) the degree for mixing up the context information from the K-nearest clusters. Meanwhile, given the support samples, ISE further uses a label-preserving loss to pull them towards their corresponding actual samples, so as to compact each cluster. Consequently, ISE reduces the "sub and mixed" clustering errors, thus improving the Re-ID performance. Extensive experiments demonstrate that the proposed method is effective and achieves state-of-the-art performance for unsupervised person Re-ID. Code is available at: \url{https://github.com/PaddlePaddle/PaddleClas}.
In grant-free sparse code multiple access (GF-SCMA) system, active user detection (AUD) is a major performance bottleneck as it involves complex combinatorial problem, which makes joint design of contention resources for users and AUD at the receiver a crucial but a challenging problem. To this end, we propose autoencoder (AE)-based joint optimization of both preamble generation networks (PGNs) in the encoder side and data-aided AUD in the decoder side. The core architecture of the proposed AE is a novel user activity extraction network (UAEN) in the decoder that extracts a priori user activity information from the SCMA codeword data for the data-aided AUD. An end-to-end training of the proposed AE enables joint optimization of the contention resources, i.e., preamble sequences, each associated with one of the codebooks, and extraction of user activity information from both preamble and SCMA-based data transmission. Furthermore, we propose a self-supervised pre-training scheme for the UAEN prior to the end-to-end training, to ensure the convergence of the UAEN which lies deep inside the AE network. Simulation results demonstrated that the proposed AUD scheme achieved 3 to 5dB gain at a target activity detection error rate of $\bf{{10}^{-3}}$ compared to the state-of-the-art DL-based AUD schemes.
Different from conventional image matting, which either requires user-defined scribbles/trimap to extract a specific foreground object or directly extracts all the foreground objects in the image indiscriminately, we introduce a new task named Referring Image Matting (RIM) in this paper, which aims to extract the meticulous alpha matte of the specific object that best matches the given natural language description, thus enabling a more natural and simpler instruction for image matting. First, we establish a large-scale challenging dataset RefMatte by designing a comprehensive image composition and expression generation engine to automatically produce high-quality images along with diverse text attributes based on public datasets. RefMatte consists of 230 object categories, 47,500 images, 118,749 expression-region entities, and 474,996 expressions. Additionally, we construct a real-world test set with 100 high-resolution natural images and manually annotate complex phrases to evaluate the out-of-domain generalization abilities of RIM methods. Furthermore, we present a novel baseline method CLIPMat for RIM, including a context-embedded prompt, a text-driven semantic pop-up, and a multi-level details extractor. Extensive experiments on RefMatte in both keyword and expression settings validate the superiority of CLIPMat over representative methods. We hope this work could provide novel insights into image matting and encourage more follow-up studies. The dataset, code and models are available at https://github.com/JizhiziLi/RIM.
High dynamic range (HDR) imaging is an indispensable technique in modern photography. Traditional methods focus on HDR reconstruction from multiple images, solving the core problems of image alignment, fusion, and tone mapping, yet having a perfect solution due to ghosting and other visual artifacts in the reconstruction. Recent attempts at single-image HDR reconstruction show a promising alternative: by learning to map pixel values to their irradiance using a neural network, one can bypass the align-and-merge pipeline completely yet still obtain a high-quality HDR image. In this work, we propose a weakly supervised learning method that inverts the physical image formation process for HDR reconstruction via learning to generate multiple exposures from a single image. Our neural network can invert the camera response to reconstruct pixel irradiance before synthesizing multiple exposures and hallucinating details in under- and over-exposed regions from a single input image. To train the network, we propose a representation loss, a reconstruction loss, and a perceptual loss applied on pairs of under- and over-exposure images and thus do not require HDR images for training. Our experiments show that our proposed model can effectively reconstruct HDR images. Our qualitative and quantitative results show that our method achieves state-of-the-art performance on the DrTMO dataset. Our code is available at https://github.com/VinAIResearch/single_image_hdr.
Due to the outstanding capability for data generation, Generative Adversarial Networks (GANs) have attracted considerable attention in unsupervised learning. However, training GANs is difficult, since the training distribution is dynamic for the discriminator, leading to unstable image representation. In this paper, we address the problem of training GANs from a novel perspective, \emph{i.e.,} robust image classification. Motivated by studies on robust image representation, we propose a simple yet effective module, namely AdaptiveMix, for GANs, which shrinks the regions of training data in the image representation space of the discriminator. Considering it is intractable to directly bound feature space, we propose to construct hard samples and narrow down the feature distance between hard and easy samples. The hard samples are constructed by mixing a pair of training images. We evaluate the effectiveness of our AdaptiveMix with widely-used and state-of-the-art GAN architectures. The evaluation results demonstrate that our AdaptiveMix can facilitate the training of GANs and effectively improve the image quality of generated samples. We also show that our AdaptiveMix can be further applied to image classification and Out-Of-Distribution (OOD) detection tasks, by equipping it with state-of-the-art methods. Extensive experiments on seven publicly available datasets show that our method effectively boosts the performance of baselines. The code is publicly available at https://github.com/WentianZhang-ML/AdaptiveMix.