Text-to-SVG systems are commonly evaluated by applying raster-image metrics to rasterised outputs, yet whether these scores capture fine-grained prompt faithfulness remains unclear. We introduce SVGFaith, a two-layer meta-evaluation framework in which the more faithful candidate of each pair is determined by construction or by human judgment. SVGFaith-Diagnostic contains 1,842 controlled contrastive pairs across eight capability axes, split evenly between two directions: two renderings compared against one prompt, and two descriptions compared against one rendering. SVGFaith-Arena contains 1,327 human-annotated comparisons of real generator outputs. On SVGFaith-Diagnostic, CLIP and three other image--text embedding metrics reach at least 0.789 pairwise accuracy on object identity but at most 0.672 on spatial relations and attribute binding, where chance is 0.5. We further introduce SVGFaith-Score, an interpretable pointwise metric. A domain-tuned decomposer turns each prompt into a dependency graph of atomic visual requirements, a visual question answering model scores whether each requirement is satisfied, and a requirement's score is discounted when its prerequisites score low. In addition to the overall score, the per-requirement scores show which requirements are judged unsatisfied and which inherit discounts from their prerequisites. SVGFaith-Score significantly outperforms all nine baselines, including a Qwen3.5-122B-A10B judge, on Arena and in both Diagnostic directions. Code and data: https://github.com/Yukinonooo/SVGFaith.
Text-based person anomaly search requires fine-grained alignment between language queries and subtle visual cues, a challenge amplified by the long-tailed nature of anomalous behaviors. While Large Multimodal Models (LMMs) offer powerful visual understanding, their generative pre-training is fundamentally misaligned with the discriminative needs of this retrieval task. Adapting them without costly fine-tuning remains a significant hurdle. We introduce \textbf{AnomalyLMM}, a training-free, coarse-to-fine framework that effectively repurposes generative LMMs for zero-shot anomaly retrieval. Our method first uses a general retrieval model to produce an initial ranking. To refine this list, we introduce a novel cloze-based re-ranking mechanism with three steps. The first step \textbf{Cloze Generation} converts the text query into a ``fill-in-the-blank'' prompt. Next, \textbf{Cloze Completion} compels the LMM to focus on specific visual regions and generate a description of the potential anomaly. The final step \textbf{Comparison \& Re-ranking} calculates semantic alignment between the LMM's generated completion and the original query, which serves as a powerful re-ranking score. Experiments on the PAB and UCC datasets show that AnomalyLMM surpasses the competitive baseline by $+1.93$\% and $+1.79$\% Recall@1 accuracy on two datasets. Crucially, our method provides highly interpretable visual-textual alignments without any task-specific training, a vital feature for real-world deployment. The code will be made publicly available.
검증-수리 루프는 코드 생성, 수학적 추론, 도구 사용에서 결함이 있는 계획을 수정하기 위한 대규모 언어 모델(LLM) 에이전트의 표준 수단이다. 검증기와 수리기 모두에 노이즈가 있으면, 수리는 이미 올바른 계획을 손상시킬 수 있으며, 보고된 수용률은 계속 상승하는 반면 실제 유효성은 하락하므로, 기존 방법들은 수리를 언제 중단해야 하는지에 대한 원칙적 근거가 부족하다. 우리는 노이즈가 있는 검증-수리-반복(verify-repair-repeat, VRR) 루프를 위한 강건한 중단 프레임워크인 VRR-Stop을 제안한다. 4-매개변수 노이즈 모델은 검증기의 거짓 수용과 거짓 거부를 수리기의 수리 및 손상 행동과 분리한다. 신념 필터링(belief filtering)은 반복된 검증 투표를 확정 유효성의 추정치로 변환하며, 루프는 모든 매개변수의 정확한 복구가 아니라 부호 식별 가능성(sign identifiability)만 필요로 하는 실제 한계 이득의 부호에 따라 확정하거나 수리한다. 검증기 판별력(discrimination)이 0에 접근하면 보정(calibration) 자체가 실패하고 추정 오류가 중단 부호를 뒤집을 수 있으므로, 우리는 VRR-Stop을 충분한 검증 마진 하에서만 현재 후보를 대체하는 추정 없는(estimation-free) 대안인 VRR-Guard와 함께 사용한다. GSM8K 스트레스 설정에서 VRR-Stop은 고정 5회 수리 대비 최종 실제 유효성을 60.6%p 향상시키며, 평균 비용은 0.72회 수리 라운드이다. 다양한 설정에서 중단 신뢰성은 추정 오류의 절대적 크기보다 검증기 판별력과 결정 마진에 의해 공동으로 좌우된다.
LLM 학습은 수동 설계 및 어노테이션에서 상호작용 기반 자기 진화로 전환되고 있다. 그러나 기존 자기 진화 방법들은 작업 다양성과 검증 신뢰성 사이의 근본적인 딜레마에 직면한다. 환경에 구속된 방법은 정밀한 피드백을 얻지만 학습을 좁은 도메인에 한정하고, 개방형 자기 생성은 작업 공간을 넓히지만 신뢰할 수 있는 검증이 부족하여 오해의 소지가 있는 보상이 학습 루프를 오염시킨다. 우리는 에이전트 스킬이 이러한 긴장을 완화하는 강력한 중간 지점임을 확인한다. 각 스킬은 특정 시나리오에서 깊고 검증 가능한 실행을 보장하는 반면, 스킬 간 동적 라우팅은 개방형 작업 다양성을 유지한다. 이 통찰을 활용하여, 우리는 제안자(proposer), 해결자(solver), 동적 스킬 컨트롤러로 구성된 공동 진화 프레임워크인 Skill Self-Play(Skill-SP)를 제안한다. 강화학습 루프를 통해 조율되며, 이 구성 요소들은 지속적인 셀프 플레이 루프에서 공동 진화한다. 제안자는 동적으로 샘플링된 스킬을 조건으로 한 도전적인 작업을 생성하고, 해결자는 역량 경계를 확장하기 위해 후보 해결책을 탐험하며, 스킬 컨트롤러는 실행 피드백을 수집하여 스킬 라이브러리를 업데이트하고 확장한다. 이러한 상호작용적 공동 진화는 구조화된 검증과 개방형 탐험 사이의 격차를 효과적으로 메운다. 도구 사용 및 추론 벤치마크에 대한 실증 평가는 견고한 진화 엔진으로서 Skill-SP가 유능한 백본의 성능 상한을 지속적으로 밀어 올리는 동시에, 초기에 정렬되지 않은 모델들의 놀라운 성과 전환을 촉진함을 보여준다. 우리 코드는 https://github.com/Qwen-Applications/skill-self-play 에서 이용할 수 있다.
Large Reasoning Models (LRMs), which autonomously produce a reasoning Chain of Thought (CoT) before producing final responses, offer a promising approach to interpreting and monitoring model behaviors. Inspired by the observation that certain CoT patterns -- e.g., ``Wait, did I miss anything?'' -- consistently emerge across tasks, we explore whether LRMs exhibit human-like cognitive habits. Building on Habits of Mind, a well-established framework of cognitive habits associated with successful human problem-solving, we introduce CogTest, a principled benchmark designed to evaluate LRMs' cognitive habits. CogTest includes 16 cognitive habits, each instantiated with 25 diverse tasks, and employs an evidence-first extraction method to ensure reliable habit identification. With CogTest, we conduct a comprehensive evaluation of 16 widely used LLMs (13 LRMs and 3 non-reasoning ones). Our findings reveal that LRMs, unlike conventional LLMs, not only exhibit human-like habits but also adaptively deploy them according to different tasks. Finer-grained analyses further uncover patterns of similarity and difference in LRMs' cognitive habit profiles, particularly certain inter-family similarity (e.g., Qwen-3 models and DeepSeek-R1). Extending the study to safety-related tasks, we observe that certain habits, such as Taking Responsible Risks, are strongly associated with the generation of harmful responses. These findings suggest that studying persistent behavioral patterns in LRMs' CoTs is a valuable step toward deeper understanding of LLM misbehavior. The code is available at: https://github.com/jianshuod/CogTest.
자기중심(egocentric) 영상으로부터의 완전한 3D 지각은 공유 metric 좌표계에서 주변 장면과 착용자의 전신 동작을 함께 복원하는 것을 요구한다. 기존 방법은 보통 장면 재구성과 동작 추정을 따로 다룬다. 장면 재구성 방법은 착용자를 무시하고, 동작 추정 방법은 명시적 장면 기하를 갖지 못하며 종종 외부 궤적에 의존한다. 두 작업을 함께 복원하는 것은 가시성이 비대칭이고 서로 다른 예측 패러다임을 요구하기 때문에 어렵다. 대부분 가시적인 장면은 결정론적 기하 회귀(regression)를 지지하는 반면, 심하게 가려진 신체는 생성적(generative) motion 추론이 필요하다. 이에 우리는 결정론적 metric 기하 재구성과 기하 조건부 motion 생성을 결합한 통합 프레임워크 RESELF(REconstructing the Scene and the sELF)를 제안한다. RESELF는 대규모 배외(exocentric) 데이터로 사전학습된 기하 파운데이션 모델을 프레임 단위 스케일 및 상대 포즈(relative-pose) 일관성 목적 함수로 자기중심 영상에 맞게 적응시킨다. 이렇게 얻은 카메라 궤적과 잠재(latent) 기하 특징이 착용자 motion을 복원하는 diffusion 모델을 조건화한다. 이후 폐루프(closed-loop) kinematic 피드백 단계가 재구성된 장면 기하를 보존하면서 카메라 헤드를 추가로 정제한다. 학습과 평가를 위해 EgoExo4D에서 자기중심 영상, sparse metric 장면 기하, 카메라 궤적, 전신 motion 어노테이션을 정렬하여 EE4D-JSM을 구축했다. 실험 결과 RESELF는 깊이 추정, 카메라 추적, 전신 motion 추정 각 개별 과제용 최신(state-of-the-art) 방법들보다 우수한 성능을 보였다. 코드, 모델, 데이터셋은 https://ka1guan.github.io/RESELF/ 에서 제공될 예정이다.
Synthesizing a target concept from a single reference image is challenging in diffusion-based personalized text-to-image generation, particularly for sticker personalization where prompts often require explicit attribute edits. With only one reference, test-time fine-tuning (TTF) methods tend to overfit, producing \textit{visual entanglement}, where background artifacts are absorbed into the learned concept, and \textit{structural rigidity}, where the model memorizes reference-specific spatial configurations and loses contextual controllability. To address these issues, we introduce \textbf{SE}mantic-aware single-image sticker person\textbf{AL}ization (\textbf{SEAL}), a plug-and-play, architecture-agnostic adaptation module that integrates into existing personalization pipelines without modifying their U-Net-based diffusion backbones. SEAL applies three components during embedding adaptation: (1) a Semantic-guided Spatial Attention Loss, (2) a Split-merge Token Strategy, and (3) Structure-aware Layer Restriction. To support sticker-domain personalization with attribute-level control, we present StickerBench, a large-scale sticker image dataset with structured tags under a six-attribute schema (Appearance, Emotion, Action, Camera Composition, Style, Background). These annotations provide a consistent interface for varying context while keeping target identity fixed, enabling systematic evaluation of identity disentanglement and contextual controllability. Experiments show that SEAL consistently improves identity preservation while maintaining contextual controllability, highlighting the importance of explicit spatial and structural constraints during test-time adaptation. The code, StickerBench, and project page will be publicly released.
언어 모델은 도구(tool)를 통해 행동하지만, 실제 에이전트(agent)는 수천 개의 인터페이스를 담은 라이브러리에 직면한다. 본 논문은 실행 전에 에이전트에게 제시되는, 사용 가능한 도구의 짧고 순서가 있는 부분집합을 도구 메뉴(tool menu)로 정의한다. 에이전트는 이 메뉴에 포함된 도구만 호출할 수 있다. 다단계 작업에는 최종 행동뿐 아니라, 그 입력을 사용 가능한 순서로 만들어 내는 선행 도구도 필요하다. 현재의 메뉴 구성 방식은 요청과의 관련성에 따라 도구의 순위를 매기므로, 최종 행동은 노출하더라도 덜 명백한 생산자(producer) 도구는 누락하거나 늦게 배치할 수 있다. 본 논문은 관찰 가능한 요청 상태에서 원하는 결과에 이르는 실행 전 경로인 상태 경로(state path)를 도입하고, 이를 학습하기 위한 State-Path Tool Menu를 제안한다. 우리의 프레임워크는 메뉴를 이러한 경로에 대한 실행 사전 분포(execution prior)로 취급한다. 인코더는 현재 상태에서 어떤 도구를 실행할 수 있는지, 도구의 출력이 이후 도구의 입력을 어떻게 충족하는지, 학습 경로에서 어떤 순서가 반복되는지를 표현한다. 리트리버(retriever)는 실행 가능한 진입 도구, 누락된 입력의 생산자 도구, 그리고 최종 행동을 포괄하도록 검색한다. 이어서 리랭커(reranker)가 생산자 도구를 소비자(consumer) 도구보다 앞에 배치한다. ToolBench에서 우리의 메뉴는 에이전트를 변경하지 않고도 온라인 성공률을 0.737에서 0.898로 높였으며, 검색, 리랭킹, 생성, 라우팅 기반 베이스라인을 능가한다. 또한 State-Path 메뉴는 32개의 도구만으로도 공식 목록이 128개로 포괄하는 것보다 더 많은 완전한 체인을 포괄하며, 그 성공률 향상은 모델 용량이 서로 다른 실행기(executor) 계열 전반에서 유지된다. 코드는 https://github.com/Met2348/State-Path 에서 확인할 수 있다.
The large domain discrepancy between faces captured in polarimetric (or conventional) thermal and visible domain makes cross-domain face verification a highly challenging problem for human examiners as well as computer vision algorithms. Previous approaches utilize either a two-step procedure (visible feature estimation and visible image reconstruction) or an input-level fusion technique, where different Stokes images are concatenated and used as a multi-channel input to synthesize the visible image given the corresponding polarimetric signatures. Although these methods have yielded improvements, we argue that input-level fusion alone may not be sufficient to realize the full potential of the available Stokes images. We propose a Generative Adversarial Networks (GAN) based multi-stream feature-level fusion technique to synthesize high-quality visible images from prolarimetric thermal images. The proposed network consists of a generator sub-network, constructed using an encoder-decoder network based on dense residual blocks, and a multi-scale discriminator sub-network. The generator network is trained by optimizing an adversarial loss in addition to a perceptual loss and an identity preserving loss to enable photo realistic generation of visible images while preserving discriminative characteristics. An extended dataset consisting of polarimetric thermal facial signatures of 111 subjects is also introduced. Multiple experiments evaluated on different experimental protocols demonstrate that the proposed method achieves state-of-the-art performance. Code will be made available at https://github.com/hezhangsprinter.
Point cloud registration is a key task in many computational fields. Previous correspondence matching based methods require the inputs to have distinctive geometric structures to fit a 3D rigid transformation according to point-wise sparse feature matches. However, the accuracy of transformation heavily relies on the quality of extracted features, which are prone to errors with respect to partiality and noise. In addition, they can not utilize the geometric knowledge of all the overlapping regions. On the other hand, previous global feature based approaches can utilize the entire point cloud for the registration, however they ignore the negative effect of non-overlapping points when aggregating global features. In this paper, we present OMNet, a global feature based iterative network for partial-to-partial point cloud registration. We learn overlapping masks to reject non-overlapping regions, which converts the partial-to-partial registration to the registration of the same shape. Moreover, the previously used data is sampled only once from the CAD models for each object, resulting in the same point clouds for the source and reference. We propose a more practical manner of data generation where a CAD model is sampled twice for the source and reference, avoiding the previously prevalent over-fitting issue. Experimental results show that our method achieves state-of-the-art performance compared to traditional and deep learning based methods. Code is available at https://github.com/megvii-research/OMNet.
Temporal action detection (TAD) aims to determine the semantic label and the temporal interval of every action instance in an untrimmed video. It is a fundamental and challenging task in video understanding. Previous methods tackle this task with complicated pipelines. They often need to train multiple networks and involve hand-designed operations, such as non-maximal suppression and anchor generation, which limit the flexibility and prevent end-to-end learning. In this paper, we propose an end-to-end Transformer-based method for TAD, termed TadTR. Given a small set of learnable embeddings called action queries, TadTR adaptively extracts temporal context information from the video for each query and directly predicts action instances with the context. To adapt Transformer to TAD, we propose three improvements to enhance its locality awareness. The core is a temporal deformable attention module that selectively attends to a sparse set of key snippets in a video. A segment refinement mechanism and an actionness regression head are designed to refine the boundaries and confidence of the predicted instances, respectively. With such a simple pipeline, TadTR requires lower computation cost than previous detectors, while preserving remarkable performance. As a self-contained detector, it achieves state-of-the-art performance on THUMOS14 (56.7% mAP) and HACS Segments (32.09% mAP). Combined with an extra action classifier, it obtains 36.75% mAP on ActivityNet-1.3. Code is available at https://github.com/xlliu7/TadTR.
Spiking neural network (SNN), as the next generation of artificial neural network (ANN), offer a closer mimicry of natural neural networks and hold promise for significant improvements in computational efficiency. However, the current SNN is trained to infer over a fixed duration, overlooking the potential of dynamic inference in SNN. In this paper, we strengthen the marriage between SNN and event-driven processing with a proposal to consider a cutoff in SNN, which can terminate SNN anytime during inference to achieve efficient inference. Two novel optimisation techniques are presented to achieve inference efficient SNN: a Top-K cutoff and a regularisation.The proposed regularisation influences the training process, optimising SNN for the cutoff, while the Top-K cutoff technique optimises the inference phase. We conduct an extensive set of experiments on multiple benchmark frame-based datasets, such asCIFAR10/100, Tiny-ImageNet, and event-based datasets, including CIFAR10-DVS, N-Caltech101 and DVS128 Gesture. The experimental results demonstrate the effectiveness of our techniques in both ANN-to-SNN conversion and direct training, enabling SNNs to require 1.76 to 2.76x fewer timesteps for CIFAR-10, while achieving 1.64 to 1.95x fewer timesteps across all event-based datasets, with near-zero accuracy loss. These findings affirms the compatibility and potential benefits of our techniques in enhancing accuracy and reducing inference latency when integrated with existing methods. Code available: https://github.com/Dengyu-Wu/SNNCutoff
Fast generation of high-quality 3D digital humans is important to a vast number of applications ranging from entertainment to professional concerns. Recent advances in differentiable rendering have enabled the training of 3D generative models without requiring 3D ground truths. However, the quality of the generated 3D humans still has much room to improve in terms of both fidelity and diversity. In this paper, we present Get3DHuman, a novel 3D human framework that can significantly boost the realism and diversity of the generated outcomes by only using a limited budget of 3D ground-truth data. Our key observation is that the 3D generator can profit from human-related priors learned through 2D human generators and 3D reconstructors. Specifically, we bridge the latent space of Get3DHuman with that of StyleGAN-Human via a specially-designed prior network, where the input latent code is mapped to the shape and texture feature volumes spanned by the pixel-aligned 3D reconstructor. The outcomes of the prior network are then leveraged as the supervisory signals for the main generator network. To ensure effective training, we further propose three tailored losses applied to the generated feature volumes and the intermediate feature maps. Extensive experiments demonstrate that Get3DHuman greatly outperforms the other state-of-the-art approaches and can support a wide range of applications including shape interpolation, shape re-texturing, and single-view reconstruction through latent inversion.
Existing 3D-from-2D generators are typically designed for well-curated single-category datasets, where all the objects have (approximately) the same scale, 3D location, and orientation, and the camera always points to the center of the scene. This makes them inapplicable to diverse, in-the-wild datasets of non-alignable scenes rendered from arbitrary camera poses. In this work, we develop a 3D generator with Generic Priors (3DGP): a 3D synthesis framework with more general assumptions about the training data, and show that it scales to very challenging datasets, like ImageNet. Our model is based on three new ideas. First, we incorporate an inaccurate off-the-shelf depth estimator into 3D GAN training via a special depth adaptation module to handle the imprecision. Then, we create a flexible camera model and a regularization strategy for it to learn its distribution parameters during training. Finally, we extend the recent ideas of transferring knowledge from pre-trained classifiers into GANs for patch-wise trained models by employing a simple distillation-based technique on top of the discriminator. It achieves more stable training than the existing methods and speeds up the convergence by at least 40%. We explore our model on four datasets: SDIP Dogs 256x256, SDIP Elephants 256x256, LSUN Horses 256x256, and ImageNet 256x256, and demonstrate that 3DGP outperforms the recent state-of-the-art in terms of both texture and geometry quality. Code and visualizations: https://snap-research.github.io/3dgp.
Identifying moving objects is a crucial capability for autonomous navigation, consistent map generation, and future trajectory prediction of objects. In this paper, we propose a novel network that addresses the challenge of segmenting moving objects in 3D LiDAR scans. Our approach not only predicts point-wise moving labels but also detects instance information of main traffic participants. Such a design helps determine which instances are actually moving and which ones are temporarily static in the current scene. Our method exploits a sequence of point clouds as input and quantifies them into 4D voxels. We use 4D sparse convolutions to extract motion features from the 4D voxels and inject them into the current scan. Then, we extract spatio-temporal features from the current scan for instance detection and feature fusion. Finally, we design an upsample fusion module to output point-wise labels by fusing the spatio-temporal features and predicted instance information. We evaluated our approach on the LiDAR-MOS benchmark based on SemanticKITTI and achieved better moving object segmentation performance compared to state-of-the-art methods, demonstrating the effectiveness of our approach in integrating instance information for moving object segmentation. Furthermore, our method shows superior performance on the Apollo dataset with a pre-trained model on SemanticKITTI, indicating that our method generalizes well in different scenes.The code and pre-trained models of our method will be released at https://github.com/nubot-nudt/InsMOS.
Denoising Diffusion Probabilistic Models (DDPM) have recently gained significant attention. DDPMs compose a Markovian process that begins in the data domain and gradually adds noise until reaching pure white noise. DDPMs generate high-quality samples from complex data distributions by defining an inverse process and training a deep neural network to learn this mapping. However, these models are inefficient because they require many diffusion steps to produce aesthetically pleasing samples. Additionally, unlike generative adversarial networks (GANs), the latent space of diffusion models is less interpretable. In this work, we propose to generalize the denoising diffusion process into an Upsampling Diffusion Probabilistic Model (UDPM). In the forward process, we reduce the latent variable dimension through downsampling, followed by the traditional noise perturbation. As a result, the reverse process gradually denoises and upsamples the latent variable to produce a sample from the data distribution. We formalize the Markovian diffusion processes of UDPM and demonstrate its generation capabilities on the popular FFHQ, AFHQv2, and CIFAR10 datasets. UDPM generates images with as few as three network evaluations, whose overall computational cost is less than a single DDPM or EDM step, while achieving an FID score of 6.86. This surpasses current state-of-the-art efficient diffusion models that use a single denoising step for sampling. Additionally, UDPM offers an interpretable and interpolable latent space, which gives it an advantage over traditional DDPMs. Our code is available online: \url{https://github.com/shadyabh/UDPM/}
As large language models (LLMs) generate texts with increasing fluency and realism, there is a growing need to identify the source of texts to prevent the abuse of LLMs. Text watermarking techniques have proven reliable in distinguishing whether a text is generated by LLMs by injecting hidden patterns. However, we argue that existing LLM watermarking methods are encoding-inefficient and cannot flexibly meet the diverse information encoding needs (such as encoding model version, generation time, user id, etc.). In this work, we conduct the first systematic study on the topic of Codable Text Watermarking for LLMs (CTWL) that allows text watermarks to carry multi-bit customizable information. First of all, we study the taxonomy of LLM watermarking technologies and give a mathematical formulation for CTWL. Additionally, we provide a comprehensive evaluation system for CTWL: (1) watermarking success rate, (2) robustness against various corruptions, (3) coding rate of payload information, (4) encoding and decoding efficiency, (5) impacts on the quality of the generated text. To meet the requirements of these non-Pareto-improving metrics, we follow the most prominent vocabulary partition-based watermarking direction, and devise an advanced CTWL method named Balance-Marking. The core idea of our method is to use a proxy language model to split the vocabulary into probability-balanced parts, thereby effectively maintaining the quality of the watermarked text. Our code is available at https://github.com/lancopku/codable-watermarking-for-llm.
Due to the complexity of annotation and inter-annotator variability, most lung nodule malignancy grading datasets contain label noise, which inevitably degrades the performance and generalizability of models. Although researchers adopt the label-noise-robust methods to handle label noise for lung nodule malignancy grading, they do not consider the inherent ordinal relation among classes of this task. To model the ordinal relation among classes to facilitate tackling label noise in this task, we propose a Unimodal-Regularized Label-noise-tolerant (URL) framework. Our URL contains two stages, the Supervised Contrastive Learning (SCL) stage and the Memory pseudo-labels generation and Unimodal regularization (MU) stage. In the SCL stage, we select reliable samples and adopt supervised contrastive learning to learn better representations. In the MU stage, we split samples with multiple annotations into multiple samples with a single annotation and shuffle them into different batches. To handle label noise, pseudo-labels are generated using the similarity between each sample and the central feature of each class, and temporal ensembling is used to obtain memory pseudo-labels that supervise the model training. To model the ordinal relation, we introduce unimodal regularization to keep the ordinal relation among classes in the predictions. Moreover, each lung nodule is characterized by three orthographic views. Experiments conducted on the LIDC-IDRI dataset indicate the superiority of our URL over other competing methods. Code is available at https://github.com/axz520/UR.
Point-supervised Temporal Action Localization (PSTAL) is an emerging research direction for label-efficient learning. However, current methods mainly focus on optimizing the network either at the snippet-level or the instance-level, neglecting the inherent reliability of point annotations at both levels. In this paper, we propose a Hierarchical Reliability Propagation (HR-Pro) framework, which consists of two reliability-aware stages: Snippet-level Discrimination Learning and Instance-level Completeness Learning, both stages explore the efficient propagation of high-confidence cues in point annotations. For snippet-level learning, we introduce an online-updated memory to store reliable snippet prototypes for each class. We then employ a Reliability-aware Attention Block to capture both intra-video and inter-video dependencies of snippets, resulting in more discriminative and robust snippet representation. For instance-level learning, we propose a point-based proposal generation approach as a means of connecting snippets and instances, which produces high-confidence proposals for further optimization at the instance level. Through multi-level reliability-aware learning, we obtain more reliable confidence scores and more accurate temporal boundaries of predicted proposals. Our HR-Pro achieves state-of-the-art performance on multiple challenging benchmarks, including an impressive average mAP of 60.3% on THUMOS14. Notably, our HR-Pro largely surpasses all previous point-supervised methods, and even outperforms several competitive fully supervised methods. Code will be available at https://github.com/pipixin321/HR-Pro.
The swift advancement in the scales and capabilities of Large Language Models (LLMs) positions them as promising tools for a variety of downstream tasks. In addition to the pursuit of better performance and the avoidance of violent feedback on a certain prompt, to ensure the responsibility of the LLM, much attention is drawn to the robustness of LLMs. However, existing evaluation methods mostly rely on traditional question answering datasets with predefined supervised labels, which do not align with the superior generation capabilities of contemporary LLMs. To address this issue, we propose a novel rational evaluation approach that leverages pre-trained reward models as diagnostic tools to evaluate the longer conversation generated from more challenging open questions by LLMs, which we refer to as the Reward Model for Reasonable Robustness Evaluation (TREvaL). Longer conversations manifest the comprehensive grasp of language models in terms of their proficiency in understanding questions, a capability not entirely encompassed by individual words or letters, which may exhibit oversimplification and inherent biases. Our extensive empirical experiments demonstrate that TREvaL provides an innovative method for evaluating the robustness of an LLM. Furthermore, our results demonstrate that LLMs frequently exhibit vulnerability to word-level perturbations that are commonplace in daily language usage. Notably, we are surprised to discover that robustness tends to decrease as fine-tuning (SFT and RLHF) is conducted. The code of TREval is available in https://github.com/Harry-mic/TREvaL.