의료 비전-언어 모델(VLM)은 임상 영상 해석에서 점차 높은 잠재력을 보여주고 있습니다. 그러나 이러한 모델들은 여러 병리학적 표현형(phenotype)의 공동 평가가 진단에 요구되는 다중 표현형 병변을 해석하는 데 여전히 어려움을 겪고 있습니다. 기존의 비전-언어 정렬(alignment) 방법으로 생성된 시각 표현은 해부학적 위계와 표현형 하위 클래스 간의 관계를 제대로 보존하지 못합니다. 이는 시각 임베딩(embedding) 공간에서 이러한 관계를 보존하기 위한 기하학적 제약 조건이 결여된 의미론적 지도(semantic supervision)에 의존하기 때문입니다. 더욱이 병변 관련 해부학적 및 표현형 표현의 희소성(sparsity)으로 인해 의료 VLM이 중요한 진단적 증거를 포착하기 어렵습니다. 이러한 한계를 해결하기 위해, 본 논문에서는 의료 VLM의 다중 표현형 병변 해석을 위한 기하 지도 시각 표현 학습 프레임워크인 \textbf{PureVision}을 제안합니다. 이는 기하 지도 표현 학습 모듈인 \textbf{PureEyes}와 해부학 유도 증거 집약 모듈인 \textbf{PureNeurons}를 결합합니다. PureEyes는 해부학적 위계와 표현형 하위 클래스 관계를 인코딩하는 이상적인 공간 분포를 통해 기하학적 지도를 제공합니다. PureNeurons는 시각 표현을 학습된 잠재 공간에 투영하고, 해당 위치를 활용하여 병변 특이적 해부학적 및 표현형 증거를 선택적으로 집약합니다. \textit{LIDC-IDRI}, \textit{CBIS-DDSM}, \textit{3DReasonKnee}에 대한 실험을 통해 PureVision이 시각적 질의응답(VQA) 및 방사선학 보고서 생성에서 병변 그라운딩(grounding)과 표현형 특성화를 개선함을 입증합니다. 코드는 https://anonymous.4open.science/r/purevision-06C2 에서 확인할 수 있습니다.
현재 월드 모델은 단일 추상화 수준에서 동작하며, 대부분은 지각적 충실도(perceptual fidelity)를 우선시하는 한편 실제 세계 하위 태스크에 필요한 공간적 추론과 의미적 이해가 부족하다. 우리는 서로 다른 시간적·추상화 스케일에서 동작하는 두 수준에 걸쳐 미래 예측을 분해하는 계층적 주행 월드 모델을 제안한다: 확장된 시간적 지평에 걸쳐 거친 장면 구조를 예측하는 고수준 예측기와, 고수준 출력을 조건으로 상세한 예측을 생성하는 저수준 생성기. 이 분해는 높은 지각적 충실도를 달성하는 동시에 강력한 공간적·의미적 표현을 포착한다. 또한 diffusion forcing 목적 함수로 사전학습하면 표준 teacher forcing 목적 함수보다 훨씬 더 풍부한 내부 표현을 얻을 수 있음을 보이며, teacher forcing—깨끗한 컨텍스트에서 다음 프레임만 예측하는 방식—은 더 안정적인 자기회귀(autoregressive) 롤아웃을 생성한다. 따라서 diffusion forcing으로 모델을 사전학습하고 teacher forcing으로 파인튜닝하는 범용 2단계 학습 패러다임을 도입하여, 전자의 표현적 이점과 후자의 롤아웃 안정성을 결합한다. 우리 접근법은 장기 생성 충실도, 반사실(counterfactual) 시나리오에서 평가한 조향 반응성, 내부 표현 품질을 포함하여 확립된 벤치마크에서 표준 주행 월드 모델 평가 전반에 걸쳐 최첨단(state-of-the-art) 결과를 달성한다. 코드, 데모, 체크포인트 및 정성적 결과가 있는 프로젝트 페이지: https://lmb-freiburg.github.io/orbis2.github.io/
GRPO 방식의 강화학습에서 균일한 크레딧 할당이 문맥상 오류 있는 저확률 꼬리 토큰을 무분별하게 강화하는 '양성 크레딧 오염(Positive-Credit Contamination)' 문제를 규명하고, 이를 해결하기 위해 각 토큰의 꼬리 위험 점수를 계산해 양성 크레딧을 보정하는 TACO를 제안한다. 3개 LLM과 8개 벤치마크 실험에서 TACO는 GRPO 기반 모델 대비 일관된 성능 향상과 학습 안정성 개선을 달성했다.
BoN은 여러 후보를 샘플링하고 프록시 점수로 하나를 선택하여 모델 출력을 개선하지만, 완전한 후보를 신뢰할 수 있게 평가할 수 있다고 가정합니다. 많은 비전-언어 태스크는 대신 부분 검증만 제공합니다. 신뢰할 수 있는 전체 응답 검증기가 없더라도 소견(finding), 스팬(span), 값(value), 영역(region), 또는 관계(relation)는 검증 가능할 수 있습니다. 또한 동일한 주장이 상반된 입장을 가진 후보들 사이에서 반복될 수 있어, 하나의 관측이 풀의 일부를 뒷받침하면서 다른 후보와 모순될 수 있습니다. 우리는 BoN 후보 풀을 고정한 채 재사용 가능한 주장을 부호가 있는 후보-요인 그래프(signed candidate--factor graph)로 표현하고, 최종 선택을 바꿀 수 있는 증거 행동(evidence actions)에 제한된 예산을 배분하는 추론 시점 선택 프레임워크인 Best-of-Evidence(BoE)를 제안합니다. BoE는 부분 검증 하에서의 선택을 형식화하고 실용적인 점수 기반 컨트롤러를 제공하며, 예산이 0인 경우 기본 BoN 결정을 복원합니다. 이론적으로, 잔여 증거 용량(residual evidence capacity)이 증거 기반 개선의 한계를 제한함을 보이고, 공유 요인 쿼리(shared factor queries)가 요인-코드(factor-code) 모델에서 Θ(K) 대비 O(log K)의 쿼리 분리(query separation)를 달성할 수 있음을 보입니다. 네 가지 의료 VQA 설정에서의 Common-ledger 실험은 증거가 신뢰할 수 있고 대조적이며 결정에 관련될 때 BoE가 고정 풀 선택을 개선하고 일부 BoN 실패를 구제할 수 있음을 보여 주며, 동시에 보편적 이득을 막는 채널 품질(channel-quality)과 후보 생성(candidate-generation) 한계를 드러냅니다.
검증 가능한 보상을 활용한 강화학습(RLVR)은 단일 응답 수준의 보상을 모든 토큰에 브로드캐스팅하는 반면, 온-정책(on-policy) 증류(OPD)는 밀집된 어드밴티지를 얻기 위해 더 강력한 교사 모델과 비교하여 각 토큰을 점수화하지만, 성능이 교사의 수준으로 제한되고 이를 뛰어넘는 탐험을 저해합니다. 이러한 상호보완성은 RLVR과 OPD의 결합을 유망하게 만들지만, 두 어드밴티지를 고정 계수로 융합할 경우 두 가지 오보정으로 인해 엔트로피 붕괴가 유발됨을 발견했습니다. 이는 토큰 수준의 OPD 어드밴티지가 유계된(bounded) RLVR 어드밴티지보다 훨씬 크게 치솟아 해당 신호를 지워버리는 크기 불일치(magnitude mismatch)와, 지속적인 최대 강도의 OPD가 학생 모델을 계속 교사 쪽으로 끌어당겨 이를 넘어서는 데 필요한 탐험을 제한하는 시간적 불일치(temporal mismatch)입니다. 본 논문에서는 OPD 어드밴티지에만 적용되는 경량 4단계 파이프라인을 통해 두 문제를 모두 해결하는 안정적 어드밴티지 융합(Stable Advantage Fusion) 프레임워크인 SAF를 제안합니다. 이는 크기 제어를 위한 '희소화 후 압축(sparsify-then-compress)' 메커니즘과 시간 제어를 위한 '웜업 후 어닐링(warm-up-then-anneal)' 메커니즘을 결합한 것으로, 각 단계는 독립적으로 전환 가능하며 오버헤드를 거의 추가하지 않습니다. RLVR을 GRPO로 구체화하여 Qwen3-1.7B/4B/8B 모델과 7개의 수학적 추론 및 코드 생성 벤치마크 전반에서 SAF를 평가한 결과, SAF는 엔트로피 붕괴를 방지하고 고정 계수 기반의 GRPO+OPD 융합보다 일관되게 우수한 성능을 보였으며, 6개의 모든 모델-도메인 설정에서 총합 점수를 0.51-2.70% 향상시키는 동시에 더 안정적인 학습을 달성했습니다.
대규모 비전-언어 모델(LVLM)의 눈부신 발전에도 불구하고, 객체 환각(object hallucination)은 신뢰할 수 있는 배포를 저해하는 근본적인 과제로 남아 있습니다. 본 연구는 실제 객체 토큰과 환각된 객체 토큰이 은닉 표상(hidden representation)에서는 명확히 분리되지만, 언어 모델링(LM) 헤드에서는 이러한 분리 가능성이 대부분 소실된다는 핵심 발견에서 출발합니다. 본 논문에서는 보조 모델이나 추가적인 추론 비용 없이 LM 헤드가 객체별 진실성 신호를 드러내도록 학습시키는 자체 평가 프레임워크인 TruthLens를 제안합니다. 구체적으로는 거의 사용되지 않는 특수 토큰을 참조 토큰으로 재활용합니다. 각 객체 토큰 위치에서 LM 헤드가 이 특수 토큰에 할당한 로그 확률을 추출하고, 사전 정의된 상수와의 차이를 진실성 점수로 정의합니다. 그런 다음 실제 객체에 대해서는 점수가 1에 가까워지고 환각된 객체에 대해서는 0에 가까워지도록 유도하는 MSE 손실 함수로 모델을 파인튜닝(fine-tuning)하며, 발산 제약(divergence constraint)을 통해 원래의 생성 성능을 보존합니다. 제한된 객체 범주 세트에서만 학습되었음에도 불구하고, TruthLens는 훨씬 더 큰 레이블 공간을 가진 벤치마크에도 효과적으로 일반화됩니다. 여러 LVLM에 걸친 광범위한 실험을 통해 최고 수준의 성능을 입증하였으며, 특히 Qwen2.5-VL-7B에서 TruthLens는 MS-COCO 기준 기존 최고 성능 대비 AUROC를 17% 이상 능가합니다. 코드 정보는 https://github.com/wyqstan/TruthLens 에서 확인할 수 있습니다.
최근 이미지 편집 시스템은 뛰어난 의미론적 이해도, 시각적 충실도, 지시어 추종(instruction-following) 능력을 달성했으나, 비디오 편집은 여전히 훨씬 더 어렵고 비용이 많이 듭니다. 본 논문에서는 엔드투엔드(end-to-end) 비디오 편집에 대한 간단한 대안을 제시합니다. 즉, 단일 비디오 편집기를 학습(training)시키는 대신 강력한 이미지 편집기를 앵커 기반 생성을 통해 비디오 편집기로 변환합니다. 우리의 핵심 통찰은 비디오 편집이 희소한 키프레임 세트를 편집하는 것과 이러한 편집 내용을 시간에 걸쳐 전파하는 두 가지 하위 문제로 분해될 수 있다는 점입니다. 이러한 관찰을 바탕으로, 우리는 2단계 프레임워크인 앵커 기반 비디오 편집(Anchor-based Video Editing, AVE)을 제안합니다. 이 프레임워크에서는 먼저 강력한 이미지 편집기가 선택된 키프레임에 대해 합성 편집을 수행하고, 모션 유도 이미지-투-비디오 확산 모델(diffusion model)이 편집된 키프레임을 고정 앵커로 처리하여 최종 비디오를 생성합니다. 이러한 설계는 최신 이미지 편집기의 장점을 직접 계승하는 동시에 비용이 많이 드는 엔드투엔드 비디오 편집 학습을 방지합니다. IVEBench 및 VIE-Bench에 대한 실험 결과는 AVE가 지시어 추종, 시간적 일관성, 콘텐츠 충실도 면에서 우수한 성능을 달성함을 보여줍니다. 추가적인 절제 연구(ablation study)에 따르면 최종 비디오 편집 품질은 이미지 편집기의 품질과 강한 상관관계가 있음이 밝혀졌으며, 이는 향후 비디오 편집의 발전이 더 강력한 이미지 편집 기반과 비디오로의 경량 전이에서 비롯될 수 있음을 시사합니다. 코드는 https://github.com/wangf3014/AVE 에서 이용 가능합니다.
We present a new method for few-shot human motion transfer that achieves realistic human image generation with only a small number of appearance inputs. Despite recent advances in single person motion transfer, prior methods often require a large number of training images and take long training time. One promising direction is to perform few-shot human motion transfer, which only needs a few of source images for appearance transfer. However, it is particularly challenging to obtain satisfactory transfer results. In this paper, we address this issue by rendering a human texture map to a surface geometry (represented as a UV map), which is personalized to the source person. Our geometry generator combines the shape information from source images, and the pose information from 2D keypoints to synthesize the personalized UV map. A texture generator then generates the texture map conditioned on the texture of source images to fill out invisible parts. Furthermore, we may fine-tune the texture map on the manifold of the texture generator from a few source images at the test time, which improves the quality of the texture map without over-fitting or artifacts. Extensive experiments show the proposed method outperforms state-of-the-art methods both qualitatively and quantitatively. Our code is available at https://github.com/HuangZhiChao95/FewShotMotionTransfer.
In this paper, we explore open-domain sketch-to-photo translation, which aims to synthesize a realistic photo from a freehand sketch with its class label, even if the sketches of that class are missing in the training data. It is challenging due to the lack of training supervision and the large geometric distortion between the freehand sketch and photo domains. To synthesize the absent freehand sketches from photos, we propose a framework that jointly learns sketch-to-photo and photo-to-sketch generation. However, the generator trained from fake sketches might lead to unsatisfying results when dealing with sketches of missing classes, due to the domain gap between synthesized sketches and real ones. To alleviate this issue, we further propose a simple yet effective open-domain sampling and optimization strategy to "fool" the generator into treating fake sketches as real ones. Our method takes advantage of the learned sketch-to-photo and photo-to-sketch mapping of in-domain data and generalizes it to the open-domain classes. We validate our method on the Scribble and SketchyCOCO datasets. Compared with the recent competing methods, our approach shows impressive results in synthesizing realistic color, texture, and maintaining the geometric composition for various categories of open-domain sketches. Our code is available at https://github.com/Mukosame/AODA
Since human-labeled samples are free for the target set, unsupervised person re-identification (Re-ID) has attracted much attention in recent years, by additionally exploiting the source set. However, due to the differences on camera styles, illumination and backgrounds, there exists a large gap between source domain and target domain, introducing a great challenge on cross-domain matching. To tackle this problem, in this paper we propose a novel method named Dual-stream Reciprocal Disentanglement Learning (DRDL), which is quite efficient in learning domain-invariant features. In DRDL, two encoders are first constructed for id-related and id-unrelated feature extractions, which are respectively measured by their associated classifiers. Furthermore, followed by an adversarial learning strategy, both streams reciprocally and positively effect each other, so that the id-related features and id-unrelated features are completely disentangled from a given image, allowing the encoder to be powerful enough to obtain the discriminative but domain-invariant features. In contrast to existing approaches, our proposed method is free from image generation, which not only reduces the computational complexity remarkably, but also removes redundant information from id-related features. Extensive experiments substantiate the superiority of our proposed method compared with the state-of-the-arts. The source code has been released in https://github.com/lhf12278/DRDL.
Previous works on image inpainting mainly focus on inpainting background or partially missing objects, while the problem of inpainting an entire missing object remains unexplored. This work studies a new image inpainting task, i.e. shape-guided object inpainting. Given an incomplete input image, the goal is to fill in the hole by generating an object based on the context and implicit guidance given by the hole shape. Since previous methods for image inpainting are mainly designed for background inpainting, they are not suitable for this task. Therefore, we propose a new data preparation method and a novel Contextual Object Generator (CogNet) for the object inpainting task. On the data side, we incorporate object priors into training data by using object instances as holes. The CogNet has a two-stream architecture that combines the standard bottom-up image completion process with a top-down object generation process. A predictive class embedding module bridges the two streams by predicting the class of the missing object from the bottom-up features, from which a semantic object map is derived as the input of the top-down stream. Experiments demonstrate that the proposed method can generate realistic objects that fit the context in terms of both visual appearance and semantic meanings. Code can be found at the project page \url{https://zengxianyu.github.io/objpaint}
Recently, deep learning models have made great progress in MWP solving on answer accuracy. However, they are uninterpretable since they mainly rely on shallow heuristics to achieve high performance without understanding and reasoning the grounded math logic. To address this issue and make a step towards interpretable MWP solving, we first construct a high-quality MWP dataset named InterMWP which consists of 11,495 MWPs and annotates interpretable logical formulas based on algebraic knowledge as the grounded linguistic logic of each solution equation. Different from existing MWP datasets, our InterMWP benchmark asks for a solver to not only output the solution expressions but also predict the corresponding logical formulas. We further propose a novel approach with logical prompt and interpretation generation, called LogicSolver. For each MWP, our LogicSolver first retrieves some highly-correlated algebraic knowledge and then passes them to the backbone model as prompts to improve the semantic representations of MWPs. With these improved semantic representations, our LogicSolver generates corresponding solution expressions and interpretable knowledge formulas in accord with the generated solution expressions, simultaneously. Experimental results show that our LogicSolver has stronger logical formula-based interpretability than baselines while achieving higher answer accuracy with the help of logical prompts, simultaneously. The source code and dataset is available at https://github.com/yangzhch6/InterMWP.
Recently, with the rise of high dynamic range (HDR) display devices, there is a great demand to transfer traditional low dynamic range (LDR) images into HDR versions. The key to success is how to solve the many-to-many mapping problem. However, the existing approaches either do not consider constraining solution space or just simply imitate the inverse camera imaging pipeline in stages, without directly formulating the HDR image generation process. In this work, we address this problem by integrating LDR-to-HDR imaging knowledge into an UNet architecture, dubbed as Knowledge-inspired UNet (KUNet). The conversion from LDR-to-HDR image is mathematically formulated, and can be conceptually divided into recovering missing details, adjusting imaging parameters and reducing imaging noise. Accordingly , we develop a basic knowledge-inspired block (KIB) including three subnetworks corresponding to the three procedures in this HDR imaging process. The KIB blocks are cascaded in the similar way to the UNet to construct HDR image with rich global information. In addition, we also propose a knowledge inspired jump-connect structure to fit a dynamic range gap between HDR and LDR images. Experimental results demonstrate that the proposed KUNet achieves superior performance compared with the state-of-the-art methods. The code, dataset and appendix materials are available at https://github.com/wanghu178/KUNet.git.
Fine-tuning large pre-trained language models on downstream tasks has become an important paradigm in NLP. However, common practice fine-tunes all of the parameters in a pre-trained model, which becomes prohibitive when a large number of downstream tasks are present. Therefore, many fine-tuning methods are proposed to learn incremental updates of pre-trained weights in a parameter efficient way, e.g., low-rank increments. These methods often evenly distribute the budget of incremental updates across all pre-trained weight matrices, and overlook the varying importance of different weight parameters. As a consequence, the fine-tuning performance is suboptimal. To bridge this gap, we propose AdaLoRA, which adaptively allocates the parameter budget among weight matrices according to their importance score. In particular, AdaLoRA parameterizes the incremental updates in the form of singular value decomposition. Such a novel approach allows us to effectively prune the singular values of unimportant updates, which is essentially to reduce their parameter budget but circumvent intensive exact SVD computations. We conduct extensive experiments with several pre-trained models on natural language processing, question answering, and natural language generation to validate the effectiveness of AdaLoRA. Results demonstrate that AdaLoRA manifests notable improvement over baselines, especially in the low budget settings. Our code is publicly available at https://github.com/QingruZhang/AdaLoRA .
This paper raises the new task of Fisheye Semantic Completion (FSC), where dense texture, structure, and semantics of a fisheye image are inferred even beyond the sensor field-of-view (FoV). Fisheye cameras have larger FoV than ordinary pinhole cameras, yet its unique special imaging model naturally leads to a blind area at the edge of the image plane. This is suboptimal for safety-critical applications since important perception tasks, such as semantic segmentation, become very challenging within the blind zone. Previous works considered the out-FoV outpainting and in-FoV segmentation separately. However, we observe that these two tasks are actually closely coupled. To jointly estimate the tightly intertwined complete fisheye image and scene semantics, we introduce the new FishDreamer which relies on successful ViTs enhanced with a novel Polar-aware Cross Attention module (PCA) to leverage dense context and guide semantically-consistent content generation while considering different polar distributions. In addition to the contribution of the novel task and architecture, we also derive Cityscapes-BF and KITTI360-BF datasets to facilitate training and evaluation of this new track. Our experiments demonstrate that the proposed FishDreamer outperforms methods solving each task in isolation and surpasses alternative approaches on the Fisheye Semantic Completion. Code and datasets are publicly available at https://github.com/MasterHow/FishDreamer.
In recent years, Text-to-Image (T2I) models have been extensively studied, especially with the emergence of diffusion models that achieve state-of-the-art results on T2I synthesis tasks. However, existing benchmarks heavily rely on subjective human evaluation, limiting their ability to holistically assess the model's capabilities. Furthermore, there is a significant gap between efforts in developing new T2I architectures and those in evaluation. To address this, we introduce HRS-Bench, a concrete evaluation benchmark for T2I models that is Holistic, Reliable, and Scalable. Unlike existing bench-marks that focus on limited aspects, HRS-Bench measures 13 skills that can be categorized into five major categories: accuracy, robustness, generalization, fairness, and bias. In addition, HRS-Bench covers 50 scenarios, including fashion, animals, transportation, food, and clothes. We evaluate nine recent large-scale T2I models using metrics that cover a wide range of skills. A human evaluation aligned with 95% of our evaluations on average was conducted to probe the effectiveness of HRS-Bench. Our experiments demonstrate that existing models often struggle to generate images with the desired count of objects, visual text, or grounded emotions. We hope that our benchmark help ease future text-to-image generation research. The code and data are available at https://eslambakr.github.io/hrsbench.github.io
The disruptive application of ChatGPT (GPT-3.5, GPT-4) to a variety of domains has become a topic of much discussion in the scientific community and society at large. Large Language Models (LLMs), e.g., BERT, Bard, Generative Pre-trained Transformers (GPTs), LLaMA, etc., have the ability to take instructions, or prompts, from users and generate answers and solutions based on very large volumes of text-based training data. This paper assesses the impact and potential impact of ChatGPT on the field of digital forensics, specifically looking at its latest pre-trained LLM, GPT-4. A series of experiments are conducted to assess its capability across several digital forensic use cases including artefact understanding, evidence searching, code generation, anomaly detection, incident response, and education. Across these topics, its strengths and risks are outlined and a number of general conclusions are drawn. Overall this paper concludes that while there are some potential low-risk applications of ChatGPT within digital forensics, many are either unsuitable at present, since the evidence would need to be uploaded to the service, or they require sufficient knowledge of the topic being asked of the tool to identify incorrect assumptions, inaccuracies, and mistakes. However, to an appropriately knowledgeable user, it could act as a useful supporting tool in some circumstances.
Cross-View Geo-Localization (CVGL) estimates the location of a ground image by matching it to a geo-tagged aerial image in a database. Recent works achieve outstanding progress on CVGL benchmarks. However, existing methods still suffer from poor performance in cross-area evaluation, in which the training and testing data are captured from completely distinct areas. We attribute this deficiency to the lack of ability to extract the geometric layout of visual features and models' overfitting to low-level details. Our preliminary work introduced a Geometric Layout Extractor (GLE) to capture the geometric layout from input features. However, the previous GLE does not fully exploit information in the input feature. In this work, we propose GeoDTR+ with an enhanced GLE module that better models the correlations among visual features. To fully explore the LS techniques from our preliminary work, we further propose Contrastive Hard Samples Generation (CHSG) to facilitate model training. Extensive experiments show that GeoDTR+ achieves state-of-the-art (SOTA) results in cross-area evaluation on CVUSA, CVACT, and VIGOR by a large margin ($16.44\%$, $22.71\%$, and $13.66\%$ without polar transformation) while keeping the same-area performance comparable to existing SOTA. Moreover, we provide detailed analyses of GeoDTR+. Our code will be available at https://gitlab.com/vail-uvm/geodtr plus.
Pulmonary diseases rank prominently among the principal causes of death worldwide. Curing them will require, among other things, a better understanding of the complex 3D tree-shaped structures within the pulmonary system, such as airways, arteries, and veins. Traditional approaches using high-resolution image stacks and standard CNNs on dense voxel grids face challenges in computational efficiency, limited resolution, local context, and inadequate preservation of shape topology. Our method addresses these issues by shifting from dense voxel to sparse point representation, offering better memory efficiency and global context utilization. However, the inherent sparsity in point representation can lead to a loss of crucial connectivity in tree-shaped structures. To mitigate this, we introduce graph learning on skeletonized structures, incorporating differentiable feature fusion for improved topology and long-distance context capture. Furthermore, we employ an implicit function for efficient conversion of sparse representations into dense reconstructions end-to-end. The proposed method not only delivers state-of-the-art performance in labeling accuracy, both overall and at key locations, but also enables efficient inference and the generation of closed surface shapes. Addressing data scarcity in this field, we have also curated a comprehensive dataset to validate our approach. Data and code are available at \url{https://github.com/M3DV/pulmonary-tree-labeling}.
Consistency Models (CM) (Song et al., 2023) accelerate score-based diffusion model sampling at the cost of sample quality but lack a natural way to trade-off quality for speed. To address this limitation, we propose Consistency Trajectory Model (CTM), a generalization encompassing CM and score-based models as special cases. CTM trains a single neural network that can -- in a single forward pass -- output scores (i.e., gradients of log-density) and enables unrestricted traversal between any initial and final time along the Probability Flow Ordinary Differential Equation (ODE) in a diffusion process. CTM enables the efficient combination of adversarial training and denoising score matching loss to enhance performance and achieves new state-of-the-art FIDs for single-step diffusion model sampling on CIFAR-10 (FID 1.73) and ImageNet at 64x64 resolution (FID 1.92). CTM also enables a new family of sampling schemes, both deterministic and stochastic, involving long jumps along the ODE solution trajectories. It consistently improves sample quality as computational budgets increase, avoiding the degradation seen in CM. Furthermore, unlike CM, CTM's access to the score function can streamline the adoption of established controllable/conditional generation methods from the diffusion community. This access also enables the computation of likelihood. The code is available at https://github.com/sony/ctm.