GeoMask3D: Geometrically Informed Mask Selection for Self-Supervised Point Cloud Learning in 3D
원문 초록
We introduce a pioneering approach to self-supervised learning for point clouds, employing a geometrically informed mask selection strategy called GeoMask3D (GM3D) to boost the efficiency of Masked Auto Encoders (MAE). Unlike the conventional method of random masking, our technique utilizes a teacher-student model to focus on intricate areas within the data, guiding the model's focus toward regions with higher geometric complexity. This strategy is grounded in the hypothesis that concentrating on harder patches yields a more robust feature representation, as evidenced by the improved performance on downstream tasks. Our method also presents a complete-to-partial feature-level knowledge distillation technique designed to guide the prediction of geometric complexity utilizing a comprehensive context from feature-level information. Extensive experiments confirm our method's superiority over State-Of-The-Art (SOTA) baselines, demonstrating marked improvements in classification, and few-shot tasks.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Efficient and Generalizable Speaker Diarization via Structured Pruning of Self-Supervised Models
Continual Self-Supervised Learning with Masked Autoencoders in Remote Sensing
Enhancing Few-shot Keyword Spotting Performance through Pre-Trained Self-supervised Speech Models
Fine-tune Before Structured Pruning: Towards Compact and Accurate Self-Supervised Models for Speaker Diarization
DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition
The Trilemma of Truth in Large Language Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.