Towards understanding deep learning with the natural clustering prior
원문 초록
The prior knowledge (a.k.a. priors) integrated into the design of a machine learning system strongly influences its generalization abilities. In the specific context of deep learning, some of these priors are poorly understood as they implicitly emerge from the successful heuristics and tentative approximations of biological brains involved in deep learning design. Through the lens of supervised image classification problems, this thesis investigates the implicit integration of a natural clustering prior composed of three statements: (i) natural images exhibit a rich clustered structure, (ii) image classes are composed of multiple clusters and (iii) each cluster contains examples from a single class. The decomposition of classes into multiple clusters implies that supervised deep learning systems could benefit from unsupervised clustering to define appropriate decision boundaries. Hence, this thesis attempts to identify implicit clustering abilities, mechanisms and hyperparameters in deep learning systems and evaluate their relevance for explaining the generalization abilities of these systems. We do so through an extensive empirical study of the training dynamics as well as the neuron- and layer-level representations of deep neural networks. The resulting collection of experiments provides preliminary evidence for the relevance of the natural clustering prior for understanding deep learning.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
GNN-ViTCap: GNN-Enhanced Multiple Instance Learning with Vision Transformers for Whole Slide Image Classification and Captioning
Hashed Watermark as a Filter: Defeating Forging and Overwriting Attacks in Weight-based Neural Network Watermarking
ViT-ProtoNet for Few-Shot Image Classification: A Multi-Benchmark Evaluation
MVNet: Hyperspectral Remote Sensing Image Classification Based on Hybrid Mamba-Transformer Vision Backbone Architecture
Beyond Accuracy: Metrics that Uncover What Makes a 'Good' Visual Descriptor
Linear Attention with Global Context: A Multipole Attention Mechanism for Vision and Physics
토론
이 논문과 연결된 포럼 글이 아직 없습니다.