Function Contrastive Learning of Transferable Meta-Representations
원문 초록
Meta-learning algorithms adapt quickly to new tasks that are drawn from the same task distribution as the training tasks. The mechanism leading to fast adaptation is the conditioning of a downstream predictive model on the inferred representation of the task's underlying data generative process, or \emph{function}. This \emph{meta-representation}, which is computed from a few observed examples of the underlying function, is learned jointly with the predictive model. In this work, we study the implications of this joint training on the transferability of the meta-representations. Our goal is to learn meta-representations that are robust to noise in the data and facilitate solving a wide range of downstream tasks that share the same underlying functions. To this end, we propose a decoupled encoder-decoder approach to supervised meta-learning, where the encoder is trained with a contrastive objective to find a good representation of the underlying function. In particular, our training scheme is driven by the self-supervision signal indicating whether two sets of examples stem from the same function. Our experiments on a number of synthetic and real-world datasets show that the representations we obtain outperform strong baselines in terms of downstream performance and noise robustness, even when these baselines are trained in an end-to-end manner.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Few-Shot Learning for Industrial Time Series: A Comparative Analysis Using the Example of Screw-Fastening Process Monitoring
Probabilistic Variational Contrastive Learning
CoReGAN: Contrastive Regularized Generative Adversarial Network for Guided Depth Map Super Resolution
CCML: Curriculum and Contrastive Learning Enhanced Meta-Learner for Personalized Spatial Trajectory Prediction
CrossMoCo: Multi-modal Momentum Contrastive Learning for Point Cloud
A Unified Framework against Topology and Class Imbalance
토론
이 논문과 연결된 포럼 글이 아직 없습니다.