Not to Overfit or Underfit the Source Domains? An Empirical Study of Domain Generalization in Question Answering
원문 초록
Machine learning models are prone to overfitting their training (source) domains, which is commonly believed to be the reason why they falter in novel target domains. Here we examine the contrasting view that multi-source domain generalization (DG) is first and foremost a problem of mitigating source domain underfitting: models not adequately learning the signal already present in their multi-domain training data. Experiments on a reading comprehension DG benchmark show that as a model learns its source domains better -- using familiar methods such as knowledge distillation (KD) from a bigger model -- its zero-shot out-of-domain utility improves at an even faster pace. Improved source domain learning also demonstrates superior out-of-domain generalization over three popular existing DG approaches that aim to limit overfitting. Our implementation of KD-based domain generalization is available via PrimeQA at: https://ibm.biz/domain-generalization-with-kd.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
SeqPE: Transformer with Sequential Position Encoding
Dynamic Chunking and Selection for Reading Comprehension of Ultra-Long Context in Large Language Models
VUDG: A Dataset for Video Understanding Domain Generalization
A Simple Linear Patch Revives Layer-Pruned Large Language Models
LogiQA 2.0—An Improved Dataset for Logical Reasoning in Natural Language Understanding
Disentangled Self-Supervision in Sequential Recommenders
토론
이 논문과 연결된 포럼 글이 아직 없습니다.