Papers with Code · CC BY-SA 4.0
Multiple Choice Question Answering (MCQA)
A multiple-choice question (MCQ) is composed of two parts: a stem that identifies the question or problem, and a set of alternatives or possible answers that contain a key that is the best answer to the question, and a number of distractors that are plausible but incorrect answers to the question.
In a k-way MCQA task, a model is provided with a question q, a set of candidate options O = {O1, . . . , Ok}, and a supporting context for each option C = {C1, . . . , Ck}. The model needs to predict the correct answer option that is best supported by the given contexts.
데이터셋
31개MedMCQABIG-bench (Hyperbaton)BIG-bench (Movie Recommendation)BIG-bench (Navigate)BIG-bench (Ruin Names)MMLU (College Biology)MMLU (Medical Genetics)MMLU (Professional medicine)MMLU (Abstract Algebra)MMLU (Astronomy)MMLU (College Chemistry)MMLU (College Mathematics)MMLU (College Physics)MMLU (Econometrics)MMLU (Electrical Engineer)MMLU (Elementary Mathematics)MMLU (Formal Logic)MMLU (High School Biology)MMLU (High School Computer Science)MMLU (High School Mathematics)MMLU (High School Statistics)BIG-bench (Novel Concepts)MMLU (College Computer Science)MMLU (High School Chemistry)MMLU (High School Physics)MMLU (Machine Learning)IndicGLUE WSTP PaMMLU (Anatomy)MMLU (Clinical Knowledge)MMLU (College Medicine)