QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questions
원문 초록
We introduce the first open-domain dataset, called QuaRTz, for reasoning about textual qualitative relationships. QuaRTz contains general qualitative statements, e.g., "A sunscreen with a higher SPF protects the skin longer.", twinned with 3864 crowdsourced situated questions, e.g., "Billy is wearing sunscreen with a lower SPF than Lucy. Who will be best protected from the sun?", plus annotations of the properties being compared. Unlike previous datasets, the general knowledge is textual and not tied to a fixed set of relationships, and tests a system's ability to comprehend and apply textual qualitative knowledge in a novel setting. We find state-of-the-art results are substantially (20%) below human performance, presenting an open challenge to the NLP community.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
TaxoAdapt: Aligning LLM-Based Multidimensional Taxonomy Construction to Evolving Research Corpora
Improving Personalized Search with Regularized Low-Rank Parameter Updates
HSSBench: Benchmarking Humanities and Social Sciences Ability for Multimodal Large Language Models
HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts
BnMMLU: Measuring Massive Multitask Language Understanding in Bengali
토론
이 논문과 연결된 포럼 글이 아직 없습니다.