태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
Emotional Intelligence (EI) is a measure of "The ability to monitor one’s own and others’ feelings, to discriminate among them, and to use this information to guide one’s thinking and action." (Salovey and Mayer, 1990). EI is further broken down into four branches: perceiving, using, understanding and managing emotions (Mayer & Salovey, 1997). Of particular relevance to language models that operate exclusively in the text modality is emotional understanding (EU). This is defined as the ability to interpret and analyse the language of emotions, to comprehend complex emotional states, and understand how these emotions can influence behaviour and decision-making.
출처에 같은 태스크 이름으로 연결된 데이터셋입니다. 동명 태스크가 포함될 수 있으므로, 비교할 때는 평가 결과의 조건도 확인하세요.
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to probe large language models and extrapolate their future capabilities. Big-bench include more than 200 tasks. Image source: https://arxiv.org/pdf/2206.04615.pdf
This dataset contains benchmark scores for EQ-Bench, a novel benchmark designed to evaluate aspects of emotional intelligence in Large Language Models (LLMs). We assess the ability of LLMs to understand complex emotions and social interactions by asking them to predict the intensity of emotional states of characters in a dialogue. The benchmark is able to discriminate effectively between a wide range of models. We find that EQ-Bench correlates strongly with comprehensive multi-domain benchmarks like MMLU (Hendrycks et al., 2020) (r=0.97), indicating that we may be capturing similar aspects of broad intelligence. Our benchmark produces highly repeatable results using a set of 60 English-language questions. We also provide open-source code for an automated benchmarking pipeline at https://github.com/EQ-bench/EQ-Bench and a leaderboard at https://www.eqbench.com.
A dataset of games played in the card game "Cards Against Humanity" (CAH), by human players, derived from the online CAH labs. Each round includes the cards presented to users - a "black" prompt with a blank or question and 10 "white" punchlines as possible responses, and which punchline was picked by a player each round, along with text and metadata. An example prompt is “TSA guidelines now prohibit ___ on airplanes”. Candidate punchlines are “Goblins”, “BATMAN!!!”, “Poor people”, and “The right amount of cocaine”. Importantly, many cards are offensive or politically incorrect. Used to explore human humor preferences. Available upon request from CAH labs: mail@cardsagainsthumanity.com Train/test splits and data processing available in the paper/code: "Cards Against AI: Predicting Humor in a Fill-in-the-blank Party Game": https://github.com/ddofer/CAH
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.