Few-Shot Text Classification

The Stanford Sentiment Treebank is a corpus with fully labeled parse trees that allows for a complete analysis of the compositional effects of sentiment in language. The corpus is based on the dataset introduced by Pang and Lee (2005) and consists of 11,855 single sentences extracted from movie reviews. It was parsed with the Stanford parser and includes a total of 215,154 unique phrases from those parse trees, each annotated by 3 human judges. Each phrase is labelled as either negative, somewhat negative, neutral, somewhat positive or positive. The corpus with all 5 labels is referred to as SST-5 or SST fine-grained. Binary classification experiments on full sentences (negative or somewhat negative vs somewhat positive or positive with neutral sentences discarded) refer to the dataset as SST-2 or SST binary.

The SST-5, also known as the Stanford Sentiment Treebank with 5 labels, is a dataset used for sentiment analysis. The SST-5 dataset consists of 11,855 single sentences extracted from movie reviews¹. It includes a total of 215,154 unique phrases from parse trees, each annotated by 3 human judges¹. Each phrase is labeled as either negative, somewhat negative, neutral, somewhat positive, or positive. This is why it's referred to as SST-5 or SST fine-grained.

The RAFT benchmark (Realworld Annotated Few-shot Tasks) focuses on naturally occurring tasks and uses an evaluation setup that mirrors deployment. RAFT is a few-shot classification benchmark that tests language models: - across multiple domains (lit reviews, medical data, tweets, customer interaction, etc.) - on economically valuable classification tasks (someone inherently cares about the task) - with evaluation that mirrors deployment (50 labeled examples per task, info retrieval allowed, hidden test set) Description from: https://raft.elicit.org/ Image source: https://raft.elicit.org/

Pano3D is a new benchmark for depth estimation from spherical panoramas. Its goal is to drive progress for this task in a consistent and holistic manner. The Pano3D 360 depth estimation benchmark provides a standard Matterport3D train and test split, as well as a secondary GibsonV2 partioning for testing and training as well. The latter is used for zero-shot cross dataset transfer performance assessment and decomposes it into 3 different splits, each one focusing on a specific generalization axis.

A dataset specifically tailored to the biotech news sector, aiming to transcend the limitations of existing benchmarks. This dataset is rich in complex content, comprising various biotech news articles covering various events, thus providing a more nuanced view of information extraction challenges. A standout feature of our dataset is its dual-layered approach: it not only identifies the general theme of the articles but also extracts specific information about the target companies involved. This dual focus significantly elevates the dataset’s relevance for applications demanding a deeper insight into the interplay of events, companies, and the broader biotech industry. The dataset encompasses 31 classes, including a ‘None’ category, to cover various events and information types such as event organisation, executive statements, regulatory approvals, hiring announcements, and more. Key aspects: - Event extraction - Multi-class classification - Biotech news domain - 31 classes - 3140 total number of examples Classes: - alliance & partnership - forming an alliance or partnership - article publication - publishing an article - clinical trial sponsorship - Sponsoring or participating in a clinical trial - closing - shutting down a facility/office/division or ceasing an initiative - company description - describing or profiling the company - department establishment - establishing a new department or division - event organization - organizing or participating in an event like a conference, exhibition, etc - event organization - organizing or participating in an event - executive appointment - appointing a new executive - executive statement - a statement or quote from an executive of a company - expanding geography - expanding into new geographical areas - expanding industry - expanding into new industries or markets - foundation - establishing a new charitable foundation - funding round - raising a new round of funding - hiring - announcing new hires or appointments at the company - investment in public company - making an investment in a public company - ipo exit - having an initial public offering or acquisition exit - m&a - mergers, acquisitions, or divestitures - new initiatives & programs - announcing new initiatives or programs - new initiatives or programs - announcing new initiatives, programs, or campaigns - None - no label - other - other events that don't fit into defined categories - participation in an event - participating in an industry event, conference, etc - partnerships & alliances - forming partnerships or strategic alliances with other companies - patent publication - publication of a new patent filing - product launching & presentation - launching or unveiling a new product - product updates - announcing updates or new versions of existing products - regulatory approval - getting approval from regulatory bodies for products, services, trials, etc. - service & product providing - launching or expanding products or services - subsidiary establishment - establishing a new subsidiary company - support & philanthropy - philanthropic activities or donations