AggreFact
The AggreFact dataset is a benchmark for evaluating the factuality of summaries generated by different summarization models. It aggregates factuality error annotations from nine existing datasets and stratifies them according to the underlying summarization model. The dataset contains the following columns: - dataset: Name of the original annotated dataset. - origin: Summarization dataset. Either cnndm or xsum. - id: Document id. - doc: Input article. - summary: Model generated summary. - model_name: Name of the model used to generate the summary. - label: Factual consistency of the generated summary. 1 is factually consistent, 0 otherwise. - cut: Either val or test. - system _score: The output score from a factuality system. - system _label: The binary factual consistency label based on the score of the factuality system.