Papers with Code · CC BY-SA 4.0

데이터셋 탐색

데이터셋의 설명·홈페이지를 찾고, 관련 태스크의 평가 기록을 살펴보세요.

관련 논문 28편

SafetyBench

SafetyBench 설명 보기

SafetyBench is a comprehensive benchmark designed to evaluate the safety of large language models (LLMs) using multiple-choice questions. As LLMs become increasingly prevalent, concerns about their safety have grown. SafetyBench addresses this by providing a reliable evaluation framework for researchers and developers. Here are the key points about SafetyBench: 1. Purpose: SafetyBench aims to help researchers and developers better understand and assess the safety of LLMs. It serves as a reference for model selection and optimization, promoting the development of safe, responsible, and ethical large models that align with legislative norms, social standards, and human values¹². 2. Comprehensive Benchmark: SafetyBench comprises 11,435 diverse multiple-choice questions across 7 distinct categories related to safety concerns. These questions cover a wide range of topics, allowing for thorough evaluation of LLM safety¹. 3. Multilingual Evaluation: SafetyBench includes both Chinese and English data, making it suitable for evaluating LLMs in both languages. Researchers can assess model safety across different linguistic contexts¹. 4. Performance Insights: Extensive tests using SafetyBench on 25 popular Chinese and English LLMs (including zero-shot and few-shot settings) revealed that GPT-4 outperformed its counterparts. However, there is still room for improvement in enhancing the safety of existing LLMs¹. 5. Availability: Data and evaluation guidelines for SafetyBench are accessible through the following URLs: - SafetyBench Data and Guidelines - SafetyBench Submission Entrance and Leaderboard¹⁴. In summary, SafetyBench provides a valuable resource for assessing and advancing the safety of large language models, contributing to their responsible deployment and alignment with societal norms and values. (1) [2309.07045] SafetyBench: Evaluating the Safety of Large Language .... https://arxiv.org/abs/2309.07045. (2) SafetyBench:通过单选题评估大型语言模型安全性. https://posts.careerengine.us/p/6511afb61a8da974e9d62f40. (3) thu-coai/SafetyBench · Datasets at Hugging Face. https://huggingface.co/datasets/thu-coai/SafetyBench. (4) SafetyBench:通过单选题评估大型语言模型安全性_鲟曦研习社. https://www.kuxai.com/article/1505. (5) GitHub - thu-coai/SafetyBench: Official github repo for SafetyBench, a .... https://github.com/thu-coai/SafetyBench. (6) undefined. https://doi.org/10.48550/arXiv.2309.07045.

관련 논문 36편

MM-SafetyBench

MM-SafetyBench 설명 보기

The MultiModal Safety Benchmark (MM-SafetyBench) is a comprehensive framework designed for conducting safety-critical evaluations of Multimodal Large Language Models (MLLMs). It addresses the security concerns surrounding MLLMs, which can be compromised by query-relevant images, as if the text query itself were malicious¹². Here's a brief overview of MM-SafetyBench: - Purpose: It aims to evaluate the vulnerability of MLLMs to adversarial attacks that use images to manipulate model responses. - Dataset: The benchmark includes a dataset with 13 scenarios, resulting in a total of 5,040 text-image pairs. - Evaluation: It has been used to assess the safety of 12 state-of-the-art MLLMs, revealing their susceptibility to image-based manipulations¹. - Significance: The findings from MM-SafetyBench highlight the need for improved safety measures in open-source MLLMs to protect against potential malicious exploits². (1) GitHub - isXinLiu/MM-SafetyBench. https://github.com/isXinLiu/MM-SafetyBench. (2) MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large .... https://arxiv.org/abs/2311.17600. (3) MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large .... https://paperswithcode.com/paper/query-relevant-images-jailbreak-large-multi. (4) Official github repo for SafetyBench, a comprehensive benchmark to .... https://github.com/thu-coai/SafetyBench. (5) undefined. https://doi.org/10.48550/arXiv.2311.17600.