arXiv:2211.17201
공식 코드 · Papers with Code
ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT
원문 초록
In this paper, we present ExtremeBERT, a toolkit for accelerating and customizing BERT pretraining. Our goal is to provide an easy-to-use BERT pretraining toolkit for the research community and industry. Thus, the pretraining of popular language models on customized datasets is affordable with limited resources. Experiments show that, to achieve the same or better GLUE scores, the time cost of our toolkit is over $6\times$ times less for BERT Base and $9\times$ times less for BERT Large when compared with the original BERT paper. The documentation and code are released at https://github.com/extreme-bert/extreme-bert under the Apache-2.0 license.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations
Tougher Text, Smarter Models: Raising the Bar for Adversarial Defence Benchmarks
Cyber-Attack Technique Classification Using Two-Stage Trained Large Language Models
Multi-label Sequential Sentence Classification via Large Language Model
Large Language Models for Anomaly Detection in Computational Workflows: from Supervised Fine-Tuning to In-Context Learning
Estimating the Level of Dialectness Predicts Interannotator Agreement in Multi-dialect Arabic Datasets
토론
이 논문과 연결된 포럼 글이 아직 없습니다.