RoQLlama: A Lightweight Romanian Adapted Language Model
원문 초록
The remarkable achievements obtained by open-source large language models (LLMs) in recent years have predominantly been concentrated on tasks involving the English language. In this paper, we aim to advance the performance of Llama2 models on Romanian tasks. We tackle the problem of reduced computing resources by using QLoRA for training. We release RoQLlama-7b, a quantized LLM, which shows equal or improved results compared to its full-sized counterpart when tested on seven Romanian downstream tasks in the zero-shot setup. Also, it consistently achieves higher average scores across all few-shot prompts. Additionally, we introduce a novel Romanian dataset, namely RoMedQA, which contains single-choice medical questions in Romanian.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Making Language Model a Hierarchical Classifier and Generator
CoLMbo: Speaker Language Model for Descriptive Profiling
Data Efficacy for Language Model Training
Enterprise Large Language Model Evaluation Benchmark
Research on Model Parallelism and Data Parallelism Optimization Methods in Large Language Model-Based Recommendation Systems
Finance Language Model Evaluation (FLaME)
토론
이 논문과 연결된 포럼 글이 아직 없습니다.