Controllable Text Generation for Open-Domain Creativity and Fairness
원문 초록
Recent advances in large pre-trained language models have demonstrated strong results in generating natural languages and significantly improved performances for many natural language generation (NLG) applications such as machine translation and text summarization. However, when the generation tasks are more open-ended and the content is under-specified, existing techniques struggle to generate long-term coherent and creative content. Moreover, the models exhibit and even amplify social biases that are learned from the training corpora. This happens because the generation models are trained to capture the surface patterns (i.e. sequences of words), instead of capturing underlying semantics and discourse structures, as well as background knowledge including social norms. In this paper, I introduce our recent works on controllable text generation to enhance the creativity and fairness of language generation models. We explore hierarchical generation and constrained decoding, with applications to creative language generation including story, poetry, and figurative languages, and bias mitigation for generation models.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Document-Level Text Generation with Minimum Bayes Risk Decoding using Optimal Transport
Enhancing Paraphrase Type Generation: The Impact of DPO and RLHF Evaluated with Human-Ranked Data
MaCP: Minimal yet Mighty Adaptation via Hierarchical Cosine Projection
Twitter Sentiment Analysis
APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation
A Reproducibility Study of Product-side Fairness in Bundle Recommendation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.