Focused Contrastive Training for Test-based Constituency Analysis
원문 초록
We propose a scheme for self-training of grammaticality models for constituency analysis based on linguistic tests. A pre-trained language model is fine-tuned by contrastive estimation of grammatical sentences from a corpus, and ungrammatical sentences that were perturbed by a syntactic test, a transformation that is motivated by constituency theory. We show that consistent gains can be achieved if only certain positive instances are chosen for training, depending on whether they could be the result of a test transformation. This way, the positives, and negatives exhibit similar characteristics, which makes the objective more challenging for the language model, and also allows for additional markup that indicates the position of the test application within the sentence.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
SeqPE: Transformer with Sequential Position Encoding
NeuralNexus at BEA 2025 Shared Task: Retrieval-Augmented Prompting for Mistake Identification in AI Tutors
JoFormer (Journey-based Transformer): Theory and Empirical Analysis on the Tiny Shakespeare Dataset
TransXSSM: A Hybrid Transformer State Space Model with Unified Rotary Position Embedding
DIBERT: Dependency Injected Bidirectional Encoder Representations from Transformers
A Bit of Progress in Language Modeling
토론
이 논문과 연결된 포럼 글이 아직 없습니다.