Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently
원문 초록
Recent zero-shot evaluations have highlighted important limitations in the abilities of language models (LMs) to perform meaning extraction. However, it is now well known that LMs can demonstrate radical improvements in the presence of experimental contexts such as in-context examples and instructions. How well does this translate to previously studied meaning-sensitive tasks? We present a case-study on the extent to which experimental contexts can improve LMs' robustness in performing property inheritance -- predicting semantic properties of novel concepts, a task that they have been previously shown to fail on. Upon carefully controlling the nature of the in-context examples and the instructions, our work reveals that they can indeed lead to non-trivial property inheritance behavior in LMs. However, this ability is inconsistent: with a minimal reformulation of the task, some LMs were found to pick up on shallow, non-semantic heuristics from their inputs, suggesting that the computational principles of semantic property inference are yet to be mastered by LMs.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
AFANet: Adaptive Frequency-Aware Network for Weakly-Supervised Few-Shot Semantic Segmentation
NeSyCoCo: A Neuro-Symbolic Concept Composer for Compositional Generalization
PersonaMagic: Stage-Regulated High-Fidelity Face Customization with Tandem Equilibrium
Grounding Descriptions in Images informs Zero-Shot Visual Recognition
SAFE: Slow and Fast Parameter-Efficient Tuning for Continual Learning with Pre-Trained Models
Happy: A Debiased Learning Framework for Continual Generalized Category Discovery
토론
이 논문과 연결된 포럼 글이 아직 없습니다.