Embedding-based statistical inference on generative models
원문 초록
The recent cohort of publicly available generative models can produce human expert level content across a variety of topics and domains. Given a model in this cohort as a base model, methods such as parameter efficient fine-tuning, in-context learning, and constrained decoding have further increased generative capabilities and improved both computational and data efficiency. Entire collections of derivative models have emerged as a byproduct of these methods and each of these models has a set of associated covariates such as a score on a benchmark, an indicator for if the model has (or had) access to sensitive information, etc. that may or may not be available to the user. For some model-level covariates, it is possible to use "similar" models to predict an unknown covariate. In this paper we extend recent results related to embedding-based representations of generative models -- the data kernel perspective space -- to classical statistical inference settings. We demonstrate that using the perspective space as the basis of a notion of "similar" is effective for multiple model-level inference tasks.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Leveraging Large Language Models for enzymatic reaction prediction and characterization
Optimization-Inspired Few-Shot Adaptation for Large Language Models
Assay2Mol: large language model-based drug design using BioAssay context
LoSiA: Efficient High-Rank Fine-Tuning via Subnet Localization and Optimization
Detecting Referring Expressions in Visually Grounded Dialogue with Autoregressive Language Models
Universal pre-training by iterated random computation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.