A Theoretical Survey on Foundation Models
원문 초록
Understanding the inner mechanisms of black-box foundation models (FMs) is essential yet challenging in artificial intelligence and its applications. Over the last decade, the long-running focus has been on their explainability, leading to the development of post-hoc explainable methods to rationalize the specific decisions already made by black-box FMs. However, these explainable methods have certain limitations in terms of faithfulness and resource requirement. Consequently, a new class of interpretable methods should be considered to unveil the underlying mechanisms of FMs in an accurate, comprehensive, heuristic, and resource-light way. This survey aims to review those interpretable methods that comply with the aforementioned principles and have been successfully applied to FMs. These methods are deeply rooted in machine learning theory, covering the analysis of generalization performance, expressive capability, and dynamic behavior. They provide a thorough interpretation of the entire workflow of FMs, ranging from the inference capability and training dynamics to their ethical implications. Ultimately, drawing upon these interpretations, this review identifies the next frontier research directions for FMs.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
야생에서의 보안 및 프라이버시 프롬프트: 사용자가 LLM에게 묻는 내용과 LLM의 응답 방식
Security and Privacy Prompts in the Wild: What Users Ask LLMs and How LLMs Respond
AI 샌드박스: 위협 모델, 분류 체계 및 측정 프레임워크
AI Sandboxes: A Threat Model, Taxonomy, and Measurement Framework
영향력 있는 다원주의적 정렬 연구를 위한 로드맵
A Roadmap to Impactful Pluralistic Alignment Research
Towards Trustworthy Physical Intelligence: From Theory to Practice Across Life Cycle
하나의 해답을 향한 세 가지 경로: 응용 연구자를 위한 AIPW, TMLE 및 이중 머신러닝의 일치화
Three Routes to One Answer: Reconciling AIPW, TMLE, and Double Machine Learning for Applied Researchers
에이전트 스킬 활용: 스킬 중개 LLM 에이전트를 위한 아키텍처 패턴과 참조 아키텍처
Harnessing Agent Skills: Architectural Patterns and a Reference Architecture for Skill-Mediated LLM Agents
토론
이 논문과 연결된 포럼 글이 아직 없습니다.