The AI off-switch problem as a signalling game: bounded rationality and incomparability
원문 초록
The off-switch problem is a critical challenge in AI control: if an AI system resists being switched off, it poses a significant risk. In this paper, we model the off-switch problem as a signalling game, where a human decision-maker communicates its preferences about some underlying decision problem to an AI agent, which then selects actions to maximise the human's utility. We assume that the human is a bounded rational agent and explore various bounded rationality mechanisms. Using real machine learning models, we reprove prior results and demonstrate that a necessary condition for an AI system to refrain from disabling its off-switch is its uncertainty about the human's utility. We also analyse how message costs influence optimal strategies and extend the analysis to scenarios involving incomparability.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
$τ^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
Comparative Analysis of AI Agent Architectures for Entity Relationship Classification
ThinkTank: A Framework for Generalizing Domain-Specific AI Agent Systems into Universal Collaborative Intelligence Platforms
AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning
Will Agents Replace Us? Perceptions of Autonomous Multi-Agent AI
From Knowledge to Noise: CTIM-Rover and the Pitfalls of Episodic Memory in Software Engineering Agents
토론
이 논문과 연결된 포럼 글이 아직 없습니다.