Fundamental Limits of Prediction, Generalization, and Recursion: An Entropic-Innovations Perspective
원문 초록
In this paper, we examine the fundamental performance limits of prediction, with or without side information. More specifically, we derive generic lower bounds on the $\mathcal{L}_p$ norms of the prediction errors that are valid for any prediction algorithms and for any data distributions. Meanwhile, we combine the entropic analysis from information theory and the innovations approach from prediction/estimation theory to characterize the conditions (in terms of, e.g., directed information or mutual information) to achieve the bounds. We also investigate the implications of the results in analyzing the fundamental limits of generalization in fitting (learning) problems from the perspective of prediction with side information, as well as the fundamental limits of recursive algorithms by viewing them as generalized prediction problems.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Optimal Conformal Prediction under Epistemic Uncertainty
Valid Selection among Conformal Sets
Anytime-valid, Bayes-assisted,Prediction-Powered Inference
Conformation-Aware Structure Prediction of Antigen-Recognizing Immune Proteins
Cautious Next Token Prediction
Where, What, Why: Towards Explainable Driver Attention Prediction
토론
이 논문과 연결된 포럼 글이 아직 없습니다.