PAC-learning gains of Turing machines over circuits and neural networks
원문 초록
A caveat to many applications of the current Deep Learning approach is the need for large-scale data. One improvement suggested by Kolmogorov Complexity results is to apply the minimum description length principle with computationally universal models. We study the potential gains in sample efficiency that this approach can bring in principle. We use polynomial-time Turing machines to represent computationally universal models and Boolean circuits to represent Artificial Neural Networks (ANNs) acting on finite-precision digits. Our analysis unravels direct links between our question and Computational Complexity results. We provide lower and upper bounds on the potential gains in sample efficiency between the MDL applied with Turing machines instead of ANNs. Our bounds depend on the bit-size of the input of the Boolean function to be learned. Furthermore, we highlight close relationships between classical open problems in Circuit Complexity and the tightness of these.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Lean Formalization of Generalization Error Bound by Rademacher Complexity
Prospective Learning: Learning for a Dynamic Future
Optimistic Rates for Learning from Label Proportions
Towards a theory of model distillation
SAT-Based PAC Learning of Description Logic Concepts
Privacy Induces Robustness: Information-Computation Gaps and Sparse Mean Estimation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.