The Universal Approximation Theorem and the Debate Over Deep vs. Shallow Networks | Chapter 9 of Why Machines Learn
The Universal Approximation Theorem and the Debate Over Deep vs. Shallow Networks | Chapter 9 of Why Machines Learn Chapter 9, “The Man Who Set Back Deep Learning,” from Why Machines Learn: The Elegant Math Behind Modern AI explores the surprising legacy of George Cybenko’s 1989 proof of the universal approximation theorem. Although now regarded as one of the foundational results in neural network theory, the theorem was ironically misinterpreted in ways that may have temporarily slowed progress toward deep learning. In this chapter, Anil Ananthaswamy connects functional analysis, the geometry of infinite-dimensional spaces, and the mysteries of modern deep networks to show how a single mathematical insight shaped decades of AI research. To follow the detailed mathematical reasoning behind Cybenko’s theorem, be sure to watch the full video summary above. Supporting Last Minute Lecture helps us continue providing accessible, academically grounded explorations of complex AI conce...