depth-width trade-offs

Survey: Approximation Theory for Neural Networks — Classical Results and New Directions Including KANs

This arxiv survey reviews four decades of universal approximation theory for feedforward neural networks, covering classical density results for single-hidden-layer networks and quantitative bounds relating approximation error to network size and target function smoothness. It gives particular emphasis to depth-width trade-offs and the parameter efficiency advantages of deeper architectures for structured function classes. The survey also covers recent theoretical developments on Kolmogorov-Arnold Networks (KANs) as an alternative architectural paradigm with emerging approximation-theoretic analysis.