On the Expressive Power of Deep Architectures
(invited lecture for ALT & DS 2011)

Author: Yoshua Bengio and Olivier Delalleau

Affiliation: Department of Computer Science and Operations Research
Université de Montréal. Montréal (QC), H3C 3J7, Canada

Abstract. Deep architectures are families of functions corresponding to deep circuits. Deep Learning algorithms are based on parametrizing such circuits and tuning their parameters so as to approximately optimize some training objective. Whereas it was thought too difficult to train deep architectures, several successful algorithms have been proposed in recent years. We review some of the theoretical motivations for deep architectures, as well as some of their practical successes, and propose directions of investigations to address some of the remaining challenges.