Watch this great overview of history and present state of Deep Learning, which is revolutionizing Machine learning, vision, robotics, and many other areas.

Here is an excellent, very comprehensive, and entertaining overview and history of deep learning and recurrent neural network, given at Berkeley in August 2014 by ProfessorJurgen Schmidhuber of IDSIA, Switzerland.

Abstract:
Machine learning and pattern recognition are currently being revolutionized by "Deep Learning" (DL) Neural Networks (NNs). This is of commercial interest (for example, Google spent over $400 million on the start-up "DeepMind," co-founded by our student). My talk will summarize our work on DL since 1991. Our recurrent NNs (RNNs) were the first to win official international competitions in pattern recognition and machine learning; our team has won more such contests than any other research group or company. In particular, our RNNs represent the state of the art in connected handwriting recognition, and aspects of speech recognition. We also built the first artificial RNN-based agent that learns from scratch complex control based on high-dimensional vision.



More information about the talk:

• https://www.icsi.berkeley.edu/icsi/events/2014/08/schmidhuber-rnn
  

Slides from the talk (PDF)

Deep Learning survey with over 850 references:

• www.idsia.ch/~juergen/deep-learning-overview.html
  

Bio: Since age 15 or so, Professor Jurgen Schmidhuber's main scientific ambition has been to build an optimal scientist through self-improving AI, then retire. His AI team has won nine international competitions in machine learning and pattern recognition and seven independent best paper/best video awards, achieved the world's first superhuman visual classification results, and established the field of mathematically rigorous universal AI and optimal universal problem solvers. His formal theory of creativity and curiosity and fun explains art, science, music, and humor.

He also generalized algorithmic information theory and the many-worlds theory of physics, and introduced the concept of Low-Complexity Art, the information age's extreme form of minimal art. Many famous companies are now using the techniques developed in his group at the Swiss AI Lab IDSIA (a Business Week Top 10 AI Lab) and USI & SUPSI. Since 2009, he has been a member of the European Academy of Sciences and Arts. He has published more than 300 peer-reviewed papers, and is recipient of the 2013 Helmholtz Award of the International Neural Networks Society. Home page: www.idsia.ch/~juergen/