A Visual Introduction to Hoeffding's Inequality - Statistical Learning Theory

This video is based on the following series of lectures:
Learning from data. Professor Yaser Abu-Mostafa, Caltech: https://tinyurl.com/4wkr7prx
Machine learning Theory. Professor Shai Ben-David and Shai Shalev-Shwartz: https://tinyurl.com/26v5btve

In this video we take a look at the strict Statistical Learning Theory framework for Supervised Classification. We take a quick look at Hoeffding's inequality, PAC Learning, the bias-complexity tradeoff and the feasibility of learning.

Timestamps :
0:00 – Intro
0:15 – Law of Large numbers
2:47 – Hoeffding’s Inequality
5:46 – Feasibility of Learning for Finite Hypothesis Classes
10:32 –The bias-complexity tradeoff
11:18 – Need for a better measure of complexity?
11:46 – The same is true for stochastic distributions as well!
_______________________________________
References:

Books:
Understanding Machine Learning: From Theory to Algorithms | Shai Shalev-Shwartz and Shai Ben-David

Learning From Data | Yaser Abu-Mostafa ,Malik Magdon-Ismail , Hsuan-Tien Lin

Foundations of Machine Learning | Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar


Blogs:
https://thirdorderscientist.org/homoclinic-orbit/2013/10/28/practical-inequalities-mdash-hoeffdings-inequality
https://scottaaronson.blog/?p=3712
https://oneraynyday.github.io/ml/2017/08/06/Journey-Through-SLT/
https://jeremykun.com/2014/01/02/probably-approximately-correct-a-formal-theory-of-learning/
https://web.stanford.edu/class/cs229t/notes.pdf
http://www.learningtheory.org/learning-has-just-started-an-interview-with-prof-vladimir-vapnik/
http://www.offconvex.org/2017/12/08/generalization1/
https://cs.stackexchange.com/questions/75327/why-is-deep-learning-hyped-despite-bad-vc-dimension
https://hunch.net/?p=496
https://hunch.net/?p=224
https://hunch.net/?cat=37&paged=2

Videos:
https://www.youtube.com/watch?v=kNWONiLbfVs&list=PLyGKBDfnk-iB4Xz_EAJNEgGF5I-6OzRNI&index=3
https://www.youtube.com/watch?v=JjsOezNsXbU
https://www.youtube.com/watch?v=3wbLr-NnIKI&list=PLTPQEx-31JXhguCush5J7OGnEORofoCW9
https://www.youtube.com/watch?v=rcR6P5O8CpU
https://www.youtube.com/watch?v=4RmpSvQ2LL0

What to watch if you find this too easy?
https://www.youtube.com/watch?v=PtA0lg_e5nA&list=PLn2-dEmQeTfQ8YVuHBOvAhUlnIPYxkeu3

__________________________________________________
Social Media:
Kartik Chincholikar Website: https://kartikchincholikar.github.io/
Github: https://github.com/kartikchincholikar
Twitter: https://twitter.com/KartikC14
_______________________________________________
I love coffee! Help fund future projects:
https://www.buymeacoffee.com/karti
_______________________________________________
Music:
Track: Such Memories
Music composed and recorded by Oak Studios
Creative Commons - Attribution ND 4.0
https://youtu.be/GspVwN-9_Fs