Softmax Function Explained In Depth with 3D Visuals
視聴回数 33,666 回
0
0
The softmax function is often used in machine learning to transform the outputs of the last layer of your neural network (the logits) into probabilities. In this video, I explain how the softmax function works and provide some intuition for thinking about it in higher dimensions. In addition to using the softmax function for classification, it’s also used in models that use attention, such as transformer models. The softmax function is very similar to the sigmoid function, except it’s generalized for higher dimensions, so if you’re also interested in learning more about the sigmoid function, check out my previous video about it linked below.
My previous video, "Why We Use the Sigmoid Function in Neural Networks for Binary Classification":
📼 https://youtu.be/WsFasV46KgQ
My other video, "Derivative of Sigmoid and Softmax Explained Visually":
📼 https://youtu.be/gRr2Q97XS2g
GitHub code for visualizing how the logit values that are passed into the softmax function change over time as the model is trained with SGD (stochastic gradient descent) or the Adam optimizer:
💻 https://github.com/elliotwaite/softmax-logit-paths
Desmos 2D graph of softmax for 4 classes:
📈 https://www.desmos.com/calculator/drqqhtb037
GeoGebra 3D graph of softmax for 2 classes (with derivatives and Gaussians):
📈 https://www.geogebra.org/classic/qhdd4afw
GeoGebra 3D graph of softmax for 3 classes (with derivatives):
📈 https://www.geogebra.org/classic/ps9gwjav
GeoGebra 3D graph of softmax with Gaussians for 3 classes:
📈 https://www.geogebra.org/classic/vgwaw7wr
GeoGebra 3D graph of the shape of the softmax input space for 4 and 5 classes:
📈 https://www.geogebra.org/classic/emjn7pmq
Join our Discord community:
💬 https://discord.gg/cdQhRgw
Connect with me:
🐦 Twitter - https://twitter.com/elliotwaite
📷 Instagram - https://www.instagram.com/elliotwaite
👱 Facebook - https://www.facebook.com/elliotwaite
💼 LinkedIn - https://www.linkedin.com/in/elliotwaite
🎵 Kazukii - Return
→ https://soundcloud.com/ohthatkazuki
→ https://open.spotify.com/artist/5d07MpiIaNmmEMTq79KAga
→ https://www.youtube.com/user/OfficialKazuki
My previous video, "Why We Use the Sigmoid Function in Neural Networks for Binary Classification":
📼 https://youtu.be/WsFasV46KgQ
My other video, "Derivative of Sigmoid and Softmax Explained Visually":
📼 https://youtu.be/gRr2Q97XS2g
GitHub code for visualizing how the logit values that are passed into the softmax function change over time as the model is trained with SGD (stochastic gradient descent) or the Adam optimizer:
💻 https://github.com/elliotwaite/softmax-logit-paths
Desmos 2D graph of softmax for 4 classes:
📈 https://www.desmos.com/calculator/drqqhtb037
GeoGebra 3D graph of softmax for 2 classes (with derivatives and Gaussians):
📈 https://www.geogebra.org/classic/qhdd4afw
GeoGebra 3D graph of softmax for 3 classes (with derivatives):
📈 https://www.geogebra.org/classic/ps9gwjav
GeoGebra 3D graph of softmax with Gaussians for 3 classes:
📈 https://www.geogebra.org/classic/vgwaw7wr
GeoGebra 3D graph of the shape of the softmax input space for 4 and 5 classes:
📈 https://www.geogebra.org/classic/emjn7pmq
Join our Discord community:
💬 https://discord.gg/cdQhRgw
Connect with me:
🐦 Twitter - https://twitter.com/elliotwaite
📷 Instagram - https://www.instagram.com/elliotwaite
👱 Facebook - https://www.facebook.com/elliotwaite
💼 LinkedIn - https://www.linkedin.com/in/elliotwaite
🎵 Kazukii - Return
→ https://soundcloud.com/ohthatkazuki
→ https://open.spotify.com/artist/5d07MpiIaNmmEMTq79KAga
→ https://www.youtube.com/user/OfficialKazuki
