Machine Learning Studio
チャンネル登録者数 5990人
1.5万 回視聴 ・ 643いいね ・ 2023/05/23
In this video, I cover 16 of the most popular optimizers used for training neural networks, starting from the basic Gradient Descent (GD), to the most recent ones, such as Adam, AdamW, and Lookahead.
#deeplearning #artificialintelligence
#neuralnetworks #computerscience
~~~~~~~~~~~~~~~
References
Nestrov: proceedings.mlr.press/v28/sutskever13.html
AdaGraD: www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf
AdaDelta: arxiv.org/abs/1212.5701
Adam & AdaMax: arxiv.org/abs/1412.6980
AMSGrad: arxiv.org/abs/1904.03590
AdaBound: arxiv.org/abs/1902.09843
AdamW: arxiv.org/pdf/1711.05101v2.pdf
Yogi: proceedings.neurips.cc/paper_files/paper/2018/file…
Nadam: openreview.net/pdf/OM0jvwB8jIp57ZJjtNEZ.pdf
Lookahead: arxiv.org/abs/1907.08610
コメント
使用したサーバー: directk
コメントを取得中...