Recommended References
No video available for this topic. Explore these curated study references:
1 / 3
article
medium.com
Comparative Analysis of Adam Optimizer and Its Variants: Adamax, RMSProp, and Adagrad — Addressing Vanishing and Exploding Gradient Problems | by Astitwa | Medium
Explore this reference material for in-depth technical documentation and background theory on AdamW (Adam with weight decay) formulation.
Source: medium.comRead Reference
AdamW (Adam with weight decay) formulation
Learning Objectives
- •Decoupled weight decay derivation
- •Effect on regularization
Weekly Outcome
By the end of this module you will be able to evaluate and select Adam variants for a given deep learning task.
Resources
Comparative Analysis of Adam Optimizer and Its Variants: Adamax, RMSProp, and Adagrad — Addressing Vanishing and Exploding Gradient Problems | by Astitwa | MediumImproving the Adam optimizer via projection-based gradient correction in deep learning - ScienceDirectPublished as a conference paper at ICLR 2018