DEV Community

#hyperparameters

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Warmup Did Nothing for My Learning Rate — Then It Fully Rescued the Wrong One
b1ackjack12 profile

Warmup Did Nothing for My Learning Rate — Then It Fully Rescued the Wrong One

#pytorch #machinelearning #deeplearning #hyperparameters
Comments
4 min read
My Gradient Clipping Was Never Clipping. It Was Secretly Multiplying My Weight Decay
b1ackjack12 profile

My Gradient Clipping Was Never Clipping. It Was Secretly Multiplying My Weight Decay

#pytorch #machinelearning #deeplearning #hyperparameters
Comments
4 min read
Weight Decay 0.05: Catastrophe in Adam, Optimum in AdamW
b1ackjack12 profile

Weight Decay 0.05: Catastrophe in Adam, Optimum in AdamW

#pytorch #machinelearning #deeplearning #hyperparameters
Comments
4 min read
EMA's Decay Isn't a Dial. It's a Plateau, a Cliff, and a 10%-Accuracy Crater
b1ackjack12 profile

EMA's Decay Isn't a Dial. It's a Plateau, a Cliff, and a 10%-Accuracy Crater

#pytorch #machinelearning #deeplearning #hyperparameters
Comments
5 min read
QLoRA hyperparameters that actually matter (rank, alpha, learning rate, epochs)
pranjulrathour profile

QLoRA hyperparameters that actually matter (rank, alpha, learning rate, epochs)

#qlora #hyperparameters #finetuning #llm
Comments
3 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.