DEV Community

#pytorch

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Discriminative Fine-Tuning: Why Your Backbone and Your Head Shouldn't Learn at the Same Speed

Discriminative Fine-Tuning: Why Your Backbone and Your Head Shouldn't Learn at the Same Speed

Comments
6 min read
DeepSpeed ZeRO vs PyTorch FSDP2, Measured: 1.8x on NVLink, Barely Finishing on PCIe (Plus How to Actually Get GPUs on GKE)

DeepSpeed ZeRO vs PyTorch FSDP2, Measured: 1.8x on NVLink, Barely Finishing on PCIe (Plus How to Actually Get GPUs on GKE)

Comments
10 min read
Undefined type Float8_e4m3fn on Apple Silicon: BF16 and GGUF Workarounds for FP8 Models

Undefined type Float8_e4m3fn on Apple Silicon: BF16 and GGUF Workarounds for FP8 Models

Comments 1
4 min read
OpenArch: PyTorch implementations of modern LLM architectures!

OpenArch: PyTorch implementations of modern LLM architectures!

Comments
5 min read
Installing PyTorch with ROCm Acceleration on Ubuntu 24.04

Installing PyTorch with ROCm Acceleration on Ubuntu 24.04

6
Comments
2 min read
SSRF in PyTorch: How a Missing URL Validation in Dataset Loading Could Leak Cloud Credentials

SSRF in PyTorch: How a Missing URL Validation in Dataset Loading Could Leak Cloud Credentials

Comments
2 min read
Neural Networks: Weights, Activation, and Backpropagation

Neural Networks: Weights, Activation, and Backpropagation

Comments
5 min read
PyTorch 2.14: clamp's Boundary Gradient Silently Changes, bfloat16 Complex Promotion Now Raises, cholesky/qr Removed

PyTorch 2.14: clamp's Boundary Gradient Silently Changes, bfloat16 Complex Promotion Now Raises, cholesky/qr Removed

Comments
4 min read
What Your Loss Function Actually Tells the Model: MSE, Cross-Entropy, and the softmax Bug That Trains Anyway

What Your Loss Function Actually Tells the Model: MSE, Cross-Entropy, and the softmax Bug That Trains Anyway

Comments
13 min read
What optimizer.step() Actually Does: One Step of SGD, Momentum and Adam by Hand

What optimizer.step() Actually Does: One Step of SGD, Momentum and Adam by Hand

Comments 1
10 min read
Backpropagation by Hand: Two Layers, a Pen, and Then Autograd Agrees

Backpropagation by Hand: Two Layers, a Pen, and Then Autograd Agrees

Comments
9 min read
Model DNA, Analyzed: Verifying 'From-Scratch' LLM Claims with Architecture, Tokenizer, and CKA (PyTorch)

Model DNA, Analyzed: Verifying 'From-Scratch' LLM Claims with Architecture, Tokenizer, and CKA (PyTorch)

Comments
6 min read
ResKV: Recovering Evicted Token Contributions via Residual KV Cache — LongBench 32/32

ResKV: Recovering Evicted Token Contributions via Residual KV Cache — LongBench 32/32

Comments
6 min read
Attention is simpler than you think - A hand-crafted superhero transformer

Attention is simpler than you think - A hand-crafted superhero transformer

Comments 3
9 min read
Your quantized model got worse, and nothing told you

Your quantized model got worse, and nothing told you

3
Comments 2
5 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.