Skip to content

DEV Community

InferenceKey

Open models on your own GPUs: experiments, measurements and the exact commands behind them. Full write-ups at inferencekey.com

inferencekey.com Joined on Oct 14, 2026

Nov 5

Serving Claude Code from DeepInfer on 4x B200: the cache, the tail and the memory ceiling

#ai #llm #gpu #devops

4 min read

Nov 4

Chat memory as a 41.9 MB file: building on MemT-1.0-Nano's save and reload

#ai #llm #machinelearning #python

5 min read

Nov 3

Serving WeVisOne-VL-4B: two engines, two copies of the weights, one chess piece apart

#ai #computervision #python #gpu

5 min read

Nov 2

Shipping Ming-Image Design: size the card by its files, and check the alpha in code

#ai #machinelearning #python #gpu

4 min read

Nov 1

Serving a 1B math model on vLLM: the memory cap sets how many answers fit, not how fast

#ai #llm #machinelearning #gpu

4 min read

Oct 31

Fitting Flint's coding agent on a 24 GB RTX 4090: context is the knob that decides

#ai #llm #gpu #selfhosted

5 min read

Oct 30

Serving MOSS-Image: step count, memory mode and the traps that change the words you ship

#ai #machinelearning #gpu #python

4 min read

Oct 29

Nemotron 3.5 Bolt on one GPU: what the 24 GB covers, the drafter flag and the llama.cpp trap

#ai #llm #gpu #opensource

5 min read

loading...