KV cache and GPU memory calculator

Type a model's id from the Hugging Face hub. The page reads the model's config.json in your browser and counts what one token costs in its cache, layer by layer, and how much memory the weights and the cache need together at the context length and number of conversations you choose. The arithmetic is the one written out in lesson 8 of the architectures course.

How it is counted

A Transformer keeps a key and a value for every token in every layer that attends over the whole text. Per token and per such layer that is 2 × key-value heads × head size numbers; models with grouped-query attention share each key-value head among several query heads, which is why their count of key-value heads, not of query heads, is what matters. DeepSeek's multi-head latent attention keeps one compressed vector and one rotary key per token instead, kv_lora_rank + qk_rope_head_dim numbers per layer, from which every head's key and value are rebuilt.

Layers with a sliding window keep only the last window of tokens, so their cost stops growing at the window's length; the page reads which layers slide from layer_types or from the window pattern in the config. Hybrid models such as Granite 4.0 H, Nemotron-H, Jamba and Qwen3-Next have recurrent layers, Mamba or linear attention, whose state has a fixed size whatever the length: they add a few megabytes per conversation and are not counted here. The weights are the number of parameters the hub reports for the checkpoint, times the bytes per number you choose.

Not counted: activations during a forward pass, the framework's own memory, and the fragmentation that paged caches reduce. Leave a margin of ten to twenty percent. Gated repositories, such as Meta's own Llama, can't be read without signing in; their public copies by Unsloth carry the same config.