Total vs Active Parameters: How Much GPU Memory Does an LLM Need?

By Sandeep Narahari, Contributor. Last updated: September 2026.

Total parameters set how much GPU memory you need. Active parameters set how much compute each token costs. Kimi K3 activates 104B parameters per token and still needs 64 or more accelerators, because sparsity governs compute, not footprint.

GPU Compute vs Memory Bandwidth: The Two Limits of LLM Inference covered which GPU resource runs out first. This part covers which model number tells you how much of that resource you need.

TL;DR

• “7B” means 7 billion parameters. At BF16 each parameter is 2 bytes, so weights alone are about 14 GB. Parameters times bytes-per-parameter is your floor.
• A dense model runs essentially every parameter for every token. An MoE model routes each token to a few experts and leaves the rest idle.
• Mixtral 8x7B is the clean example. The Mistral team’s paper puts it plainly: each token has access to 47B parameters but uses 13B active parameters during inference.
• The same paper states the rule directly: memory cost is proportional to the total (sparse) parameter count, while active parameters are proportional to compute cost.
• Kimi K3 shows it at scale. It is 2.8T total with 104B active, and Moonshot recommends 64 or more accelerators to serve it.

What does 7B, 70B, or 400B actually mean?

The number is parameters, in billions: the learned weights inside the model. They have to sit in memory before the model produces a token, so the floor is parameters multiplied by bytes per parameter, which depends on precision:

GPU memory floor (GB) = total parameters (B) × bytes per parameter 2 bytes at BF16/FP16, 1 at FP8, 0.5 at MXFP4/INT4.

The takeaway: precision moves the memory bill as much as model size does. These are weights only. KV cache and activations sit on top, which is why real deployments need headroom. See H100 vs H200 for Long-Context LLMs for how fast KV cache grows.

Real checkpoints rarely match the naive arithmetic, because models mix precisions. OpenAI’s gpt-oss-120b holds 116.8B parameters but ships as a 60.8 GiB checkpoint, since its MoE weights are MXFP4 while attention and embeddings are not.

What is the difference between a dense model and an MoE model?

A dense model uses essentially its full parameter set for every token. A mixture-of-experts (MoE) model splits its feed-forward layers into parallel copies called experts, and a small router picks a few per token.

Llama 3.1 405B is dense: all 405B parameters are active for every token. Mixtral 8x7B is MoE: each layer holds eight expert feed-forward blocks and a router selects two per token. Attention layers are shared rather than duplicated, which is why eight copies of a 7B model total 47B and not 56B.

The Mixtral paper states the design goal cleanly: raising the expert count while holding the number selected per token fixed grows parameter count while keeping computational cost effectively constant. Capacity grows, arithmetic does not.

What are total vs active parameters?

Total parameters are how much model exists. Active parameters per token are how much of that model runs for a given token. In a dense model the two are identical. In an MoE model the active count is a fraction of the total.

Mixtral 8x7B is the canonical example: 47B parameters available to each token, 13B active during inference. It matched or beat Llama 2 70B across most benchmarks while using roughly 5x fewer active parameters. Current models push the ratio much further:

The takeaway: sparsity ratios have gone from under 4:1 in 2023 to nearly 30:1 in 2026. Sources: Mixtral paper, gpt-oss-120b model card, vLLM Qwen3.8-Flash-Next recipe, DeepSeek-V3 repository, Kimi K3 model card. Figures as of September 2026.

Two counts need care. DeepSeek-V3’s main model is 671B, but the published checkpoint is 685B because it includes a 14B multi-token-prediction module. Qwen3.8-Flash-Next is quoted as 125B, 176B, or 180B depending on whether its 51B N-gram table and 4B MTP head are counted. Read the model card before sizing anything.

Do you size GPUs by total or active parameters?

Size by total parameters. The Mixtral paper says so directly: the analysis of active parameters reflects inference compute cost and does not account for memory cost, and memory cost for serving Mixtral is proportional to its 47B total. Every expert must be resident, because the router can select any combination on any token.

The takeaway: MoE cuts your compute bill, not your memory bill. Teams that plan capacity from the active number find the model will not load at all.

One honest caveat on speed. GPU Compute vs Memory Bandwidth showed token generation is usually limited by memory bandwidth rather than math, so a lower active count does not translate one-for-one into faster tokens. The Mixtral authors flag two specific costs: the routing mechanism itself adds overhead, and running more than one expert per device increases memory loads. They conclude MoE suits batched workloads where arithmetic intensity is high, which is the same threshold that piece described.

Why does a sparse model still need so much VRAM?

Because sparsity governs which weights compute, not which weights exist. Kimi K3 is the clearest demonstration to date.

Moonshot AI’s K3 routes each token to 16 of 896 experts, about 104B active out of 2.8 trillion, using MXFP4 weights from quantization-aware training. Per token it does roughly the arithmetic of a 104B model, and Moonshot still recommends supernode configurations with 64 or more accelerators.

So K3 prices per token like a mid-size model and deploys like the largest open model shipped. Nothing about the 27:1 ratio changes the footprint. Documented deployments across the range, from each model’s own maintainers:

The takeaway: a 117B model fits on one GPU and a 125B model needs eight, because total footprint and precision decide, not the headline parameter count.

How many GPUs do you need to run a 70B or 405B model?

At BF16, a 70B model’s ~140GB floor fits on two 80GB GPUs (H100/A100) or one 141GB H200; a 405B model’s ~810GB floor needs at least six H200s or ten 80GB GPUs. That is a bare-weights minimum with zero room for KV cache, activations, or the parallelism scheme the model actually ships with, so treat it as a starting point, not an order.

The formula above gives a floor, not an answer. Qwen3.8-Flash-Next makes the gap concrete.

Its FP8 checkpoint is 172.78 GiB, which by arithmetic alone would fit inside two H200s. The vLLM recipe instead documents TEP8 across eight H200s, and notes plain TP8 is incompatible with the checkpoint’s 128-wide quantization blocks. Parallelism strategy and kernel constraints decide the configuration, not just capacity.

That model also bends the “everything must be in VRAM” rule. Its 51B N-gram embedding table can be asynchronously offloaded to host memory, needing at least 51 GB of system RAM instead of GPU memory. Lookup-style parameters with little per-token compute are the exception: the weights doing matrix multiplications still have to be on the GPU.

The practical sequence: compute the floor from total parameters and precision, add headroom for KV cache and activations, then check the maintainer’s validated recipe before ordering anything.

Sizing to total parameters means you are usually buying memory rather than FLOPs, and the two are not priced proportionally. GPU rental rates vary widely by provider, so an eight-GPU H200 configuration is a rate you compare, not a fixed number. Our H100 rental price roundup found rates spanning roughly $2 to $12 per GPU-hour as of August 2026, a 6x spread on identical silicon.

FAQ