Nvidia Explains NVFP4 and KV Cache Tiering as Responses to Memory Shortage
Posted As of
On November 5, Nvidia said in an official blog post that memory supply remains tight and prices are rising, and described NVFP4 checkpoints and KV cache tiering as its responses. The figures are Nvidia's own claims.
What happened
This summary is based on Nvidia's official blog post (dated November 5, 2026, written by Amr Elmeleegy). It is an explanatory article, not a product announcement.
- Outlook: As models grow, contexts get longer, and agent apps multiply, the burden on GPU memory is increasing. The post expects memory supply to stay tight and prices to keep rising in the near term. (Official)
- Models that released NVFP4 checkpoints: FLUX.2 [klein] (Black Forest Labs), Inkling (Thinking Machines Lab), Beam 1.0 (Reflection), Muse Spark (Meta), Step 5 Preview (StepFun), Nemotron 3.5 Super VL (Nvidia). NVFP4 takes less space than FP8. (Official)
- KV cache responses: Lowering precision lets more context fit in HBM, Vera CPU LPDDR holds paused contexts, and Dynamo moves inactive contexts to a shared storage tier called CMX and back again. There is also a feature that treats GPU memory, CPU memory and storage as one resource and reuses computed contexts. (Official)
- Generational comparison (Nvidia's claims): Going from Blackwell Ultra to Vera Rubin, HBM capacity per GPU rose about 3%. In a SemiAnalysis AgentX DeepSeek V4 Pro workload (125 tokens per second per user), Vera Rubin NVL72 is said to deliver up to 30 times the throughput per dollar of GB300 NVL72. These are manufacturer figures for a specific workload. (Official, self-claimed)
Not in the post: new consumer GPUs, and effects on personal PCs.
Our take
As memory gets scarcer, 4-bit formats like NVFP4 and KV cache savings are becoming the industry's default response. This suggests quantized models in these formats may reach local AI first, so when choosing hardware, check RAM capacity and supported formats together. However, the cost figures above are Nvidia's claims about data center configurations and do not directly apply to personal devices.
Sources and related posts
- 엔비디아 블로그: More Performance From Every Byte (2026-11-05) Primary source