GitSeonjeap
English

← News

Local AI

vLLM Publishes DeepSeek-V4.1-Flash Performance on Vera Rubin NVL72

Posted As of

The vLLM blog published performance results for serving DeepSeek-V4.1-Flash on a Vera Rubin NVL72 rack. The 58x figure over GB300 NVL72 compares at a specific interactivity point and should not be read as an overall speedup.

What happened

  • Per the official vLLM blog, the post was published on 2026-11-12.
  • The benchmark framework is SemiAnalysis AgentX.
  • (Official) At 387 P90 tokens/s per user, it reported 1.7M tokens/s per GPU, which it says is 58x GB300 NVL72.
  • (Official) At peak, it presented 2.52M tokens/s per GPU and 1,810 tokens/s for a single user.
  • (Third-party analysis) It noted the gap at peak throughput is 2.7x.
  • The post is the third in a GB300 series and explains that it moves the same stack to Vera Rubin NVL72.

Our take

These are datacenter-class rack results that are hard to compare directly with setups individuals run on a Mac or PC. What matters for local users is mainly how inference engine optimization changes alongside hardware generations. Independent verification has not been confirmed, and model precision, batch configuration, and cost are not confirmed in the excerpt.

Sources and related posts

More in Local AI

  1. NVIDIA Releases Nemotron for Science for Scientific Agents (nvidianews.nvidia.com)

    NVIDIA announced a package based on Nemotron 3.5 Super VL. It includes a chemistry dataset and a post-training recipe, and the IR copy states that the model is optimized to run locally on a 128GB DGX Spark.

    Local AI| |1 sources |Our take included
  2. NVIDIA RTX Spark Compact Desktops Begin Shipping (blogs.nvidia.com)

    Compact RTX Spark desktops from ASUS, Dell, and MSI have launched as of the announcement date and are positioned for local AI agents and model fine-tuning. Pricing was not confirmed on NVIDIA's page.

    Local AI| |1 sources |Our take included
  3. llama.cpp b12497 adds tiny-voca support (github.com)

    llama.cpp pre-release b12497 adds tiny-voca support to the multimodal path and improves G2P conversion. The release page has no performance figures or description of the target model.

    Local AI| |1 sources |Our take included

All news