GitSeonjeap
English

Sign in

Reading needs no sign-in. Sign in to write posts or comments, or to recommend and report, in the community.

By signing in you agree to the notices below. Community rules · Privacy notice

Keep browsing without signing in

← News

Local AI

Ollama v0.40.16 Stable Release Notes Memory Savings on Apple Silicon

Posted As of

Ollama has released the v0.40.16 stable tag. The official release notes state that memory usage for Qwen3.8 Flash Next on Apple silicon has decreased, and they include CUDA 13.5 support for Windows on ARM.

What happened

  • Source: Ollama GitHub release (v0.40.16 stable tag)
  • Official: the release states that memory usage for Qwen3.8 Flash Next on Apple silicon decreased by about 27 GiB
  • /v1/responses now respects the truncation setting and no longer truncates silently at the default value
  • Added CUDA 13.5 support for Windows on ARM
  • Includes a llama.cpp update
  • The release notes do not describe the test environment behind the 27 GiB figure

Our take

For Mac users running Qwen3.8 Flash Next locally on Apple silicon, the memory headroom may differ. However, the actual reduction may vary with model size, quantization, and context length, and this cannot be confirmed from the release notes alone. The real-world effect of CUDA 13.5 on Windows on ARM devices is also not yet known.

Sources and related posts

More in Local AI

  1. Ollama v0.40.16-rc0: pplx-decider added, API behavior fixed (github.com)

    Ollama pre-release v0.40.16-rc0 adds Perplexity's 27B decider model, and changes how the OpenAI-compatible endpoint handles default sampler values. The stable release may differ from these notes.

    Local AI| |1 sources |Our take included
  2. llama.cpp b12200, Qualcomm OpenCL quantization kernel improvements (github.com)

    The llama.cpp b12200 release contains one change that reworks OpenCL q8_0 and small-batch MoE processing. No performance figures were published.

    Local AI| |1 sources |Our take included
  3. Moonshot Releases Open-Weight Model Kimi K3.1 Pro (huggingface.co)

    Moonshot AI has uploaded the weights of Kimi K3.1 Pro, a multimodal agent model, to Hugging Face. The parameter count is listed as 2.8T (official) and the model is under a dedicated license.

    Local AI| |1 sources |Our take included

All news