diff options
| author | Henry J. Webster <hwebs@hwebs.info> | 2026-08-03 21:38:08 -0500 |
|---|---|---|
| committer | Henry J. Webster <hwebs@hwebs.info> | 2026-08-03 21:39:51 -0500 |
| commit | 71e36370770acc7498748a890cd0365b3d351027 (patch) | |
| tree | 60c4180135368f928bdc651f7b5d769886c56441 /machines | |
| parent | e9b9671d238bb3de779c0dbf4a518b41280be778 (diff) | |
kusanagi/ollama: fix ROCm GPU fallback to CPU for Gemma
The RDNA3 card reports as gfx1101/gfx1102/gfx11-generic, which the ROCm
build bundled with ollama-rocm has no kernels for, so ollama found no
usable device and ran 100% on CPU after the nixpkgs bump. Force it to
present as gfx1100 via HSA_OVERRIDE_GFX_VERSION.
Also set OLLAMA_FLASH_ATTENTION=1 so the existing q8_0 KV cache actually
engages instead of silently falling back to f16 and inflating VRAM use.
Assisted-by: Claude-code:Opus-4.8
Diffstat (limited to 'machines')
| -rw-r--r-- | machines/kusanagi/default.nix | 8 |
1 files changed, 8 insertions, 0 deletions
diff --git a/machines/kusanagi/default.nix b/machines/kusanagi/default.nix index 5e6dc1e..a22843d 100644 --- a/machines/kusanagi/default.nix +++ b/machines/kusanagi/default.nix @@ -212,6 +212,14 @@ package = pkgs.ollama-rocm; environmentVariables = { HIP_VISIBLE_DEVICES = "0"; + # RDNA3 card reports as gfx1101/gfx1102/gfx11-generic, which the ROCm + # build bundled with ollama-rocm doesn't have kernels for -> it finds no + # usable device and silently runs 100% on CPU. Force it to present as + # gfx1100 (7900 XT/XTX), which is fully supported and binary-compatible. + HSA_OVERRIDE_GFX_VERSION = "11.0.0"; + # KV-cache quantization (below) is a no-op without flash attention; ollama + # falls back to f16 KV cache, inflating VRAM use and forcing CPU offload. + OLLAMA_FLASH_ATTENTION = "1"; OLLAMA_CONTEXT_LENGTH = "32768"; OLLAMA_KV_CACHE_TYPE = "q8_0"; }; |