sparkrun-recipes

Mooglest team recipes for sparkrun, served on NVIDIA DGX Spark clusters.

Recipes

Recipe	Model	Runtime	Nodes	Context	Concurrency
glm-5.3-flash-nvfp4	LibertAIDAI/GLM-5.3-Flash-NVFP4 (320B/18B MoE)	SGLang	2 (TP=2)	64K	2 running
glm-5.3-flash-nvfp4-hiconc	same	SGLang	2 (TP=2)	256K	2 running / 16 queued
glm-5.3-flash-exl3-tr3-4bpw	Mia-AiLab/GLM-5.3-Flash-EXL3-TR3-4bpw (320B/?)	vLLM	2	256K	4 running
deepseek-v4-flash-vision-exp-mooglest	deepseek-ai/DeepSeek-V4-Flash-Vision-Exp	vLLM (B12X / eugr)	2 (TP=2)	1,048,576 (auto @ gmu 0.85)	8 running

deepseek-v4-flash-vision-exp-mooglest is a copy of @eugr/deepseek-v4-flash-vision-exp, converted from the upstream v1 eugr recipe to format v2 (builder: eugr + build_args: [--exp-b12x] + the instanttensor-hybrid-draft-loader mod, resolved from the trusted eugr registry). Keep the eugr registry enabled. Settings match upstream (0.85, CUDA graphs on); the format conversion is the only current divergence.

The -mooglest suffix is deliberate: sparkrun list (which sparkrun-ui renders) is the one-row-per-filename-stem view, and the eugr registry already ships a deepseek-v4-flash-vision-exp.yaml. An identically named copy here would be deduped away and never appear in the UI; the distinct stem keeps this one addressable as @mooglest/….

Why these settings

• SGLang, not vLLM — on DGX Spark (GB10 / sm_121), vLLM produces wrong output for the glm5_next architecture (NoPE MLA kernel asserts pe_dim == 64 while the model is qk_rope_head_dim = 0). SGLang's TileLang DSA path has the NoPE kernel.
• NVFP4, not BF16/FP8 — the full-fat (~598 GiB) and official FP8 (~306 GiB) checkpoints don't fit two GB10 nodes; the community NVFP4 (~181 GiB) does.
• pre_exec TileLang patch — stock TileLang on GB10 requests more dynamic shared memory than the device allows; the pre_exec retunes the sparse-MLA kernel (block_I=64→32, num_stages=2→1, threads=256→128).
• 64K context — the model's native window is 1M, but the 34 KDA linear-attention layers make a per-request state cache the binding constraint on GB10; 64K is the verified serviceable point.
• gpu_memory_utilization: 0.85 + CUDA graphs (DeepSeek V4 only) — this matches upstream. An earlier revision of this recipe ran 0.80 + --enforce-eager after the config crashed on the 2-node cluster twice, for two unrelated reasons: first an earlyoom SIGTERM (host available memory fell under earlyoom's 2% trigger, ~2.5 GB of the 128 GB unified pool), then, at 0.80 with graphs still enabled, an assert active_topk_width >= cm.max_seq_len // compress_ratio failure inside DeepSeek V4 sparse_mla CUDA-graph capture. --enforce-eager disabled TorchInductor and CUDA-graph capture, so the assertion was never evaluated — at the cost of context (--max-model-len auto resolved to 410,880 instead of 1,048,576). That workaround has been reverted in favour of upstream's config, so graphs (and the captured-graph knobs) are live again. If the sparse_mla capture assertion returns, re-try --enforce-eager first — it, not the 0.05 of memory, is what bounds the captured graph size. The earlyoom trigger itself is host configuration that this recipe does not and cannot set — see Host earlyoom below.

Host earlyoom — manual, out of band

earlyoom is host state: a systemd service on each DGX Spark with its own /etc/default/earlyoom. No recipe key controls it, and the recipes in this repo do not apply it. Everything below is operator guidance, deliberately kept out of the recipe YAML.

sparkrun setup earlyoom (wizard phase 6) is the only sparkrun-managed path, and it is not parameterizable: it writes EARLYOOM_ARGS="-m 2 -s 80 --prefer … --avoid …", with -m (minimum available memory %) and -s (minimum free swap %) hardcoded as literals in its install script. Its CLI exposes only --prefer and --avoid — there is no -m/-s flag, and the legacy eugr run-recipe.sh shim rejects --earlyoom-args outright. So a recipe can only lower what the workload asks for; it can never move earlyoom's line.

Target state

Both hosts should be standardized to -m 1 -s 80 with the stock sparkrun --prefer / --avoid regexes. This is a target, not a description of the current hosts — as of 2026-09-16 no earlyoom change has been applied anywhere, and both hosts still run the values in the drift table below:

EARLYOOM_ARGS="-m 1 -s 80 --prefer (vllm|VLLM|sglang|llama-server|llama-cli|trtllm|tritonserver|ray|python3|python) --avoid (systemd|sshd|dockerd|containerd|dbus-daemon|NetworkManager)"

Current drift (live argv, verified 2026-09-16)

Live process command lines, read from /proc/<earlyoom>/cmdline on each host:

gladstone  /usr/bin/earlyoom -m 2 -s 80 --prefer (vllm|VLLM|…) --avoid (systemd|…)
sadstone   /usr/bin/earlyoom -r 3600

Host	Threshold args	Resulting SIGTERM trigger
gladstone	-m 2 -s 80 (sparkrun-managed)	avail ≤ 2% and swap free ≤ 80%
sadstone	none — inherits earlyoom's built-in -m 10 -s 10	avail ≤ 10% and swap free ≤ 10%

sadstone is not merely "different" — it is more aggressive than gladstone. earlyoom's built-in -m default is 10%, so sadstone starts SIGTERMing around 10% available (~12.8 GB on a 128 GB unified pool) instead of 2% (~2.5 GB). Note -r 3600 is only earlyoom's report interval (upstream ships that value in its own /etc/default/earlyoom) and says nothing about thresholds — which is why it reads as "configured" while behaving as stock.

Procedure (per host, not from a recipe)

sudoedit /etc/default/earlyoom     # set EARLYOOM_ARGS to the target above
sudo systemctl restart earlyoom
systemctl is-active earlyoom
cat "/proc/$(pgrep -o earlyoom)/cmdline" | tr '\0' ' '; echo

Confirm the change from the resulting argv with that last command rather than trusting the file.

Gotchas

• earlyoom needs both thresholds breached. It sends SIGTERM only when available memory and free swap are both below their minima, so with -m 1 -s 80 the memory figure is the binding constraint and the swap figure is a loose backstop. (SIGKILL then fires at each KILL_PERCENT, which defaults to PERCENT/2 — -m 1 → 0.5% available.)
• The --prefer / --avoid patterns are active. gladstone's live argv above shows them intact and unquoted, so inference processes really are killed first and sshd / dockerd / systemd really are protected. Recording this because upstream's /etc/default/earlyoom warns that quotes in EARLYOOM_ARGS can reach the process literally and make the regex match nothing — that does not happen here, so sparkrun's quoted form (--prefer '$PREFER') is fine and there is nothing to fix.
• A manual edit is not durable. sparkrun setup earlyoom rewrites /etc/default/earlyoom wholesale, so any later wizard or setup run reverts the host to -m 2 -s 80. Re-check after cluster re-setup. sparkrun setup check reports only installed/active, never the threshold.
• This is the crash to remember. The first failure in the DeepSeek V4 history above was an earlyoom SIGTERM, not a CUDA fault — the workload was killed at the 2% line, and nothing in a recipe bounds or prevents that.

Register

# ~/.config/sparkrun/registries.yaml
registries:
  - name: mooglest
    url: https://git.mooglest.com/mooglest/sparkrun-recipes.git
    subpath: recipes
    trusted: true

If the repo is private, sparkrun needs credentials to clone it. HTTPS with an embedded token (https://<user>:<token>@git.mooglest.com/...) works, or use an SSH deploy key mounted in the serving pod — note the pod's cluster-owned SSH key is not authorized for Forgejo, so a plain ssh://git@… URL will fail there.

Then:

sparkrun registry update
sparkrun run @mooglest/glm-5.3-flash-nvfp4-hiconc --cluster gander