What runs well on two Pascal P100 cards
Which local model is the best all-around choice on this box? A month of benchmarks on accuracy, file editing with test feedback, tool use, speed and tuning. One strand also picks the local planner for the AIOS control-plane prototype. All raw data is in this repository.
Where things stand (25 Oct)
The default question for these tests is now the best all-around model: accuracy, editing, tool use, speed and context together. Only tests that name another goal (like the AIOS planner) are judged differently.
- Most capable:
qwen3.6-27b-abliterated, the incumbent. It has the best reasoning scores (and finishes every thinking question), the best editing (28/50 Aider Polyglot) and the best tool use (66 % tau-bench). On HumanEval it is within noise of the top score, and every significant gap is in its favour. It is slow as a dense 27B: 18.8 tok/s with tensor split, about 40 on planner edits with a borrowed MTP head. - Best balance:
qwen3.6-35b-a3b, the MoE. On editing it is statistically level with the incumbent (24 vs 28 of 50, p=0.45) at 3.6× the speed: 68 tok/s, 111 on planner edits with its MTP version. It has never been scored on MMLU-Pro, HumanEval or tau-bench, so that is the gap to close before it can be called the best all-around model. - Fastest:
gpt-oss-20bat 100 tok/s. It came last on editing (4/25) and on tool use (45 % tau-bench), and does not repeat itself reliably. - Production :8080:
devstralis the weakest coder on the shelf (4/25 Aider, 84.8 HumanEval). Tensor split would take it from 12.9 to 22.3 tok/s. - Qwen3.8 vs 3.6 (5 Oct): with the anti-loop thinking budget, Flash-Next and the Qwen3.8-27B UD-Q6_K tie the incumbent on thinking-on MMLU-Pro (87.1 / 88.6 vs 87.1, all p ≥ 0.11). On editing the incumbent leads both 3.8s that were run: Flash-Next 21/50 (p=0.09) and the 27B Q8 17/50 (p=0.01). Every Polyglot run so far had thinking off.
- Context: the UD-Q6_K runs at 126k with nothing on CPU, at 28 tok/s. The Q8_0 needed 8 layers' FFN on CPU to reach 64k, at 16.5 tok/s. Flash-Next at 256k froze the host (RAM exhausted with
--fit), so 128k is its ceiling. - Flash-Next on Strata (12–25 Oct): the GSQ-RCO IQ3_S on Strata, an expert-offload engine (the most-used experts in VRAM, all of them in RAM, the CPU computes the rest), reads a fresh 128k prompt at 402 tok/s and decodes at 35.7 tok/s, with 4.3 GiB of RAM to spare. A fixed layer split of 36 with per-card weight trimming put 10,653 experts on the two cards (8,789 before) and gave +10 % prompt and +3 % decode over the first run. Accuracy at IQ3_S is not scored. Details
- Image and audio (8 Oct, ComfyUI):
FLUX.2 klein 4Bfor images (23 s at 1024², twice as fast as anything else, top CLIP score),Stable Audio 3 Mediumfor sound effects (2 s per 10 s clip), Chatterbox or VibeVoice for speech. ComfyUI needs a cu126 PyTorch on these cards. Details - Still to measure: accuracy for the MoE and K2 Horizon; Aider Polyglot with thinking on; MTP and tensor split on the full 40-task planner bench; pi's tool-call loops.
Model scoreboard
Every model that loads on the CUDA build. Speed is decode tok/s with tensor split (llama-bench tg128, 2 Oct sweep; hover for layer split). Reasoning is 70 MMLU-Pro questions (5 per category × 14). Coding is all 164 HumanEval problems. Editing is Aider Polyglot pass@2 as a percentage (of 50 exercises for the top three, of 25 for the rest). Seed 42, temperature 0. Click a column header to sort.
Result (21 Sep, still standing): keep the incumbent qwen3.6-27b-abliterated. It has the best reasoning scores and is the only model that finishes every thinking question. Its HumanEval is within noise of the top score (96.3 vs 97.6), and every statistically significant gap (McNemar exact, p<0.05) is in its favour. Its own Q8_K_P and a stock Q8_0 gave no resolvable gain and cost most of the context headroom, so both were deleted (30 Sep).
Update (2 Oct): HumanEval is saturated (92–97.6 %) and does not predict editing. qwen3.8-27b Q8 tops HumanEval but drops to third on Aider Polyglot. See Coding.
Update (5 Oct): with a 6,144-token thinking budget no model runs away on thinking questions any more. Flash-Next (87.1) and the Qwen3.8-27B UD-Q6_K (88.6) both tie the incumbent (87.1): every gap is p ≥ 0.11, so neither 3.8 shows a gain over 3.6 on reasoning. On editing the incumbent still leads.
| Model | Speed tok/s | Reasoning, quick | Reasoning, thinking | Coding (HumanEval) | Editing (Aider) |
|---|
* = the model spent its whole 12,288-token thinking budget on some questions and never answered them. The true score is higher, so a starred value is a floor. † = run on 5 Oct through the :8080 preset with the anti-loop fix (6,144-token thinking budget, medium effort), at temperature 0; every question was answered. Without the fix Flash-Next scored 78.6* (59 of 70 answered). The UD-Q6_K's quick and HumanEval scores are from its 21 Sep run (Ollama); it has no llama-bench speed (28 tok/s with MTP through the router at 40k and 128k alike, against 33 for the Q8_0 on the same prompts). "none" = no thinking mode; "not run" = never tested. gpt-oss-20b is non-deterministic: it scored 77.1 once and 71.4 in four of five repeats. Flash-Next's speed is from 48 distinct prompts on a server, with part of the model in RAM. Three shelf models do not load on llama.cpp: qwen3.6-27b-opus-distill and qwen3.6-35b-a3b-abliterated-vl (vision GGUFs) and gemma4-e4b-abliterated. Sources: cuda-matrix, aider-polyglot, speed-sweep, flash-next-full, think-fix-full.
Speed against coding accuracy
Coding with test feedback: Aider Polyglot
Exercism exercises in six languages (C++, Go, Java, JavaScript, Python, Rust). The model edits the stub file to the spec and the real unit tests run. If they fail, the test output goes back for a second try. This uses aider's own harness, called unmodified, in its container. Thinking is off, temperature is 0, and replies use the whole-file edit format. Every model works through the same fixed, language-stratified order, so every comparison is paired.
Result (2 Oct): the incumbent qwen3.6-27b-abliterated is the best coder. The MoE qwen3.6-35b-a3b is statistically level with it (10/6 discordant, p=0.45) at 3.6× the speed. qwen3.8-27b Q8, which has the best HumanEval on the box, is significantly worse than the incumbent (13/2, p=0.01). devstral, the production coding agent, ties gpt-oss-20b for last and lost 12–0 to the incumbent.
Top 4, 50 exercises
All five, first 25 exercises
Paired comparisons (exact McNemar on pass@2)
| Pair | Exercises | Only the first passed / only the second | p |
|---|---|---|---|
qwen3.6-27b-abliterated vs devstral-patched | 25 | 12 / 0 | <0.001 |
qwen3.6-27b-abliterated vs gpt-oss-20b | 25 | 14 / 2 | 0.004 |
qwen3.6-35b-a3b vs devstral-patched | 25 | 10 / 1 | 0.01 |
qwen3.6-27b-abliterated vs qwen3.8-27b-unsloth | 50 | 13 / 2 | 0.01 |
qwen3.6-35b-a3b vs qwen3.8-27b-unsloth | 50 | 9 / 2 | 0.07 |
qwen3.6-27b-abliterated vs qwen3.8-flash-next | 50 | 10 / 3 | 0.09 |
qwen3.6-35b-a3b vs qwen3.8-flash-next | 50 | 8 / 5 | 0.58 |
qwen3.6-27b-abliterated vs qwen3.6-35b-a3b | 50 | 10 / 6 | 0.45 |
devstral-patched vs gpt-oss-20b | 25 | 3 / 3 | 1.00 |
Every model at least doubled from pass@1 to pass@2, so the feedback try is where most of the points are, and HumanEval never measures it. Devstral's misses are genuine wrong code: zero malformed replies, and the tests ran and failed on logic. The test runner was validated in both directions before any model ran: 96/100 reference solutions pass and 99/100 unsolved stubs fail. Caveats: one run per model, and the same exercise flipped between identical runs even at temperature 0. Thinking was off throughout. These are 50 of the 225 exercises. Flash-Next ran the same 50 on 2–3 Oct with --fit on (experts in RAM). Sources: aider-polyglot-10h/REPORT.md, flash-next-full.
Tool use: tau-bench retail
A multi-turn tool-calling agent loop against the retail domain. There were 0 parse failures across 375 tasks and ~4,000 model calls on four architectures, so tool-call emission is solid and every failure was reasoning or policy.
The customer is scripted, so a model that asks a question never gets an answer and scores zero. These scores are not comparable to published tau-bench numbers. g9v3 is a partial run (30 of 115 tasks). The MoE qwen3.6-35b-a3b has not been run. Full report
Speed and tuning
GP100 has no tensor cores and no dp4a, CUDA 13 dropped it, and Ollama's Vulkan path hides a lot of performance. Most of the speed here came from configuration, not from model choice.
Recipe (2 Oct): CUDA 12.6 sm_60 llama.cpp, -sm tensor -fa on with an f16 KV cache, and NCCL_P2P_DISABLE=1 in the environment (without it tensor split hangs on the first token). Add the model's MTP head where one exists (--spec-type draft-mtp, n-max 3 on MoE models, 4 on dense ones). Do not use n-gram drafting on MoE models.
Split mode: layer vs tensor, every model (2 Oct sweep)
llama-bench tg128, f16 KV, flash attention on, 3 reps, one session. Gains: +24–30 % on the MoE models, +50–60 % on gpt-oss and K2 Horizon, +66–73 % on every dense 24–27B. Prefill gains 24–124 % too (devstral 175 → 391), and VRAM is unchanged. The driver reports P2P as OK, but GPU1 sits in a Gen3 x4 slot behind the Ryzen root complex, so NCCL has to go through host memory. g9v3 needs its own build. Sources: speed-sweep, p100-optimize.
MTP speculative decoding, every model with a head (planner edits)
Decode tok/s on 5 planner-bench tasks, temperature 0, grammar on, all answers correct. About 99 % of drafts are accepted here because a plan is mostly a copy of the input file; on general prompts round 2 saw 77 %, so expect less in everyday use. The planner twin and the stock Qwen3.6-27B are unsloth's MTP-preserving GGUFs. The incumbent has no head of its own; Qwen3.8's MTP file works on it with 95 % acceptance. A built-in head needs less VRAM than the separate file, which carries its own copy of the embeddings. Baselines for the unsloth, HauhauCS, stock-Q4 and incumbent rows come from the llama-bench sweep. p100-optimize §3–4
Backend: CUDA sm_60 vs Ollama/Vulkan
Generation tok/s at ctx 8192, layer split (19 Sep). The backend also inverts the ranking: devstral is slowest on Vulkan and fastest on CUDA. MMLU-Pro moved by one question in total, so accuracy does not depend on the backend. cuda-rerun
KV cache: f16 vs q8_0 (devstral Q8_0)
f16 reads KV at 546 GB/s (91 % of the ceiling). q8_0 reads it at 104 GB/s, because any q8_0 in the cache puts flash attention on its slow path. Production :8080 still runs q8_0 @ 49k for the context room; tensor split would need f16. bandwidth report
Findings
| Finding | Measurement | Source |
|---|---|---|
| Hardware and runtime | ||
| HBM2 ceiling | 599 GB/s read, 520 copy, per card (82 % of nameplate). Devstral generation streams at 309 GB/s, 52 % of that. | bandwidth |
| Multi-GPU split | tensor is the fastest mode for every model once NCCL_P2P_DISABLE=1 is set. Without it, it hangs with both cards at 100 % and 37 W; this was the 18 Sep "deadlock". layer is the default and works. row fails to load. --parallel 2 gives 23.4 tok/s aggregate at −1.2 % per stream (devstral, layer), so decode is weight-streaming-bound. | p100-optimize, bandwidth |
| KV cache type | q8_0 KV costs devstral 17 % at 28k depth (9.75 vs 11.68 tok/s) and the MoE planner 12 % at 16k (45.8 vs 51.9), with no prefill difference. Tensor split requires f16 anyway. | bandwidth, p100-optimize |
| Clocks | Under load both cards boost to their 1328 MHz maximum on their own, so raising application clocks gains nothing. | p100-optimize |
| Silent partial offload | llama.cpp defaults to --fit on with a 1024 MiB margin per card. A file that nearly fills 32 GB gets a few layers quietly left in RAM (−15 % speed, so a speed floor misses it). Check VRAM against non-embedding weights + 1082 MiB + 72 KB/token. Force with --fit off -ngl 999 and split by hand (-ts 36,28): the last card also holds the output tensor. | quant test |
| Vulkan tuning (before CUDA) | Disabling the integer-dot-product path made prompt processing 14–23× faster, and num_batch 256 added 9 % to prefill. Superseded by the CUDA build. | qwen3.8 tuning, devstral tuning |
| Speculative decoding | ||
| MTP heads | Every model with a head speeds up: the MoE planner 66 → 111 tok/s (n-max 3; n-max 4 falls back to 84), Ornith 65 → 100, the dense 27Bs 17–20 → 40–49 (n-max 4). Outputs match the non-MTP run byte for byte on 4 of 5 tasks, and the fifth is also correct. Measured on planner edits only. | p100-optimize |
| DFlash | Qwen3.8-27B Q8_0 + DFlash: 22.72 tok/s, 2.09× layer split, 76–79 % accepted on general prompts (MTP was 1.79× there). On planner edits, Q4_K_M + DFlash reaches 28.8. It crashes under tensor split (ggml-backend-meta assert). | round 2, p100-optimize |
| Draft-free n-gram | A net loss in every test: ngram-cache accepts 11 % on general prompts. On the MoE planner, ngram-simple/ngram-mod give 35–43 tok/s against 66 without, even at 55 % acceptance. | round 2, p100-optimize |
| Models and quants | ||
| Quant choice on GP100 | No dp4a, so K-quants are slower per byte than Q8_0: Q4_K_M + MTP is slower than Q8_0 + MTP. MXFP4 (gpt-oss) works on sm_60. | round 1, p100-optimize |
| Higher quant of the incumbent | Q5_K_P → Q8_K_P (Q8_0 + F16) or a stock Q8_0: no resolvable accuracy gain, ~4 % slower, and the Q8_K_P tops out at ctx ~16k against the Q5's ~180k. Qwen3.8's Q4 → Q8 coding gain (p=0.016) does not carry over from a Q5 start. | quant test |
| Context by spilling to CPU | Qwen3.8-27B Q8_0 (29 GB): each FFN block moved to CPU frees ~135 MiB per card and costs ~4–5 ms per pass. 64k took 8 blocks: 32.9 → 16.5 tok/s, prefill 188 → 95. 128k took 24 blocks: 8 tok/s. With mmap the CPU tensors made prefill CPU-bound (52 tok/s); load-mode = none roughly doubled it. A smaller quant is the better route: the UD-Q6_K fits 126k with nothing on CPU at 28 tok/s. | spill, Q6_K |
| q8_0 KV under tensor split | Qwen3.8-27B at 80k with a q8_0 cache ran out of VRAM on the first prompt (cuBLAS pool alloc), although its cache is the size of the 40k f16 one. Halving the KV is not a free 2× context here. | spill |
| Flash-Next at 256k | Loading the 75 GB IQ2_XXS at ctx 262144 with --fit froze the host, screen on, with no kernel log: --fit pushes more experts into RAM to make room for the cache, on 61 GB of RAM with zram swap. 131072 is the working ceiling. Probes since then run under a host-RAM watchdog. | max-context |
| Router presets | Command-line --ctx-size, --cache-type-*, --split-mode and --ubatch-size on the router override every per-model preset, so they live in [*] of models.ini. Every preset keeps ≥0.6 GB free per card at a full fill: devstral loaded at 64k and then OOMed in warmup. | pi-context |
| Context cost | The Qwen 27B models measured 72 KB/token of KV, not the 260 KB in earlier notes. Devstral measured 168 KB/token f16 (about 84 KB at q8_0, derived rather than measured), with room for ~100k tokens. | cuda-matrix |
| Models bigger than VRAM | qwen3.8-flash-next (512-expert MoE): the 82 GB UD-IQ3_XXS ran at 5.3 tok/s steady through Ollama (MMLU-Pro 70.0). The 75 GB IQ2_XXS runs at 19.3 tok/s on llama.cpp with --fit putting experts in RAM. The quant and the runtime both changed, so the two causes are not separated. A sparse MoE must get distinct prompts: repeating one gave a bogus 12.53 tok/s on 19 Sep. | flash-next, flash-next IQ2 |
| Flash-Next on Strata | Strata's CUDA 12 engine built for sm_60 runs the GSQ-RCO IQ3_S as a layer split. The cards take turns per verify window, so a 70/30 utilisation split is expected, not idle capacity. GPU1's Gen3 x4 link (3.3 GB/s) makes a missed expert ~6× as costly there, so auto gave it only layers 39–47. STRATA_STAGE_TRIM=1 (each card loads only its own layers' dense weights) plus a fixed split of 36 raised the GPU expert slots 8,789 → 10,653 and the decode hit rate to ~93 %. 128k fresh prompt: 366 → 402 tok/s, decode 34.9 → 35.7. Split 36 started from a learned expert profile, so it is not a pure A/B against 39. | first run, trim, split 36 |
| Benchmarks | ||
| Thinking budget (anti-loop) | Qwen3.8 models could think until the 12,288-token cap and give no answer (Flash-Next: 3 of 14 MMLU batches, 0/5 on history and law). Preset reasoning-budget = 6144 + reasoning-effort = medium + Qwen's thinking-mode sampling: 70/70 answered on both 3.8s, with 1.8–3.2× fewer thinking tokens than their no-fix runs; a batch that reaches the budget is cut off and still answers. The budget does the work on the 27B; medium effort alone fixes most of Flash-Next. | think-fix-full, 27B, Flash-Next |
| Flash-Next + MTP | Works only on a PR build with the head appended to the GGUF: 18.6 → 20.7 tok/s at n-max 3 with p-min 0.75 (92 % accepted). Ungated drafting is slower than none, because a rejected draft costs a pass over experts in RAM. | flash-next-mtp |
| HumanEval vs editing | HumanEval ranked qwen3.8-27b Q8 first (97.6). On Aider Polyglot it came third and was significantly worse than the incumbent (17 vs 28 of 50, p=0.01). A score on single functions from a docstring does not predict editing. | polyglot |
AIOS planner benchmark
A purpose-specific test, not part of the all-around ranking. The AIOS contract is one-shot and grammar-constrained: the planner emits a single write_lab_file with the full replacement content and the fixed check sequence, with a 120 s initial and 240 s maximum deadline. That makes generation throughput × output size the binding constraint on these cards. 40 tasks per configuration.
Result (23 Sep): the dense 27B models are always right but too slow (only files up to ~4.7 KiB fit in 240 s). The MoE qwen3.6-35b-a3b drops a check in over half its plans on its own, but with the check list pinned in the grammar it scores 39/40, all within 240 s, and follows 0/5 injections.
Update (2 Oct): tensor split takes the pinned MoE from 53 to 66 tok/s (the 15.0 KiB edit: 127 → 102 s). Its MTP version reaches 111 tok/s, 5/5 correct on a 5-task subset. Dense qwen3.8-27b q8_0 with tensor split + MTP decodes at ~49 tok/s, so every size up to 15.0 KiB fits in 240 s (151 s for the largest; 9/9 correct). Swapping the planner file changes the SHA-256 the AIOS spec pins, and neither configuration has had the full 40-task run yet. p100-optimize
Size ladder: wall seconds per edit as the file grows
The same two edits at each file size. Green = correct, amber = failed validation, red = wrong. Underlined = over the 240 s ceiling. The two 2 Oct rows ran only the port edit at six sizes; – = not run.
Escaped replacement content tokenizes at about 2.4–2.7 bytes/token. Emitting it raw instead of as a JSON string would save 22–26 % of tokens on the Mistral, gpt-oss and g9v3 tokenizers, but only ~4 % on the Qwen tokenizer, so the grammar kept JSON. Sources: planner-bench/RESULTS.md, tokenize-escaping.
Full-stack rig (AIOS)
The rig runs multi-transaction scenarios through the whole flow: planner, strict validation, optional reviewer and reporter, a simulated human, checks, commit. Mistakes carry forward. Every configuration runs under an oracle human and a rubber-stamp human. Under the rubber stamp, committed wrong is the number that matters.
Result (26–27 Sep): use one model and no reviewer. The careful-hash planner persona went 54/54 with 0 wrong commits. Both model reviewers were net negative: they blocked 4 and 8 correct plans, and injected text steered them. Do the stale-hash check deterministically in the controller instead.
Fix check (27 Sep): with reporter from/to typed as any JSON value, the 27B's reports went from 5/13 to 13/13 complete and exact. The new tool-loop-careful persona followed 0/6 injections. Sources: first-stack, fixes-check, rig source.
Image and audio generation: ComfyUI
One night (7–8 Oct) of text-to-image and text-to-audio models in ComfyUI 0.35, one server per card. Every model got the same prompts in the same order with fixed seeds. Images are 1024×1024 from GenEval prompts, scored by CLIP; sound effects use AudioCaps captions and music uses 20 prompts, both scored by CLAP; speech is scored by a Whisper round trip.
- Images:
FLUX.2 klein 4B. At 23 s an image it is twice as fast as anything else, it has the top CLIP score, and it gets counts and positions right in the grid below. Z-Image Turbo is the runner-up on looks at 69 s. FLUX.1 dev (229 s) is not worth it on Pascal. - Sound effects:
Stable Audio 3 Medium: 2 s for a 10 s clip and CLAP 0.191, against 11 s and 0.104 for Stable Audio Open 1.0. - Speech: Chatterbox, Chatterbox Turbo and VibeVoice 1.5B all come back through Whisper at under 1 % word error rate, faster than real time. VibeVoice does two-speaker dialogue but needs a reference voice.
- Music: ACE-Step 1.5 turbo only works in fp32 with
--gpu-onlyand a fresh server per song; 3 of 20 vocal songs still ran out of memory.
| Image model | Setup | s / image | CLIP, 80 shared prompts |
|---|---|---|---|
| FLUX.2 klein 4B | bf16, 4 steps, fp16 compute | 23.0 | 0.338 |
| RealVisXL V5 (SDXL) | 25 steps, cfg 6.8, fp16 | 48.2 | 0.336 |
| Z-Image Turbo 6B | GGUF Q8_0, 8 steps, fp16 | 68.6 | 0.334 |
| Qwen-Image-2512 20B | GGUF Q4_K_M + Lightning 4-step LoRA, fp32 | 76.2 | 0.331 |
| FLUX.1 dev 12B | GGUF Q8_0, 20 steps, fp16 | 228.9 | 0.329 |
| FLUX.1 schnell 12B | GGUF Q8_0, 4 steps, fp16 | 48.3 | 0.326 |
The CLIP spread is 0.012, too small to rank by; it mostly shows the objects are there. Judge quality from the grid.
Sound effects and music
| Model | s / clip | RTF | CLAP |
|---|---|---|---|
| Stable Audio 3 Medium 100 × 10 s | 2.0 | 0.20 | 0.191 |
| Stable Audio Open 1.0 100 × 10 s | 11.3 | 1.12 | 0.104 |
| ACE-Step 1.5 turbo 17 × 60 s | 52.3 | 0.87 | 0.265 |
RTF is generation seconds per second of audio. ACE-Step's time includes reloading its 10 GB checkpoint for every song, and its CLAP is against style tags, so it does not compare with the AudioCaps rows.
Speech (40 sentences)
| Model | s / clip | RTF | WER |
|---|---|---|---|
| Chatterbox | 2.0 | 0.57 | 0.000 |
| Chatterbox Turbo | 1.5 | 0.40 | 0.005 |
| VibeVoice 1.5B +5 dialogues | 3.8 | 0.86 | 0.006 |
Word error rate from Whisper large-v3-turbo transcribing the output; VibeVoice's five two-speaker dialogues scored 0.000.
What Pascal changes in ComfyUI
| Finding | Detail |
|---|---|
| PyTorch build | The CUDA 13 wheels dropped sm_60, so ComfyUI needs a cu126 build (2.14.1+cu126 here) in its own venv. |
| fp16 is off by default | On Linux ComfyUI runs Pascal in fp32 and keeps text encoders on the CPU. With --force-fp16, FLUX.1 dev went from 316 to 228 s an image (−28 %), Z-Image from 65 to 69 s (+5 %), with the same CLIP score; Qwen-Image produced blank images and ran in fp32. |
| ACE-Step 1.5 | Its lyrics / audio-code model must run on CUDA, so it fails with the encoder on the CPU. --force-fp16 also makes the audio VAE fp16, which saturates every song. --gpu-only (fp32) works but never offloads, so the server has to restart between songs. |
| VibeVoice node | Refuses to run without at least one reference voice clip. |
Models on the shelf
GGUF weights (2–82 GB each) are too big for a Git repo, so this is the inventory: the files, their sizes, and where each one lives. The fast tier sits on NVMe (13 s cold load for 29 GB). The archive sits on an 8 TB HDD (~160 s). Speeds are decode tok/s, layer → tensor split, from the 2 Oct sweep.
Every test in this repository
P100 runs from 15 September onward, oldest first. Each card links to the write-up and the raw folder (logs, per-run JSON, drivers).
Reusable code
Early runs (1–6 September, before the P100s)
These ran through Ollama before the P100 runs began on 15 September. The 1 and 2 September reports record a GTX 1660 SUPER. The 4–6 September runs don't name the GPU, but their speeds match the 1660 figures (27B ≈ 2.6 tok/s). Speed columns are not comparable with the P100 numbers above. Accuracy is still valid, because accuracy was later shown not to depend on the backend.
Image generation: RealVisXL (2 September)
The best- and worst-scoring image by CLIP alignment for each GenEval and T2I-CompBench category. This is a sample; the full run was 2,953 images, and the repo keeps scores and manifests rather than every PNG.