turintech

/

Kev-9B-Artemis-GGUF

Like 11

Follow

TurinTech AI 43

curl http://localhost:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "My Stripe integration keeps failing. Please help ASAP.",
  "questions": {
    "urgency": {"type": "noul", "instructions": "Does this message express urgency?"}
  }
}'

Keep -ub 4096 -b 4096: the numbers on this card were measured with them, and the batch size moves the probabilities slightly (in an RTX 5090 test on a separate 854-token input, the default file gave 0.730 at the default of 512 and 0.727 at 4096). Inputs longer than the batch size are accepted: with llama.cpp 1e6f04a and b11609, a 4,914-token input was answered at both settings. -c 16384 stops the server from reserving memory for the full training context. The GPU memory on this card was measured with two slots (--parallel 2); with llama.cpp 1e6f04a, the commands here start llama-server's default of four slots, so add --parallel 2 to reproduce the measured setting. An explicit slot count also divides -c between the slots (llama.cpp b11609 and b11617 do), so that gives two slots of 8,192 tokens each, where the default's four slots share all 16,384 and one request can use them all. For several concurrent clients, -np 8 -c 32768 --cache-ram 0 gives eight slots of 4,096 tokens with the host prompt cache off; in an RTX 5090 test with llama.cpp fcac190 this used about 0.6 GiB more GPU memory than the command above as written (without --parallel 2). To run a downloaded file:

hf download turintech/Kev-9B-Artemis-GGUF Kev-9B-Artemis-IQ3_M.gguf --local-dir .
llama-server -m Kev-9B-Artemis-IQ3_M.gguf --port 8080 -c 16384 -ub 4096 -b 4096

Every file here also loads and answers with this command on an Apple M3 Max (36 GB) with llama.cpp's Metal build (b11617).

Ollama (0.40.4 tested, on a CPU, 14 October 2026) runs the package's tags as decision models with --ask, and serves /v1/systemone on its own port with a model field added; the package sets the card's context and batch sizes. Ollama's own batch handling moves the probabilities a little (0.978 against 0.977 from llama-server on this example):

echo "My Stripe integration keeps failing. Please help ASAP." | ollama run turintech/kev-9b-artemis:iq3_m --ask "Does this message express urgency?"

LM Studio (0.4.26 tested, CPU engine 2.64.0, 25 October 2026) loads the Hub entry turintech/kev-9b-artemis as a decision model with the same settings; keep the @iq3_m label when downloading, then send decision requests to /v1/systemone on port 1234 with the model named. The chat window and /v1/chat/completions do not run it as a decision model. Of the buttons under "Use this model", llama.cpp starts the file as a chat model, which is not supported, and LM Studio imports it without the context and batch settings above; the commands here are the tested paths.

lms get turintech/kev-9b-artemis@iq3_m --gguf
lms load turintech/kev-9b-artemis

• Same answer as BF16: the share of decisions where the file picks the same answer as the BF16 original, mistakes included. Higher is closer.
• Mean KL to BF16: KL divergence between the original's and the file's answer probabilities, as the protocol describes. Lower is closer.
• Accuracy: the share of decisions that match the panel's labels. Accuracy change: the file's accuracy minus the original's, in points.
• Confident answers changed: answers the original gave with more than 80% probability that the file changed. Lower is closer.

n/a means the card's tables or our run records do not have that value.

Protocol a100-2026-10

One NVIDIA A100 80 GB, October 2026 (the run behind the first card).

• Reference: ggml-org's BF16 GGUF Kev-9B-BF16.gguf (SHA-256 7097686ceab17858c478644e14bee187b5bec775fe4fd6a14aaf02715a903f0a) of jaredpalmer/kev-9b
• Panel test: typed-decisions test split (selection panel); dataset: LocalLLaMA/typed-decisions; split: test; cases: 400; decisions: 2,000; text SHA-256: 4f294f218ea1da27f3efef936359389c62ea4d3973a41457732990f1d31b647c
• Panel train_split: typed-decisions train split, 200 cases drawn outside the calibration window (seed 20261010); dataset: LocalLLaMA/typed-decisions; split: train; cases: 200; decisions: 1,000
• Panel decision_index: Decision Index stratified sample (ANLI, ARC-Challenge, ARC-Easy, BANKING77, CLINC150 with out-of-scope, HellaSwag, MMLU, MMLU-Pro, When2Call, WinoGrande; 30 items each); dataset: ANLI, ARC, BANKING77, CLINC150, HellaSwag, MMLU, MMLU-Pro, When2Call and WinoGrande (300-item stratified sample); cases: 300; decisions: 300; text SHA-256: 9cb4301cc7d7d96dde9663263c53020912ad628201e5609ab2d65bf16ebb652e
• Tool: llama-server /v1/systemone, which returns a probability for every allowed answer, with all of a case's questions in one request. Same answer: the file's most likely answer is the original's. Mean KL: KL divergence from the original's answer probabilities to the file's, averaged over decisions. Accuracy: against the panel's labels, which are synthetic for typed-decisions and each benchmark's own answer for the Decision Index sample. Accuracy change: the file's accuracy minus the original's on the same decisions, in points. Confident answers changed: decisions where the original gave an answer with more than 80% probability and the file chose another. Ranges on the results page are 95% case-bootstrap intervals
• llama.cpp build: b11540 (commit 1e6f04a75e91da4bb66b5248f3fd4bdeff169d41)
• Command: llama-server -m <file.gguf> -c 16384 -ub 4096 -b 4096 --parallel 2 -ngl 99 with our decision suite as the client
• Hardware: one NVIDIA A100-SXM4-80GB per server; the IQ3_M file's checks were split across four A100s
• Settings: 16,384-token context, batch 4,096, two slots, all layers on the GPU
• Measured by: TurinTech measured every file in these tables, including ggml-org's Q8_0 and Q4_K_M, with the same build and settings. Our records of this run are the release tables: the per-decision files, the exact dates (the first card dates the run 12 October 2026), the KL to the original and our files' revision and SHA-256 are not in them; our files are the ones of these sizes, published with the same bytes. The IQ3_M file was measured as a candidate after the others, with its memory taken back to back with the IQ4_XS file on one A100
• Selection: candidate recipes were ranked on the first 100 test cases and then measured on all 2,000 test decisions, so the test decisions were used to choose the published files and are not an independent test. The first card's record says the train-split decisions were not used to choose the files, except that the default IQ3_M file was chosen partly on them, and that their independence from calibration and from the choice of files has not been verified for every file; 170 of them were inside every file's calibration text. The Decision Index sample was measured on the candidates and the records do not say whether it informed the choice
• Source, on the results page: the table columns "Same answer as original, % of 2,000", "Accuracy, %", "Accuracy change, points", "Confident answers changed", "Same answer as original, % of 1,000", "Accuracy, %, train split", "Decision Index accuracy, % of 300", "Decision Index same answer, %"
• Comparison files: ggml-org: ggml-org/Kev-9B-GGUF, Kev-9B-Q8_0.gguf, SHA-256 d30b225bfdc985d1856bb04a990be76008d0cf40dccbf78938e3069744bc614c; ggml-org: ggml-org/Kev-9B-GGUF, Kev-9B-Q4_K_M.gguf, SHA-256 86a6084984a6ef6818eb12c07cdc67ed3c47b00208d41a8816c64301bc6bcbda

Protocol a100-2026-10, panel test (typed-decisions test split (selection panel)):

Protocol a100-2026-10, panel train_split (typed-decisions train split, 200 cases drawn outside the calibration window (seed 20261010)):

Protocol a100-2026-10, panel decision_index (Decision Index stratified sample (ANLI, ARC-Challenge, ARC-Easy, BANKING77, CLINC150 with out-of-scope, HellaSwag, MMLU, MMLU-Pro, When2Call, WinoGrande; 30 items each)):

Protocol rtx5090-2026-10-13

NVIDIA RTX 5090 cards, 13 October 2026.

• Reference: ggml-org's BF16 GGUF Kev-9B-BF16.gguf (17,924,897,248 bytes, SHA-256 7097686ceab17858c478644e14bee187b5bec775fe4fd6a14aaf02715a903f0a) of jaredpalmer/kev-9b, from ggml-org/Kev-9B-GGUF revision ec2bbfe, run first and alone on one GPU. Its answer probabilities on the same GPU model and build are the reference for every file
• Panel test: typed-decisions test split (selection panel); dataset: LocalLLaMA/typed-decisions; split: test; cases: 400; decisions: 2,000; text SHA-256: 4f294f218ea1da27f3efef936359389c62ea4d3973a41457732990f1d31b647c
• Panel train_split: typed-decisions train split, 200 cases drawn outside the calibration window (seed 20261010); dataset: LocalLLaMA/typed-decisions; split: train; cases: 200; decisions: 1,000
• Panel decision_index: Decision Index stratified sample (ANLI, ARC-Challenge, ARC-Easy, BANKING77, CLINC150 with out-of-scope, HellaSwag, MMLU, MMLU-Pro, When2Call, WinoGrande; 30 items each); dataset: ANLI, ARC, BANKING77, CLINC150, HellaSwag, MMLU, MMLU-Pro, When2Call and WinoGrande (300-item stratified sample); cases: 300; decisions: 300; text SHA-256: 9cb4301cc7d7d96dde9663263c53020912ad628201e5609ab2d65bf16ebb652e
• Tool: llama-server /v1/systemone, which returns a probability for every allowed answer, with all of a case's questions in one request. Same answer: the file's most likely answer is the original's. Mean KL: KL divergence from the original's answer probabilities to the file's, averaged over decisions. Accuracy: against the panel's labels, which are synthetic for typed-decisions and each benchmark's own answer for the Decision Index sample. Accuracy change: the file's accuracy minus the original's on the same decisions, in points, exact to two decimals. Confident answers changed: decisions where the original gave an answer with more than 80% probability and the file chose another. Ranges on the results page are 95% case-bootstrap intervals (1,000 resamples, seed 20261010)
• llama.cpp build: b11540 (commit 1e6f04a75e91da4bb66b5248f3fd4bdeff169d41), built with CUDA 13.3 for sm_120
• Command: llama-server -m <file.gguf> -c 16384 -ub 4096 -b 4096 --parallel 2 --fit off -ngl 99 -t 32 with our decision suite as the client, two workers
• Hardware: NVIDIA GeForce RTX 5090 32 GB cards (driver 580.167.08), one server per GPU at a time, up to three files at once on separate GPUs
• Settings: 16,384-token context, batch 4,096, two slots, all 33 layers on the GPU
• Measured by: TurinTech downloaded every file, ours at revision 398b3fe and ggml-org's at ec2bbfe, checked each by SHA-256, and measured all of them with the same build, settings and GPU model; the numbers are recounted from the run's per-decision files
• Selection: the test decisions were used to choose the published files, so they are not an independent test; the train-split decisions and the Decision Index sample are as described under protocol a100-2026-10
• Source, on the results page: the table columns "Same answer as original, % of 2,000, RTX 5090", "KL to original, RTX 5090", "Accuracy, %, RTX 5090", "Accuracy change, points, RTX 5090", "Confident answers changed, of 415", "Same answer as original, % of 1,000, RTX 5090", "KL to original, train split, RTX 5090", "Accuracy, %, train split, RTX 5090", "Accuracy change, points, train split, RTX 5090", "Confident answers changed, of 196", "Accuracy, % of 300, RTX 5090", "Same answer as original, %, RTX 5090"
• Comparison files: ggml-org: ggml-org/Kev-9B-GGUF at ec2bbfe66202, Kev-9B-Q8_0.gguf, SHA-256 d30b225bfdc985d1856bb04a990be76008d0cf40dccbf78938e3069744bc614c; ggml-org: ggml-org/Kev-9B-GGUF at ec2bbfe66202, Kev-9B-Q4_K_M.gguf, SHA-256 86a6084984a6ef6818eb12c07cdc67ed3c47b00208d41a8816c64301bc6bcbda

Protocol rtx5090-2026-10-13, panel test (typed-decisions test split (selection panel)):

Protocol rtx5090-2026-10-13, panel train_split (typed-decisions train split, 200 cases drawn outside the calibration window (seed 20261010)):

Protocol rtx5090-2026-10-13, panel decision_index (Decision Index stratified sample (ANLI, ARC-Challenge, ARC-Easy, BANKING77, CLINC150 with out-of-scope, HellaSwag, MMLU, MMLU-Pro, When2Call, WinoGrande; 30 items each)):

Protocol gb10-2026-10-13

One NVIDIA GB10, 13 October 2026.

• Reference: ggml-org's BF16 GGUF Kev-9B-BF16.gguf (17,924,897,248 bytes, SHA-256 7097686ceab17858c478644e14bee187b5bec775fe4fd6a14aaf02715a903f0a) of jaredpalmer/kev-9b, from ggml-org/Kev-9B-GGUF revision ec2bbfe, run on the same machine. Its answer probabilities on the same machine and build are the reference for every file
• Panel test: typed-decisions test split (selection panel); dataset: LocalLLaMA/typed-decisions; split: test; cases: 400; decisions: 2,000; text SHA-256: 4f294f218ea1da27f3efef936359389c62ea4d3973a41457732990f1d31b647c
• Panel train_split: typed-decisions train split, 200 cases drawn outside the calibration window (seed 20261010); dataset: LocalLLaMA/typed-decisions; split: train; cases: 200; decisions: 1,000
• Panel decision_index: Decision Index stratified sample (ANLI, ARC-Challenge, ARC-Easy, BANKING77, CLINC150 with out-of-scope, HellaSwag, MMLU, MMLU-Pro, When2Call, WinoGrande; 30 items each); dataset: ANLI, ARC, BANKING77, CLINC150, HellaSwag, MMLU, MMLU-Pro, When2Call and WinoGrande (300-item stratified sample); cases: 300; decisions: 300; text SHA-256: 9cb4301cc7d7d96dde9663263c53020912ad628201e5609ab2d65bf16ebb652e
• Tool: llama-server /v1/systemone, which returns a probability for every allowed answer, with all of a case's questions in one request. Same answer: the file's most likely answer is the original's. Mean KL: KL divergence from the original's answer probabilities to the file's, averaged over decisions. Accuracy: against the panel's labels, which are synthetic for typed-decisions and each benchmark's own answer for the Decision Index sample. Accuracy change: the file's accuracy minus the original's on the same decisions, in points, exact to two decimals. Confident answers changed: decisions where the original gave an answer with more than 80% probability and the file chose another. Ranges on the results page are 95% case-bootstrap intervals (1,000 resamples, seed 20261010)
• llama.cpp build: b11540 (commit 1e6f04a75e91da4bb66b5248f3fd4bdeff169d41), built for CUDA 13.0 (sm_121)
• Command: llama-server -m <file.gguf> -c 16384 -ub 4096 -b 4096 --parallel 2 -ngl 99 -t 20 with our decision suite as the client, two workers
• Hardware: one NVIDIA GB10 (128 GB of memory shared by the CPU and GPU), one file at a time
• Settings: 16,384-token context, batch 4,096, two slots, all layers on the GPU
• Measured by: TurinTech downloaded every file, ours at revision 398b3fe and ggml-org's at ec2bbfe, checked each by SHA-256, and measured all of them with the same build, settings and machine; the numbers are recounted from the run's per-decision files
• Selection: the test decisions were used to choose the published files, so they are not an independent test; the train-split decisions and the Decision Index sample are as described under protocol a100-2026-10
• Source, on the results page: the table columns "Same answer as original, % of 2,000, GB10", "KL to original, GB10", "Accuracy, %, GB10", "Accuracy change, points, GB10", "Confident answers changed, of 414", "Same answer as original, % of 1,000, GB10", "KL to original, train split, GB10", "Accuracy, %, train split, GB10", "Accuracy change, points, train split, GB10", "Decision Index accuracy, % of 300, GB10", "Decision Index same answer, %, GB10"
• Comparison files: ggml-org: ggml-org/Kev-9B-GGUF at ec2bbfe66202, Kev-9B-Q8_0.gguf, SHA-256 d30b225bfdc985d1856bb04a990be76008d0cf40dccbf78938e3069744bc614c; ggml-org: ggml-org/Kev-9B-GGUF at ec2bbfe66202, Kev-9B-Q4_K_M.gguf, SHA-256 86a6084984a6ef6818eb12c07cdc67ed3c47b00208d41a8816c64301bc6bcbda

Protocol gb10-2026-10-13, panel test (typed-decisions test split (selection panel)):

Protocol gb10-2026-10-13, panel train_split (typed-decisions train split, 200 cases drawn outside the calibration window (seed 20261010)):

Protocol gb10-2026-10-13, panel decision_index (Decision Index stratified sample (ANLI, ARC-Challenge, ARC-Easy, BANKING77, CLINC150 with out-of-scope, HellaSwag, MMLU, MMLU-Pro, When2Call, WinoGrande; 30 items each)):

Protocol rtx3060-2026-10-12

One NVIDIA RTX 3060 12 GB, 12 October 2026 (partial suite).

• Reference: ggml-org's BF16 GGUF Kev-9B-BF16.gguf (17,924,897,248 bytes, SHA-256 7097686ceab17858c478644e14bee187b5bec775fe4fd6a14aaf02715a903f0a) of jaredpalmer/kev-9b, run on the same machine with 10 of its 32 blocks on the GPU and the rest on the CPU
• Panel test: typed-decisions test split (selection panel); dataset: LocalLLaMA/typed-decisions; split: test; cases: 400; decisions: 2,000; text SHA-256: 4f294f218ea1da27f3efef936359389c62ea4d3973a41457732990f1d31b647c
• Panel train_split: typed-decisions train split, 200 cases drawn outside the calibration window (seed 20261010); dataset: LocalLLaMA/typed-decisions; split: train; cases: 200; decisions: 1,000
• Panel decision_index: Decision Index stratified sample (ANLI, ARC-Challenge, ARC-Easy, BANKING77, CLINC150 with out-of-scope, HellaSwag, MMLU, MMLU-Pro, When2Call, WinoGrande; 30 items each); dataset: ANLI, ARC, BANKING77, CLINC150, HellaSwag, MMLU, MMLU-Pro, When2Call and WinoGrande (300-item stratified sample); cases: 300; decisions: 300; text SHA-256: 9cb4301cc7d7d96dde9663263c53020912ad628201e5609ab2d65bf16ebb652e
• Tool: llama-server /v1/systemone, which returns a probability for every allowed answer, with all of a case's questions in one request. Same answer: the file's most likely answer is the original's. Mean KL: KL divergence from the original's answer probabilities to the file's, averaged over decisions. Accuracy: against the panel's labels, which are synthetic for typed-decisions and each benchmark's own answer for the Decision Index sample. Accuracy change: the file's accuracy minus the original's on the same decisions, in points, exact to two decimals. Confident answers changed: decisions where the original gave an answer with more than 80% probability and the file chose another. Ranges on the results page are 95% case-bootstrap intervals (1,000 resamples, seed 20261010)
• llama.cpp build: b11540 (commit 1e6f04a75e91da4bb66b5248f3fd4bdeff169d41), CUDA build (driver 535.309.01)
• Command: llama-server -m <file.gguf> -c 16384 -ub 4096 -b 4096 --parallel 2 -t 12 --fit off -ngl 99 (the original with -ngl 10) with our decision suite as the client, two workers
• Hardware: one NVIDIA GeForce RTX 3060 12 GB (driver 535.309.01) with an AMD Ryzen 9 5900X and 62 GB of RAM, one file at a time
• Settings: 16,384-token context, batch 4,096, two slots; our files and ggml-org's Q4_K_M fully on the GPU; the test decisions, train-split decisions, Decision Index sample and confidence bins only (the behaviour checks did not run)
• Measured by: TurinTech measured every file in these tables on this machine against the original run there, with the same build and settings; our files matched the published SHA-256s and ggml-org's Q4_K_M the hash of the A100 run; the numbers are recounted from the run's per-decision files. The IQ3_M file did not exist when this run took place
• Selection: the test decisions were used to choose the published files, so they are not an independent test; the train-split decisions and the Decision Index sample are as described under protocol a100-2026-10
• Source, on the results page: the table columns "Same answer as original, % of 2,000, RTX 3060", "KL to original, RTX 3060", "Accuracy, %, RTX 3060", "Accuracy change, points, RTX 3060", "Confident answers changed, of 415, RTX 3060", "Same answer as original, % of 1,000, RTX 3060", "KL to original, train split, RTX 3060", "Accuracy, %, train split, RTX 3060", "Accuracy change, points, train split, RTX 3060", "Decision Index accuracy, % of 300, RTX 3060", "Decision Index same answer, %, RTX 3060"
• Comparison files: ggml-org: ggml-org/Kev-9B-GGUF, Kev-9B-Q4_K_M.gguf, SHA-256 86a6084984a6ef6818eb12c07cdc67ed3c47b00208d41a8816c64301bc6bcbda

Protocol rtx3060-2026-10-12, panel test (typed-decisions test split (selection panel)):

Protocol rtx3060-2026-10-12, panel train_split (typed-decisions train split, 200 cases drawn outside the calibration window (seed 20261010)):

Protocol rtx3060-2026-10-12, panel decision_index (Decision Index stratified sample (ANLI, ARC-Challenge, ARC-Easy, BANKING77, CLINC150 with out-of-scope, HellaSwag, MMLU, MMLU-Pro, When2Call, WinoGrande; 30 items each)):