Instructions to use openai/gpt-oss-20b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use openai/gpt-oss-20b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="openai/gpt-oss-20b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("openai/gpt-oss-20b")
model = AutoModelForCausalLM.from_pretrained("openai/gpt-oss-20b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
AMD Developer Cloud
Local Apps Settings

vLLM

How to use openai/gpt-oss-20b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "openai/gpt-oss-20b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openai/gpt-oss-20b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/openai/gpt-oss-20b

SGLang

How to use openai/gpt-oss-20b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "openai/gpt-oss-20b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openai/gpt-oss-20b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "openai/gpt-oss-20b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openai/gpt-oss-20b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use openai/gpt-oss-20b with Docker Model Runner:
```
docker model run hf.co/openai/gpt-oss-20b
```

Agentic coding test of 5 local models: WinForms exe + 5 console languages (full matrix)

#278

by MR1KRAKEN - opened about 12 hours ago

Discussion

MR1KRAKEN

about 12 hours ago

Agentic coding test of 5 local models: WinForms exe + 5 console languages

I ran five local models as autonomous coding agents on two tasks and measured the results. Full logs,
per-model details and Russian versions are in the companion repository:
https://github.com/MR1KRAKEN/local-llm-agent-battle-tests (Russian docs live in the ru/ folder there).

Testing was carried out under the supervision of DeepSeek-V4.1-Flash (DSH orchestrator). It started
the servers, assigned the tasks, ran the automated checks and collected the metrics. It did not write or
fix model code.

Task

Write a working C#/WinForms calculator (.NET 8) in a given folder, build it and publish a single-file
exe (dark theme, 4×5 button grid, keyboard support, division-by-zero handling).
Write console calculators in Python, C, JavaScript, Rust and Java, each verified against eight fixed
expressions (2+3, 7*8, 100/4, 2.5+1.5, -4+10, 0.1+0.2, (2+3)*4, 10/0).

Bench

Ryzen 7 9800X3D, 64 GB DDR5, RTX 4070 Ti 12 GB, Windows 11, .NET SDK 8. llama.cpp (ctx 65536,
--jinja, --reasoning-budget 0, KV q8_0) for GGUF models, Ollama for gpt-oss:20b.

Results

Model	WinForms exe	Python	C	JS	Rust	Java	Console total
Nemotron-3.5-Lightning-30B-A3B Q4_0	⚠️ exe built, mouse works, keyboard crashed	8/8	7/8	8/8	8/8	6/8	37/40
Qwen3.5-9B Q4_K_M	⚠️ exe built, UI unusable	4/8	0/8	8/8	7/8	0/8	19/40
LFM2-24B-A2B Q4_K_M	⏱ no tool call reached the harness	—	—	—	—	—	—
Bonsai-27B Q1_0	❌ degeneration, no build	—	—	—	—	—	—
gpt-oss:20b (Ollama)	❌ no exe in the main run	8/8	8/8	8/8	8/8	8/8*	40/40*

n/8 — matched cases out of eight; x/40 — sum across five languages. * — gpt-oss Java was compiled
only after a separate orchestrator instruction; without it the total would be 32/40.

Per-model reports

Links to the individual reports are in the comments below.

Main findings

Harness matters more than weights. The same models scored zero exes in an earlier one-shot harness
and three exes once compiler feedback and surgical edit calls were available.
"Builds" is not "works". All three produced exes had functional defects (crash on keypress, dead
input, broken decimals). The formal check (process alive 8 s) missed them; manual mouse/keyboard tests
found them.
Local single-line fixes land, parser rewrites do not. Removing base.OnKeyDown and changing
printf("%.10f") to %.10g succeeded. Rewriting a parenthesis parser failed for every model.
Self-verification is unreliable. Every model claimed success while the checks failed — including one
that attached a log showing its own failure.
Profiles differ sharply. Nemotron is the all-rounder; gpt-oss is the console champion with a weak GUI;
Qwen is fastest but narrow (0/8 in C and Java); Bonsai at Q1_0 degenerates; LFM2 is blocked by the
integration, not by its ability to emit tool calls.

MR1KRAKEN

about 12 hours ago

Per-model reports:

MR1KRAKEN

about 11 hours ago

Follow-up: what other people report on these exact weights

Since publishing, I read the existing threads and upstream issues. Four findings are worth adding, two of them
correct our own wording:

Nemotron Q4_0 file had a metadata bug, now fixed. The official artifact wrote
attention.head_count_kv as zeros, so block 5 was treated as an SSM block
(llama.cpp issue #27718, fixed by
PR #27729 on 2026-08-26). Our local file is
SHA256 61f87e75…0e3c, i.e. the re-uploaded fixed one — our numbers are unaffected.
LFM2 uses a Pythonic tool-call format. Per the official card, calls look like
<|tool_call_start|>[func(arg="x")]<|tool_call_end|> with tools passed as JSON in the system prompt.
Our harness sends OpenAI-style tools, which explains the mismatch we reported.
Bonsai Q1_0 expects a patched runtime. The official repo asks for the PrismML llama.cpp fork with
dedicated 1-bit kernels. We re-ran it there: tool calls work, it writes real code, JavaScript scores 8/8,
C 6/8, Python 0/8, WinForms still fails to compile. The degeneration we first reported belongs to the stock
build, not to the model.
gpt-oss has known template bugs: the commentary channel left valid without tools drops BFCL_v4 to ~3%
(#274), and the template crashes on tool calls
with content: null (#272).

Per-model threads now carry the relevant part of this.

Russian version of the full report: https://github.com/MR1KRAKEN/local-llm-agent-battle-tests (folder ru/).

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment