Instructions to use Qwen/Qwen3-14B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Qwen/Qwen3-14B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Qwen/Qwen3-14B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-14B")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-14B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
AMD Developer Cloud
Local Apps Settings

vLLM

How to use Qwen/Qwen3-14B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Qwen/Qwen3-14B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3-14B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Qwen/Qwen3-14B

SGLang

How to use Qwen/Qwen3-14B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Qwen/Qwen3-14B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3-14B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Qwen/Qwen3-14B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Qwen/Qwen3-14B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Qwen/Qwen3-14B with Docker Model Runner:
```
docker model run hf.co/Qwen/Qwen3-14B
```

PSA: check your served context length — Ollama gave me 4k for a model declaring 40960

#22

by SatoshiAkiyama - opened Aug 20

Discussion

SatoshiAkiyama

Aug 20

•

edited Aug 20

Not a problem with the model — a serving-layer issue that had me attributing
weeks of results to the model itself, so posting in case anyone else is in the
same position.

On ollama 0.32.9, ollama show qwen3:14b reports a context length of 40960.
Load it with nothing specified and ollama ps reports:

NAME         ID              SIZE      PROCESSOR    CONTEXT    UNTIL
qwen3:14b    bdbd181c33f2    9.6 GB    100% GPU     4096       4 minutes from now

No num_ctx in the Modelfile, no OLLAMA_CONTEXT_LENGTH set. 16GB card, which
puts it in the 4k tier described on Ollama's context-length page.

It does not error. Send a prompt longer than that and you get HTTP 200,
prompt_eval_count 2050, and an answer built from whatever survived.
Truncation takes the front, so the system prompt, your formatting rules and
the earlier turns of a conversation are what disappear first. It reads exactly
like a model that stops following instructions on long inputs.

With a needle on the first line of a ~7.9k-token prompt, the reply quoted a
fragment from a filler line near the end instead. Adding "num_ctx": 16384 to
the request, same prompt, it answers correctly.

Some models on the same server return a clear HTTP 400 for the identical
request instead of truncating. I couldn't work out what separates them: it
isn't model size (a stock Qwen3-4B truncates the same way this one does), it
isn't the declared context length, and it isn't ollama create — a derivative
built from a Modelfile containing nothing but FROM qwen3:14b behaves
identically to its parent.

Quickest check

Read prompt_eval_count in the API response and compare it with how many
tokens you actually sent. It's the number evaluated, not the number submitted.
ollama ps also shows the CONTEXT column on builds that have it.

Script that runs the comparison for you (standard library only, no
dependencies), plus a longer writeup:
https://github.com/Bigbonus/ollama-context-window-check

python check_context.py qwen3:14b

Reported upstream as https://github.com/ollama/ollama/issues/17889.

Happy to compare notes if anyone else has hit this — particularly if you can
tell me why the truncated count is 2050 when the allocated window is 4096. It
reproduces across models and prompts and I have no explanation for it.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment