Instructions to use LiquidAI/LFM3-1.7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LiquidAI/LFM3-1.7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LiquidAI/LFM3-1.7B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LiquidAI/LFM3-1.7B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM3-1.7B", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LiquidAI/LFM3-1.7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LiquidAI/LFM3-1.7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM3-1.7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LiquidAI/LFM3-1.7B

SGLang

How to use LiquidAI/LFM3-1.7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LiquidAI/LFM3-1.7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM3-1.7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LiquidAI/LFM3-1.7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM3-1.7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LiquidAI/LFM3-1.7B with Docker Model Runner:
```
docker model run hf.co/LiquidAI/LFM3-1.7B
```

LFM3-1.7B on WebGPU straight from your GGUF, faster than the ONNX recipe — could you list it as a browser option?

#19

by KikoCis - opened 10 days ago

Discussion

KikoCis

10 days ago

Hi Liquid AI team,

Congratulations on LFM3. We've added the lfm3 architecture to Elffuss Engine, our Apache-2.0 WebGPU runtime that reads GGUF files directly, and made LFM3-1.7B its default model.

What it does with LFM3-1.7B:

Runs your official GGUFs (LFM3-1.7B-Q8_0.gguf and LFM3-1.7B-QAD-Q4_0.gguf) in the browser, with no conversion step.
Covers the full architecture: the gated short convolutions, the sliding-window and global attention layers with rope_freqs, the per-head output gate and the sandwich norms. It also uses your chat template, including <think>.
Checked against llama.cpp layer by layer and token by token: greedy outputs match until the first near-tie in the reference's own logprobs.
Compared with the recipe on your ONNX model card (onnxruntime-web/webgpu, model_q4f16, prompt in chunks of 256), in the same browser with no flags, alternating the two runtimes:
- with your QAD-Q4_0 GGUF: about 1.3× faster on a 512-token prompt and about 1.8× faster generating;
- with the Q8_0 GGUF: about 1.3× and 1.5×.

It is live as the default local model at claw.elffuss.com and code.elffuss.com, and the code is at github.com/utopia-ia/elffuss-engine.

Your model cards list the inference engines each model supports. Would you consider adding Elffuss Engine there as a browser/WebGPU option that runs the GGUF as published? We're happy to share the validation tests (layer by layer against llama-eval-callback, plus greedy decoding) and the benchmark script, or to help if you want to reproduce it on your side.

Thanks for releasing the GGUFs with the chat template and sampling defaults inside: that made the port much easier.

— The Elffuss team (UtopiaIA)

KikoCis

5 days ago

Update: LFM3-1.7B now also runs on the CPU in Elffuss Engine, for browsers with no usable WebGPU (WebAssembly SIMD; threads when the page is cross-origin isolated). Greedy output matches llama.cpp token for token on the QAD-Q4_0 GGUF.

In the same browser session, against llama.cpp compiled to WebAssembly: the prompt is read 2.7x faster and text is written 2.0x faster on one thread; with eight threads, 2.7x and 1.35x.

You can try it at https://elffuss.com/?cpu (the address forces the CPU path, also on devices that have WebGPU).

A question for the team: is there, or will there be, a small draft model that shares LFM3's tokenizer? It would let a device run speculative decoding with the draft on the CPU and LFM3-1.7B on the GPU.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment