Instructions to use LiquidAI/LFM3-1.7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LiquidAI/LFM3-1.7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LiquidAI/LFM3-1.7B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LiquidAI/LFM3-1.7B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM3-1.7B", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LiquidAI/LFM3-1.7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LiquidAI/LFM3-1.7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM3-1.7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LiquidAI/LFM3-1.7B

SGLang

How to use LiquidAI/LFM3-1.7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LiquidAI/LFM3-1.7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM3-1.7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LiquidAI/LFM3-1.7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM3-1.7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LiquidAI/LFM3-1.7B with Docker Model Runner:
```
docker model run hf.co/LiquidAI/LFM3-1.7B
```

a dspark model would be wonderful, even if silly

#23

by Smorty100 - opened 8 days ago

Discussion

Smorty100

8 days ago

The model already runs very quickly, but its weird seeing how all lfm 2.5 models seem to have dspark drafters, including the small 2.6B, but this new one doesn't.

of course this is a little silly, since its already sich a small model, but higher speeds are still nice to have.

Magyarmex

8 days ago

i agree.

rasyosef

4 days ago

•

edited 1 day ago

LFM3-1.7B-DSpark: ⚡Up to 3.1× faster LFM3-1.7B inference with a DSpark draft model (vLLM + llama.cpp)

DSpark Model: https://huggingface.co/rasyosef/LFM3-1.7B-DSpark
GGUFs: https://huggingface.co/rasyosef/LFM3-1.7B-DSpark-GGUF

A lossless DSpark draft model for LiquidAI/LFM3-1.7B, trained with speculators (training code).

Up to 3.14× faster on math and 2.63× on code, 2.88× on average (default generation params)
Mean acceptance length 4.10 on math and 3.24 on code, 2.83 overall (block size 8)

Usage

vLLM

Requires vLLM from PR #66323 (Python-only changes, so precompiled kernels can be reused):

VLLM_USE_PRECOMPILED=1 uv pip install \
  "vllm @ git+https://github.com/vllm-project/vllm.git@refs/pull/66323/head"

Then serve:

vllm serve LiquidAI/LFM3-1.7B --port 8000 \
  --speculative-config '{"model": "rasyosef/LFM3-1.7B-DSpark", "num_speculative_tokens": 8}'

llama.cpp (GGUF)

A GGUF build of this drafter is available at rasyosef/LFM3-1.7B-DSpark-GGUF. Pair it with a LFM3-1.7B GGUF target:

llama-server -hf LiquidAI/LFM3-1.7B-GGUF:Q8_0 \
  -hfd rasyosef/LFM3-1.7B-DSpark-GGUF:Q8_0 \
  --spec-type draft-dspark --spec-draft-n-max 4 --spec-draft-n-min 0 \
  -fa on -ngl 99

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment