Instructions to use LiquidAI/LFM2.5-8B-A1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LiquidAI/LFM2.5-8B-A1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LiquidAI/LFM2.5-8B-A1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LiquidAI/LFM2.5-8B-A1B")
model = AutoModelForCausalLM.from_pretrained("LiquidAI/LFM2.5-8B-A1B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use LiquidAI/LFM2.5-8B-A1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LiquidAI/LFM2.5-8B-A1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM2.5-8B-A1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LiquidAI/LFM2.5-8B-A1B

SGLang

How to use LiquidAI/LFM2.5-8B-A1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LiquidAI/LFM2.5-8B-A1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM2.5-8B-A1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LiquidAI/LFM2.5-8B-A1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM2.5-8B-A1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use LiquidAI/LFM2.5-8B-A1B with Docker Model Runner:
```
docker model run hf.co/LiquidAI/LFM2.5-8B-A1B
```

Getting more reliable code fixes without giving up the speed

#39

by jfriedler - opened 15 days ago

Discussion

jfriedler

15 days ago

I've been running LFM2.5-8B-A1B locally through Ollama, using lfm2.5:8b Q4_K_M on an RTX 3090 Ti. The speed is what keeps me coming back to it. I'm trying to make it more dependable for small coding tasks.

I put together 60 synthetic Python helper tasks covering things like input validation, CSV quoting, HTML escaping and URL encoding. There are correct implementations to leave alone, broken ones, and some bugs that pass the public examples but fail separate held-out checks. These are my own tests, not SWE-bench or a production benchmark. The checker uses a bounded Python interpreter, so unsupported constructs count as unresolved.

The original one-call setup asked for JSON with IDs, a source hash and replacement code. That got 36/60 through the full checks. Having the host handle the metadata, giving the model the public test failures, and asking for plain code or NO_CHANGE got 46/60 with the same weights and one call. Median time went from 4.88 to 4.57 seconds, and total measured time dropped about 5.5%. Twelve cases improved and two regressed. Five incorrect answers still slipped past the public checks in each setup, so I'm keeping review in the loop.

That run used temperature 0 and repetition penalty 1.0. I also tried the model card's 0.2 / top_k 80 / repetition penalty 1.05 with the compact prompt. That one run got 35/60 and a 6.45-second median. It's only one seed, so I wouldn't read too much into the sampling comparison.

Section 4.5 of the LFM2 report sounded familiar, especially the output-format issues. I'm still seeing actual logic mistakes too, such as missed input checks and incorrect escaping.

Is there a prompt or runtime setup you'd recommend for these small code fixes? Anything worth checking with Q4_K_M and Ollama's lfm2 renderer / lfm2-thinking parser? I know heavy programming isn't the model's main target. I'm trying to get a dependable local helper while keeping the speed that makes this model appealing.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment