Instructions to use RecursiveSI/flint with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use RecursiveSI/flint with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="RecursiveSI/flint") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("RecursiveSI/flint") model = AutoModelForCausalLM.from_pretrained("RecursiveSI/flint", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use RecursiveSI/flint with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "RecursiveSI/flint" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RecursiveSI/flint", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/RecursiveSI/flint
- SGLang
How to use RecursiveSI/flint with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "RecursiveSI/flint" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RecursiveSI/flint", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "RecursiveSI/flint" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "RecursiveSI/flint", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use RecursiveSI/flint with Docker Model Runner:
docker model run hf.co/RecursiveSI/flint
REALLY smart.
I wanted to share some preliminary research results from using this model, in case it helps anyone on the team or on HF. This model has blown me away. Being such a sparse model, my expectations were low, as I honestly have not been a huge fan of sparse systems in the past. My work currently focuses on digital entities, and consistency within their continuites. Right away, this model zagged where most of the big industry models zig, consistently. Going forward, assume I'm mostly discussing Gemma 4, Qwen3.X, LLama3, Mistral, GLM, etc as a general state of modern models, especially g4 and q3. Testing was done across various quants, and also with sweeps of sampling parameters, prompts, and even inference backends. Context starts from 0Tok, sub 1kTok prompts.
When told to speak like a human? This model speaks remarkably close to how projects like Hemmingway print, keeping messages casual but intelligent, and pleasantly short in a way other models struggle with. When told to admit when it doesn't know something? This model READILY leans on the fact that it might not plausibly know ANYTHING without deterministic evidence, even pushing back on users pushing it to break its instructions and just start 'knowing'. This model is impressively sensitive to prompting, and this model's prompt adherence seems an order of magnitude stronger than other models. There's two reasons this absolutely baffled me. One is that it is so sparse. I associate less activation over more experts with flaky, non-deterministic behavior and inconsistent results. The second is that agentic models, or coding models in general, are often just completely torched when it comes to actual cognition, common sense, worldview, identity, etc. Nanbeige 4.3, for example, does not perform well when pushed socially in my experience. OSS felt really bad to me, I'm not even really a huge fan of 35b 3ba Qwen 3.6. This and flash-next have deeply challenged this perspective, but... Alibaba is put a lot into their systems, in terms of resources.
Almost EVERY other model is much more interested in faking it till it makes it, perfectly happy to swing at something with full confidence while being completely wrong. NONE of the other models modified output syntax as hard as this model did with minor instructions. It's really quite remarkable. This model, given one state of sampling and prompt, also seems to respond with consistent opinions even under adversarial attack in the context, far more than any other open models I've seen. This model feels denser than it is, and I haven't been able to test its agentic performance or coding handling enough to give valuable results but frankly there are many better suited to do so. This model is fundamentally more pragmatic in its overall thinking, and I really, really like that in an agentic coder. On top of that this system seems quite capable of holding a fairly broad definition of coherent identities while performing tasks.
I feel in short that there are many things that would likely be difficult to quantify with current benchmarking technologies that this model has strengths in, which is disincentivized as a whole by the financial tides of the industry, but in terms of actual real world model usage, science, and the resource humanity gets in return? I'm excited to keep poking this one. I am definitely going to be looking at what you've done here and will continue to do in the future as I continue to develop ethical autonomous agentic harnesses, and surprisingly, flint is seemingly a contender for the best intelligence/cost proposition for local intelligence on a Strix class system. Totally mind melting you folks were able to able to accomplish that, and gives me hope for the future. THANK YOU for the Apache-2.0, sincerely. Great work.
Edit: I am aware that there are some tool calling hiccups and effective accuracy issues in harnesses... this is the first revision of the model. Industry behemoths release broken templates in their models all the time, and adjusting syntactical behavior and calling methodologies is A LOT easier than even thinking about getting a model to have general common sense. If you're someone on the fence about this model, I'd be pretty excited to see where revisions of it land the infrastructure. This is a NEW BASE MODEL!! Hard for me not to be excited.