CEIA Lumi-4B

Lumi is a small language model developed by the NLP team at CEIA/UFG, available as a 4B instruct model. Built with a focus on Brazilian Portuguese, Lumi was developed through Continual Pre-Training (CPT) combined with Knowledge Distillation (KD), followed by Supervised Fine-Tuning (SFT), starting from Nemotron 3 Nano 4B.

During CPT, we trained the model on 50 billion tokens, using Nemotron 3.5 Lightning as the distillation teacher. The training mixture included web and document corpora alongside specialized and SFT-style pretraining datasets. Subsequently, we fine-tuned the model using chat and instruction-following data for conversational interaction in Brazilian Portuguese.

This release is the instruct checkpoint after CPT with KD and SFT. The Lumi training pipeline for this release does not include additional Reinforcement Learning (RL), Reinforcement Learning with Verifiable Rewards (RLVR), or dedicated model-identity training. Thinking must remain disabled for this release. These extensions are planned as next steps.

Highlights

• Bilingual Proficiency: Optimized for Brazilian Portuguese and English.
• Retained STEM Performance: CPT and SFT pipelines designed to preserve mathematics and coding capabilities.
• Enhanced Educational Quality: Portuguese pretraining data curated with an educational-quality classifier trained by our team, retaining only high-quality educational content.

Release History

• Nov 6, 2026: Released Lumi-4B-v0.2, trained with an updated data blend for continual pretraining (CPT), improving performance across Portuguese/English benchmarks.
• Oct 29, 2026: Released the Lumi 4B.

Model Details

• Model type: Please refer to NVIDIA model card for details on the model architecture
• Language(s): Portuguese, English
• Tokenizer: Please refer to NVIDIA model card
• Contact: diogo_fernandes@discente.ufg.br

Model Performance

Model evolution

Portuguese tasks

English tasks

Usage

vLLM

The following examples are templates. Replace ceia-br/lumi-4b with the final repository ID and validate the runtime version and chat template with the released checkpoint. They assume the chat template retains the base model's enable_thinking control and explicitly disable thinking for this instruct release.

Create an environment and install vLLM and the API client:

uv venv --python 3.12 --seed
source .venv/bin/activate
uv pip install vllm openai

The following command downloads the model and starts the server:

vllm serve ceia-br/lumi-4b --host 127.0.0.1 \
  --default-chat-template-kwargs '{"enable_thinking": false}'

Commonly used options include:

• --port: Port number for the API server.
• --tensor-parallel-size: Number of GPUs for tensor parallelism.
• --gpu-memory-utilization: Fraction of GPU memory available to the model executor.
• --max-model-len: Maximum combined length of the prompt and generated output, within the context window supported by the checkpoint.

For the full list of available options, please refer to the official vLLM documentation.

Once the server is running, send requests using the OpenAI-compatible API:

from openai import OpenAI

model_name = "ceia-br/lumi-4b"
client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="EMPTY",
)

result = client.chat.completions.create(
    model=model_name,
    messages=[
        {
            "role": "user",
            "content": "Explique o que é inteligência artificial em português brasileiro, com um exemplo simples.",
        }
    ],
    max_tokens=512,
    temperature=0,
    extra_body={
        "chat_template_kwargs": {"enable_thinking": False},
    },
)

print(result.choices[0].message.content)

Best Practices

Use the chat template distributed with the released checkpoint and keep thinking disabled. Set enable_thinking=False in the chat-template arguments, as illustrated above. When using Transformers directly, pass enable_thinking=False to tokenizer.apply_chat_template, following the base model's documented control. The other generation settings in the example above are illustrative.

TODO: Add the recommended generation parameters, validated context length, and tested inference versions.

Unvalidated use cases

• Tool Use: TODO: Document whether function calling was included in post-training and evaluation.
• Thinking / Reasoning Mode: Thinking must remain disabled for this instruct release. Support for enabling thinking is planned for a future release.
• Model Identity: Dedicated identity training has not yet been performed and is planned as a next step.

Training Datasets

CPT (Continual Pre-Training)

The following datasets were used for Continual Pre-Training (CPT) with Nemotron 3.5 Lightning as the knowledge-distillation teacher. Training was conducted using NVIDIA Megatron-Bridge with a context size of 8k (8,192) over a total of 50 billion tokens.

For the Portuguese component of the CPT mixture, we trained an educational-quality classifier and used it to select high-quality educational content. Only the Portuguese data retained through this selection process were included in CPT.

Portuguese and Multilingual Corpora

• FinePDFs-Edu
• FineWeb2
• GigaVerbo-v2
• Nemotron-CC-v2.1
• Nemotron-Pretraining-Specialized-v1.1
• Nemotron-Pretraining-SFT-v1
• CEIA-Pretraining-High-Quality-Educational
• CEIA-Pretraining-High-Quality-Synthetic
• CEIA-Pretraining-High-Quality-Translated-To-Portuguese

SFT (Supervised Fine-Tuning)

• Nemotron-SFT-Multilingual-v2
• Nemotron-SFT-Instruction-Following-Chat-v2
• Nemotron-SFT-Instruction-Following-Chat-v3
• Nemotron-Science-v1
• Nemotron-SFT-Safety-v2
• CEIA-Portuguese-Instruction-Following-Chat
• CEIA-Educational-SFT-Portuguese
• CEIA-Portuguese-Chat

Planned Next Steps

The current release is the instruct checkpoint obtained after SFT. Planned next steps include:

• RL and RLVR: Additional reinforcement-learning post-training, including training with verifiable rewards.
• Thinking: Training and validation for thinking-enabled operation in a future release. Thinking remains disabled in the current release.
• Model Identity: Dedicated identity training for Lumi's self-identification and attribution to the NLP team at CEIA/UFG.

Risks and Limitations

Lumi 4B may generate inaccurate, biased, or unsupported content. Performance outside Brazilian Portuguese and in specialized domains requires dedicated evaluation. Users should assess the model in the context of their intended application.

Acknowledgements

We thank NVIDIA for the Nemotron models used as the starting point and distillation teacher. We also thank Hugging Face, Polyglot, and NVIDIA for providing the datasets used during continual pre-training.

Authors

NLP team — CEIA/UFG

How to cite

If you find Lumi helpful, please cite the model using the entry below. Replace the placeholder repository URL and confirm the preferred author attribution before use.

@misc{ceiaufg2026lumi4b,
  title        = {Lumi 4B},
  author       = {{NLP Team, CEIA/UFG}},
  year         = {2026},
  howpublished = {Hugging Face model repository},
  url          = {https://huggingface.co/ceia-br/lumi-4b}
}

References

[NVIDIA] Nemotron 3 Nano 4B Model Card.