Repository navigation
5. Model Management
Jing Jin edited this page May 19, 2025
·
2 revisions
- When you select a capability, models compatible with that task are listed.
- If a model isn't on your device, a "Download" button will be visible after you expand the model item. Tapping it will initiate the download.
- Once downloaded, the button will change to "Try It”, allowing you to use the model.
- Feel free to switch apps or lock your device. The download will continue in the background. We'll send a notification when it's done.
- Some models (e.g., Gemma 3) are gated and require you to log in with your Hugging Face (HF) account and agree to their specific license terms before downloading.
- The app will prompt you to log in via a web browser flow when you attempt to download such a model.
- Successfully logging in and accepting terms will allow the download to proceed. Your HF token is stored securely for future downloads.
For LLM tasks, after the model generates a response, you can view performance metrics:
- Look for a "Stats" button below the response.
- Metrics typically include:
- Time to First Token (TTFT)
- Prefill Speed (tokens/s)
- Decode Speed (tokens/s)
- Latency (sec)
- To delete a downloaded model, simply click on the delete icon next to the model in the model selection screen.