Instructions to use google/gemma-4-E4B-it with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use google/gemma-4-E4B-it with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("google/gemma-4-E4B-it") model = AutoModelForMultimodalLM.from_pretrained("google/gemma-4-E4B-it", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- AMD Developer Cloud
Add IFBench evaluation result
Add IFBench evaluation result
Summary
This PR adds the IFBench score reported for google/gemma-4-E4B-it to .eval_results/, following the Hugging Face Hub evaluation-results specification. IFBench (allenai/IFBench_test) is now an official Hub benchmark, with prompt-level loose accuracy as its headline metric.
Benchmarks Added
- IFBench: 44.0
Source
- Gemma 4 technical report: https://arxiv.org/abs/2607.02770 (row: "IFBench")
Files Added
.eval_results/ifbench.yaml
Verification
This score is self-reported in the source above; no verified token is provided because it was not run via HF Jobs with inspect-ai. The Hub task is prompt-level loose accuracy (0-100). If your number was computed with a different IFBench variant (strict, or instruction-level), let us know and we will fix or close this PR.
Closing this: the source doesn't say which IFBench variant this score is (the Hub task is prompt-level loose accuracy), so we can't map it reliably. If you can confirm it's prompt-level loose, we're happy to reopen.
Reopening: keeping this score as reported. If it isn't prompt-level loose accuracy, let us know and we'll update or close it.