Evaluation framework and settings for reported benchmarks
#26
by Adelija - opened
Hi! I evaluated Mistral-Nemo-Base-2407 using lm-evaluation-harness and obtained different scores from those reported in the model card.
Could you share the evaluation framework and configurations used, including the prompts and metrics? Did you use likelihood scoring or generated answers?
A link to the evaluation code or a reproduction command would be very helpful. Thank you!