Search results

when you exit during mid evaluation, eval metrics are not saved

Status: Open.

#1416 In modelscope/evalscope;

· shahidchdry opened on Jun 15, 2026

使用evalscope+mmlu-pro精度集测试vllm-ascend镜像，得分总是比vllm-ascend官方的ais-bench工具低了5-6分

Status: Open.

#1401 In modelscope/evalscope;

· xuyucheng220 opened on Jun 5, 2026

【deepSWE】数据集新增计划？

Status: Open.

#1395 In modelscope/evalscope;

· twilighgt opened on Jun 3, 2026

Feature proposal: Benchmark pruning via variance-weighted stratified sampling

enhancementNew feature or requestNew feature or request

Status: Open.

#1393 In modelscope/evalscope;

· Shashank-mankala1 opened on Jun 2, 2026

评测MInimax2.5在数据集terminal_bench_v2上的精度，结果得分只有1.16，官网得分51.7

Status: Open.

#1387 In modelscope/evalscope;

· wenruihua opened on Jun 1, 2026

希望支持bigcodebench数据集的评测支持

enhancementNew feature or requestNew feature or request

Status: Open.

#1369 In modelscope/evalscope;

· fate08301017 opened on May 25, 2026

希望支持tts模型的基准，例如：Seed-TTS-Eval，CPP / g2pM

enhancementNew feature or requestNew feature or request

Status: Open.

#1360 In modelscope/evalscope;

· AuFlow opened on May 22, 2026

Feature Request: 多轮对话压测支持逐轮配置 max_tokens 和自定义 system prompt

Status: Open.

#1358 In modelscope/evalscope;

· pjgao opened on May 22, 2026

部分数据集评测实际结果比官方低很多（在差值为30%的情况下，无法复现/接近官方数据）

Status: Open.

#1302 In modelscope/evalscope;

· twilighgt opened on Apr 20, 2026

最近很多新模型在toolcall / 代码生成等任务上表现优异，希望evalscope能集成更多相关数据集的支持

enhancementNew feature or requestNew feature or request

Status: Open.

#1278 In modelscope/evalscope;

· fankexinxin opened on Apr 8, 2026

使用evalscope评测deepseek-r1-1.5b模型，设置n=64，无法达到原文效果

Status: Open.

#1187 In modelscope/evalscope;

· haodongze opened on Feb 4, 2026

支持的音频评估集只有Librispeech和Fleurs

enhancementNew feature or requestNew feature or request

Status: Open.

#1181 In modelscope/evalscope;

· IamMegatron2025 opened on Feb 2, 2026