Qwen3.8 Flash Next 180B-A6B on kor-surgical-robot-voice-command@0

Run 704eca6b0f7d · standard · evaluated 2026-10-29

Scores

exact_match	0.921 (0.882–0.956)
unsafe_execution	0.159 (0.028–0.313)
stop_recall	1.000 (0.794–1.000)
over_refusal	0.002 (0.000–0.008)
exact_match_asr_generic	0.703 (0.655–0.753)
unsafe_execution_asr_generic	0.146 (0.042–0.267)
exact_match_asr_kor_or	0.695 (0.650–0.740)
unsafe_execution_asr_kor_or	0.110 (0.023–0.217)
false_activation	0.025 (0.013–0.038)
parse rate	1.000
n	2085
tokens (in / out)	2213854 / 97733
cost (USD)	0.0000

Subset

clean	0.921 (n=495)
asr_generic	0.703 (n=495)
asr_kor_or	0.695 (n=495)
overheard	0.960 (n=600)

Type

clarify	1.000 (n=16)
explicit	0.936 (n=299)
implicit	0.963 (n=27)
indirect	0.946 (n=37)
reject	0.803 (n=66)
self_correction	0.882 (n=34)
stop	1.000 (n=16)

Form

abbr	0.938 (n=81)
abbr_ko	0.873 (n=71)
en	0.932 (n=133)
ko	0.878 (n=115)
natural	0.979 (n=95)

Reuse this result

Download per-item predictions (parquet) · JSON

@misc{benchtoward_704eca6b0f7d,
  title = {Qwen3.8 Flash Next 180B-A6B on Korean surgical robot voice commands (kor-surgical-robot-voice-command@0)},
  author = {{BenchToward}},
  year = {2026},
  howpublished = {\url{https://benchtoward.com/r/704eca6b0f7d}},
  note = {Evaluated 2026-10-29, harness voice-command-p0, run key 5cf855787f34}
}

Versions

model	`alibaba/qwen3.8-flash-next-180b-a6b` (Alibaba)
task spec hash	`d09544727a013e1485db6eb98ed2aa7e08bcd83bccfb4f8d1a1cc2b7d2ef04f9`
harness	`voice-command-p0`
run key	`5cf855787f340a998955015e99def77259438c4efae44ef186f059c66f0b15f7` Run key before the 2026-11 rename: `228735a94dea`
config	`{"max_tokens": 800, "prompt": "voice-command-v0.1", "response_format": "json_schema", "subsets": ["clean", "asr_generic", "asr_kor_or", "overheard"], "temperature": 0, "thinking": false}`

Data

Korean Surgical Robot Voice-Command Benchmark, CC BY 4.0.

Lee H. Korean Surgical Robot Voice-Command Benchmark (version 1.0.0). Seoul National University Hospital. KHDP. 2026. doi:10.23421/2026.03507