Benchmarks

How each BenchToward benchmark is made: the published evaluation it adopts, the open SNUH data it runs on, and where it stands. Built by the BenchToward benchmark builder; watch builds on Live.

Code on GitHub: benchtoward/benchmarks (Apache-2.0, problem reports as issues). Data on KHDP, the data platform of Seoul National University Hospital: each benchmark links its dataset there, and datasets BenchToward builds are uploaded to KHDP with a DOI.

All 16 Building 1 In review 7 Published 8

Benchmark	Data	Adopted from	Status
Korean clinical knowledge (KMLE-level MCQ) clinicalqa@0	ClinicalQA: Korean Medical Knowledge QA Benchmark	BenchToward definition	published	code · 20 results
30-day postoperative in-hospital mortality inspire-mortality@0	INSPIRE	INSPIRE authors' 30-day in-hospital mortality task (gbm_mortality.py)	published	code · 3 results
Korean operating-room speech recognition k-or-speech-asr@0	Synthetic K-OR Speech Audio	BenchToward definition	published	code · 2 results
Korean surgical robot voice commands kor-surgical-robot-voice-command@0	Korean Surgical Robot Voice-Command Benchmark	Dataset author's evaluation code (prompt, scorer, paired comparison), ported unchanged	published	code · 14 results
ICU cardiac arrest within 0.5-24 h from a 5-min ECG epoch scope-arrest@0	SCOPE: ICU ECG and PPG before cardiac arrest, death or discharge	BenchToward definition	published	code · 4 results
Synthetic-SNUH Text-to-SQL (OMOP CDM, Korean and English) synthetic-snuh-text2sql@0	Synthetic-SNUH v4	EHRSQL 2024 shared task scoring (reliability score) with Spider test-suite execution matching	published	code · 4 results
Abnormal rhythm in 5-second intraoperative ECG (lead II) vitaldb-arrhythmia-abnormal@0	VitalDB Arrhythmia Database	Eun et al. 2026 (Technical Validation)	published	code · 2 results
Intraoperative hypotension 5 min ahead (numeric MAP) vitaldb-ioh@0	VitalDB	BenchToward definition	published	code · 4 results