Benchmarks
How each BenchToward benchmark is made: the published evaluation it adopts, the open SNUH data it runs on, and where it stands. Built by the BenchToward benchmark builder; watch builds on Live.
Code on GitHub: benchtoward/benchmarks (Apache-2.0, problem reports as issues). Data on KHDP, the data platform of Seoul National University Hospital: each benchmark links its dataset there, and datasets BenchToward builds are uploaded to KHDP with a DOI.
All 16 Building 1 In review 7 Published 8
| Benchmark | Data | Adopted from | Status | |
|---|---|---|---|---|
| Intraoperative hypotension, the authors' unbiased dataset vitaldb-ioh@1 |
vitaldb | Yang et al., Br J Anaesth 2025 (hyunlimy/intraop_hypotension @ 8045a73) | building | Build |
| Multi-turn evidence-grounded QA over a patient's Korean notes syn-snuh-chatqa@0 |
syn-snuh | EHRNote-ChatQA (Kim et al. 2026) | in review | Baselines |
| Clinical error detection in synthetic Korean SNUH notes (paired) syn-snuh-note-errors@0 |
syn-snuh | Paired clinical error detection (arXiv 2608.16643) | in review | Checks |
| First-day SOFA from a Korean ICU record (SYN-ICU) syn-icu-sofa@0 |
syn-icu | mimic-code first-day SOFA (MIT-LCP) @ 303d26c, MedCalc-Bench exact match | in review | Checks |
| Postoperative AKI (KDIGO stage 2-3) after noncardiac surgery from pre-operative data inspire-aki@0 |
inspire | Do J et al., JAMIA Open 2026 (Inspire-AKI-prediction v1.0.0-jamia-open, 4dc799c) | in review | Checks |
| 12-lead paper-ECG images (rendered from LYDUS-SNUH-ECG-50K): classification and digitisation lydus-ecg-image@0 |
lydus-snuh-ecg-50k | PhysioNet/CinC Challenge 2024 (Reyna et al.), evaluation-2024 @ 9af7dfb; ecg-image-kit @ 27b90f5 | in review | Checks |
| 12-lead ECG diagnosis: multi-label, PhysioNet/CinC 2021 score lydus-ecg12-dx@0 |
lydus-snuh-ecg-50k | PhysioNet/CinC Challenge 2021 scoring (evaluation-2021 @ e2a75fc) | in review | Checks |
| BIS value from 60 s of raw frontal EEG vitaldb-bis@0 |
vitaldb | vitaldb/openeeg BIS reproduction (cohort.py, metrics.py, predict_bis) | in review | Baselines |
| Korean clinical knowledge (KMLE-level MCQ) clinicalqa@0 |
ClinicalQA: Korean Medical Knowledge QA Benchmark | BenchToward definition | published | code · 20 results |
| 30-day postoperative in-hospital mortality inspire-mortality@0 |
INSPIRE | INSPIRE authors' 30-day in-hospital mortality task (gbm_mortality.py) | published | code · 3 results |
| Korean operating-room speech recognition k-or-speech-asr@0 |
Synthetic K-OR Speech Audio | BenchToward definition | published | code · 2 results |
| Korean surgical robot voice commands kor-surgical-robot-voice-command@0 |
Korean Surgical Robot Voice-Command Benchmark | Dataset author's evaluation code (prompt, scorer, paired comparison), ported unchanged | published | code · 14 results |
| ICU cardiac arrest within 0.5-24 h from a 5-min ECG epoch scope-arrest@0 |
SCOPE: ICU ECG and PPG before cardiac arrest, death or discharge | BenchToward definition | published | code · 4 results |
| Synthetic-SNUH Text-to-SQL (OMOP CDM, Korean and English) synthetic-snuh-text2sql@0 |
Synthetic-SNUH v4 | EHRSQL 2024 shared task scoring (reliability score) with Spider test-suite execution matching | published | code · 4 results |
| Abnormal rhythm in 5-second intraoperative ECG (lead II) vitaldb-arrhythmia-abnormal@0 |
VitalDB Arrhythmia Database | Eun et al. 2026 (Technical Validation) | published | code · 2 results |
| Intraoperative hypotension 5 min ahead (numeric MAP) vitaldb-ioh@0 |
VitalDB | BenchToward definition | published | code · 4 results |