Benchmarks
How each BenchToward benchmark is made: the published evaluation it adopts, the open SNUH data it runs on, and where it stands. Built by the BenchToward benchmark builder; watch builds on Live.
Code on GitHub: benchtoward/benchmarks (Apache-2.0, problem reports as issues). Data on KHDP, the data platform of Seoul National University Hospital: each benchmark links its dataset there, and datasets BenchToward builds are uploaded to KHDP with a DOI.
All 16 Building 1 In review 7 Published 8
| Benchmark | Data | Adopted from | Status | |
|---|---|---|---|---|
| Multi-turn evidence-grounded QA over a patient's Korean notes syn-snuh-chatqa@0 |
syn-snuh | EHRNote-ChatQA (Kim et al. 2026) | in review | Baselines |
| Clinical error detection in synthetic Korean SNUH notes (paired) syn-snuh-note-errors@0 |
syn-snuh | Paired clinical error detection (arXiv 2608.16643) | in review | Checks |
| First-day SOFA from a Korean ICU record (SYN-ICU) syn-icu-sofa@0 |
syn-icu | mimic-code first-day SOFA (MIT-LCP) @ 303d26c, MedCalc-Bench exact match | in review | Checks |
| Postoperative AKI (KDIGO stage 2-3) after noncardiac surgery from pre-operative data inspire-aki@0 |
inspire | Do J et al., JAMIA Open 2026 (Inspire-AKI-prediction v1.0.0-jamia-open, 4dc799c) | in review | Checks |
| 12-lead paper-ECG images (rendered from LYDUS-SNUH-ECG-50K): classification and digitisation lydus-ecg-image@0 |
lydus-snuh-ecg-50k | PhysioNet/CinC Challenge 2024 (Reyna et al.), evaluation-2024 @ 9af7dfb; ecg-image-kit @ 27b90f5 | in review | Checks |
| 12-lead ECG diagnosis: multi-label, PhysioNet/CinC 2021 score lydus-ecg12-dx@0 |
lydus-snuh-ecg-50k | PhysioNet/CinC Challenge 2021 scoring (evaluation-2021 @ e2a75fc) | in review | Checks |
| BIS value from 60 s of raw frontal EEG vitaldb-bis@0 |
vitaldb | vitaldb/openeeg BIS reproduction (cohort.py, metrics.py, predict_bis) | in review | Baselines |