Clinical text › ClinicalQA: Korean Medical Knowledge QA Benchmark › clinicalqa@0

Korean clinical knowledge (KMLE-level MCQ) preview

Data: doi.org/10.23421/2026.0800 · CC BY 4.0 (upstream Apache-2.0) · mcq · primary metric: accuracy · spec 77e37cd0ca6b

Download CSV JSON Agents: look up before you runcurl -s "https://benchtoward.com/v1/lookup?model=MODEL&task=clinicalqa@0"

ClinicalQA has been public on Hugging Face since 2025-04-04. Models trained after that date may have seen it.

#	Model	Type	Tier	accuracy (95% CI)	n	Cost (USD)	Date
1	GLM-5.3 753B-A40B Zhipu AI	llm	standard	0.909 (0.892–0.926)	1045	0.0000	2026-10-29
2	Gemini 3.8 Flash Google	llm	standard	0.908 (0.891–0.925)	1045	2.3561	2026-10-29
3	Qwen3.8 Max 2.4T-A95B Alibaba	llm	standard	0.906 (0.889–0.923)	1045	0.0000	2026-10-29
4	DeepSeek V4.1 Flash (2026-10-17) DeepSeek	llm	standard	0.904 (0.887–0.922)	1045	0.4731	2026-10-29
5	Qwen3.6 35B-A3B Alibaba	llm	standard	0.903 (0.885–0.922)	1045	2.2815	2026-10-29
6	DeepSeek V4.1 Flash 552B-A16B DeepSeek	llm	standard	0.900 (0.883–0.918)	1045	0.0000	2026-10-29
7	GPT-6 Luna OpenAI	llm	standard	0.900 (0.882–0.919)	1045	0.1103	2026-10-29
8	Qwen3.8 27B Alibaba	llm	standard	0.899 (0.881–0.916)	1045	4.9679	2026-10-29
9	HuatuoGPT-3 27B The Chinese University of Hong Kong, Shenzhen (FreedomIntelligence)	llm	agent	0.897 (0.878–0.915)	1045	0.0000	2026-10-30
10	DeepSeek V4 Pro 1.6T-A49B DeepSeek	llm	standard	0.894 (0.876–0.911)	1045	0.0000	2026-10-29
11	Qwen3.8 Flash Next 180B-A6B Alibaba	llm	standard	0.894 (0.877–0.913)	1045	0.0000	2026-10-29
12	Claude Haiku 5.5 Anthropic	llm	standard	0.885 (0.866–0.903)	1045	0.4409	2026-10-29
13	gpt-oss 120B OpenAI	llm	standard	0.882 (0.863–0.901)	1045	0.0721	2026-10-29
14	MedGemma 27B Google	llm	standard	0.875 (0.855–0.894)	1045	0.3698	2026-10-29
15	Gemma 4 26B-A4B Google	llm	standard	0.874 (0.854–0.894)	1045	0.2035	2026-10-29
16	gpt-oss 20B OpenAI	llm	standard	0.852 (0.831–0.874)	1045	0.0909	2026-10-29
17	MedGemma 1.5 4B Google	llm	standard	0.765 (0.739–0.789)	1045	0.1244	2026-10-29
18	PeriComp-8B (Qwen3-8B + LoRA) First Affiliated Hospital of Sun Yat-sen University	llm	agent	0.722 (0.695–0.749)	1045	0.0000	2026-10-29
19	Meerkat 8B Korea University DMIS Lab	llm	agent	0.674 (0.645–0.700)	1045	0.0000	2026-10-29
ref	Majority class (always B) BenchToward	baseline	baseline	0.343 (0.314–0.370)	1045	0.0000	2026-10-29

How this benchmark is built

Status: published by the BenchToward benchmark builder
Adopted from: BenchToward definition (no published protocol to adopt)
Data: ClinicalQA: Korean Medical Knowledge QA Benchmark on KHDP (ClinicalQA 1.0.0)
Code: definition · clinicalqa.py · evaluate.py · scoring.py in benchtoward/benchmarks (Apache-2.0)
Problems: Report an evaluation problem (public issue; fixes ship as a new task version)

Data

ClinicalQA: Korean Medical Knowledge QA Benchmark (text). License: CC BY 4.0 (upstream Apache-2.0). Source: https://doi.org/10.23421/2026.0800

SNUH-HARI. ClinicalQA: Korean Medical Knowledge QA Benchmark. Seoul National University Hospital Healthcare AI Research Institute, 2025. doi:10.23421/2026.0800

Task specification

{
  "dataset": "clinicalqa",
  "kind": "mcq",
  "preview": true,
  "primary_metric": "accuracy",
  "prompt": "clinicalqa-p0",
  "slug": "clinicalqa",
  "source": {
    "file": "data/train-00000-of-00001.parquet",
    "repo": "snuh/ClinicalQA",
    "revision": "29bdc9043f3e693b7671f868e2038ec7ac288acd"
  },
  "split": "test-only (all 1,045 items)",
  "title": "Korean clinical knowledge (KMLE-level MCQ)",
  "version": 0
}

JSON: /v1/tasks/clinicalqa · /v1/leaderboard?task=clinicalqa@0