Gemma 4 26B-A4B on clinicalqa@0
Run d3265cd5bd82 · standard · evaluated 2026-10-29
Scores
| accuracy | 0.874 (0.854–0.894) |
|---|---|
| parse rate | 0.999 |
| n | 1045 |
| tokens (in / out) | 408483 / 542513 |
| cost (USD) | 0.2035 |
By category
| Allergy and Immunology | 0.917 (n=36) |
|---|---|
| Cardiology | 0.944 (n=89) |
| Dermatology | 0.750 (n=12) |
| Emergency Medicine | 0.969 (n=32) |
| Endocrinology | 0.897 (n=68) |
| Endocrinology, metabolic, and nutritional disorders | 1.000 (n=1) |
| Forensic Medicine | 1.000 (n=1) |
| Gastroenterology | 0.884 (n=112) |
| General Surgery | 0.862 (n=58) |
| Geriatric Medicine | 1.000 (n=1) |
| Gynecology | 0.771 (n=70) |
| Hematology and Oncology | 0.958 (n=48) |
| Infectious Diseases | 0.846 (n=52) |
| Musculoskeletal (e.g., costochondritis) | 1.000 (n=1) |
| Nephrology | 0.820 (n=111) |
| Neurology / Neurosurgery | 0.898 (n=49) |
| Obstetrics | 0.667 (n=18) |
| Ophthalmology | 1.000 (n=1) |
| Orthopedic Surgery | 0.917 (n=24) |
| Otolaryngology | 1.000 (n=14) |
| Otolaryngology (ENT) | 0.929 (n=14) |
| Pediatrics | 0.839 (n=56) |
| Pharmacology | 1.000 (n=1) |
| Psychiatry | 0.822 (n=45) |
| Pulmonology (Respiratory Medicine) | 0.877 (n=65) |
| Radiology | 1.000 (n=1) |
| Rheumatology | 0.898 (n=49) |
| Urology | 0.923 (n=13) |
| Vascular Surgery | 1.000 (n=2) |
| nan | 0.000 (n=1) |
Reuse this result
Download per-item predictions (parquet) · JSON
@misc{benchtoward_d3265cd5bd82,
title = {Gemma 4 26B-A4B on Korean clinical knowledge (KMLE-level MCQ) (clinicalqa@0)},
author = {{BenchToward}},
year = {2026},
howpublished = {\url{https://benchtoward.com/r/d3265cd5bd82}},
note = {Evaluated 2026-10-29, harness p0.1, run key 4b7fa25dee84}
}
Versions
| model | google/gemma-4-26b-a4b (Google) |
|---|---|
| task spec hash | 77e37cd0ca6bded7fba03792ec1922342b3f6a7dc3ce602881ce196e6aa7377c |
| harness | p0.1 |
| run key | 4b7fa25dee84cb4ead2714ec5a3c63fd62454a93e06f16290e15b7882a234404 |
| config | {"max_tokens": 16384, "prompt": "clinicalqa-p0", "temperature": 0} |
Data
ClinicalQA: Korean Medical Knowledge QA Benchmark, CC BY 4.0 (upstream Apache-2.0).
SNUH-HARI. ClinicalQA: Korean Medical Knowledge QA Benchmark. Seoul National University Hospital Healthcare AI Research Institute, 2025. doi:10.23421/2026.0800