gpt-oss 20B on synthetic-snuh-text2sql@0

Run 071979f7a992 · standard · evaluated 2026-10-30

Scores

hard_ex	0.193 (0.127–0.271)
rs0	0.223 (0.158–0.292)
hard_rs5	-3.614 (-4.005–-3.183)
hard_rsN	-154.777 (-167.847–-139.718)
easy_ex	0.899 (0.857–0.938)
parse rate	0.951
n	510
tokens (in / out)	2399594 / 1432101
cost (USD)	0.2708

Reuse this result

Download per-item predictions (parquet) · JSON

@misc{benchtoward_071979f7a992,
  title = {gpt-oss 20B on Synthetic-SNUH Text-to-SQL (OMOP CDM, Korean and English) (synthetic-snuh-text2sql@0)},
  author = {{BenchToward}},
  year = {2026},
  howpublished = {\url{https://benchtoward.com/r/071979f7a992}},
  note = {Evaluated 2026-10-30, harness text2sql-p0, run key c492b3b663fb}
}

Versions

model	`openai/gpt-oss-20b` (OpenAI)
task spec hash	`ac017b1c69ca1976ff65ebd4469eb681344d4db9dacf9f137f61347344340a25`
harness	`text2sql-p0`
run key	`c492b3b663fb31442176f740278e4360f8120839406d404e5fcd056204a17994` Run key before the 2026-11 rename: `0687d608194f`
config	`{"max_tokens": 16384, "prompts": {"easy": "text2sql-omop-p0", "hard": "text2sql-omop-hard-p0"}, "temperature": 0, "tiers": ["hard", "easy"]}`

Data

Synthetic-SNUH v4, CC BY-NC 4.0.

Lee H-C, Seoul National University Hospital. Synthetic-SNUH v4.0.0: synthetic OMOP CDM and KR Core FHIR records for 10,000 virtual patients (version 4.0.0). KHDP. 2026. doi:10.23421/2026.03183