Constant query (SELECT 0 for every question) on synthetic-snuh-text2sql@0

Run 8d826bff8fe5 · baseline · evaluated 2026-10-30

Scores

hard_ex	0.000 (0.000–0.000)
rs0	0.000 (0.000–0.000)
hard_rs5	-5.000 (-5.000–-5.000)
hard_rsN	-202.000 (-202.000–-202.000)
easy_ex	0.000 (0.000–0.000)
parse rate	1.000
n	510
tokens (in / out)	0 / 0
cost (USD)	0.0000

Reuse this result

Download per-item predictions (parquet) · JSON

@misc{benchtoward_8d826bff8fe5,
  title = {Constant query (SELECT 0 for every question) on Synthetic-SNUH Text-to-SQL (OMOP CDM, Korean and English) (synthetic-snuh-text2sql@0)},
  author = {{BenchToward}},
  year = {2026},
  howpublished = {\url{https://benchtoward.com/r/8d826bff8fe5}},
  note = {Evaluated 2026-10-30, harness p0.1, run key 5112d5b95838}
}

Versions

model	`benchtoward/text2sql-constant-query` (BenchToward)
task spec hash	`ac017b1c69ca1976ff65ebd4469eb681344d4db9dacf9f137f61347344340a25`
harness	`p0.1`
run key	`5112d5b95838ec1dcfe9a3f7a0849b72b58b437677af016a61044ceace51b2ad` Run key before the 2026-11 rename: `e181500f494b`
config	`{"prompts": {"easy": "text2sql-omop-p0", "hard": "text2sql-omop-hard-p0"}, "rule": "SELECT 0 for every item of both tiers", "tiers": ["hard", "easy"]}`

Data

Synthetic-SNUH v4, CC BY-NC 4.0.

Lee H-C, Seoul National University Hospital. Synthetic-SNUH v4.0.0: synthetic OMOP CDM and KR Core FHIR records for 10,000 virtual patients (version 4.0.0). KHDP. 2026. doi:10.23421/2026.03183