Compare top models

LLM Benchmarks

See how today's leading models write SpacetimeDB code, scored task by task on live modules.

Gemini 3.1 Pro

Typescript · With Guidelines
100.0%Task Pass Rate
89/89Checks Passed
47Total Evals

Pass Rate Over Time

Click a data point to view results from that date.

auth

100% · 8 evals
StatusTask IDChecksRun Time
t_026_auth_identity_check1/112.7s
t_027_private_vs_public_table1/111.0s
t_041_registered_user_gate1/19.5s
t_042_admin_bootstrap1/111.6s
t_043_role_based_access1/113.2s
t_044_ban_list1/114.0s
t_045_rate_limit1/113.0s
t_046_shared_document1/116.2s

basics

100% · 15 evals
StatusTask IDChecksRun Time
t_000_empty_reducers1/111.6s
t_001_basic_tables1/111.3s
t_002_scheduled_table1/111.1s
t_003_struct_in_table1/16.4s
t_004_insert2/27.3s
t_005_update3/39.0s
t_006_delete3/311.0s
t_007_crud4/413.1s
t_008_index_lookup3/310.8s
t_009_init4/47.7s
t_010_connect1/19.4s
t_011_helper_function3/38.3s
t_038_schedule_at_time1/113.4s
t_039_cancel_schedule1/113.8s
t_040_lifecycle_player1/17.2s

data modeling

100% · 6 evals
StatusTask IDChecksRun Time
t_024_event_table1/17.9s
t_025_optional_fields1/19.2s
t_028_cascade_delete1/19.2s
t_029_filter_and_aggregate1/113.5s
t_030_two_table_join1/113.2s
t_031_unique_constraint1/19.0s

queries

100% · 8 evals
StatusTask IDChecksRun Time
t_022_view_basic1/19.8s
t_023_view_per_user1/18.2s
t_032_range_query1/118.3s
t_033_sort_and_limit1/116.6s
t_034_find_first1/110.3s
t_035_select_distinct1/114.1s
t_036_count_without_collect1/120.7s
t_037_multi_column_filter1/118.8s

schema

100% · 10 evals
StatusTask IDChecksRun Time
t_012_spacetime_product_type3/37.7s
t_013_spacetime_sum_type3/310.6s
t_014_elementary_columns3/38.6s
t_015_product_type_columns3/39.8s
t_016_sum_type_columns3/310.4s
t_017_scheduled_columns2/210.7s
t_018_constraints3/310.2s
t_019_many_to_many5/517.6s
t_020_ecs7/716.3s
t_021_multi_column_index5/511.2s