Compare top models

LLM Benchmarks

See how today's leading models write Spacetime code, scored task by task on live modules.

Spacetime vs Postgres

← Back

Run Analysis

OpenAI: GPT-6 Astra · 2026-09-10

No repository changes recommended.

Failure patterns

Snake_case view name mismatch in module export (1 task)

Classification: Model limitation
Confidence: Medium

What the model wrote:

export const activeAnnouncements = spacetimedb.anonymousView(
  { name: 'activeAnnouncements', public: true },

What was expected:

export const activeAnnouncements = spacetimedb.anonymousView(
  { name: 'active_announcements', public: true },

What the error says: schema_parity: failed
Why this happened: The generated module used camelCase for the registered view name instead of the snake_case name expected by the schema parity check. The surrounding code was otherwise structurally close, so this appears to be a localized generation mistake rather than a repository issue.
Suggested action: None
Suggested area: None
Affected tasks: t_022_view_basic
AI guidelines gap: Needs manual review; the complete AI guidelines was not supplied

OpenAI: GPT-6 Astra

TypeScript · With Guidelines · 2026-09-10

99.5%Task Pass Rate

189/190Checks Passed

83Total Evals

auth

100% · 9 evals

Pass	Task	Checks	Run Time
✓	Auth Identity Check	1/1	6.8s
✓	Private Vs Public Table	1/1	6.3s
✓	Registered User Gate	1/1	7s
✓	Admin Bootstrap	1/1	7.5s
✓	Role Based Access	1/1	7.4s
✓	Ban List	1/1	8.5s
✓	Rate Limit	1/1	9.2s
✓	Shared Document	1/1	11.2s
✓	Secure Projection	2/2	7.3s

basics

100% · 15 evals

Pass	Task	Checks	Run Time
✓	Empty Reducers	1/1	6.5s
✓	Basic Tables	1/1	5.6s
✓	Scheduled Table	1/1	6.4s
✓	Struct In Table	1/1	6.2s
✓	Insert	2/2	7.9s
✓	Update	3/3	7.7s
✓	Delete	3/3	6s
✓	Crud	4/4	6.9s
✓	Index Lookup	3/3	7.1s
✓	Init	4/4	7.8s
✓	Connect	1/1	7.1s
✓	Helper Function	3/3	6.8s
✓	Schedule At Time	1/1	8.8s
✓	Cancel Schedule	1/1	13.4s
✓	Lifecycle Player	1/1	5.6s

data modeling

100% · 6 evals

Pass	Task	Checks	Run Time
✓	Event Table	1/1	5.9s
✓	Optional Fields	1/1	6.4s
✓	Cascade Delete	1/1	6.1s
✓	Filter And Aggregate	1/1	7.5s
✓	Wo Table Join	1/1	7.1s
✓	Unique Constraint	1/1	5s

lifecycle

100% · 3 evals

Pass	Task	Checks	Run Time
✓	Scheduled Materialization	4/4	10.5s
✓	Connection Scoped Presence	4/4	7.6s
✓	Scheduled Private	3/3	14.6s

migrations

100% · 3 evals

Pass	Task	Checks	Run Time
✓	Automatic Migration	3/3	9.9s
✓	Incremental Migration	4/4	12s
✓	Hot Swap Compatibility	2/2	11.5s

procedures

100% · 8 evals

Pass	Task	Checks	Run Time
✓	Procedure Return	2/2	5.3s
✓	Http Fetch	2/2	6.6s
✓	Fetch And Store	3/3	6.8s
✓	Scheduled Procedure	3/3	8.3s
✓	Http Handler	2/2	5.4s
✓	Http Router	2/2	6s
✓	Idempotent Webhook	3/3	11.2s
✓	External Upload Flow	4/4	11s

queries

88% · 8 evals

Pass	Task	Checks	Run Time
✗	View Basic	0/1	7.6s
✓	View Per User	1/1	5.2s
✓	Range Query	1/1	8.5s
✓	Sort And Limit	1/1	7.6s
✓	Find First	1/1	7.6s
✓	Select Distinct	1/1	7s
✓	Count Without Collect	1/1	7.2s
✓	Multi Column Filter	1/1	8.7s

reducers

100% · 6 evals

Pass	Task	Checks	Run Time
✓	Atomic Idempotent Transfer	5/5	12.3s
✓	Nested Update	3/3	10.3s
✓	Nested Cascade Delete	11/11	12.2s
✓	Batched Delete	4/4	13.2s
✓	Deterministic Context	5/5	7.5s
✓	Reducer Result Table	2/2	5.2s

schema

100% · 10 evals

Pass	Task	Checks	Run Time
✓	Spacetime Product Type	3/3	7.6s
✓	Spacetime Sum Type	3/3	11.9s
✓	Elementary Columns	3/3	9.5s
✓	Product Type Columns	3/3	6.1s
✓	Sum Type Columns	3/3	8.6s
✓	Scheduled Columns	2/2	7s
✓	Constraints	3/3	8.6s
✓	Many To Many	5/5	9.7s
✓	Ecs	7/7	14.6s
✓	Multi Column Index	5/5	8.6s

tables

100% · 8 evals

Pass	Task	Checks	Run Time
✓	Normalized Collection	1/1	5.9s
✓	Heartbeat Isolation	1/1	5.7s
✓	Binary Storage	2/2	8.3s
✓	Normalized Schema	1/1	7.6s
✓	Denormalized Index	4/4	11.5s
✓	Autoinc Reference	2/2	7.1s
✓	Default Values	2/2	8.9s
✓	Special Types	1/1	7.2s

views

100% · 7 evals

Pass	Task	Checks	Run Time
✓	Hree Table Join	2/2	11.1s
✓	Semijoin Intersection	2/2	7.8s
✓	Imestamp Window	2/2	8.6s
✓	Index And Filter	2/2	7.4s
✓	Materialized Aggregate	2/2	20s
✓	Query Builder View	2/2	5.9s
✓	View Primary Key	2/2	7.4s