Skip to main content
Version: 2.0.0

Call an AI model from a sidecar

Prerequisites

You need a module and a client connected to it, Node.js to run the sidecar, and an API key from the provider of the AI model you want to use. If you don't have a module yet, follow the quickstart for your language first.

There's no C++ version of this guide yet.

In this guide, you'll let players ask an AI model for a reply, and have a sidecar answer them. A sidecar is a separate program that runs next to your database: it connects to it like any client, picks up the requests players make, calls the model, and writes the replies back.

Compared with calling the model from your module, a sidecar can use your provider's official SDK, stream replies, wait as long as a reply takes, and keep your API key out of the database. The module only stores requests and replies, so the same pattern works for any slow or external work, not just AI.

How it works​

  • An ai_request table holds every request: who asked, the prompt, and the reply or error once it's done.
  • Players call a request_ai_reply reducer to add a request, and read their own requests through a my_ai_requests view.
  • The sidecar subscribes to a pending_ai_requests view, calls the model for each request, then calls a complete_ai_request reducer with the reply.
  • An ai_worker table lists the identities allowed to read the queue and complete requests, so only your sidecar can answer.

Define the tables​

Add the ai_request and ai_worker tables. Both are private: players and the sidecar read them only through the views you'll add below. The done column has an index so the sidecar's view can find pending requests without reading every request.

import { schema, table, t, SenderError } from 'spacetimedb/server';

const aiRequest = table(
  { name: 'ai_request' },
  {
    id: t.u64().primaryKey().autoInc(),
    requester: t.identity().index('btree'),
    prompt: t.string(),
    text: t.string(),
    error: t.string(),
    done: t.bool().index('btree'),
    createdAt: t.timestamp(),
  }
);

const aiWorker = table(
  { name: 'ai_worker' },
  {
    identity: t.identity().primaryKey(),
  }
);

const spacetimedb = schema({ aiRequest, aiWorker });
export default spacetimedb;

Let players request a reply​

Add a request_ai_reply reducer that players call with their prompt. It rejects empty or overly long prompts, and players who already have a request waiting, so one player can't fill the queue.

Then add a my_ai_requests view that returns the caller's own requests. Players subscribe to it to get their replies, and they never see anyone else's prompts.

const MAX_PROMPT_LENGTH = 2000;

export const requestAiReply = spacetimedb.reducer({ prompt: t.string() }, (ctx, { prompt }) => {
  if (prompt.length === 0 || prompt.length > MAX_PROMPT_LENGTH) {
    throw new SenderError(`The prompt must be 1 to ${MAX_PROMPT_LENGTH} characters long`);
  }
  // Each player can have one request waiting at a time.
  for (const request of ctx.db.aiRequest.requester.filter(ctx.sender)) {
    if (!request.done) {
      throw new SenderError('You already have a request waiting for a reply');
    }
  }
  ctx.db.aiRequest.insert({
    id: 0n,
    requester: ctx.sender,
    prompt,
    text: '',
    error: '',
    done: false,
    createdAt: ctx.timestamp,
  });
});

export const myAiRequests = spacetimedb.view(
  { name: 'my_ai_requests', public: true },
  t.array(aiRequest.rowType),
  ctx => Array.from(ctx.db.aiRequest.requester.filter(ctx.sender))
);

The view has a primary key, so when a request changes, clients receive an update to the existing row instead of a deletion and a new row.

Let the sidecar answer​

Add a pending_ai_requests view that returns the requests waiting for a reply, and a complete_ai_request reducer that stores a reply. Both check that the caller is listed in ai_worker: anyone else gets an empty queue and can't complete requests.

export const pendingAiRequests = spacetimedb.view(
  { name: 'pending_ai_requests', public: true },
  t.array(aiRequest.rowType),
  ctx => {
    // Only AI workers can see the queue.
    if (!ctx.db.aiWorker.identity.find(ctx.sender)) {
      return [];
    }
    return Array.from(ctx.db.aiRequest.done.filter(false));
  }
);

export const completeAiRequest = spacetimedb.reducer(
  { id: t.u64(), text: t.string(), error: t.string() },
  (ctx, { id, text, error }) => {
    if (!ctx.db.aiWorker.identity.find(ctx.sender)) {
      throw new SenderError('Only AI workers can complete requests');
    }
    const request = ctx.db.aiRequest.id.find(id);
    if (!request) {
      throw new SenderError('No such request');
    }
    if (request.done) {
      throw new SenderError('This request is already complete');
    }
    ctx.db.aiRequest.id.update({ ...request, text, error, done: true });
  }
);

Completed requests stay in ai_request, so players can read their past replies. Delete old ones from time to time, for example from a scheduled reducer, so the table doesn't grow forever.

Write the sidecar​

The sidecar is a Node.js program written in TypeScript. In a new directory next to your module, install the SpacetimeDB SDK, tsx to run TypeScript, and your provider's SDK, then generate the bindings for your module:

npm install spacetimedb tsx @anthropic-ai/sdk
spacetime generate --lang typescript --out-dir src/module_bindings --module-path ../spacetimedb

Create src/sidecar.ts. The sidecar connects to your database, saves its token so it keeps the same identity when it restarts, and subscribes to pending_ai_requests. For each pending request, it calls the model and then complete_ai_request. If the model call fails, it completes the request with an error instead, so the player isn't left waiting.

import { existsSync, readFileSync, writeFileSync } from 'node:fs';
import { DbConnection, tables } from './module_bindings';
import type { AiRequest } from './module_bindings/types';

// The sidecar saves its token, so it keeps the same identity when it restarts.
const TOKEN_FILE = 'sidecar-token.txt';

const conn = DbConnection.builder()
  .withUri('ws://localhost:3000')
  .withDatabaseName('my-game')
  .withToken(existsSync(TOKEN_FILE) ? readFileSync(TOKEN_FILE, 'utf8') : undefined)
  .onConnect((ctx, identity, token) => {
    writeFileSync(TOKEN_FILE, token);
    console.log(`Sidecar connected as ${identity.toHexString()}`);
    ctx.subscriptionBuilder().subscribe([tables.pendingAiRequests]);
  })
  .build();

// Requests the sidecar is working on, so each one is handled only once.
const inProgress = new Set<bigint>();

// Runs for every pending request, including those waiting when the sidecar starts.
conn.db.pendingAiRequests.onInsert((_ctx, request) => {
  void handleRequest(request);
});

async function handleRequest(request: AiRequest) {
  if (inProgress.has(request.id)) {
    return;
  }
  inProgress.add(request.id);
  try {
    const text = await callModel(request.prompt);
    conn.reducers.completeAiRequest({ id: request.id, text, error: '' }).catch(console.error);
  } catch (error) {
    const message = error instanceof Error ? error.message : String(error);
    conn.reducers.completeAiRequest({ id: request.id, text: '', error: message }).catch(console.error);
  } finally {
    inProgress.delete(request.id);
  }
}

Then add the callModel function for your provider at the end of the file. Each provider's SDK reads its API key from an environment variable, so the key never enters your database.

The model names below are examples. Providers release new models often, so check your provider's documentation for the model you want.

This uses the Anthropic TypeScript SDK. fallbacks asks Anthropic to retry on another model if the model declines the request; a request that is still declined comes back with a refusal stop reason.

import Anthropic from '@anthropic-ai/sdk';

// Reads the API key from the ANTHROPIC_API_KEY environment variable.
const anthropic = new Anthropic();

async function callModel(prompt: string): Promise<string> {
  const message = await anthropic.beta.messages.create({
    model: 'claude-opus-5-5',
    max_tokens: 4096,
    output_config: { effort: 'low' },
    betas: ['server-side-fallback-2026-07-01'],
    fallbacks: 'default',
    messages: [{ role: 'user', content: prompt }],
  });
  if (message.stop_reason === 'refusal') {
    throw new Error('The model declined this request');
  }
  return message.content.map(block => (block.type === 'text' ? block.text : '')).join('');
}

Run the sidecar​

Start the sidecar with your provider's API key:

ANTHROPIC_API_KEY='your-api-key' npx tsx src/sidecar.ts

The first time it runs, the sidecar prints its identity, for example Sidecar connected as c200…. Add that identity to ai_worker with SQL, prefixed with 0x. Only the database owner and collaborators can write to a table this way:

spacetime sql my-game "INSERT INTO ai_worker (identity) VALUES (0x{identity})"

where identity is the hexadecimal identity the sidecar printed. The sidecar receives the pending requests as soon as the row is added, and keeps the same identity on later runs because it reuses the token it saved in sidecar-token.txt. Keep that file private: anyone with the token can act as your sidecar.

Run a single sidecar per database. Two sidecars would both answer the same requests.

Get replies on the client​

On the client, call request_ai_reply with the prompt, and subscribe to my_ai_requests. When the sidecar completes a request, the client receives an update to its row, with the reply in text, or the reason it failed in error.

import { DbConnection, tables } from './module_bindings';
import type { AiRequest } from './module_bindings/types';

const conn = DbConnection.builder()
  .withUri('ws://localhost:3000')
  .withDatabaseName('my-game')
  .onConnect(ctx => {
    ctx.subscriptionBuilder().subscribe([tables.myAiRequests]);
  })
  .build();

// Call this when the player sends a message.
function askModel(prompt: string) {
  conn.reducers.requestAiReply({ prompt }).catch(console.error);
}

// A request's row is updated when the sidecar completes it.
conn.db.myAiRequests.onUpdate((_ctx, _oldRequest, request) => showReply(request));

function showReply(request: AiRequest) {
  if (!request.done) {
    return;
  }
  if (request.error) {
    console.error('The AI request failed:', request.error);
  } else {
    console.log(request.text);
  }
}

Stream the reply​

To show the reply while the model is still writing it, have the sidecar store the text so far as it arrives. Add an update_ai_request_text reducer that only AI workers can call:

export const updateAiRequestText = spacetimedb.reducer(
  { id: t.u64(), text: t.string() },
  (ctx, { id, text }) => {
    if (!ctx.db.aiWorker.identity.find(ctx.sender)) {
      throw new SenderError('Only AI workers can update requests');
    }
    const request = ctx.db.aiRequest.id.find(id);
    if (!request || request.done) {
      throw new SenderError('No such pending request');
    }
    ctx.db.aiRequest.id.update({ ...request, text });
  }
);

In the sidecar, give callModel an onText callback that receives the text generated so far, and use your provider's streaming API:

async function callModel(prompt: string, onText: (textSoFar: string) => void): Promise<string> {
  const stream = anthropic.beta.messages.stream({
    model: 'claude-opus-5-5',
    max_tokens: 4096,
    output_config: { effort: 'low' },
    betas: ['server-side-fallback-2026-07-01'],
    fallbacks: 'default',
    messages: [{ role: 'user', content: prompt }],
  });
  let text = '';
  for await (const event of stream) {
    if (event.type === 'content_block_delta' && event.delta.type === 'text_delta') {
      text += event.delta.text;
      onText(text);
    }
  }
  const message = await stream.finalMessage();
  if (message.stop_reason === 'refusal') {
    throw new Error('The model declined this request');
  }
  return text;
}

Then, in handleRequest, pass a callback that sends the text to the database. Every call to update_ai_request_text is a transaction that updates the player's view, so send at most a few updates per second rather than one per word:

let lastUpdate = 0;
const text = await callModel(request.prompt, textSoFar => {
  // Send at most 4 updates per second.
  if (Date.now() - lastUpdate >= 250) {
    lastUpdate = Date.now();
    conn.reducers.updateAiRequestText({ id: request.id, text: textSoFar }).catch(console.error);
  }
});

On the client, show request.text in the update callback even while done is false: it holds the reply so far.

What's next?​

You now have a sidecar that answers players' AI requests:

  • Players add requests with request_ai_reply and read their replies through the my_ai_requests view.
  • The sidecar reads the queue through the pending_ai_requests view, calls the model with your provider's SDK, and stores replies with complete_ai_request.
  • The ai_worker table makes sure only your sidecar can read the queue and answer requests, and your API key stays with the sidecar.

To learn more about the features used here, see Views, Reducers and the TypeScript client SDK.

To call the model from your module instead, without running a separate program, see Call an AI model from your module.