What it is
A durable task-retry system that lives inside a SpacetimeDB server module, letting you schedule background work, automatically re-run it with increasing delays when it fails, and inspect pending tasks and attempt history through admin views.
Where it fits
- Transactional email delivery: Sending a receipt or notification after a purchase where a temporary downstream failure should not silently drop the message.
- External API calls: Calling a third-party service from a reducer where the service may be temporarily unavailable and the call needs to be retried a bounded number of times.
- Eventual consistency across systems: Propagating a state change to an external data store where the write might fail and must be retried without duplicating side effects.
- Admin-monitored background jobs: Any background work where operators need visibility into what is queued and what has failed, without building a custom audit log.
- User-initiated async operations: Handling requests where the caller should not wait for completion but the work must eventually succeed or be explicitly abandoned.
How it works
When a task is submitted, it is stored with its arguments, a maximum attempt count, and a base delay. The first attempt runs immediately. If the handler signals failure or throws an unexpected error, the task is rescheduled with a delay that doubles on each subsequent attempt. This continues until the handler succeeds or the attempt limit is reached.
Handlers are expected to be safe to run more than once for the same input. Each attempt runs inside the module's normal transaction model, so any writes the handler makes before a failure are committed as part of that attempt. If a task's arguments are fundamentally invalid, the handler can signal that explicitly, which permanently stops retries for that task without consuming the remaining attempt budget.
A rolling history of the most recent attempts is kept and exposed through admin-only views, alongside a list of tasks still waiting to run. Access to submit tasks directly or manage administrators is controlled by an identity-based admin list that is seeded when the module is first initialized.
