Task Queue
Pluggable task queue supporting Trigger.dev and BullMQ — auto-detected via environment variables.
Bunship's AI generation pipeline uses a pluggable task queue with two backends. The system auto-detects which backend to use based on the environment variables you provide.
Backend Comparison
| Trigger.dev | BullMQ | |
|---|---|---|
| Tasks run on | Their cloud | Your server |
| Vercel Hobby (10s) | ✅ Works | ❌ No persistent process |
| Vercel Pro (300s) | ✅ Works | ❌ No persistent process |
| Self-hosted (VPS) | ✅ Works | ✅ Works |
| Requires Redis | No | Yes |
| Stale recovery | Scheduled task (cloud) | setInterval |
Auto-Detection
The adapter is selected by the first matching environment variable:
TRIGGER_SECRET_KEY -> Web/API runtime uses the Trigger.dev adapter (highest priority)
REDIS_URL -> Web/API or standalone API runtime uses BullMQ
(fallback) -> BullMQNo code changes needed. Set the right env var for your deployment.
Option A: Trigger.dev (Recommended for Vercel)
Best for Vercel deployments. Tasks run on Trigger.dev's cloud infrastructure.
Setup
- Create account at cloud.trigger.dev
- Create a project, note your Project ID and the Secret Key used by Web/API to enqueue runs
- Confirm the SDK and CLI versions match. The current repo pins them in
apps/ship-api/package.jsonat4.4.6:
@trigger.dev/sdk=4.4.6
@trigger.dev/build=4.4.6
trigger.dev CLI=4.4.6- Add dispatch variables to the Web/API container runtime:
TRIGGER_SECRET_KEY=sk_dev_your_key_here
TRIGGER_PROJECT_ID=your_project_idTRIGGER_SECRET_KEY lets the app enqueue runs on Trigger.dev. It is not a Docker build variable and it is not required in Trigger cloud task runtime.
Local Development
# Terminal 1: Start Trigger.dev local dev worker
cd apps/ship-api
bun run trigger:dev
# Terminal 2: Start your app
bun run devDeploy Tasks
Manual deploy:
cd apps/ship-api
bun run trigger:deployThis registers ai-generate and the ai-stale-recovery schedule on Trigger.dev.
If your local shell has OTEL_* environment variables (e.g. from Shelltime or other monitoring tools), deploy will fail with cannot merge resource due to conflicting Schema URL. Either clear these variables before deploying or use the CI workflow below.
CI/CD Deploy (GitHub Actions)
A workflow at .github/workflows/deploy-trigger.yml automatically deploys tasks when you push changes to main.
Triggers:
- Push to
mainthat modifies Trigger task directories,apps/ship-api/package.json,trigger.config.ts, or the workflow - Manual dispatch from the Actions tab
Required GitHub Secrets / Variables:
| Secret | Description |
|---|---|
TRIGGER_ACCESS_TOKEN | Personal Access Token (tr_pat_ prefix). Generate at cloud.trigger.dev/account/tokens |
TRIGGER_PROJECT_ID | Your Trigger.dev project ref (e.g. proj_xxx). Optional for the main repo if the default in trigger.config.ts is correct |
For forked repos: Fork users set their own TRIGGER_ACCESS_TOKEN and TRIGGER_PROJECT_ID in their repo's Settings → Secrets. The workflow deploys to their own Trigger.dev project — no conflict with the upstream repo.
Keep Trigger-Related Env Separate
| Scenario | Variables | Where they live |
|---|---|---|
| Deploy tasks | TRIGGER_ACCESS_TOKEN, TRIGGER_PROJECT_ID | GitHub Secrets / Variables, consumed by .github/workflows/deploy-trigger.yml |
| Web/API dispatch | TRIGGER_SECRET_KEY, optional TRIGGER_PROJECT_ID | Web/API container runtime env |
| Trigger cloud task execution | DATABASE_URL, S3_*, PUBLIC_S3_URL_BASE, provider API keys, AI tuning vars | Trigger.dev project Environment Variables, or synced by trigger.config.ts syncEnvVars |
Production Env Vars (Trigger.dev Cloud Task Runtime)
After deployment, tasks run on Trigger.dev cloud. Configure these in Trigger.dev project Environment Variables for prod:
| Variable | Required | Purpose |
|---|---|---|
DATABASE_URL | Yes | Read/write task, billing, and recovery data |
S3_ENDPOINT | Yes | Object storage endpoint |
S3_REGION | Yes | Object storage region |
S3_ACCESS_KEY | Yes | Object storage key |
S3_SECRET_KEY | Yes | Object storage secret |
S3_BUCKET | Yes | Output bucket |
PUBLIC_S3_URL_BASE | Yes | Public output URL base; legacy NEXT_PUBLIC_S3_URL_BASE still works |
SITE_URL | Recommended | Used when absolute links are generated |
PUBLIC_AUTH_PROVIDER | Recommended | Keeps auth provider selection consistent if auth code is imported |
AUTH_SECRET / BETTER_AUTH_SECRET | Recommended for Better Auth | May be needed if auth-related code is imported by tasks |
Provider-specific vars (only for enabled providers):
| Variable | Needed when |
|---|---|
FAL_API_KEY | Using fal provider |
REPLICATE_API_TOKEN | Using replicate provider |
KIE_API_KEY | Using kie provider |
KIE_API_BASE_URL | Optional override for kie API base URL |
OPENAI_API_KEY | CMS/admin AI copilot |
OPENAI_API_BASE | Optional OpenAI-compatible base URL |
OPENAI_API_MODEL | Optional default model override |
Optional tuning vars:
| Variable | Default | Purpose |
|---|---|---|
AI_TASK_RETRY_BASE_DELAY_MS | 2000 | Base retry delay |
AI_TASK_RETRY_MAX_DELAY_MS | 60000 | Maximum retry delay |
AI_TRIGGER_RUN_GUARD_TIMEOUT_MS | 270000 | Trigger task execution guard timeout |
AI_TRIGGER_RELEASE_TIMEOUT_MS | 2000 | Timeout for releasing concurrency slots |
AI_OUTBOX_WORKER_ENABLED | 1 | Set 0 to disable the outbox worker |
AI_TASK_HEARTBEAT_AUDIT | 1 | Set 0 to disable heartbeat audit |
Option B: BullMQ
Best for self-hosted deployments with a persistent server (Railway, Fly.io, VPS).
Requirements
- Redis instance (Upstash, Railway Redis, or self-hosted)
- Persistent process (not serverless)
Setup
- Ensure
REDIS_URLis set and noTRIGGER_SECRET_KEYis present:
REDIS_URL=redis://your-redis-url:6379- Deploy
ship-apias a standalone server:
cd apps/ship-api
bun run src/index.tsThe BullMQ worker starts automatically with admission control, retries, and stale-task scanning.
Stale Task Recovery
Regardless of backend, Bunship includes a safety net that automatically fails stuck tasks and refunds credits.
| Backend | Recovery Mechanism |
|---|---|
| Trigger.dev | ai-stale-recovery scheduled task (every 5 min, cloud) |
| BullMQ | setInterval in persistent worker process |
Code Map
apps/ship-api/src/services/ai/queue/
├── types.ts # TaskQueueAdapter interface
├── processor.ts # Shared execution logic
├── adapter-bullmq.ts # BullMQ implementation
├── adapter-trigger.ts # Trigger.dev implementation
└── index.ts # Auto-detection + adapter export