What is this?
effect-mq is an Effect-native background job and message queue library: schema-first job definitions, a storage-agnostic queue core, a worker runtime, and stores for Postgres (via Drizzle) and Redis.
effect-mq is an Effect-native background job and message queue library: schema-first job definitions, a storage-agnostic queue core, a worker runtime, and stores for Postgres (via Drizzle) and Redis.Is this a library I import, or a separate service I run?It is a TypeScript library split into tree-shakeable modules (core, drizzle-postgres store, redis store, testing kit) that you wire into your own Effect application; there is no standalone server binary.
It is a TypeScript library split into tree-shakeable modules (core, drizzle-postgres store, redis store, testing kit) that you wire into your own Effect application; there is no standalone server binary.How do I wire it into my project?Install effect-mq plus the store module you need, re-export the provided drizzle schema factories so drizzle-kit manages the job tables' migrations, then define jobs and run the worker runtime against that store.
Install effect-mq plus the store module you need, re-export the provided drizzle schema factories so drizzle-kit manages the job tables' migrations, then define jobs and run the worker runtime against that store.Do I need Effect and a Postgres or Redis instance to use this?Yes, it's built on Effect v4 and needs at least one backing store — Postgres (claims via FOR UPDATE SKIP LOCKED and LISTEN/NOTIFY) or Redis (atomic Lua scripts with pub/sub) — plus an in-memory driver for testing or disposable jobs.
Yes, it's built on Effect v4 and needs at least one backing store — Postgres (claims via FOR UPDATE SKIP LOCKED and LISTEN/NOTIFY) or Redis (atomic Lua scripts with pub/sub) — plus an in-memory driver for testing or disposable jobs.What happens if a worker crashes or a job stalls mid-run?Delivery is at-least-once with token-guarded locks, heartbeat renewal, and stalled recovery, and every attempt is persisted to a run ledger in the store rather than kept in worker memory, so retries survive crashes.
Delivery is at-least-once with token-guarded locks, heartbeat renewal, and stalled recovery, and every attempt is persisted to a run ledger in the store rather than kept in worker memory, so retries survive crashes.How do repeatable cron jobs avoid double-firing across multiple workers?Each occurrence is claimed and enqueued in one atomic store operation, guaranteeing exactly one job per occurrence no matter how many workers are running, regardless of retention settings.
Each occurrence is claimed and enqueued in one atomic store operation, guaranteeing exactly one job per occurrence no matter how many workers are running, regardless of retention settings.What deduplication options does it offer?It supports pending-dedup, throttle, debounce, and replace-while-delayed, all using a separate dedup key so the actual job id is never rewritten.
It supports pending-dedup, throttle, debounce, and replace-while-delayed, all using a separate dedup key so the actual job id is never rewritten.Can I send different jobs to different stores?Yes, jobs can be bound to named stores — for example Postgres for business-critical runs and memory or Redis for disposable ones — and this routing is enforced at compile time.
Yes, jobs can be bound to named stores — for example Postgres for business-critical runs and memory or Redis for disposable ones — and this routing is enforced at compile time.Is there a built-in dashboard UI?No UI is bundled, but effect-mq exposes a dashboard data layer: listing with metadata filters, keyset pagination, polling, retry, and per-job and per-store retention, which you can build a dashboard on top of.
No UI is bundled, but effect-mq exposes a dashboard data layer: listing with metadata filters, keyset pagination, polling, retry, and per-job and per-store retention, which you can build a dashboard on top of.What are similar projects?bull, rq, tork, carno.js, react-starter-kit, rankme.fast, dockhand, asynq
bull, rq, tork, carno.js, react-starter-kit, rankme.fast, dockhand, asynqWhich topics is it in?job-queue, task-queue, background-jobs, drizzle-orm, message-queue, cron, scheduler, bun, redis, sql, ci, observability
job-queue, task-queue, background-jobs, drizzle-orm, message-queue, cron, scheduler, bun, redis, sql, ci, observability