The transactional outbox pattern makes a business change and its event record part of the same database transaction. A separate relay publishes committed outbox rows and retries failures. This closes the gap created when an application commits a database write and publishes a message as two independent operations.
This guide implements the pattern with PostgreSQL 18, Node.js 26 LTS, and the `pg` driver. The relay sends events to a small HTTP consumer so the example can run without assuming a specific broker's delivery guarantees. The same outbox can feed a broker through a publisher adapter or a change-data-capture (CDC) pipeline.
The dual-write problem#
Imagine an order service that saves an order and then publishes order.created:
await db.query("INSERT INTO orders ...");
await broker.publish("order.created", event);
There are two independent systems involved: the database and the event transport. No ordinary database transaction makes both operations commit atomically. That leaves two failure windows:
- Database first: the order commits, then the process crashes or the broker call fails. The order exists, but downstream services never hear about it.
- Event first: the event is accepted, then the database transaction rolls back or the process fails before commit. A consumer reacts to an order that does not exist.
Retries do not remove the ambiguity. If a publish call times out, the producer may not know whether the remote system accepted the event. Retrying can create a duplicate; not retrying can lose a notification.
A timeline makes the issue clear:
- The API receives
POST /orders. - The service writes the order and commits the database transaction.
- The service attempts to publish
order.created. - The process stops between steps 2 and 3, leaving committed state without an event.
Reversing steps 2 and 3 simply changes the failure into an event for an uncommitted change. The problem is not the order of the calls. It is that the calls do not share one atomic commit boundary.
How the transactional outbox works#
Add an outbox_events table to the same database as the business data. In one transaction, the application:
- Inserts or updates the business record.
- Inserts an event row describing that committed change.
- Commits both writes together.
A separate relay reads committed outbox rows, publishes them, and marks them as published. If the application transaction rolls back, neither the business change nor its outbox event remains. If the relay is unavailable, the event stays in the database until delivery can resume.
The outbox does not make the database and remote transport one distributed transaction. It moves the atomic boundary to the database and makes delivery a retryable process. That distinction is the core of the pattern.
Sequence, in words#
- Order API → PostgreSQL: begin a transaction; insert the order; insert
order.createdintooutbox_events; commit. - Polling relay → PostgreSQL: claim eligible, unpublished rows and commit the claim.
- Polling relay → Consumer endpoint: send each event with its stable event ID.
- Consumer → Consumer database: record the event ID and apply the consumer's local side effect in one transaction.
- Consumer → Relay: return success; the relay marks the outbox row as published.
- On failure: the relay records the error and schedules a retry. If it crashes after the consumer commits but before marking the outbox row, the same event can be delivered again; the consumer's idempotency check absorbs it.
Implementation: PostgreSQL and Node.js#
The example uses PostgreSQL 18 and Node.js 26 LTS, with pg for database access. It uses native fetch for HTTP delivery. Node.js 26 entered Long-Term Support in October 2026; use a supported LTS line in your own deployment. Node.js release information.
1. Create the tables#
Create schema.sql and apply it to the application database. The outbox row has a stable UUID, aggregate identity, event type, JSON payload, retry time, lease fields, and publication timestamp. The partial index keeps the relay's pending-row lookup focused on unpublished records.
CREATE TABLE orders (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
customer_id uuid NOT NULL,
total_cents integer NOT NULL CHECK (total_cents >= 0),
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE outbox_events (
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
aggregate_type text NOT NULL,
aggregate_id uuid NOT NULL,
event_type text NOT NULL,
payload jsonb NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(),
available_at timestamptz NOT NULL DEFAULT now(),
attempts integer NOT NULL DEFAULT 0,
locked_by text,
locked_until timestamptz,
published_at timestamptz,
last_error text
);
CREATE INDEX outbox_pending_idx
ON outbox_events (available_at, created_at, id)
WHERE published_at IS NULL;
CREATE TABLE consumer_inbox (
event_id uuid PRIMARY KEY,
received_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE received_order_events (
event_id uuid PRIMARY KEY REFERENCES consumer_inbox(event_id),
order_id uuid NOT NULL,
payload jsonb NOT NULL,
handled_at timestamptz NOT NULL DEFAULT now()
);
consumer_inbox and received_order_events represent the receiving service's database in this single-database demonstration. In a real deployment, the consumer owns its own inbox and business tables. Keep the event ID unique so a repeated delivery cannot apply the same local effect twice.
2. Install and configure the Node.js application#
Create package.json:
{
"type": "module",
"scripts": {
"demo": "node --env-file=.env src/demo.js",
"relay": "node --env-file=.env src/relay.js",
"consumer": "node --env-file=.env src/consumer.js"
},
"dependencies": {
"pg": "^8.16.3"
}
}
Install dependencies with npm install. Create .env and set the connection details; keep the file out of version control:
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/outbox_demo
CONSUMER_URL=http://localhost:4000/events
Create the database, then apply the schema with psql "$DATABASE_URL" -f schema.sql. The snippets below assume src/ exists and the schema has already been applied.
Create src/db.js:
import pg from "pg";
export const pool = new pg.Pool({
connectionString: process.env.DATABASE_URL,
max: 10,
connectionTimeoutMillis: 2_000,
idleTimeoutMillis: 30_000,
});
pool.on("error", (error) => {
console.error("Unexpected idle PostgreSQL client error", error);
});
3. Write the order and outbox event together#
The transaction must use one checked-out pg client for every statement. node-postgres explicitly warns against using pool.query() for a transaction because the pool can dispatch separate statements to different clients. node-postgres transaction documentation.
Create src/create-order.js:
import { pool } from "./db.js";
export async function createOrder({ customerId, totalCents }) {
const client = await pool.connect();
try {
await client.query("BEGIN");
const orderResult = await client.query(
`INSERT INTO orders (customer_id, total_cents)
VALUES ($1, $2)
RETURNING id, customer_id, total_cents, created_at`,
[customerId, totalCents],
);
const order = orderResult.rows[0];
const event = {
orderId: order.id,
customerId: order.customer_id,
totalCents: order.total_cents,
createdAt: order.created_at,
};
await client.query(
`INSERT INTO outbox_events
(aggregate_type, aggregate_id, event_type, payload)
VALUES ($1, $2, $3, $4::jsonb)`,
["order", order.id, "order.created", JSON.stringify(event)],
);
await client.query("COMMIT");
return order;
} catch (error) {
await client.query("ROLLBACK");
throw error;
} finally {
client.release();
}
}
The event payload is constructed from the row returned by the insert, not from untrusted request fields alone. The transaction ensures the order and event row either both commit or both roll back. PostgreSQL documents this all-or-nothing behavior for transaction blocks. PostgreSQL transaction tutorial.
Add src/demo.js to create one order and populate the outbox:
import { randomUUID } from "node:crypto";
import { createOrder } from "./create-order.js";
import { pool } from "./db.js";
try {
const order = await createOrder({
customerId: randomUUID(),
totalCents: 2_599,
});
console.log("Created order:", order.id);
} finally {
await pool.end();
}
Run the consumer in one terminal with npm run consumer, the relay in another with npm run relay, then run npm run demo. The generated customer UUID is illustrative because this minimal schema does not model a customers table.
4. Poll, claim, publish, and retry#
The relay must support more than one worker without letting them claim the same pending row at the same time. PostgreSQL's FOR UPDATE SKIP LOCKED is intended for queue-like tables where concurrent workers should skip rows already locked by another worker. PostgreSQL `SELECT` documentation.
The claim operation below uses a short transaction to lease a batch, then commits before making network calls. It also prevents a later unpublished event for the same aggregate from being claimed while an earlier one remains pending. This preserves per-aggregate progression in this polling example; it does not establish a global order across all aggregates.
Create src/relay.js:
import { randomUUID } from "node:crypto";
import { setTimeout as sleep } from "node:timers/promises";
import { pool } from "./db.js";
const workerId = randomUUID();
const endpoint = process.env.CONSUMER_URL ?? "http://localhost:4000/events";
const batchSize = 20;
const leaseSeconds = 30;
async function claimBatch() {
const client = await pool.connect();
try {
await client.query("BEGIN");
const result = await client.query(
`WITH candidates AS (
SELECT e.id
FROM outbox_events e
WHERE e.published_at IS NULL
AND e.available_at <= now()
AND (e.locked_until IS NULL OR e.locked_until < now())
AND NOT EXISTS (
SELECT 1
FROM outbox_events earlier
WHERE earlier.aggregate_type = e.aggregate_type
AND earlier.aggregate_id = e.aggregate_id
AND earlier.published_at IS NULL
AND (earlier.created_at, earlier.id) < (e.created_at, e.id)
)
ORDER BY e.available_at, e.created_at, e.id
LIMIT $1
FOR UPDATE SKIP LOCKED
)
UPDATE outbox_events e
SET locked_by = $2,
locked_until = now() + ($3 * interval '1 second'),
attempts = attempts + 1
FROM candidates c
WHERE e.id = c.id
RETURNING e.id, e.aggregate_type, e.aggregate_id,
e.event_type, e.payload, e.attempts`,
[batchSize, workerId, leaseSeconds],
);
await client.query("COMMIT");
return result.rows;
} catch (error) {
await client.query("ROLLBACK");
throw error;
} finally {
client.release();
}
}
async function markPublished(eventId) {
await pool.query(
`UPDATE outbox_events
SET published_at = now(), locked_by = NULL,
locked_until = NULL, last_error = NULL
WHERE id = $1 AND locked_by = $2`,
[eventId, workerId],
);
}
async function scheduleRetry(event, error) {
const delayMs = Math.min(300_000, 1_000 * 2 ** Math.min(event.attempts, 8));
await pool.query(
`UPDATE outbox_events
SET available_at = now() + ($2 * interval '1 millisecond'),
locked_by = NULL, locked_until = NULL, last_error = $3
WHERE id = $1 AND locked_by = $4`,
[event.id, delayMs, String(error).slice(0, 2_000), workerId],
);
}
async function deliver(event) {
const response = await fetch(endpoint, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({
eventId: event.id,
aggregateType: event.aggregate_type,
aggregateId: event.aggregate_id,
type: event.event_type,
payload: event.payload,
}),
signal: AbortSignal.timeout(5_000),
});
if (!response.ok) {
throw new Error(`Consumer returned HTTP ${response.status}`);
}
}
async function run() {
for (;;) {
const events = await claimBatch();
if (events.length === 0) {
await sleep(1_000);
continue;
}
for (const event of events) {
try {
await deliver(event);
await markPublished(event.id);
} catch (error) {
console.error("Outbox delivery failed", event.id, error);
await scheduleRetry(event, error);
}
}
}
}
run().catch((error) => {
console.error("Relay stopped", error);
process.exitCode = 1;
}).finally(async () => {
await pool.end();
});A lease lets another worker recover a row if the process dies before it releases the claim. Set the lease longer than the normal delivery timeout, and renew it if a delivery can legitimately take longer. The sample uses sequential delivery within a batch to keep the flow easy to follow; increase throughput deliberately, while retaining the per-aggregate ordering rule.
5. Make the consumer idempotent#
The relay marks a row as published only after receiving a successful HTTP response. Consider the failure window where the consumer commits its database transaction, but the relay crashes before updating published_at. The row becomes eligible again after its lease expires. The consumer must recognize the repeated event ID.
Create src/consumer.js:
import { createServer } from "node:http";
import { pool } from "./db.js";
const server = createServer(async (req, res) => {
if (req.method !== "POST" || req.url !== "/events") {
res.writeHead(404).end();
return;
}
try {
let raw = "";
for await (const chunk of req) {
raw += chunk;
if (raw.length > 1_000_000) throw new Error("Request too large");
}
const event = JSON.parse(raw);
if (!event.eventId || event.type !== "order.created") {
res.writeHead(400).end("Invalid event");
return;
}
const client = await pool.connect();
try {
await client.query("BEGIN");
const inserted = await client.query(
`INSERT INTO consumer_inbox (event_id)
VALUES ($1)
ON CONFLICT (event_id) DO NOTHING
RETURNING event_id`,
[event.eventId],
);
if (inserted.rowCount === 0) {
await client.query("COMMIT");
res.writeHead(200).end("Duplicate ignored");
return;
}
await client.query(
`INSERT INTO received_order_events (event_id, order_id, payload)
VALUES ($1, $2, $3::jsonb)`,
[event.eventId, event.payload.orderId, JSON.stringify(event.payload)],
);
await client.query("COMMIT");
res.writeHead(200).end("Accepted");
} catch (error) {
await client.query("ROLLBACK");
throw error;
} finally {
client.release();
}
} catch (error) {
console.error("Consumer request failed", error);
res.writeHead(500).end("Retry later");
}
});
server.listen(4000, () => console.log("Consumer listening on :4000"));
The inbox insert and local side effect share one consumer transaction. A duplicate event ID causes the insert to return no row, so the handler skips the side effect and responds successfully. PostgreSQL's ON CONFLICT DO NOTHING provides the uniqueness-based guard used here. PostgreSQL `INSERT` documentation.
The demo creates an order, the relay delivers its outbox event, and the consumer records it in received_order_events. The code is a runnable teaching example, not a tested deployment; add authentication, request validation, graceful shutdown, structured logging, and separate database credentials before production use.
Polling versus change data capture#
Polling is the approach above: a worker queries the outbox on an interval, claims rows, sends them, and updates their status.
- It is straightforward to operate with an ordinary database and application process.
- You control batch size, retry policy, leases, and cleanup.
- It adds periodic queries and creates a delay between commit and the next poll.
- The relay owns more delivery-state logic, including leases, retries, and poison-event handling.
Change data capture (CDC) reads database change records and streams inserts from the outbox to a downstream transport. Debezium's Outbox Event Router, for example, is designed to capture an outbox table and transform its rows into routed event messages. Its documented table model includes an event ID, aggregate type, aggregate ID, event type, and payload. Debezium Outbox Event Router.
CDC can avoid frequent polling queries and can integrate with an established log-based data pipeline. It also introduces connector deployment, database-log configuration and retention, schema mapping, and connector monitoring. Choose it when the organization already operates CDC infrastructure or needs its streaming characteristics. Polling is often the smaller first implementation. Neither option removes the need for idempotent consumers.
Delivery semantics, duplicates, and ordering#
Treat the relay as at-least-once: an event can be delivered more than once. This is a property of the retry workflow, not a promise that every transport behaves identically. The consumer must be safe when the same event ID arrives again.
The inbox pattern works when the inbox record and the consumer's database-side effect commit in the same transaction. If the side effect is an external API call, email, payment, or another database, the inbox transaction alone cannot make that remote effect atomic. Use the downstream system's idempotency mechanism where available, or create another local outbox for the next boundary.
Ordering needs an explicit design. created_at ordering is not a universal global-order guarantee, especially with multiple workers, retries, and independent aggregates. The sample blocks later pending events for the same aggregate behind the earliest unpublished row. For stricter domain ordering, store an aggregate sequence number allocated in the business transaction and have consumers detect gaps or stale versions. Do not let one failed event silently allow a later event for the same aggregate to overtake it unless the domain permits that behavior.
Operating the outbox#
A table that never drains becomes an operational problem. Treat the relay as a production subsystem:
- Retries: use bounded exponential backoff with jitter in production. Keep retryable transport errors distinct from permanent validation or schema errors.
- Poison events: after a defined retry policy, move the event to an inspectable dead-letter state or alert an operator. Do not delete it silently.
- Leases: make claims expire so crashed workers do not strand rows. Ensure the lease exceeds ordinary request timeouts or renew it.
- Cleanup: retain published rows long enough for investigation and replay policy, then delete or archive them in bounded batches. Never clean unpublished rows merely because they are old.
- Monitoring: track pending row count, oldest pending age, delivery attempts, retry rate, publish failures, and time from
created_attopublished_at. Alert on growing lag, not only process health. - Capacity: size polling batches and worker concurrency against database load and downstream capacity. Watch index growth and vacuum behavior as rows accumulate and are removed.
- Security: avoid placing secrets or unnecessary personal data in event payloads. Validate event schemas and authenticate the relay-to-consumer request.
The most useful first alert is usually the age of the oldest unpublished event. A relay process can be alive while the queue is not making progress.
When you do not need an outbox#
An outbox adds a table, relay, retries, cleanup, monitoring, and consumer idempotency work. It may be unnecessary when the event is best-effort telemetry, when no downstream system depends on it, or when a lost notification has no meaningful consequence and can be reconstructed cheaply.
It may also be unnecessary when the system already uses a documented atomic integration mechanism that covers the same boundary, or when the database itself is the only source of truth and consumers can safely query it. Do not add an outbox merely because an architecture diagram contains an event arrow. Use it when a committed state change must reliably produce a downstream signal.
What to do next#
The transactional outbox is one building block in a broader distributed-systems design. Use the Architecture Patterns Reference to compare it with related integration patterns, and the Node.js Production-Readiness Checklist to review operational concerns before deployment.
References#
- PostgreSQL 18 — Transactions
- PostgreSQL 18 — SELECT and row locking
- PostgreSQL 18 — INSERT and ON CONFLICT
- PostgreSQL 18 — UUID functions
- node-postgres — Transactions
- node-postgres — Pool API
- Debezium — Outbox Event Router
- Node.js — Releases
Software Engineer and Researcher, writing about AI, developer tooling and automation.
