The Transactional Outbox Pattern: Making Event Publishing Atomic

The Gap Between a Write and a Publish

In the last post, the order service does two things when an order comes in: it saves the order, then it publishes order.created to the exchange. That code is correct as far as it goes, but it glosses over something. Those are two separate systems, a database and a message broker, and nothing makes the two calls atomic.

Picture the failure: the order is saved, the commit succeeds, and then before the publish call goes out the process crashes, or the network to the broker drops for a second. The order now exists. The event never will. Nobody gets an error. Nothing retries. The order just sits there, quietly missing whatever was supposed to happen next.

The obvious fix, wrap both calls in a try/catch and retry the publish on failure, doesn't actually close the gap. It just moves it: now the retry logic has to survive the same process crash that caused the original problem, and it usually can't, because the information needed to retry (that this specific order still needs its event published) only ever lived in memory.

The Fix: Write the Event, Not Just the Order

The database is already good at making multiple writes atomic, that's what a transaction is for. The trick is to stop treating "save the order" and "publish the event" as two steps across two systems, and turn it into one step inside one system: save the order, and save a row describing the event, in the same transaction.

Nothing talks to RabbitMQ at this point. The order service writes to the database and nothing else. Either the order and its outbox row both commit, or neither does, because they're the same transaction. There is no window where one exists without the other.

A separate process, the relay, is the only thing that ever reads that row and actually publishes it. That's the whole idea. Everything else in this post is just the details of making the relay reliable.

The Outbox Table

The outbox is a plain collection sitting next to orders in the same tenant database. Each row is one event waiting to go out:

// models/outboxEvent.model.js
const outboxEventSchema = new Schema({
  routingKey: { type: String, required: true },   // 'order.created'
  payload: { type: Object, required: true },       // the event body
  createdAt: { type: Date, default: Date.now },
  publishedAt: { type: Date, default: null },       // null = not sent yet
});

export default (conn) =>
  conn.models.OutboxEvent || conn.model('OutboxEvent', outboxEventSchema);

Since orders live in a per-tenant database, the outbox row for a given order has to land in that same tenant's database, on the same connection, so the transaction actually covers both writes. That connection is exactly req.db, the one handed out by the connection manager. The session for the transaction comes from there:

// controllers/order.controller.js
export async function createOrder(req, res) {
  const session = await req.db.startSession();

  try {
    let order;

    await session.withTransaction(async () => {
      const Order = req.db.model('Order');
      const OutboxEvent = req.db.model('OutboxEvent');

      order = new Order({
        tenantId: req.tenantId,
        items: req.body.items,
        total: req.body.total,
      });
      await order.save({ session });

      const event = new OutboxEvent({
        routingKey: 'order.created',
        payload: { orderId: order._id, total: order.total },
      });
      await event.save({ session });
    });

    res.status(201).json({ success: true, orderId: order._id });
  } finally {
    session.endSession();
  }
}

One thing worth knowing if you haven't used MongoDB transactions before: startSession() and withTransaction() only work against a replica set, not a standalone instance. MongoDB Atlas is always a replica set, so this is a non-issue in production, but a local single-node mongod needs to be started as a one-node replica set for this to work in dev.

The Relay

The relay's whole job is to look for rows where publishedAt is still null, publish them, and mark them done. The simplest version just polls:

// outbox/relay.js
const POLL_INTERVAL_MS = 2000;
const BATCH_SIZE = 100;

async function relayLoop(db, channel) {
  setInterval(async () => {
    const OutboxEvent = db.model('OutboxEvent');

    const pending = await OutboxEvent.find({ publishedAt: null })
      .sort({ createdAt: 1 })
      .limit(BATCH_SIZE);

    for (const event of pending) {
      try {
        await publishEvent(channel, event.routingKey, event.payload, event._id.toString());
        event.publishedAt = new Date();
        await event.save();
      } catch (err) {
        // left unpublished on purpose, picked up again next poll
        console.error(`[outbox] failed to publish ${event._id}:`, err.message);
      }
    }
  }, POLL_INTERVAL_MS);
}

publishEvent here is the same confirm-channel publisher from the RabbitMQ post, unchanged. The outbox row's own id is passed through as the event id, so idempotency downstream still works exactly the way it already did.

That single db parameter is doing some quiet work, though. Since orders live in a database per tenant, there's no one database to poll, there are as many as there are tenants. Running a naive version of this loop against every tenant's database on a timer would mean opening a connection to every tenant just to ask "anything new?", which is the exact problem the lazy connection manager exists to avoid. The relay sidesteps this by riding on the same connection manager instead of managing its own: it only polls tenants that already have an active connection open for other reasons, and skips the rest. A tenant with no live traffic has no open connection, so it also has no polling happening against it, its outbox rows just wait until the next request wakes that connection back up.

Still At-Least-Once

The outbox closes the gap where an event could be lost. It doesn't make publishing exactly-once, and it isn't trying to. If the relay publishes an event, gets the broker's confirmation, and then crashes before event.save() writes publishedAt, that row is still null when the relay restarts. It publishes again.

That's fine, and it's fine for a specific reason: the failure mode changed from "event silently missing" to "event possibly delivered twice," and duplicates are exactly what the idempotent consumers from the RabbitMQ post already handle. A dropped event is invisible and unrecoverable. A duplicate is a no-op. The outbox doesn't need to solve duplicates because that problem was already solved on the other end.

Cleaning Up

Left alone, the outbox collection grows forever. Once a row has been published, it's just an audit trail, so a periodic job clears out anything old enough that nobody's going to need it for debugging:

// outbox/prune.js
async function pruneOutbox(db) {
  const OutboxEvent = db.model('OutboxEvent');
  const cutoff = new Date(Date.now() - 48 * 60 * 60 * 1000); // 48h

  await OutboxEvent.deleteMany({
    publishedAt: { $ne: null, $lt: cutoff },
  });
}

48 hours is a reasonable default, enough runway to debug a "did this event go out" question without the collection turning into an unbounded history. Rows that are still unpublished are never touched by this job, on purpose, they're the ones that still need the relay's attention.

Polling vs Change Streams

Polling every two seconds is simple and needs no extra infrastructure, but it means a new event can sit for up to two seconds before anyone even looks at it, and the relay is running a query against the collection on a timer even when there's nothing new.

MongoDB has a built-in alternative: change streams, which tail the replica set's oplog and push new documents to a listener as they're written, no polling interval involved.

// outbox/relay-changestream.js
async function watchOutbox(db, channel) {
  const OutboxEvent = db.model('OutboxEvent');

  const stream = OutboxEvent.watch([{ $match: { operationType: 'insert' } }], {
    fullDocument: 'updateLookup',
  });

  stream.on('change', async (change) => {
    const event = change.fullDocument;

    try {
      await publishEvent(channel, event.routingKey, event.payload, event._id.toString());
      await OutboxEvent.updateOne({ _id: event._id }, { publishedAt: new Date() });
    } catch (err) {
      console.error(`[outbox] failed to publish ${event._id}:`, err.message);
    }
  });
}

The trade is operational: a change stream needs to resume from where it left off after the relay restarts (MongoDB gives you a resume token for exactly this), or a burst of orders placed while the relay was down would be missed. A polling loop doesn't have that problem, it just rescans whatever is still unpublished on its next tick. For most workloads the two-second polling delay is not worth trading for that extra piece of resume-token bookkeeping. It only tends to matter once near-real-time delivery is an actual product requirement, not just a nice-to-have.

A Full Walkthrough, Revisited

Same order-placement example as before, corrected:

1. A customer checks out. The request reaches the order service.
2. The order service opens a transaction on the tenant's connection, saves the order, and saves an outbox row for order.created in that same transaction, then commits.
3. The order service responds to the customer immediately. Nothing about the relay or RabbitMQ has happened yet, and none of it can slow this response down.
4. Separately, the relay notices the new outbox row within one poll cycle and publishes it to platform.events, then marks the row published.
5. From here it's exactly the flow from the RabbitMQ post: the notification service consumes it, checks Redis for that event id, sends the confirmation email, retries through the dead letter queue if it fails.

The difference is step 2. If the process crashes the instant after that commit, the order and its outbox row both exist, or neither does. There's no longer a state where the order is real and the event is gone. The relay will get to it whenever it comes back.

Where Else This Pattern Applies

Order confirmations aren't special here. The outbox is the right tool anywhere a database write needs to reliably cause something else to happen afterward:

• Sending a welcome email when a new account row is created
• Re-indexing a product in search after its listing is edited
• Charging a saved payment method on a subscription's renewal date
• Invalidating a cache entry after a price or inventory change

The pattern doesn't even require RabbitMQ specifically. The relay just needs to do something reliably with an unpublished row, it could call a webhook, write directly into a search index, or hit any other downstream system. RabbitMQ is the convenient target here because it's already the backbone for everything else, but the outbox itself doesn't care what's on the other end.

The common thread to watch for: any time code reads "save this, then also do that," where the "also do that" step talks to a different system, there's a silent-gap risk in the making. The outbox is the general answer to that shape of problem, not a one-off fix for order events.

What This Trades Away

Quick Reference

Closing Thoughts

None of this replaces anything from the RabbitMQ post, it sits in front of it. The exchange, the retry queue, the dead letter queue, the idempotent consumers, all of that is unchanged. The outbox only fixes the one step that was never actually safe: the moment between saving something and telling the rest of the system it happened.

It's a small pattern. One table, one small relay process, a few lines of transaction code. What it removes is a failure mode that's invisible until the day someone asks why an order from three weeks ago never got a confirmation email, and there's no error log anywhere that explains it.

The write and the event it causes shouldn't be able to disagree about what happened.