Transactional Outbox Pattern: Reliability in Messaging

The Transactional Outbox Pattern ensures that a database update and a message publication happen atomically. It prevents the system from being in an inconsistent state where a database update succeeds but the message is never sent (or vice versa).

🏗️ The Problem

In microservices, you often save data (e.g., an Order) and then publish an event (e.g., OrderCreated). If the service crashes after saving the Order but before publishing the event, downstream services (like Shipping or Inventory) will never know the order exists.

🚀 The .NET Implementation

The most reliable way is to save the message to an Outbox Table within the same database transaction as your main business data.

1. The Outbox Message Entity

public class OutboxMessage
{
    public Guid Id { get; set; }
    public string Type { get; set; } // e.g., "OrderCreated"
    public string Content { get; set; } // Serialized JSON
    public DateTime CreatedAt { get; set; }
    public DateTime? ProcessedAt { get; set; } // Null if not sent
}

2. Saving Atomically (The Producer)

public async Task CreateOrderAsync(Order order)
{
    using var transaction = await _context.Database.BeginTransactionAsync();

    try {
        // 1. Save the main data
        _context.Orders.Add(order);

        // 2. Add to the Outbox (same transaction!)
        _context.OutboxMessages.Add(new OutboxMessage {
            Id = Guid.NewGuid(),
            Type = "OrderCreated",
            Content = JsonSerializer.Serialize(new { OrderId = order.Id }),
            CreatedAt = DateTime.UtcNow
        });

        // 3. Commit both!
        await _context.SaveChangesAsync();
        await transaction.CommitAsync();
    } catch {
        await transaction.RollbackAsync();
    }
}

3. The Background Publisher (The Worker)

A separate background service periodically polls the Outbox table and publishes the messages.

public class OutboxProcessor : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            var messages = await _db.OutboxMessages
                .Where(m => m.ProcessedAt == null)
                .ToListAsync();

            foreach (var message in messages)
            {
                // 1. Send message to RabbitMQ/Kafka
                await _bus.Publish(message.Content);

                // 2. Mark as processed
                message.ProcessedAt = DateTime.UtcNow;
                await _db.SaveChangesAsync();
            }

            await Task.Delay(1000, stoppingToken);
        }
    }
}

💡 Why use Outbox?

Atomicity: Guarantees that the message will eventually be sent if the data is saved.
Reliability: Messages are persisted. If the broker is down, the Worker will retry later.
Zero-Data-Loss: No events are lost even during service crashes or broker downtime.