This post is part of a series on NServiceBus. In the previous articles, we looked at fundamental concepts for messaging and how to get started with NServiceBus. This spooky season, we’ll shine a light on how to solve some of the terrifying challenges of message delivery using the Outbox Pattern, helping you fight off the zombies and ghosts that can haunt your distributed systems. 🎃
The Monsters in Messaging 😈
In the world of distributed systems, we often encounter two terrifying monsters: Zombies and Ghosts. 😱
Unfortunately this Halloween night, monsters have made their way into our messaging systems, causing chaos and confusion! Our distributed systems are under attack. Let’s take a closer look at our OrderService which has been plagued by these frightful foes.
public class OrderService
{
private readonly IMessageSession _messageSession;
private readonly ApplicationDbContext _dbContext;
public OrderService(IMessageSession messageSession, ApplicationDbContext dbContext)
{
_messageSession = messageSession;
_dbContext = dbContext;
}
public async Task CreateOrder(Order order)
{
await _dbContext.Orders.AddAsync(order);
var message = new OrderCreatedEvent { OrderId = order.Id };
await _messageSession.Send(message);
}
}Zombies 🧟♂
Zombie records can occur when our system successfully writes a new Order record to the database but fails to send the OrderCreatedEvent. This can happen due to network issues, service crashes, or other unforeseen errors. As a result, the system is left in an inconsistent state, with a new Order created but no notification sent to other services. These undead records linger in our database, causing confusion and potential errors down the line.
Ghosts 👻
In an attempt to fix this zombie problem, we might change the order of operations in our service code. However, if we publish the OrderCreatedEvent before committing the Order entity to the database, we risk creating ghost messages. These invisible messages are sent out to other services, wreaking havoc as they reference orders that don’t actually exist in the database. This can lead to failed processing, errors, and a hauntingly bad user experience (and nightmares for the developer tracking down the issues 💥).
Enter the Outbox Pattern 📬
The Outbox Pattern is a powerful technique to combat these messaging monsters. It ensures that domain data and messages are reliably persisted and processed, even in the face of failures. By storing outgoing messages in a durable outbox table within the same transaction as your business data, you can guarantee that either both the business data and the message are committed, or neither are.
Rewriting our OrderService with the Outbox Pattern
Our updated implementation of the OrderService now includes an OutboxEntry entity to store outgoing messages. When creating an order, we add both the Order and the corresponding OutboxEntry to the same transaction. This way, if the transaction commits successfully, both the order and the message are persisted. If it fails, neither is saved, preventing both zombies and ghosts without breaking a sweat 💪. Then, a separate process can read from the outbox table and send the messages reliably while acknowledging successful processing.
public class OutboxEntry
{
public int Id { get; set; }
public string Type { get; set; }
public string Data { get; set; }
public DateTime CreatedOn { get; set; }
public bool IsProcessed { get; set; }
public DateTime? ProcessedOn { get; set; }
}
public class OrderService
{
private readonly ApplicationDbContext _context;
public OrderService(ApplicationDbContext context)
{
_context = context;
}
public async Task CreateOrder(Order order)
{
using var transaction = await _context.Database.BeginTransactionAsync();
try
{
_context.Orders.Add(order);
var message = new OrderCreatedEvent { OrderId = order.Id };
var outboxEntry = new OutboxEntry
{
Type = nameof(OrderCreatedEvent),
Data = JsonSerializer.Serialize(message),
CreatedOn = DateTime.UtcNow,
IsProcessed = false
};
_context.OutboxEntries.Add(outboxEntry);
await transaction.CommitAsync();
}
catch
{
await transaction.RollbackAsync();
throw;
}
}
}NServiceBus Outbox Integration 🛠️
NServiceBus provides built-in support for the outbox pattern, making it easier to protect your system from messaging monster attacks. By enabling the outbox feature in your NServiceBus configuration, you can ensure that messages are stored and processed reliably. We can then use ITransactionalSession or IMessageHandlerContext to ensure that our database operations and message sending are part of the same transaction. ✅
Related Patterns and Practices 🛡️
The outbox pattern is often used alongside patterns like Unit of Work to ensure data consistency and reliability in distributed systems. Additionally, implementing idempotent message handlers can help mitigate the effects of duplicate messages, further strengthening your system against the perils of messaging monsters.
It’s also important to note here that the outbox pattern can be used for more than just sending messages. It can be used for most operations that integrate with some external dependency, such as sending emails with retries.
Dawning a New Era of Reliable Messaging 🌅
It’s now the morning after Halloween. We’ve deployed our outbox as defense, and our messaging monsters have been vanquished! 💨 By implementing the outbox pattern and leveraging NServiceBus’s capabilities, we’ve fortified our distributed systems against the threats of Zombies and Ghosts. With these strategies in place, we can focus on the challenges of our application’s domain rather than fighting the monsters of our messaging technology.
Happy Halloween and happy coding! 🎃👻🧟