When should we use Event-Driven architecture?
- to reverse dependencies
- to split loads, eg. 10k invoice generation takes a few mins but 10k payments take hours, so we can use a queue to decouple the 2 processes
- eventual consistency (requires additional event storage to provide the capability of replay on the events' producer)
There are 2 patterns used in our systems.
Pattern 1: one producer → many consumers (SNS)
- Lambda is used if a processor (such as generating invoices) is required to handle the events
- An SQS is introduced if a service needs to know the events
- Emails are sent if a third party system (such as alerting) needs to be integrated
Pattern 2: one producer → one consumer (SQS)
- A DLQ can be easily configured with the original SQS.
- A simple lambda can be introduced to copy messages between queues for replaying failed messages.
- idempotency (retry-safe design) that each event should be safely executed multiple times without side effects
- event order insensitive: the system should not assume the events always come in order. It should be able to handle the events regardless of the order.
- the fully automated monitoring is in place
- the producer should not care about the process results (success or failure) from consumers