Design a notification system

Think of it like this

Analogy

A post office sorting room. Letters arrive from many senders. Clerks check each recipient's forwarding preferences (“email me instead”, “no junk mail”, “not on Sundays”), sort the mail into bins for each delivery method, and couriers for each method work through their own bins. Urgent registered mail jumps the queue. If a courier can't deliver, the letter goes to a “return to sender” shelf rather than being tried forever.

The problem it solves

Every part of a product wants to notify users: “your order shipped”, “someone commented”, “your password was changed”, “weekly digest”. If each team calls push, SMS and email providers directly, you get inconsistent preferences, duplicate messages, users spammed at night, and outages when a provider slows down. Security messages must arrive in seconds; marketing can wait.

A real example

Example

“Your order has shipped”

The warehouse marks your parcel as shipped and emits an event. The notification service picks it up and checks your preferences: push yes, SMS no, email yes. It's 23:30 where you live and you've set quiet hours, so the push is held until 08:00. The email is transactional, so it goes now.

Each message is rendered from a template in your language and placed on that channel's queue. Push workers send through the phone platforms' push services; email workers send through an email provider. Both respect the providers' rate limits and retry with backoff when a provider is busy.

Every message carries the event ID. If a worker crashes after sending but before confirming, the retry sees that the ID was already sent and stops. You get one email, not two.

How it works

1. Requirements

Functional: any service can request a notification for a user; channels are push, SMS, email and in-app; templates and localization; user preferences and opt-outs; scheduling; delivery status.

Non-functional: transactional notifications delivered within seconds; no duplicates; no loss; handle bursts of millions; respect provider rate limits.

2. Design

Architecture

Services publish notification requests; the notification service applies preferences and templates, then routes to one queue per channel.

Producers publish a request: user ID, notification type, data, priority, and an idempotency key (for example order-42-shipped).
The notification service de-duplicates by key, loads the user's preferences (channels, opt-outs, quiet hours, time zone), renders the template in the user's language, and decides which channels to use.
It writes one message per channel onto that channel's queue, split by priority: transactional queues are separate from bulk.
Channel workers call providers (APNs and Firebase Cloud Messaging for push, an SMS gateway, an email service) with provider rate limits and retries.
Delivery results (sent, failed, bounced, opened) are recorded and fed back, for example to remove invalid device tokens.

3. Reliability

At-least-once + idempotency: queues may redeliver, so each worker checks a “sent” record for the notification ID before sending, and passes idempotency keys to providers where supported.
Retries with backoff on provider errors; after N attempts, a dead-letter queue and an alert.
Fallback channels: if push fails for an important alert, try SMS.
Provider outages: isolated per channel queue; backlog drains when the provider recovers.

4. Respecting users

Preferences per notification type and channel, checked on every send, with one-click unsubscribe for email.
Quiet hours and time-zone-aware scheduling for non-urgent messages.
Per-user rate limits and digests: “12 people liked your post” instead of 12 pushes.
Transactional and security messages bypass marketing opt-outs (but not legal requirements).

5. Scale

Bulk campaigns are expanded into per-user requests by batch jobs that feed the bulk queues at a controlled rate, so they never starve transactional traffic. Workers auto-scale on queue backlog (lesson 8).

Trade-offs

What you gain

One place for preferences, templates, de-duplication and auditing.
Channel isolation keeps one provider's problems contained.
Priority queues keep critical messages fast during big campaigns.

What it costs you

Another central service many teams depend on, so it must be very reliable.
Asynchronous delivery means status arrives later.
Many external providers, each with their own limits and failure modes.

When not to use it

A small app sending a few emails can call an email service directly from a background job.
Don't route latency-critical one-time codes through a slow bulk path; give them their own high-priority lane.

In the cloud

Push notificationsEvery major cloud sells this as a managed service, so you configure it rather than build it.

AWS2

Amazon SNS mobile push (opens vendor docs)
Sends push notifications to iOS, Android and other devices through their platform services.
AWS End User Messaging (opens vendor docs)
SMS, voice and push APIs (formerly Pinpoint's channels; Pinpoint itself ended Oct 30, 2026).

Azure2

Azure Notification Hubs (opens vendor docs)
Sends push notifications to any platform from one API, at scale.
Azure Communication Services (opens vendor docs)
APIs for email, SMS, chat and calling.

Google Cloud1

Firebase Cloud Messaging (opens vendor docs)
Free cross-platform push messaging for Android, iOS and web.

How you’d set it up

Services publish events; a notification service fans them out to push, SMS and email workers.

Event source
SNS topic
SQS queues per channel
SNS mobile push / SES / End User Messaging

Interview questions

Try answering out loud first, then open the model answer.

How do you prevent users from getting the same notification twice?

Model answer

Make the whole pipeline idempotent. Producers send an idempotency key that identifies the event (for example comment-881-reply-to-user-7); the notification service stores keys it has accepted and drops repeats within a window. Channel workers record each notification-channel pair as sent in the same step as sending, check it before sending, and pass idempotency keys to providers that support them. Retries then don't duplicate, even with at-least-once queues.

How would you send a campaign to 50 million users without hurting transactional notifications?

Model answer

Run the campaign as a batch job that expands the audience in chunks, applies preferences and opt-outs, and enqueues into a separate low-priority bulk queue at a controlled rate, scheduled by time zone. Transactional notifications use their own high-priority queues and workers, so they never wait behind bulk. Bulk workers respect provider rate limits and back off on throttling. Monitor bulk progress and allow pausing.

How do you handle invalid device tokens or bouncing email addresses?

Model answer

Process provider feedback: APNs and FCM report unregistered tokens, and email providers report bounces and complaints. Remove or mark those addresses in the user's device and contact records so you stop sending to them, which also protects your sender reputation. Keep a suppression list for hard bounces and complaints, and alert if failure rates spike, which may mean a configuration or certificate problem rather than bad addresses.

Quiz

Pick an answer to see whether it’s right and why.

0 of 6 answered

Key terms

Pub/sub: Publish/subscribe. A publisher sends a message to a topic and every subscriber gets its own copy.
Message queue: A buffer where producers leave messages and consumers pick them up later, so neither waits for the other.
Dead-letter queue: A side queue for messages that keep failing, so they can be inspected instead of retried forever.
Idempotency: Doing the same operation twice has the same effect as doing it once, which makes retries safe.
Rate limiting: Capping how many requests a client can make in a period of time.
Retry: Trying a failed request again, usually after a short wait.