Module 02 // ScalingLesson 07 / 3413 min read
Stateless vs stateful services
A stateless server keeps nothing about you between requests, so any copy can serve you; a stateful one remembers things locally, which makes it harder to scale and to replace.
Think of it like this
Analogy
A stateless service is like a fast-food counter: you can walk up to any cashier, because your order is all on the ticket you hand over. A stateful one is like a tailor who keeps your measurements in a notebook at their own shop. You have to go back to that tailor, and if their shop burns down, your measurements are gone.
The problem it solves
You put three servers behind a load balancer (lesson 6), and strange things start happening. Users get logged out at random. Shopping carts empty themselves. Uploaded files can't be found. The cause is always the same: a server kept something in its own memory or disk, and the next request landed on a different server.
Horizontal scaling, auto-scaling and painless deploys all depend on servers being interchangeable.
A real example
Example
Shopping carts that vanish during a deploy
An online shop keeps each user's cart in the memory of whichever server they first reached. The load balancer uses sticky sessions to keep sending them back there.
Every deploy restarts the servers, and every restart empties thousands of carts. During a sale, auto-scaling removes a server and its users' carts disappear with it. Support tickets pile up.
The fix is to make the app servers stateless. Carts move to Redis, keyed by a session ID stored in a cookie. Now any server can handle any request, sticky sessions can be turned off, and servers can be added, removed or restarted at any time without anyone noticing.
How it works
A stateless server handles each request using only what's in the request plus shared systems such as databases and caches. A stateful server keeps something locally that later requests depend on: sessions, uploaded files, in-progress work, WebSocket connections.
- Users to Load balancer
- Load balancer to App server A
- Load balancer to App server B
- App server A to Session store
- App server B to Session store
- App server A to Database
- App server B to Database
- App server B to Object storage
Where sessions can live
A session is what the server needs to remember between requests, at minimum who you are.
| Option | How it works | Good | Watch out |
|---|---|---|---|
| Server memory + sticky sessions | Load balancer pins each user to one server | No code change | Uneven load; sessions lost when a server dies or deploys |
| Shared session store | Session ID in a cookie; data in Redis or a database | Any server can serve anyone; easy to revoke | One more dependency, about 1 ms per lookup |
| Signed token (JWT) | User ID and expiry are inside a signed token the client sends | No lookup at all | Hard to revoke early; larger requests |
Most apps use a shared store for web sessions, or short-lived tokens plus a refresh token for APIs.
Other state to move out
- Uploaded files go to object storage (lesson 13), not the server's disk.
- Background work goes on a queue (lesson 17), not in a server's memory where a restart loses it.
- Scheduled jobs should run once, not once per server: use a scheduler service or a lock.
- Local caches are fine as long as losing them only costs speed, not correctness.
Some things must be stateful
Databases, caches, queues and WebSocket connection servers hold state by design. The goal isn't zero state. It's to keep state in a few places built for it, which you replicate and back up, and keep everything else disposable.
Trade-offs
What you gain
- Any server can handle any request, so load spreads evenly.
- Servers can be added, removed or replaced at any moment: auto-scaling and rolling deploys just work.
- A crashed server loses nothing important.
What it costs you
- Every request may need a lookup in a shared store, adding a little latency.
- The shared store becomes critical and must be highly available.
- Moving existing state out of a legacy app can be real work.
When not to use it
- Long-lived connections (WebSockets, game sessions) are inherently tied to one server. Accept that, and keep the durable part (messages, scores) in shared storage.
- For a single-server internal tool with no plans to scale, in-memory sessions are fine.
In the cloud
Shared session storageEvery major cloud sells this as a managed service, so you configure it rather than build it.
AWS2
- Amazon ElastiCache (opens vendor docs)
A fast shared store for session data with expiry (TTL) on each key.
- Amazon DynamoDB (opens vendor docs)
Durable key-value store; its TTL feature deletes expired sessions automatically.
Azure2
- Azure Managed Redis (opens vendor docs)
Shared in-memory session store.
- Azure Cosmos DB (opens vendor docs)
Durable session store with per-item time to live.
Google Cloud2
- Memorystore for Valkey (opens vendor docs)
Shared in-memory session store.
- Firestore (opens vendor docs)
Durable document store with TTL policies for expiring sessions.
Any server can handle any request because sessions live in a shared store, not in server memory.
- Application Load Balancer
- Stateless EC2 or ECS tasks
- ElastiCache sessions
Interview questions
Try answering out loud first, then open the model answer.
How do you handle user sessions across many web servers?
Model answer
Store a random session ID in a secure, HTTP-only cookie and keep the session data in a shared, replicated store such as Redis, with a TTL so idle sessions expire. Every server looks it up on each request. For APIs and mobile apps I'd use short-lived signed access tokens (minutes) plus a refresh token stored server-side, which gives lookup-free checks on most requests and a way to revoke access. I'd avoid sticky sessions except as a stopgap.
What's wrong with sticky sessions?
Model answer
They tie users to servers. Load can become uneven when some users are heavy. When a server dies or is replaced during a deploy, its users lose their sessions. Auto-scaling helps less because existing users stay on old servers. They're acceptable as a quick fix, or for genuinely stateful connections, but moving state out is the real solution.
How would you make a file-upload feature work with several servers?
Model answer
Don't write files to the server's disk. Have the client upload straight to object storage with a presigned URL, or stream through the server into object storage, then save the object key and metadata in the database. Any server can then serve the file, ideally through a CDN.
Quiz
Pick an answer to see whether it’s right and why.
0 of 6 answered
Key terms
- Stateless service
- A service that keeps no user data between requests, so any copy of it can answer any request.
- Stateful service
- A service that keeps data about users or work in its own memory or disk between requests.
- Session
- Data the server remembers about a logged-in user between requests, such as who they are and what's in their cart.
- Sticky session
- Sending every request from the same user to the same server, usually with a cookie. Also called session affinity.
- JWT
- JSON Web Token. A signed token that carries claims like user ID and expiry, which servers can check without a lookup.
- Horizontal scaling
- Adding more machines and splitting the work between them. Also called scaling out.
Progress is saved in this browser only.