Stateless Services
Learn why stateless services scale and recover better, where state should live, and how sessions, files, jobs, and graceful shutdown affect production behavior.
What you will be able to do
A stateless service does not require a specific instance to remember user or workflow state between requests. Any healthy instance can handle the next request.
Why Statelessness Matters
Horizontal scaling, load balancing, rolling deploys, autoscaling, and failure recovery all become easier when service instances are replaceable. If one instance dies, another can continue because durable state lives outside the instance.
| State type | Unsafe location | Better location |
|---|---|---|
| User session | One API server memory | Signed token or shared session store/cache. |
| Uploaded file | Local disk on one container | Object storage. |
| Pending work | In-memory list | Queue. |
| Business data | Process memory | Database. |
| Config/secrets | Hardcoded in instance | Config service, environment, secret manager. |
Scenario: Assessment Submission
A learner submits a strict assessment. The system should not lose the attempt if an API container restarts after accepting the request. Local memory is the wrong place for submitted answers or scoring jobs.
| Step | State decision | Reason |
|---|---|---|
| Receive submission | Validate request and user auth in API | Short synchronous path. |
| Store submitted answers | Durable database/object record | Submission must survive crash. |
| Start scoring | Queue scoring job | Scoring may take time and can retry. |
| Track progress | Database status field | User can refresh and see state. |
| Notify completion | Async event/notification | Should not block durable score. |
Kubernetes and Runtime Behavior
Stateless design matters in Kubernetes and container platforms because instances can be moved, restarted, scaled down, or replaced. The service must handle lifecycle signals correctly.
| Concept | Meaning | Design implication |
|---|---|---|
| Readiness probe | Can this instance receive traffic? | Fail readiness during startup or graceful shutdown. |
| Liveness probe | Is the process stuck or dead? | Restart only when local process is unhealthy. |
| SIGTERM | Shutdown signal | Stop accepting new work, finish or hand off in-flight work. |
| Grace period | Time before forced kill | Keep requests short or checkpoint long work. |
| Horizontal Pod Autoscaler | Adds/removes instances | Requires externalized state and safe concurrency. |
Design principle
If losing one instance loses accepted user work, the service is not stateless enough for reliable horizontal scaling.
12-Factor Connection
12-factor style applications push config into environment, treat backing services as attached resources, keep processes stateless, and write logs as event streams. These ideas make services easier to deploy and scale.
| 12-factor idea | System design meaning |
|---|---|
| Config in environment | Deploy the same build to different environments. |
| Backing services | Database, cache, queue, and object storage are external resources. |
| Processes | Instances are disposable and stateless. |
| Logs | Write logs to stdout/event pipeline, not local files. |
| Concurrency | Scale by process/container count. |
Cloud Stateless Platforms
| Need | AWS | GCP | Azure |
|---|---|---|---|
| Container app | ECS/EKS | Cloud Run/GKE | Container Apps/AKS |
| Serverless function | Lambda | Cloud Functions | Azure Functions |
| Shared cache/session | ElastiCache | Memorystore | Azure Cache for Redis |
| Object storage | S3 | Cloud Storage | Blob Storage |
| Queue for work | SQS | Pub/Sub/Cloud Tasks | Service Bus/Storage Queues |
| Secrets | Secrets Manager/Parameter Store | Secret Manager | Key Vault |
Beginner Mistakes
- Storing sessions only in local memory and expecting load balancing to work.
- Writing uploaded files to container disk.
- Starting long background work inside an API request without durable tracking.
- Ignoring SIGTERM and dropping in-flight requests during deploys.
- Using liveness probes to check deep dependencies and causing restart loops.
Guided Practice
Practice task
A resume upload API stores files on local disk and starts parsing in memory. Redesign it so instances can scale horizontally and restart safely.
Sample Answer
| Problem | Fix |
|---|---|
| Local file disk | Upload directly or move immediately to object storage. |
| In-memory parse job | Create durable parse job in queue. |
| No status tracking | Store uploadId, file URL, parse status, and errors in database. |
| Duplicate retries | Use idempotency key or upload session ID. |
| Unsafe shutdown | On SIGTERM stop accepting new work and let queue retry unfinished jobs. |
Before You Continue
- You should know why stateless services scale better.
- You should identify unsafe local state.
- You should understand readiness, liveness, SIGTERM, and externalized state.
- You are ready to learn how caching and CDNs reduce repeated work.