Rate Limiting: Algorithms, Response Codes, and Fair

Rate limiting caps how frequently a client can call your API within a window.

Rate Limiting: Protecting Your API From Your Best and Worst Customers

Rate limiting is the practice of capping how many requests a client can make to your API within a time window. It exists to protect three things: your infrastructure from being overwhelmed, other customers from noisy-neighbor effects, and your paid tiers from being trivially bypassed. Every serious public API — Stripe, GitHub, Twilio, Slack — implements rate limits. The interesting design decisions are about which algorithm, what the limits should be, and how to communicate them to clients.

The main algorithms

(1) Fixed window — count requests per calendar minute; simplest but allows 2x burst at window boundaries. (2) Sliding window — smooths the boundary problem at slightly higher storage cost. (3) Token bucket — each client has a bucket of tokens that refills at a steady rate; requests consume tokens; supports bursts up to bucket size. Widely used because it matches human intuition. (4) Leaky bucket — requests queue and drain at a fixed rate; smooths out bursty clients but adds latency. Token bucket is the default choice for most APIs; sliding window is best when strict fairness matters more than burst tolerance.

Response codes and headers

Return 429 Too Many Requests when a client exceeds their limit. Include headers so clients can back off intelligently: X-RateLimit-Limit (their cap), X-RateLimit-Remaining (how many left in this window), X-RateLimit-Reset (when the window resets, as Unix timestamp), and Retry-After (seconds to wait, standardized). Clients that respect Retry-After are well-behaved; clients that ignore it and retry immediately are the ones that need aggressive limits. Log rate-limit violations by client — repeat offenders often indicate integration bugs worth reaching out about.

Where to limit

Layered defense. (a) Edge — CDN or load balancer, coarse-grained IP-based limits to filter obvious abuse before it hits your app. (b) Gateway — per-API-key limits enforced by API gateway or reverse proxy; this is where most fair-use limits live. (c) Application — fine-grained business-logic limits (e.g., 'no more than 10 password resets per email per hour'). Limits at only one layer leave gaps; limits at every layer with clear separation of concerns produces defense in depth without duplication.

Setting the numbers

Start with limits well above legitimate usage — measure the 99th percentile of current customer behavior, set limits at 2-5x that. Tighten later based on data. Common tiers: unauthenticated (10-60 req/min per IP), authenticated free (60-100 req/min per key), paid (1000+ req/min, or per-plan quotas). Publish limits in your API docs; hidden limits generate support tickets. Provide a mechanism for legitimate users to request higher limits; the request itself is useful signal.

Multi-dimensional limits

A single request-per-minute limit is often too coarse. Real APIs layer: per-endpoint limits (heavy endpoints like search get lower limits than lightweight ones), per-resource limits (bulk operations have their own quota), concurrent-request limits (max in-flight, not just rate), and compute-cost limits (GraphQL uses query complexity). GitHub's REST API uses points-based quotas; Shopify uses a leaky bucket with points per query. The added complexity is worth it once your API has diverse workloads.

Frequently asked questions

Redis for rate limiting or in-memory?
Redis for anything running on more than one server — in-memory limits are per-instance and easily bypassed by round-robin across replicas. Redis atomic INCR with EXPIRE is the canonical implementation; sub-millisecond overhead per request.
Should we rate-limit our own frontend?
Yes, and with the same infrastructure. Session-bound limits on the same endpoints your public API uses catch runaway retry loops in your own client code — a genuinely useful safety net. Excluding your own frontend from limits is a common source of self-inflicted outages.
How do we handle legitimate spikes (Black Friday, launch day)?
Anticipated spikes get temporary limit uplifts for specific customers, requested in advance. Unanticipated legitimate spikes are why you have monitoring and on-call — a customer suddenly hitting limits is either a legitimate business event or an integration bug, and both deserve human attention rather than silent rejection.

Related fundraising guides (40)

Investor directory · Fundraising library · Articles A–Z · Company funding database