REST APIs in Production: Pagination, HATEOAS, and the Patterns That Actually Matter (Part 3 of 3)

This is the last post in a three-part series on REST API design. The first post covered REST's constraints and HTTP semantics. The second covered resource design, versioning, and authentication. This one covers what separates a demo API from a production one.

Pagination 🔗

Always paginate collections that can grow without bound. Returning 50,000 records in one response is a denial-of-service problem waiting to happen.

Offset-Based Pagination 🔗

GET /orders?offset=0&limit=20
GET /orders?offset=20&limit=20
GET /orders?page=2&per_page=20

Simple to implement and query. Allows jumping to arbitrary pages.

Weakness: inconsistent results under concurrent writes. A record inserted between page 1 and page 2 requests can show up as a duplicate. If one is deleted, you may skip a record. Performance also degrades at high offsets in most databases.

Cursor-Based Pagination 🔗

GET /orders?limit=20
# Returns: { "data": [...], "cursor": "eyJpZCI6MjB9" }

GET /orders?after=eyJpZCI6MjB9&limit=20

The cursor encodes a position in the result set. It's typically the ID or sort key of the last item returned. Each page request is a stable query from a known position.

Strengths: consistent under concurrent writes, efficient at any depth, suits real-time feeds.

Weaknesses: cannot jump to arbitrary pages (only forward/backward), and clients must treat the cursor as opaque.

Keyset Pagination 🔗

A variant of cursor pagination that expresses the position as an explicit field value:

GET /orders?after_id=789&limit=20

Less opaque than encoded cursors, but ties pagination semantics to visible fields.

Response Envelope for Pagination 🔗

Always include pagination metadata:

{
  "data": [ ... ],
  "pagination": {
    "total": 1540,
    "limit": 20,
    "offset": 40,
    "has_next": true,
    "has_previous": true
  },
  "links": {
    "self":  "/orders?offset=40&limit=20",
    "next":  "/orders?offset=60&limit=20",
    "prev":  "/orders?offset=20&limit=20",
    "first": "/orders?offset=0&limit=20",
    "last":  "/orders?offset=1520&limit=20"
  }
}

The links object connects this to HATEOAS, more on that next.

HATEOAS 🔗

HATEOAS (Hypermedia as the Engine of Application State) is REST's most misunderstood, most often omitted constraint. According to Fielding himself, it's also the part that makes an API truly RESTful.

The idea: a client moves through an API using links in responses. It needs little upfront knowledge of URI structure. A human clicks through a website by following links. A REST client should do the same instead of hardcoding URI structure it was never given.

What It Looks Like 🔗

GET /orders/789

{
  "id": "789",
  "status": "pending",
  "total": 142.50,
  "customer": {
    "id": "42",
    "name": "Alice"
  },
  "_links": {
    "self":     { "href": "/orders/789" },
    "cancel":   { "href": "/orders/789/cancel", "method": "POST" },
    "customer": { "href": "/users/42" },
    "payment":  { "href": "/payments?orderId=789" }
  }
}

The server controls what actions are available. If the order is already cancelled, the cancel link disappears. The client doesn't need to know the cancellation rules. It just checks whether the link is present.

Media Types and HATEOAS 🔗

The _links pattern above is informal. There are standardized hypermedia formats:

• HAL (Hypertext Application Language): uses _links and _embedded, widely adopted
• JSON:API: a more opinionated full specification for request/response format
• Siren: includes actions with typed fields, more expressive for write operations
• Collection+JSON: designed specifically for collection resources

The Reality of HATEOAS Adoption 🔗

HATEOAS is theoretically correct and has genuine benefits for client decoupling. In practice, most production APIs don't implement it. Most clients don't use it even when offered. The reasons are pragmatic. Statically typed clients often prefer a well-documented contract over dynamic link discovery. Tooling for hypermedia clients also isn't as mature as tooling for OpenAPI-described APIs.

Know this so you decide deliberately, not by default. For public APIs with diverse clients and long lifespans, HATEOAS is worth the investment. For internal APIs between services you control, a well-maintained OpenAPI spec may be more practical.

Filtering, Sorting, and Searching 🔗

Filtering 🔗

Use query parameters to filter collections:

GET /orders?status=pending
GET /orders?status=pending&customerId=42
GET /products?minPrice=10&maxPrice=100&category=electronics

For complex filters, some APIs use a structured query language:

GET /products?filter=price:lt:100,category:eq:electronics

Or a JSON-encoded filter (though this can be awkward in URLs):

GET /products?filter={"price":{"lt":100},"category":"electronics"}

Be consistent and document your filter syntax explicitly.

Sorting 🔗

GET /orders?sort=createdAt
GET /orders?sort=createdAt:desc
GET /orders?sort=-createdAt          # minus prefix for descending
GET /orders?sort=status,createdAt:desc

Always document the default sort order, and make it stable. Include a unique field as a tiebreaker.

Searching 🔗

Full-text search usually warrants a dedicated endpoint or query parameter:

GET /products?q=wireless+headphones
GET /search?q=wireless+headphones&type=products

For sophisticated search requirements, dedicated search endpoints with POST bodies can be appropriate:

POST /products/search

{
  "query": "wireless headphones",
  "filters": { "inStock": true, "priceMax": 200 },
  "sort": { "field": "relevance", "order": "desc" },
  "pagination": { "limit": 20, "offset": 0 }
}

POST works for search when the query is too complex for a URL. It's pragmatic, but it sacrifices cacheability.

Sparse Fieldsets and Field Selection 🔗

Large resources with many fields are expensive to transfer. They often contain fields a given client doesn't need. Field selection lets clients request only what they need:

GET /users/42?fields=id,name,email
GET /orders?fields=id,status,total&include=customer.name

This is particularly important for mobile clients on limited bandwidth.

Content Negotiation 🔗

REST embraces HTTP's content negotiation mechanism. Clients can request specific response formats:

# Request JSON
curl -H "Accept: application/json" https://api.example.com/users/42

# Request XML
curl -H "Accept: application/xml" https://api.example.com/users/42

The server indicates what it returned:

Content-Type: application/json; charset=utf-8

If the server cannot produce the requested format, it returns 406 Not Acceptable.

For versioning via content type:

curl -H "Accept: application/vnd.myapi.v2+json" https://api.example.com/users/42

Rate Limiting 🔗

Every public API needs rate limiting. Return standard headers so clients can manage their behavior:

X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 742
X-RateLimit-Reset: 1735689600
Retry-After: 3600

When a client exceeds the limit, return 429 Too Many Requests with a Retry-After header.

Common rate limiting strategies:

• Fixed window: count requests in a fixed time window (simple, but allows burst at window boundary)
• Sliding window: rolling time window (smoother, more expensive)
• Token bucket: tokens replenish at a fixed rate, bursts allowed up to bucket size
• Leaky bucket: requests processed at a fixed rate, excess queued or dropped

Provide different rate limit tiers for different clients (authenticated vs. anonymous, free vs. paid).

Idempotency Keys 🔗

For non-idempotent operations like POST, idempotency keys let clients retry safely. There's no risk of duplicate side effects:

curl -X POST https://api.example.com/payments \
  -H "Idempotency-Key: 7f3a2b1c-4d5e-6f7g-8h9i-0j1k2l3m4n5o" \
  -H "Content-Type: application/json" \
  -d '{"amount": 9900, "currency": "USD"}'

The server caches the response for the idempotency key for a defined window. A retry with the same key returns the cached response without reprocessing the payment.

This pattern is essential for payment APIs and order creation. It matters for any operation with financial or irreversible consequences.

Conditional Requests and Optimistic Concurrency 🔗

HTTP provides mechanisms for conditional operations via ETags and timestamps.

ETags are version identifiers for a resource:

GET /orders/789

HTTP/1.1 200 OK
ETag: "abc123"

A client can make conditional requests using If-Match to prevent lost updates:

# Only update if the resource still has this ETag (optimistic locking)
curl -X PUT https://api.example.com/orders/789 \
  -H "If-Match: \"abc123\"" \
  -d '{ "status": "processing" }'

Someone else may have modified the resource since the client retrieved it. If so, the server returns 412 Precondition Failed.

For cache revalidation, clients use If-None-Match:

# Only send the body if the resource has changed
curl -H "If-None-Match: \"abc123\"" https://api.example.com/orders/789
# Returns 304 Not Modified if unchanged

If-Modified-Since / Last-Modified work similarly but with timestamps.

Long-Running Operations 🔗

Some operations take seconds or minutes. Don't block. Return 202 Accepted and provide a way to track progress:

POST /reports/generate

HTTP/1.1 202 Accepted
Location: /jobs/report-abc123

{
  "jobId": "report-abc123",
  "status": "pending",
  "statusUrl": "/jobs/report-abc123"
}

GET /jobs/report-abc123

{
  "jobId": "report-abc123",
  "status": "running",
  "progress": 42,
  "startedAt": "2026-03-27T09:00:00Z"
}

GET /jobs/report-abc123

{
  "jobId": "report-abc123",
  "status": "completed",
  "result": { "url": "/reports/2026-q1.pdf" }
}

Alternatively, use webhooks to push completion notifications rather than requiring polling.

API Design Best Practices 🔗

Be Consistent 🔗

Inconsistency is the number one user-experience failure in APIs. Pick conventions for naming, casing, error format, date format, pagination, and versioning, then apply them everywhere.

• Field names: camelCase or snake_case, pick one
• Dates: ISO 8601 (2026-03-27T09:00:00Z) always, never Unix timestamps in mixed APIs
• Booleans: isActive, not active, enabled, or status: "true"

Document Everything 🔗

An API without documentation is incomplete. Use OpenAPI 3.x to describe your API. It enables generated documentation, client SDK generation, request validation, and contract testing.

Document:

• Every endpoint, method, and path parameter
• Every request body field (required vs. optional, constraints)
• Every response status code and body
• Authentication requirements
• Rate limits
• Deprecation timelines

Use HTTPS Exclusively 🔗

Never offer plain HTTP. Redirect all HTTP traffic to HTTPS. Use HSTS (Strict-Transport-Security) to prevent downgrade attacks.

Design for Your Consumer 🔗

APIs are products. Talk to the developers who will use your API. Understand their workflows. APIs that are technically correct but operationally awkward get abandoned. I always advise treating your API contract like a public function signature in a library.

Backward Compatibility is a Promise 🔗

Once you publish an endpoint, clients depend on it. Make breaking changes rare, communicate them well, and support them with a migration path and a generous deprecation window.

Keep Actions Atomic 🔗

An endpoint should either succeed completely or fail completely. Partial success is a consistency nightmare. If a batch operation partially fails, return a clear, structured response. It should identify which items succeeded and which failed, with specific error details.

Don't Return 200 with an Error Body 🔗

This bears repeating. Use the HTTP status code correctly. Your monitoring, your clients, and your own sanity will thank you.

Closing 🔗

Most "REST APIs" in production implement a subset of the constraints. They ignore the rest, particularly HATEOAS. That's a pragmatic reality of software engineering. It's fine, as long as it's a deliberate choice.

Think about it. The difference between a good API and a frustrating one usually isn't the technology. It's the care taken with every small decision. The status code, the error message, the version strategy, the pagination contract. Those decisions compound. Clients you don't know yet will inherit them.

Build APIs you'd want to consume yourself.

PS: Let me know if I missed anything. Ping me on Twitter/X or LinkedIn, and follow along there if you'd like more of this. Let's chat.

rest-api hateoas pagination rate-limiting openapi