REST APIs in Production: Pagination, HATEOAS, and the Patterns That Actually Matter (Part 3 of 3)

⏱ 13 min read

by Irina Scurtu

This is the last post in a three-part series on REST API design. The first post covered REST's constraints and HTTP semantics. The second covered resource design, versioning, and authentication. This one covers what separates an API that works in a demo from one that survives production traffic.

Pagination 🔗

Collections that can grow without bound should always be paginated. Returning 50,000 records in a single response is a denial of service waiting to happen.

Offset-Based Pagination 🔗

GET /orders?offset=0&limit=20
GET /orders?offset=20&limit=20
GET /orders?page=2&per_page=20

Simple to implement and query. Allows jumping to arbitrary pages.

Weakness: inconsistent results under concurrent writes. If a record is inserted between page 1 and page 2 requests, you may see a duplicate. If one is deleted, you may skip a record. Performance also degrades at high offsets in most databases.

Cursor-Based Pagination 🔗

GET /orders?limit=20
# Returns: { "data": [...], "cursor": "eyJpZCI6MjB9" }

GET /orders?after=eyJpZCI6MjB9&limit=20

The cursor encodes a position in the result set (typically the ID or sort key of the last item returned). Each page request is a stable query from a known position.

Strengths: consistent under concurrent writes, efficient at any depth, suits real-time feeds.

Weaknesses: cannot jump to arbitrary pages (only forward/backward), cursor must be treated as opaque.

Keyset Pagination 🔗

A variant of cursor pagination where the position is expressed as an explicit field value:

GET /orders?after_id=789&limit=20

Less opaque than encoded cursors, but ties pagination semantics to visible fields.

Response Envelope for Pagination 🔗

Always include pagination metadata:

{
  "data": [ ... ],
  "pagination": {
    "total": 1540,
    "limit": 20,
    "offset": 40,
    "has_next": true,
    "has_previous": true
  },
  "links": {
    "self":  "/orders?offset=40&limit=20",
    "next":  "/orders?offset=60&limit=20",
    "prev":  "/orders?offset=20&limit=20",
    "first": "/orders?offset=0&limit=20",
    "last":  "/orders?offset=1520&limit=20"
  }
}

The links object connects this to HATEOAS - more on that next.

HATEOAS 🔗

HATEOAS - Hypermedia as the Engine of Application State - is the most misunderstood and most often omitted constraint of REST. It is also, according to Fielding himself, the part that makes an API truly RESTful.

The idea: a client should be able to move through an API entirely through links provided in responses, with minimal out-of-band knowledge about URI structure. Just as a human clicks through a website by following links, a REST client should follow links through an API instead of hardcoding URI structure it was never given.

What It Looks Like 🔗

GET /orders/789

{
  "id": "789",
  "status": "pending",
  "total": 142.50,
  "customer": {
    "id": "42",
    "name": "Alice"
  },
  "_links": {
    "self":     { "href": "/orders/789" },
    "cancel":   { "href": "/orders/789/cancel", "method": "POST" },
    "customer": { "href": "/users/42" },
    "payment":  { "href": "/payments?orderId=789" }
  }
}

The server controls what actions are available. If the order is already cancelled, the cancel link disappears. The client doesn't need to know the business rules around when cancellation is allowed - it just checks whether the link is present.

Media Types and HATEOAS 🔗

The _links pattern above is informal. There are standardized hypermedia formats:

  • HAL (Hypertext Application Language): uses _links and _embedded, widely adopted
  • JSON:API: a more opinionated full specification for request/response format
  • Siren: includes actions with typed fields, more expressive for write operations
  • Collection+JSON: designed specifically for collection resources

The Reality of HATEOAS Adoption 🔗

HATEOAS is theoretically correct and has genuine benefits for client decoupling. In practice, most production APIs don't implement it, and most clients don't use it even when offered. The reasons are pragmatic: statically typed clients often prefer a well-documented contract over dynamic link discovery, and the tooling for hypermedia clients is not as mature as for OpenAPI-described APIs.

This is worth knowing so you can make an informed decision, not ignore it by default. For public APIs with diverse clients and long lifespans, HATEOAS is worth the investment. For internal APIs between services you control, a well-maintained OpenAPI spec may be more practical.

Filtering, Sorting, and Searching 🔗

Filtering 🔗

Use query parameters to filter collections:

GET /orders?status=pending
GET /orders?status=pending&customerId=42
GET /products?minPrice=10&maxPrice=100&category=electronics

For complex filters, some APIs use a structured query language:

GET /products?filter=price:lt:100,category:eq:electronics

Or a JSON-encoded filter (though this can be awkward in URLs):

GET /products?filter={"price":{"lt":100},"category":"electronics"}

Be consistent and document your filter syntax explicitly.

Sorting 🔗

GET /orders?sort=createdAt
GET /orders?sort=createdAt:desc
GET /orders?sort=-createdAt          # minus prefix for descending
GET /orders?sort=status,createdAt:desc

Always document the default sort order and make it stable (include a unique field as a tiebreaker).

Searching 🔗

Full-text search usually warrants a dedicated endpoint or query parameter:

GET /products?q=wireless+headphones
GET /search?q=wireless+headphones&type=products

For sophisticated search requirements, dedicated search endpoints with POST bodies can be appropriate:

POST /products/search

{
  "query": "wireless headphones",
  "filters": { "inStock": true, "priceMax": 200 },
  "sort": { "field": "relevance", "order": "desc" },
  "pagination": { "limit": 20, "offset": 0 }
}

Using POST for search is a pragmatic choice when the query is too complex for a URL, though it sacrifices cacheability.

Sparse Fieldsets and Field Selection 🔗

Large resources with many fields are expensive to transfer and often contain fields a given client doesn't need. Field selection lets clients request only what they need:

GET /users/42?fields=id,name,email
GET /orders?fields=id,status,total&include=customer.name

This is particularly important for mobile clients on limited bandwidth.

Content Negotiation 🔗

REST embraces HTTP's content negotiation mechanism. Clients can request specific response formats:

# Request JSON
curl -H "Accept: application/json" https://api.example.com/users/42

# Request XML
curl -H "Accept: application/xml" https://api.example.com/users/42

The server indicates what it returned:

Content-Type: application/json; charset=utf-8

If the server cannot produce the requested format, it returns 406 Not Acceptable.

For versioning via content type:

curl -H "Accept: application/vnd.myapi.v2+json" https://api.example.com/users/42

Rate Limiting 🔗

Every public API needs rate limiting. Return standard headers so clients can manage their behavior:

X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 742
X-RateLimit-Reset: 1735689600
Retry-After: 3600

When a client exceeds the limit, return 429 Too Many Requests with a Retry-After header.

Common rate limiting strategies:

  • Fixed window: count requests in a fixed time window (simple, but allows burst at window boundary)
  • Sliding window: rolling time window (smoother, more expensive)
  • Token bucket: tokens replenish at a fixed rate, bursts allowed up to bucket size
  • Leaky bucket: requests processed at a fixed rate, excess queued or dropped

Provide different rate limit tiers for different clients (authenticated vs. anonymous, free vs. paid).

Idempotency Keys 🔗

For non-idempotent operations (POST), idempotency keys allow clients to safely retry requests without risk of duplicate side effects:

curl -X POST https://api.example.com/payments \
  -H "Idempotency-Key: 7f3a2b1c-4d5e-6f7g-8h9i-0j1k2l3m4n5o" \
  -H "Content-Type: application/json" \
  -d '{"amount": 9900, "currency": "USD"}'

The server caches the response for the idempotency key for a defined window. A retry with the same key returns the cached response without reprocessing the payment.

This pattern is essential for payment APIs, order creation, and any operation with financial or irreversible consequences.

Conditional Requests and Optimistic Concurrency 🔗

HTTP provides mechanisms for conditional operations via ETags and timestamps.

ETags are version identifiers for a resource:

GET /orders/789

HTTP/1.1 200 OK
ETag: "abc123"

A client can make conditional requests using If-Match to prevent lost updates:

# Only update if the resource still has this ETag (optimistic locking)
curl -X PUT https://api.example.com/orders/789 \
  -H "If-Match: \"abc123\"" \
  -d '{ "status": "processing" }'

If the resource has been modified by someone else since the client retrieved it, the server returns 412 Precondition Failed.

For cache revalidation, clients use If-None-Match:

# Only send the body if the resource has changed
curl -H "If-None-Match: \"abc123\"" https://api.example.com/orders/789
# Returns 304 Not Modified if unchanged

If-Modified-Since / Last-Modified work similarly but with timestamps.

Long-Running Operations 🔗

Some operations take seconds or minutes. Don't block. Return 202 Accepted and provide a way to track progress:

POST /reports/generate

HTTP/1.1 202 Accepted
Location: /jobs/report-abc123

{
  "jobId": "report-abc123",
  "status": "pending",
  "statusUrl": "/jobs/report-abc123"
}
GET /jobs/report-abc123

{
  "jobId": "report-abc123",
  "status": "running",
  "progress": 42,
  "startedAt": "2026-03-27T09:00:00Z"
}
GET /jobs/report-abc123

{
  "jobId": "report-abc123",
  "status": "completed",
  "result": { "url": "/reports/2026-q1.pdf" }
}

Alternatively, use webhooks to push completion notifications rather than requiring polling.

API Design Best Practices 🔗

Be Consistent 🔗

Inconsistency is the number one user-experience failure in APIs. Pick conventions for naming, casing, error format, date format, pagination, and versioning - then apply them everywhere.

  • Field names: camelCase or snake_case - pick one
  • Dates: ISO 8601 (2026-03-27T09:00:00Z) always, never Unix timestamps in mixed APIs
  • Booleans: isActive, not active, enabled, or status: "true"

Document Everything 🔗

An API without documentation is incomplete. Use OpenAPI 3.x to describe your API. It enables generated documentation, client SDK generation, request validation, and contract testing.

Document:

  • Every endpoint, method, and path parameter
  • Every request body field (required vs. optional, constraints)
  • Every response status code and body
  • Authentication requirements
  • Rate limits
  • Deprecation timelines

Use HTTPS Exclusively 🔗

There is no reason to offer plain HTTP. Redirect all HTTP traffic to HTTPS. Use HSTS (Strict-Transport-Security) to prevent downgrade attacks.

Design for Your Consumer 🔗

APIs are products. Talk to the developers who will use your API. Understand their workflows. APIs that are technically correct but operationally awkward get abandoned. I always advise treating your API contract the same way you treat a public function signature in a library.

Backward Compatibility is a Promise 🔗

Once you publish an endpoint, clients depend on it. Breaking changes should be rare, well-communicated, and supported with a migration path and a generous deprecation window.

Keep Actions Atomic 🔗

An endpoint should either succeed completely or fail completely. Partial success is a consistency nightmare. If a batch operation partially fails, return a clear, structured response that identifies which items succeeded and which failed with specific error details.

Don't Return 200 with an Error Body 🔗

This bears repeating. Use the HTTP status code correctly. Your monitoring, your clients, and your own sanity will thank you.

Closing 🔗

Most "REST APIs" in production implement a subset of the constraints and ignore the rest, particularly HATEOAS. That's a pragmatic reality of software engineering, and it's fine - as long as it's a deliberate choice.

Think about it: the difference between a good API and a frustrating one usually isn't the technology. It's the care taken with every small decision - the status code, the error message, the version strategy, the pagination contract. Those decisions compound. Clients you don't know yet will inherit them.

Build APIs you'd want to consume yourself.

PS: Let me know if I missed anything - ping me on Twitter/X or LinkedIn, and follow along there if you'd like more of this. Let's chat.

rest-api hateoas pagination rate-limiting openapi

Enjoyed this?

I write about .NET, messaging, and distributed systems most weeks - the parts that don't make it into a LinkedIn post.