Ask HN: RESTful API Pagination Strategies
4 comments
We follow the github approach and put it all in the headers:
< HTTP/1.1 200 OK
* Server nginx/1.9.11 is not blacklisted
< Server: nginx/1.9.11
< Date: Thu, 10 Mar 2016 03:19:48 GMT
< Content-Type: application/json
< Content-Length: 247620
< Connection: keep-alive
< Count: 120
< X-Frame-Options: SAMEORIGIN
< Vary: Cookie
< Link: <https://xxx.com/api/someresource/?limit=50&offset=50; rel="next">
< Allow: GET, POST, HEAD, OPTIONS
< X-Server: zzz
< X-Frame-Options: SAMEORIGIN
<Twitter has some good points on the problem with paging and uses a "max_id" & "since_id" instead.
https://dev.twitter.com/rest/public/timelines
https://dev.twitter.com/rest/public/timelines
You don't need to know of changes unless the user is on that page.
You can use websockets to broadcast/listen for changed data and updated the DOM accordingly (if you're on the web).
You can use websockets to broadcast/listen for changed data and updated the DOM accordingly (if you're on the web).
I'm hoping to start a discussion on the topic of paging/pagination as there seems to a hole in most implementations that I'm only seeing addressed by some API's.
The example is pretty simple:
GET /some_resource/
Returns: 2,000 results (200 per page)
Loop 10 times to get all results(GET page=1; GET page=2, etc, etc...)
If any record is updated on page 1 while I'm on page 9 (just an example), I will miss it unless I go back and do a reconciliation.
Facebook's Graph API (https://developers.facebook.com/docs/graph-api/using-graph-api -> go to Cursor-based Pagination) tries to address this with a Cursor-based Pagination approach, where they lock a given data set to be within two hashed id's. I don't fully grok the implementation of this strategy though at first glance - [it] seems vast.
Stripe's API tries to do the same https://stripe.com/docs/api#pagination . I don't work with Stripe's API but it appears they want you to specify an "object ID", which I'm guessing is some hashed id + timestamp. Then if a record was updated on page 1 (per my example above), a new hashed ID is generated and would likely be returned on a subsequent page (say page 11 in my example above).
I might be interpreting those implementations incorrectly (by all means correct me here) but I would love to hear from some HN members who have dealt with this issue.
Immediate questions:
- Is there an API out there that solves this really well? - Any further reading folks here might recommend?