Pagination implementation

Hi Danny,

If you’re having a single page app, you can call “fetchNextPage” when the user clicks on the “next” link. The done callback would not be called unless the user clicked through all the pages. That’s actually perfectly fine.

Alex

Ah ok, I think i understand it more - i had a callback in the done() that was necessary to render the information, but I moved that callback into the eachPage() itself (after looping through the records) and that seems fine.

Now I have two more questions :slightly_smiling_face:

• Is there a way to retrieve the previousPage - fetchPreviousPage() - so a user would be able to move back and forth between them?
• Is fetchNextPage() a member of the Airtable class? Do I call it like var base = new Airtable(…); base.fetchNextPage()?

Hi Danny,

There’s isn’t a previousPage - you could just memoize those once your client fetches it. fetchNextPage is not a class member - it’s a helper that’s passed into a page callback.

Here’s exactly where it’s created https://github.com/Airtable/airtable.js/blob/2dad5fe2b5d7551fe67971c67fd7ad1be0a5f80f/lib/query.js#L82

Alex

Ah ok - I can add memoization to save the previous results. However, I think I’m in a bind calling fetchNextPage(). My airtable has a couple of tables where the records of one are needed to fetch the records of another. Using async.waterfall, I’m calling 3 tables, using the results from the previous table to get more records needed.

The first stage of the waterfall, where I call select().eachPage() for many records, I was calling the waterfallCallback() in the airtable’s select().done() (which was called after reaching the last page of records). I can move the waterfallCallback() into the .eachPage() so I can get all the information I need for the next waterfall stage, but then I cannot call fetchNextPage() because it would call the same waterfallCallback().

Is there an offset that can be retrieved from getting a page, or can an offset or pageNumber be passed to a request?

You may be better off using the lower level API. Take a look at
https://github.com/Airtable/airtable.js/blob/2dad5fe2b5d7551fe67971c67fd7ad1be0a5f80f/lib/table.js#L103 . As it’s a private method, it may be refactored, but if you have a strong use case, we can consider making it public.

With that, you can even store the offsets separately and fetch pages on demand.

Note, the offsets should be valid for a while, but not indefinitely - hours, not days. That may break some of the single page app logic.

Alex

Please add the offset property in Node.js, That would be amazing!

You can follow this article @Weiwei_Li

How to paginate records in Airtable when using the API | Chinara James

Airtable’s API is great and so is their documentation. However, I came across a small stumbling block while using it to build a simple project. The problem, Airtable returns a maximum of 100 records…

Ah yes, I found that. It will return paginated data, but it seems to me like it still requires you to retrieve every single record of the table at the get-go.
So if you have 5000 records, it will need to keep calling fetchNextPage() until all 5000 records are retrieved. Only then is the data being paginated. So it doesn’t seem to be a very scalable solution.
I think my understanding is correct here?

IF we could access the offset property (which is available in curl), we could retrieve the first 100 records. Then, when an event is passed in to retrieve the next set of 100 records, the offset property could help us get those records by starting the record retrieval at the 101th record.

You may be right that it could have a performance hit for say 5000 records but the Airtable node client does implement caching so that could help with the performance.
It’s never been slow in my experience to retrieve all the records but I have not tested with a base of 1000+ records.
I too was looking for the offset parameter but it has not been implemented yet to my knowledge.

thanks for the reply! i didn’t know that caching was implemented.

You are correct that the offset property hasn’t been implemented for node.js, but I don’t think it is actually required based on Evan’s response here:

Issue: Offset property with node.js

Hi, I would like to use the offset property to paginate through records with Node.js. I understand it exists for curl,...

also, thanks for the great articles you have written on your blog. they have been very helpful.

Hi, I would like to use the offset property to paginate through records with Node.js. I understand it exists for curl,...

I believe that I tried that approach, that is, trying to call fetchNextPage only when the user requires it but there are still challenges. In order to create the pagination markup that the tutorial is about you need to know the total number of records and pages and current page. You can’t do that when eachPage only gives you the first 100 but doesn’t tell you the total records.

And you can’t call fetchNextPage outside of the eachPage function so getting counts and current page in conjunction with calling fetchNextPage on the fly doesn’t seem possible but I could be wrong.

If you’re using the NodeJs client and found a better way, do share.

Also like he say Airtable isn’t for backend use per say and I was going to point that out when you mentioned 5000 record. If you app reaches that, it’s probably time to scale to firebase or mongodb. I just use Airtable for smaller projects.