Pagination implementation

[Danny_Alvarez]

Hi there! I’m getting a bunch of records using the API and I would like to paginate the results - in the end, I’d like to display to the user 10 records on a page, then when they hit ‘Next Page’ I can show them the next 10, and so on, requesting the next 10 from the server each time (as opposed to grabbing all the records myself and paginating them in the client).

I see that there is eachPage() and fetchNextPage(), but I may be confused about how they work. I use eachPage to get the records and massage the data I get by looping through the records. However, the done() function doesn’t seem to get called unless I call fetchNextPage() and I want to call fetchNextPage() after a user chooses to get to the next page of results. Any help/advice would be awesome!

[Alexander_Sorok]

Hi Danny,

If you’re having a single page app, you can call “fetchNextPage” when the user clicks on the “next” link. The done callback would not be called unless the user clicked through all the pages. That’s actually perfectly fine.

Alex

[Danny_Alvarez]

Ah ok, I think i understand it more - i had a callback in the done() that was necessary to render the information, but I moved that callback into the eachPage() itself (after looping through the records) and that seems fine.

Now I have two more questions :slightly_smiling_face:

• Is there a way to retrieve the previousPage - fetchPreviousPage() - so a user would be able to move back and forth between them?
• Is fetchNextPage() a member of the Airtable class? Do I call it like var base = new Airtable(…); base.fetchNextPage()?

[Alexander_Sorok]

Hi Danny,

There’s isn’t a previousPage - you could just memoize those once your client fetches it. fetchNextPage is not a class member - it’s a helper that’s passed into a page callback.

Here’s exactly where it’s created https://github.com/Airtable/airtable.js/blob/2dad5fe2b5d7551fe67971c67fd7ad1be0a5f80f/lib/query.js#L...

Alex

[Danny_Alvarez]

Ah ok - I can add memoization to save the previous results. However, I think I’m in a bind calling fetchNextPage(). My airtable has a couple of tables where the records of one are needed to fetch the records of another. Using async.waterfall, I’m calling 3 tables, using the results from the previous table to get more records needed.

The first stage of the waterfall, where I call select().eachPage() for many records, I was calling the waterfallCallback() in the airtable’s select().done() (which was called after reaching the last page of records). I can move the waterfallCallback() into the .eachPage() so I can get all the information I need for the next waterfall stage, but then I cannot call fetchNextPage() because it would call the same waterfallCallback().

Is there an offset that can be retrieved from getting a page, or can an offset or pageNumber be passed to a request?

[Alexander_Sorok]

You may be better off using the lower level API. Take a look at
https://github.com/Airtable/airtable.js/blob/2dad5fe2b5d7551fe67971c67fd7ad1be0a5f80f/lib/table.js#L... . As it’s a private method, it may be refactored, but if you have a strong use case, we can consider making it public.

With that, you can even store the offsets separately and fetch pages on demand.

Note, the offsets should be valid for a while, but not indefinitely - hours, not days. That may break some of the single page app logic.

Alex

[Weiwei_Li]

Please add the offset property in Node.js, That would be amazing!

[Chinara_James]

You can follow this article @Weiwei_Li

Chinara James – 5 Nov 18

How to paginate records in Airtable when using the API | Chinara James

Airtable’s API is great and so is their documentation. However, I came across a small stumbling block while using it to build a simple project. The problem, Airtable returns a maximum of 100 records…

[Weiwei_Li]

Ah yes, I found that. It will return paginated data, but it seems to me like it still requires you to retrieve every single record of the table at the get-go.
So if you have 5000 records, it will need to keep calling fetchNextPage() until all 5000 records are retrieved. Only then is the data being paginated. So it doesn’t seem to be a very scalable solution.
I think my understanding is correct here?

IF we could access the offset property (which is available in curl), we could retrieve the first 100 records. Then, when an event is passed in to retrieve the next set of 100 records, the offset property could help us get those records by starting the record retrieval at the 101th record.

[Chinara_James]

You may be right that it could have a performance hit for say 5000 records but the Airtable node client does implement caching so that could help with the performance.
It’s never been slow in my experience to retrieve all the records but I have not tested with a base of 1000+ records.
I too was looking for the offset parameter but it has not been implemented yet to my knowledge.