pagination for external API

[auekk2787]

I'm writing a script in Airtable Extensions to pull data from Canvas API. I've set the per_page limit to 100, because that's the maximum number of objects Airtable will retrieve at once for an external API.

I need to use pagination to download the rest of the results. According the Canvas API reference, it's in the response header Link value. I'm able to see this in Postman but how do I retrieve this in an Airtable script? 

How do I set up a loop to paginate until the end? All the online discussions are about Airtable's API and I can't find how to retrieve response headers.

let currentCourses = await remoteFetchAsync("https://XXXX?per_page=100", requestOptions);
const currentCoursesData = await currentCourses.json();
console.log(currentCoursesData);

[ScottWorld]

Sorry, I don’t know how to do this with a script, so somebody else will need to help you with that below.

However, you could always automate this with Make’s Canvas Automations. Make doesn’t have a maximum number of objects that it can receive, and also automatically takes care of any pagination for you,

There is a small learning curve with Make, which is why I created this basic navigation video to help. I also provide the links to a few other Make training resources there as well. For example, to instantly trigger your Make scenarios from Airtable with a script instead of a button, check out this thread.

[auekk2787]

Unfortunately my organization doesn't allow data integrations like Make or Zapier. That's why I'm exploring Airtable scripting and APIs.

[auekk2787 · Accepted solution]

I figured this out. You have to use a headers.get and specify the Key from the response, like this:

let request = await remoteFetchAsync("https://XXXX?per_page=100", requestOptions);
let responseLink = request.headers.get('Link');
console.log(responseLink);