Skip to main content

Hey

I am using Postman to make an API call from Airtable to list all 14,000 of my records to store in a variable and pass onto voiceflow.

This is my pre- request script:

 

function getAllRecords() {
let records = [];

// First request
requestAirtable(url, null, function(initial_response) {
records.push(...initial_response.records);
let offset = initial_response.offset;

// Handle pagination recursively
function fetchNextPage(offset) {
if (!offset) {
console.log(`✅ Data Fetch Complete! Total records: ${records.length}`);
pm.environment.set("voiceflow_data", JSON.stringify({ fields: records }));
return;
}

requestAirtable(url, offset, function(response) {
records.push(...response.records);
fetchNextPage(response.offset);
});
}

fetchNextPage(offset);
});
}

function requestAirtable(url, offset, callback) {
const api_key = pm.environment.get("AIRTABLE_API_KEY"); // Securely store API key

if (offset) {
url += `?offset=${offset}`;
}

const options = {
url: url,
method: 'GET',
headers: { // Fix: Correct 'header' to 'headers'
'Authorization': `Bearer ${api_key}`, // Fix: Ensure key is inside string
'Content-Type': 'application/json'
}
};

pm.sendRequest(options, function(err, res) {
if (err) {
console.log("Airtable Request Error:", err);
return;
}
callback(res.json());
});
}

// Run the function
getAllRecords();
 
I used ChatGPT for this as I am not a developer, but it's not working. Any help would be greatly appreciated.
 
Thanks 

Did you change the relevant variables (base ID, table ID, api key)?

Are any error messages returned when you run the script? This would be very helpful in determining what's wrong.


Did you change the relevant variables (base ID, table ID, api key)?

Are any error messages returned when you run the script? This would be very helpful in determining what's wrong.


Hi

Thanks for getting in touch. 

Yes I changed the base ID, table ID and api key. The issue is that rather than the function running until all records are received it stops at the first 100.


The standard API has a limit of 100 records per call. You need to use additional calls with pagination to get more than 100 records.

Here are some topics on the forum that have scripts that other users have used:

https://community.airtable.com/t5/development-apis/how-to-get-more-than-100-rows-using-airtable-api-in-python-using/td-p/69753

https://community.airtable.com/t5/development-apis/show-more-than-100-records-with-api-calls-using-javascript-ajax/td-p/66648#:~:text=You%20can%20only%20retrieve%20a,get%20(even%20with%20pagination)

 


The standard API has a limit of 100 records per call. You need to use additional calls with pagination to get more than 100 records.

Here are some topics on the forum that have scripts that other users have used:

https://community.airtable.com/t5/development-apis/how-to-get-more-than-100-rows-using-airtable-api-in-python-using/td-p/69753

https://community.airtable.com/t5/development-apis/show-more-than-100-records-with-api-calls-using-javascript-ajax/td-p/66648#:~:text=You%20can%20only%20retrieve%20a,get%20(even%20with%20pagination)

 


Because I have a lot of records I was trying to do it automatically, but I have accepted that it needs to be manually in batches.


Reply