Difficulty with the Airtable API Retrieve function

[Ollie]

Hello, first of all, the Airtable API documentation is great. That said, closely following the example code, i’m trying to understand how I might do something functional with the retrieve method.

Here is the example I am given from the API docs for my table ‘Users’

base('Users').select({
    // Selecting the first 3 records in Grid view:
    maxRecords: 3,
    view: "Grid view"
}).eachPage(function page(records, fetchNextPage) {
    // This function (`page`) will get called for each page of records.

    records.forEach(function(record) {
        console.log('Retrieved', record.get('ID'));
    });

    // To fetch the next page of records, call `fetchNextPage`.
    // If there are more records, `page` will get called again.
    // If there are no more records, `done` will get called.
    fetchNextPage();

}, function done(err) {
    if (err) { console.error(err); return; }
});

What i’d like to do is wrap this example up in a function and return an array of all records. But I don’t see how I might do that from the current documentation.

But additionally, having explored the source code, I’m wondering whether there is something missing?

In the firstPage() and all() functions, I notice the done() function has two signatures: error case: done(err, null) and success case: done(null, allRecords). But in the eachPage function in the success case, the records are not passed on, it’s simply done(null).

I do have limited experience with javascript and node.js though so maybe this all fine. But seeing as there are one or two other threads on this forum of people having a similar issue, it might be at least worth giving a few more examples in the Airtable API documentation on how I might retrieve all records using the eachPage function.

Thanks for any help and clarity you might be able to provide.

Thanks,
Oliver

[ROARK · Accepted solution]

Hi Ollie,

I ran into the same kind of trouble a while ago.

Here is a function I came up with:

async function getRecordsFromAirtable(table, options = {}) {
    let recordsArray = [];
    await table.select(options).eachPage((records, fetchNextPage) => {
        recordsArray = [...recordsArray, ...records];
        fetchNextPage();
    })
        .catch(error => { console.error(error); return false; })
    return recordsArray;
}

With the example from the documentation you could adjust it as follows:

async function getRecordsFromAirtable() {
    let recordsArray = [];

    await base('Users').select({
        maxRecords: 3,
        view: "Grid view"
    }).eachPage((records, fetchNextPage) => {
        recordsArray = [...recordsArray, ...records];
        fetchNextPage();
    })
        .catch(error => { console.error(error); return false; })

    return recordsArray;
}

Hope that helps.

[Ollie]

Hi Roark,

This is very helpful thankyou. As a predominantly Python programmer, I haven’t yet got async calls sussed and my attempts to populate an array were not going so well. Thanks for sharing a working example.

[Bill_French]

Ollie:

I haven’t yet got async calls sussed and my attempts to populate an array were not going so well.

You are not alone. Airtable should consider adding a Python scripting engine. I know - the engineers at Airtable are at this moment saying -

But Bill, you complained for three years about no javascript capability - we added that and now you’re not happy; you want another language?

Yes, that’s exactly what I’m saying. Why? Simple answer…

• Airtable is rapidly becoming a data store of lots of business and organizational data.
• The platform’s advances in automation and integration capabilities have caused it to attract increasingly more data.
• The world is rapidly embracing data science as a key competitive advantage and Python is the logical and most powerful choice for data science.

Ergo, if you want to become the no/low-code platform of choice for data AND data science, add Python right away. The folks at SeaTable saw this train coming but it’s not too late to get on.