Accessing Airtable's Pagination Using Python's Requests Library

Hey Guys,

How can I access Airtable’s pagination by including a given offset in either the url or query?

My attempt below at including in the url has failed with the error “KeyError: ‘offset’”.

I’ve been able to access the 1st offset per the function at_page(). For simplicity’s sake, I took the result from the function and hardcoded it to the variable offset_01.

Thank you!

import requests
import pandas as pd

offset_01 = 'itrCG3VDp2c34x1j1/recKTJGYMICe13iA8'
url = 'https://api.airtable.com/v0/PRIVATETABLEKEY/accounts' + \
    '&offset=' + offset_01
headers = {'Authorization': 'Bearer PRIVATEAPIKEY'}


# Calling API
response = requests.get(
    url,
    headers=headers,
)
json_response = response.json()


# Finding AT page offset

def at_page(at_json):
    df_initial = pd.DataFrame(at_json)
    at_offset = df_initial['offset'][0]

    return at_offset

offset = at_page(json_response)

PS also have an open stackoverflow:

1 Like

Any reason you’re not using a library for this?

I think this is sort’a what you want -

def get_iter(self, **options):
“”"
Record Retriever Iterator

    Returns iterator with lists in batches according to pageSize.
    To get all records at once use :any:`get_all`

    >>> for page in airtable.get_iter():
    ...     for record in page:
    ...         print(record)
    [{'fields': ... }, ...]

    Keyword Args:
        max_records (``int``, optional): The maximum total number of
            records that will be returned. See :any:`MaxRecordsParam`
        view (``str``, optional): The name or ID of a view.
            See :any:`ViewParam`.
        page_size (``int``, optional ): The number of records returned
            in each request. Must be less than or equal to 100.
            Default is 100. See :any:`PageSizeParam`.
        fields (``str``, ``list``, optional): Name of field or fields to
            be retrieved. Default is all fields. See :any:`FieldsParam`.
        sort (``list``, optional): List of fields to sort by.
            Default order is ascending. See :any:`SortParam`.
        formula (``str``, optional): Airtable formula.
            See :any:`FormulaParam`.

    Returns:
        iterator (``list``): List of Records, grouped by pageSize

    """
    offset = None
    while True:
        data = self._get(self.url_table, offset=offset, **options)
        records = data.get("records", [])
        time.sleep(self.API_LIMIT)
        yield records
        offset = data.get("offset")
        if not offset:
            break
1 Like

Thank you so much! Will review this and properly reply shortly.

I chose not to use the wrapper as I’m concerned it’ll lose support and get stale. It’s not supported by any legit orgs. There’s also a lot of stale issues open, which can speak to a small community as it is.

Requests won’t have that issue.

Yep - I fear this as well, but the code itself might provide you with some insights that you can apply to your own code.

I think there may be another one out there too with more support.

1 Like

Hi Bill,

Funny enough, your discussion with @Bastiaan_Bijl helped me find how to query within the requests library:

It’s the params={‘offset’: ‘itrXXXXXX/recXXXXXX’} query that allowed me to access the next 100 records.

# Calling API
response = requests.get(
    url,
    headers=headers,
    params={'offset': 'itrXXXXXX/recXXXXXX'}
)
json_response = response.json()

Thank you for your contributions and looking forward to continue to support the community as well :slight_smile:

1 Like

This topic was automatically closed 24 hours after the last reply. New replies are no longer allowed.