Skip to main content

Hey Guys,



How can I access Airtable’s pagination by including a given offset in either the url or query?



My attempt below at including in the url has failed with the error “KeyError: ‘offset’”.



I’ve been able to access the 1st offset per the function at_page(). For simplicity’s sake, I took the result from the function and hardcoded it to the variable offset_01.



Thank you!



import requests


import pandas as pd





offset_01 = 'itrCG3VDp2c34x1j1/recKTJGYMICe13iA8'


url = 'https://api.airtable.com/v0/PRIVATETABLEKEY/accounts' + \


    '&offset=' + offset_01


headers = {'Authorization': 'Bearer PRIVATEAPIKEY'}








# Calling API


response = requests.get(


    url,


    headers=headers,


)


json_response = response.json()








# Finding AT page offset





def at_page(at_json):


    df_initial = pd.DataFrame(at_json)


    at_offset = df_initialt'offset']t0]





    return at_offset





offset = at_page(json_response)


PS also have an open stackoverflow:




Accessing Airtable's Pagination Using Python's Requests Library







python-3.x, api, python-requests, airtable







asked by





empty__cup



on 06:54PM - 08 Jan 20 UTC





















Any reason you’re not using a library for this?



I think this is sort’a what you want -



def get_iter(self, **options):


“”"


Record Retriever Iterator



    Returns iterator with lists in batches according to pageSize.


    To get all records at once use :any:`get_all`





    >>> for page in airtable.get_iter():


    ...     for record in page:


    ...         print(record)


    b{'fields': ... }, ...]





    Keyword Args:


        max_records (``int``, optional): The maximum total number of


            records that will be returned. See :any:`MaxRecordsParam`


        view (``str``, optional): The name or ID of a view.


            See :any:`ViewParam`.


        page_size (``int``, optional ): The number of records returned


            in each request. Must be less than or equal to 100.


            Default is 100. See :any:`PageSizeParam`.


        fields (``str``, ``list``, optional): Name of field or fields to


            be retrieved. Default is all fields. See :any:`FieldsParam`.


        sort (``list``, optional): List of fields to sort by.


            Default order is ascending. See :any:`SortParam`.


        formula (``str``, optional): Airtable formula.


            See :any:`FormulaParam`.





    Returns:


        iterator (``list``): List of Records, grouped by pageSize





    """


    offset = None


    while True:


        data = self._get(self.url_table, offset=offset, **options)


        records = data.get("records", r])


        time.sleep(self.API_LIMIT)


        yield records


        offset = data.get("offset")


        if not offset:


            break


Any reason you’re not using a library for this?



I think this is sort’a what you want -



def get_iter(self, **options):


“”"


Record Retriever Iterator



    Returns iterator with lists in batches according to pageSize.


    To get all records at once use :any:`get_all`





    >>> for page in airtable.get_iter():


    ...     for record in page:


    ...         print(record)


    b{'fields': ... }, ...]





    Keyword Args:


        max_records (``int``, optional): The maximum total number of


            records that will be returned. See :any:`MaxRecordsParam`


        view (``str``, optional): The name or ID of a view.


            See :any:`ViewParam`.


        page_size (``int``, optional ): The number of records returned


            in each request. Must be less than or equal to 100.


            Default is 100. See :any:`PageSizeParam`.


        fields (``str``, ``list``, optional): Name of field or fields to


            be retrieved. Default is all fields. See :any:`FieldsParam`.


        sort (``list``, optional): List of fields to sort by.


            Default order is ascending. See :any:`SortParam`.


        formula (``str``, optional): Airtable formula.


            See :any:`FormulaParam`.





    Returns:


        iterator (``list``): List of Records, grouped by pageSize





    """


    offset = None


    while True:


        data = self._get(self.url_table, offset=offset, **options)


        records = data.get("records", r])


        time.sleep(self.API_LIMIT)


        yield records


        offset = data.get("offset")


        if not offset:


            break

Thank you so much! Will review this and properly reply shortly.



I chose not to use the wrapper as I’m concerned it’ll lose support and get stale. It’s not supported by any legit orgs. There’s also a lot of stale issues open, which can speak to a small community as it is.



Requests won’t have that issue.

Thank you so much! Will review this and properly reply shortly.



I chose not to use the wrapper as I’m concerned it’ll lose support and get stale. It’s not supported by any legit orgs. There’s also a lot of stale issues open, which can speak to a small community as it is.



Requests won’t have that issue.




Yep - I fear this as well, but the code itself might provide you with some insights that you can apply to your own code.



I think there may be another one out there too with more support.



Yep - I fear this as well, but the code itself might provide you with some insights that you can apply to your own code.



I think there may be another one out there too with more support.


Hi Bill,



Funny enough, your discussion with @Bastiaan_Bijl helped me find how to query within the requests library:





It’s the params={‘offset’: ‘itrXXXXXX/recXXXXXX’} query that allowed me to access the next 100 records.



# Calling API


response = requests.get(


    url,


    headers=headers,


    params={'offset': 'itrXXXXXX/recXXXXXX'}


)


json_response = response.json()


Thank you for your contributions and looking forward to continue to support the community as well 🙂

Reply


Terms & Conditions