Accessing Airtable's Pagination Using Python's Requests Library
Hey Guys,
How can I access Airtable’s pagination by including a given offset in either the url or query?
My attempt below at including in the url has failed with the error “KeyError: ‘offset’”.
I’ve been able to access the 1st offset per the function at_page(). For simplicity’s sake, I took the result from the function and hardcoded it to the variable offset_01.
Returns iterator with lists in batches according to pageSize.
To get all records at once use :any:`get_all`
>>> for page in airtable.get_iter():
... for record in page:
... print(record)
b{'fields': ... }, ...]
Keyword Args:
max_records (``int``, optional): The maximum total number of
records that will be returned. See :any:`MaxRecordsParam`
view (``str``, optional): The name or ID of a view.
See :any:`ViewParam`.
page_size (``int``, optional ): The number of records returned
in each request. Must be less than or equal to 100.
Default is 100. See :any:`PageSizeParam`.
fields (``str``, ``list``, optional): Name of field or fields to
be retrieved. Default is all fields. See :any:`FieldsParam`.
sort (``list``, optional): List of fields to sort by.
Default order is ascending. See :any:`SortParam`.
formula (``str``, optional): Airtable formula.
See :any:`FormulaParam`.
Returns:
iterator (``list``): List of Records, grouped by pageSize
"""
offset = None
while True:
data = self._get(self.url_table, offset=offset, **options)
records = data.get("records", r])
time.sleep(self.API_LIMIT)
yield records
offset = data.get("offset")
if not offset:
break
Returns iterator with lists in batches according to pageSize.
To get all records at once use :any:`get_all`
>>> for page in airtable.get_iter():
... for record in page:
... print(record)
b{'fields': ... }, ...]
Keyword Args:
max_records (``int``, optional): The maximum total number of
records that will be returned. See :any:`MaxRecordsParam`
view (``str``, optional): The name or ID of a view.
See :any:`ViewParam`.
page_size (``int``, optional ): The number of records returned
in each request. Must be less than or equal to 100.
Default is 100. See :any:`PageSizeParam`.
fields (``str``, ``list``, optional): Name of field or fields to
be retrieved. Default is all fields. See :any:`FieldsParam`.
sort (``list``, optional): List of fields to sort by.
Default order is ascending. See :any:`SortParam`.
formula (``str``, optional): Airtable formula.
See :any:`FormulaParam`.
Returns:
iterator (``list``): List of Records, grouped by pageSize
"""
offset = None
while True:
data = self._get(self.url_table, offset=offset, **options)
records = data.get("records", r])
time.sleep(self.API_LIMIT)
yield records
offset = data.get("offset")
if not offset:
break
Thank you so much! Will review this and properly reply shortly.
I chose not to use the wrapper as I’m concerned it’ll lose support and get stale. It’s not supported by any legit orgs. There’s also a lot of stale issues open, which can speak to a small community as it is.
Requests won’t have that issue.
Thank you so much! Will review this and properly reply shortly.
I chose not to use the wrapper as I’m concerned it’ll lose support and get stale. It’s not supported by any legit orgs. There’s also a lot of stale issues open, which can speak to a small community as it is.
Requests won’t have that issue.
Yep - I fear this as well, but the code itself might provide you with some insights that you can apply to your own code.
I think there may be another one out there too with more support.
Yep - I fear this as well, but the code itself might provide you with some insights that you can apply to your own code.
I think there may be another one out there too with more support.
Hi Bill,
Funny enough, your discussion with @Bastiaan_Bijl helped me find how to query within the requests library:
It’s the params={‘offset’: ‘itrXXXXXX/recXXXXXX’} query that allowed me to access the next 100 records.
