How do I send the offset request?

[ids]

I've seen this question, but not the type of answer I'm hoping for. A CURL example is a great for me to translate the request. Similar to the CURL examples in the docs for List Records, it looks like I should be able to request records 101-200 using the offset value returned in the original request. The docs say it's a Query, therefore:

...?offset=itrvDOnB9cbRDnn63/rec1VA0JPORC2S9Zz

It's not working. I haven't inspected the returned headers yet. 

So the questions:

-- should the request be RAW or POST?

--should/could the request be a JSON payload instead of querystring params?

I need to retrieve thousands of records. My follow up question for data from a different table, USING JSON Payloads can I somehow filter fields? There are many fields I really don't want to process on my end.

Thanks,

Scott

IDS

PS: I plan on using POSTMAN for testing.

[TheTimeSavingCo · Accepted solution]

It's a GET and it's part of your query params, here's an example:

https://api.airtable.com/v0/appz6YcyO3dqDWV/tblsCy3oC7N7FE4c?offset=itrloILUxLWba21F/recreX96bXOkThVj

To filter fields, use the `fields` query param

https://airtable.com/developers/web/api/list-records

[ids]

Thank You.

I have been using GET and the same formatting. trying a POST was just a test of sorts. Also changed to using the Table Name instead of the ID. Same result. I finally did an inspection of the returned Headers:

HTTP/1.1 422 Unprocessable Entity
Date: Fri, 29 Mar 2024 05:44:26 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 56
Connection: close

What is the error: 422 Unprocessable Entity for this call?

https://api.airtable.com/v0/applpByfqnul0sn21/tblUxVun7fwac4OpJ?offset=itrHj5qqR9DmV2LVn/rec1VA0JPORC2S9Zz

My experience in other APIs, this error might indicate a bad token. Or, a session has closed and I can't access the data I was attempting to lookup. Is that possible with the offset record/string?

Thanks,

Scott

IDS

[TheTimeSavingCo]

Yeap that's possible, and if so the body of the response says something like LIST ITERATOR NOT FOUND or something like that.  What was the body of the 422 you got? 

[ids]

Yes, that error is in the body.

{"error":{"type":"LIST_RECORDS_ITERATOR_NOT_AVAILABLE"}}

Is this because of the offset? Solutions?

Found partial answer: I realized what I was looking at in the doc page for list records. 422 is a timeout of sorts.

What is the timeout for this session?

IOW, When the 1st session retrieves the first 100 records with the "offset" string, how long is that window open to retrieve the next 100 records? 60 seconds? 120 seconds? Something different?

Scott

IDS

[TheTimeSavingCo]

The offset's timed out I reckon; if you do the call again and try the new offset it should work fine.  I'm not sure how long the window is I'm afraid

[ids]

Thanks Adam. I was able to get this working as expected. I had been restarting the script after getting the offset values. My fix was capturing the offset value real time and running the API call again, essentially looping through until all the rows, 100 per, were retrieved or when no offset value was returned.

[mliu]

Hi ids,

Can you help to share more details how you make this finally workable to load all rows into Power BI, thank you! it would be great to share some snapshot.