Say there’s an Airtable database with 50 000 records and restricted to public.
We use a script to retrieve its data via Airtable API: php sdk GitHub - sleiman/airtable-php: A PHP client for the Airtable API.
This API has limits: 5 requests per second & 100 items per request.
If we try to get the data in a faster pace we go over limits and get blocked for 30 seconds by the server (429 status code and 30 seconds cooldown before subsequent requests will succeed)
Is there a way to get around these API restrictions and recieve the data quicker?
You can use a webhook automation that runs a script that gathers data for more records and then posts to the data to your server. Note that scripting automations have their limitations, but you should be able to get more than 500 records per second.
I would be super cautious about trying to use an undocumented feature of this scale. Plus, how would you even know what to include in your header with no documentation?
I can theoretically see conditional gets as useful at the record level, except that the data for a single record tends to be fairly small and probably not worth it for Airtable to build out the architecture. I don’t see how conditional gets would really work when getting a set of records, especially if you are using filterByFormula.
Furthermore, I don’t think that conditional gets would workaround the API request limit. The limit is on the number of requests received. A conditional get would still count as a request received.
The nature of the API is such that any and all activity/bandwidth consumed affects all users and processes. Given that (at scale) Airtable struggles in the enterprise, conditional GETs and ETags can provide significant reduction in the number of requests an app may need to perform.
Imagine a request of a thousand rows turns up no changes; with conditional GETs the rows don’t have to traverse the internet, only a few bytes tell the requester that there are no changes. This information can be used to better pace the calls should there be known changes.
Don’t think so. They have event handlers to know if data has changed and when. They simply need to support response headers that share this information.
