I am currently working with the Webhooks notifications (https://airtable.com/developers/web/api/webhooks-overview) to process record updates when a set of files/fields have changed. While I love the power and flexibility of the notification, especially the `fromSources` filtering and metadata, I have hit a few roadbumps in my implementations and I'm wondering if I am missing any obvious solutions that others have discovered. - Cursor value. Initially I thought I could possibly track the cursor value, but then I realized that this is likely just the cursor from the start of the payload list, so after 7 days of payloads, the max cursor value from the payloads will be less than my locally saved cursor, so this doesn't seem viable.
- `baseTransactionNumber`. Since AirTable states that "each payload includes a transaction number, which is scoped to that webhook and sequentially increases", we can use this number to ensure we don't process multiple payloads more than once. However, we are not given a way to use this transaction number at the List Payloads endpoint. Since payloads don't expire for 7 days, this means that if we don't track the cursor, we may be required to make multiple API requests (since we are limited to retrieving 50 payloads per call) to the listing endpoint. And if we use something like AWS Lambdas which can scale depending on the load, then we will certainly start hitting the 5 requests per base rate limit. Does anyone have any advice for a cleaner way to track/retrieve payloads besides some transaction number/cursor combination?
- On the note of the 5 requests per base limit, does anyone know if this rate is separate from the normal API usage or is it included in the normal API limit?
For what it's worth, here are the things AirTable could provide to make this easier
- Include a "payload list count" or better yet the matching transaction number in the notification message.
- Allow querying with start/stop values where (stop - start) <= 50 based on the transaction numbers
