Oct 05, 2022 09:38 PM
We have built an automation that syncs data from our CRM with Airtable. Each insert/update/delete in the CRM post a message to a webhook created from an Automation. This works perfectly fine for single transactions in the CRM but for a bulk update or an import in the CRM of e.g. 500 records it only processes a limited number of records (e.g. 168 out of 800).
Is there a limitation in the number of instances of a script that can run in parallel? And if so, is there a way in Airtable to work around this as I have no idea to buffer the webhooks coming in.
Oct 06, 2022 03:25 AM
I’m not sure if there is a limit to how many times a webhook can be triggered per minute/second. You may want to email firstname.lastname@example.org about that, and post back here with what they say.
However, there are 2 other limitations:
Each webhook request has a maximum payload size limit of 100kb per request. So if you’re sending 800 records at a time to your webhook, it’s possible that you’re going over the 100kb limit.
Your billing plan for your workspace will give you a limited amount of automation runs that you can execute per month. Each triggered webhook counts as one automation run.
If you’re continuing to have problems with Airtable’s webhooks, I would highly recommend using the professional webhooks from Make, because they have no limits at all. No size limits, no run limits, and no limits on the number of times that you can trigger a webhook. They also offer custom webhook responses and custom mailhooks, too.
Oct 07, 2022 11:26 AM
One webhook will not include 500 changed records from the CRM. Pretty sure this is not what’s happening.
So, I could run a trillion webhooks and there would be no phone call or consequences from Make?
Importing a big change into the CRM typically impacts any outbound webhooks one record at a time. But, this is likely to be very fast and will overload the available webworkers on the Airtable webhook receiver which I think is limited to five simultaneous workers at a time (I suspect records in excess are simply ignored by Airtable). I have a hunch the limit is identical to the API limit (5 per second I think).
The challenge is to throttle the CRM at times like this by ensuring that bulk injections include a status field of some type that either the CRM uses to avoid firing the web hooks or such field is used on the Airtable side to ignore such bulk entries. Then, use a more gated process to introduce these new uploaded entries into Airtable in a distributed fashion over time such as when that status flag is lifted, allow it to be posted to Airtable.
Oct 24, 2022 01:14 PM
Adam here from the Airtable Support Content Ops team. Great question on the limit. Currently, the limit is 5 webhook requests per second along with the other 100kb size limit that Scott mentioned. I’ve gone ahead and updated our documentation to reflect that in the FAQs section. Hope this helps and thanks for tagging me in @ScottWorld
Oct 24, 2022 05:41 PM
Thanks! In your post above, you wrote 5 requests per minute, but in the FAQ on the support article, it says 5 requests per second. Is it 5 per second or 5 per minute?
Oct 24, 2022 06:07 PM
The limit on automation runs is per workspace, not per base.
Oct 24, 2022 06:11 PM
Oh wow, thanks for clarifying. That’s an even greater restriction than what I thought it was! I’ve edited my post above.
Oct 24, 2022 10:37 PM
I had some conversations with Airtable staff on the issue and they explained:
Oct 25, 2022 02:42 AM
@Adam_Minich of Airtable will need to clarify for us whether this is 5 per second or 5 per minute.
As @kuovonne mentioned above, this is (sadly) per workspace, not per base. @Adam_Minich It might be nice to clarify this on Airtable’s pricing page.
Oct 25, 2022 08:06 AM
Whoops! Fixed that. It’s per second.
Oct 25, 2022 01:01 PM
Thanks, @Adam_Minich! :grinning: Do you think that the pricing page could be updated to let people know that automation runs are per workspace instead of per base?
Oct 25, 2022 02:43 PM
@ScottWorld Sure thing! Forgot to add that to my response. I requested that the pricing page be updated to mention the per workspace limit and/or link to the support article that discusses this limit in more detail.
Oct 25, 2022 04:22 PM