I’m heavily caching Airtable data on my own server via the API, and if Airtable could notify my server when a record is updated, created, or destroyed, that would help me invalidate the cached data and re-request it.
Ultimately that would mean less calls from my server to the Airtable API, which I imagine would please Airtable.
Welcome to the community @Puneet_Kaura, and thanks for your well-intentioned interest in filling a high demand feature gap in Airtable.
In as much as Airtable’s [current] feature set contains no such ability to configure webhooks based on events affecting tables, suggesting any third-party solution is able to do this is not being entirely transparent and may set consumer expectations that you possess something that even Airtable is unable to provide.
Until such time as Airtable adds webhooks as an integral aspect of its underlying architecture, we are all beholden to “polling” for changes in datasets and tracking such change states over time. This is not a trivial approach, it requires the API, and consumes unnecessary bandwidth. It’s my opinion that this approach only serves to mimic the essence of webhooks; it is not really “webhooks”. I like to refer to this as “fauxhooks”. :winking_face:
While it does seem like a distant and perhaps unlikely event (pun intended), Airtable really has no choice; it must support webhooks and perhaps not for the reasons you might expect.
While we tend to think that all features are driven by customer demand, the need for webhooks does not fall neatly into demand-driven requirements. Rather, Airtable has a bigger problem - a tsunami of API requests used for many use cases - and every one of them is driven by a polling architecture - i.e., every minute, ask Airtable if anything has changed in a table).
Something’s Gotta’ Give
Imagine 10,000 Zaps and another 10,000 Integromat calls every minute to the Airtable API - each asking the same question across thousands of tables - got anything new?
The vast majority of these API calls are hollow actions - i.e., the request comes back “no” - nothing new here. All of these calls are fundamentally wasted bandwidth, wasted server cycles, and most important - wasted distractions that Airtable must respond to, otherwise, the API itself would not function.
If we extrapolate the likely trajectory of this architectural underpinning, sustaining this climate is not only a bad idea, it will result in continual demand for vastly more computing power. We don’t have to look far or wide to find examples where companies have ignored the benefits of modernized event-driven architectures and suffered serious consequences.
This is why I believe we will see webhooks [eventually] and they will change everything, including the way we use glue services, the way we build systems using the API, and the possibilities for new and seriously innovative apps that will emerge with Airtable under the covers.
Spot on. I’m not interested in “faux hooks” either. It has to be integrated into Airtable to be efficient, because 3rd parties are always stuck with polling the Airtable database (which is inefficient if they do it, say, once a second, to even compete with the speed of a true webhook).
I am presently using Zapier in order to let my only-average-computer-savvy father manage his portfolio web site. Each time we decide something is worthy of going online, we check the field “show on web” then Zapier sends the webhook to netlify to go rebuild. But this isn’t a very good system! Zapier’s options for Airtable are limited to “when a new row shows up” which I think does not account for “when a row has been removed” The points Bill makes are all too real. I know that when we’re in there mucking about we don’t need the site redeploying over and over so even a simple option for debouncing the hook would be welcome too. As a user of Airtable, I know that in order for Airtable to hum smoothly the servers need to not be bombarded, and in order for that to be a reality, we need webhooks.
I believe I saw someone here in the forums creating a Block for the Airtable block contest and it may be a block only available for PRO (only they can create blocks) users.
Just finished creating a simple firefox extension that intercepts the requests done in the background for the most common operations: when a cell is updated, either text, attachments or relationships or when a row is added or deleted. This only grab the request body and sends it to the webhook url set on the extension preferences. This does not make any request to their API endpoints
I know this is not the best and some people may require for this to happen without every user that uses their bases to install it but I think it can work for someone. Here is the repository for the extension https://github.com/joseadrian/airtable-firefox-extension
This is the one and only feature I need to make AirTable my solution to everything. As mentionned by so many people before, I am now coding a “ping anything new ? pong probably not” webhook workaround that will ultimately consume all my API request quota when it could be so much less data consuming. I hope this will become obsolete very soon !
Thanks for sharing this into a climate where we are desperate for webhook support from Airtable. Hopefully, this and all the other polling attempts to simulate a true webhook architecture will be obsoleted soon.
BTW - this approach seems to require a complete duplicate of every record to know if anything has changed. This could impact performance and costs especially in tables where fields are abundant.
I’ve used a different approach that’s similar to this but only that the logic is near the same. Where I deviate is the underlying nature of change detection and persisting historical records. Instead of making a JSON copy of the record and saving into a field, I simply compute a UUID3 hash and store that in a single text field. It’s far smaller and updates are very fast.
The polling process simply recomputes the hash during the change test process and compares it with what’s stored - if the same, no changes occurred. Otherwise, a new hash key is a certainty that something changed in that record since the hash was last computed.
In its simplest form, this is a flawed model if (and only if) you need to perform any logic based on the historical value of any given field. But even this can be overcome by establishing hash keys for certain or groups of fields.
Thanks for the reply! I’ll add that note about record size to our README.
In its simplest form, this is a flawed model if (and only if) you need to perform any logic based on the historical value of any given field. But even this can be overcome by establishing hash keys for certain or groups of fields.
This is a requirement for us. I’m curious about your solution, wouldn’t you need to reverse the hash if you wanted to compare to historical values? in that case, wouldn’t it be easier to store the previous / historical value?
Indeed, there are use cases where you need a versioned record in its entirety. But, if all you care about is (a) the record has changed, and (b) replace the old with new - you can save yourself a lot of bandwidth and quota consumption.
For most applications involving a sync process, the historical values are not a concern so it may make sense to think about a full-on versioning of the data the exception, and the simpler requirements the rule - just sayin’ …
Easier, yes, but potentially problematic in large data sets.
Hey all—another +1 here for real Webhooks in Airtable. Like many of you I’ve sifted through the available options but haven’t found anything worthwhile.
I spent some nights building something that works for me. It’s a “faux-hook” that polls once a minute and notifies the changes. It does store the last known data, so if you feel uncomfortable about that let me know and I can look into some way of not actually saving the data.
