Show live data in a web/hybrid application

@Alejandro_Sanchez,

Real-time updates into web pages require a definition since [presently] Airtable doesn’t provide webhooks or event handlers in the API. These are necessary to create truly responsive integrations. Lacking these features, we must poll for changes.

While integration services like Zapier claim to support an event-like behavior (i.e., a new record is added and near-instant updates are processed), they are only mimicking an event-like behavior. I believe Integro requires you to add date/time fields to expedite the polling nature of this objective by limiting tests against tables based on recent polling intervals.

When you say “live data”, you should qualify this by narrowing down the latency your users are willing to tolerate.

• < 5 minutes
• < 1 minute
• < 10 seconds
• < 1 second
• < 100 milliseconds

Many users think they are getting live data when it’s never older than 5 minutes. Others, such as the transit center in Los Angeles hold my company to a much faster standard when a bus driver is attacked - 500 milliseconds to be exact. A camera has to recognize the attack and report the event in less than 1/2 second to be considered “live data”.

In any case, it’s a good idea to consider the latency tolerance before choosing any specific implementation approach.

Let’s assume by “live” you mean updates should appear on the web page in less than a minute.

In this case, you could use airtable.js, but it would require a relatively complex web page that uses a polling interval established with a timer loop in the javascript running in that page. This is required because web pages are pretty dumb - they don’t refresh data unless nudged at an interval that matches the minimum latency tolerance for your use case.

This is not ideal because timer loops – in general – are fairly brittle machinery for the web. They are not impervious to each client’s available memory and other environmental conditions. And with each look, Airtable.js must make a new call to the Airtable API to see if anything has changed in the target base/table.

Ideally, you need a real-time architecture for data updates and synchronization to the browser. Indeed, in a real-time climate the objective is to simply synchronize the data between the table and the browser web page. This is best achieved with sockets - a technical concept that sustains an open connection between the browser page and the database.

Firebase is famous for this capability and I use it often to achieve near-instant replication of data from servers and disparate services. Another technology that makes this possible is PubNub, a real-time data streaming network. Depending on the nature of the application you are building Firebase and PubNub may be overkill, but they are both near-free (and free at lower tiers of volume).

In my view, the most reliable and less brittle way to achieve what you want to do is to try to eliminate as many moving parts as you can. Airtable.js certainly achieves this - it is a single client-side SDK that talks directly to the Airtable API.

But it requires all the logic to be running on the client so you will likely encounter some issues with security, processing performance, and the brower support headaches (i.e., different browser vendors support javascript differently). When faced with these challenges at the client, developers often escape this madness by using server-side script - i.e., the client browser is always listening to the server for updates, and the server is always polling Airtable for new data. Spreading this architecture over multiple systems (Zapier, Server, Client) is not ideal - the complexities create many points of failure. Best to unify the process architecture when you can.

Feel free to describe your desired latency and scale of the app and I’ll put a finer point on this topic.

Hey @Alejandro_Sanchez :wave:

There is another great tool that can help you: Ably the pub/sub messaging platform with a suite of integrated services to deliver complete realtime functionality directly to end-users.

Coincidentally, we just published a new post where we show how to use Airtable as a database to store realtime messages using a chat group app built with Vue.js as an example.

Here is a preview of the architecture:

Let me know if you have questions or there is anything I can help you with.
Disclaimer: I work for Ably :grinning: