Soon You'll Need a CDN

[Bill_French]

One of the best features of Airtable is the ability to drag images, files, and even videos (i.e., digital artefacts) right into an attachment field. When you do this, Airtable makes a copy of your image and stores metadata about the image into your base. It also quietly moves the copy of your image to its content delivery network (CDN). The URL to your image has been openly accessible providing a wonderful way to share access to your assets For almost eight years, this has been possible but that’s going to change in significant ways later this fall.

Long Time Coming

I predicted this would happen in 2019. They warned us then and they are giving you almost a year to make plans. A lot of folks complained here, but this change is most certainly going to happen and it is in everyone’s best interest.

Now that Airtable has officially announced new specifications for its CDN, any solutions that depend heavily on images that need to be accessed outside the Airtable UI will probably require that you host them in your own CDN.

What’s a CDN?

Cloudflare provides one of the better definitions:

A content delivery network (CDN) refers to a geographically distributed group of servers which work together to provide fast delivery of Internet content. A CDN allows for the quick transfer of assets needed for loading Internet content including HTML pages, javascript files, stylesheets, images, and videos. The popularity of CDN services continues to grow, and today the majority of web traffic is served through CDNs, including traffic from major sites like Facebook, Netflix, and Amazon.

Airtable, of course, has its own CDN to provide access to the images and files you upload to attachment fields. The intent of this CDN is for it’s paying customers; not as a global hosting server. If you need to host your Airtable-bound digital artifacts, you’re going to need to essentially replicate Airtable’s CDN, so you might want to read on.

Thanks Google Drive!

One might think that Google Drive, or other services such as DropBox or Box will host these files for me, but that could be a mistake. These services don’t want to be your CDN any more than Airtable does. They include ToS clauses that allow them to throttle open access to your documents despite being fully shared to the open Internet.

One Approach

There are many ways to create a CDN for your attachment files. In fact, Cloudinary is an ideal ready-made CDN that also provides many features for sizing and rendering images in custom formats and styles. However, it’s not a magic bullet and the pricing might be a little steep depending on your needs.

The complex part requires that you create some sort of an integration that pulls (or pushed) your attachments from Airtable (perhaps using the API or script automation) and upload yet another copy into your CDN environment. If you have a serious commitment to attachments, this could be a sizeable challenge. There’s also the issue of keeping your CDN updated in near-real-time, a key requirement that is more easily addressed with Firebase.

These complexities may not be easy to build depending on your selection of a premade SaaS CDN solution. And, you might also get lucky with Zapier or another glue-factory approach. Experiment a little and see what works best for your situation.

I have already helped a few friends design an approach that uses Firebase, a Google Cloud Platform service that is very fast and provides a generous free tier and an integrated CDN.

WARNING - CDNs are not universal in features. Some are globally distributed while others provide localized servers in your country. The costs vary greatly as well. And it also depends on your hosting and delivery requirements.

[Olpy_Acaflo]

Bill.French:

One of the best features of Airtable is the ability to drag images, files, and even videos (i.e., digital artefacts) right into an attachment field.

Thank you Bill for documenting this subject a bit.

For “big projects”, I had never imagined delegating a CDN role to AIRTABLE since it would have been a design flaw from the start, it seemed logical to me, and more so since you had warned (as well as many other things) that one should really not design a project with such a starting bias.

My little prototypes did collect the URL of an Attachment Field (Image, PDF, JSON file) to consume it in another little prototype “built on top of airtable” but they are only little prototypes that I’m the only one to consume: I already knew from your previous warnings that it wouldn’t last.

Storing the References of Attachments housed in a real CDN (not a DRIVE) in my Tables instead of storing the Attachment itself is what I used to do before I discovered the ease of making it simpler in Airtable, and before I discontinued this practice inspired by your warnings.
As long as I can do it simply myself by means of my keyboard, my mouse, in my smaller bases.
To satisfy a programmatic, automation approach, I expect to have to struggle a bit with Script and CDN API but I have done it before and will do it again.

What bothers me a lot though, is the depreciation of the visual comfort of the Attachment Field in the AIRTABLE UI!
Imagine a FIELD populated with URLs of CDN instead of pretty images or PDF or .txt, .json files!
The loss is there.

I have imagined a possible solution:

Proxy: definition: the proxy of a Media File (Image or Video) is an Image compressed to lighten it while keeping it acceptably readable and manipulable.
This precision, although incomplete, seems to me sufficient to read and understand what follows.

Let’s imagine that for each image stored in a real CDN, I can get the URL of its Proxy because the CDN offers this possibility in its Pricing Plan.
This is a pure hypothesis but I have already seen it in very professional Media Storage & Management products rented by Companies.
Let’s imagine that by Airtable Script, I can query the CDN API to:

• List the Proxy Names stored in the CDN;
• Filter the Result by query or by parsing the response;
• Get the URL of the filtered Proxy based on its Name;
• Upload this Proxy in the Attachment Field of the Record whose Name (Primary Key Field) matches the name of the Proxy;
  (this is only a simplistic example without batch process, without exploitation of an index neither on CDN side, nor on Airtable side).
  Wouldn’t that be cool?
  Well, no, because it would only work randomly, since this feature of AIRTABLE has a frequent bias, very well documented by you in the Community, I thank you again.
  In the past and in the present, AIRTABLE has tried to find the bug and then ended up accepting it without fixing it.
  Let’s not make assumptions about the future but just a probability: it will not work in the near future.

So, what to do ?
AIRTABLE could offer us a new FIELD which would be a kind of I-Frame with restricted features for security reasons.
This Field would offer a URL property.
When we paste the URL of an Image Proxy stored in a real CDN, this Field would display a Preview of this Proxy in the table UI.
This is already possible visually using a Field Attachment but then the Proxy would be saved in Airtable, to put it more simply. (see details in your Message).
And this is only possible from my Keyboard and Mouse.
As the new Field I am proposing is a Display Component, to my mind there is no question of storing anything in Airtable’s internal CDN.
You paste the URL of your Proxy in the Field I propose, by hand or by Script,
and you get visually your proxy in your table UI.

Did I make myself clear ?
I’m thinking of drawing a mockup of all this in the current UI of Airtable if my proposal was not disqualified by the experts of the Community
because it would be issued from a biased thought or because a better inspired proposal than mine would replace it.
Fork me here please !
Thanks in advance.

olπ

[kuovonne]

Olpy_Acaflo:

AIRTABLE could offer us a new FIELD which would be a kind of I-Frame with restricted features for security reasons

It isn’t a new field type, but third party apps can do this to a certain extent. I believe Amplify can preview a wide variety of urls for a monthly subscription. My app Low Tech PDF can preview image URLs for a one-time flat fee.

[Bill_French]

Olpy_Acaflo:

What bothers me a lot though, is the depreciation of the visual comfort of the Attachment Field in the AIRTABLE UI! Imagine a FIELD populated with URLs of CDN instead of pretty images or PDF or .txt, .json files! The loss is there.

What a wonderful observation. You are a visionary on par with the engineers at Coda. :clap:

To demonstrate how important this is, at my company we needed a utility app that would blend our image collection for a highway project with the ability to streamline the tagging of those images for more precise adjustments to our AI model. The images are in a secure CDN-of sorts but the database app itself must integrate them for rapid visual inspection and search.

We don’t use Airtable for this for precisely the reason you suggest - you can’t see or inspect the image in the database UI. So while storage by reference is an important design choice, Airtable eliminates the opportunity to enjoy the benefits of integrated image assets.

The app’s UI allows us to search, filter, and group AI data with a thumbnail view of the images and full view if needed. Buttons allow us to nudge the AI model with additional training biases. A custom pack updates our database in real-time which is always updating the AI model in real-time.

Why the purple tint? We use special cut filters on our image sensors combined with infrared light to see inside the vehicle even if it has really deep tinted glass.

How’d we do this? In Coda, a URL has the option of being rendered as a string or as the image it represents.

Olpy_Acaflo:

This is a pure hypothesis but I have already seen it in very professional Media Storage & Management products

Yep, it’s a reality (in Coda).

Olpy_Acaflo:

When we paste the URL of an Image Proxy stored in a real CDN, this Field would display a Preview of this Proxy in the table UI.

This is precisely how they do it but most important, it doesn’t have to be URLs from their own CDN - it supports URLs from any source even computed Base64 values.

[Bill_French]

kuovonne:

It isn’t a new field type

Perhaps, but it is certainly a new field behaviour; a requirement that seems obvious.

[kuovonne]

Bill.French:

it is certainly a new field behaviour;

We’ve had conversations before about wanting to make fundamental changes to how fields work. One of these concepts was to have a clear separation between how a field’s value is stored and how it is displayed. This proposed change to how a url field type is displayed is just another extension of this idea.

Here’s a recap of my thoughts for anyone who might not have seen it before

• simplify the number of field types
• give each field type more extensive display options
• give each field a validation formula (preferably a formula that can reference other fields!)
• give each field conversion rules when non-conforming data is pasted or imported

For example

• a rating field could be a number field type with a validating formula that only accepts integers from 0-10, and display options that shows a user-specified character (emoji) the set number of times
• a currency field could be a number field with a validating formula that accepts numbers in a user defined range and display options that include a currency symbol and user defined symbols for the decimal and grouping symbol.
• a percent field is a number field with formation options that multiplied the number by 100 and tacks on a percent sign
• a phone number field is a text field with special formatting depending on the number of digits
• a url field could be a text field with a formula that optionally prefixes ‘https://‘ to the url if it is missing, and display options to render a thumbnail of the webpage (or the favicon, or a larger preview, or specific meta info, etc.)
• a text field could have formatting options to state if the display should be rendered as plain text, markdown, or html
• a date/time field could display options to always display in gmt, the local user time, or a set timezone determines by the field configuration (ugh, did I really say that? Timezones are confusing enough as is.)
• all of the formatting options should also be available to formula and rollup fields. Thus, a formula field could build a url that is then displayed as a preview of that url. A rollup field could be displayed as a fully formatted html page with line items.

But this would require a major overhaul of lots of existing code, a method for migrating existing data, gobs of testing, and a massive education campaign. As much as I would like to see this, I don’t think Airtable has the resources or inclination to do this right now. (Of course, I would love to be wrong about this guess.)

[Bill_French]

kuovonne:

give each field a validation formula (preferably a formula that can reference other fields!)

LOL! I love this one because y’all told me I was nuts when I suggested this in 2019. I complained endlessly in a very lengthy thread that a field is incapable of having both a value and a formula that optionally sets that value, thus making a “formula” type field necessary and all the baggage that comes with it.

kuovonne:

give each field type more extensive display options

100% agree.

kuovonne:

this would require a major overhaul of lots of existing code, a method for migrating existing data, gobs of testing, and a massive education campaign.

Correct - the ship sailed and is unlikely to ever return.

[kuovonne]

Bill.French:

I love this one because y’all told me I was nuts when I suggested this in 2019.

I don’t recall saying you were nuts. I recall that I was unable to comprehend how this would work. You’ve taught me a thing or two since then. And to be clear, I’m talking two different formulas in my previous post: a data validation formula (to flag and/or reject invalid values), and a display options formula.

Bill.French:

Correct - the ship sailed and is unlikely to ever return.

This makes me sad.

But since I’m dreaming, I’ll expand on the dream …

• simplify the number of field types
• give each field type more extensive display options, including a formula as part of the display options
• give each field a validation formula (preferably a formula that can reference other fields!) in order to reject or flag invalid data
• give default values for each field that works across all platforms (records created in the regular interface, via forms, and via automations / api / scripting / custom apps, etc.)
• give each field conversion rules when non-conforming data is pasted or imported
• give copy/export options for copying/exporting either the underlying data or the displayed data

[Olpy_Acaflo]

Hi @kuovonne , Hi @Bill.French ,

I thank you for sharing these thoughts of yours and your conversations about the FIELDS because I had lost sight of them.
It was worth reading again, as far as I am concerned.
It is a pity that you are not Project Managers at AIRTABLE: AIRTABLE would be even better to say it briefly.

For some time, you have been dealing with :

• Data Validation ;
• Indexed Bases Search ;
• High End Reporting ;
• Automation ;
• Integration ;
• APIs ;
• Scripting ;

and suggesting workarounds, by illustrating your remarks with personal projects in AIRTABLE and/or in GOOGLE CLOUD, CODA.

It doesn’t solve the category of problems that should rather be prioritized internally at AIRTABLE but it inspires a lot, especially when you match them with some theoretical supplements like good practices versus troublesome practices to be avoided based on your professional experience, your contact with Customers, extended Projects but running every day in real life.

oLπ

[Olpy_Acaflo]

Bill.French:

Yep, it’s a reality (in Coda).

Olpy_Acaflo:

When we paste the URL of an Image Proxy stored in a real CDN, this Field would display a Preview of this Proxy in the table UI.

This is precisely how they do it but most important, it doesn’t have to be URLs from their own CDN - it supports URLs from any source even computed Base64 values.

Thank you for unveiling this part of your project.

You look inside the cockpit? :grinning_face_with_big_eyes:
You probably detect Humans asleep or busy on their Mobile Device instead of looking at the road.

In terms of Ai, it’s interesting: a Middle-Man facing your CODA’s UI tweaks the training of an Ai if I understood correctly.
I first had dreamed of this formula while experimenting with GitHub and Datasets offered to learn
because it seemed to me to be the learning solution taking advantage of the presence of a Human in front of the training experience of a model, before discovering it working in some AsAService business Ai-based product that popped-up on my phone’s instagram client.

CODA is used as a UI of a Database host, knowing you a little bit by reading you, in a Firebase Project:
it will have been necessary that I see CODA in this function at your place for me to register, loggedIn CODA and that I started to follow you there too.

Is your CODA querying your database with SQL expressions ?
In its case, how do you get the payload after querying it ?
Or REST queries (where you get JSON queries responses payload) ?

In August 2018, I simultaneously discovered Notion, Coda and Airtable in the same article
but it was something like Airtable I was looking for.
I bet on Airtable, fearing that the 3 would not survive, but only CODA remained in my toDo
partly because you use it by the way.
So I’m going to continue to explore CODA and its Community.

This will not exempt me from proposing the FIELD idea on Coda’s Way that I had just imagined on my side to AIRTABLE Staff but it’s for the act: they must have known for a long time that CODA has this and they must have their opinion on the subject: this FIELD still does not exist in their company despite all the interest that it presents, despite the discontinuation of the CDN role by Airtable.

oLπ