Attachment download inconsistant

[Valerian_Leber1]

Hi
We use a script to create airtable records through API.
Attachement (pictures) are stored on a public webserver. The URL is passed by API request and Airtable dowload the attachement on his own cloud to be available in the attachment field.
This solution worked properly for a year maybe.

Since december 2019 we get some “broken files”. It seems that Airtable fails at downloading the attachment ant the attachment appears whit a generic icon instead of a picture preview. The picture can’t be downloaded properly.

It is very inconsistant : we have records with for exemple 4 pictures in the attachment field and one of them appear broken. Deleting the record and recreating it the same way may fix the record. The picture is always available on our webserver.

As we have a long history I could determine that this problem began to occurr on last december.

I read some similar issues on the forum that have been fixed by “patching” the airtable with the support

Can I have any help on this issue?

[Bill_French]

Phil_Deschaine:

Then when you look at the history of the record, it confusingly shows you uploading and deleting an attachment.

Yep - this is also a poor aspect of the API. But, even accounting for bad URLs, good URLs fail as well from time-to-time.

Phil_Deschaine:

confusingly shows you uploading and deleting an attachment.

It’s more nuanced that this. The history is reflecting on a process that failed to make a copy of the document at the provided URL. To be more clear - the process takes your URL, makes a temporary copy of the document, and makes a permanent copy which is tantamount to pushing it into the Airtable CDN (as far as I know).

It is this final step that seems to fail for some good URLs and all bad URLs.

[Florian_Verdon2]

Hello everyone,

I am facing the same problem while uploading files and this is very annoying.

Before trying every possible solution blindly, I would like to check if someone found a workaround for this ?

• Does the file location make a difference ? (I am using Google Drive but could host files on S3 or self-host if it solves the problem)
• Custom apps, scripts or API are all subject to that intermittent failure ?
• Any other suggestion ?

@Bill.French, I know you did a lot of experiments on this problem. Are you still using you retry strategy to handle that intermittent error ?

Thank you

[Bill_French]

Florian_Verdonck:

Are you still using you retry strategy to handle that intermittent error ?

Yes, but retries typically fail as well. I have 30 clients who regularly see this issue bubble up from time-to-time.

[Valerian_Leber1]

Hi everyone,

On my side I tried reducing the size of the picture before upload, but no this has no significant change.

However I observed a strange behaviour : when I add through API a bunch of records with attachments, lots of them have broken attachments (picture icon without thumbnail and broken link of data).

If I delete these records and retry to add the same records through API, most of the records will be OK. I suspect that once Airtable hase cached once the link, everything goes better.

Retries could be a good strategy, but silence frome Airtable on this issue is really desapointing.

For the record, this wasn’t an issue a few time ago and started to be totaly broken on december 2019 (I have airtable bases showing this chronology…)

[Bill_French]

Valerian_Lebert:

this wasn’t an issue a few time ago and started to be totaly broken on december 2019 (I have airtable bases showing this chronology…)

As do I. I reported this initially on Dec 19, 2019 and to this day, the errors are intermittent and difficult to reproduce unless the platform is in a state of mind to fail.

URL to Attachment

My attention to this issue began in Nov 2019 and by Dec 19, 2019 we noticed a steep regression (i.e., many more failures).

[Ockenden_Tech]

@Valerian_Lebert, @Bill.French, @Florian_Verdonck, I can report that I do also observe similar very strange behavior with Airtable attachments seeming to be uploaded, and then automatically being deleted seconds later. This happens regardless of which application is being used to drive the API, whether third party integrators like Zapier/Integromat, Airtable automation or Airtable script.

I tried @Florian_Verdonck’s method with Airtable automations (thanks for the tip @Florian_Verdonck!), and that also suffers from the same issue, and worse, gets stuck in an infinite loop because it just keeps running ad infinitum with the automation continually adding the attachment, and then it getting deleted seconds later, so it just keeps going and going.

What I have found does work though is to wait for some time, e.g. half an hour or an hour, and then to process them in bulk using the Convert URLs to attachments script. For some reason, there seems to be some throttling of attachment upload bandwidth, hence why one needs to wait a while.

Any further updates from Airtable on this issue?

On a side note, I am curious as to whether you all have found a way to add in the file names when doing the uploads either via automations or via Airtable scripts.

[Florian_Verdon2]

Hello @Ockenden_Tech,

I am not sure if this issue occurs regardless the source of the file.
My feeling is that it happens “more” when using Google URLs but this is a feeling and I never did real tests about this.

You may try to create a function that “proxy” the source file to check if it increases the success rate.
I would suggest two options for this :

1. Without code : Create an Integromat Scenario with a Webhook as trigger, Download File and the return a webhook response with the file in the body. (so you return the file FROM integromat, and not the original source)

2. Low-code : Create a Pipedream (or AWS Lambda etc) function that do the same thing (probably cheaper if huge volumes)

Side note : yes it is possible in Script and Automations (with a script block). Update a record action do not allow specifying the filename (it always takes the original filename I think).

[Valerian_Leber1]

Hi @Ockenden_Tech

Thanks for the information. I did’t know the Convert URLs to attachments trick.
I just tested it, and I have the same errors. See screen attached : some empty attachment fields and some “broken pictures”:

From my point of view, I observe an improvement in general on this bug (more attachment get uploaded without erros than before). But maybe it is also because I changed some process on my side (resizing the pictures to smaller files).

However it is still a pain that attachment upload is not 100% reliable.

Missing attachment can be fixed with your script as the stript will retry to upload them. But “broken ones” (when you don’t see the thumbnail) are really difficult to handle.

Valérian

[Bill_French]

Florian_Verdonck:

My feeling is that it happens “more” when using Google URLs but this is a feeling and I never did real tests about this.

I did - it’s clearly more likely with Google-based resources, but it is not limited to Google URLs. I’ve proven this with a variety of endpoints suggesting the flaw is in Airtable.

Ockenden_Tech:

and worse, gets stuck in an infinite loop because it just keeps running ad infinitum with the automation continually adding the attachment, and then it getting deleted seconds later, so it just keeps going and going.

My approach has an additional retry test to ensure it doesn’t go on forever. This sort of API behaviour will negatively impact only your users, not Airtable themselves.

Ockenden_Tech:

This happens regardless of which application is being used to drive the API, whether third party integrators like Zapier/Integromat, Airtable automation or Airtable script.

I have seen this issue with the Airtable API but have performed few tests with the SDK so that avenue is inconclusive for me.

Ockenden_Tech:

What I have found does work though is to wait for some time, e.g. half an hour or an hour,

Indeed, but let’s be clear - waiting does not remedy the issue in some unknown way. Waiting only influences when attempts are made. Running these processes earlier is no different than running them later. It’s like a slot machine - do you think waiting a half hour will change the bet outcome? :winking_face:

[Bill_French]

Attachment download inconsistant

And BTW - the polarity of this title seems inverse. The issue is really “uploading”; ergo - creating a new attachment is an upload process into Airtable’s CDN. I tend to think of my local file system in terms of “downloads” in the same manner that browsers place files retrieved from the Interwebs in a “Download” folder.