Diagnose script interactions

[Richard_Foxwort]

I am trying to figure out an issue where attachments in my Airtable records are being deleted - allegedly by me according to the record history - when I am doing nothing of the sort.

I am using Integromat to create an Airtable record from incoming emails - the script includes populating an Attachments field in Airtable with certain email attachments.

I then have a native Airtable automation that classifies these incoming email message records based on their content and updates a Message_type field

That all works just fine, however in many, but strangely not all cases, the attachment then mysteriously gets deleted…

Here’s the record history pane from one of these records (I’ve obscured the data fields)

The Activity history shows the initial creation of the record via API, including an attachment, from Integromat, then the classification step using my Airtable script - in this case is a ‘no match’. All this script does is set a value in a ‘Message type’ field based on sting matching against content in the email body

But then the Activity history it shows the attachment is deleted - by me! But I did not delete it!

How can if work out what is actually driving this unwanted deletion of the attachment?

[Bill_French]

Martin_Kranz:

I am also in touch with support about this, who respond the same way

I was referencing your own response from Airtable support which clearly indicates they have not studied the community’s conversations. No reasonable person could digest all that has been written about this topic and conclude that it is because we failed to provide publicly accessible URLs.

[Justin_Barrett]

Martin_Kranz:

PS apologies, i was under the impression you’re community moderator

No worries. I can see how the “Community Leader” label might create that impression, but it just means that a) I post a lot, and b) my ramblings tend to make sense more often than not. :winking_face: The only official mods are Airtable staff, who have the “Airtable” label by their names.

[Richard_Foxwort]

Just adding a little bit more info to this thread re an Airtable support recommendation that does not fix this issue.

Airtable support suggested modifying the GDrive link to remove &export=download from the end of the URL string - no idea why they’d suggest that since AFAIK this is the mechanism to tell GDrive we want a direct file download that skips the GDrive file viewer.

I tried it as follows:

Perhaps unsurprisingly, this does not solve the problem.

[Bill_French]

Richard_Foxworthy:

no idea why they’d suggest that since AFAIK this is the mechanism to tell GDrive we want a direct file download that skips the GDrive file viewer.

It’s actually a little more nuanced than this, and this article explains all of the different document access URL formats that Google supports. But setting this aside, you certainly need to provide Airtable with a publicly accessible URL that is not subject to an iFramed web viewer and I believe in all your tests this has been the case.

This is my point though - there’s a preponderance of data indicating a serious issue in the document attachment feature and it seems they just keep pushing the support narrative that we are not instrumenting the links properly or the links are not openly shared, or Google Drive must have been down. I think they need to appoint a special counsel on this one - someone who really owns the issue.

[Richard_Foxwort]

Hi @Bill.French

I think they need to appoint a special counsel on this one - someone who really owns the issue.

I responded to Airtable support about how their suggested fix to remove &export=download from the URL does not work for me, and how this issue needs more serious attention, and got this rather encouraging reply:

Peter here, thanks so much for following back up about this. We have a few engineers on our team who are actually actively looking into this very issue, and I’d be happy to keep you updated about how their efforts progress. Hoping we can have this sorted out soon.

Let’s hope we see a fix sooner than later.

[Bill_French]

Richard_Foxworthy:

got this rather encouraging reply:

Good to hear.

I first reported this with 7 pages of documentation, repeatable tests, and some ideas exactly 360 days ago today. Support’s response was almost identical. I truly hope they mean it this time.

[Martin_Kranz]

I have received the same message from the same Peter this week, also shared a example base with them so they could see it fail in real life. let’s hope they do something with it!

[Richard_Foxwort]

@Bill.French @Martin_Kranz here’s a little bit more from Airtable support:

Our engineering team has been taking a look into this issue, and are trying to get to the root of why these attachment uploads are failing (while it looks like the attachments are uploading and then getting deleted, the underlying behavior is just that the attachments are failing to upload. i.e. what you’re seeing is what normally happens when an attachment fails to upload).

Just to keep you on the same page – Airtable uses the service Filestack in processing attachments, and our working hypothesis is that Google Drive limits downloads from Filestack, which causes transient failures like those observed here.

The (relatively) good news is that when these files fail to upload, you should be able to simply retry uploading them at a later date. That said, we do not have very good signaling for attachment failures; the attachment value is just removed from the cell.

Also, to be up front, while our team is taking a look into this issue to fully understand it, it isn’t at this point clear when we might be able to have a full fix in place.

I’m not sure if they are suggesting there is a specific issue between Gdrive and Filestack - in which case swapping out Gdrive and replacing with Dropbox etc might help? And also, given that Airtable’s file upload is via Filestack, I guess this could mean that the problem is actually with Filestack rather than with Airtable per se?

[Bill_French]

Richard_Foxworthy:

working hypothesis is that Google Drive limits downloads from Filestack, which causes transient failures like those observed here.

Well, of course they do. All services have limits and likely watched very closely by Google. In fact, they probably have some quotas that are triggering when the Filestack system looks aggressive.

Richard_Foxworthy:

in which case swapping out Gdrive and replacing with Dropbox etc might help?

There have been tests that show this issue occurs with other services as well, but they may be blocking Filestack too. Besides, ruling out Google Drive is not an option; there are 100 million businesses using it.

Richard_Foxworthy:

The (relatively) good news is that when these files fail to upload, you should be able to simply retry uploading them at a later date.

There is some validity to this idea, however, immediate retries almost always fail; I’ve tested up to ten retries without success. And to be clear - AIRTABLE should implement this at the service instead of asking thousands of users to create a patchwork of crap to overcome this issue. We didnd’t pick Filestack, after all, eh?

Richard_Foxworthy:

I guess this could mean that the problem is actually with Filestack rather than with Airtable per se?

Perhaps, but ultimately, the issue is with Airtable because we have no standing with Filestack.

[Richard_Foxwort]

And here is another bit of info from Airtable support responding to a question if this issue is GDrive specific:

That’s correct – our understanding is that this issue is GDrive specific, so we wouldn’t necessarily expect this issue to occur in Dropbox (or another similar service). Please do let us know whether you have success with a different service (and sorry for not having thought to propose that kind of solution sooner).

Completely agree with you @Bill.French that buck must stop with Airtable, even if it’s specifically a Filestack <–> GDrive issue, but I intend try using a different cloud file system for now, and see it it alleviates the issue for me.