Diagnose script interactions

I am trying to figure out an issue where attachments in my Airtable records are being deleted - allegedly by me according to the record history - when I am doing nothing of the sort.

I am using Integromat to create an Airtable record from incoming emails - the script includes populating an Attachments field in Airtable with certain email attachments.

I then have a native Airtable automation that classifies these incoming email message records based on their content and updates a Message_type field

That all works just fine, however in many, but strangely not all cases, the attachment then mysteriously gets deleted…

Here’s the record history pane from one of these records (I’ve obscured the data fields)

The Activity history shows the initial creation of the record via API, including an attachment, from Integromat, then the classification step using my Airtable script - in this case is a ‘no match’. All this script does is set a value in a ‘Message type’ field based on sting matching against content in the email body

But then the Activity history it shows the attachment is deleted - by me! But I did not delete it!

How can if work out what is actually driving this unwanted deletion of the attachment?

You’re running into an issue that sometimes happens when trying to add attachments via Integromat. If you search the forum, you’ll likely find other threads on this subject, but long story short, the URL of the attachment you’re adding needs to be a permanent place where an actual document lives online. If the Integromat module you’re using provides a URL for a file that’s served up on the fly, that will often lead to the situation you’re encountering: the file is added, then it disappears moments later.

Could you share more details about the modules you’re using in Integromat? We can probably provide more specific guidance based on that.

Hi Justin, I think you actually helped me with the classifier Airtable automation I’m now using, so I guess this is now an example of no good deed going unpunished… Sorry about that, and thanks for all your help :grinning:

The relevant piece of my Integromat flow looks like this:

I’m taking file attachments from incoming emails, filtering for PDFs only, saving them to Gdrive, then adding them into an Attachments field in an Airtable record. I don’t actually need the files in Gdrive at all, but this was the only way I could figure out how to handle the scenario where there are multiple file attachments

Since its Gdrive, there is a stable URL where these documents do actually permanently live, and since this does work some of the time, I figure this cant be a permissions issue on the Gdrive…

A potentially relevant bit of info is that this behaviour seems to be inconsistent - the attachment deletion seems to happen a bit more than 50% of the time, but not every time.

Are you possibly mistaking a deletion with cases that the attachment seemed to have occurred (should have occurred) but wasn’t actually sustained?]

This is a well known issue that lots of people have experienced, but this would also be the first known case where Integromat was involved.

Not from my experience. I ran into this a while ago when I was using Integromat more often. Only recently did I figure out the right combination of modules to make it work reliably. It’s a minor pain, but it does the job. I’m tied up at the moment, but I’ll post an update later with details on the solution I found.

Good to know. For more than a year this error condition has been discussed in various threads and one common element has been that integration systems like Zapier and Integromat never exhibited these issues. Even @ScottWorld was quick to point out that Integromat suffered no such problems.

If Integromat can be used to demonstrate this problem in a repeatable way, it may be explained if they chose to update their platform to use the REST API instead of the original partner-developed code supported through Airtable prior to the REST API being made available.

Looking forward to your next post.

I’d love to see the workflow that @Justin_Barrett created for this issue!

I haven’t personally experienced this issue myself, but I wonder if the attachment hasn’t been fully uploaded to Google Drive before you try to insert the attachment into Airtable, especially if it’s a larger attachment? Maybe try adding a “Sleep” module after the Google Drive module to pause the workflow for a few seconds?

That’s just my guess, but I’m not sure.

I fully tested this hypothesis and many times. Google Drive will not return a URL until the document has been fully instantiated. Ergo, it’s impossible to have a URL to a document before the document has been fully saved to Drive.

This approach has been tested 60,000 times by me; it has no bearing on the success rate. We’ve also tested a framework that delays and retries up to ten times with no known impact on success rates.

All of my tests suggest there is a failure occurring in the Airtable platform when attachments are introduced from the public API.

Since this issue has never presented itself in Zapier (or Integromat as far as I know), I can only assume that neither of these platforms utilize the public [REST] API that we all must use.

This is great information to know! Thank you!!

Very interesting.

Sorry for the delay. Here’s the relevant part of my scenario (the full thing is the mess I was remembering earlier; this part is actually pretty simple):

Even though the “Upload a File” module does provide a sharing URL for the uploaded file, it’s not persistent enough for Airtable’s taste.

I think that Google has been updating its Integromat modules, because I don’t recall seeing the “Get a Share Link” module when I ran my earliest tests last year. I could be wrong. However, when I built this test scenario a little over a month ago, I discovered it, tested it, and subsequently rejoiced. The link that it provides is sufficiently persistent, and the file doesn’t vanish from Airtable shortly after arriving like a trick-or-treater through a convenient trap door that’s triggered by the doorbell…wait, what was I talking about?

Here’s the setup for “Get a Share Link” if it helps.

In the Airtable module, the attachment field setup looks like this (I’m using my own custom filename created in Airtable, but you could probably use the native file from Google if you want):

@Richard_Foxworthy In this case, my gut says that the “Get a Share Link” module would need to go inside the aggregation structure. I haven’t used Integromat aggregators yet, but I’m guessing your current setup aggregates share URLs from the “Upload a File” step. Switch it to pull the share URLs from the “Get a Share Link” module instead.

Hi @Justin_Barrett and @Richard_Foxworthy

You are right Justin, Google did an upgrade with GDrive and you now need the first two modules, in tandem, to work.

Hope this might help!

Mary K

Thanks @Justin_Barrett and all, I will apply that second Gmail step which hopefully will work.

I’ve found a further variation on this that may be related - another of my file attachment scenarios also displayed this behaviour - guess I’m just lucky - but the Integromat scenario is different.

In this second case, I’m parsing a file download link from the email message body, then inserting the link into an Airtable Attachments field. This works often enough, but also fails regularly.

Airtable support had this to offer:

That activity in your revision history typically means Airtable was unable to process the attachment. So although it looks like it was uploaded and then deleted, it’s actually the case that we never had the full file. This could be due to the URL not being a publicly available URL (e.g., it requires a login to access), or the Integromat integration is sending URLs in a format we can’t parse.

Would you mind confirming that the URLs you’re sending are upload-able using the UI when you access the base itself? For example, if you click the “+” in an attachment field and select the the URL option below, does the file upload successfully that way?

And sure enough my example URL failed in the Airtable upload tool, even though it succeeds in my browser. I’ll re-engineer my Integromat flows to include an explicit save-file-to-GDrive step followed by the Get Share Link that @Justin_Barrett identified - -hopefully that will cure both my problems!

thanks again everyone for help on this.

2 Likes

Unfortunately this additional Gdrive step has proven not to solve my issue.

I’m now using an Integromat flow like this:

I have today found an example where the generated Gdrive Sharelink URL does work in the Airtable manual file upload tool, but it fails in the same way as before when the same URL is used from Integromat.

I’m using the ‘Web Content Link’ URL from the Integromat ‘get a share link’ module - this is a Gdrive URL with &export=download appended to the URL.

Which seems to suggest that this issue may actually be related.