Diagnose script interactions

[Richard_Foxwort]

I am trying to figure out an issue where attachments in my Airtable records are being deleted - allegedly by me according to the record history - when I am doing nothing of the sort.

I am using Integromat to create an Airtable record from incoming emails - the script includes populating an Attachments field in Airtable with certain email attachments.

I then have a native Airtable automation that classifies these incoming email message records based on their content and updates a Message_type field

That all works just fine, however in many, but strangely not all cases, the attachment then mysteriously gets deleted…

Here’s the record history pane from one of these records (I’ve obscured the data fields)

The Activity history shows the initial creation of the record via API, including an attachment, from Integromat, then the classification step using my Airtable script - in this case is a ‘no match’. All this script does is set a value in a ‘Message type’ field based on sting matching against content in the email body

But then the Activity history it shows the attachment is deleted - by me! But I did not delete it!

How can if work out what is actually driving this unwanted deletion of the attachment?

[Justin_Barrett]

You’re running into an issue that sometimes happens when trying to add attachments via Integromat. If you search the forum, you’ll likely find other threads on this subject, but long story short, the URL of the attachment you’re adding needs to be a permanent place where an actual document lives online. If the Integromat module you’re using provides a URL for a file that’s served up on the fly, that will often lead to the situation you’re encountering: the file is added, then it disappears moments later.

Could you share more details about the modules you’re using in Integromat? We can probably provide more specific guidance based on that.

[Richard_Foxwort]

Hi Justin, I think you actually helped me with the classifier Airtable automation I’m now using, so I guess this is now an example of no good deed going unpunished… Sorry about that, and thanks for all your help :grinning:

The relevant piece of my Integromat flow looks like this:

I’m taking file attachments from incoming emails, filtering for PDFs only, saving them to Gdrive, then adding them into an Attachments field in an Airtable record. I don’t actually need the files in Gdrive at all, but this was the only way I could figure out how to handle the scenario where there are multiple file attachments

Since its Gdrive, there is a stable URL where these documents do actually permanently live, and since this does work some of the time, I figure this cant be a permissions issue on the Gdrive…

A potentially relevant bit of info is that this behaviour seems to be inconsistent - the attachment deletion seems to happen a bit more than 50% of the time, but not every time.

[Bill_French]

Richard_Foxworthy:

the attachment deletion seems to happen a bit more than 50% of the time, but not every time.

Are you possibly mistaking a deletion with cases that the attachment seemed to have occurred (should have occurred) but wasn’t actually sustained?]

Justin_Barrett:

the situation you’re encountering: the file is added, then it disappears moments later.

This is a well known issue that lots of people have experienced, but this would also be the first known case where Integromat was involved.

[Justin_Barrett]

Bill.French:

This is a well known issue that lots of people have experienced, but this would also be the first known case where Integromat was involved.

Not from my experience. I ran into this a while ago when I was using Integromat more often. Only recently did I figure out the right combination of modules to make it work reliably. It’s a minor pain, but it does the job. I’m tied up at the moment, but I’ll post an update later with details on the solution I found.

[Bill_French]

Justin_Barrett:

I ran into this a while ago when I was using Integromat more often.

Good to know. For more than a year this error condition has been discussed in various threads and one common element has been that integration systems like Zapier and Integromat never exhibited these issues. Even @ScottWorld was quick to point out that Integromat suffered no such problems.

If Integromat can be used to demonstrate this problem in a repeatable way, it may be explained if they chose to update their platform to use the REST API instead of the original partner-developed code supported through Airtable prior to the REST API being made available.

Looking forward to your next post.

[ScottWorld]

I’d love to see the workflow that @Justin_Barrett created for this issue!

I haven’t personally experienced this issue myself, but I wonder if the attachment hasn’t been fully uploaded to Google Drive before you try to insert the attachment into Airtable, especially if it’s a larger attachment? Maybe try adding a “Sleep” module after the Google Drive module to pause the workflow for a few seconds?

That’s just my guess, but I’m not sure.

[Bill_French]

ScottWorld:

I wonder if the attachment hasn’t been fully uploaded to Google Drive before you try to insert the attachment into Airtable

I fully tested this hypothesis and many times. Google Drive will not return a URL until the document has been fully instantiated. Ergo, it’s impossible to have a URL to a document before the document has been fully saved to Drive.

ScottWorld:

Maybe try adding a “Sleep” module after the Google Drive module to pause the workflow for a few seconds?

This approach has been tested 60,000 times by me; it has no bearing on the success rate. We’ve also tested a framework that delays and retries up to ten times with no known impact on success rates.

All of my tests suggest there is a failure occurring in the Airtable platform when attachments are introduced from the public API.

Since this issue has never presented itself in Zapier (or Integromat as far as I know), I can only assume that neither of these platforms utilize the public [REST] API that we all must use.

[ScottWorld]

Bill.French:

I fully tested this hypothesis and many times. Google Drive will not return a URL until the document has been fully instantiated. Ergo, it’s impossible to have a URL to a document before the document has been fully saved to Drive.

This is great information to know! Thank you!!

All of my tests suggest there is a failure occurring in the Airtable platform when attachments are introduced from the public API.

Very interesting.

[Justin_Barrett]

Sorry for the delay. Here’s the relevant part of my scenario (the full thing is the mess I was remembering earlier; this part is actually pretty simple):

Even though the “Upload a File” module does provide a sharing URL for the uploaded file, it’s not persistent enough for Airtable’s taste.

I think that Google has been updating its Integromat modules, because I don’t recall seeing the “Get a Share Link” module when I ran my earliest tests last year. I could be wrong. However, when I built this test scenario a little over a month ago, I discovered it, tested it, and subsequently rejoiced. The link that it provides is sufficiently persistent, and the file doesn’t vanish from Airtable shortly after arriving like a trick-or-treater through a convenient trap door that’s triggered by the doorbell…wait, what was I talking about?

Here’s the setup for “Get a Share Link” if it helps.

In the Airtable module, the attachment field setup looks like this (I’m using my own custom filename created in Airtable, but you could probably use the native file from Google if you want):

@Richard_Foxworthy In this case, my gut says that the “Get a Share Link” module would need to go inside the aggregation structure. I haven’t used Integromat aggregators yet, but I’m guessing your current setup aggregates share URLs from the “Upload a File” step. Switch it to pull the share URLs from the “Get a Share Link” module instead.