New error importing images from Airtable record into Firebase

[Dylan_Spurgin]

I have a Firebase database that mirrors data from an Airtable view. Functionality is in place that allows an Admin user to trigger an import of the Airtable data over to Firebase. This has been working for a few months without issue.

Recently, I have started to see that my Firebase function that pulls the images from Airtable records into Firebase is failing. I was wondering if anything has changed or if anyone knows any reason for this.

In a Firebase Function I am importing the image from Airtable with code like:
let fromUrl = firebaseRecord.fields['Headshot'][0].url; let filename = firebaseRecord.fields['Headshot'][0].filename; firebaseAdmin.storage().bucket(config.firebase.bucket).upload(fromUrl, {destination: filename})

And I get an error like this in Firebase (URL truncated for privacy):
ENOENT: no such file or directory, stat 'https://dl.airtable.com/.attachments/be2454...b0c70d14/d3...f0/600-uVJ...B3.jpeg'

If I then try to open that URL in a browser, the image is returned and displayed correctly.

Thanks in advance for any and all responses.

[Bill_French]

Dylan_Spurgin:

If I then try to open that URL in a browser, the image is returned and displayed correctly.

Have you tried opening that URL in a browser instance where no security context with Airtable is currently present?

[Dylan_Spurgin]

Thanks for the reply, Bill.

Yes, the image loads even in an Incognito instance of Chrome.

[Bill_French]

Okay - well it sounds like Firebase is unable to access it, so a security context is a likely candidate. However, as near as I can tell, URLs into attachments are always public. I have a note to ask the folks at Airtable why these URLs are openly accessible.

In any case, I’m noodling on your problem - perhaps it will come to me. Your code seems fine.

[BillFrench]

If the process failing all the time Or intermittently?

And how much request volume is occurring?

[Teneo_Team]

It is failing every time. It fails when I try three images consecutively or 300.

[Bill_French]

Teneo_Team:

or 300

I’m confused - “…or 300” what exactly? 300 image extractions?

Firebase is very fast about its HTTP requests. Do you have anything built into your function that paces the requests to the Airtable API?

[Dylan_Spurgin]

Yes, I do at times pull 300 or so images from Airtable. Sometimes I only pull 3. Either way, all of them fail. I don’t currently have any rate limiting built in as it has never been a need. I don’t see that as being the likely cause of this issue though.

[Dylan_Spurgin]

I found that Firebase has removed support for passing a remote URL to bucket.upload. So the solution to this is to rewrite how I import files. Not Airtable related.

[Bill_French]

Dylan_Spurgin:

Firebase has removed support for passing a remote URL to bucket.upload .

Oops! Thanks for saying this - I have at least one client who does exactly this.