Help

Re: Automatically get files from (protected) URLs using Sripting

4361 2
cancel
Showing results for 
Search instead for 
Did you mean: 
Jonas_Kockx
4 - Data Explorer
4 - Data Explorer

Hey
I wanted to automate the process of fetching files from URLs and putting them in a Attachment filed. I found a script to do so at:

However, the URLs I want to fetch require being logged in into Webflow. So I was wondering if someone knew a method of making sure the URLs are fetching ‘as it was working in your own browser’. In my browser I can simply copy the URLs as I’m automatically logged in to that website.
Kind regards
Jonas

18 Replies 18

Hi Jonas, and welcome to the community!

This is a noble challenge, however, it requires some deep thought to achieve the stated requirement. The deepest of the thought is central to what Airtable requires in an automated process, not so much what users require.

When Airtable is presented with a URL, it uses that endpoint to retrieve a document (like an image, or a PDF, etc). In that process, Airtable has no understanding of a security context unless, of course, it is already established in the context of your browser when performing attachment additions manually.

Making attachments automatically is a completely different security climate; there is no browser and the automated process can only hand-off to Airtable’s API URLs that are publically accessible. And the only option to make this work with attachment URLs that are private is a proxy that does possess the ability to establish a security context with the platform hosting the secure URLs (ergo, the platform possessing the private URLs must also provide a modern API).

I have created this approach many times but it’s complicated. The proxy must mimic (for Airtable’s benefit) document URLs that are (in fact) public. It must do this for a few seconds; long enough for Airtable to make a copy of the target URL and then immediately thereafter, it must invalidate that URL.

The Cat is Out of the Bag

And doing this makes the documents once protected by private URLs, suddenly publically accessible because Airtable attachments are open URLs; if you know the link, you can access the attachments.

@Jonas_Kockx I don’t know about WebFlow in particular, but to do anything like this, you would need to tap into the software’s API.

WebFlow has an API that lets you retrieve any items from your collections, which seems to include the URLs of your files (which is what Airtable needs to create a file in the attachment field).

So you would need to write a script in Airtable that taps into WebFlow’s API.

Alternatively, if you don’t want to (or don’t know how to) write a script, that’s where you would turn to Integromat’s WebFlow integrations and automations with Airtable.

Integromat is a no-code integration platform lets you tap into APIs without writing any code. (Note that I am a Registered Integromat Partner, and that link contains my personal referral code.)

And how would that script hand off a public (non-private) URL such that it could be absorbed into an attachment?

In general, when accessing any API, the API requires the user to authenticate their API call using their API key which lets them access any private information that is only accessible to them. In other words, the process of authentication gives the API call full access to anything that they would normally have access to, as if they were currently logged into their system.

Ok, thank you for your answer! I’ll look into integrating WebFlow’s API in my script!

Hi @ScottWorld,

If I do that WITHOUT any PROXY, should I expose my API-Key into my Airtable-Script-App ?
I believe YES and I don’t want to do that.
So I would have to use a PROXY anyway ?
I believe YES.

Thank you to explain if I’m right or wrong.

Second thought: maybe this kind of PROXY to hide an API KEY is much easier to write / adapt / setup or to rent than the full PROXY Solution but I still don’t know which of Bill’s or Scott’s way should be the best for me on some lightly or deeply different API Use Cases. (maybe both are right for me but, and it’s personal, I don’t use what Bill’s calling Glu-Factories: this is only a personal initial choice.)
I don’t use WebFlow but some other API so I will continue to follow this thread.

Best,

olπ

That’s actually another good reason to use a 3rd-party service like Integromat — because it shields your API key from your users.

As far as I know, using Airtable’s scripting app reveals your API key to all of your users, even read-only users.

Yes.

Thank you @ScottWorld

oLπ

Indeed, this is how APIs work. But you need to address the core requirement of Airtable attachments. It doesn’t matter if your script can authenticate with the target platform and access the data including the private URLs. What matters is what URL will you hand off to Airtable for loading an attachment. If the API retrieves a secure URL and hands it off to Airtable, the attachment process will fail because Airtable itself requires ALL attachments to be publicly accessible.

Once again - my question - how will any process via script or any other means such as Integromat overcome this essential attachment requirement?

Am I missing something?