Help

Save the date! Join us on October 16 for our Product Ops launch event. Register here.

Re: Automatically get files from (protected) URLs using Sripting

3271 4
cancel
Showing results for 
Search instead for 
Did you mean: 
Jonas_Kockx
4 - Data Explorer
4 - Data Explorer

Hey
I wanted to automate the process of fetching files from URLs and putting them in a Attachment filed. I found a script to do so at:

However, the URLs I want to fetch require being logged in into Webflow. So I was wondering if someone knew a method of making sure the URLs are fetching ‘as it was working in your own browser’. In my browser I can simply copy the URLs as I’m automatically logged in to that website.
Kind regards
Jonas

18 Replies 18

Yes @Bill.French and @ScottWorld,
this is the main problem to solve anyway !

Many Thanks,

olπ

@Bill.French @Olpy_Acaflo

Ahhh, I see, that’s a good question! And something I overlooked above. It definitely depends on the API. For example, Google Drive has the option to allow access to a private URL, as long as you know the private URL.

I’m not familiar with WebFlow, so if it doesn’t operate in a similar fashion, then some intermediate steps would be needed in Integromat to temporarily transfer that file into a publicly-accessible space (such as Google Drive) where it can temporarily gain a public URL.

So, an Integromat scenario could look like this:

Authenticate with WebFlow > download the file to Google Drive > send the Google Drive URL to Airtable > delete the file from Google Drive.

LOL! Finally, I got through. :slightly_smiling_face:

Actually, it depends on the visibility options of “private” URLs in the target document platform.

Ergo, it’s not really private nor is it secure (at least not in the sense or spirit of the target document platform). It is private and secure if your definition includes links to documents that are openly accessible by anyone in possession of said link. :winking_face:

Ergo - you need a proxy. But this definition of a proxy likely violates the spirit of why the documents are not available as openly accessible URLs in the first place. Perhaps this is not really a requirement, but we have to assume it is because documents are typically secured for good reason.

Your suggested process is missing a very important step -

… it must also set the security settings for the downloaded file in Google to “Accessible to anyone with the link…”.

What you’ve described here is similar to what I described in my first response. Can Integromat perform all these steps? Perhaps it can. I’ve been fortunate enough to get hired many times to create proxies like this because it’s very difficult to achieve this process without precise timing in a no-code environment.

One thing to remember - any Google Drive document with the security setting “Accessible to anyone with the link…” is also discoverable by anyone in the G-Suite domain for which the account exists. And by “discoverable” I mean searchable, viewable, copyable - it (by definition) is a wide-open document to everyone in the domain even if they do not possess the URL.

To be clear, in my approach, I do not instantiate “intermediate” temporary documents in Google Drive that are exposed publicly or at the domain because it increases the security attack surfaces by a factor of 3 and also broadly publishes the document to the organization’s users.

The most secure way to do this while satisfying the spirit of the secure document URLs is to established a signed URL that is accessible only through a webhook in Google Apps Script. This makes it possible to host (for a few seconds) a document that can be retrieved, but which doesn’t actually exist in Google Drive.

I don’t think this is the case any longer. Indeed, use of any third-party API in a script requires that script has knowledge of a security context of some type. However, isn’t this how you defend against overtly making API tokens and other login credentials difficult to view?

Yes, Integromat has the ability to change permissions of Google Drive files.

Cool. And can it reverse the permissions and/or remove the file when (and only when) Airtable’s attachment process finishes?

Yes, it can do both things.

Excellent! Someone should publish that use case because it’s very costly to script.

Actually, even better, Integromat can temporarily download files into its own cache while running a scenario… so you could eliminate Google Drive altogether and just download the attachment into Integromat, upload it to Airtable, and it will be instantly deleted when the scenario is finished.