Check whether URL exists

[Stefan_Kunz]

Hi,

is there a way to check the validity of a URL from a record? I would consider a URL to be valid if it is reachable and does not lead to a 404 error.

Using Google I have seen examples as the following:

var request = new XMLHttpRequest();  
request.open('GET', 'http://www.mozilla.org', true);
request.onreadystatechange = function(){
    if (request.readyState === 4){
        if (request.status === 404) {  
            alert("Oh no, it does not exist!");
        }  
    }
};
request.send();

But apparently XMLHttpRequest() is not known in Airtable Scripting block.

Any hint would be appreciated!
Thanks, Stefan

[Bill_French]

Stefan_Kunz:

But apparently XMLHttpRequest() is not known in Airtable Scripting block.

Correct; try this approach… (set gcpURL to the endpoint)

//
// http get test
//
output.markdown('## HTTP GET Test');

let getOptions = {
    method: "get",
    headers: {
        'Accept'       : 'application/text',
    }
}

const getResults = await fetch(gcpUrl, getOptions);
output.inspect(getResults);

[Stefan_Kunz]

Hi @Bill.French,

thanks for your reply! I just tried it out (only replaced gcpUrl with my test domain ‘https://stefan.kunzweb.net/’ and now get the following error message on script execution:

Thanks, Stefan

[Bill_French]

CORS…

This is unfortunate and the best advice is to start here to get a basic understanding of why CORS issues arise. In a nutshell, the response is an indication the server you are requesting content from is not entirely comfortable (security-wise) with such a request coming through something other than a browser (i.e., a javascript-based request).

One would think a GET request in javascript is no different than a web browser making the same request. But this is not the case; there are more complexities that arise as compare to a browser.

Backing up a bit - is your concern to test the URL …

1. To be certain it is a web page that actually functions?
2. To be certain that the URL is really a valid URL?
3. To fetch data from the target URL?

[Stefan_Kunz]

Thank you @Bill.French! Yes, this is unfortunate. I have tested the call with different URLs (e.g. also with Wikipedia) but I always got the CORS error.

Never mind - it is not a critical task and I can use other solutions outside of Airtable to solve that based on exported Airtable data. My intention was just to make sure that all URLs which I have in my DB are still valid, i.e. to be alerted if a URL is no longer existent, e.g. becuase it has moved to a different location. I don’t need to fetch any data from it.

Thanks, Stefan

[Noamsay]

Thanks @Bill.French for your piece of code. It was very helpful.
I got the same CORS error, but it worked for me after adding

mode: 'no-cors'

to the option like this:

let getOptions = {
    method: "GET",
    headers: {
        'Content-Type': 'application/json',
    },
    body: JSON.stringify(data),
    mode: 'no-cors'
}

[Bill_French]

Noamsay:

I got the same CORS error, but it worked for me after adding mode: 'no-cors'

Yep - this is a harsh aspect of security for all aspects of integration and the CORS issue stems from the services you are calling. As such, systems built in Google Apps Script can be shaped to avoid this, but in most cases, you are at the mercy of the endpoint providers.

[Gustavo_VD]

Couldn’t you try to use something like Integromat or Zapier? This would be a simple task to achieve using Integromat, for example.

[Jorge_Gonzalez]

Is there a way or script that can check the url’s that are in a column and give me the status code for them like 404, 200 etc?