Checking 404 status of all URLs in a Column

[Sanjida_Begum]

I have pasted around 500-600 URls in a column and I want to know 404 statuses of all the URLs in the Next Column or in “view Blocks” (want to know which URLs are live and which are not). I am not so good at scripting. Can You Please help?

[Jorge_Gonzalez]

I am also interested in something like this. For example, my column is named textarea-address I would like for the script to check all the URLs that are inserted in that column

[Alex_Nesbitt]

Any luck with finding a solution? I am looking for the same.

[ROARK]

That’s actually fairly straightforward to do.

Inside an automation you can use the fetch() method to call the url and it will return the status code.

Here is an example automation of how this could be implemented:

First we set our url field as an input variable on the lefthand side of the automations view.

Then we run the following script:

const inputConfig = input.config();
const url = inputConfig.url;

let status;

try {
    const response = await fetch(url);
    status = response.status;
} catch (error) {
    status = 'error';
}

output.set('status', status);

This will try to fetch the url and if successful outputs the status code, which will be 200, 404, etc.

You can now use that output value in the next part of you automation.

There is one caveat when calling this method inside an Airtable automation: it won’t follow redirects and just throw an error instead:

Response status was 301 ‘Moved Permanently’ but redirect mode was set to ‘error’

Normally you’d be able to setup fetch() in a way that it will follow redirects, but that’s not supported in Airtable automation scripts:

TypeError: Requests that follow redirects are not supported currently

This happens for example when there is a http → https redirect.

The problem is, that we won’t get a response code and the error message we get inside the catch block is useless.

Depending on your use case, just setting the output text to “error” could suffice.

[Dominik_Bosnjak]

This is just off the top of my head (i.e., untested), but how about something like this:

const results = []

await base.getTable('Table 1').selectRecordsAsync({fields:['Your_Url_Field_Name']})
.then(query => query.records.flatMap(rec => remoteFetchAsync('https://www.sammobile.com/no-way-this-url-exists')
    .then(response => {
        if(response.status === 404)
            results.push({id:rec.id,fields:{'Your_Status_Check_Result_Field':'404'}})
    })));

while(results.length)
    base.getTable('Table 1').updateRecordsAsync(results.splice(0,50))

Since the original requirement only seeks to mark 404s?

[Manal_Belghali]

Hey! Thanks for this.
Any idea on how we can get the output to be the actual error/status code (404, 301…)?

Summary

This text will be hidden

[morr]

I have tried to run this with time based automation, and I can't seem to do that, it doesn't let me choose a dataset when I am triggering this on time. Anyone know how to fix it? 

[PCE]

Hi, this sounds like a fairly easy set up. Can you explain it in a bit more detail—like you're explaining it to someone who knows next to nothing about automations and scripts in Airtable. What specific options do you select from start to finish in Airtable to set this up? This could help my entire team better manage our urls.