Having trouble with Fetch in scripting block

Jerry_Harris · ‎May 31, 2020

Hello,
I was working on a fetch to a site to retrieve JSON data with Basic Authentication. I got around the CORS block by setting " mode: ‘no-cors’" in my call to fetch. But I ran into a more serious problem. My calls to fetch() are not returning any data even if successful. Here’s my test program:
var url = ‘https://www.google.com/thisshouldfail’;
var url2 = ‘https://www.google.com/’;

var response = await fetch(url, {method:'GET', mode: 'no-cors'});
var response2 = await fetch(url2, {method:'GET', mode: 'no-cors'});

output.inspect(response);
output.text(response.status);

output.inspect(response2);
output.text(response2.status);

The output to this running is:
{}
0

{}
0

Looking at the browser’s network log I see the calls return the correct HTTP codes:
GENERAL:
Request URL: https://www.google.com/thisshouldfail
Request Method: GET
Status Code: 200
Remote Address: 172.217.9.228:443
Referrer Policy: no-referrer-when-downgrade

GENERAL:
Request URL: https://www.google.com/
Request Method: GET
Status Code: 200 
Remote Address: 172.217.9.228:443
Referrer Policy: no-referrer-when-downgrade

The fetch() function call is not returning either 404 or 200 codes, nor the results of the successful call to google.com.

Is there another flag I need to get this to work?

Bill_French · ‎May 31, 2020

Hi @Jerry_Harris,

Not sure why your tests are failing, but it could be something to do with the Google domain which is a heavily guarded domain. I suspect the folks at Google are not amused by anyone testing API calls on their domain address.

Here are the results of two GET tests I whipped up in script block. One calls an API that returns a JSON a list of dog breeds. The other is a JSONP web service that simply returns a payload with whatever parameters you call it with. You should be able to just drop this in and run it to examine the responses.

This is the code I quickly hacked together …

//
// simple http get test (Airtable example)
//
output.markdown('## Simple HTTP GET Test (Airtable Example)');

const response = await fetch('https://dog.ceo/api/breeds/list/all');
const jsonTest = await response.json();
output.inspect(jsonTest);

// establish the cgp web service endpoint
let gcpUrl = "https://script.google.com/macros/s/AKfycbxGO0QVOYEiiBkseEh3AUSnHFMpS0OXidRtAUMTNIjj_EaFdBo/exec";

//
// http get test (from GCP)
//
output.markdown('## HTTP GET Test (from GCP)');

getOptions = {
    method: "get",
    headers: {
        'Accept'       : 'application/json',
    }
}

getResults = await fetch(gcpUrl + "?field=name&value=Bill%20French", getOptions);
const getJson    = await getResults.json();

output.markdown("Display JSON Object");
output.inspect(getJson);
output.markdown(getJson.field + " = " + getJson.value);

kuovonne · ‎May 31, 2020

Notice that @Bill.French is not directly outputting the response from the fetch.
He is taking an additional step of converting the response to something meaningful (in this case json). Notice that he has to await this conversion process.

Your code is trying to directly output the response, which contains the body of the data, but isn’t directly the data itself. Think of it as a package that you need to unwrap when you order something online. If you order a pencil and get it delivered, you can’t start writing with the pencil as soon as it arrives–you have to first take it out of the box. How you unwrap the package depends on the type of data in it.

response.text() – read the response and return as text,
response.json() – parse the response as JSON,
response.blob() – return the response as Blob

Bill_French · ‎May 31, 2020

A super-important step because parsing can take a little time. :winking_face:

Jerry_Harris · ‎May 31, 2020

Thanks, Bill. Your script works for me, except when I modify the Options by adding mode: ‘no-cors’:
getOptions = {
method: “get”,
mode: ‘no-cors’, // added by me
headers: {
‘Accept’ : ‘application/json’,
}
}

Then, Airtable returns {} even though Chrome’s developer tools report that the URL succeeded with a 200 status.

I need the ‘no-cors’ for the real project I’m working on. I want to grab data from our Jira instance and create a new version. If I don’t put in mode: ‘no-cors’, it fails with the preflight check b/c Jira’s website does not response with Access-Control-Allow-Origin “*” in its header. (It doesn’t have that Header item.) But all my tests so far have shown that adding mode: ‘no-cors’ fails to return any data from the call, even when Chrome’s debugger shows the call worked.

I can’t find any documentation whether Airtable is suppose to support ‘no-cors’. It either doesn’t, or it’s a bug.

Thanks again for your help.
Jerry

Bill_French · ‎May 31, 2020

Okay - well, that’s a little deeper and complex question because there are many endpoints for Jira data and they behave differently. Which one are you using?

This is a security requirement your browser is enforcing. CORS errors are only caused because your browser notes that the domain for the URL you’re requesting does NOT match the domain for the URL you are currently on. In a script block context, you are running script in the client, not in a server context, so this error restriction expected.

I believe it’s often (not always) a mistake to disable CORS as it exposes some key security features that protect more than simply your client (i.e., your web browser running Airtable and script blocks).

Perhaps you have seen this article describing the conditions that allow a client-side request. Or maybe this one?

Jerry_Harris · ‎Jun 01, 2020

Hi Bill,
The endpoint I’m attempting to access is :
https://{myCompanyName}.atlassian.net/rest/api/2/project/{projectId}/versions

I have seen the OAuth documentation at Atlassian and am aware OAuth2 is more secure than an API key and Basic Auth. I wanted to do a quick experiment to determine if this would work within an Airtable script block first. I was hoping setting mode: no-cors would work. As I pointed out above, the call does go through to the server but no data is returned to my airtable script,

Do you have any knowledge of an OAuth2 API call working from within a script block? It will require user confirmation to complete the OAuth dance. Is this something Airtable will allow? I’m not confident it will.

Thanks again for your time and help. I suppose my next step would be to spin up a small server that proxies the call to Atlassian.

Regards,
Jerry

Jerry_Harris · ‎Jun 01, 2020

Hello Yvonne,
I had noticed that difference and tried it to no avail. Thanks for your help.

Regards,
Jerry

Bill_French · ‎Jun 01, 2020

I have never had the occasion to integrate via oAuth2 (in Script Blocks) but I suspect it can be done.

Um, this is not true actually. We build all sorts of oAuth2 integrations that are fully automated. The “dance” can be orchestrated in a manner that does not require human interaction.

Unfortunately, there’s is nothing about oAuth2 that is quick or simple.

If that’s the plan, one must ask - why even use the script block at the headwater of the integration process? Might be better to just spin up an integration layer in a free Google Apps Script project where it can be fully automated and secure. Or, use Zapier or Integromat.

Jerry_Harris · ‎Jun 02, 2020

Hi Bill,
Thanks again for your help. It seems Jira implements only 3lo oauth v2 and they seem pretty explicit about requiring a user to grant authorization to Jira resources. This was going to be a convenience feature so not worth spending more time on.

Thanks for the tips on using Google apps or Zapier or Integromat. If I re-visit this, I’ll look at those options.

Regards,
Jerry