Help

Re: API calls to a NodeJS/Express application with no domain name

1885 1
cancel
Showing results for 
Search instead for 
Did you mean: 
Matthew_Thomas
7 - App Architect
7 - App Architect

As in my previous post, and several other posts here about fetch(), I’m not sure how much of this question is: 1) my own coding error; 2) issue with/how the scripting block functions; or 3) a combination of those and more.

I’ve gotten simple fetch() calls to work, from examples posted on this forum and in the documentation. I’ve also gotten a call to my desired Vimeo endpoint to partially work (with some restrictions mentioned in the earlier linked post).

The following code works (copied and modified from other examples on this forum), and since the example video here isn’t restricted (it’s a random video I picked on Vimeo’s homepage), all the information I need is present (namely: the “duration” field).


let url = "https://vimeo.com/api/oembed.json?url=https%3A//vimeo.com/392348731";
let headers = {
    'Accept': 'application/json',
    'Referer': 'https://airtable.com',
};

let info = await fetch(url, {
    "method": 'get',
    "headers": headers
});
const json = await info.json();
output.inspect(json);

with the output seen here:

Screen Shot 2020-02-19 at 1.21.08 PM

But given the Vimeo embed restriction, I haven’t gotten this approach to work for my videos with restrictions. No problem, I already have a NodeJS/Express app that deals with the regular Airtable API to fetch this Vimeo information for a bunch of records. Could I just call this app from a Scripting block to run it all within Airtable?

Well, the longer answer is my API endpoints would need to be tweaked (none currently return JSON data, but rather uses it to render a HTML page). To see if any interaction would be possible, I tried tweaking the code from above to set the URL to a new test endpoint, similar to

let url = http://IP.address.located.here:8080/runtime/test";

where the IP address is not a domain name, and the app runs on port 8080. I added the following code to the ‘runtime’ router

router.get('/test', (req, res) => {
	res.status(200).json({user: 'test json data'});
});

such that visiting the URL response with test JSON data on a GET request. Visiting the actual URL in the browser is successful,

Screen Shot 2020-02-19 at 1.28.54 PM

but trying the same call in the Scripting block hangs (with the spinny-circle) — no error message.

Screen Shot 2020-02-19 at 1.30.38 PM

and checking the logs on the server don’t show any record of a ping to the URL.

The only idea I have is it is [some kind of client-side issue similar to this SendBlock post] from Jonathan. Or that it is some DNS/port issue with my URL being located at an IP address with port 8080.

Any ideas on if this sounds like the potential issue, or other ideas on what it could be? And if this is the issue, does anyone have suggestions on how to workaround it? @Bill.French mentioned in the previous post setting up a Heroku app as a proxy, but I’m not sure if that would run into similar issues/problems as the NodeJS server app I already have (hosted on Digital Ocean).

Thanks for any help/advice! I’m off to a slow start with this scripting block, but I definitely see a lot of potential here. Really great work from the Airtable team!

-Matthew

14 Replies 14

But didn’t you indicate above that the NodeJS app doesn’t have this issue? If true, you can build a block that calls your NodeJS app (as a proxy) to Vimeo, right?

That was my idea, but at the moment, I can’t even get a simple proof-of-concept call working. I’m not sure if that’s because of a bug in my code, or with how the scripting block is calling the proxy app.

As a tl;dr of the above, a fetch() to Vimeo returns successful JSON :thumbs_up: . A fetch() to my proxy endpoint (which does return JSON via a browser or curl call) returns nothing in the scripting block and results in the hung spinny-circle indicator :-1: .

I assume your proxy server is set up to respond to GET requests (since it works with a browser).

Send me an example cURL command or browser URL and I’ll look at iut with some script block tools I’ve created for my own debugging.

openside
10 - Mercury
10 - Mercury

are you checking via your browser debugger what is returned on the ajax call?

My guess is you don’t have your server set up for proper CORS handling, which only comes into play when making calls from a browser (and why its working outside the browser)

I think his issue is the opposite - it works fine in a browser and fails from a Script Block which probably indicates the issue is in the script block.

maybe - i thought he meant when he typed it into the browser url path it worked (ie a direct GET). but maybe he meant it worked via ajax call.

This is how I got the impression the service was properly instrumented for CORS policy and the Script Block is what’s failing.

openside
10 - Mercury
10 - Mercury

Im doing some testing and Im pretty sure you’re just missing the CORS headers in your server.

An easy way to test is to modify your url to use a CORS proxy like this one https://cors-anywhere.herokuapp.com/

So, change your URL to be:

and i bet it will work. You’ll want to use the https://www.npmjs.com/package/cors library in your node app

Yes, thanks @openside! It does appear to be a CORS issue; testing with the proxy you mentioned does return my test JSON via the scripting block. I’ll have to do some more testing once I get the new package up and running in my server app, but definitely looking a lot more promising!

Matthew_Thomas
7 - App Architect
7 - App Architect

While writing this post, I think I discovered the issue (after installing the cors package) was not a CORS issue, but an HTTPS issue. I’ve moved that question to a follow-up post, but decided to keep my original text as reference for others.

Some further testing has so far proved unsuccessful. I’ve tried simple tests using

that @openside cited above, as well as trying to to write my own middleware for these sample requests. Most puzzling to me is that, I don’t see any attempt to access the server through log statements. Going through the Proxy will log a ping, but straight from the scripting block does nothing.

Here are the relevant pieces of code I can think about for how I incorporated the cors library, following the documentation. If anyone can spot my error (or if it is a different problem altogether), let me know!

app.js

// Installed middleware following package documentation
const cors = require('cors');
app.use(cors());

/**
 * Alternative method trying to handle the request myself.
 */

/**
app.use((req, res, next) => {
    res.set({
        'Access-Control-Allow-Origin': '*',
        'Access-Control-Allow-Methods': 'GET,POST,OPTIONS',
        'Access-Control-Allow-Headers': 'Content-Type, Access-Control-Allow-Headers',
        'Content-Type': 'application/json',
    });

    if(req.method == 'OPTIONS') {
        console.log('PREFLIGHT option');
        return res.status(200).end();
    }

    // This log statement should appear regardless of CORS-success??
    console.log('Attempt!');  
    next();
});

*/

// Ideally I would factor this out into a router, but figured
// I should try the simpler approach first.
app.post('/api/runtime', cors(), (req, res, next) => {
    let jsonMessage = {
        'msg': 'This is a CORS-enabled message!',
    };
    console.log('AIRTABLE request. Should succeed');
    res.json(jsonMessage);
});

Scripting block code

let url = 'http://ip.address.goes.here:8080/api/runtime';
let content = {
    'videos': [
        {
            'id': 'recXXXXXXX',
            'fields': { 'Vimeo Link': 'https://vimeo.com/XXXXX' }
        }
    ]
};

let response = await fetch(url, {
    method: 'POST',
    headers: {
        'Accept': 'application/json'
    }
    body: JSON.stringify(content);
});

let data = await response.json();
output.inspect(data);
Matthew_Thomas
7 - App Architect
7 - App Architect

It seems this is correct. Airtable was not even reaching my server because it is served over HTTP instead of HTTPS (and is why calls through the Heroku proxy did succeed). I figured something was up when I noticed the error messages differed between the Electron app and the Safari browser. Running the script via Chrome delivered the same error message as the Electron app, and the inspector was most helpful by displaying the following:

Screen Shot 2020-03-04 at 9.57.09 AM

So, it looks like I’ll need to go about getting a certificate and setting that up. But before tackling that task, I wanted to poll the community and see if:

  • There’s a workaround to enable mixed content (my guess is the scripting block is setup to reject non-HTTP content for security reasons).
  • There’s any additional steps I might be missing (CORS, HTTPS, or other-related issues)

If not, I’ve heard of Let’s Encrypt and would probably go down that route first. Has anyone had success with this, or could recommend anything else?

openside
10 - Mercury
10 - Mercury

A couple options:

I recommend you check out cloudflare. You’ll get ssl cert for free and for $5/month you can explore their serverless platform called cloudflare workers which is pretty amazing and much easier to use than aws lambda or Google functions.

Also, some easier options:

Online IDE, Editor, and Compiler - Fast, Powerful, Free

Repl.it is a simple yet powerful online IDE, Editor, Compiler, Interpreter, and REPL. Code, compile, run, and host in 50+ programming languages: Clojure, Haskell, Kotlin (beta), QBasic, Forth, LOLCODE, BrainF, Emoticon, Bloop, Unlambda, JavaScript,...


Or use Heroku which I believe comes with free ssl setup out of the box

Okay, so - when I said this 2 weeks ago, I was probably on the right compass heading … :winking_face:

I’m pretty sure there’s not a single host anywhere on the planet who will support open HTTP POSTs.

Indeed, it does. And this is why I typically do not rule out G-Suite which includes SSL without cost or configuration and every script app and every document is a micro-service just waiting to be exploited.

@Bill.French for custom apps, is it possible to have a server? Or are custom apps just a React frontend and you should make calls out to a server hosted elsewhere? I don’t see anything about this in the docs.