API calls to a NodeJS/Express application with no domain name

Matthew_Thomas · ‎Feb 19, 2020

As in my previous post, and several other posts here about fetch(), I’m not sure how much of this question is: 1) my own coding error; 2) issue with/how the scripting block functions; or 3) a combination of those and more.

I’ve gotten simple fetch() calls to work, from examples posted on this forum and in the documentation. I’ve also gotten a call to my desired Vimeo endpoint to partially work (with some restrictions mentioned in the earlier linked post).

The following code works (copied and modified from other examples on this forum), and since the example video here isn’t restricted (it’s a random video I picked on Vimeo’s homepage), all the information I need is present (namely: the “duration” field).


let url = "https://vimeo.com/api/oembed.json?url=https%3A//vimeo.com/392348731";
let headers = {
    'Accept': 'application/json',
    'Referer': 'https://airtable.com',
};

let info = await fetch(url, {
    "method": 'get',
    "headers": headers
});
const json = await info.json();
output.inspect(json);

with the output seen here:

But given the Vimeo embed restriction, I haven’t gotten this approach to work for my videos with restrictions. No problem, I already have a NodeJS/Express app that deals with the regular Airtable API to fetch this Vimeo information for a bunch of records. Could I just call this app from a Scripting block to run it all within Airtable?

Well, the longer answer is my API endpoints would need to be tweaked (none currently return JSON data, but rather uses it to render a HTML page). To see if any interaction would be possible, I tried tweaking the code from above to set the URL to a new test endpoint, similar to

let url = http://IP.address.located.here:8080/runtime/test";

where the IP address is not a domain name, and the app runs on port 8080. I added the following code to the ‘runtime’ router

router.get('/test', (req, res) => {
	res.status(200).json({user: 'test json data'});
});

such that visiting the URL response with test JSON data on a GET request. Visiting the actual URL in the browser is successful,

but trying the same call in the Scripting block hangs (with the spinny-circle) — no error message.

and checking the logs on the server don’t show any record of a ping to the URL.

The only idea I have is it is [some kind of client-side issue similar to this SendBlock post] from Jonathan. Or that it is some DNS/port issue with my URL being located at an IP address with port 8080.

Any ideas on if this sounds like the potential issue, or other ideas on what it could be? And if this is the issue, does anyone have suggestions on how to workaround it? @Bill.French mentioned in the previous post setting up a Heroku app as a proxy, but I’m not sure if that would run into similar issues/problems as the NodeJS server app I already have (hosted on Digital Ocean).

Thanks for any help/advice! I’m off to a slow start with this scripting block, but I definitely see a lot of potential here. Really great work from the Airtable team!

-Matthew

Matthew_Thomas · ‎Mar 04, 2020

While writing this post, I think I discovered the issue (after installing the cors package) was not a CORS issue, but an HTTPS issue. I’ve moved that question to a follow-up post, but decided to keep my original text as reference for others.

Some further testing has so far proved unsuccessful. I’ve tried simple tests using

that @openside cited above, as well as trying to to write my own middleware for these sample requests. Most puzzling to me is that, I don’t see any attempt to access the server through log statements. Going through the Proxy will log a ping, but straight from the scripting block does nothing.

Here are the relevant pieces of code I can think about for how I incorporated the cors library, following the documentation. If anyone can spot my error (or if it is a different problem altogether), let me know!

app.js

// Installed middleware following package documentation
const cors = require('cors');
app.use(cors());

/**
 * Alternative method trying to handle the request myself.
 */

/**
app.use((req, res, next) => {
    res.set({
        'Access-Control-Allow-Origin': '*',
        'Access-Control-Allow-Methods': 'GET,POST,OPTIONS',
        'Access-Control-Allow-Headers': 'Content-Type, Access-Control-Allow-Headers',
        'Content-Type': 'application/json',
    });

    if(req.method == 'OPTIONS') {
        console.log('PREFLIGHT option');
        return res.status(200).end();
    }

    // This log statement should appear regardless of CORS-success??
    console.log('Attempt!');  
    next();
});

*/

// Ideally I would factor this out into a router, but figured
// I should try the simpler approach first.
app.post('/api/runtime', cors(), (req, res, next) => {
    let jsonMessage = {
        'msg': 'This is a CORS-enabled message!',
    };
    console.log('AIRTABLE request. Should succeed');
    res.json(jsonMessage);
});

Scripting block code

let url = 'http://ip.address.goes.here:8080/api/runtime';
let content = {
    'videos': [
        {
            'id': 'recXXXXXXX',
            'fields': { 'Vimeo Link': 'https://vimeo.com/XXXXX' }
        }
    ]
};

let response = await fetch(url, {
    method: 'POST',
    headers: {
        'Accept': 'application/json'
    }
    body: JSON.stringify(content);
});

let data = await response.json();
output.inspect(data);

Matthew_Thomas · ‎Mar 04, 2020

It seems this is correct. Airtable was not even reaching my server because it is served over HTTP instead of HTTPS (and is why calls through the Heroku proxy did succeed). I figured something was up when I noticed the error messages differed between the Electron app and the Safari browser. Running the script via Chrome delivered the same error message as the Electron app, and the inspector was most helpful by displaying the following:

So, it looks like I’ll need to go about getting a certificate and setting that up. But before tackling that task, I wanted to poll the community and see if:

There’s a workaround to enable mixed content (my guess is the scripting block is setup to reject non-HTTP content for security reasons).
There’s any additional steps I might be missing (CORS, HTTPS, or other-related issues)

If not, I’ve heard of Let’s Encrypt and would probably go down that route first. Has anyone had success with this, or could recommend anything else?

openside · ‎Mar 04, 2020

A couple options:

I recommend you check out cloudflare. You’ll get ssl cert for free and for $5/month you can explore their serverless platform called cloudflare workers which is pretty amazing and much easier to use than aws lambda or Google functions.

Also, some easier options:

Online IDE, Editor, and Compiler - Fast, Powerful, Free

Repl.it is a simple yet powerful online IDE, Editor, Compiler, Interpreter, and REPL. Code, compile, run, and host in 50+ programming languages: Clojure, Haskell, Kotlin (beta), QBasic, Forth, LOLCODE, BrainF, Emoticon, Bloop, Unlambda, JavaScript,...

Or use Heroku which I believe comes with free ssl setup out of the box

Bill_French · ‎Mar 04, 2020

Okay, so - when I said this 2 weeks ago, I was probably on the right compass heading … :winking_face:

I’m pretty sure there’s not a single host anywhere on the planet who will support open HTTP POSTs.

Indeed, it does. And this is why I typically do not rule out G-Suite which includes SSL without cost or configuration and every script app and every document is a micro-service just waiting to be exploited.

Stevan_Popovic · ‎Jan 20, 2021

@Bill.French for custom apps, is it possible to have a server? Or are custom apps just a React frontend and you should make calls out to a server hosted elsewhere? I don’t see anything about this in the docs.