Is the external URL property for a dataset supported in the Vega Lite app?

[Nose_Croquet]

Hi,

Has anyone used the URL property described here:

Vega-Lite

Data

Vega-Lite - a high-level grammar for statistical graphics. Vega-Lite provides a higher-level grammar for visual analysis, comparable to ggplot or Tableau, that generates complete Vega specifications. Vega-Lite specifications consist of simple...

to load data from an external (public) URL into a Vega Lite specification within the Airtable app? This is the kind of syntax that works externally in the Vega Lite editor:

"data": {"url": "https://vega.github.io/vega-datasets/data/unemployment-across-industries.json"}

I’d like to include some external annotation data in a chart (and join it using a lookup, which also works OK in the Vega Lite editor), but I’m having trouble getting it working. I know the Vega Lite block binds the table data at runtime, but I’m not quite sure whether it’s able to make external requests or not.

It doesn’t throw an error when I do it, but on the other hand the error message when you put in a nonexistent field name says this:

fieldname is not a valid transform name, inline data name, or field name from the Imported table table

which sort of implies that those are the only valid data sources. FWIW the binding of the table data happens late enough that you can successfully do transforms on the data.

[Bill_French · Accepted solution]

Nose_Croquet:

I know the Vega Lite block binds the table data at runtime, but I’m not quite sure whether it’s able to make external requests or not.

The block cannot make external calls. But, I believe they provide access to the source in Github so you could create a branch that allows this. The other approach is to use automations, rollups, and the API to make the data viz-ready in a single flat table and then use Vega to plot the results.

[Nose_Croquet]

The block cannot make external calls.

Aha; as I feared, thanks. It’s evidently making one external call to the Vega Lite schema, though:

https://vega.github.io/schema/vega-lite/v4.json

(even though that JSON is in the GitHub code, it still breaks if you put something bogus in there, so I’m pretty sure it’s being loaded at runtime). From looking at the source here:

github.com

Airtable/apps-vega-lite/blob/243b46a5304c6328d640428bc78f2f82b4c765a8/frontend/data.js#L129

}

/**
 * validateFieldDefinitions ensures that only real field names
 * from the selected table are used to build charts.
 *
 * @param  {Table} table          The selected table
 * @param  {Object} specification A parsed vega-lite specification JSON string
 * @return {Array}                An array of errors. Should only ever have one entry.
 */
export function validateFieldDefinitions(table, specification) {
    const errors = [];

    if (!table) {
        return errors;
    }

    // Produce a list of all of the "real" table field names
    const fieldNames = table.fields.map(({name}) => name);

    // Collect a list of the "inline data" field name values, which

It kind of looks like it’s actually just throwing an error if the field name is not in one of the three allowed places, so you’re right, maybe it would be doable to hack it a bit. I don’t really know what the heck I’m doing with Javascript though, so the design is a bit hard for me to get my head around.

The other approach is to use automations, rollups, and the API to make the data viz-ready in a single flat table and then use Vega to plot the results.

Yeah, I think this is pretty much the workaround I’d come up with yesterday. I’m pushing data into the table via the API already, so I can put whatever I want in in whatever format I want it, but I’d like the users to be able to filter what’s in the plot by filtering a column in (a) table. Thing is, the data needs to be in ‘long’ format (as in one observation per row) to be plotted, and this makes filtering a bit less pleasant for users; so it would be nice if they could see it in ‘wide’ (crosstabulated, Excel-type) format, then choose the bits they want to appear on the plot.

Vega also has the ‘fold’ transform, which is very powerful and nearly makes this easy but it requires you to specify the names of all the fields you’re going to transform in advance, meaning I’d have to update the JSON every time a new field was added (& there’s no way to push that programmatically).

Thanks for your help!

[Bill_French]

Good assessment of the challenges @Nose_Croquet - I will likely benefit from this eventually as a number of clients have begun to get restless concerning data source flexibility.