+4Hey,
I absolutely love Airtable. Is there a way to transform Airtable into a graph database and/or a way to export/integrate it so it can be usable with graph DB tools (e.g. Neo4J).
Furthermore, is there an Graph DB alternative similar to Airtable?
Thanks a lot in advance for your replies!
Sincerely,
Timo
I just moved from Airtable as a backend to Neo4j as a backend, with the latter sitting behind a GraphQL server. I’m still in the development phase, with everything running on my laptop. I’ll test deploying it in the next few days using Netlify functions and a Neo4j Sandbox.
I intend to keep using Airtable to edit data, so I wrote a command line program (actually a REPL so that the Airtable data is held in memory between commands) that exports data from multiple Airtable bases and imports it into a single Neo4j database through GraphQL mutations.
These mutations use Cypher MERGEs, so duplicate nodes and relationships aren’t created when I import the same data. During the transfer process I rename all Airtable record ids to a hash of their content; that way I deduplicate data across bases.
Both my web app and REPL are written in Elm, which is a statically typed language. I have most of my backend and most of my frontend auto-generated with Neo4j’s GRANDstack and elm-graphql respectively. So whenever I change my schema, the Elm compiler can pinpoint the code changes I need to make (beyond the autogenerated changes). I’m sure you can achieve something similar with Typescript and other languages.
Last year I wrote a fair bit of Elixir code to export a collection of Airtable bases, merge and deduplicates data across the bases in-memory, and then exports it back to an Airtable “meta-base”. I got quite fancy with the deduplication: for example, calculating perceptual hashes of images from different bases and removing those that were 70% similar.
But I think the graph database approach is going to be much more capable because one will be able to run graph algorithms inside Neo4j to find and connect similar data. Neo4j has advanced algorithms built-in with excellent documentation, including free books.
Also, Neo4j makes it easier to track historical data, which one can choose to include or exclude in queries. That will depend on your data model (and custom queries and mutations in your GraphQL resolvers) since there are several ways to model historical data.
Because I’m working in a specific problem domain, my code isn’t general. But I see that there are new solutions for getting the structure of Airtable databases that don’t depend on the API. Makes me wonder if anyone has written code that autogenerates a GraphQL schema from an Airtable base. But perhaps it still makes sense to model your Airtable data as you’d like to edit the data, and model your GraphQL schema as you’d like to query it.
Anyway, I encourage you to explore this direction. Airtable is still the best tool for viewing and editing data (especially in a collaborative environment) and Neo4j is the best way to model (and query) many forms of data, and is, in particular, a natural fit with GraphQL. I also plan to export Neo4j subgraphs for editing in Airtable.
There have been various requests for Airtable to provide a GraphQL API to developers. And perhaps people have made solutions with AWS Lambda, etc. But none of those solutions will have the richness of querying a Neo4j database, for things like recommendations, or for running offline graph algorithms to find patterns in your data.
+4I just moved from Airtable as a backend to Neo4j as a backend, with the latter sitting behind a GraphQL server. I’m still in the development phase, with everything running on my laptop. I’ll test deploying it in the next few days using Netlify functions and a Neo4j Sandbox.
I intend to keep using Airtable to edit data, so I wrote a command line program (actually a REPL so that the Airtable data is held in memory between commands) that exports data from multiple Airtable bases and imports it into a single Neo4j database through GraphQL mutations.
These mutations use Cypher MERGEs, so duplicate nodes and relationships aren’t created when I import the same data. During the transfer process I rename all Airtable record ids to a hash of their content; that way I deduplicate data across bases.
Both my web app and REPL are written in Elm, which is a statically typed language. I have most of my backend and most of my frontend auto-generated with Neo4j’s GRANDstack and elm-graphql respectively. So whenever I change my schema, the Elm compiler can pinpoint the code changes I need to make (beyond the autogenerated changes). I’m sure you can achieve something similar with Typescript and other languages.
Last year I wrote a fair bit of Elixir code to export a collection of Airtable bases, merge and deduplicates data across the bases in-memory, and then exports it back to an Airtable “meta-base”. I got quite fancy with the deduplication: for example, calculating perceptual hashes of images from different bases and removing those that were 70% similar.
But I think the graph database approach is going to be much more capable because one will be able to run graph algorithms inside Neo4j to find and connect similar data. Neo4j has advanced algorithms built-in with excellent documentation, including free books.
Also, Neo4j makes it easier to track historical data, which one can choose to include or exclude in queries. That will depend on your data model (and custom queries and mutations in your GraphQL resolvers) since there are several ways to model historical data.
Because I’m working in a specific problem domain, my code isn’t general. But I see that there are new solutions for getting the structure of Airtable databases that don’t depend on the API. Makes me wonder if anyone has written code that autogenerates a GraphQL schema from an Airtable base. But perhaps it still makes sense to model your Airtable data as you’d like to edit the data, and model your GraphQL schema as you’d like to query it.
Anyway, I encourage you to explore this direction. Airtable is still the best tool for viewing and editing data (especially in a collaborative environment) and Neo4j is the best way to model (and query) many forms of data, and is, in particular, a natural fit with GraphQL. I also plan to export Neo4j subgraphs for editing in Airtable.
There have been various requests for Airtable to provide a GraphQL API to developers. And perhaps people have made solutions with AWS Lambda, etc. But none of those solutions will have the richness of querying a Neo4j database, for things like recommendations, or for running offline graph algorithms to find patterns in your data.
Hi Stephen!
First of all, sorry for the long delay in my answer. I initially saw your answer and wanted to reply but then it got buried with other stuff.
Second, thank you so much for that spontaneous and very complete answer. I am at the very beginning of a research project and I wish I was already at the stage where I could use your insights concretely. However, I am not there yet and my understanding is still limited, especially in terms of implementing anything.
I am very looking forward to take all your answers into consideration!
Again, it’s amazing the power of a community. I would never have expected such an answer.
Sincerely,
Timo
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
