Re: State of "Airtable Backup & Restoration" 2020 - Summary of existing tools

TFP · ‎Dec 08, 2020

Introduction

I’ve analysed and made the below “State of the Art” regarding the “Backup & Restoration” topic.
This analysis has been done on December 8, 2020.

I’ve also defined what are the “key points” that matters to me, for a reliable Backup & Restoration system. (You might have a different opinion/priorities)

I’ve summarized my findings through an Executive Summary below, and then analysed all tools I could find that might help regarding either “Backup” or “Restoration”, and summarized their limitations.

Also, I created a poll below regarding what you need/expect from such a tool. :bar_chart:

Context

I realized such analysis for my company’s interests first (Unly), but thought it could be useful to the Airtable community.

We might release our internal CLI “Restore backup” tool as open-source tool, because I couldn’t come across any similar available tooling, despite its current limitations.

I’m also considering creating an Airtable App dedicated to “backup/restoration”, maybe open source, maybe paid.

I’d love to get your feedback about how you’d like to use a backup/restoration tool, what are your use-cases, to help me design such a tool. Please share your inputs as comments! :pray:

Key points:

Performing backups and performing backup restoration are two different needs and suffer from different limitations.
Storing Airtable backups into CSV format is too limiting to be used by any restoration tool, it’s not possible to do Airtable > CSV > Airtable, because CSV won’t allow to represent complex structure such as formula or database schema
Storing Airtable backups into JSON format is possible, but there is no tool (official or paid/community) that converts JSON to Airtable (no such “import” tool).
It’s not possible to know the database structure through the Airtable API, but the recent Airtable Apps API offers more information, even though:
It’s not possible to know what formula are used exactly. It’s easier to detect a Formula field, but storing the formula being used doesn’t seem possible.
Remembering the full Airtable configuration (database schema) of one base is not humanly possible and therefore needs to be performed automatically (backup + restoration) to reach 100% confidence in one’s ability to properly restore an Airtable base to a working state .

A 100% reliable backup/restore solution must have the following abilities:

Automated, runs automatically (cron-ish)
Backups data
- Optionally backup row attachments or url (url by default to keep backup overall size to a minimum)
- Resolve relationship (which table, which record)
Backup of all tables by default (not necessarily need user input about what tables should be backed-up)
Backup data structure
- Backup of table/fields configuration (formula, etc.)
Restoration of a full backup (structure + data) into a new Airtable Base

Executive summary:

There exist no tool that allows to perform backup/restoration reliably at this time (December, 2020).

There is no solution that allows to make a full database export (schema + data) and then restore it later on.

All potential solutions fall short and are limited. Some solution only back up the data, some only back up the attachments. Some create CSV backups that cannot be used reliably. Some create JSON backups that don’t contains the data structure of the database (because Airtable doesn’t allow developers to fetch this information through the current API).

The native Airtable Snapshot seems to be the most powerful for a “full recovery”, but is too limited, and we lack control over when it performs said snapshots, which is very likely to lead to data loss. Also, it’s a “black box” that cannot be accessed by any “in-house” recovery tools.

External paid tools (Zapier, MiniExtensions) don’t allow to recreate a base from scratch, but only to restore data into an existing one.

External Open-Source tools (Unly:airtable-backups-boilerplate), even though free, require advanced technical skills (AWS) and aren’t usable by everyone.

It is also unclear what the community really needs regarding Backups & Restoration.

Do you need a full backup recovery system? (create new Base + restore database schema + restore data)
Do you only need a data recovery system? (restore data into an existing Base)
- How would conflicts be handled? Would the backup overwrite existing records? Leave as-it? Manually choose for each conflict?

What do you need? (select all that you’d be able to do)

Manual backups (data only)
Manual backups (database schema + data)
Managed and automated backups (data only)
Managed and automated backups (database schema + data)
Data recovery system (restore data into an existing Base)
Full backup recovery system (create new Base + restore database schema + restore data)

0 voters

Analysis of existing tools and ways to perform backups (and data/schema restoration):

Many tools and quirks have been created to help about Airtable base backups and restoration. Here is a list of those that we could find.

TL;DR; - My personal opinion/recommandation

The easiest way for a full backup system of all your Airtable bases is to use Pro Backup paid 3rd party service (see 9).

Alternatively, if you don’t want to allow any 3rd party to access your data, and have advanced technical skills, you can use the free and open-source Airtable Backups Boilerplate (see 7)
- Disclaimer: I’m the author

In any case, whether you use an extra layer of protection, you can rely on the tools Airtable offers:
- Base Snapshots being the most useful of them for disaster-recovery. (recovers a full snapshot into a new base)
- Undo/redo for common mistakes that can be reverted immediately.
- The trash, up to 7 days after deleting a Base/Table/Field/Records.

1) Zapier

Documentation: How to set up automatic external backups for Airtable in 20 minutes

Release date: October 2018

Cost: ~$50/month (pricing might have changed since then)

Analysis:

:red_circle: Attachments backup:
- Doesn’t allow exporting attachments, backups contain only their url
:red_circle: Schema backup:
- Not possible to back up the database schema, only the data How to set up automatic external backups for Airtable in 20 minutes
:warning: Data backup:
- Data are exported as CSV
- Table names must be configured one by one to be backed up (not automatic)
:red_circle: Full backup restoration:
- Not possible, only the data can be (partially) restored
- Relationships seem to be respected (not tested)

2) Mini Extension - Export Airtable Base as CSVs to Dropbox

Documentation: https://miniextensions.com/airtable-dropbox-csvs/

Release date: March 2020

Cost: $39/month (package containing all mini-extensions, see https://miniextensions.com/pricing/)

Analysis:

:red_circle: Attachments backup:
- Doesn’t allow exporting attachments, backups contain only their url
- Recommends using https://miniextensions.com/bulk-download-attachments/ to work around this limitation
:red_circle: Schema backup:
- Not possible to back up the database schema, only the data
:warning: Data backup:
- Data are exported as CSV
- Table names must be specified to be backed up (not automatic)
- Relationships show the Record ID, “you’ll need to use a Lookup field to show the record’s name”
:red_circle: Full backup restoration:
- Not possible, only the data can be (partially) restored
- Relationships seem to be respected (not tested)

3) Mini Extension - Create Dropbox Folders For Airtable Records

Documentation: https://miniextensions.com/airtable-dropbox-folder/

Release date: May 2020

Cost: $39/month (package containing all mini-extensions, see https://miniextensions.com/pricing/)

Analysis:

:white_check_mark: Attachments backup:
- Automatically creates a Dropbox folder that contains all the attachments of an Airtable record.
:red_circle: Schema backup:
- Not possible to back up the database schema
:red_circle: Data backup:
- Not possible to back up the database data
:red_circle: Full backup restoration:
- Not possible at all

This solution is interesting because it could act as a “support” way to perform automated backups for Airtable Attachments. It’s not a backup solution, but could be used alongside one.

4) Bulk Download All Attachments from Airtable

Documentation: https://miniextensions.com/bulk-download-attachments/

Release date: January 2020

Cost: $39/month (package containing all mini-extensions, see https://miniextensions.com/pricing/)

Analysis:

:white_check_mark: Attachments backup:
- Manually exports all attachments (from a table/view) as a single .zip file
- “ :warning: Tested with thousands of records and multiple GB.”
:red_circle: Schema backup:
- Not possible to back up the database schema
:red_circle: Data backup:
- Not possible to back up the database data
:red_circle: Full backup restoration:
- Not possible at all

Similar to the previous solution, it can be used as a “support” way to perform manual backups for Airtable Attachments. It’s not a backup solution, but could be used alongside one.

5) Automatically Import CSVs into Airtable

Documentation: https://miniextensions.com/airtable-import-csv/

Release date: April 2020

Cost: $39/month (package containing all mini-extensions, see https://miniextensions.com/pricing/)

Analysis:

:red_circle: Attachments backup:
- Doesn’t perform any backup
:red_circle: Schema backup:
- Doesn’t perform any backup
:red_circle: Data backup:
- Doesn’t perform any backup
:red_circle: Full backup restoration:
- Not possible because it doesn’t know the data structure, thus won’t be able to recreate a base from scratch
- Allows restoring/updating records in an existing base/table
- “Can only sync a few thousands of records”

Similar to the previous solution, it can be used as a “support” way to perform automated restoration of data.

It’s not a backup solution at all, but could be used alongside one. (i.e: one tool that performs the backup as CSV and this tool that performs the restoration)

6) Native Airtable "Snapshot"

Documentation: https://support.airtable.com/hc/en-us/articles/202584799-Taking-and-restoring-base-snapshots

Release date: Unknown (2019 ?)

Pricing: Included in your Airtable plan. Features depend on the plan.

Known limitations:

Snapshots are only accessible for a certain amount of time depending on your workspace’s pricing plan.
For the Free plan, you will be able to see the past 2 weeks ’ worth of snapshot history.
For the Plus plan, 6 months ’ worth of snapshot history will be saved starting at your time of upgrade.
For the Pro and Enterprise plans, 1 year 's worth of snapshot history will be saved starting at your time of upgrade.

Analysis:

:red_circle: Attachments backup:
- Not possible to create an external backup, it’s handled in-house by Airtable and the data aren’t accessible
:red_circle: Schema backup:
- Not possible to create an external backup, it’s handled in-house by Airtable and the data aren’t accessible
:red_circle: Data backup:
- Not possible to create an external backup, it’s handled in-house by Airtable and the data aren’t accessible
:white_check_mark: Full backup restoration:
- Completely restores an Airtable Base to a previous state, into a new Base
- Not possible to configure the frequency of the snapshots (you might lose data since last snapshot)

It’s the only known way to completely restore a Base, but it does so with such limitations that it cannot be considered as “reliable” at all. You don’t have any control over when it triggers , nor when it expires .

But, it’s the only solution that allows you to create a new base based on an existing backup , while keeping all the base’s configuration intact. ( :warning: not tested! I expect it keeps Formulas, etc. but probably not Apps and other advanced stuff “airtable-related”)

7) :tada: Unly Open-Source “Airtable Backups Boilerplate”, powered by AWS

Documentation:

Release date: January 2020

Cost: Free, as long as you don’t go over the AWS limits (which are quite large).

:warning: Unlike other above-mentioned solution, this one is probably the most technical , and will need to be configured by someone familiar with AWS. (Engineer-ish)

On the other hand, you’re in complete control over your backups, and everything is done through your own services. No 3rd parties involved. (GDPR-friendly)

Analysis:

:red_circle: Attachments backup:
- Doesn’t allow exporting attachments, backups contain only their url
:warning: Schema backup:
- Partially, fields are exported, but advanced field configuration isn’t exported (because not available through Airtable API)
- No Formula/Rollup support (only the result is exported)
:warning: Data backup:
- Data are exported as JSON
- Table names must be specified to be backed up (not automatic)
- Relationships are exported, but the related table isn’t explicit and would need to be resolved by the restoration tool
:red_circle: Full backup restoration:
- Not possible, it’s a backup tool, not a restoration tool
- If building a backup tool based on the generated backup file:
  - The data might be (partially) restored into an existing Base or a new Base
  - Attachments will be restored using their url (because Airtable Attachments links do not expire at this time)
  - Relationships will need to be resolved by the restoration tool (Record ID will change upon restoration)
  - Formulas/Rollup fields will be lost (only the generated values will be kept)

😎 [Unly private] CLI allowing you to restore backups generated from https://github.com/UnlyEd/airtable-backups-boilerplate

Documentation: None public, private doc

Release date: No public release date

Cost: Free (doesn’t cost anything to run)

:warning: Unlike other above-mentioned solution, this one is both very technical and hasn’t been publicly released yet.

It basically allows to restore a backup into an empty clone of your Base. (Duplicate Base > Disable “Duplicate records”)

9) ProBackup

Documentation: https://probackup.io/backup/airtable
Release date: End of 2020
Cost: Depends on plan, 15-99$/month, 15 days free trial

This is a paid 3rd party solution, which I analyze thoroughly on the technical, security and legal/compliance levels.

Disclaimer: While I’m a not affiliated with them, I might benefit from their referral program if you subscribe using the above link.

:tada: : Unlike any solution above, this one backs up all your bases automatically. It doesn’t need to know about which bases and which tables you want to back up. It’s the most automated tool there is, and it can be used by about anyone, there is no technical entry barrier. Also, their backup contains the most metadata about your base schema. It’s the only solution which also backs-up your Airtable Formulas (the formula itself, alongside the calculated value).

:warning: Even though they don’t provide a way to restore a full backup, at this time (e.g: Re-create a base from scratch and generate all tables/fields/views/etc. automatically), ProBackup is the only tool that might be able to do so in the future. Also, it’s on their product roadmap.

The company behind this is B4B IT BV, which is specialized in backup systems (Airtable, Asana, Podio, Monday).

The backups are stored into their AWS Account (S3), and located in Ireland (Dublin). (I believe they might also provide US-located backups storage)
They allow you to sync those backups with your own Google Drive, where backups are stored as Google Sheet files (Data + Fields + Views)

:balance_scale: They’re GDPR compliant and provide a DPA. https://b4bit.be/role-statement.html

They ask you to install a Chrome Extension and that’s about it. The Chrome extension is used to access your Airtable account and the sync will work as long as you’re logged in onto Airtable.
The sync will stop as soon as you log out, or uninstall the chrome extension.

Analysis:

:white_check_mark: Attachments backup:
- Google Sheet backups contains the URL of the attachments
- A Google Drive folder “Backup Attachments” containing all the attachments is also available for download
:white_check_mark: Schema backup:
- The full schema is included in the backup (including raw Formulas)
:white_check_mark: Data backup:
- Data are stored as Google Sheet on Google Drive
- All tables are automatically included in the backup
- Data might be stored using another format (JSON?) into their AWS Account
:warning: Full backup restoration:
- Not possible yet, but partial recovery is possible
- Full recovery is on the product roadmap (views, formula, tables, fields, fields configuration/options, data, etc.)

Additional resources:

Airtable Backup & Recovery Strategies: Simply, a Nightmare State of “Airtable Backup & Restoration” 2020 - Summary of existing tools

Willem_Dewulf · ‎Dec 08, 2020

Hi TFP, thanks for the clear & elaborate overview! I just wanted to let you know that my team has been working hard on a dedicated backup tool for Airtable, called [Pro Backup](https://probackup.io/backup/airtable), that might cover some of your needs. It is currently still in early phase, but it offers the following:

Backup of attachments, comments & views.
Schema database is also covered
Data backup
- Data can be exported to CSV via the snapshot feature
- Data is synced to Google Sheets as well.
Full backup restoration: Currently we offer only a basic recovery feature that allows you to recover permanently deleted data. However we are planning to add more advanced restore capabilities in the near future.

Feel free to DM me if you would like to have more info on this. Else visit https://probackup.io/backup/airtable and reach out to our Support team.

TFP · ‎Dec 09, 2020

After analysing the various causes and scenarios that might lead to data loss, I wrote another piece of knowledge at Analysis of causes and scenarios leading to data loss, and what "Backup & Restoration strategy" migh...

My personal opinion is that having our own “full backup recovery system (schema + data)” is not an absolute necessity . After analysing all potential risks, we figured most of the risks we idenfied can be resolved easily enough by using the tools provided by airtable (undo, trash and snapshot).

Even though, we will keep making our own backups , through our open-source tool https://github.com/UnlyEd/airtable-backups-boilerplate, because we feel necessary to have a backup of the data, at the very least. But this backup will not allow us to make a full recovery, and it’s not its purpose.

Building a “full backup recovery system” is quite a gigantic task, and we’re not up to it. The tools provided by Airtable are good enough for most cases. I would still recommend any professional to make their own backups is a way that can be scripted, to ensure they can use their backups to recover from disasters. (or rely on a professional 3rd party that does exactly that)

TFP · ‎Dec 11, 2020

After a thorough analysis of the paid 3rd party Pro Backup, I edited the above post (see 9).

Also, I added a TL;DR; section for people who want to go straight to a recommended and opinionated solution.

@Bill.French I believe the Pro Backup will eventually offer a professional and full backup recovery solution that might suit the needs you’ve expressed multiple times in this Community forum, those past years. I’d love your opinion on the subject! :slightly_smiling_face:

Bill_French · ‎Dec 11, 2020

I agree. I think this product - of all approaches and tactics - is best positioned to but a very big dent in the backup/restoration challenge for any firm who has invested in Airtable solutions.