Access to bases on mobile devices while offline


#21

Hi, everyone! First off, we hear you loud and clear on the desire for offline mode. We’ve spent much time internally exploring the possibility of offline mode (and even building some prototypes), but it’s a fundamentally difficult feature to build. Since Airtable is a collaborative product, an ideal offline solution would need to gracefully handle merge conflicts, and do so in the context of Airtable’s relational data structure and rich field types. For comparison, Google Docs uses an approach termed operational transforms, but this only works on simpler data structures (i.e. a Google word document is represented as a single long string of text, and a spreadsheet is represented as a shapeless 2-dimensional array of cells).

As we continue to explore the tradeoffs for various implementations of offline mode, it would be very useful for our team to get some detailed feedback from dedicated Airtable users like yourselves about why and how you would like offline to be designed. Just a sampling of potential things to think about:

  • How would you want to use Airtable offline? Tell us your story. Are you a world traveler, wanting to find a way to look at your elaborately designed base of sightseeing spots without having to pay exorbitant roaming charges? Are you a construction baron, sending out remote workers to to survey your newly acquired plots of land? Are you a deep sea diver on a solo expedition in the Marianas Trench, logging undiscovered species of hagfish on the ocean floor? Do you just have really terrible WiFi in the boiler room under the stairs, where the boss moved your office?
  • Do you use Airtable alone, or with collaborators? (Resolving merge conflicts when someone is the sole collaborator on a database is far simpler than doing so when you have multiple collaborators, some of whom are online and some of whom are offline, and all of whom are editing different things.)
  • How much do you need to be able to do in Airtable while offline? Do you just need a read-only version of the contents of your bases that you can access while offline? Or do you need to have the ability to edit the contents of records while offline? Or do you need to have the ability to change the database schemata while offline? (In order, these are: comparatively simple to implement; difficult to implement; incredibly difficult to implement.)
  • For how long do you need to be able to use Airtable offline? For a few minutes, as you drive through a tunnel? (Please don’t Airtable and drive.) For a few hours, as you sit in an airport terminal and refuse to pay for overpriced WiFi? For a few weeks, as you trek through the farthest reaches of frigid Nunavut?
  • How tolerant would you be of having to resolve merge conflicts manually? One potential way of alleviating the issues that arise with resolving merge conflicts is to force end users to make decisions on how to solve specific conflicts. This, however, comes with its own set of potential problems (e.g. being stalled by having to resolve a merge conflict when all you want to do is work, your co-workers getting mad that you merged over their data without their permission).
  • Would you primarily be using offline mode on desktop or mobile? Anecdotally, it seems like many of the people who want offline mode want it for mobile, but it would be good for us to get some clarification on that front.

#22

How would you want to use Airtable offline? I’m a wedding photographer! I want to use Airtable at wedding fairs, expos, and events to create a database of couples that I talk to - I create a personal CRM-type form as I go, and I don’t always have internet.

Do you use Airtable alone, or with collaborators? I use it alone!

How much do you need to be able to do in Airtable while offline? Mostly, I’d like to be able to create records and then edit those records.

How tolerant would you be of having to resolve merge conflicts manually? VERY tolerant. As long as I knew it before I went offline, it’d be way worth it to still have the offline option. If someone wasn’t up for that, they could just choose not to use the offline features.

Would you primarily be using offline mode on desktop or mobile? Mobile! I use it on my ipad for this.


#23

Hi Katherine, I am going to respond to your questions but I also want to ask a couple simple questions for you to consider too, that may help to find an offline solution quickly, with little or no modification to the way your servers currently process requests from collaborators…

-How does Airtable CURRENTLY handle collaboration between users when changes are made by UserA who is using a satellite internet connection at 2000 ms latency, and UserB who is using cable internet with 10-12 ms latency. For example; if a request is “timestamped” (with a UTC Universal Coordinated Time) by UserA’s device BEFORE UserB’s device, but the request from UserB arrives at Airtable’s server first due to lower latency, which request does Airtable process first if the request effects the same record?

-In the hypothetical circumstances described above, does Airtable notify either of the users of the conflict or does it simply overwrite the older change with the newer change and the users are oblivious to the conflict?

-Using the same hypothetical situation described above, now increase the 2000 ms latency to 900,000 ms (15 mins) and I assume the servers would process the requests the same way (provided that Airtable increased the allowed amount of latency between requests and did not reject the requests as being invalid, as I assume it does now). So now, using the same logic, if Airtable servers were programmed to “expect” and process requests with large amounts of latency, then couldn’t we increase this to any amount of time?

I am interested to know the answers to those questions, and then maybe we aren’t very far from making offline access a reality. Then when a user “toggles offline mode” or is “forced” offline by bad service, they can continue to work, knowing that as soon as they toggle back online, or when service becomes available again, their changes will be processed in the same way they would be otherwise.

Now I will respond to your questions…
-here are a few use cases; journal, fitness, vehicle maintenance, inventory, hiking, fishing, camping, navigating, road trips, travel, photography, geocache, contacts, tasks, you name it… databases are great for it but you can’t count on coverage being everywhere. we aren’t asking for this for the 99% uptime we have… we are asking because of the 1% downtime which can last from minutes (tunnel like you said) to days (camping trip or road trip or anything that takes you off the grid)
-most of the world and even most of the united states has incomplete and/or intermittent coverage (see attached), especially rural areas. i could (and have) listed many use cases as examples of why i need offline coverage but in the end, it comes down to… we need offline coverage regardless of whether we are using it for a glorious, enterprise solution that connects 1000 employees with contact data, product information and task management… or someone simply wants to keep track of their oil changes. big or small, complex or simple… if people are going to get on board with a database system they need to know it works under all internet conditions, not just the ideal ones. if airtable is going to be widely adopted (which i believe it can become) it needs to provide solutions for users in a wide variety of circumstances, not just those working in modern cities. some of this may come across rude but actually i hold you guys in high respect for what you have created; just remember your customer base could be much greater when everyone, including low income, rural, low bandwidth customers can all use airtable and recommend it to their friends and family.


#24

since i ranted and forgot to answer some of the questions, i will respond to those now…

How would you want to use Airtable offline?
[see previous reply] journal, fitness, hiking, camping, fishing, maintenance schedules, tasks, photography, apps and software database, contacts, inventory, messages, calendar events, gift ideas, and lots more. my ideas are mostly for personal use at the moment but i also work for a company that has branches all over the world and i am considering using airtable to coordinate on quotes, item data, contact info, and other things that are currently exchanged via complex spreadsheets (yuck).

Do you use Airtable alone, or with collaborators?
currently alone, but i would recommend it more to others and find more collaborative uses for it if were not dependent on internet connectivity which varies from place to place, home to home, business to business, and street to street.

How much do you need to be able to do in Airtable while offline?
create, edit and delete records, but not change table structure or relationships

How tolerant would you be of having to resolve merge conflicts manually?
tolerant, if the changes were presented in a concise, easy to read format with a simple “accept” or “reject” button for each change, each table, each base, or all changes.

Would you primarily be using offline mode on desktop or mobile?
primarily mobile (iphone), however i also use mac at home, windows at home, and windows at work, and browsers everywhere so all platforms are important.

hope this feedback helps!


#25

CEO of Airtable here. Thanks everyone for your feedback so far!

I’ll address joshua_soule’s question, which I interpret as/distill into “can offline read/write access be thought of in the same way as online collaboration, but with exaggerated latency.” The answer is: not entirely.

Some Airtable user actions, such as editing a cell or creating/deleting new rows, are performed immediately on the client’s local data representation (see related concepts of “optimistic UI updating,” “optimistic concurrency,” and “eventual consistency”). Thus, they could theoretically be performed offline. In fact, you can do so today by disconnecting your internet connection without closing the browser tab or the iOS/Android app, making changes, then reconnecting—in absence of any conflicting changes from other clients, your offline edits should seamlessly sync.

The problem is that if changes are made from multiple clients simultaneously—whether by a team collaborator, or yourself from multiple offline devices—these clients increasingly diverge in their local states. In other words, if the Airtable server is the “real world,” each client gets more and more carried away with their imagination and starts to construct their own alternate reality. (Actually, this problem would remain even if you took away the Airtable server and instead made the system peer to peer, at which point you would have to grapple with consensus determination, which becomes increasingly complicated as the data model becomes more sophisticated, i.e. an entire relational database instead of simple numerical values).

Some amount of (or more accurately, “some types of”) divergence can be cleanly merged together: for instance, if you and I edit completely different cell values without making any other changes. In these cases, it’s obvious that our intended changes are completely independent of each other and can be trivially resolved. (For some very simple data structures, you can even construct a highly limited “world” where changes can always be cleanly merged together, but this doesn’t work for more sophisticated applications).

However, this isn’t always true. In some cases, our respective intents come into conflict. Within these cases, it’s sometimes still easy for Airtable to arbitrate our dispute and make a decision on what the outcome should be—in the case of two people overwriting the same cell value, the last edit received by the server wins. Note that for many cell types, Airtable’s behavior is actually a bit more advanced than “last-edit-wins”. For instance, if a multi select cell contains options Albatross and Buzzard, and you remove option Albatross from the cell (thus locally believing the resultant state to be Buzzard) while I simultaneously add option Cockatoo to the multi select cell (thus locally believing the resultant state to be Albatross, Buzzard, Cockatoo), Airtable actually resolves the resultant state to Buzzard and Cockatoo so that both of our intents are preserved, rather than simply clobbering one of our changes with the other .

Sometimes, though, there’s no clear way (not just for Airtable, but on a theoretical level) to resolve the conflicting intents—if one user changes a column, and another simultaneously changes values in the old column, for instance, it’s unclear what the human-desirable outcome is. One of the users’ intents must prevail (at least partially) over the other’s. This is where having 2 seconds of online latency is different from 2 days of latency (i.e. offline mode). With 2 seconds of latency, the most work you’ll lose or need to manually re-perform is 2 seconds. With 2 days of latency, you could potentially lose up to 2 days of work, as your subsequent changes could depend on the first change, which will be rejected. In other words, offline mode prevents you as the user from getting near-realtime feedback that a change being rejected due to another client’s conflicting changes, and instead lets you spiral increasingly into an invalid world of local state which becomes growingly impossible to resolve with the other clients’ world(s).

As far as I know, there are no database systems that gracefully handle all of the above problems. (Airtable is the first end-user relational database that handles fully realtime collaboration). We do believe this is an important feature, but there are necessarily tradeoffs to be made in its design. Your detailed feedback will help guide us to design the right tradeoffs.


#26

Hi Howie,

Thanks so much for taking the time to explain these obstacles in more detail. One solution comes to mind, but of course it has drawbacks like any other solution. I wonder if this might be a potential solution…

While still connected, a user may intentionally request (via toggle switch) to put a specific base in offline mode. In response to this request, the server notes the UserName and UTC timestamp of the request, sends a response back to the user’s device and the device makes a local copy and goes offline.

While offline, the user’s device prohibits structure changes to the base, such as addition or deletion of tables and fields, editing field types, formulas, relationship types, etc while offline. When the user requests to go back online, the device sends a copy of the base to the server, the server compares each table to existing tables and informs the user if any changes conflict with other user changes and if conflicts cannot be resolved via the timestamp and normal rules, then the server offers to export a summary of conflicts in the form of a common text file format (csv for example) containing the table names, record names and list of conflicts. The device should also include a warning message when the user makes the request to go offline, that offline usage carries some risks that incompatible changes could be made by other users while the device is offline and if that happens, a clean merge could become impossible when device goes back online. The user must acknowledge and accept the terms in order to enter offline mode. Maybe the base owner would also receive a copy of the request, or maybe just a summary of the conflicts, or none.

By making offline mode an intentional request, with a warning, and with specific limits, I think this would greatly decrease the chances of conflicts and offer a manual solution when conflicts did arise. If the offline user was not the base owner, maybe the base owner should receive the copy of the conflicts and decide what to do with them. Maybe the base owner could also accept / reject the requests to go offline. Or maybe the base owner just delegates users who are allowed to use offline mode, without further approvals. All just ideas that would definitely need fine tuning.

One last twist to this is that maybe other collaborators with edit rights can see a visual indicator when some users are working offline and popup a warning message if attempts are made to change structural and functional parts of the base, besides merely adding, deleting and editing data.

Thoughts?


#27

Hi Joshua, this is one (valid) proposal amongst a permutative spectrum of design possibilities. This particular approach is not too far off, directionally, from some of our internal explorations. As with any other possibility, though, it comes with its own set of advantages and disadvantages. For instance, it would be quite tedious to manually resolve conflicts. (I’ve personally experienced this pain as an Evernote user; conflicted notes/notebooks appear quite frequently in my usage of that product and require a frustrating amount of extra work to resolve.) Requiring an explicit toggle for offline mode may not help in situations where the internet disappears unexpectedly, or when one forgets to hit the switch ahead of time (i.e. you’re boarding a long flight, and neglect to make the change). In any case, even if the offline client does not allow structural changes, the same class of merge conflicts will arise if other online clients do perform structural changes during that time.

While we appreciate the suggestions for technical implementation, at this point the most useful thing to inform the design is to receive detailed, specific usage stories directly from as many users as possible. With those data points, we can better assemble the various technical mechanisms at our disposal into a cohesive experience that delivers the best set of tradeoffs, where best is informed by our users.


#28

How would you want to use Airtable offline?
I travel and I collect music. My main database is on my collection and I do need to check the database abroad, in the odd cellar and other strange places to avoid bying the same album more than twice. :wink:

Do you use Airtable alone, or with collaborators?
So far I’ve used my data alone, but I do see how I can share this with people who’s looking for a gift to give me.

How much do you need to be able to do in Airtable while offline?
Create and edit records. I do not need to change the database as such.

How tolerant would you be of having to resolve merge conflicts manually?
Somewhat … Conflicts should be presented in a clear and understandable manner and with a simple way of resolving those conflicts.

Would you primarily be using offline mode on desktop or mobile?
Offline mode on mobile only is more than acceptable. One can always find a Starbucks for other uses.

One last note; What I do need most is access to my data, anywhere, at anytime on any device. The last is important; when I travel I use Android & Chromebook, at home I use all the iStuff, PCs and Linux depending on where I’m at. But I do need to be able to have at least read access.


#29

I would be glad to have Airtable offline available exclusively on the read-only mode. It would solve some of the problems above, I guess.


#30

How would you want to use Airtable offline?
Not for profit providing services to seniors to help them remain in their homes.
Contractors and volunteers need to enter visit data. Ideally, much of this would
be pull down selections both from pick lists, i.e. service provided and selection
of actual data, i.e. client name. When client name is entered it would be good
to populate contact information like phone and address.

Do you use Airtable alone, or with collaborators?
Currently alone but planning for future collaboration.

How much do you need to be able to do in Airtable while offline?
Mostly add service data but to do so may require client identification.

How tolerant would you be of having to resolve merge conflicts manually?
In this case conflict management could be tolerated to a small degree, but since remote
data entry is being done by contractors and volunteers the level of technical expertise is minimal.
Typically the only type of management that might work is if they attempt to enter the same service record more than once.

Would you primarily be using offline mode on desktop or mobile?
Primarily mobile, but I suppose since we have a very mixed technology case
it could be either.


#31

Option to set the initial tables as single user upon creation? I like the form / structure / views of airtable as a database as a tool. if your not worried about collaboration and maybe just sharing read only views . That might be an option.


#32

Offline mode is difficult. Merge conflicts WILL occur for sure. But Please consider implementing a read-only offline access first. It will not result in any merge conflict but will be extremely useful. Think of it like printing off a document and take the paper version with you onto a plane.

As for how much data to cache? That should be easy to work out based on user browsing history. Show an “unavailable” sign for anything not cached.

I believe some offline data is better than no offline data at all. At the moment all my data is unreadable during a flight or on high speed rail trains!


#33

Having read-only access off-line would work perfectly for me. Think about creating a database of 1001 travel sites with associated restaurants and attractions (like a book). This could be built prior to travel and viewed on a laptop or phone at any location. It would be necessary to place the database in read-only mode while connected, which would be fine. Edits along the way could be added when reconnected to wifi.


#34

How would you want to use Airtable offline?
The First Tee of Greater Richmond uses Airtable to track student progress in our programs. Right now, we coaches use our own data to connect to Airtable on our mobile devices when we need to record information out on the golf course. If mobile data coverage is an issue out on the course, we are forced to write things down and perform data entry when we’re back in the clubhouse.

Also, coaches without a mobile device and mobile data plan must write everything down and record it later. It would be great to give them a WiFi-only iPad to use while coaching.

Do you use Airtable alone, or with collaborators?
With collaborators.

How much do you need to be able to do in Airtable while offline?
Ideally, everything we can do online other than modify table structure.

How tolerant would you be of having to resolve merge conflicts manually?
Honestly, not very tolerant. Our coaches have varying degrees of technical sophistication.

Would you primarily be using offline mode on desktop or mobile?
Exclusively mobile.


#35

Just a read only access for the existing database is huge for most people. it will be super helpful to quick search on the database even without internet connection!


#36

How would you want to use Airtable offline?
I want to use Airtable as a small, easy-to-use database tool for personal and small business use, whether online or offline. That, I thought from your marketing/branding, was the whole point of the app. I don’t want a Filemaker, I want a Bento. So all your worry about all these multiple users potentially making changes at exactly the same time that might mess up the database doesn’t make sense to me; that sounds like an enterprise app problem. I think you need to make up your mind what market you are targeting. As far as how I want to use it offline, I want to be able to make a few changes while I’m flying, either on the OS X app on my MacBook Air or on the IOS app on my iPad, when I’m up in the air with no internet access, and have them automatically sync next time I connect with wifi or cellular. Or maybe I’m out on my boat for the weekend, same thing. Or maybe I’m in a client’s office or the audience at a symposium and I’m having trouble getting the wifi password; there are many different scenarios where I temporarily can’t connect. I’m not looking to do huge amounts of work offline.

Do you use Airtable alone, or with collaborators?
Right now mostly alone, although I anticipate collaborating with a small handful of, say, 1-5 people.

How much do you need to be able to do in Airtable while offline?
Would like it to be fully functional.

How tolerant would you be of having to resolve merge conflicts manually?
That would be time-consuming (time-wasting), so not very tolerant.

Would you primarily be using offline mode on desktop or mobile?
Both.

I’ve been searching around for a goo, easy-to-use simple database program. Started playing around with HanDbase but don’t like the complicated syncing methodology, so thought I’d look for something better. Your product is getting good reviews, and looks good to me, and the syncing problem is non-existent, as long as I’m online. The inability to work offline is a deal-breaker. May have to deal with the hassle of HanDbase syncing, at least I can work offline.


#37

I am using Airtable in Africa. Internet connections are not stable. Dificult to be used when connections are interrupted. An offline modus could possibly prevent data loss and resume syncing when online again. Would be a very helpful feature, whcih would enable us to use the product on a much larger scale!


#38

I would really appreciate an offline/sync feature.
E.g. if you planned your trip at home using Airtable and in the foreign country you do not always have internet access but would like to see what you planned.

Maybe a checkbox for each base which will allow offline/sync until the next WLAN Hotspot is available.

Read only as a 1st version will be cool.
But add/update of data for existing bases should also be possible.

Regards,
Stefan


#39

Thank you for asking !!

How would you want to use Airtable offline?
Teacher/coach/artistic director,mo have a database of exercises and data about projects and evolution.
I (very) often work in places with not coverage, and one wifi.
I see much more a database as being based on my device and backed up on the net (+possibility to sing with my other devices).

Do you use Airtable alone, or with collaborators?
I use it alone.
I had Bento and iDatabase, but they stop supporting their apps :frowning:
(Bento is a good example as the sync was between my MacBook and the I devices, without an online backup. IDatabase offered the possibility to backup (and also sync) with Dropbox) would you consider that as a possibility?

How much do you need to be able to do in Airtable while offline?
I would say: make it fully operational offline, so to have full autonomy is using the database. And the online connection is a bonus to backup and sync.
I would literally need it to be that way in order to be able to use it.

How tolerant would you be of having to resolve merge conflicts manually?
I can’t really answer this one for now.

I find your online facilities to be VERY USEFUL and answering my personal need of a database :slight_smile:)
So the real thing that I miss is the full offline use.

Thank you so much !

Ami

Would you primarily be using offline mode on desktop or mobile?
Exclusively mobile.


#40

How would you want to use Airtable offline?
I use AirTable for work editing a magazine. When working on an aeroplane or in a foreign country, I want to access a contact database and publishing schedule. Otherwise, I use Workflowy for less structured information, which works well offline.

Do you use Airtable alone, or with collaborators? Alone

How much do you need to be able to do in Airtable while offline? Read only would be better than nothing. Otherwise editing and adding records. I wouldn’t need to make any structural changes.

How tolerant would you be of having to resolve merge conflicts manually?
I like manual.

Would you primarily be using offline mode on desktop or mobile? Desktop on the plane, but mobile in the taxi.