[Misdiagnosis, Airtable not at fault] API reporting creation success but records missing

[Simon_Bowyer]

I noticed three records missing from a table, which had been created via an API call. The API call response indicated success, including a response object containing the same number of records (and with recIDs) as had been POSTed in the call. However, these records do now not appear in Airtable. There is no record of manual deletion in the base trash, and the view is not filtered.

I was wondering what could be going on and so decided to look at an automated Airtable snapshot taken on the same day. The records were not in the snapshot, but I did notice that the snapshot was taken at the exact same minute that the API record creation POST call was made for the missing three records.

Is it possible that the snapshot process has interfered with the record creation call in some way (perhaps a DB transaction lock or similar), resulting in the Airtable API response to prematurely report successful creation?

[ScottWorld]

I would double-check your API call to make sure you’re calling the same base ID (the “app” ID) & same table name that you’re looking at now.

If you’re sure it’s all the same, then I would email support@airtable.com about it, and post back here with what they say.

[Simon_Bowyer]

Thanks – a useful double-check. We actually record the base id and timestamp of the record creation API call in our SQL DB, and so I am certain of which base the call was made to. I shall ask support directly.

[ScottWorld]

Cool, post back with what they say!

[Simon_Bowyer]

A screenshot of the Airtable snapshots on that day (British Summer Time), and of our separate record of the API POST call of the missing records (UTC). BST is 1 hour ahead of UTC.

[ScottWorld]

@Simon_Bowyer

Was that the only time that your API call failed?

If so, it’s extremely likely that you’ve stumbled upon a major & very significant bug in Airtable. (And there is no shortage of bugs in Airtable, so this wouldn’t be unusual.)

And very good sleuthing on your end to figure out what causes this bug! If a snapshot is being taken while an API call is being made, then Airtable will lose the API call.

This definitely needs to be reported to them.

[Simon_Bowyer]

I’m not so worried about why a record wasn’t created (and it may not be snapshot interference, that is just one suggestion). It’s that the create records POST call had a 200 response, and the number of records in the response object matched the number of records sent. I.e. it was reported by the Airtable API as a successful call.

I’ve begun writing a validation method to check that the response object contains the expected keys, and that the id begins with rec, and the createdTime exists and is a datetime string.
I’ve also begun writing a verification method which collects the record ids from the response object and then immediately calls GET (using a filterByFormula) to make sure the records were actually created by Airtable.

Hopefully these two extra methods will shine some more light on what is going on, but it would be nice if the Airtable API responses could be trusted! [Edit: the Airtable API responses were trustworthy all along]

[ScottWorld]

Well, you shouldn’t have to go through all of that, obviously. And 100% of my API calls have always resulted in accurate responses thus far. So it’s either a bug, or something else is going on with your setup.

1. But you didn’t answer my question from above: Is that the only API call that failed? The one that happened during the snapshot?

2. Please report this to Airtable Support, and let us know what they say!

[Simon_Bowyer]

The API call reported as succeeded, but the outcome appears to have failed. It’s the only one we noticed so far and not attributed to manual deletion or ‘something weird’ and moved on without investigating. The vast majority of our record creation calls do actually create records, but for a while we have had occasional, rare, mentions by our staff that something which was expected wasn’t there.

[Simon_Bowyer · Accepted solution]

I asked Airtable support, and they asked some good, detailed, questions. These questions led me to check our code which produces the exported_at timestamps, and I found that the name of our column exported_to_airtable_at would actually be more accurately named considered_for_export_to_airtable_at. In this case the three records were considered by our code, found unsuitable, and so never made it into an Airtable API POST call at all.

So, the Airtable API is completely without fault, and the snapshots were a red herring. The temptation to jump to a conclusion when a coincidence appears is strong, and I succumbed.