Skip to main content
Solved

Random 404 responses from airtable api


Did this topic help you find an answer to your question?
Show first post

36 replies

  • Participating Frequently
  • 37 replies
  • October 4, 2019
Bill_French wrote:

I’m not sure. My money was on the Zap or the data as the likely cause. @Paul_Kortman insisted it was Airtable.

Do we know what the true cause was yet? I have no clue how the user-agent is controlled in a Zap.


In reality, it may be a bit of both. I know Airtable has some weird Zapier-specific code, and it’s possible that Zapier stopped triggering that special behavior. So it could be everyone’s fault!


  • Author
  • Inspiring
  • 14 replies
  • October 4, 2019
EvanHahn wrote:

In reality, it may be a bit of both. I know Airtable has some weird Zapier-specific code, and it’s possible that Zapier stopped triggering that special behavior. So it could be everyone’s fault!


@EvanHahn I’m still waiting on Zapier support to respond to my latest message (from last night) but they did remind me of something I knew from testing but had no confirmation on:

If a webhook makes a request to https://api.airtable.com/v0/{appId}/{table name} it does not work with the User-Agent: Zapier header, which we automatically add. It returns a 404, “table not found” error. However, if you use the table id , or if you remove the User-Agent: Zapier header, it will work.

This is why I specified a user-agent of Postman, however the log files they provided show the opposite is true, when its sent with a user-agent of Zapier its sucessful with a {table name} and when sent with a user agent of Postman and a {table name} it fails.

I’m guessing you have it written in the API to detect the user agent of Zapier and then accept only {table id} for the API calls, correct? Why on earth would you do that in the API and not add a switch versus a user-agent detection?


  • Participating Frequently
  • 37 replies
  • October 4, 2019
Paul_Kortman wrote:

@EvanHahn I’m still waiting on Zapier support to respond to my latest message (from last night) but they did remind me of something I knew from testing but had no confirmation on:

If a webhook makes a request to https://api.airtable.com/v0/{appId}/{table name} it does not work with the User-Agent: Zapier header, which we automatically add. It returns a 404, “table not found” error. However, if you use the table id , or if you remove the User-Agent: Zapier header, it will work.

This is why I specified a user-agent of Postman, however the log files they provided show the opposite is true, when its sent with a user-agent of Zapier its sucessful with a {table name} and when sent with a user agent of Postman and a {table name} it fails.

I’m guessing you have it written in the API to detect the user agent of Zapier and then accept only {table id} for the API calls, correct? Why on earth would you do that in the API and not add a switch versus a user-agent detection?


I believe that’s the case, yes. I also think there are some other pieces of the API that behave differently in this case, but I can’t elaborate on the specifics.

A lot of these decisions were made before I even joined the company, so I don’t have a lot of historical context on this. If it’s important, I can try to trace down the answer, but I’m not sure we’d even be able to share that publicly even if I did get the answer.

If it were up to me, we would have no Zapier-specific code on our end, but we have a little bit, which might be explaining your issue.


  • Author
  • Inspiring
  • 14 replies
  • October 4, 2019
EvanHahn wrote:

I believe that’s the case, yes. I also think there are some other pieces of the API that behave differently in this case, but I can’t elaborate on the specifics.

A lot of these decisions were made before I even joined the company, so I don’t have a lot of historical context on this. If it’s important, I can try to trace down the answer, but I’m not sure we’d even be able to share that publicly even if I did get the answer.

If it were up to me, we would have no Zapier-specific code on our end, but we have a little bit, which might be explaining your issue.


aka I could tell you but then I’d have to kill you…,… nice

I don’t need the answer, just someone else to commiserate with me that having an API behave differently based on a user-agent is not best practice and kinda absurd imo.

We’re on the same page here.

I think the most frustrating thing is the lack of consistency within Zapier, it should always succeed or always fail if it were consistent in sending over the user-agent I specified.


  • Author
  • Inspiring
  • 14 replies
  • October 4, 2019
EvanHahn wrote:

In reality, it may be a bit of both. I know Airtable has some weird Zapier-specific code, and it’s possible that Zapier stopped triggering that special behavior. So it could be everyone’s fault!


To my point with @Bill.French it isn’t Zapier that is mis-interpreting the error. (It is indeed Zapier’s fault for not being consistent) but (and I don’t know how the logic would work on this) perhaps there is a better error than 404 when a table name is given but a table id is expected… perhaps @EvanHahn that should be added to the future features, even if it is a 404, to be able to say in the error message a table ID was expected ... table IDs start with tbl etc OR something like that, so when someone is passing a table name and getting an error they know a table id was expected, and vice versa.


  • Participating Frequently
  • 37 replies
  • October 4, 2019
Paul_Kortman wrote:

aka I could tell you but then I’d have to kill you…,… nice

I don’t need the answer, just someone else to commiserate with me that having an API behave differently based on a user-agent is not best practice and kinda absurd imo.

We’re on the same page here.

I think the most frustrating thing is the lack of consistency within Zapier, it should always succeed or always fail if it were consistent in sending over the user-agent I specified.


I agree, and I disagree with our old decision. There may be some additional logic, too, and the User-Agent check just verifies that it’s coming from Zapier (for example, we might check that the incoming IP addresses matches Zapier and double-check the header, too), but I am not sure. In any case, I wish we had no code specific to a particular vendor. In practice, that sometimes has to happen for various reasons.

I’ll keep that in mind!


Forum|alt.badge.img+19
  • Inspiring
  • 3264 replies
  • October 4, 2019
EvanHahn wrote:

I agree, and I disagree with our old decision. There may be some additional logic, too, and the User-Agent check just verifies that it’s coming from Zapier (for example, we might check that the incoming IP addresses matches Zapier and double-check the header, too), but I am not sure. In any case, I wish we had no code specific to a particular vendor. In practice, that sometimes has to happen for various reasons.

I’ll keep that in mind!


@EvanHahn - Thanks for helping with this issue.

Setting aside the absurdity part, I agree with Paul on this point:

I don’t think anyone intentionally tries to infuse absurd concepts into an API - they happen over time when it’s too costly to repay code debt. And I do know that there was a time when Zapier was cutting corners to make stuff work too.

Zapier comes with a lot of benefits and some risks as demonstrated in this thread. Any time middle-ware machinery is added to process automation, there are increased risks that have to be factored into the make-or-buy equation. Bring an off-road vehicle; it could get a little bumpy.

We can avoid pitfalls like this by building the middle-ware from scratch (my preference), but you need to bring a paving machine if you want a smooth road.

All said and now better understood, I don’t feel compelled to buy Paul a beer, but I think you both could use a beer so I’ll try to work out those logistics.


  • Participating Frequently
  • 37 replies
  • October 4, 2019
Bill_French wrote:

@EvanHahn - Thanks for helping with this issue.

Setting aside the absurdity part, I agree with Paul on this point:

I don’t think anyone intentionally tries to infuse absurd concepts into an API - they happen over time when it’s too costly to repay code debt. And I do know that there was a time when Zapier was cutting corners to make stuff work too.

Zapier comes with a lot of benefits and some risks as demonstrated in this thread. Any time middle-ware machinery is added to process automation, there are increased risks that have to be factored into the make-or-buy equation. Bring an off-road vehicle; it could get a little bumpy.

We can avoid pitfalls like this by building the middle-ware from scratch (my preference), but you need to bring a paving machine if you want a smooth road.

All said and now better understood, I don’t feel compelled to buy Paul a beer, but I think you both could use a beer so I’ll try to work out those logistics.


I can’t speak for Paul, but I’m good on the beer front—just glad I could help!


  • Author
  • Inspiring
  • 14 replies
  • Answer
  • October 9, 2019

Update from my side on this… a Zapier engineer discovered that I was using user-agent in my webhook header and not User-Agent the capitalized version would overwrite the default User-Agent:"Zapier" whereas the lower case would be sent along with the default upper case.

We did see that the headers are sent in somewhat of a random order each time, so sometimes when two “user-agent” headers were sent the User-Agent:"Zapier" would go first, sometimes thw user-agent:"postman" would go first, thus the randomness.

Our guess is that airtable took the first one (regardless of capitalization) and would opperate that way.

So it is my fault for not capitalizing the U and the A… but also Zapier’s fault for sending headers in a random order, and they couldn’t explain why that happens.

Thanks for your help in diagnosing this problem!


  • Participating Frequently
  • 37 replies
  • October 9, 2019

Glad we could get this solved!


Paul, you’re a lifesaver!

For those wandering through who bumped into this same problem hitting the Airtable API from a Zapier webhook, the simple fix is this: add a header User-Agent with any value other than Zapier and your call will go through.


Reply