Help

Webhook payload - still limited access to nested data?

Topic Labels: Automations
4406 20
cancel
Showing results for 
Search instead for 
Did you mean: 
Luke_Concannon
6 - Interface Innovator
6 - Interface Innovator

Tena Koutou!

I got momentarily excited by the received webhook trigger in automations (old news I know, just got round to it seeming like a time saver!) only to learn that there is limited access to nested data within the json payload…

I’m sending a payload via webhook from Typeform => Airtable. Works great. I just can’t access the nested data I need. super frustrating. Seems like a fairly standard issue feature to be able to access nested data within a JSON object.

Has there been any update to this? Anyone have any work arounds that dont involve third parties? I’m happy proxying the webhook through vercel serveless function but just seemed like a quick win to manage it in airtable.

Any advice hugely appreciated!
Luke

20 Replies 20

You should be able to write a JavaScript to access the nested data.

Otherwise, you would need to use a 3rd-party service. For a no-code webhook solution, I personally use Make:

There has been no update to this as far as I know.

If you know the exact shape of the data, you can use individual input variables to get a specific nested value.

But overall, you are correct that you need some third party system. One third party option is to JSON stringify the payload and store that string in a single key. (Essentially stringifying twice.) Then have a script parse the JSON from that single key.

Should you? Yes. Can you? No.

The entire point of inbound webhooks is to avoid middleware that increases solution complexity and latency.

This is only possible if you control the inbound hook. The entire point of supporting webhooks is to enable arbitrary conveyances of data payloads, many of which are predictably complex and varied such that the webhook listener must be able to process the payload with action scripts.

While this seriously critical requirement continues to make integrations vastly more difficult than they should be, Airtable focuses on improvements like these.

image

I get that your complaint is with Airtable’s limitations, but if it’s our only solution at the moment, then that’s all we can do for now. :man_shrugging:

I’m not dismissing your very legitimate complaint about Airtable’s engineering focuses.

Also, it looks like incoming webhooks are still in Alpha. At least that is what the user interface still says. This issue might be one of the reasons why it is still in Alpha.

Yeah, I’m good with that as an answer, but after 14 months in “alpha” testing and tens of thousands of actions depending on it, I’m less inclined to be kind. After all, this shortfall is simply missing the ability to map the root of the JSON payload into a field or a parameter; all other elements are mappable, just not this one.

I see this more as an indication that it is a low priority for Airtable, not as a need for more testing. I’m guessing that Airtable knows that this is an issue, but that it is not their focus, and so they are choosing to leave it in alpha because they know they can’t reasonably take it out of alpha without implementing this feature and they don’t want to spend their resources there now.

@ScottWorld - That’s clearly because you have some sort of commercial relationship with them though right? You’ve been recommending this third party on every thread I read yesterday.

@Bill.French are all other elements mappable? The data I need lies 5 layers deep and I can’t figure how to access it. In trying to map these to inputs I can only access array.length or make new lists of properties that exist within the object.

webhook payload

data I need

Screen Shot 2022-09-01 at 10.26.00 AM

Luke_Concannon
6 - Interface Innovator
6 - Interface Innovator

It seems like such a basic requirement for a really neat feature. Like so basic that it actually renders the feature a veneer of what it is proposed to be.

@kuovonne @Bill.French I have to say I fully enjoyed the time spent reading through your back-and-forths. very playful :winking_face:

If you know of better solutions than the ones I’ve offered, feel free to post your better solutions. If I knew of better solutions, i would post them myself. I used to recommend Zapier until I discovered Make. If there was a better tool than Make, I would recommend that one.

As a top Airtable consultant, my entire job is helping people find the best, easiest, cheapest, simplest tools for their needs.

I am a Make affiliate, but i have no other relationship with the company. I have volunteered 840 days in these forums helping people out — for free. Solving other people’s problems. If you have a problem with the $25 per month that I make from my Make affiliate link (which in my personal opinion is often the best tool for the job), then I don’t know what else to tell you. It would be nice to show a little gratitude. :man_shrugging:

fair play. It was a spicy response on my behalf and I apologise for trolling. I was just incredibly explicit when I stated:

Good on you for helping people out bro. And loving the link dropping to your website and airtable profile :joy: . But seriously, thanks for your contributions.

I’d be hugely appreciative if you could expand on:

I assume you mean bash some script to access the nested data in the webhook payload. Can you take a look at my specific use case and help me ‘write a javascript’ to get to what I need? Because I dont think its possible, but just wanted this confirmed so I know to write this webhook trigger off as a potential solution.

No worries! Apology accepted! :slightly_smiling_face:

Ha, you are 100% correct… you specifically asked for no third-party workarounds. I apologize for ignoring your request on this.

In case it makes a difference, there was actually a method to my madness. I was trying to provide an answer that wouldn’t just be helpful for you, but that would also be helpful for any visitors in the future who searched on the words “webhook payload” or “nested data” — which would then hopefully save me time in the future from typing up the same answer again.

I had noticed that you already mentioned Vercel as a 3rd-party solution, but I went to their website and I couldn’t figure it out, so I thought that it wouldn’t hurt to mention another 3rd-party solution that might be easier for people. (Also, I’m not financially motivated to recommend Make. I’m just super-passionate about their product, and I use their product almost every day with my clients. The fact that they throw a few dollars my way is nice, but that’s just icing on the cake.)

Oh, I’m so sorry about not being more clear on this. I actually don’t know Javascript at all — which is actually the reason why I use Make so much! Lol. Make is my substitute for JavaScript.

When I wrote this, I was under the impression that someone with JavaScript skills “should” be able to access the nested data. This was just an assumption on my part. Since I don’t know JavaScript, I don’t actually know for sure with 100% certainty that this can be done. Both @Bill.French and @kuovonne are the JavaScript experts, so they would be able to tell you if this can be done with a JavaScript, and if so, they could tell you how to do it.

Correct, it can’t, given the current constraints in Airtable.

This is at the crux of this discussion. Not only can it not be done in the context of an Airtable webhook listener, but it is also in direct conflict with the entire purpose of a webhook listener. It’s one thing to say that Airtable supports inbound webhooks; it’s an entirely different claim to say that it supports arbitrary webhooks from a third-party system that gives you ZERO control over the JSON payload. That said, it’s important to point out that …

VASTLY! Web services that provide outbound webhook capabilities DO NOT allow users to shape the JSON payloads.

This is tantamount to building a really nice yacht with integrated yacht-rock-radio (i.e., webhook listeners in Airtable) but delivering it without a bow. Ninty per cent of the boat is cool and functional, but it still does not float.

This is precisely the type of statement that conflates, confuses, and misleads readers.

… done with a javascript …

There are no predicates, and because there is no context, it reaches into the stratosphere of ambiguity. Topics like this require predicates so that we can all understand the contexts of our comments.

  • Done with javascript running on a middleware proxy like AutoCode. (possible)
  • Done with javascript running on a middleware proxy like Make. (possible)
  • Done with javascript running on a middleware proxy like Zapier. (possible)
  • Done with javascript running inside an Airtable automation script. (not possible)

Bottom line - asked and answered (many times) - it can be done with javascript, but not in a manner that is without inelegance.

Hi,
As I’m currently experimenting with webhooks between GCP and Airtable in both directions,
I take the permission to add a small summary of this thread so far,
because it helps me, and publicly in case it could help someone who starts reading this thread now.
Thank you all for your generous thoughts and Q/A and see you soon,
oLπ (feel free to quote me & edit me if I would have misunderstood something.)

?

If you are able to build both sides of the integration process - i.e., Google Cloud Platform sending the webhook, you can serialize the entire JSON payload and thus sidestep this issue; Airtable’s webhook is able to capture the entire payload as a single serialized element and do as it pleases with the entire payload.

Thank you @Bill.French , I suspected as much but I didn’t want to talk about it more without having something actual to show.
So thank you for explicitly stating it: it is a positive fact in the context of my present GCP playground.
oLπ

Adam_Hodson
5 - Automation Enthusiast
5 - Automation Enthusiast

For anyone still dealing with this issue I was able to create a workaround by using a Zapier Webhook to receive my data first, transform it, and then pass to the Airtable webhook.  I know this is a third-party solution and you would need a premium zapier account to use the webhooks, but it's a much quicker setup than shifting to make or GCP I would think. 

@airtable Where are we with a resolution on this? The webhook featured is seriously handicapped by this bug/limitation.