Dealing with duplicates

[Pete_Thompson]

Hi All, 

I'm putting a Zap together and linking our recruitment ATS with Airtable. I'm adding job adverts that come from our ATS in to Airtable first, and then I can send to another platform via the Zap. I've run in to an issue that I hope Airtable will solve. Our ATS sends job adverts to 3 job boards, so when it comes from the ATS to Airtable, it enters the job advert record 3 times. So it's duplicated twice. Unfortunately the ATS record has a data field that has all 3 job board ID's together so I can't filter the record by just one job board.

Is there a way I can mark the duplicates with a true/false or checkbox field using some form of automation? I can then filter them out in another view and use the view to pass on the information to another platform from Airtable via the Zap without any duplicates. I hope that makes sense! Any help would be much appreciated. 

[ScottWorld]

Have you checked out the DeDupe extension? It can't be automated, but you can manually run it whenever you need to clean up duplicates.

[Pete_Thompson]

Thanks Scott. Yes I saw that. Unfortunately I can't use that as a workaround as it's too manual and those duplicate records would have created new records via the Zap on another platform. I'm waiting for our ATS provider to fix their api, but it won't be a priority. 

I tried to do an automation: when a record matches conditions, find records, then update duplicate records my check box for duplicate field. But I couldn't make "find records" find duplicates of the original. 

[ScottWorld]

Yeah, I don't know of any way to do it with Zapier... I only know how to do it with Make, which would let you delete all but one of the found records. That's because the search will always return ALL of the records... so you have to tell it to skip over one of the records. Make can do this, but Zapier can't.

Also, if you know JavaScript or can find someone who knows Javascript, this can likely be automated with a Javascript. 

[Pete_Thompson]

Thanks Scott. I'll have a look at Make. Perhaps I could Zap records to Airtable from our ATS, and then use a Make for Airtable to Webflow. WIll look into this. 

[TheTimeSavingCo]

Hm, what if you had an automation that runs when a record's created in the table, and it'd have a "Find Record" action to find a record with the same name (or however you're identifying duplicates), and if it did find a record that had the same name as the triggering record, it'd check that "Duplicate" checkbox you mentioned?

[Pete_Thompson]

I like that idea Adam. I tried that the other day using the Job Ad ID as that is a good duplicate identifier. See attached. When I added a record manually to test it, it would check the duplicate checkbox even if it wasn't a duplicate. So I gave up at that point. I was going to use the duplicate checkbox to filter in another view. 

[TheTimeSavingCo]

Hmm, your automation doesn't have a conditional logic step, so it's always going to mark the checkbox?

Could you try adding a conditional logic step?

[Pete_Thompson]

Aren't there two conditional logic steps? If JA Ad ID isn't empty and if JA Ad ID is not JA Ad ID. Any suggestion on how I properly lay the logic out? 

[TheTimeSavingCo]

The logic for the steps would be whether the "Find Record" action found anything: if it did, it'd mark the checkbox, and if it didn't it'd do nothing

You've already created an automation with the conditional logic steps that isn't working I take it?  Could you provide screenshots of your new setup?