I would love to have a feature that would search a given table by a field or set of fields to find and surface duplicate rows. It would then be great to be able to take a pre-defined action on duplicated rows such as delete, or merge. Dealing with dupes is always a pain with large data sets or importing new data into a set where some records may already exist. A way to deal with dupes in a somewhat automated way would be really helpful.
[SOLVED] Record Duplication Detection, Deduping, and Duplicate Merging
+1 for auto detect dups! Major weakness of the platform and tons of ppl looking! and without a COUNTIF function all the workarounds are too much for a simple ask.
+1!
Some more context on my use case…
I am using Airtable to manage an events calendar which my organization (~100 people) uses too see what events others are going to, speaking at etc. I’m also encouraging everyone to contribute to the events calendar by filling out a form.
I’m looking for a mechanism that will stop people from recording the same event twice, and for a way to add or edit existing events on the calendar.
I’ve been struggling with this problem for a long time also, but I walked through @W_Vann_Hall’s demo and it was extremely helpful. I was looking (like I think lots of ppl are) for a way to constantly check for duplicates, so the ARRAY issues concerned me.
Here’s my attempt to simplify the demo:
- My own example table is posted here: https://airtable.com/shrEZNMgiKrQdv8lp/tblcywQ2RsDL3H9iH/viwd63P4gD3yMRUqD
- Generate a “test duplicates” base with ARRAYJOIN roll-up to your list (this creates a string variable to pass back to your individual list items
- In your main base, pass the list back using an ARRAYJOIN roll-up. I know this is counter-intuitive since you are joining only 1 other, but its necessary to read the column as a string
- Create a formula to detect duplicates:
IF(LEN(List) - LEN(SUBSTITUTE(List,Name,"")) > LEN(Name),“yes”,“no”)
Hope this is helpful for others!
I have a problem with the final step…
Could anyone help me with this: ??
Please, please, this has been requested for so long and is a pretty obvious feature, especially in a relational database where I want my primary key to be unique!
I wonder what the technical issues with this are. Surely at least duplication DETECTION shouldn’t be such a hard thing to implement?
Hi folks! Our deduplication block is currently in beta right now. You can use it to delete duplicate records or merge individual fields from duplicate records together.
If you’re a Pro plan user and you’re interested in testing it out (and giving us your feedback!), we’d love for you to sign up using this form.
Any update on this @Katherine_Duh? The form to sign up doesn’t seem to be available anymore.
The form loaded fine for me by clicking the link @Katherine_Duh posted above:
(just now)
And I can update you that the beta is live. I have been able to try out the De-duping block.
+27They have officially released a de-duplication block for pro users. More information here: https://support.airtable.com/hc/en-us/articles/360008332253
+19- Employee
Looks like @Kamille_Parks beat me to the community forum announcement — the Dedupe block is now officially launched and available to all Pro users! You can install it right now from the blocks gallery.
Special thanks to those of you who participated in the beta and gave us valuable feedback. Happy deduping!
+2Just a heads up that this thread will automatically close in 24 hours. Thanks for the discussion, everyone!
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.