Help

This Product Ideas board is currently undergoing updates, but please continue to submit your ideas.

[SOLVED] Record Duplication Detection, Deduping, and Duplicate Merging

cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Aaron_Owen
7 - App Architect
7 - App Architect
Status: Launched

I would love to have a feature that would search a given table by a field or set of fields to find and surface duplicate rows. It would then be great to be able to take a pre-defined action on duplicated rows such as delete, or merge. Dealing with dupes is always a pain with large data sets or importing new data into a set where some records may already exist. A way to deal with dupes in a somewhat automated way would be really helpful.

93 Comments
Julian_Kirkness
10 - Mercury
10 - Mercury

Just thought I would add a suggestionโ€ฆ

You could find duplicate rows by using a formula field to concatenate the fields you want to match by and then Grouping a view based upon the formula field. You could then simply look through the data and find any group with more than one record in and act accordingly. Data reduplication is very often NOT an exact science - you may have to manually edit data from one record to another for example - so this is quite a reasonable approach.

If only you could filter by groups with a record count of > 1!

Julian

Matthew_Chapman
4 - Data Explorer
4 - Data Explorer

Not sure how many people will need to say it before it gets prioritized, but Iโ€™m adding my voice to this chorus.

Kevin_Murray
5 - Automation Enthusiast
5 - Automation Enthusiast

Another voice. I need this to detect duplicate contacts, similar to the way Google Contacts works.

ripwit
6 - Interface Innovator
6 - Interface Innovator

Iโ€™m using a calculated field for my primary key, concatenating a few other fields. I would have thought that Airtable would throw an error, or at least not store duplicate records when the primary key is the same.

This is totally opposite the way other RDBMSs work. The primary has to be unique.

ripwit
6 - Interface Innovator
6 - Interface Innovator

Wow! Thanks for the workaround but it seems very convoluted. Should be baked into the implementation!

Christoff
7 - App Architect
7 - App Architect

In Airtable, thereโ€™s no such thing as using a field for โ€œyourโ€ primary key. Airtable uses an internal field that is not visible through the user interface but is accessible through the API.

The first field at the left in Grid View is not the primary key. Itโ€™s what shows up in a linked record and it can be changed without affecting the link. Thereโ€™s no need for it to be unique.

ripwit
6 - Interface Innovator
6 - Interface Innovator

Thanks for the explanation. I guess I mentally glossed over the difference between the primary field and primary key.

Geoffrey_Henry
5 - Automation Enthusiast
5 - Automation Enthusiast

Iโ€™m looking for a way to delete and/or merge duplicates as well!

Polen
7 - App Architect
7 - App Architect

I see people are still suffering because of this, so I want to bring up this suggestion which didnโ€™t seem to get much attention and it will absolutely help those who really need to find [at least some kind of] duplicates with some ease.

Helmi
6 - Interface Innovator
6 - Interface Innovator

As with lots of other issues this ia longly requested one.

Iโ€™ll add my +1 also by making a suggestion: How about allowing to filter grouped results.

Example: If I group by an email field in a table of 5000 entries with 10 duplicates, i can find duplicate email entries but I have to scroll through 4990 grouped rows. If I would be able to filter out grouped results with a result count <= 1 this could be much easier.

This feature would also help in other situations.