Reject new record when existing record with identical value in field URL exists

[Bastianbb]

Hi all 🙂

I am trying to avoid duplicate entries from making it to my airtable table.

Background: I am running multiple webscraping actors collecting entries from public sources multiple times a day. The actor itself can not determine which entries it already has collected, but just get's the 10 latest entries 5x a day. This way i assure that I get all the entries which are posted. However, when I now add the entries to airtable, it will create duplicates. I am currently using the "Datastore" of Make automation, which will automatically "reject" any duplicate entry as i am using the unique URL of the entry as the unique key in the Make datastore. This is desired as each time an actor runs, only the new entries make it to the DB. 

However, when trying to switch from Make Datastore to Airtable as a DB provider, I realised that airtable doesn't allow me to specify a "key" or "record id" when adding new entries.

Question: Is there anyway to make airtable reject new records from coming into the table if another record with an identical value for URL already exists?

I am aware I could run a script to automatically delete this duplicates at regular intervalls, but I thought I ask if there is a more elegant solution.

[ScottWorld]

You would use Make’s Airtable modules to do the rejecting before sending that data to Airtable.

There is a small learning curve with Make, which is why I created this basic navigation video to help people who are new to Make. I also provide the links to a few other Make training resources.