Automate CSV Import

Hi,
I’m looking for a file-linked or import-only table so that we can regularly push data from our CRM into our Airtable project management app without having to manually re-import the CSVs every day.
It would be nice if we could schedule import or cause the re-import/update to be triggered whenever the CSVs are changed.
-Donald

2 Likes

Have you looked into Zapier? You can use it to connect Airtable and Google Sheets, and set it up so that when new records are created in Sheets they are also created in Airtable. I believe finding/matching/updating records when records are modified (rather than created) can also be done, though it is slightly more complicated.

2 Likes

You can also connect Excel (Office 360) and Airtable through Zapier

1 Like

An app automatically creates a CSV in my Dropbox. How would I automatically update an Airtable view with this information? Using Zapier?

1 Like

Something that might be relevant: one of the blocks from our newly released Airtable Blocks platform is the CSV import block.

Now, the CSV import block doesn’t import CSVs automatically. However, the CSV import block will remember your field mappings when you import other CSVs in the future. That way, all you need to do is drag and drop the CSV file onto the CSV import block, and click one button.

import%20csv%202

3 Likes

Are there any plans to somehow allow import of CSV files to be triggered via Zapier or similar. Being able to automate the import of data from a csv would be incredibly powerful for our use cases.

Automated import depends on two very key requirements;

  1. The generation and the delivery approach of the source [CSV] data, and …
  2. The acceptance/ingestion mechanism for the target database.

Let’s say that Airtable provides a means for accepting CSVs and through it provides the automated loading of the data into a table without any human interaction. This addresses only half of the technical requirements; the CSV has to be delivered to Airtable in a manner that supports an automated ingestion gateway.

Tell me, how would the source CSV get generated automatically and then be conveyed such that a new Airtable ingest feature could receive it? What communications protocol would it use? And what security measures would it require?

Hi Bill, I was thinking along these lines…

  1. Our stock management system exports the required data to a dropbox folder in csv format.
  2. This event triggers Zapier.
  3. Zapier triggers the Import csv block that is pre-configured with the correct mappings to update the relevant records in Airtable.

Does that answer your questions?

I get the sense that this process is currently working for you, right?

Why is the CSV import block (or any new Airtable feature for that matter) needed when Zapier is perfectly capable of placing the data directly into the target table?

Is it because you believe …

  • Airtable should eliminate the need for the Zapier route and provide this functionality in a more seamless manner?
  • The Zapier route is needlessly complex or costly?
  • The Zapier integration is unreliable?

My curiosity is to understand the underlying requirements and how they could be made better.

In case it’s not apparent, I have a proclivity to avoid CSVs altogether and not withstanding legacy systems integration, the answer to automated CSV integration is to not do it. :wink: This is why they made the API.

CSVs come with some serious issues:

  • Security issues
  • Delimiter issues
  • Typically not automatically generated
  • Conveyed through email [predominantly]
  • Concurrent updates are problematic

In contrast, APIs avoid all of these issues.

Yes, Bill you are right. I am sure API would be a far better solution. The truth of the matter is that the relative difficulty of generating good api connections (for the uneducated), as compared to using already available simplistic tools built into both our stock management platform (scheduled csv export routines) and the Airtable Import CSV Block is what is putting me off going down the api route. One day I will bite the bullet and get a handle on developing api connections :slight_smile:

I am interested to hear that we could use Zapier to inject data to an already formed Airtable table though.

To explain in a little more detail, we have a table with a list of products and values associated with each product (Product SKU), such as inventory (Qty). Would we be able to use Zapier to inject these Qty values, thus overwriting those already in Airtable by matching records agaisnt the Product SKU (in the way we can do using mapped CSV import), rather than creating new records? If so, do you have any links to where I could best read up on how this is done?

Please forgive my ignorance here, I am relativity new to Airtable and have little experience of Zapier, neither, unfortunately, am I a programmer (one of the reason I am using this platform). That said, I do understand data integrity and the fundamentals of relationship databases etc.

I am really very greatful for your time and input here. :slight_smile:

No worries; the best innovations are created by business people, not coders. :slight_smile:

I think Airtable would love to make every feature possible where otherwise, code or complex Zapier configurations are required. But, these are resource-intensive features and reliance on their after-market-place is necessary.

I believe there is a way to do this without using the API, but since I tend to work more in API-land, I defer to a few of the Zapier and Integromap experts here in the community. I might also call out to @openside because I think they have a special toolset to do exactly this without writing code - could be wrong, but worth an investigation.

1 Like

@Katherine_Duh: How are you getting the CSV import block to remember your field mapping for future imports? That only seems to happen for me if I don’t need to import a CSV with a different set of field names afterward.

My main request would be if AirTable would let you actually save a mapping of field names, so you could load it later, saving time and preventing errors. I don’t need a full automation of CSV import, but that would help greatly.

Zapier doesn’t solve the issue for me because the size of our CSVs is too large for their 100/record/time rate limit, and we don’t have a Plan to cover the number of Tasks/mo. that would be needed.

Do AirTable staff ever review and reply to feature requests?

I believe this is now possible in a script block.

@Bill.French: Thanks so much for the suggestion!

I’ll look into this further. For now, I don’t have much familiarity with the Scripting Block, since I’ve never developed in Javascript except for display-layer on websites.

This was the only seemingly-related discussion on the Scripting Block forum, but it didn’t have specific code: Script - Use form to intake csv file that adds to existing table

Is there any other example yet, that you know of?

Note that I had started working on a mapping between a SQL database and my AirTable Base using the server-side node.js API prior to Scripting Block being released, but my data model has changed since then, so it would be ideal if I could define and save some kind of mapping - like you can when doing a CSV import in Salesforce, or setting up a Zapier Zap to import to AirTable from Sheets.

Then you are qualified. :wink:

It’s really not that different. The biggest change is to learn the interfaces into your Airtable tables, records and fields. There are basic functions that allow you to read the records and access the field values. Once you get those basics down, you are just doing every-day javascript stuff to create the process and perform the steps for the app.

Yes, when you install your first script block, you will see the documentation which includes a number of good examples.

@Bill.French: Thanks - I did see some of the examples in the scripting block, but they looked fairly basic, and I didn’t see one that would interface into the CSV import block, or else create a way for me to do CSV imports. But I may need to look harder.

Well, CSV imports are achieved typically by reading the file from a web location (like Google Drive) or from a field that pulls in an attachment via an Airtable form. There’s no way to access existing blocks from the script blocks so you need to write the block in a way that performs the entire process.

I’m pretty sure you could read the CSV documents from a field in a table and then perform an import into another (or same) table.

1 Like

Ah, thanks - now I understand.

and… the entire table/field schema is accessible in script blocks so you could create detailed mappings and then sustain those mappings in – wait for it – Airtable itself. :wink:

Ok, I did see an example that lets you pull the schema via Scripting Block. So yes, I could create a meta-table so that others besides me could revise the mapping.

Great suggestions!

1 Like