"Can I pull data from a website into Airtable?" comes up here every few months, so here is a short map of the options I have tried, with the point where each one stops working well.
Disclosure first: I make the three Universe bases linked at the end, so read it knowing that. Everything below works without them.
1. CSV export + import
Works on every plan. Most review sites, Reddit tools and job boards can give you a CSV somehow.
Where it stops: it is manual. Re-importing creates duplicates unless you have a stable ID field and merge on it.
2. Scripting (extension or "Run script" automation action)fetch() can call any public JSON API and write records. Good for small, clean APIs.
Where it stops: most sites have no public API or block plain requests, automation scripts have a short run-time limit, and the Scripting extension is not on the Free plan. You also own the maintenance when the site changes.
3. Sync / fetch extensions and Zapier-style tools
Good when the source already has an API or an integration (RSS, a SaaS app). Scheduling is built in.
Where it stops: they still need an API on the other side. For a page that only exists as HTML (a review page, a subreddit search, a job board), there is nothing to connect to.
4. A scraper that runs elsewhere and writes into Airtable
This is what I use for sources without an API. The scraper runs on a scraping platform under your own account (I use Apify), and its Airtable integration writes each result as a row. You connect it with OAuth, so you never paste an Airtable token anywhere. Schedule it, and a stable ID field (review ID, post ID, job URL) stops duplicates.
Where it stops: you pay per result to the scraping platform, and you depend on the scraper staying up to date with the site.
What made the biggest difference for me in the base itself
- One unique ID field from the source, and merge on it. Without it every scheduled run makes duplicates.
- Your own fields next to the imported ones: Status, Topics, Next action date, Notes. The imported data is read-only in spirit; your fields are where the work happens.
- A kanban view by Status, plus 2-3 filtered views for the things you act on (low ratings, no reply yet, new this week).
- A small "Start here" table inside the base with the setup steps, so a teammate can rebuild the connection without asking you.
The three bases (free to copy)
- Trustpilot Review Tracker: https://www.airtable.com/universe/expxMMT5P9xY6PYwi/trustpilot-review-tracker-apify
- Reddit Mentions Tracker: https://www.airtable.com/universe/expllyGuZGTB4g63D/reddit-mentions-tracker-apify
- Job Search Across Job Boards (uses the Scripting extension, so it needs a paid Airtable plan): https://www.airtable.com/universe/expQSXJThbw2saMYm/job-search-across-job-boards-apify
Curious what others use for sources that have no API. Has anyone found a clean way to do it with automations only?