Purus suspendisse a ornare non erat pellentesque arcu mi arcu eget tortor eu praesent curabitur porttitor ultrices sit sit amet purus urna enim eget. Habitant massa lectus tristique dictum lacus in bibendum. Velit ut viverra feugiat dui eu nisl sit massa viverra sed vitae nec sed. Nunc ornare consequat massa sagittis pellentesque tincidunt vel lacus integer risu.
Mauris posuere arcu lectus congue. Sed eget semper mollis felis ante. Congue risus vulputate nunc porttitor dignissim cursus viverra quis. Condimentum nisl ut sed diam lacus sed. Cursus hac massa amet cursus diam. Consequat sodales non nulla ac id bibendum eu justo condimentum. Arcu elementum non suscipit amet vitae. Consectetur penatibus diam enim eget arcu et ut a congue arcu.
Vitae vitae sollicitudin diam sed. Aliquam tellus libero a velit quam ut suscipit. Vitae adipiscing amet faucibus nec in ut. Tortor nulla aliquam commodo sit ultricies a nunc ultrices consectetur. Nibh magna arcu blandit quisque. In lorem sit turpis interdum facilisi.
Vitae vitae sollicitudin diam sed. Aliquam tellus libero a velit quam ut suscipit. Vitae adipiscing amet faucibus nec in ut. Tortor nulla aliquam commodo sit ultricies a nunc ultrices consectetur. Nibh magna arcu blandit quisque. In lorem sit turpis interdum facilisi.
“Nisi consectetur velit bibendum a convallis arcu morbi lectus aecenas ultrices massa vel ut ultricies lectus elit arcu non id mattis libero amet mattis congue ipsum nibh odio in lacinia non”
Nunc ut facilisi volutpat neque est diam id sem erat aliquam elementum dolor tortor commodo et massa dictumst egestas tempor duis eget odio eu egestas nec amet suscipit posuere fames ded tortor ac ut fermentum odio ut amet urna posuere ligula volutpat cursus enim libero libero pretium faucibus nunc arcu mauris sed scelerisque cursus felis arcu sed aenean pharetra vitae suspendisse ac.
Imagine you want to keep track of the templates available on the Webflow website. You’re interested in monitoring when new templates are added and when existing templates are removed from the homepage. Manually checking the website and updating a spreadsheet would be a tedious and time-consuming task. That’s where our Next.js web scraper and Airtable integration come into play.
We’ll build a Next.js application that scrapes the Webflow templates page, extracts relevant information, and stores it in an Airtable base. The application will consist of two main components:
webflow-scraper.js
file defines an API route that fetches the HTML content of the Webflow templates page using the axios
library.cheerio
library to parse the HTML and extract the desired data, such as the template name, URL, and publish date.2. Compare and Update API Route:
compare-and-update.js
file defines another API route that compares the scraped data with the existing records in the Airtable base.
To integrate with Airtable, we use the airtable
package and configure it with our Airtable API key and base ID. We define the necessary field IDs and table name in the lib/airtable.js
file, making it easy to reference them throughout the application.
To automate the scraping and comparison process, we utilize the Vercel Cron Jobs feature. By adding a vercel.json
file with the desired cron schedule, we can configure the compare-and-update
API route to run at specific intervals (e.g., every 12 hours). This ensures that our Airtable base stays up to date with the latest changes on the Webflow templates page.
At Create Something, our mission is to automate every integration through innovative AI and API solutions. Explore our popular papers to gain insights into the latest advancements and practical applications in automation and integration.