📊 SERP competitor research with Scrape.do API & Google Sheets
⚡ 294 views · 📊 Market Research & Insights
💡 Pro Tip — HTTP Request scraping tends to break when sites update their markup. If you’re scraping a major platform, check if ScraperNode covers it — it has maintained scrapers for LinkedIn, Instagram, TikTok, YouTube, and 20+ other platforms that return structured data.
Description
🔍 Extract Competitor SERP Rankings from Google Search to Sheets with Scrape.do
This template requires a self-hosted n8n instance to run.
A complete n8n automation that extracts competitor data from Google search results for specific keywords and target countries using Scrape.do SERP API, and saves structured results into Google Sheets for SEO, competitive analysis, and market research.
📋 Overview
This workflow provides a lightweight competitor analysis solution that identifies ranking websites for chosen keywords across different countries. Ideal for SEO specialists, content strategists, and digital marketers who need structured SERP insights without manual effort.
Who is this for?
- SEO professionals tracking keyword competitors
- Digital marketers conducting market analysis
- Content strategists planning based on SERP insights
- Business analysts researching competitor positioning
- Agencies automating SEO reporting
What problem does this workflow solve?
- Eliminates manual SERP scraping
- Processes multiple keywords across countries
- Extracts structured data (position, title, URL, description)
- Automates saving results into Google Sheets
- Ensures repeatable & consistent methodology
⚙️ What this workflow does
- Manual Trigger → Starts the workflow manually
- Get Keywords from Sheet → Reads keywords + target countries from a Google Sheet
- URL Encode Keywords → Converts keywords into URL-safe format
- Process Keywords in Batches → Handles multiple keywords sequentially to avoid rate limits
- Fetch Google Search Results → Calls Scrape.do SERP API to retrieve raw HTML of Google SERPs
- Extract Competitor Data from HTML → Parses HTML into structured competitor data (top 10 results)
- Append Results to Sheet → Writes structured SERP results into a Google Sheet
📊 Output Data Points
| Field | Description | Example |
|---|---|---|
| Keyword | Original search term | digital marketing services |
| Target Country | 2-letter ISO code of target region | US |
| position | Ranking position in search results | 1 |
| websiteTitle | Page title from SERP result | Digital Marketing Software & Tools |
| websiteUrl | Extracted website URL | https://www.hubspot.com/marketing |
| websiteDescription | Snippet/description from search results | Grow your business with HubSpot’s tools… |
⚙️ Setup
Prerequisites
- n8n instance (self-hosted)
- Google account with Sheets access
- Scrape.do account with SERP API token
Google Sheet Structure
This workflow uses one Google Sheet with two tabs:
Input Tab: “Keywords”
| Column | Type | Description | Example |
|---|---|---|---|
| Keyword | Text | Search query | digital marketing |
| Target Country | Text | 2-letter ISO code | US |
Output Tab: “Results”
| Column | Type | Description | Example |
|---|---|---|---|
| Keyword | Text | Original search term | digital marketing |
| position | Number | SERP ranking | 1 |
| websiteTitle | Text | Title of the page | Digital Marketing Software & Tools |
| websiteUrl | URL | Website/page URL | https://www.hubspot.com/marketing |
| websiteDescription | Text | Snippet text | Grow your business with HubSpot’s tools |
🛠 Step-by-Step Setup
- Import Workflow: Copy the JSON → n8n → Workflows → + Add → Import from JSON
- Configure Scrape.do API:
- Endpoint:
https://api.scrape.do/ - Parameter:
token=YOUR_SCRAPEDO_TOKEN - Add
render=truefor full HTML rendering
- Endpoint:
- Configure Google Sheets:
- Create a sheet with two tabs: Keywords (input), Results (output)
- Set up Google Sheets OAuth2 credentials in n8n
- Replace placeholders:
YOUR_GOOGLE_SHEET_IDandYOUR_GOOGLE_SHEETS_CREDENTIAL_ID
- Run & Test:
- Add test data in
Keywordstab - Execute workflow → Check results in
Resultstab
- Add test data in
🧰 How to Customize
- Add more fields: Extend HTML parsing logic in the “Extract Competitor Data” node to capture extra data (e.g., domain, sitelinks).
- Filtering: Exclude domains or results with custom rules.
- Batch Size: Adjust “Process Keywords in Batches” for speed vs. rate-limits.
- Rate Limiting: Insert a Wait node (e.g., 10–30 seconds) if API rate limits apply.
- Multi-Sheet Output: Save per-country or per-keyword results into separate tabs.
📊 Use Cases
- SEO Competitor Analysis: Identify top-ranking sites for target keywords
- Market Research: See how SERPs differ by region
- Content Strategy: Analyze titles & descriptions of competitor pages
- Agency Reporting: Automate competitor SERP snapshots for clients
📈 Performance & Limits
- Single Keyword: ~10–20 seconds (depends on Scrape.do response)
- Batch of 10: 3–5 minutes typical
- Large Sets (50+): 20–40 minutes depending on API credits & batching
- API Calls: 1 Scrape.do request per keyword
- Reliability: 95%+ extraction success, 98%+ data accuracy
🧩 Troubleshooting
- API error → Check
YOUR_SCRAPEDO_TOKENand API credits - No keywords loaded → Verify Google Sheet ID & tab name =
Keywords - Permission denied → Re-authenticate Google Sheets OAuth2 in n8n
- Empty results → Check parsing logic and verify search term validity
- Workflow stops early → Ensure batching loop (
SplitInBatches) is properly connected
🤝 Support & Community
- n8n Forum: https://community.n8n.io
- n8n Docs: https://docs.n8n.io
- Scrape.do Dashboard: https://dashboard.scrape.do
🎯 Final Notes
This workflow provides a repeatable foundation for extracting competitor SERP rankings with Scrape.do and saving them to Google Sheets.
You can extend it with filtering, richer parsing, or integration with reporting dashboards to create a fully automated SEO intelligence pipeline.
🔗 Nodes Used
Google Sheets, HTTP Request
📥 Import
Download workflow.json and import into n8n:
Workflow menu → Import from File