Tagged web-scraping

Scrape Google search results and track keyword rank with a scrape.do credit governor

Find and install a CLI, API wrapper or scraper for a task from a library

Export local business leads from Google Maps to CSV with phones, emails and websites

Scrape Instagram, TikTok, YouTube and Google data with Apify actors into JSON or CSV

Install and run the MediaCrawler client to collect Douyin, Bilibili and Weibo data

Collect Bilibili videos, comments and creator evidence for topic and audience research

Collect Douyin keyword search, video detail and comments for trend research

Collect Kuaishou videos, comments and creator profiles for trend research

Collect Baidu Tieba threads and replies to study community pain points

Collect Weibo posts, comments and creator IDs for discourse and trend research

Collect Zhihu answers, articles and comments for expert discourse research

Find SEO opportunities from keyword expansion, SERP scraping and competitor pages

Collect historical URLs and hidden endpoints from Wayback, Common Crawl and OTX

Crawl sites and JavaScript to discover endpoints and forms with katana

Combine DNS brute force, OSINT and crawling to find more subdomains

Scrape BOSS Zhipin job listings and salaries via Chrome CDP into JSON or CSV

Generate and validate web scraper scripts using verified page selectors

Scrape listings, tables and feeds into data with a reusable Playwright script

Export Feishu or Lark docs, wikis and minutes transcripts to faithful Markdown

Archive public WPS and KDocs pages, including mind maps, as SVG, PNG and Markdown

Audit any domain for AI crawler access, llms.txt, schema and SSR, scored 0 to 100

Extract web page data from plain-language prompts with ScrapeGraph

Scrape blog posts from RSS feeds and track competitor blogs by keyword

Automate browser sessions, scrape pages and run browser tasks with Notte

Scrape rival ads from Meta, TikTok, Google and LinkedIn and write a teardown

Pull speaker names, titles and bios from conference sites for outreach lists

Scrape a competitor's Google Ads by domain for creatives and formats

Search Hacker News stories and comments through the free Algolia API

Check AI crawler access, indexing and the pages behind lost search questions

Pull Instagram profiles, posts and reels into structured data

Scrape LinkedIn and Indeed job postings with titles, salaries and apply links

Scrape LinkedIn job postings into a pipeline with the JobSpy Python library

Scrape recent LinkedIn posts from chosen profiles to track founder activity

Pull LinkedIn profiles, company pages and posts into structured data

Scrape competitor ads from Meta's Ad Library with spend, reach and creatives

Scrape Product Hunt trending launches with Apify to track competitors

Search and scrape Reddit posts by keyword, subreddit and time range with Apify

Scrape G2, Capterra and Trustpilot reviews to track competitors and sentiment

Pull G2, Capterra and Trustpilot product reviews to watch competitor sentiment

Crawl a sitemap into a full content inventory grouped by topic and type

Scrape social profiles, posts and ads through the ScrapeCreators API

Scrape websites into a schema you define, returning clean structured records

Search the Wayback Machine for archived pages and recover lost customer lists

Scrape pages, extract structured data and automate browsers from any URL

Crawl websites at scale and extract answers from pages with the Olostep API

Search, crawl and extract web content with Tavily and run deep research

Scrape X timelines and home feeds by user or keyword into RSS, JSON or Markdown

Scrape Xiaohongshu search results into lists and full note text as JSON or Markdown

Diagnose why a page ranks poorly across crawl, index, on-page and content signals