Scraping & Extraction

Query X and Twitter data for tweet search, follower lists and engagement metrics

Fetch a URL as markdown, text or HTML through 9Router's web fetch providers

Ingest a whole blog, newsletter or RSS archive into source pages with dedup

Turn large audio, email and document archives into knowledge pages at scale

Research sources, extract structured records and maintain deduplicated tracker pages

Ingest videos, audio, books and PDFs into the brain with entity extraction

Extract text from large PDF collections with GPU-accelerated document processing

Build validated scrapers, document extractors and parsing pipelines with Firecrawl

Convert X (Twitter) tweets and articles to Markdown with front matter

Fetch any web page as Markdown using site adapters for X, YouTube and Hacker News

Download videos and extract content from Bilibili, YouTube, TikTok and other platforms

Scrape Instagram, LinkedIn, TikTok, Google Maps and Amazon via Apify actors

Scrape web pages with escalating fallbacks, including Bright Data for CAPTCHAs

Extract clean readable Markdown from web pages before ingesting them into your vault

Parse PDFs and images into structured Markdown and extract fields with LandingAI ADE

Build batch and classify-then-extract document pipelines for RAG with LandingAI ADE

Generate custom recognition rules that fix how torrent and file names are parsed

Pull video stats and transcripts from Douyin, Xiaohongshu and WeChat Channels links

Fetch Reddit posts and comments through its JSON API when pages are blocked

Download WeChat official account articles and comments as HTML, PDF, Word or Markdown

Extract text, tables and metadata from 107 document formats with Xberg

Extract text from many files at once with shared config and capped parallelism

Extract tables from PDFs, spreadsheets and images as markdown or JSON

Run OCR on scanned PDFs and photographed pages with language packs and tuning

Pick the output format for an LLM, parser or archive from extracted documents

Convert JS-rendered or login-only web pages into clean Markdown with metadata

Search and collect Douyin videos, users, comments and feeds with media downloads

Search and collect Kuaishou posts, accounts, comments and media files

Pull public Kuaishou live-room info, replays and bounded event streams

Find the RSS or Atom feed of any website and parse its articles

Search and collect TikTok videos, accounts, comments and media files

Search and collect Weibo posts, accounts, comments and media files

Fetch full WeChat Official Account articles, post lists and album collections

Search and collect X posts, accounts, replies and media files

Search and collect Xiaohongshu notes, users, comments and media in bulk

Extract text from scanned PDFs and images offline with local OCR engines

Fetch websites with curl using browser headers and validate the file types

Scrape Google Maps for business leads, reviews and contact details with Docker

Search, read, crawl and extract live web pages and platform data

Extract 1688.com wholesale product titles, pricing, variants and sellers

Fetch full Airbnb listing details, amenities and reviews by listing ID

Extract Airbnb search results with prices, ratings, photos and pagination

Look up Amazon product details, price and ratings from an ASIN

Find top-selling Amazon products by keyword and extract their data

Scrape Amazon bestseller category pages into ranked product lists

Monitor Amazon buy box prices and other sellers by ASIN

Extract Amazon product listings, prices and specs for search and tracking

Extract full product data from any Amazon product page URL, in any region

Extract Amazon search results for keywords, brands and price tracking