ScrapeGraphAI
NewScrapeGraphAI is an LLM-powered scraping API with scrape, extract, search, crawl and monitor endpoints that return markdown or schema-shaped JSON, plus an MIT-licensed Python library you can run yourself.
ScrapeGraphAI
Explore features, practical uses and pricing below.
ScrapeGraphAI is a scraping API that uses language models to turn web pages into structured data. Instead of writing CSS selectors, you send a URL and a prompt such as "list every product with its price and rating," optionally with a JSON schema, and get typed JSON back. There's also an MIT-licensed Python library of the same name you can run with your own model provider.
The five ScrapeGraphAI endpoints
Version 2 of the API is organized around five operations, documented in the ScrapeGraphAI docs:
- Scrape fetches a page as markdown, HTML, a screenshot, links, images, a summary or brand details.
- Extract sends a URL, raw HTML or markdown plus a prompt to an LLM and returns JSON, shaped by a schema if you supply one.
- Search runs a web search and can extract data from the results in the same request.
- Crawl works through multiple pages of a site asynchronously, with start, stop and resume controls.
- Monitor re-fetches a page on a five-field cron schedule, records the diff against the previous run and can post each run to a webhook.
Fetch settings let you pick a render mode (auto, fast or JavaScript) and turn on a stealth option for sites that block ordinary requests. Saved schemas and a request history round out the API.
Calling it from code and agent frameworks
Official SDKs cover Python and JavaScript/TypeScript, with a mocking mode for credit-free tests. In Python, you can pass a Pydantic model's JSON schema straight into an extract call. The docs cover MCP setup for Cursor, Claude Desktop, Claude Code and Codex, tool wrappers for LangChain, LangGraph, LlamaIndex, CrewAI, Agno and the Vercel AI SDK, and no-code nodes for n8n, Make and Zapier.
Pulling open roles from careers pages
Suppose a recruiting-analytics team wants titles, locations and salary ranges from 40 company careers pages. A developer writes a Pydantic model with those three fields, calls Extract for each URL with the prompt "every open role on this page," and uses JavaScript render mode because most careers pages load listings dynamically. Pages that come back empty are retried with stealth enabled. For the ten companies the team cares about most, a Monitor runs on a daily cron with a webhook, so a new posting triggers an alert instead of a manual check.
Trade-offs to weigh before building on it
LLM extraction copes with layout changes better than fixed selectors, but it can misread a page or fill a field with something plausible, so validate against your schema and spot-check. Credit use adds up across options. The pricing page lists Extract at 5 credits per call, stealth at 5 more, Search at 2 credits per result (5 with a prompt), and Monitor runs billed at the scrape cost plus 5 credits when a change is detected. The open-source library avoids API credits, but you supply and pay for the model, install Playwright and maintain the pipeline. Site terms and data-protection rules still apply.
Who ScrapeGraphAI is built for
It suits developers building data pipelines, retrieval systems or agents who would rather describe the data than maintain selectors. It's a weaker fit for non-technical users who want a point-and-click scraper, which is the approach Browse AI takes. Teams that mainly need clean markdown from whole sites might also compare Firecrawl.
Credits and plans
The pricing page lists a Free plan with 500 one-time credits and 10 requests per minute, then Starter at $20 a month for 10,000 credits, Growth at $100 for 100,000 credits and Pro at $500 for 750,000 credits, each with a higher rate limit, plus custom Enterprise terms. PDF processing costs 1 credit per page, with a 25-page default cap.
ScrapeGraphAI questions
Is the open-source library the same thing as the API?
The docs call the GitHub project the open-source core. The library runs pipelines such as SmartScraperGraph and SearchGraph locally with OpenAI, Groq, Azure, Gemini or local Ollama models. The docs don't spell out feature parity with the hosted endpoints, so check before assuming a library pipeline maps to an API call.
Do I need a JSON schema?
No. Extract requires a prompt and treats the schema as optional, though a schema makes the output shape predictable.