MCP serverSafeBrowser automation & web dataSome setup

Scrapling

An adaptive Python web scraping framework that handles single requests to full-scale crawls, featuring anti-bot bypass, proxy rotation, and an MCP server for agent integration.

🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

89k stars9.2kLicense: BSD-3-ClauseUpdated 3 days ago0 opens

Preview

▍

What you can do with it

Automate the extraction of structured data from websites for market research or data analysis.
Build robust web crawlers that can bypass anti-bot measures and adapt to website design changes.
Integrate advanced web scraping capabilities into an AI agent's workflow via its MCP server.

Try asking your agent

“Using Scrapling, extract all blog post titles and their URLs from 'https://www.example.com/blog/'.”
“Crawl 'https://www.fakestore.com/products' to get the name, price, and image URL for the first 10 products, handling potential anti-bot measures.”
“Periodically monitor 'https://news.example.com' for new articles containing the keyword 'AI' and report their headlines.”

Safety check

Safe

No risky patterns found in the files we scanned.

Automated scan of the repo files. Always review code you run.

From the README

Effortless Web Scraping for the Modern Web

<strong>Selection methods</strong>
&middot;
<strong>Fetchers</strong>
&middot;
<strong>Spiders</strong>
&middot;
<strong>Proxy Rotation</strong>
&middot;
<strong>CLI</strong>
&middot;
<strong>MCP</strong>

Scrapling is a Python Web Scraping framework that brings fast HTTP requests, stealth browsers, powerful parsing, and concurrent crawling into one library.

Scrape static pages, render JavaScript, interact with websites, and extract data from the API calls behind them. Build crawlers with reusable sessions, proxy rotation, automatic retries, pause/resume, and built-in data exports.

Use it from Python, the terminal, or your AI agent through its MCP server and Agent Skill. From extracting a few fields to collecting a whole website as clean Markdown, Scrapling covers the full scraping workflow.

What you can do with Scrapling

Fetch & interact: Fast HTTP requests, stealth browsers, JavaScript rendering, Cloudflare Turnstile/Interstitial handling, persistent sessions, and background API capture.
Parse & extract: CSS, XPath, text and regex search, adaptive element tracking, similar-element matching, and clean Markdown output.
Crawl & export: Concurrent spiders, proxy rotation, retries, AutoThrottle, pause/resume, streaming, ready-made templates, and JSON/JSONL/CSV/XML exports.
Scrape with AI or from your terminal: An MCP server for AI agents to browse and extract data, an Agent Skill, CLI commands, and an interactive shell.

Start with a single page:

from scrapling.fetchers import Fetcher

page = Fetcher.get('https://quotes.toscrape.com/')
quotes = page.css('.quote .text::text').getall()
print(quotes)

Or scale up with a spider:

from scrapling.spiders import Spider, Response

class QuotesSpider(Spider):
    name = "quotes"
    start_urls = ["https://quotes.toscrape.com/"]

    async def parse(self, response: Response):
        for quote in response.css('.quote'):
            yield {
                "text": quote.css('.text::text').get(),
                "author": quote.css('.author::text').get(),
            }

result = QuotesSpider().start()
print(len(result.items))
result.items.to_json("quotes.json")

This prints 10 and saves the quotes and authors to quotes.json.

Platinum Sponsors

NodeMaven - The most efficient proxy provider for Web Scraping and Automation with the Highest Quality IP on the market. Use code SCRAPLING35 for 35% discount.




  
    
  

 Scrapling handles Cloudflare Turnstile. For enterprise-grade protection, 
    <b>Hyper Solutions</b>
   provides API endpoints that generate valid antibot tokens for <b>Akamai</b>, <b>DataDome</b>, <b>Kasada</b>, and <b>Incapsula</b>. Simple API calls, no browser automation required. 



  
    
  


  
    <b>Evomi</b>
  : residential proxies from $0.49/GB. Scraping browser with fully spoofed Chromium, residential IPs, auto CAPTCHA solving, and anti-bot bypass. </br>
  <b>Scraper API for hassle-free results. MCP and N8N integrations are available.</b>




  
    
  


Read a full review of Scrapling on The Web Scraping Club (Nov 2025), the #1 newsletter dedicated to Web Scraping.




  
    
  


Swiftproxy provides scalable residential proxies with 80M+ IPs across 195+ countries, delivering fast, reliable connections, automatic rotation, and strong anti-block performance. Free trial available.




  
    
  


Byteful supplies ethical residential, mobile, ISP, and datacenter proxies, plus scraping and AI automation tools. Use code SCRAPLING10: 10% off residential proxies.




  
    
  


ProxyLane: Reduce blocks and CAPTCHAs by 90%. 28M Premium Residential IPs for Heavy Web Scraping. 195 countries. City/ISP/ZIP targeting. Sticky/rotating sessions. Non-expiring traffic from $2/GB. Code <strong>SCRAPEANDTAKE</strong>: <em>25% off</em>.




  
    
  


LundProxy: Stop paying per GB. Unlimited residential traffic, 100M+ IPs, 190+ countries. Pay by bandwidth. Request a free 50 Mbps residential proxies trial.




  
    
  


$1/GB at DataImpulse. Ethical first-party proxies. 90M+ IPs, 195+ countries. Residential, datacenter, mobile. Pay-as-you-go = no subscription. Traffic never expires.

Do you want to show your ad here? Click here