scrapy

Web Scraping

scrapy/scrapy

64.9K12.0KPythonBSD-3-Clause

A Python web scraping framework that helps you quickly crawl website data, suitable for batch collection and data analysis.

🕷️ Web Scraping

scrapy preview — Preview card generated by GitHub from the repository. Star counts on the card may lag behind the figures above.

Tool Info

Type🐙 Open Source

Stars64.9K

Forks12.0K

LanguagePython

LicenseBSD-3-Clause

Unlock the full analysis — free

You're reading the public preview of scrapy. The AI-written pros & cons, use cases, setup guide and audience fit are free with an account — no credit card. Pro adds AI comparisons (50 per 30 days) and member daily & weekly reports.

Compare scrapy with alternatives

See how it stacks up against other web scraping tools.

More Tools

firecrawl

193.9K

A web scraping and search API that helps AI agents and developers easily obtain web data.

Scrapling

89.0K

An adaptive web scraping framework that automatically extracts data from a single page or an entire website, saving time and effort.

crawl4ai

86.0K

Automatically crawl web content and organize it into a format AI can directly use, convenient for data analysis and knowledge bases.

MediaCrawler

67.6K

One-click crawling of notes, videos, and comments from platforms like Xiaohongshu, Douyin, and Bilibili to help you quickly collect social media data.

legado

47.2K

An Android reader that allows custom book sources, aggregating novel resources from multiple platforms, supporting TTS reading and eye-care reading.