Footer

Your go-to destination for discovering trending open-source projects and uncovering the insights that matter

Legal

  • Terms and conditions
  • Privacy policy

More

  • Pricing
  • Advertise
  • onedump
  • gosnakego
  • Contact us

Social

  • X (Twitter)
  • GitHub
Trendshift owl
What they build speaks louder than words
© 2026 Trendshift, created by Julian Li with ❤️
SignalAdvertise
Trendshift
SignalAdvertise
Sign in
Trendshift owl

Discover trends that matter

SignalAdvertise
DailyWeeklyMonthlyYearly
Live mentionsTopics
RepositoriesDevelopers
InsightsStats
Bifrost
Bifrost

scrapy/scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

#Web scraping
Visit GitHub
Python
65.2k
12.2k
610 contributors
BSD 3-Clause "New" or "Revised" License
website
AchievementsView all →
FeaturedAdvertise to 180K+ monthly visitors
Bifrost

Fastest Enterprise AI Gateway for Routing, Governing, and Securing all your AI Traffic

Momentum

How fast scrapy/scrapy is gaining stars, and where it lands on Trendshift and GitHub Trending

Activity and trending

Stars, forks and merged pull requests each day or month, lined up with the repository's rank on Trendshift Trending

Sep 5 – Nov 4, 2026

Stars +1,025 in 61 days

02040
34

Forks +263 in 61 days

0510
10

Merged PRs 295 in 61 days

02040
34

Trendshift Trendingwhere the repository ranked each day it made the top 25

Not in the Trendshift top 25 in this period

Sep 5Sep 13Sep 21Sep 29Oct 7Oct 15Oct 23Oct 31
The left and right arrow keys step from day to day; Home and End go to the first and the last

Stars, forks, merged pull requests and Trendshift Trending rank, by day
DayStars gainedForks gainedMerged PRs
Sat, Sep 5, 2026+7+10
Sun, Sep 6, 2026+4+53
Mon, Sep 7, 2026+25+27
Tue, Sep 8, 2026+14+14
Wed, Sep 9, 2026+17+42
Thu, Sep 10, 2026+14+38
Fri, Sep 11, 2026+12+60
Sat, Sep 12, 2026+9+30
Sun, Sep 13, 2026+20+100
Mon, Sep 14, 2026+27+72
Tue, Sep 15, 2026+13+14
Wed, Sep 16, 2026+10+314
Thu, Sep 17, 2026+17+113
Fri, Sep 18, 2026+14+61
Sat, Sep 19, 2026+8+30
Sun, Sep 20, 2026+16+13
Mon, Sep 21, 2026+11+43
Tue, Sep 22, 2026+9+610
Wed, Sep 23, 2026+14+31
Thu, Sep 24, 2026+9+26
Fri, Sep 25, 2026+15+43
Sat, Sep 26, 2026+12+10
Sun, Sep 27, 2026+8+223
Mon, Sep 28, 2026+10+53
Tue, Sep 29, 2026+18+40
Wed, Sep 30, 2026+5+46
Thu, Oct 1, 2026+5+56
Fri, Oct 2, 2026+12+37
Sat, Oct 3, 2026+12+47
Sun, Oct 4, 2026+15+222
Mon, Oct 5, 2026+13+11
Tue, Oct 6, 2026+17+20
Wed, Oct 7, 2026+28+52
Thu, Oct 8, 2026+34+66
Fri, Oct 9, 2026+19+56
Sat, Oct 10, 2026+22+60
Sun, Oct 11, 2026+22+30
Mon, Oct 12, 2026+30+20
Tue, Oct 13, 2026+27+210
Wed, Oct 14, 2026+19+411
Thu, Oct 15, 2026+24+312
Fri, Oct 16, 2026+22+634
Sat, Oct 17, 2026+13+45
Sun, Oct 18, 2026+10+51
Mon, Oct 19, 2026+19+66
Tue, Oct 20, 2026+21+914
Wed, Oct 21, 2026+17+710
Thu, Oct 22, 2026+22+64
Fri, Oct 23, 2026+21+50
Sat, Oct 24, 2026+19+90
Sun, Oct 25, 2026+21+30
Mon, Oct 26, 2026+34+97
Tue, Oct 27, 2026+21+32
Wed, Oct 28, 2026+27+92
Thu, Oct 29, 2026+0+60
Fri, Oct 30, 2026+21+91
Sat, Oct 31, 2026+18+50
Sun, Nov 1, 2026+22+51
Mon, Nov 2, 2026+26+101
Tue, Nov 3, 2026+19+21
Wed, Nov 4, 2026+15+00
Total+1,025+263295
  • 1#1 on the list
  • 7Rank that day
  • Not trending
2

    • GitHub trending history

    Where the repository ranked on GitHub Trending, one dot for each day it was on a list

    Jun 4 – Jun 7, 2025Best #3

    #1#5#10#15#20#25
    Top 10
    Nov 2023MayNovMayNovMay
    The left and right arrow keys step from day to day; Home and End go to the first and the last

    Mentions and achievements

    Where people are talking about scrapy/scrapy, and every badge it has earned
    1. Nov 41 post

      • Web scraping is getting harder. JS rendering. DOM changes. Bot checks. Lazy loading. Huge pages with tiny useful content. 8 OSS repos worth knowing: Firecrawl — 198K★ Scrapling — 95K+★ Crawl4AI —…

        Read full on X, posted by @opinion_bits

        @opinion_bits · x.com

    2. 13 days earlier
    3. Oct 221 post

      • 公开网页数据还在一页页复制？🕷️ Scrapy 用 Python 写爬虫，自动跟进链接并提取结构化数据。采集前记得遵守网站规则。📄 #Python #浇朋友 #蓝朋友 #互关 #fo

        Read full on X, posted by @T1doo_AI

        @T1doo_AI · x.com

    4. 7 days earlier
    5. Oct 151 post

      • 😤 批量抓公开数据从零写脚本太折腾。Scrapy 是成熟爬虫框架，调度、去重、导出全有，只写解析逻辑。六万四千星。

        Read full on X, posted by @laogeai

        @laogeai · x.com

    6. Oct 91 post

      • 想用 Hermes 和 Openclaw 白薅全网数据、又不想被反爬逮住的，收拢这 10 个仓库就齐了！ 1️⃣ Firecrawl：URL 往里一撂，它自个儿就把整站刨穿，吐出来直接是 AI 张口就能吞的干净数据，JS 渲染的页面也照扛，15.2 万★，杀进 GitHub Top 100。 🔗 2️⃣ Crawl4AI：把网站捏成 LLM 扫一眼就能读的文本，API key…

        Read full on X, posted by @0xGky

        @0xGky · x.com

    7. Oct 81 post

      • 5. 专业级框架，该有的全有：代理轮换、自动重试、浏览器指纹伪装、任务队列。 爬虫公司一个月收你几千块的东西。 这里免费。

        Read full on X, posted by @Techbruneth

        @Techbruneth · x.com

    8. Oct 71 post

      • 还是不敢撒手扒数据，这10个开源工具能把封锁成本给你压下去。 1️⃣ Firecrawl：URL丢进去，整站结构化输出，AI直接消费，JS渲染也不怕，14万★的GitHub Top 100项目。 🔗 2️⃣ Crawl4AI：把网页转成LLM爱吃的纯文本格式，完全免费不要key，16美元月费逼出来的爆款，7万★。 🔗 3️⃣…

        Read full on X, posted by @Manorgw

        @Manorgw · x.com

    9. Oct 31 post

      • 4. 专业级爬虫框架。代理轮换、自动重试、浏览器指纹伪装、队列管理一应俱全。这是爬虫公司靠啥赚几千块的全套工具。 现在免费给你。

        Read full on X, posted by @Lucy_love_AI

        @Lucy_love_AI · x.com

    10. Oct 21 post

      • 反爬搞得你焦头烂额，IP 一被 ban 就彻底歇菜？这 10 个开源工具，直接把网站的反爬成本拉到爆表。 1️⃣ Firecrawl：丢一个URL进去，整站结构化数据直接吐出来，AI应用拿来就能用，JS渲染的页面也照扒不误，GitHub Top 100里躺了14万星。 🔗 2️⃣…

        Read full on X, posted by @kfhshl

        @kfhshl · x.com

    11. 7 days earlier
    12. Sep 251 post

      • 5. 专业框架，什么都带：代理轮换、自动重试、浏览器指纹伪装、任务队列。 爬虫公司每月收你几千块的东西。 这里免费。

        Read full on X, posted by @CoderDaMing

        @CoderDaMing · x.com

    13. Sep 201 post

      • 5. 专业框架，什么都带：代理轮换、自动重试、浏览器指纹伪装、任务队列。 爬虫公司每月收你几千块的东西。 这里免费。

        Read full on X, posted by @AIExplorerTim

        @AIExplorerTim · x.com

    You might also like

    Repositories close to scrapy/scrapy
    FeaturedAdvertise to 180K+ monthly visitors
    Bifrost

    Fastest Enterprise AI Gateway for Routing, Governing, and Securing all your AI Traffic

    D4Vinci/Scrapling

    Python

    🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

    96.3k10.4k

    scrapinghub/scrapy-poet

    Python

    Page Object pattern for Scrapy

    12929

    howie6879/talospider

    Python

    talospider - A simple,lightweight scraping micro-framework

    574

    tech-engine/goscrapy

    Go

    GoScrapy: High perfomance webscraping framework in Go, inspired by Python's Scrapy.

    37441

    omkarcloud/botasaurus

    Python

    The All in One Framework to Build Undefeatable Scrapers

    5.8k517

    BitingSnakes/silkworm

    Python

    Async web scraping framework on top of Rust. Works with Free-threaded Python (`PYTHON_GIL=0`).

    765

    gocolly/colly

    Go

    Elegant Scraper and Crawler Framework for Golang

    25.6k1.9k

    alirezamika/autoscraper

    Python

    A Smart, Automatic, Fast and Lightweight Web Scraper for Python

    8.1k848

    apache/stormcrawler

    Java

    A scalable, mature and versatile web crawler based on Apache Storm

    1k302