Footer

Your go-to destination for discovering trending open-source projects and uncovering the insights that matter

Legal

  • Terms and conditions
  • Privacy policy

More

  • Pricing
  • Advertise
  • onedump
  • gosnakego
  • Contact us

Social

  • X (Twitter)
  • GitHub
Trendshift owl
What they build speaks louder than words
© 2026 Trendshift, created by Julian Li with ❤️
SignalAdvertise
Trendshift
SignalAdvertise
Sign in
Trendshift owl

Discover trends that matter

SignalAdvertise
DailyWeeklyMonthlyYearly
Live mentionsTopics
RepositoriesDevelopers
InsightsStats

Omnigraph
Omnigraph
It's a plan
It's a plan
Busbar
Busbar
Bifrost
Bifrost
OfinTech Desk
OfinTech Desk

apify/crawlee

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

#Web scraping
Visit GitHub
TypeScript
26.5k
1.9k
132 contributors
Apache License 2.0
website
AchievementsView all →
FeaturedAdvertise to 180K+ monthly visitors
Omnigraph

Versioned graph database on object storage. Git-style workflows and multimodal retrieval

It's a plan

Open-source Linear alternative

An issue tracker where AI agents take the work you assign them

Busbar

Control what AI can do before it acts

Bifrost

Fastest Enterprise AI Gateway for Routing, Governing, and Securing all your AI Traffic

OfinTech Desk

An open-source AI workspace for finance and accounting teams, in which every figure is traced to its source

Momentum

How fast apify/crawlee is gaining stars, and where it lands on Trendshift and GitHub Trending

Activity and trending

Stars, forks and merged pull requests each day or month, lined up with the repository's rank on Trendshift Trending

Sep 21 – Nov 20, 2026

Stars +651 in 61 days

02040
35

Forks +124 in 61 days

0510
7

Merged PRs 185 in 61 days

01020
17

Trendshift Trendingwhere the repository ranked each day it made the top 25

Not in the Trendshift top 25 in this period

Sep 21Sep 29Oct 7Oct 15Oct 23Oct 31Nov 8Nov 16
The left and right arrow keys step from day to day; Home and End go to the first and the last

Stars, forks, merged pull requests and Trendshift Trending rank, by day
DayStars gainedForks gainedMerged PRs
Mon, Sep 21, 2026+10+31
Tue, Sep 22, 2026+5+02
Wed, Sep 23, 2026+12+14
Thu, Sep 24, 2026+5+16
Fri, Sep 25, 2026+13+13
Sat, Sep 26, 2026+10+01
Sun, Sep 27, 2026+12+10
Mon, Sep 28, 2026+11+00
Tue, Sep 29, 2026+9+34
Wed, Sep 30, 2026+16+15
Thu, Oct 1, 2026+10+04
Fri, Oct 2, 2026+4+12
Sat, Oct 3, 2026+10+20
Sun, Oct 4, 2026+9+20
Mon, Oct 5, 2026+5+317
Tue, Oct 6, 2026+17+28
Wed, Oct 7, 2026+31+20
Thu, Oct 8, 2026+35+71
Fri, Oct 9, 2026+11+27
Sat, Oct 10, 2026+19+20
Sun, Oct 11, 2026+6+31
Mon, Oct 12, 2026+14+42
Tue, Oct 13, 2026+14+20
Wed, Oct 14, 2026+14+22
Thu, Oct 15, 2026+7+19
Fri, Oct 16, 2026+9+20
Sat, Oct 17, 2026+7+00
Sun, Oct 18, 2026+9+20
Mon, Oct 19, 2026+12+26
Tue, Oct 20, 2026+3+21
Wed, Oct 21, 2026+13+51
Thu, Oct 22, 2026+10+22
Fri, Oct 23, 2026+13+02
Sat, Oct 24, 2026+13+60
Sun, Oct 25, 2026+14+11
Mon, Oct 26, 2026+5+00
Tue, Oct 27, 2026+7+73
Wed, Oct 28, 2026+6+10
Thu, Oct 29, 2026+14+13
Fri, Oct 30, 2026+7+13
Sat, Oct 31, 2026+1+21
Sun, Nov 1, 2026+7+10
Mon, Nov 2, 2026+9+14
Tue, Nov 3, 2026+8+211
Wed, Nov 4, 2026+9+513
Thu, Nov 5, 2026+15+18
Fri, Nov 6, 2026+8+28
Sat, Nov 7, 2026+7+10
Sun, Nov 8, 2026+12+00
Mon, Nov 9, 2026+17+38
Tue, Nov 10, 2026+16+42
Wed, Nov 11, 2026+12+34
Thu, Nov 12, 2026+8+41
Fri, Nov 13, 2026+6+44
Sat, Nov 14, 2026+8+30
Sun, Nov 15, 2026+9+20
Mon, Nov 16, 2026+13+35
Tue, Nov 17, 2026+15+30
Wed, Nov 18, 2026+13+17
Thu, Nov 19, 2026+6+18
Fri, Nov 20, 2026+1+00
Total+651+124185
  • 1#1 on the list
  • 7Rank that day
  • Not trending
2
  • GitHub trending history

    Where the repository ranked on GitHub Trending, one dot for each day it was on a list

    Nov 23, 2023 – Aug 18, 2024Best #16

    #1#5#10#15#20#25
    Top 10
    Dec 2023JunDecJunDecJun
    The left and right arrow keys step from day to day; Home and End go to the first and the last

    Mentions and achievements

    Where people are talking about apify/crawlee, and every badge it has earned
    1. Nov 61 post
      • 9 GITHUB REPOS THAT LET YOU SCRAPE ANY WEBSITE WITHOUT GETTING BLOCKED. Most scrapers get banned in the first 100 requests. These ones do not. Crawl4AI — AI-powered crawler built for LLM pipelines.…

        Read full on X, posted by @cyrilXBT

        @cyrilXBT · x.com

    2. Nov 51 post
      • 🚀 3 powerful GitHub repos for web scraping & data extraction! Automate crawling, extract website data, and build research workflows. 1. Crawlee: 2. Botasaurus: 3. Spider:

        Read full on X, posted by @Shoumya23

        @Shoumya23 · x.com

    3. Nov 42 posts
      • You can build your own AI-powered personal shopping assistant now Here are 10 open-source repos worth checking out: 1. LangChain 2. Browser Use 3. Playwright 4. Firecrawl 5. Crawlee 6. Qdrant 7. Mem0…

        Read full on X, posted by @RoundtableSpace

        @RoundtableSpace · x.com

      • Web scraping is getting harder. JS rendering. DOM changes. Bot checks. Lazy loading. Huge pages with tiny useful content. 8 OSS repos worth knowing: Firecrawl — 198K★ Scrapling — 95K+★ Crawl4AI —…

        Read full on X, posted by @opinion_bits

        @opinion_bits · x.com

    4. Oct 311 post
      • Top 10 Open Agent Web & Browser Tools: 1. Firecrawl - 2. Browser Use - 3. Agent-Reach - 4. Scrapling - 5. Crawl4AI - 6. MediaCrawler - 7. Agent Browser - 8. Crawlee - 9. Stagehand - 10. Skyvern -…

        Read full on X, posted by @AsWill11

        @AsWill11 · x.com

    5. 9 days earlier
    6. Oct 221 post
      • 网页数据要一页页复制?🕷️ Crawlee 用 JavaScript/TypeScript 抓取页面,支持浏览器自动化和批量采集。整理公开资料时很实用。🌐 #WebScraping #浇朋友 #蓝朋友 #互关 #fo

        Read full on X, posted by @T1doo_AI

        @T1doo_AI · x.com

    7. Oct 181 post
      • 写爬虫,总被链接队列和浏览器操作折腾?🕸️ Crawlee 提供爬取队列、浏览器自动化和数据存储。 适合采集获准访问的公开数据。🧰 #WebScraping #Automation #浇朋友 #蓝朋友 #互关 #fo

        Read full on X, posted by @T1doo_AI

        @T1doo_AI · x.com

    8. 9 days earlier
    9. Oct 91 post
      • 想用 Hermes 和 Openclaw 白薅全网数据、又不想被反爬逮住的,收拢这 10 个仓库就齐了! 1️⃣ Firecrawl:URL 往里一撂,它自个儿就把整站刨穿,吐出来直接是 AI 张口就能吞的干净数据,JS 渲染的页面也照扛,15.2 万★,杀进 GitHub Top 100。 🔗 2️⃣ Crawl4AI:把网站捏成 LLM 扫一眼就能读的文本,API key…

        Read full on X, posted by @0xGky

        @0xGky · x.com

    10. Oct 81 post
      • 4. 一个像人一样控制 Chrome 的 agent:点击、滚动、登录、填表单、抓数据。 苏黎世联邦理工开发。95k+ 星标。 专门对付那些会屏蔽传统爬虫的复杂网站。MIT 协议。

        Read full on X, posted by @Techbruneth

        @Techbruneth · x.com

    11. Oct 71 post
      • 还是不敢撒手扒数据,这10个开源工具能把封锁成本给你压下去。 1️⃣ Firecrawl:URL丢进去,整站结构化输出,AI直接消费,JS渲染也不怕,14万★的GitHub Top 100项目。 🔗 2️⃣ Crawl4AI:把网页转成LLM爱吃的纯文本格式,完全免费不要key,16美元月费逼出来的爆款,7万★。 🔗 3️⃣…

        Read full on X, posted by @Manorgw

        @Manorgw · x.com

    You might also like

    Repositories close to apify/crawlee
    FeaturedAdvertise to 180K+ monthly visitors
    OfinTech Desk

    An open-source AI workspace for finance and accounting teams, in which every figure is traced to its source

    Bifrost

    Fastest Enterprise AI Gateway for Routing, Governing, and Securing all your AI Traffic

    It's a plan

    Open-source Linear alternative

    An issue tracker where AI agents take the work you assign them

    Omnigraph

    Versioned graph database on object storage. Git-style workflows and multimodal retrieval

    Busbar

    Control what AI can do before it acts

    apify/crawlee-python

    Python

    Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.

    9.7k863

    coder-hxl/x-crawl

    TypeScript

    Flexible Node.js crawler library

    1.9k117

    ScrapingBee/scrapingbee-node

    TypeScript

    A Node library to interact with ScrapingBee's API for headless browsers and proxy rotation

    1104

    scrapfly/python-scrapfly

    Python

    Official Python SDK for the Scrapfly platform: web scraping, screenshots, AI extraction, crawling, and a remote anti-bot browser. Integrates with Scrapy, LlamaIndex, and LangChain.

    6217

    isdk/web-fetcher.js

    HTML

    A web automation library that lets AI agents browse the web using declarative JSON, with a unified dual-engine API and powerful anti-bot evasion.

    244

    stretchcloud/deepscrape

    TypeScript
    33930

    vakra-dev/reader

    TypeScript

    Open source web infrastructure for AI. Scrape, crawl, and automate the web, clean markdown, browser sessions, ready for your agents.

    57646

    D4Vinci/Scrapling

    Python

    🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

    104.4k11.3k

    h4ckf0r0day/obscura

    Rust

    The headless browser for AI agents and web scraping

    36.6k3k