Footer

Your go-to destination for discovering trending open-source projects and uncovering the insights that matter

Legal

  • Terms and conditions
  • Privacy policy

More

  • Pricing
  • Advertise
  • onedump
  • gosnakego
  • Contact us

Social

  • X (Twitter)
  • GitHub
Trendshift owl
What they build speaks louder than words
© 2026 Trendshift, created by Julian Li with ❤️
SignalAdvertise
Trendshift
SignalAdvertise
Sign in
Trendshift owl

Discover trends that matter

SignalAdvertise
DailyWeeklyMonthlyYearly
Live mentionsTopics
RepositoriesDevelopers
InsightsStats

Trina AX
Trina AX
It's a plan
It's a plan
Omnigraph
Omnigraph
OfinTech Desk
OfinTech Desk
Bifrost
Bifrost
Busbar
Busbar

LanguageMachines/ucto

Unicode tokeniser. Ucto tokenizes text files: it separates words from punctuation, and splits sentences. It offers several other basic preprocessing steps such as changing case that you can all use to make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation. Ucto comes with tokenisation rules for several languages and can be easily extended to suit other languages. It has been incorporated for tokenizing Dutch text in Frog, our Dutch morpho-syntactic processor. http://ilk.uvt.nl/ucto --

Visit GitHub
C++
72
14
5 contributors
GNU General Public License v3.0
website
FeaturedAdvertise to 180K+ monthly visitors
Trina AX

A self-hosted team AI workspace for company files, with per-person folder permissions

It's a plan

Open-source Linear alternative

An issue tracker where AI agents take the work you assign them

Omnigraph

Versioned graph database on object storage. Git-style workflows and multimodal retrieval

OfinTech Desk

An open-source AI workspace for finance and accounting teams, in which every figure is traced to its source

Bifrost

Fastest Enterprise AI Gateway for Routing, Governing, and Securing all your AI Traffic

Busbar

Control what AI can do before it acts

Momentum

How fast LanguageMachines/ucto is gaining stars, and where it lands on Trendshift and GitHub Trending

Activity and trending

Stars, forks and merged pull requests each day or month, lined up with the repository's rank on Trendshift Trending

Nov 16 – Nov 22, 2026

Stars +0 in 7 days

01

Forks +0 in 7 days

01

Merged PRs 0 in 7 days

01

Trendshift Trendingwhere the repository ranked each day it made the top 25

Not in the Trendshift top 25 in this period

Nov 16Nov 17Nov 18Nov 19Nov 20Nov 21Nov 22
The left and right arrow keys step from day to day; Home and End go to the first and the last

Stars, forks, merged pull requests and Trendshift Trending rank, by day
DayStars gainedForks gainedMerged PRs
Mon, Nov 16, 2026+0+00
Tue, Nov 17, 2026+0+00
Wed, Nov 18, 2026+0+00
Thu, Nov 19, 2026+0+00
Fri, Nov 20, 2026+0+00
Sat, Nov 21, 2026+0+00
Sun, Nov 22, 2026+0+00
Total+0+00
  • 1#1 on the list
  • 7Rank that day
  • Not trending

You might also like

Repositories close to LanguageMachines/ucto
FeaturedAdvertise to 180K+ monthly visitors
Busbar

Control what AI can do before it acts

Omnigraph

Versioned graph database on object storage. Git-style workflows and multimodal retrieval

It's a plan

Open-source Linear alternative

An issue tracker where AI agents take the work you assign them

OfinTech Desk

An open-source AI workspace for finance and accounting teams, in which every figure is traced to its source

Trina AX

A self-hosted team AI workspace for company files, with per-person folder permissions

Bifrost

Fastest Enterprise AI Gateway for Routing, Governing, and Securing all your AI Traffic

neurosnap/sentences

Go

A multilingual command line sentence tokenizer in Golang

47243

sanderland/script_tok

Python

Code for the paper "BPE stays on SCRIPT", "Which Pieces Does Unigram Tokenization Really Need?" and MinGram

225

wikimedia/sentencex

Rust

A sentence segmentation library with wide language support optimized for speed and utility.

14721

textlint-rule/sentence-splitter

TypeScript

Split {Japanese, English} text into sentences.

13820

WeBuddhist/Botok

Python

🏷 བོད་ཏོག [pʰøtɔk̚] Tibetan word tokenizer in Python

8417

segment-any-text/wtpsplit

Python

Toolkit to segment text into sentences or other semantic units in a robust, efficient and adaptable way.

1.4k83

brown-uk/nlp_uk

Groovy

This is a project to demonstrate NLP API from LanguageTool for Ukrainian language.

8114

vanderlee/php-sentence

PHP

Simple text sentence splitting and counting. Supports atleast english, german and dutch, possibly more. If you find it works well enough for your language, please let me know!

8423

spencermountain/compromise

JavaScript

modest natural-language processing

12.2k671