wikimedia/sentencex

Toolkit to segment text into sentences or other semantic units in a robust, efficient and adaptable way.

A multilingual command line sentence tokenizer in Golang

Simple text sentence splitting and counting. Supports atleast english, german and dutch, possibly more. If you find it works well enough for your language, please let me know!

Real-time processing and delivery of sentences from a continuous stream of characters or text chunks.

🚀 The Fastest Chunker in the West 🇺🇸 Upto 1TB/s "semantic" chunking, quick and easy!

One library to split them all: Sentence, Code, Docs. Chunk smarter, not harder; built for LLMs, RAG pipelines, and beyond.

🍱 Semantically create chunks from large document for passing to LLM workflows

Stanford NLP Python library for tokenization, sentence segmentation, NER, and parsing of many human languages

A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.