Flight shuffle now compresses with LZ4 by default. Frames are compressed once on the map side and stay compressed across disk and the wire. 1 TB TPC-H repartition sweep: ~10% faster on local NVMe,…

🚢 Daft v0.7.15 just shipped. try_cast() converts types without crashing your pipeline — invalid values become null instead of throwing a runtime error. Also in this release: LZ4 flight shuffle…

Flight shuffle now compresses with LZ4 by default. Frames are compressed once on the map side and stay compressed across disk and the wire. 1 TB TPC-H repartition sweep: ~10% faster on local NVMe…

做 RAG、训练数据清洗或多模态检索时，数据经常散在 Parquet、图片、音频、视频、HuggingFace 数据集里。预处理脚本越堆越乱，最后很难复用。Daft 是一个面向 AI 和多模态工作负载的数据引擎。 GitHub： 它把这些数据处理任务放进统一的 DataFrame / ETL 管线里，支持 Parquet、Iceberg、Ray 等生态，也能处理非结构化媒体。适合把…

Processes structured data alongside images, audio, and video

🚢 Daft v0.7.14 has shipped Parquet reader rewrite — up to 17x faster remote reads Streaming distributed limits Native UUIDv7 generation JSON array/object functions

Distributed limits in v0.7.14 are now streaming. Ray actor holds atomic (skip, take, done) state. Workers claim per morsel, slice in place. When budget is exhausted, scheduler cancels remaining…

Arrow PyCapsule Interface in v0.7.11. Zero-copy DataFrame exchange between Daft, PyArrow, Polars, cuDF — direct pointers via the C Data Interface. No more serializing through the Python heap. via

Daft now has bidirectional ASOF joins. Backward: "what happened before this?" >>> latest right row where right.key <= left.key Forward: "what happens next?" >>> earliest right row where right.key >=…