unew

unew is a high-performance command-line utility designed for efficiently processing and managing unique lines from input streams, offering functionalities similar to `sort`, `uniq`, and `tee`. Its primary use case is deduplication and data organization, featuring advanced capabilities such as case-insensitive processing, file splitting, shuffling, and the ability to append new unique lines to existing files. Benchmarked against similar tools, unew demonstrates superior speed and memory efficiency, making it suitable for handling large datasets.