uniseg

uniseg implements Unicode Text Segmentation (UAX #29), Unicode Line Breaking (UAX #14), and monospace font string-width calculation (similar to wcwidth) as a pure Go library with zero external dependencies. It solves a problem the Go standard library leaves unaddressed: a single code point is not the same thing as a user-perceived character, and naive byte or rune counting produces wrong results for combining marks, emoji sequences, regional-indicator flags, and Hangul syllables.

The package exposes three tiers of API: single-call convenience functions like GraphemeClusterCount and StringWidth for the common case, a Graphemes iterator class for readable loop-based consumption, and allocation-free Step/StepString functions for performance-critical callers that need to track parser state across calls themselves. It is a foundational dependency for terminal UI ecosystems such as tview, tcell, and bubbletea, where accurate cursor positioning and line wrapping depend on correct grapheme and width calculations.

What You Get

• GraphemeClusterCount and StringWidth for one-line character counting and terminal-width calculation
• A Graphemes iterator class for ergonomic loop-based traversal of grapheme clusters with word/sentence/line-break flags attached
• Allocation-free Step and StepString functions that return cluster, boundary, and width information in a single pass for performance-sensitive code
• Specialized First* functions (FirstWord, FirstSentence, FirstLineSegment, etc.) for callers who only need one type of boundary
• ReverseString, which reverses a string while preserving grapheme cluster integrity instead of corrupting multi-rune characters

Common Use Cases

• Calculating accurate cursor positions and line wrapping in terminal UI frameworks and text editors
• Counting the true number of user-perceived characters in strings containing emoji, combining marks, or CJK text
• Implementing word/sentence-boundary-aware text selection or search in a text-processing application
• Wrapping text to a fixed display width when rendering to a monospace terminal or fixed-width font

Under The Hood

Architecture uniseg is a single flat Go package with no internal layering: boundary logic for each Unicode algorithm lives in its own file (grapheme.go, word.go, sentence.go, line.go), width.go handles monospace width rules, and step.go unifies all four boundary types plus width into one combined scan. Large generated data files (graphemeproperties.go, wordproperties.go, sentenceproperties.go, lineproperties.go, eastasianwidth.go, emojipresentation.go) encode Unicode character-property lookup tables consumed by explicit finite-state-machine transition functions (transitionGraphemeState, transitionWordBreakState, etc.). Parser state is packed into a single int and threaded through successive calls so iteration can resume without allocation; there is no dependency injection or interface layer, since the entire package is a stateless, streaming, functional API plus a thin Graphemes struct that wraps StepString for convenience. Because every boundary function and the Graphemes wrapper share the same packed-state encoding, changing that encoding would require updating every transition function in lockstep.

Tech Stack The package depends on nothing beyond the Go standard library (only unicode/utf8 is imported), targets Go 1.18 per go.mod, and ships generator scripts (gen_properties.go, gen_breaktest.go) that regenerate the Unicode property and conformance-test tables from official Unicode Character Database source files via go generate. There is no build system beyond the Go toolchain, no database, and no network calls; it is consumed as an imported library by terminal UI frameworks such as tview, tcell, and bubbletea rather than run as a standalone program.

Code Quality Test coverage is extensive: eleven test files mirror nearly every source file, and several of them (graphemebreak_test.go, wordbreak_test.go, sentencebreak_test.go, linebreak_test.go) are large conformance suites sourced directly from the official Unicode UCD test data, meaning correctness is verified against the authoritative reference rather than only hand-picked cases. Naming is consistent, idiomatic Go, exported functions carry detailed godoc comments, and the design avoids error returns entirely since the API is pure computation with no I/O. No CI workflow configuration is present in the repository (only a FUNDING.yml under .github), and no linter config file was found, though the README displays a Go Report Card badge claiming an A+ grade.

API Design The library’s three-tier API is a deliberate developer-experience choice: single-call helpers (GraphemeClusterCount, StringWidth) cover the common case with no setup, the Graphemes class gives a familiar iterator pattern for general use, and the zero-allocation Step/StepString functions serve performance-critical callers willing to manage state themselves. The specialized First* function family lets callers opt into exactly one boundary type instead of paying for all four, which is unusual ergonomic care for a low-level Unicode segmentation library and lets it serve both quick scripts and hot rendering loops in terminal UI code from the same codebase.