unicode-security

unicode-security is a Rust crate from the unicode-rs organization that implements the checks defined in Unicode Technical Standard #39 (Unicode Security Mechanisms). It gives Rust programs a way to detect visually confusable strings, mixed-script identifiers, and characters restricted from safe identifier use — the class of Unicode tricks used in phishing domains, spoofed usernames, and homoglyph attacks.

The crate exposes four focused APIs: skeleton() for confusable-string detection, GeneralSecurityProfile for identifier-allowed checks, MixedScript/AugmentedScriptSet for mixed-script detection, and RestrictionLevelDetection for classifying how permissive a string’s script usage is. It is no_std by default and depends only on unicode-script and unicode-normalization, making it easy to embed in security-sensitive parsers, validators, or the Rust standard library itself (it ships an optional rustc-dep-of-std feature).

What You Get

• skeleton() — computes the UTS #39 confusable skeleton of a string, so two visually-identical-but-different strings normalize to the same skeleton and can be compared for spoofing
• GeneralSecurityProfile trait — identifier_allowed() and identifier_type() to check whether a character is safe to use in an identifier per the Unicode Identifier Status/Type tables
• MixedScript / AugmentedScriptSet — detects whether a string mixes multiple Unicode scripts in a way that could indicate a spoofing attempt
• RestrictionLevelDetection — classifies a string’s restriction level (ASCII-only through Unrestricted) as defined by UTS #39
• no_std support with an optional rustc-dep-of-std feature for embedding directly in the Rust standard library toolchain

Common Use Cases

• Flagging visually confusable usernames or handles during account registration to block impersonation
• Validating identifiers in a parser or compiler front-end against Unicode’s identifier security profile
• Detecting mixed-script domain names or URLs as part of anti-phishing checks
• Classifying user-submitted text by restriction level before allowing it into a namespace that assumes a single script

Under The Hood

Architecture The crate is organized as a flat set of single-purpose modules under src/ — confusable_detection.rs, general_security_profile.rs, mixed_script.rs, and restriction_level.rs — plus a large generated tables.rs holding the Unicode Character Database lookup tables that every algorithm reads from. lib.rs is a thin re-export layer wiring each module to one UTS #39 mechanism (skeleton, GeneralSecurityProfile, MixedScript, RestrictionLevelDetection), with minimal cross-module coupling: general_security_profile depends only on tables::identifier, mixed_script wraps the sibling unicode-script crate’s ScriptExtension into an AugmentedScriptSet, and restriction_level composes both. The crate is no_std by default, so there’s no I/O or allocation-heavy machinery; the real “breaking” surface is tables.rs, since every public function is a lookup or iterator over those static tables and needs regeneration whenever the underlying Unicode version bumps.

Tech Stack A small Rust crate (edition 2018) with exactly two runtime dependencies — unicode-script for Script/ScriptExtension data and unicode-normalization for NFD normalization — both built with default-features = false to preserve no_std compatibility. An optional rustc-dep-of-std feature pulls in rustc-std-workspace-std/core and compiler_builtins, letting the crate be vendored directly into the Rust standard library. There’s no build script; the Unicode data tables are generated offline by a Python script (scripts/unicode.py) and committed as src/tables.rs, with CI re-running the generator and diffing the output to catch drift. GitHub Actions runs cargo build/cargo test on the beta toolchain plus a separate rustfmt --check job.

Code Quality Unit tests live in src/tests.rs and exercise identifier_allowed, single/mixed-script resolution, confusable skeleton generation, and mixed-script-confusable character detection against literal Unicode strings and characters — direct example-based tests rather than property-based or fuzz testing. Error handling is minimal by design: every public function returns a bool or enum rather than a Result, since none of the checks are fallible. The crate enforces #![deny(missing_docs, unsafe_code)] at the top level, so every public item must carry a doc comment and no unsafe is permitted anywhere. CI runs cargo fmt --check and cargo test but there is no clippy or coverage job.

API Design The public surface is intentionally narrow: four independent entry points (skeleton(), GeneralSecurityProfile::identifier_allowed(), MixedScript::is_single_script(), RestrictionLevelDetection::detect_restriction_level()), each reachable as an extension-trait method directly off char or &str with zero setup or wrapper types. The crate-level doc comment includes a runnable example showing the identifier-allowed check end to end. Documentation beyond that top-level example is thin — most trait methods carry a one-line doc comment linking to the relevant UTS #39 section rather than explaining edge cases in-crate, with edge-case behavior (empty strings, control characters) demonstrated only in the test suite.