pyahocorasick

pyahocorasick is a Python library, implemented as a C extension with a pure-Python fallback, that lets you search for many string patterns in a single pass over an input text using the Aho-Corasick algorithm. Patterns are added to an Automaton, which behaves as a dict-like Trie keyed by string, and then finalized into a matching automaton with make_automaton(); after that, Automaton.iter() returns every occurrence of every pattern found in the haystack in linear time relative to the input length.

Because the automaton can be built once and reused, pyahocorasick is well suited to workloads where a fixed or slowly-changing dictionary of keys needs to be matched repeatedly against large or streaming inputs — intrusion detection, content filtering, bioinformatics sequence scanning, and log/text analytics. Built automatons are picklable, so an expensive index build can be done offline and reloaded at runtime instead of being rebuilt on every process start.

What You Get

• An Automaton class usable as both a dict-like Trie (add_word, get, membership testing) and, after calling make_automaton(), a full Aho-Corasick search engine via iter()
• A CPython C extension for performance, compiled at install time via a standard C toolchain, compatible with Python 3.9+ on Linux, macOS, and Windows
• Pickle support so a built automaton can be serialized to disk and reloaded instead of rebuilt on every run
• Configurable string storage (unicode vs. bytes) controlled at build time via the AHOCORASICK_UNICODE preprocessor flag
• A pure-Python reference implementation of the automaton included in the source tree (etc/py/) for environments where compiling a C extension isn’t practical

Common Use Cases

• Scanning large volumes of text or network payloads for a large, fixed dictionary of signatures in intrusion detection or anti-virus style tooling
• Matching thousands to millions of biological sequence keys (e.g. CRISPR guides) against streams of DNA sequencing reads in a single pass
• Content filtering or keyword spotting across user-submitted text against a curated blocklist/allowlist
• Persisting a prebuilt automaton via pickle to avoid rebuilding a large key index on every service restart

Under The Hood

Architecture - the core data structure is a Trie built from added string keys, which make_automaton() converts into an Aho-Corasick automaton by computing failure links between nodes; the Automaton class in the C extension (src/) exposes both the pre-conversion Trie/dict-like interface and the post-conversion iter() search method over the same object. Tech Stack - primarily C (52% of the codebase) for the extension core with a Python wrapper layer and a documented pure-Python fallback under etc/py/ for environments without a C compiler; built via setup.py/distutils with a Makefile driving the standard test-and-build workflow, CI runs through GitHub Actions. Code Quality - a substantial tests/ directory with dedicated regression tests per historical GitHub issue (test_issue_5.py through test_issue_133.py) alongside general test_basic.py/test_unit.py/test_unpickle.py files, reflecting an issue-driven regression-test discipline; the project has been in moderate/low-activity maintenance mode with roughly 1.3 commits/month recently, though releases remain periodic. API Design - the Automaton class deliberately overloads two mental models (dict-like Trie before make_automaton(), iterator-based search engine after) onto a single object, which keeps the surface small but requires understanding the two-phase build-then-search lifecycle to use correctly.