Pygments

Pygments is a generic syntax highlighter written in pure Python. It supports more than 500 languages and text formats, converting source code into richly marked-up output (HTML, LaTeX, RTF, terminal ANSI, and more) for use in code hosting platforms, documentation generators, forums, wikis, and any application that needs to prettify source code.

The library is built around a pluggable lexer/formatter/style architecture: lexers tokenize source text into a stream of typed tokens, formatters render that token stream into a target output format, and styles control the color/markup applied to each token type. This separation of concerns lets Pygments support an enormous range of languages while keeping each individual lexer small and focused, and it has made the project a de facto standard dependency behind tools like Sphinx, Jupyter, and countless static site generators.

What You Get

• Over 500 built-in lexers covering mainstream and extremely niche programming, markup, and configuration languages
• Multiple output formatters: HTML (with CSS classes or inline styles), ANSI terminal, LaTeX, RTF, SVG, and raw image output via Pillow
• A large library of bundled color styles (Monokai, Solarized, Dracula-compatible themes, etc.) plus a style API for custom themes
• The pygmentize CLI for highlighting files or piping code from stdin without writing any Python
• A plugin/extension system so third-party packages can register additional lexers, formatters, filters, and styles
• Filters for transforming token streams (e.g. stripping whitespace, adding line numbers, highlighting specific lines)

Common Use Cases

• Highlighting code blocks in static site generators and documentation tools (Sphinx, MkDocs, Jekyll plugins)
• Rendering syntax-highlighted code snippets on code-hosting and forum/wiki platforms
• Producing colorized terminal output for CLI tools that display source code or diffs
• Generating print-ready or PDF documentation with LaTeX-formatted code listings

Under The Hood

Architecture Pygments’ core pipeline is Lexer → token stream → Formatter, with pygments.lex() and pygments.highlight() in pygments/__init__.py as the primary entry points. pygments/lexer.py defines the base Lexer, RegexLexer, and ExtendedRegexLexer classes, where each concrete lexer (under pygments/lexers/) declares a set of named states with regex-to-token-action rules; RegexLexer.get_tokens_unprocessed walks the input applying the current state’s rules and can push/pop lexer states, enabling handling of nested or context-sensitive syntax (e.g. embedded code blocks). pygments/formatter.py defines the abstract Formatter base that concrete formatters (pygments/formatters/) subclass to turn (token_type, value) pairs into markup, while pygments/style.py and pygments/styles/ define color themes as declarative token-to-style mappings that formatters consult when rendering. pygments/cmdline.py implements the pygmentize CLI, and pygments/plugin.py wires in the setuptools/importlib entry-point plugin system that lets external packages register lexers, formatters, styles, and filters without modifying the core package. Tech Stack Pygments is pure Python (supporting Python 3.9+ per pyproject.toml), built with the hatchling build backend, and has effectively zero required runtime dependencies — the only optional dependency is colorama for Windows terminal color support. The project vendors small amounts of non-Python code (Emacs Lisp, StringTemplate, C, and dozens of other sample languages) purely as lexer test fixtures/example files, not as build dependencies. Code Quality The tests/ directory is extensive, including test_basic_api.py, test_cmdline.py, an examplefiles/ corpus with real-world snippets in hundreds of languages used to exercise each lexer, and snippets/ for targeted regression tests; the project is also continuously fuzzed via OSS-Fuzz per the README, and the maintainers explicitly document their approach to catastrophic-backtracking regex risk. Code style is consistent across the 500+ lexer modules, following a shared template of tokens dicts and small mixin classes for reuse. API Design The primary API surface is deliberately tiny — highlight(code, lexer, formatter) plus get_lexer_by_name()/guess_lexer() and get_formatter_by_name() factory functions — making the common case a one-line call, while the Lexer/Formatter/Style base classes offer a well-documented extension path for anyone adding new languages or output formats.