Gitingest

Gitingest is a Python CLI tool and library that converts a Git repository — remote URL, private repo, specific subdirectory, or local path — into a single structured text digest: a summary, a directory tree, and the concatenated file contents, all sized for pasting directly into an LLM context window. The core trick is a URL substitution: swap hub for ingest in any GitHub URL and get the same digest instantly through the hosted gitingest.com front-end, without installing anything.

Under the hood it respects .gitignore and a custom .gitingestignore, supports include/exclude glob patterns, caps file size and total digest size, and reports token counts via tiktoken so users know whether the output will fit their model’s context window. Beyond the CLI and importable Python API (ingest/ingest_async), the project ships a self-hostable FastAPI server (Docker/Docker Compose, optional S3-backed storage), official Chrome/Firefox/Edge browser extensions, and a public API — all built on the same underlying ingestion library.

What You Get

• A gitingest CLI command that ingests a local directory or remote Git URL and writes a digest to a file or stdout
• An importable Python API (ingest and ingest_async) for embedding the same ingestion logic in scripts and notebooks
• Automatic .gitignore and .gitingestignore handling, plus include/exclude pattern filters and file-size caps
• Token-count estimation via tiktoken so the digest size can be checked against a target model’s context window
• Support for specific branches, tags, commits, subdirectories, and private repositories via a GitHub personal access token
• A self-hostable FastAPI web server (Docker/Docker Compose, optional S3 storage) mirroring the gitingest.com experience

Common Use Cases

• Pasting a whole repository (or just one subdirectory) into an LLM chat to ask questions about the codebase
• Feeding a codebase digest into a coding agent or RAG pipeline as context
• Quickly reviewing an unfamiliar open-source project’s structure and contents in one text file
• Generating reproducible, versioned code snapshots for documentation or prompt templates
• Running a private, self-hosted digest server inside a company network instead of using the public gitingest.com

Under The Hood

Architecture The codebase separates concerns cleanly: query_parser.py resolves a URL or local path into an IngestionQuery pydantic model, clone.py performs the actual git clone, ingestion.py walks the resulting file tree into FileSystemNode objects while applying include/exclude patterns and the size caps defined in config.py (MAX_FILE_SIZE, MAX_DIRECTORY_DEPTH, MAX_FILES, MAX_TOTAL_SIZE_BYTES), and output_formatter.py renders the final summary/tree/content triple. entrypoint.py orchestrates all of this behind an async context manager that guarantees the temporary clone is removed even if ingestion raises. A parallel src/server package layers a FastAPI app (routers, query processing, S3 utilities) on top of the same core library instead of duplicating its logic, so the CLI, the Python API, and gitingest.com are three thin front-ends over one ingestion pipeline.

Tech Stack The core library targets Python 3.8+, packaged with setuptools in a src/ layout, using Click for the CLI, GitPython for repository cloning, httpx for async HTTP, pydantic for its query and filesystem-node models, pathspec for gitignore-style pattern matching, and tiktoken for token counting, with loguru for structured logging. The optional [server] extra adds FastAPI with uvicorn, boto3 for S3-backed digest storage, slowapi for rate limiting, prometheus-client for metrics, and sentry-sdk for error tracking; self-hosting is supported via a Dockerfile and a Compose file with dev/prod profiles plus a bundled MinIO service for local S3 emulation.

Code Quality The test suite covers the CLI, cloning, git utilities, gitignore handling, ingestion, notebook processing, and pattern matching, with dedicated subpackages for query parsing and the server, run under pytest across Ubuntu, macOS, and Windows on Python 3.8 through 3.13 in CI, with coverage tracked on one matrix leg. Linting enforces Ruff’s full rule set (with a short, explicitly tracked list of temporary exceptions) alongside pre-commit hooks and pylint configuration, the codebase uses pydantic models with from __future__ import annotations and type hints throughout, and errors are raised as custom typed exceptions rather than swallowed.

What Makes It Unique Its differentiator is less the core idea — turning a repo into text isn’t new — and more the execution: a memorable hub-to-ingest URL substitution that needs no installation, a token-aware output format tuned specifically for LLM context windows, and a consistent experience across a CLI, an importable library, a hosted web UI, official browser extensions, and a self-hostable server with S3 storage, all sharing one ingestion core.