botocore
The low-level, foundational Python SDK that underpins both boto3 and the AWS CLI
Repository Health
Technical Analysis
botocore is Amazon Web Services’ low-level Python interface to virtually every AWS service. Rather than hand-writing a client for each service, botocore generates clients from JSON service models, giving it broad and immediately up-to-date coverage as AWS ships new APIs and services.
It is not typically used directly by application developers; instead it is the shared foundation that boto3 (the higher-level, resource-oriented SDK) and the official AWS CLI are both built on. Anyone building session handling, credential resolution, request signing, or a custom AWS tool at a lower level than boto3 provides depends on botocore directly.
What You Get
- A
botocore.session.get_session()entry point that creates low-level clients for any AWS service (session.create_client('ec2'), etc.) - Automatic credential resolution across environment variables, shared config/credentials files, IAM roles, and SSO
- AWS Signature Version 4 request signing and endpoint resolution handled transparently per service/region
- Service clients generated from bundled JSON API models, so new AWS services and operations become available as botocore is updated rather than requiring hand-written bindings
- Optional
awscrt-backed extras for CRT-accelerated signing/checksums on performance-sensitive workloads
Common Use Cases
- Serving as the dependency that boto3 and the AWS CLI use internally for every AWS API call
- Building a custom, lightweight AWS tool that needs direct client/session control without boto3’s resource abstractions
- Implementing custom credential providers or request signing/middleware hooks for AWS API calls via botocore’s event system
- Powering third-party libraries (e.g. cloud storage adapters, infrastructure tools) that need AWS API access without the full boto3 surface
Under The Hood
Architecture: The core loop runs through session.py (Session/credential/config orchestration), client.py (dynamic per-service client class generation from model data), model.py (parses the JSON service models under data/ into a queryable ServiceModel/OperationModel structure), endpoint.py/endpoint_provider.py (region/partition endpoint resolution), auth.py (SigV4/SigV4A request signing), and hooks.py (the pluggable event system other AWS Python tools, including boto3, hook into to customize behavior). Tech Stack: Pure Python with a small dependency set (jmespath for response filtering, python-dateutil, urllib3), plus an optional crt extra pulling in awscrt for native-accelerated signing/checksums; the data/ directory bundles versioned JSON service models for every supported AWS service. Code Quality: Ruff-linted with a strict 79-char line length (Black-compatible), a pytest suite with dedicated slow and validates_models markers (the latter validating the bundled service models themselves), and a well-documented CONTRIBUTING process reflecting Amazon’s ownership and formal SDK maintenance policy. API Design: The session.create_client('service-name') pattern is uniform across all ~300+ AWS services since clients are generated from the same model schema, but the API is intentionally low-level (raw dict request/response shapes, manual pagination in some cases) — developers wanting resource objects, higher-level abstractions, or waiters typically reach for boto3 instead, which layers those on top of botocore’s client objects.
Used by 14 apps in this directory
Apache Airflow
Data Engineering
Define, schedule, and monitor complex data workflows as Python code — with a powerful UI, 80+ provider integrations, and battle-tested scalability across thousands of production deployments.
ClickHouse
Databases · Analytics · Data Engineering
Open-source column-oriented database that delivers real-time analytical queries on petabyte-scale data with millisecond latency.
Hatchet
AI Development · Developer Tools · Automation
A Postgres-backed orchestration engine for background tasks, AI agents, and durable workflows that replaces Redis queues and multi-datastore durable execution platforms with a single self-hostable service.
headroom
AI Development · Developer Tools
Compress everything your AI agent reads — tool outputs, logs, RAG chunks, and files — before it reaches the LLM, achieving 60–95% fewer tokens with the same answers.
knowhere
AI Development · Developer Tools
Transform messy, unstructured documents into persistent, navigable memory that AI agents can actually use.
Label Studio
AI Development · Data Engineering
Label Studio is an open-source, multi-type data labeling platform that lets teams annotate images, text, audio, video, and time series data with a configurable XML-based UI and export annotations in formats ready for any ML framework.
LanceDB
Databases · AI Development
Open-source, embedded vector database built on the Lance columnar format for fast multimodal search across billions of vectors, backed by Y Combinator (W23).
LearnHouse
Learning Management · CMS
Open-source LMS with AI tutoring, real-time collaboration boards, live code execution, and built-in course monetization — self-hosted in minutes.
MLflow
AI Development · Monitoring
The open source AI engineering platform for debugging, evaluating, monitoring, and optimizing production LLMs and agents at scale.