Machine Learning & Deep Learning Packages
Machine-learning and deep-learning frameworks and model libraries for training, fine-tuning, and serving models (PyTorch, scikit-learn, Transformers).
Packages in Machine Learning & Deep Learning
PyTorch
A Python-first tensor library with GPU acceleration and a dynamic, define-by-run autograd engine for building and training deep neural networks.
Transformers
State-of-the-art pretrained models for text, vision, audio, and multimodal inference and training.
scikit-learn
Simple and efficient tools for machine learning and data analysis in Python.
tiktoken
OpenAI's fast, Rust-backed BPE tokenizer for counting and encoding text into the exact tokens GPT models see.
Datasets
One-line loading and fast, Arrow-backed processing for thousands of ML datasets
sentence-transformers
A Python library for computing state-of-the-art sentence, text, and image embeddings, rerankers, and sparse and multi-vector representations.
Tokenizers
Fast, production-grade tokenizer implementations for NLP and LLMs, written in Rust
huggingface_hub
The official Python client and hf CLI for the Hugging Face Hub, for downloading, uploading, and managing models, datasets, and Spaces.
ONNX Runtime
Cross-platform, high-performance ML inference and training accelerator for ONNX models
accelerate
A thin PyTorch wrapper that runs the same raw training script on CPU, multi-GPU, or TPU with mixed precision, DeepSpeed, and FSDP support.
qdrant-client
Official Python SDK for the Qdrant vector search engine
vision
PyTorch's official library for datasets, pretrained models, and image transformations in computer vision.
PEFT
Hugging Face's library of state-of-the-art parameter-efficient fine-tuning methods for large models.
pymilvus
The official Python SDK for Milvus, the open-source vector database for embedding-based similarity search.
spaCy
Industrial-strength Natural Language Processing library for Python
