All 158 Dependencies

Every package PostgresML depends on, ranked by repo health score.

PostgresML is a PostgreSQL extension that brings machine learning and large language model inference directly into the database engine. Built in Rust using the pgrx framework, it exposes SQL functions like pgml.train, pgml.predict, pgml.embed, pgml.rank, and pgml.transform that execute ML operations in the same process as PostgreSQL — eliminating external API calls, data serialization overhead, and the latency of moving data to separate model servers.

The core insight is architectural: data for ML systems is inherently larger and more dynamic than the models themselves, so it is more efficient to bring models to the data rather than constantly moving data to the models. PostgresML realizes this by embedding Python bindings (via PyO3) inside the Rust extension, enabling direct invocation of Hugging Face Transformers, scikit-learn, XGBoost, LightGBM, and other libraries from within SQL queries — with optional GPU acceleration through CUDA.

The ecosystem extends beyond the extension itself. The project includes a Rocket-based web dashboard for managing models and visualizing metrics, a multi-language SDK (Rust, Python, JavaScript, C) for building RAG pipelines, integration with pgvector for approximate nearest neighbor search, and pgcat for horizontal connection pooling and sharding. Both self-hosted Docker deployments and a managed cloud service are available, making it accessible from local development to production-scale workloads.

22 file-lock 2 23 @codemirror/lang-cpp ^6.0.2 23 @codemirror/lang-rust ^6.0.1 24 blas-src 0.10 outdated 25 @codemirror/lang-json ^6.0.1 25 @codemirror/lang-python ^6.1.3 27 transformers-stream-generator 28 blas 0.22 outdated 29 is-terminal 0.4 31 hash32 =0.2.1 outdated 32 CodeMirror State ^6.2.1 33 lang-javascript ^6.2.1 33 csv-async 1 33 mdurl ==0.1.2 37 CodeMirror ^6.0.1 37 pyo3-asyncio 0.18 outdated 38 openblas-src 0.10 40 dotenv (Rust) 0.15 40 once_cell 1 41 @codemirror/view ^6.21.0 41 signal-hook 0.3 outdated 43 colored 2 outdated 43 ndarray-stats 0.5 outdated 45 six ==1.16.0 outdated 45 yaml-rust 0.4 46 async-timeout ==4.0.3 outdated 46 lazy_static 1.4 46 walkdir 2.4.0 47 Autosize ^6.0.1 48 markdown-rs 1.0.0-alpha.14 48 md5 0.7 outdated 48 rmp-serde 1.3 49 ctrlc 3 49 inquire 0.6 outdated 52 pgvector-rust 0.3 outdated 53 num-traits 0.2 53 python-dateutil ==2.8.2 outdated 54 aho-corasick 0.7 outdated 54 csv 1.3 56 PyYAML ==6.0.1 outdated 57 convert_case 0.6 outdated 57 Evaluate 57 ndarray 0.15 outdated 59 tracing 0.1.37 59 tracing 0.3.17 outdated 60 owo-colors 3 outdated 61 typetag 0.2 62 glob 0.3 62 rich ^13.4.2 outdated 63 einops 64 console-subscriber * 64 neon 0.10 outdated 64 Scraper 0.17 outdated 65 multiprocess ==0.70.15 outdated 66 Certifi ==2023.7.22 outdated 66 markdown-it-py ==3.0.0 outdated 66 python-dotenv ^1.0.0 66 SacreBLEU 67 url 2.4 68 base64 0.21 outdated 68 parking_lot 0.12 68 pendulum ^2.1.2 outdated 69 dill ==0.3.7 outdated 69 itertools 0.10 outdated 69 orjson 69 pytz ==2023.3 outdated 69 tzdata ==2023.3 outdated 70 indexmap 2.7 72 proc-macro2 1.0 72 sea-query 0.30.7 outdated 72 sea-query 0.5.0 outdated 73 SQLx 0.7.3 outdated 75 anyhow 1 75 dotenv ^16.3.1 outdated 75 torchaudio 75 xFormers 76 serde_json 1 76 toml 0.7 outdated 77 aiosignal ==1.3.1 outdated 77 async-trait 0.1.71 77 Chrono 0.4 77 flate2 1.0 77 indicatif 0.17.6 outdated 77 log 0.4 77 quote 1.0.9 77 serde 1 78 env_logger 0.10 outdated 78 sailfish 0.8 outdated 80 idna ==3.4 outdated 80 Optimum 81 frozenlist ==1.4.0 outdated 81 fsspec ==2023.6.0 outdated 81 regex 1 82 rand 0.8 82 xxhash ==3.3.0 outdated 83 attrs ==23.1.0 outdated 83 Charset Normalizer ==3.2.0 outdated 83 heapless 0.7 outdated 83 lopdf 0.31.0 outdated 83 pgrx =0.12.9 outdated 83 pgrx =0.12.9 outdated 83 Requests ^2.31.0 83 reqwest 0.11 outdated 83 sqlparser 0.38 outdated 84 uuid 1.3.3 85 notify 6 outdated 85 notify 0.3 outdated 85 packaging ==23.1 outdated 86 serde_with 3.8.1 87 multidict ==6.0.4 outdated 88 dbt-postgres ^1.5.2 88 filelock ==3.12.3 outdated 88 tantivy 0.19 outdated 88 Tokenizers 88 typing_extensions ==4.7.1 outdated 89 Click ^8.1.6 90 Black ^23.7.0 outdated 90 Clap 4 90 DOMPurify ^3.0.6 90 PEFT 90 tqdm ==4.66.1 outdated 91 accelerate 91 bitsandbytes 91 comrak 0.17 outdated 91 PyO3 0.23 outdated 91 pytest-asyncio 91 time 0.3 92 futures 0.3.29 92 LangChain 92 Marked ^9.1.0 outdated 92 sentence-transformers 92 Sentry Rust SDK 0.31 outdated 92 Sentry Rust SDK 0.31 outdated 92 Sentry Rust SDK 0.31 outdated 92 syn 2.0.17 outdated 92 urllib3 ==2.0.4 outdated 92 yarl ==1.9.2 outdated 93 Tokio 1 94 Datasets 94 DeepSpeed 94 jest ^29.6.1 outdated 94 Bolt for Python ^1.18.0 94 TRL 95 aiohttp ==3.8.5 outdated 95 dbt-core ^1.5.2 95 huggingface_hub 95 LightGBM 95 OpenAI Python SDK >=1.3.7 95 PyArrow ==13.0.0 outdated 95 vision 95 Transformers 96 CatBoost 96 NumPy ==1.25.2 outdated 96 pytest 96 PyTorch 96 TypeScript ^5.1.6 outdated 97 XGBoost 98 pandas ==2.0.3 outdated

Join founders buildingwith open source

Opinionated takes, migration guides, cost-saving tips, and insights from the open source ecosystem.

Subscribe on Substack
Join 750+ subscribers

Search