onnxruntime-node

onnxruntime-node is the official Node.js binding for ONNX Runtime, Microsoft’s cross-platform machine-learning inference and training accelerator. It lets Node.js and Electron applications load ONNX models exported from PyTorch, TensorFlow, scikit-learn, and other frameworks, then run inference locally with native-speed execution.

The package ships prebuilt native binaries for Windows, macOS, and Linux across CPU and GPU execution providers (CUDA, DirectML, CoreML, WebGPU), so most consumers get hardware-accelerated inference with a single npm install and no local build step. It sits underneath higher-level JavaScript AI tooling and is the standard way to serve ONNX models from a Node.js backend.

What You Get

• Prebuilt native binaries for Windows, macOS, and Linux (x64 and arm64) with no local compilation required
• Multiple execution providers: CPU by default, plus CUDA, DirectML, WebGPU, and CoreML for hardware acceleration
• A shared onnxruntime-common TypeScript API surface also used by the browser (onnxruntime-web) and React Native bindings, so model code ports across platforms
• Support for both the stable release channel and nightly @dev builds for early access to new execution providers
• Compatibility with Electron apps in addition to plain Node.js runtimes

Common Use Cases

• Serving a PyTorch or TensorFlow model exported to ONNX from a Node.js API backend without a separate Python inference service
• Running local, on-device inference in an Electron desktop application
• GPU-accelerated inference pipelines on Linux/Windows servers via CUDA or DirectML
• Backing JavaScript AI SDKs and libraries (e.g. embedding or tokenizer pipelines) that need a portable ONNX runtime under the hood

Under The Hood

Architecture: onnxruntime-node is a thin native-addon layer over the shared ONNX Runtime C++ inference engine that lives in the same monorepo. src/inference_session_wrap.cc wraps the core InferenceSession C++ class via node-addon-api, exposing it to JS as binding.ts -> index.ts, which in turn implements the platform-agnostic InferenceSession/Tensor interfaces declared in the shared onnxruntime-common package (also consumed by onnxruntime-web and onnxruntime-react-native). Execution provider selection (CPU/CUDA/DirectML/CoreML/WebGPU) is resolved at session-creation time and dispatches into the corresponding native EP built into the prebuilt binary. Tech Stack: The public surface is TypeScript compiled to both CJS and ESM; the native layer is C++ built with node-addon-api and packaged per-platform via postinstall/script/install, with adm-zip and global-agent handling binary download/proxying. Code Quality: The repo carries an extensive C++/JS test suite (test/test-main, mocha-based JS tests, CI matrices across OS/EP combinations) reflecting Microsoft’s production usage; the JS layer itself is a thin, well-typed wrapper so most correctness risk sits in the native engine, which has its own dedicated test suites. API Design: The InferenceSession.create() -> run() pattern is minimal and consistent across all three JS packages (node/web/react-native), so a model pipeline written once ports with little change between server, browser, and mobile targets; the tradeoff is that platform-specific concerns (execution providers, binary installation) leak into setup/install steps rather than being abstracted away.