human

Human is a browser- and Node.js-compatible library that bundles a wide range of human-analysis AI models behind a single unified API: 3D face detection with rotation correction, face description and recognition, body pose tracking, 3D hand and finger tracking, iris analysis, age/gender/emotion prediction, gaze tracking, gesture recognition, and body segmentation. Rather than shipping one model, it orchestrates dozens of independently-trained TensorFlow.js models through an attention-pipeline approach, running only the models a given configuration enables and skipping re-inference on unchanged frames to keep real-time video processing fast.

It targets both client and server environments: in the browser it runs against WebGPU, WebGL, WASM, or CPU backends (including inside a WebWorker), and in Node.js it can use the WASM backend or tfjs-node/tfjs-node-gpu for hardware-accelerated inference. The package ships multiple prebuilt bundles (ESM, ESM-nobundle, IIFE, Node, Node-WASM, Node-GPU) so consumers can pick the build that matches their runtime and backend without pulling in unused code paths.

What You Get

• A single Human class exposing face, body, hand, gesture, and segmentation detection behind one configuration object and one detect() call
• Prebuilt bundles for every target runtime — browser ESM/IIFE, WebWorker, Node with tfjs-node, Node-GPU, and Node-WASM — so you only ship the backend you need
• A library of independently swappable models (BlazeFace, FaceMesh, MoveNet, BlazePose, EfficientPose, HandTrack, CenterNet, RVM, Meet segmentation) selectable per feature via config flags
• Temporal interpolation and frame-skip caching (skipFrames/skipTime) that keep video processing smooth without re-running every model on every frame
• TypeScript type definitions and TSDoc-generated API reference for the full Config, Result, and Human surface
• Ready-to-run browser and Node.js demo apps (webcam, face match, Face ID, multi-thread, canvas export) covering common integration patterns

Common Use Cases

• Face detection, description, and 1:1/1:N face matching for login/verification flows (the Face ID demo pattern)
• Real-time webcam-based body pose and gesture tracking for fitness, motion-capture, or interactive-art applications
• Age, gender, and emotion estimation for audience-analytics or kiosk-style installations
• Background segmentation for virtual-camera / video-call background replacement without a server round-trip
• Server-side batch or video processing of face/body data via the Node.js bundles and demos (folder processing, ffmpeg-based video demo)

Under The Hood

Architecture The Human class in src/human.ts is the single orchestration point: it merges user config with defaults (src/config.ts), selects a TensorFlow.js backend (src/tfjs/backend.ts negotiates tensorflow → webgpu → webgl → wasm → cpu), and then delegates to one independently-implemented module per capability — face/facemesh.ts, face/blazeface.ts, body/movenet.ts, body/blazepose.ts, hand/handtrack.ts, object/centernet.ts, segmentation/rvm.ts, gesture/gesture.ts — each following the same load/warmup/predict convention and only running when its config.enabled flag is set. Results from whichever modules ran are merged into a single typed Result object (src/result.ts), with util/interpolate.ts smoothing values across frames and draw/draw.ts providing optional canvas rendering. The design is modular rather than monolithic: swapping or disabling a detection model touches one file and one config block, not the orchestration core.

Tech Stack Written in TypeScript targeting ESNext, Human has zero runtime npm dependencies — TensorFlow.js itself is vendored into the build via @vladmandic/tfjs rather than pulled in as a consumer-facing dependency. The custom build pipeline (build.js, using @vladmandic/build and esbuild) produces the six distinct bundle targets, and Microsoft’s api-extractor rolls the TypeScript sources into a single .d.ts. Development tooling includes ESLint (airbnb-base + @typescript-eslint) and a devDependency-only footprint of @tensorflow/tfjs-* packages used purely to build and test the backend-specific bundles.

Code Quality Tests live under test/ as a custom harness (test/node.js) that forks the compiled demo scripts as child processes and tallies pass/fail counts through a custom logger, rather than using a conventional assertion-based framework like Jest or Vitest — this is closer to integration/smoke testing across the shipped bundles and backends than unit testing of individual functions. tsconfig.json enables many strict compiler flags (alwaysStrict, noImplicitThis, noImplicitReturns) but leaves noImplicitAny off, and the only CI workflow present is CodeQL security scanning rather than an automated test run. Naming and module layout are consistent throughout, and public APIs carry TSDoc comments used to generate the published API reference.

What Makes It Unique Human’s distinguishing choice is combining a large catalog of independently-trained detection and analysis models — face, body, hand, gaze, gesture, segmentation, age/gender/emotion — behind one coherent, config-driven API with automatic backend negotiation and cross-frame temporal caching, rather than shipping a single-purpose detector. That orchestration layer, plus prebuilt bundles tuned to six different runtime/backend combinations, sets it apart from most TensorFlow.js model wrappers, which typically expose one model with no cross-model result merging or frame-skip optimization.