Artillery

Artillery is an open-source load testing and functional testing platform for Node.js. It ships as an artillery CLI (built on oclif) driven by declarative YAML test scripts that describe traffic phases (ramp-up, sustained load, spikes), weighted scenarios, and multi-step request chains. The core execution engine (@artilleryio/int-core) has built-in support for HTTP, WebSocket, and Socket.io protocols, and a dedicated Playwright engine drives real headless-browser sessions as first-class virtual users using the same script format.

What sets Artillery apart from most load testing tools is how it scales: rather than requiring you to provision and manage your own fleet of load-generator machines, the CLI can package and run distributed tests directly on AWS Lambda, AWS Fargate/ECS, or Azure Container Instances. There is no separate control plane or agent to install; artillery run --platform swaps out the execution backend for the same test script.

The project is structured as an npm workspaces monorepo of 17 packages, split into the CLI, the shared core runner, and independently versioned engines and plugins that are resolved by naming convention (artillery-engine-<name>, artillery-plugin-<name>). Official plugins add response assertions (expect), pass/fail thresholds (ensure), Apdex scoring, per-endpoint metrics, fake test data generation, memory inspection, and Slack notifications, while a publish-metrics plugin ships first-party integrations for CloudWatch, Datadog, Dynatrace, Mixpanel, New Relic, OpenTelemetry, Prometheus, and Splunk.

Most of the codebase is MPL-2.0 licensed and works fully self-hosted with no license key. Azure-specific modules are carved out under a separate Business Source License that restricts production use, and an optional plugin can report results to Artillery’s hosted Artillery Cloud dashboard, but neither is required to use the tool.

What You Get

• Multi-protocol load testing engines (HTTP, WebSocket, Socket.io) built into the core runner, plus a Playwright engine for real headless-browser scenarios
• Serverless distributed test execution on AWS Lambda, Fargate/ECS, or Azure Container Instances, launched directly from the CLI with no worker infrastructure to manage
• A plugin API with official plugins for response assertions (expect), pass/fail thresholds (ensure), Apdex scoring, per-endpoint metrics, fake test data, memory inspection, and Slack notifications
• First-party metrics integrations for publishing results to CloudWatch, Datadog, Dynatrace, Mixpanel, New Relic, OpenTelemetry, Prometheus, and Splunk
• Declarative YAML test scripts with weighted scenarios, multi-step request chains, and CSV/Redis-backed data-driven payloads
• An oclif-based CLI (artillery run, quick, run-lambda, run-fargate, run-aci, report) with dozens of runnable examples covering CI/CD, Kubernetes, GraphQL, gRPC, and SOAP

Common Use Cases

• Load testing REST or GraphQL APIs to find capacity limits before a launch or a known traffic spike
• Load testing real user flows in a browser (login, search, checkout) using the Playwright engine instead of raw HTTP requests
• Running large-scale, distributed load tests from AWS Lambda or Fargate without standing up dedicated load-generation infrastructure
• Wiring load tests into CI/CD pipelines or Kubernetes jobs as automated performance regression gates via the ensure plugin
• Functional and regression testing of APIs by reusing load test scripts with the expect plugin’s response assertions
• Load testing WebSocket, Socket.io, or custom protocol (gRPC/Twirp/SOAP) backends alongside standard HTTP APIs

Under The Hood

Architecture The monorepo splits cleanly into a CLI layer (packages/artillery), a shared test-execution engine (packages/core, published as @artilleryio/int-core), a commons layer, and independently versioned protocol engines/plugins (artillery-engine-playwright, artillery-engine-posthog, and eight artillery-plugin-* packages) resolved at runtime by naming convention — loadEngines() in packages/core/lib/runner.js requires artillery-engine-<name> for anything not built in, and the CLI’s plugin loader does the equivalent for plugins. The CLI itself is layered into cmds/ (oclif command definitions such as run, quick, run-lambda, run-fargate, run-aci) and platform/ (per-target execution backends: local, aws-lambda, aws-ecs, az, cloud), so a --platform flag swaps the entire execution backend without touching the core runner. Inside core, runner.js owns the load-generation loop itself: a phaser schedules ramp/arrival-rate phases, a weighted-pick module selects scenarios, and virtual users are dispatched through whichever protocol engines (http, ws, socketio, or a dynamically loaded Playwright engine) the script’s YAML config declares. This is a genuinely modular, plugin-oriented design — swapping engines or adding a new artillery-plugin-* package doesn’t require touching the scheduler — though it does mean every engine must honor the same (script, events, engineUtil) constructor contract, which is the abstraction the whole ecosystem depends on.

Tech Stack Artillery is a Node.js CLI built on oclif as its command framework, structured as an npm workspaces monorepo orchestrated by Turborepo for builds, tests, and linting. Protocol support comes from an HTTP client, a WebSocket client, and a Socket.io client inside @artilleryio/int-core, plus a dedicated Playwright engine for real-browser load testing. Distributed and cloud execution leans heavily on the AWS SDK v3 (separate client packages for Lambda, ECS, EC2, CloudWatch, S3, SQS, SSM, STS, and IAM) and, for Azure, on Azure’s identity and storage SDKs plus an Azure Container Instances client — the Azure integration is the piece carved out under a separate license. Metrics and observability integrations talk directly to CloudWatch, Datadog, Dynatrace, New Relic, OpenTelemetry, Prometheus, and Splunk, config and schema validation use joi and js-yaml, and the whole repo is published to npm as more than a dozen independently versioned packages through dedicated release workflows.

Code Quality Testing is consistent across packages via the tap test framework, with dedicated unit and acceptance suites, a separate Windows test workflow, and cloud end-to-end suites that exercise real AWS execution paths. Test doubles come from mocking and stubbing libraries alongside a real HTTP test-target server for acceptance coverage of the HTTP engine. Linting is centralized on a single formatter/linter tool enforced through pre-commit hooks and commit-message linting for conventional commits. CI is extensive, including static security analysis, multi-platform test runs, and workflows that actually exercise distributed cloud execution rather than just unit tests. Error handling in the core runner is largely explicit — engine load failures are caught and collected into a warnings object rather than crashing the whole run — though the codebase is overwhelmingly plain JavaScript, with TypeScript confined mostly to a dedicated schema-generation package, so type safety is limited outside of that boundary.

What Makes It Unique Artillery’s Playwright engine treats full headless-browser sessions as first-class virtual users rather than bolting browser testing on as an afterthought — the same YAML phase/arrival-rate model that drives raw HTTP load tests also drives Playwright scenarios, with built-in sampling logic to cap how many concurrent and total trace recordings are captured so tracing overhead stays bounded. Its serverless-native distributed execution model — packaging worker code and running it directly on Lambda, Fargate/ECS, or Azure Container Instances from the CLI with no separate control-plane server or agent fleet — is a genuinely distinct operational choice compared to tools that expect you to provision and manage your own load-generator workers. The engine/plugin resolution-by-package-name convention is a lightweight, ecosystem-friendly extension mechanism, though that pattern itself is common across the wider JavaScript tooling ecosystem rather than unique to Artillery.

Self-Hosting

Licensing Model Most of the codebase — the CLI, core engines, and most plugins — is MPL-2.0 licensed and fully usable in self-hosted deployments with no license key. Azure-specific modules (the az platform backend for Azure Container Instances) are licensed separately under the Business Source License 1.1, which converts to MPL 2.0 four years after each release.

Self-Hosting Restrictions

• No license key or paid plan is required to run Artillery locally, in CI, or distributed across AWS Lambda, Fargate, or ECS.
• Production use of the Azure Container Instances backend requires a commercial license from Artillery Software Inc; the default grant only covers evaluation and non-production use.

Enterprise Features No enterprise/, ee/, or pro/ source directories or license-gated feature flags were found in the repository. The CLI defines a hidden pro command topic, but it has no implementation in this repo.

Cloud vs Self-Hosted Artillery ships an opt-in cloud-reporting plugin (packages/artillery/lib/platform/cloud/cloud.js) that sends test run results to Artillery’s hosted dashboard when a --record flag or API key environment variable is supplied. This is a separate commercial SaaS product; self-hosted and local test runs work identically without it.

License Key Required No — except for production use of the Azure Container Instances backend, which requires a commercial license under the BSL grant.