One script, every device. No per-seat licenses. No vendor lock-in.
Live demo • Features • Get started • Documentation • Community • Release notes
VirtualPyTest captures what a device shows (HDMI, camera, screen mirroring, browser) and drives it back (IR, Bluetooth remote, ADB, Appium, Playwright). On top of that it gives you a navigation graph of your app, a test runner, 24/7 monitoring, and Grafana analytics.
It runs on Linux, Raspberry Pi, Docker, or the cloud, and targets set-top boxes, Android TV, mobile phones, web apps, and anything else you can point a capture card at. It is built to replace commercial device-testing suites that cost $50k+ per year.
VirtualPyTest in 60 seconds — dashboard, devices, heatmap, a test run, its report and the KPI it measured.
 Map your app once Screens as nodes, actions as edges, one goto that drives a real browser |
 Proof, not a checkmark Every step with its screenshot, verification and evidence |
 Ask the AI, it acts Answers from the docs, then runs a script on a device from the chat |
 Every navigation, timed A KPI on every run, measured from the frames, with its report |
 Why did it fail? Reference, screen and pixel diff, side by side, from a failed step |
 Automatic zap detection Zap with the remote: the backend detects and times it, no code to write |
 Automatic incident detection Freezes caught on their own, and a 24 h review buffer on every device, no script |
 QuickTest Builder No code, just steps: press CH+, check the picture moves, run |
All videos, with new ones as they come, are on the Videos page.
 Map your app once, every test reuses it. Every screen and path becomes a reusable graph. Change a screen once and every test follows. |
 No-code for the team, Python for the engineers. Visual drag-and-drop builder for everyone. Full Python the moment you need real logic. |
 One script, every platform. The navigation graph keeps the UI out of the script. Named variants override only what differs per model. |
 It proves what is actually on screen. Image matching, OCR text, and AI detectors. Every check shows source, reference, and pixel diff. |
 Black screen, freeze, audio loss, caught in real time. Every device watched around the clock. Incidents alerted the moment they happen and tracked to resolution. |
 Talk to your lab. The AI drives the device. A real agent on a real model, with a built-in MCP server so any AI client can inspect and control devices. |
Show 10 more features (references, A/V quality, reports, Grafana, heatmap and rewind, fleet, device info, permissions, web UI, health)
 Reference images and text, easy to maintain. One library for every reference. Auto OCR and focus detection, one-click recapture when the UI changes. |
 Audio and video quality, measured every minute. Per-minute A/V quality KPIs per device, with trends per platform and software version. |
 Every run, a report you can read. Pass/fail summary with initial and final state, video, and KPI and zapping times captured automatically. |
 Every result, in native Grafana dashboards. Pass rates, duration, and volume per script. Per-step KPI radars across versions. Standard Grafana, yours to extend. |
 A fleet heatmap and 24h of screen to scrub back. One glance shows where the fleet hurts. Every device's screen is recorded, so you can see what happened at 3 a.m. |
 Your whole fleet, status and live screen. Hosts, devices, and system metrics on one dashboard. Live screen of every device, restart services from the browser. |
 Device details, read straight off the screen. Model, firmware, and version auto-extracted from the device's own About screen, in any language. |
 Permissions, down to the last action. Scoped per workspace, team, and user. Grant or deny every single action. |
 Runs in any browser, desktop to phone. Nothing to install. Fully responsive, so you can operate the whole lab from anywhere. |
 Every device accounted for, every host healthy. Device occupancy (busy vs idle, manual vs script) and host CPU, memory, disk, and temperature, 24/7. |
Full feature tour with larger screenshots: virtualpytest.angelstreet.io/docs/features
git clone https://github.com/AngelStreetCorp/virtualpytest.git
cd virtualpytest
./setup/docker/install_docker.sh # only if Docker is not installed yet
./setup/docker/launch.sh # full stack: database, server, host, web UI, Grafanalaunch.sh generates every secret, builds the images and prints the URLs when the API answers —
the web UI is on http://<this-machine>:5073. Every install path (Docker, one VM, Proxmox
fleet, developer setup) starts from the Get Started guide.
For Docker there are two supported shapes: use the command above for the complete Compose stack,
or run ./setup/docker/launch.sh --host-only when this machine only owns devices and should join
a server elsewhere. The registry publishes separate virtualpytest-server, virtualpytest-host,
and virtualpytest-frontend images; Compose pulls them together rather than providing one
monolithic image. See Install with Docker and
Add a host.
| VirtualPyTest | Commercial tools | |
|---|---|---|
| Cost | Free, open source | $50k+ / year |
| Runs on | Linux, Raspberry Pi, Docker, cloud | Often Windows only |
| Source code | Yours to read and modify | Vendor locked |
| Monitoring, analytics, AI | Included | Paid add-ons |
Stb-tester gives you Python. Witbe gives you no-code. VirtualPyTest gives you both, and AI can drive all of it.
| VirtualPyTest | Stb-tester | Witbe | |
|---|---|---|---|
| Free | ✅ | ❌ | ❌ |
| Open source | ✅ | ✅ | ❌ |
| Python scripting | ✅ | ✅ | ❌ |
| No-code builder | ✅ | ❌ | ✅ |
| AI drives the whole platform through MCP | ✅ | ❌ | ❌ |
| Hosts on any hardware, Raspberry Pi included | ✅ | ❌ | ❌ |
| Integration: Grafana, TestRail, Postman, Langfuse,slack | ✅ | ❌ | ❌ |
| Farms SauceLabs/BrowserStack and emulators | ✅ | ❌ | ❌ |
| **Fine-grained permissions and Workspace ** | ✅ | ❌ | ❌ |
| Heatmap across devices | ✅ | ❌ | ❌ |
| Automatic KPI measurement | ✅ | ❌ | ❌ |
| Automatic incident detection | ✅ | ❌ | ❌ |
| Requirements and test coverage | ✅ | ❌ | ❌ |
| Ask AI assistant | ✅ | ❌ | ❌ |
| 24h review buffer on every device | ✅ | ❌ | ❌ |
| Add your own controllers for new devices and test systems | ✅ | ❌ |
The five questions worth asking any testing vendor, answered for VirtualPyTest. More in the full FAQ.
Can you test our production app on multiple platforms without changing our code?
Yes. Nothing is installed in your app: no SDK, no instrumentation, no test build. The screen is captured from the outside (HDMI, camera, screen mirroring, browser) and the device is driven the way a user drives it (IR or Bluetooth remote, ADB, Appium, Playwright). The same script runs on STB, Android TV, mobile and web; the navigation tree holds what differs per platform. See Unified controller.
Can you show a live event monitored on multiple platforms, with a recording of what viewers saw?
Yes. Every device's screen is visible at once, with a fleet heatmap flagging devices in trouble. Each device's HDMI output is recorded continuously (video, audio, transcript) on a rolling 24-hour buffer, and incidents keep their start and end frames as evidence. See Visual capture.
Do you measure picture and sound quality, or only whether a screen appeared?
Both, on every captured frame, 24/7: black screen, freeze, blur, blockiness; audio loss, silence, loudness (LKFS), saturation; subtitles (OCR + language detection), zapping, banners; plus an approximate MOS (1–5) per minute in Grafana. The MOS is an indicator, not a certified lab measurement (no VMAF). See AV quality.
What happens to our tests when we redesign the app or switch language?
The scripts don't change. Scripts name destinations like navigate_to("settings"), never pixels or selectors, so a redesign only touches the navigation tree: recapture reference images in one click or let AI exploration rebuild nodes. Screens are recognised by a layout fingerprint that tolerates translated text, and text checks use OCR with language detection and fuzzy matching. A full redesign still needs someone to review the updated tree.
What will 24/7 monitoring on all our devices cost per month, with no surprises?
Software: €0 a month. Open source (AGPL v3), self-hosted, no licence, per-device or usage fees. Hardware is a one-time cost: ~€420 for 1 device (Raspberry Pi 5 + HDMI capture card), ~€480 for up to 4 devices on one host, ~€7 per extra capture card. Running costs are electricity, plus storage beyond the default 24 h of recordings (~7.5 GB per device per day). See the hardware guide.
- 🐛 Issues: report bugs or request features on GitHub Issues.
- 💬 Discussions: ask questions and share ideas in GitHub Discussions.
- 🤝 Contributing: see CONTRIBUTING.md.
- 💖 Sponsor: support the project on GitHub Sponsors.
VirtualPyTest is open source under the GNU AGPL v3. Free to use, study, modify, and self-host, including for commercial internal use. If you distribute a modified version or offer it to others as a service, you must publish your modifications under the same license. For commercial licensing outside these terms, contact the author.