obs-cv-plugin

A background-segmentation video filter for OBS Studio that runs on OpenVINO (CPU, Intel iGPU, or an NVIDIA GPU through OpenVINO 2026's GPU plugin), and the Python tooling that measures what every stage of it costs per frame.

The plugin is the companion code for the CondadosAI post Build a computer-vision plugin for OBS Studio: https://condados.ai/blog/obs-computer-vision-plugin-openvino. The point of the post is the plumbing: how a filter gets pixels off the GPU, where inference has to live so OBS never drops a frame, and what the whole trip from camera to virtual camera costs.

Webcam / media source ──► [ obs-cv-plugin filter ] ──► OBS scene ──► Virtual Camera ──► Zoom, Meet, …
                             │ GPU: draw source at 256×256 → stage → map (last frame's copy)
                             │ worker thread: BGRA→RGB float → OpenVINO → mask
                             │ GPU: upload R8 mask → composite in an .effect

Quickstart: build it and use it (no Python needed)

The segmentation models are committed under plugin/data/models/, so building the plugin needs only the C++ toolchain. The Python tooling further down is for regenerating the models and reproducing the measurements; you can ignore it entirely on a first pass.

OBS must come from the obsproject/obs-studio PPA (or another install that ships the libobs development files). A snap or flatpak OBS provides neither the CMake config this build needs nor the plugin directory it installs into.

# 1. OBS + headers and build tools. intel-opencl-icd is what makes the Intel iGPU
#    exist for OpenVINO ("GPU.0"); skip it and only CPU shows up, which also works.
sudo add-apt-repository ppa:obsproject/obs-studio
sudo apt install obs-studio cmake ninja-build build-essential pkg-config libopencv-dev intel-opencl-icd

# 2. OpenVINO C++ runtime (no root needed): the official archive
#    (on Ubuntu 24.04, use the ubuntu24 archive from the same directory listing)
mkdir -p deps && cd deps
curl -L -o ov.tgz https://storage.openvinotoolkit.org/repositories/openvino/packages/2026.3/linux/openvino_toolkit_ubuntu22_2026.3.0.22451.bd8d6542e3c_x86_64.tgz
tar xzf ov.tgz && mv openvino_toolkit_* openvino && rm ov.tgz && cd ..

# 3. Build and install into your OBS profile
cmake -S . -B build -G Ninja -DOpenVINO_DIR=$PWD/deps/openvino/runtime/cmake
cmake --build build
cmake --build build --target install-user   # → ~/.config/obs-studio/plugins/obs-cv-plugin/

Then, in OBS:

1. Add a video source (your webcam, or a looping media source).
2. Right-click the source → Filters → + under Effect Filters → Background Segmentation (OpenVINO). The background turns green immediately; the filter's properties choose the device, the background colour or transparency, and the mask threshold/feather.
3. To use it in a call: Start Virtual Camera in the main window, then pick OBS Virtual Camera in Zoom/Meet/Teams. On Linux OBS asks to install v4l2loopback the first time; on machines with Secure Boot the module must be signed or the prompt fails silently (check sudo dmesg | grep v4l2loopback).

Every load, model compile and error is logged with an [obs-cv-plugin] prefix in the OBS log (Help → Log Files), and a stats line appears there every 5 seconds while the filter runs; that line is the quickest confirmation the plugin is alive.

Troubleshooting

• The filter is not in the list. Either OBS is the snap/flatpak build (see the note above), or the module failed to load: search the OBS log for os_dlopen. The most common cause of the latter is that deps/openvino/ moved or was deleted after building; the module's RPATH points at that absolute path, so keep the repo where it was built or rebuild after moving it.
• The filter is there but the video is unchanged. A failed model load or effect compile degrades to passthrough on purpose; the reason is in the OBS log under [obs-cv-plugin]. If the 5-second stats line is present, the filter is running and the problem is elsewhere (threshold at an extreme, transparent background over a transparent scene).
• No GPU.0 in the device dropdown. The Intel iGPU only enumerates once the OpenCL compute runtime is installed (sudo apt install intel-opencl-icd); this is a driver, not a hardware limitation. An NVIDIA card may appear as GPU.1 through an undocumented OpenVINO code path; treat it as an experiment, not a supported target.

Layout

obs-cv-plugin/
├── CMakeLists.txt                 # the plugin build (libobs + OpenVINO + OpenCV)
├── plugin/
│   ├── src/plugin-main.cpp        # module entry, registers the filter
│   ├── src/cv-filter.cpp          # the filter: readback, worker thread, mask upload, composite
│   ├── src/segmenter.cpp/.hpp     # OpenVINO wrapper (twin of src/obscv/core/segmenter.py)
│   └── data/                      # shipped with the plugin
│       ├── effects/mask-composite.effect
│       ├── locale/en-US.ini
│       └── models/<variant>/{fp32,fp16,int8}.{xml,bin}   (committed; `obscv publish-models` regenerates them)
├── src/obscv/                     # Python tooling (uv project, `obscv` CLI)
│   ├── config.py                  # paths, model ids, protocol constants
│   ├── core/segmenter.py          # OpenVINO runtime, same preprocessing as the C++
│   ├── core/data.py               # model download, EasyPortrait subset via HTTP range reads
│   ├── core/export.py             # ONNX → IR (FP32/FP16), NNCF INT8
│   ├── core/benchmark.py          # precision × device speed, mask agreement
│   ├── core/timecode.py           # wall-clock block code for end-to-end latency
│   ├── core/obsbench.py           # drives OBS over obs-websocket
│   └── cli.py
├── output/                        # saved artifacts the post cites
├── DESIGN.md                      # the brainstorm and the measurement plan
└── deps/openvino/                 # OpenVINO C++ archive (not committed)

Adapting it to your own model

The split to keep: plugin-main.cpp and cv-filter.cpp are the shell and the plumbing and do not know what the model does; segmenter.cpp is the only file that does. To swap the payload:

1. Export your model to OpenVINO IR (ov.convert_model, static shape) and put the .xml/.bin pair under plugin/data/models/<variant>/. The input size is read from the IR in the Segmenter constructor, so a different resolution needs no code change.
2. The input contract is hard-coded in Segmenter::run: BGRA resized with INTER_LINEAR, converted to RGB, scaled by 1/255 with no mean/std normalisation, packed as NCHW float32. If your model normalises differently, that is the convertTo/split block to change.
3. The output decode assumes a single-channel float map in [0, 1] (alpha.convertTo(mask_u8, CV_8UC1, 255.0)). A different head (boxes, keypoints, a full image) replaces that line and the compositing in plugin/data/effects/mask-composite.effect.
4. src/obscv/core/segmenter.py mirrors the C++ preprocessing step for step. Update it to match and the whole benchmark suite (obscv benchmark, obscv obs-bench) runs against your model unchanged.

The Python tooling: regenerate the models, reproduce the numbers

Everything below is optional; the committed IRs and artifacts came from these commands. Tested on Ubuntu 22.04. The project is managed with uv (curl -LsSf https://astral.sh/uv/install.sh | sh if you do not have it):

uv sync                         # creates the venv and installs the pinned deps

uv run obscv download           # ONNX models + 200 EasyPortrait test images/masks (into data/, gitignored)
uv run obscv export             # → models/<variant>/{fp32,fp16}.xml, checked against onnxruntime
uv run obscv quantize           # → models/<variant>/int8.xml (NNCF PTQ, 100 calibration images)
uv run obscv benchmark          # → output/benchmark.md, speed.csv, quality.csv, latency.png
uv run obscv publish-models     # copy the IRs into plugin/data/models/

Measuring inside OBS

obs --websocket_port 4455 --websocket_password secret &      # server must be enabled once in Tools → WebSocket Server Settings
OBS_WS_PASSWORD=secret uv run obscv obs-bench                # 720p + 1080p renditions × devices × modes × readbacks → output/plugin_stages.csv
# the clip below comes from `uv run obscv download`
OBS_WS_PASSWORD=secret uv run obscv obs-bench --clips data/pexels-5941016-1080p25.mp4 \
    --devices GPU.0,CPU --readbacks model --heavy 20 --out plugin_stages_heavy.csv   # emulate a 20× heavier model

The plugin's Model repeats per frame property (and the --heavy values above) runs the network N times per frame so the sync-vs-worker comparison can be made with a model that does not fit in the frame budget.

End-to-end latency (camera in → virtual camera out)

sudo modprobe v4l2loopback devices=2 video_nr=10,20 exclusive_caps=1,0 \
     card_label="OBS Virtual Camera","obscv fakecam"
# exclusive_caps=1 on the OBS side so browsers accept it as a camera; 0 on the fake
# camera, or the device stops announcing itself as an output after OBS has opened it
OBS_WS_PASSWORD=secret uv run obscv latency-bench --device GPU.0    # → output/e2e_latency.csv

latency-bench starts the fake camera, adds it to OBS as a V4L2 source, starts the Virtual Camera, and measures five paths (no filter, worker, sync, and both modes with a 10× model). The pieces are also available separately: obscv fakecam writes the timecode frames, obscv latency reads the virtual camera and decodes them.

Licence

The code in this repository is Apache-2.0 (see LICENSE). Things it links to or downloads carry their own terms:

Component	Terms	Note
libobs (OBS Studio 30.2.3)	GPL-2.0-or-later	The built plugin links libobs; distribute the binary under GPL-compatible terms (Apache-2.0 source is compatible with GPLv3).
OpenVINO 2026.3 runtime	Apache-2.0	Downloaded archive, not redistributed here.
OpenCV 4.5.4	Apache-2.0	Ubuntu package.
onnx-community/mediapipe_selfie_segmentation	Apache-2.0	Community ONNX re-export of Google's MediaPipe selfie-segmentation model; the IR files in plugin/data/models/ are derived from it.
EasyPortrait (Kapitanov, Kvanchiani, Kirillova, 2023)	CC BY-SA 4.0	200 test images are fetched at run time for calibration and evaluation and are not redistributed.