regression-detection

tracking performance of rustc-generated binaries over time

Detects when a deploy silently ships a noindex, drops a schema block, or rewrites your canonical. Baselines a page's SEO contract, diffs every change with Critical/Warning/Info severity, and points fixes to claude-seo sub-skills. A Claude Code skill. Open source.

Agentic eval framework for Laravel: golden datasets, LLM-as-judge, adversarial harness, regression detection

Continuous Evaluation Infrastructure for Production AI

Statistical regression detection and CI quality gates for AI agents

Fast, lightweight Rust benchmarking library. Dev mode: profile like Criterion. Prod mode: monitor with <10ns overhead. Features: async benchmarks, regression alerts, health metrics, p95/p99 latency tracking. Thread-safe, minimal deps. Benchmark any code block or function with one macro.

Detects semantic regressions across PRs that git's merge conflict detection misses

Early warning when a new GitHub Actions runner image starts breaking builds across many repos. Opt-in, anonymized, MIT.

JAX benchmarking, profiling and evaluation metrics for Flax NNX: a registry of pure-function metrics (regression, classification, calibration, uncertainty, forecasting, generative, image, text, audio, graph, fairness), XLA FLOP counting, roofline analysis, GPU and energy monitoring, regression detection, publication exports, W&B and MLflow.

A new package that helps developers integration-test AI and LLM applications by validating structured outputs. It takes a user's test scenario or prompt as input, sends it to an LLM, and uses pattern

Nextflow plugin for cross-run benchmarking. Records per-process metrics for every run into a local or shared history store, compares runs against their own history, and flags regressions with likely causes, over-provisioned resources, and retries. Includes a zero-dependency dashboard, an MCP server for AI assistants, and opt-in AI narration.

Linux/C++ CLI for benchmarking, perf-counter capture, and regression detection with configurable thresholds and statistical validation.

AI-powered benchmark regression analyzer with CLI, REST API, and interactive Dashboard. Compares cloud performance runs via OpenSearch + LLM, with automated regression detection.

This program enables users to microbenchmark their Java code to detect regression in performance.

Online evaluation, regression detection, and statistical A/B experimentation for LLM systems in production

Catch silent agent failures in CI. Pytest plugin with regression detection for LLM agents.

Verdict de performance argumenté pour les tests de charge — résultats JMeter et LoadRunner DevWeb, régression détectée par la statistique, PASS/WARN/FAIL dans la CI. Aucune clé API requise.

SQL service observability demo: SLA monitoring, regression detection, root-cause analysis, and capacity forecasting.

Did your coding agent change after an upgrade? Per-version behaviour metrics from your local Claude Code and Codex session logs, with significance tests.

Operator console for AI agent fleets. Runs, traces, cost budgets, regression detection, SLA scoring. Datadog-shaped layer between agent platforms and SRE/SecOps teams.