Telemetry payload v3: clear failures, complete data by MickLesk · Pull Request #175 · community-scripts/core

MickLesk · 2026-10-27T13:05:16Z

✍️ Description

New payload fields error_message (first msg_error of the failing step), error_step, error_phase (host/container) and failed_function; payload_version 3
The trace holds the failing command's output, or the failing step's log lines, instead of the whole-log tail; apt/dpkg/apk, download, build, pct-create and xtrace chatter is removed
IPv4 anonymisation matches four octets only, so version numbers survive
Explanations no longer say "its output says why"; they name the program (cargo exited with status 101, grep found no match, Arithmetic expression evaluated to 0)
A helper's return N is replaced by the call that failed; a silent() failure a helper passed up keeps its own command and output
The container loads api/exitcodes.func and api/errorlog.func, so the fallback exit-code table in error_handler.func is gone
Fix: the container prefetch never ran, because lxc-attach carried the host's COMMUNITY_SCRIPTS_CORE_DIR into the container
Fix: a detached install that finished after a console hangup was reported as 199 (race on the .done marker)
Fix: Ctrl-C during apt was reported as APT error 100; it is now 130 and counts as aborted
Fix: host checks that fail before the install record (storage, ID, template) are now reported
Fix: VM runs get one log and a capture, so VM failures carry output; VM cleanup reports even when the install record could not be sent
Fix: a bare return inside the EXIT trap re-entered the error handler in VM scripts and looped; the senders run with errexit and the ERR trap off
TELEMETRY_URL and TELEMETRY_PAYLOAD_VERSION reach the container; Incus no longer defaults DIAGNOSTICS to yes; a missing exit code is 199 on both hosts
Tests: new tests/telemetry-trace.sh, exit-code-hints and stale-silent-record extended; api/API.txt and ui/API.txt regenerated
Tested live on PVE 9.2 (x86) and Incus 7.5 (arm64): success, apt, plain command, missing command, helper return, release download, substitution, missing input, storage check, console hangup, Ctrl-C, VM error paths

🔗 Related Issue

Fixes #

✅ Prerequisites (X in brackets)

Self-review completed – Code follows project standards.
Tested thoroughly – Changes work as expected (Proxmox VE and/or Incus, as applicable).
No security risks – No hardcoded secrets, unnecessary privilege escalations, or permission issues.
API.txt regenerated – If a function signature or behavior changed, API.txt was updated to match.

🤖 AI Assistance (X in brackets)

If you used an AI tool (GitHub Copilot, Claude, ChatGPT, etc.) to write or generate any code in this PR, you must confirm compliance below.
Select exactly one option.

No AI used – Code was written without AI assistance.
AI was used – The output has been reviewed and corrected, and I take responsibility for it matching this repo's conventions.

🛠️ Type of Change (X in brackets)

🐞 Bug fix – Resolves an issue without breaking functionality.
✨ New feature – Adds new, non-breaking functionality.
💥 Breaking change – Alters existing behavior in a way that may require updates in every downstream script.
🔧 Refactoring / Code Cleanup – Improves readability or maintainability without changing functionality.
📝 Documentation update – Changes to README, docs/, CONTRIBUTING.md, or other docs.

1-3 told the reader to look at the output; 64 and 115 described what the engine and scripts no longer use them for.

error_message, error_step, error_phase and failed_function travel as their own fields. The trace holds the failing command's output or the failing step's log lines with package and download progress removed, and IPv4 anonymisation no longer eats version numbers. A helper's "return N" is replaced by the call that failed, and a silent() failure a helper passed up keeps its capture. Tool and add-on runs use the same payload.

The container now also loads the exit code table and the log filter, so error_handler drops its own copy of the table. Sourcing the files by path lets the error handler read a failing line back.

…-install race A detached install that wrote .done just before exiting was reported as a lost exit code. The recovery menu checks the failing step's output instead of the whole run. Incus no longer opts a container into telemetry when the host value is unset, and a missing exit code is 199 on both hosts.

get_active_logfile returned a new file per call, so VM failures carried no output. msg_* now log steps, silent() writes a capture, and the error handlers record command, line and function before reporting.

…ntainer prefetch The new explanation returned from inside case and loop blocks; with errtrace on and the engine sourced from a pipe, bash lost track of its function contexts and the container's failure was never written. lxc-attach also carried the host's engine directory into the container, which skipped the container's own prefetch.

A run that stopped at the storage, ID or template checks sent nothing, because a terminal status without an install record was dropped. Such a failure now sends the install record first; cancelled dialogs and update runs still send nothing. Exit 64 counts as a configuration problem.

The keystroke reaches the program in the foreground, which exits with its own code; bash then never runs the INT trap, and the run was reported as an APT failure.

github-actions · 2026-10-27T13:05:26Z

Try this branch

The engine and the scripts resolve independently, so a production script can
be run against the engine from this PR by setting one variable:

COMMUNITY_SCRIPTS_CORE_URL=https://raw.githubusercontent.com/community-scripts/core/feat/telemetry-v3 \
bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/ct/debian.sh)"

Swap ct/debian.sh for whatever exercises the change.

This PR touches both backends. Worth running on a Proxmox VE and an Incus host.

Run a script from a fork as well

curl -fsSL https://raw.githubusercontent.com/community-scripts/core/feat/telemetry-v3/tools/run.sh |
  bash -s -- https://raw.githubusercontent.com/YOU/DevScripts/your-branch ct/debian.sh \
             https://raw.githubusercontent.com/community-scripts/core/feat/telemetry-v3

Note that run.sh is reached through a pipe, so the script it starts inherits
an exhausted stdin. Whiptail is fine — it opens /dev/tty — but a plain read
would see EOF. The single-variable form above does not have that problem.

Useful flags while testing

dev_mode=net logs every engine fetch with status and duration, which is the
quickest way to confirm the branch is really being used. dev_mode=keep stops a
failed build from deleting the container along with the evidence.

json_escape and friends ended with a bare return, which inside an EXIT trap returns the status from before the trap. In a VM script, whose ERR trap is inherited into functions, that fired error_handler during the final report, and every nested call reported again. The senders now run with errexit and the caller's ERR trap off, and the VM cleanup reports a failure even when the install record could not be sent.

MickLesk added 8 commits October 27, 2026 13:07

Describe exit statuses 1-3, 64 and 115 without filler

ceaec05

1-3 told the reader to look at the output; 64 and 115 described what the engine and scripts no longer use them for.

Load the engine from the container's prefetched copy

8646149

The container now also loads the exit code table and the log filter, so error_handler drops its own copy of the table. Sourcing the files by path lets the error handler read a failing line back.

Give VM runs one log and a failure capture

827b067

get_active_logfile returned a new file per call, so VM failures carried no output. msg_* now log steps, silent() writes a capture, and the error handlers record command, line and function before reporting.

Count a Ctrl-C that apt or pip survived as an interrupt

3c51426

The keystroke reaches the program in the foreground, which exits with its own code; bash then never runs the INT trap, and the run was reported as an APT failure.

MickLesk added 3 commits October 27, 2026 14:17

Describe payload v3 and the capture in the api docs

db57b48

Regenerate ui/API.txt for the telemetry functions

e85e8af

MickLesk changed the title ~~Feat/telemetry v3~~ Telemetry payload v3: clear failures, complete data Oct 27, 2026

MickLesk added the automated pr Automated pull request label Oct 27, 2026

MickLesk marked this pull request as ready for review October 27, 2026 13:57

MickLesk merged commit af0fe77 into main Oct 27, 2026
5 checks passed

Navigation Menu

Repository navigation

Telemetry payload v3: clear failures, complete data - #175

Telemetry payload v3: clear failures, complete data#175
MickLesk merged 11 commits into
mainfrom
feat/telemetry-v3

MickLesk commented Oct 27, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 27, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

MickLesk commented Oct 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✍️ Description

🔗 Related Issue

✅ Prerequisites (X in brackets)

🤖 AI Assistance (X in brackets)

🛠️ Type of Change (X in brackets)

Uh oh!

github-actions Bot commented Oct 27, 2026

Try this branch

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

MickLesk commented Oct 27, 2026 •

edited

Loading