Skip to content

ci: capture pytest exit codes and build test-status.json per label #536

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
CagriYonca wants to merge 1 commit into
base: master
Choose a base branch
from
Open

ci: capture pytest exit codes and build test-status.json per label #536

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 46 additions & 13 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,9 +201,18 @@ jobs:
update-currency-versions:
executor:
name: base
ruby_version: "4.0"
ruby_version: "3.4"

@GSVarsha GSVarsha Oct 21, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specific reason to change the ruby_version here?

@CagriYonca CagriYonca Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grpc installation was failing in Ruby 4.0

-- C level backtrace information -------------------------------------------
/usr/local/lib/libruby.so.4.0(rb_print_backtrace+0x14) [0x7503990738ef] /usr/src/ruby/vm_dump.c:1108
/usr/local/lib/libruby.so.4.0(rb_vm_bugreport) /usr/src/ruby/vm_dump.c:1453
/usr/local/lib/libruby.so.4.0(rb_bug_for_fatal_signal+0x106) [0x750398e26846] /usr/src/ruby/error.c:1131
/usr/local/lib/libruby.so.4.0(sigabrt+0x42) [0x750398fa0fe2] /usr/src/ruby/signal.c:962
/lib/x86_64-linux-gnu/libc.so.6(0x750398933d70) [0x750398933d70]
/lib/x86_64-linux-gnu/libc.so.6(0x75039898869c) [0x75039898869c]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x12) [0x750398933c42]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x22) [0x75039891c4ac]
/root/project/gemfiles/vendor/bundle/ruby/4.0.0/gems/grpc-1.84.0-x86_64-linux-gnu/src/ruby/lib/grpc/4.0/grpc_c.so(0x75037b90cc28) [0x75037b90cc28]
/root/project/gemfiles/vendor/bundle/ruby/4.0.0/gems/grpc-1.84.0-x86_64-linux-gnu/src/ruby/lib/grpc/4.0/grpc_c.so(0x75037b90cff5) [0x75037b90cff5]
/root/project/gemfiles/vendor/bundle/ruby/4.0.0/gems/grpc-1.84.0-x86_64-linux-gnu/src/ruby/lib/grpc/4.0/grpc_c.so(0x75037b6c3ee1) [0x75037b6c3ee1]
/root/project/gemfiles/vendor/bundle/ruby/4.0.0/gems/grpc-1.84.0-x86_64-linux-gnu/src/ruby/lib/grpc/4.0/grpc_c.so(0x75037b7bcdc5) [0x75037b7bcdc5]
/root/project/gemfiles/vendor/bundle/ruby/4.0.0/gems/grpc-1.84.0-x86_64-linux-gnu/src/ruby/lib/grpc/4.0/grpc_c.so(0x75037b7be4dd) [0x75037b7be4dd]
/root/project/gemfiles/vendor/bundle/ruby/4.0.0/gems/grpc-1.84.0-x86_64-linux-gnu/src/ruby/lib/grpc/4.0/grpc_c.so(0x75037b732590) [0x75037b732590]
/root/project/gemfiles/vendor/bundle/ruby/4.0.0/gems/grpc-1.84.0-x86_64-linux-gnu/src/ruby/lib/grpc/4.0/grpc_c.so(0x75037b73489b) [0x75037b73489b]
/root/project/gemfiles/vendor/bundle/ruby/4.0.0/gems/grpc-1.84.0-x86_64-linux-gnu/src/ruby/lib/grpc/4.0/grpc_c.so(0x75037b73490d) [0x75037b73490d]
/usr/local/lib/libruby.so.4.0(RB_FL_TEST_RAW+0x0) [0x750398e453eb] gc/default/default.c:2874
/usr/local/lib/libruby.so.4.0(run_final) gc/default/default.c:2878
/usr/local/lib/libruby.so.4.0(finalize_list) gc/default/default.c:2908
/usr/local/lib/libruby.so.4.0(finalize_deferred_heap_pages) gc/default/default.c:2932
/usr/local/lib/libruby.so.4.0(rb_gc_impl_shutdown_call_finalizer+0x263) [0x750398e55446] gc/default/default.c:3097
/usr/local/lib/libruby.so.4.0(rb_objspace_call_finalizer) /usr/src/ruby/gc.c:1739
/usr/local/lib/libruby.so.4.0(rb_ec_finalize+0x19) [0x750398e348c9] /usr/src/ruby/eval.c:177
/usr/local/lib/libruby.so.4.0(rb_ec_cleanup) /usr/src/ruby/eval.c:268
/usr/local/lib/libruby.so.4.0(ruby_run_node+0x95) [0x750398e34bb5] /usr/src/ruby/eval.c:332
/usr/local/bin/ruby(rb_main+0x21) [0x5bb6c5cc5110] ./main.c:42
/usr/local/bin/ruby(main) ./main.c:62
/lib/x86_64-linux-gnu/libc.so.6(0x75039891dca8) [0x75039891dca8]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85) [0x75039891dd65]
[0x5bb6c5cc5151]

It works after switching to Ruby 3.4

@arjun-rajappa arjun-rajappa Oct 21, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use some what like this

ruby-sensor/.circleci/config.yml

Line 429 in 86777de

exclude:

@CagriYonca CagriYonca Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In libraries_ruby_34_40, excluding grpc for Ruby 4.0 still allows it to run on other Ruby versions because it's a matrix job.

However, update-currency-versions runs in a single environment to generate version artifacts for all 18 gems. If we run it on Ruby 4.0 and skip grpc, installed_grpc.txt won't be generated and grpc will be missing from the currency report entirely.

Since Ruby 3.4 is the latest stable release, running this job on 3.4 allows all 18 gems (including grpc) to be installed, tested, and tracked for currency without skipping anything.

steps:
- checkout
- run:
name: Install jq and curl
command: |
apt-get update
apt-get install -y jq curl
- run:
name: Wait for all workflow test suites and retries to finish
command: |
./.circleci/wait_for_workflows.sh
- run:
name: Update RubyGems and Bundler
command: |
Expand All @@ -214,26 +223,29 @@ jobs:
name: Capture installed gem versions and test with latest versions
command: |
#!/usr/bin/env bash
set -e
mkdir -p bundle-list
mkdir -p test-results
mkdir -p _junit

run_bundle_list_and_test() {
local label="$1"
local gemfile="$2"
echo "=== Processing $label from $gemfile ==="

# Install dependencies
BUNDLE_GEMFILE="$gemfile" bundle check 2>/dev/null \
|| BUNDLE_GEMFILE="$gemfile" bundle install --quiet
if ! (BUNDLE_GEMFILE="$gemfile" bundle check 2>/dev/null || BUNDLE_GEMFILE="$gemfile" bundle install --quiet); then
echo " ✗ bundle install failed for $label"
echo "FAIL" > "test-results/${label}_status.txt"
return 0
fi

# Capture installed versions
BUNDLE_GEMFILE="$gemfile" bundle list > "bundle-list/installed_${label}.txt"
echo " Captured $(grep -c '^\s\+\*' bundle-list/installed_${label}.txt) gems"
BUNDLE_GEMFILE="$gemfile" bundle list > "bundle-list/installed_${label}.txt" || true
echo " Captured $(grep -c '^\s\+\*' "bundle-list/installed_${label}.txt" 2>/dev/null || echo 0) gems"

# Run tests to verify the versions work
# Run tests with APPRAISAL_INITIALIZED=1 so Rakefile runs only this gem's tests
echo " Running tests for $label..."
if BUNDLE_GEMFILE="$gemfile" bundle exec rake test 2>&1 | tee "test-results/${label}_test.log"; then
if APPRAISAL_INITIALIZED="1" BUNDLE_GEMFILE="$gemfile" bundle exec rake test 2>&1 | tee "test-results/${label}_test.log"; then
echo "PASS" > "test-results/${label}_status.txt"
echo " ✓ Tests passed for $label"
else
Expand All @@ -260,6 +272,27 @@ jobs:
run_bundle_list_and_test "graphql" "./gemfiles/graphql_20.gemfile"
run_bundle_list_and_test "roda" "./gemfiles/roda_30.gemfile"
run_bundle_list_and_test "net-http" "./gemfiles/net_http_01.gemfile"

# Assemble test-status.json in bundle-list directory
JSON="{"
FIRST=1
for STATUS_FILE in test-results/*_status.txt; do
[ -f "$STATUS_FILE" ] || continue
LABEL=$(basename "$STATUS_FILE" _status.txt)
STATUS=$(cat "$STATUS_FILE" | tr -d '[:space:]')
if [ "$STATUS" = "FAIL" ]; then
[ $FIRST -eq 0 ] && JSON="${JSON},"
JSON="${JSON} \"${LABEL}\": \"fail\""
FIRST=0
fi
done
JSON="${JSON} }"
echo "$JSON" > bundle-list/test-status.json

echo "=== test-status.json ==="
cat bundle-list/test-status.json
echo ""
ls -la bundle-list/
- store_artifacts:
path: bundle-list
- store_artifacts:
Expand Down Expand Up @@ -355,11 +388,11 @@ workflows:
only:
- master
requires:
- lint
- test_core-ruby-base-3.2
- test_core-ruby-base-3.3
- test_core-ruby-base-3.4
- test_core-ruby-base-4.0
- lint: terminal
- test_core-ruby-base-3.2: terminal
- test_core-ruby-base-3.3: terminal
- test_core-ruby-base-3.4: terminal
- test_core-ruby-base-4.0: terminal
libraries_ruby_32_33:
max_auto_reruns: 2
jobs:
Expand Down
61 changes: 61 additions & 0 deletions .circleci/wait_for_workflows.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
#!/usr/bin/env bash
set -e

if [[ -z "${CIRCLE_TOKEN:-}" ]]; then
echo "WARNING: CIRCLE_TOKEN is not set. Skipping waiting for sibling workflows."
exit 0
fi

PROJECT_SLUG='https://circleci.com/api/v2/project/gh/instana/ruby-sensor'
PIPELINE_SLUG='https://circleci.com/api/v2/pipeline'
WORKFLOW_SLUG='https://circleci.com/api/v2/workflow'

echo "=== Getting Pipeline ID for build #${CIRCLE_BUILD_NUM} ==="
PIPELINE_ID=$(curl -s \
-H "Circle-Token: ${CIRCLE_TOKEN}" \
"${PROJECT_SLUG}/job/${CIRCLE_BUILD_NUM}" \
| jq -rj '.pipeline.id // empty')

if [[ -z "${PIPELINE_ID}" ]]; then
echo "WARNING: Could not retrieve PIPELINE_ID from job info. Proceeding."
exit 0
fi

echo "Pipeline ID: ${PIPELINE_ID}"

echo "=== Querying workflows for pipeline ==="
WORKFLOW_INFO=$(curl -sf \
-H "Circle-Token: ${CIRCLE_TOKEN}" \
"${PIPELINE_SLUG}/${PIPELINE_ID}/workflow" \
| jq -r '.items[] | select(.name != "core" and .name != "report_coverage" and .name != "publish") | "\(.id)\t\(.name)"')

echo "Discovered test workflows to wait for:"
echo "${WORKFLOW_INFO}"
echo ""

# Loop and wait until every sibling workflow reaches a terminal state
while IFS=$'\t' read -r workflow_id workflow_name; do
[[ -z "${workflow_id}" ]] && continue

echo "Waiting for workflow '${workflow_name}' (${workflow_id}) to complete..."
MAX_ATTEMPTS=120 # up to 30 mins (120 * 15s)
ATTEMPT=0
while [[ ${ATTEMPT} -lt ${MAX_ATTEMPTS} ]]; do
STATUS=$(curl -s \
-H "Circle-Token: ${CIRCLE_TOKEN}" \
"${WORKFLOW_SLUG}/${workflow_id}" \
| jq -r '.status // "unknown"')

if [[ "${STATUS}" != "running" && "${STATUS}" != "on_hold" && "${STATUS}" != "unknown" ]]; then
echo " ✓ Workflow '${workflow_name}' finished with status: ${STATUS}"
break
fi

echo " ... '${workflow_name}' is currently: ${STATUS} (waiting 15s, attempt $((ATTEMPT+1))/${MAX_ATTEMPTS})"
sleep 15
ATTEMPT=$((ATTEMPT + 1))
done
done <<< "${WORKFLOW_INFO}"

echo ""
echo "=== All sibling test workflows have finished (including auto-retries) ==="
Loading