test: fix smoke test failures on xen/vmware by weizhouapache · Pull Request #13628 · apache/cloudstack

weizhouapache · 2026-07-16T11:34:21Z

Description

This PR fixes some smoke test failures on vmware or xenserver/xcpng

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

weizhouapache · 2026-07-16T11:34:36Z

@blueorangutan package

blueorangutan · 2026-07-16T11:36:05Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

Copilot

Pull request overview

This PR targets integration smoke test stability across hypervisors (notably XenServer/XCP-ng and VMware) by skipping KVM-specific checks/tests when not running on KVM, and by adjusting a few timing/host prerequisite assumptions that can cause intermittent failures.

Changes:

Skip KVM-only paths in smoke tests when the detected hypervisor is not KVM.
Add additional host-tool prerequisite checks for a VPC source NAT IP update test.
Add delays around affinity-group migration steps and make the events-resource test more tolerant around volume attach/detach operations.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
test/integration/smoke/test_nonstrict_affinity_group.py	Adds waits before migration steps in non-strict affinity group tests.
test/integration/smoke/test_network_extension_namespace.py	Skips KVM prerequisite checks on non-KVM hypervisors; expands required host tools for one test.
test/integration/smoke/test_kms_lifecycle.py	Skips a VM root-disk encryption test on non-KVM hypervisors.
test/integration/smoke/test_events_resource.py	Wraps volume attach/detach operations to avoid hard failures in some environments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

codecov · 2026-07-16T11:43:10Z

Codecov Report

❌ Patch coverage is 20.00000% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 19.73%. Comparing base (6764ca5) to head (2addfa1).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...tack/storage/motion/AncientDataMotionStrategy.java	0.00%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #13628      +/-   ##
============================================
- Coverage     19.73%   19.73%   -0.01%     
+ Complexity    19959    19954       -5     
============================================
  Files          6371     6371              
  Lines        575784   575788       +4     
  Branches      70478    70480       +2     
============================================
- Hits         113653   113625      -28     
- Misses       449778   449809      +31     
- Partials      12353    12354       +1

Flag	Coverage Δ
uitests	`3.41% <ø> (ø)`
unittests	`21.01% <20.00%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

blueorangutan · 2026-07-16T12:24:11Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18593

weizhouapache · 2026-07-16T12:35:32Z

@blueorangutan test ubuntu26 xcpng82 keepEnv

blueorangutan · 2026-07-16T12:36:03Z

@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu26 mgmt + xcpng82) has been kicked to run smoke tests

weizhouapache · 2026-07-16T12:37:19Z

@blueorangutan test ubuntu26 vmware-80u3e keepEnv

blueorangutan · 2026-07-16T12:38:05Z

@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu26 mgmt + vmware-80u3e) has been kicked to run smoke tests

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

weizhouapache · 2026-07-17T11:34:58Z

@blueorangutan package

blueorangutan · 2026-07-17T11:36:04Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-07-17T12:24:46Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18602

weizhouapache · 2026-07-17T12:45:07Z

@blueorangutan test ubuntu26 xcpng82

blueorangutan · 2026-07-17T12:46:03Z

@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu26 mgmt + xcpng82) has been kicked to run smoke tests

weizhouapache · 2026-07-17T12:46:57Z

@blueorangutan test ubuntu26 vmware-80u3e

blueorangutan · 2026-07-17T12:48:04Z

@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu26 mgmt + vmware-80u3e) has been kicked to run smoke tests

blueorangutan · 2026-07-17T13:46:10Z

[SF] Trillian Build Failed (tid-7)

weizhouapache · 2026-07-17T15:36:37Z

@blueorangutan test ubuntu26 vmware-80u3

blueorangutan · 2026-07-17T15:38:03Z

@weizhouapache a [SL] Trillian-Jenkins test job (ubuntu26 mgmt + vmware-80u3) has been kicked to run smoke tests

blueorangutan · 2026-07-17T23:16:50Z

[SF] Trillian Build Failed (tid-9)

blueorangutan · 2026-08-25T20:52:43Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18987

blueorangutan · 2026-08-26T05:01:04Z

[SF] Trillian test result (tid-16820)
Environment: vmware-80u3 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 66723 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr13628-t16820-vmware-80u3.zip
Smoke tests completed. 142 look OK, 3 have errors, 11 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_deploy_more_vms_than_limit_allows	`Error`	178.69	test_deploy_vms_in_parallel.py
test_08_upgrade_kubernetes_ha_cluster	`Failure`	828.38	test_kubernetes_clusters.py
test_05_list_volumes_isrecursive	`Failure`	0.05	test_list_volumes.py
test_07_list_volumes_listall	`Failure`	0.05	test_list_volumes.py
all_test_volumes	`Skipped`	---	test_volumes.py
all_test_vpc_conserve_mode	`Skipped`	---	test_vpc_conserve_mode.py
all_test_vpc_firewall_rules	`Skipped`	---	test_vpc_firewall_rules.py
all_test_vpc_ipv6	`Skipped`	---	test_vpc_ipv6.py
all_test_vpc_redundant	`Skipped`	---	test_vpc_redundant.py
all_test_vpc_router_nics	`Skipped`	---	test_vpc_router_nics.py
all_test_vpc_vpn	`Skipped`	---	test_vpc_vpn.py
all_test_webhook_delivery	`Skipped`	---	test_webhook_delivery.py
all_test_webhook_lifecycle	`Skipped`	---	test_webhook_lifecycle.py
all_test_host_maintenance	`Skipped`	---	test_host_maintenance.py
all_test_hostha_kvm	`Skipped`	---	test_hostha_kvm.py

…e.py

…tween pools

…l.py

weizhouapache · 2026-08-28T08:26:33Z

@blueorangutan package

blueorangutan · 2026-08-28T08:28:05Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

+            except Exception as e:
+                retries_left -= 1
+                if "accounts to cleanup" not in str(e):
+                    raise Exception("Warning: Exception during cleanup : %s" % e)


+            if (Hypervisor.HypervisorType.XenServer.equals(((VolumeInfo) srcData).getHypervisorType())) {
+                return false;
+            }
+
            if (destData instanceof VolumeInfo) {
+                if (Hypervisor.HypervisorType.XenServer.equals(((VolumeInfo) destData).getHypervisorType())) {
+                    return false;
+                }


blueorangutan · 2026-08-28T09:14:56Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19010

weizhouapache · 2026-08-28T09:17:05Z

@blueorangutan test matrix

blueorangutan · 2026-08-28T09:18:04Z

@weizhouapache a [SL] Trillian-Jenkins matrix job (EL8 mgmt + EL8 KVM, Ubuntu22 mgmt + Ubuntu22 KVM, EL8 mgmt + VMware 7.0u3, EL9 mgmt + XCP-ng 8.2 ) has been kicked to run smoke tests

blueorangutan · 2026-08-29T02:19:30Z

[SF] Trillian test result (tid-16837)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 57510 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr13628-t16837-kvm-ol8.zip
Smoke tests completed. 156 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

blueorangutan · 2026-08-29T02:56:22Z

[SF] Trillian test result (tid-16838)
Environment: kvm-ubuntu22 (x2), zone: Advanced Networking with Mgmt server u22
Total time taken: 60561 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr13628-t16838-kvm-ubuntu22.zip
Smoke tests completed. 156 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

blueorangutan · 2026-08-29T05:27:42Z

[SF] Trillian test result (tid-16839)
Environment: vmware-70u3 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 68381 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr13628-t16839-vmware-70u3.zip
Smoke tests completed. 155 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_deployVMInSharedNetwork	`Error`	392.19	test_network.py

blueorangutan · 2026-08-29T07:10:09Z

[SF] Trillian test result (tid-16840)
Environment: xcpng82 (x2), zone: Advanced Networking with Mgmt server ol9
Total time taken: 75729 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr13628-t16840-xcpng82.zip
Smoke tests completed. 156 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

weizhouapache · 2026-08-29T07:54:26Z

@blueorangutan test ol8 vmware-80u3

blueorangutan · 2026-08-29T07:56:04Z

@weizhouapache a [SL] Trillian-Jenkins test job (ol8 mgmt + vmware-80u3) has been kicked to run smoke tests

Copilot AI lite review requested due to automatic review settings July 16, 2026 11:34

boring-cyborg Bot added component:integration-test Python Warning... Python code Ahead! labels Jul 16, 2026

weizhouapache added this to the 4.23.0 milestone Jul 16, 2026

Copilot started reviewing on behalf of weizhouapache July 16, 2026 11:34 View session

weizhouapache removed this from the 4.23.0 milestone Jul 16, 2026

weizhouapache added this to the 4.23.0 milestone Jul 16, 2026

Copilot AI reviewed Jul 16, 2026

View reviewed changes

Comment thread test/integration/smoke/test_nonstrict_affinity_group.py

Comment thread test/integration/smoke/test_nonstrict_affinity_group.py

Comment thread test/integration/smoke/test_events_resource.py Outdated

Copilot AI review requested due to automatic review settings July 17, 2026 09:41

Copilot started reviewing on behalf of weizhouapache July 17, 2026 09:42 View session

Copilot AI reviewed Jul 17, 2026

View reviewed changes

Comment thread test/integration/smoke/test_network_extension_namespace.py

Comment thread test/integration/smoke/test_kms_lifecycle.py

Comment thread test/integration/smoke/test_events_resource.py

weizhouapache mentioned this pull request Aug 26, 2026

Smoke tests results of 4.23.0.0 RC #13978

Closed

weizhouapache added 10 commits August 28, 2026 10:25

test: fix smoke test failures on xen/vmware

5e7950a

test: sleep 10 seconds before attaching volume in test_events_resourc…

04508f5

…e.py

test: add more time.sleep(10)

3800290

test: add more time.sleep(10) in test_vm_strict_host_tags.py

48bdf4d

test: fix test_usage.py

e2c3e26

xenserver/xcpng: do not bypass secondary storage when copy volumes be…

961bff3

…tween pools

test: move time.sleep(10) in test_host_maintenance.py

972fe0f

VMware: fix import DC

27fe416

test: retry domain deletion in tearDown of test_deploy_vms_in_paralle…

d6abf0d

…l.py

test: fix tests failures caused by others in drs and host maintenance

2addfa1

Copilot AI review requested due to automatic review settings August 28, 2026 08:26

weizhouapache force-pushed the 4.23-fix-smoke-test-failures branch from 440ca34 to 2addfa1 Compare August 28, 2026 08:26

boring-cyborg Bot added the component:vmware label Aug 28, 2026

Copilot started reviewing on behalf of weizhouapache August 28, 2026 08:30 View session

Copilot AI reviewed Aug 28, 2026

View reviewed changes

weizhouapache marked this pull request as ready for review August 29, 2026 07:48

Uh oh!

Conversation

weizhouapache commented Jul 16, 2026

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

weizhouapache commented Jul 16, 2026

Uh oh!

blueorangutan commented Jul 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blueorangutan commented Jul 16, 2026

Uh oh!

weizhouapache commented Jul 16, 2026

Uh oh!

blueorangutan commented Jul 16, 2026

Uh oh!

weizhouapache commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blueorangutan commented Jul 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

weizhouapache commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

weizhouapache commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

weizhouapache commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

weizhouapache commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

blueorangutan commented Jul 17, 2026

Uh oh!

blueorangutan commented Aug 25, 2026

Uh oh!

blueorangutan commented Aug 26, 2026

Uh oh!

weizhouapache commented Aug 28, 2026

Uh oh!

blueorangutan commented Aug 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

codecov Bot commented Jul 16, 2026 •

edited

Loading

weizhouapache commented Jul 16, 2026 •

edited

Loading