00:00:00.001 Started by upstream project "nightly-trigger" build number 3086 00:00:00.001 originally caused by: 00:00:00.001 Started by timer 00:00:00.007 Checking out git https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool into /var/jenkins_home/workspace/autotest-spdk-master-vs-dpdk-v23.11_script/33b20b30f0a51e6b52980845e0f6aa336787973ad45e341fbbf98d1b65b265d4 to read jbp/jenkins/jjb-config/jobs/autotest-upstream/autotest.groovy 00:00:00.008 The recommended git tool is: git 00:00:00.008 using credential 00000000-0000-0000-0000-000000000002 00:00:00.010 > git rev-parse --resolve-git-dir /var/jenkins_home/workspace/autotest-spdk-master-vs-dpdk-v23.11_script/33b20b30f0a51e6b52980845e0f6aa336787973ad45e341fbbf98d1b65b265d4/jbp/.git # timeout=10 00:00:00.023 Fetching changes from the remote Git repository 00:00:00.025 > git config remote.origin.url https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool # timeout=10 00:00:00.037 Using shallow fetch with depth 1 00:00:00.037 Fetching upstream changes from https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool 00:00:00.037 > git --version # timeout=10 00:00:00.047 > git --version # 'git version 2.39.2' 00:00:00.047 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:00.048 Setting http proxy: proxy-dmz.intel.com:911 00:00:00.048 > git fetch --tags --force --progress --depth=1 -- https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool refs/heads/master # timeout=5 00:00:02.289 > git rev-parse origin/FETCH_HEAD^{commit} # timeout=10 00:00:02.304 > git rev-parse FETCH_HEAD^{commit} # timeout=10 00:00:02.317 Checking out Revision 10da8f6d99838e411e4e94523ded0bfebf3e7100 (FETCH_HEAD) 00:00:02.317 > git config core.sparsecheckout # timeout=10 00:00:02.329 > git read-tree -mu HEAD # timeout=10 00:00:02.345 > git checkout -f 10da8f6d99838e411e4e94523ded0bfebf3e7100 # timeout=5 00:00:02.366 Commit message: "scripts/create_git_mirror: Update path to xnvme submodule" 00:00:02.366 > git rev-list --no-walk 10da8f6d99838e411e4e94523ded0bfebf3e7100 # timeout=10 00:00:02.935 [Pipeline] Start of Pipeline 00:00:02.950 [Pipeline] library 00:00:02.952 Loading library shm_lib@master 00:00:02.952 Library shm_lib@master is cached. Copying from home. 00:00:02.978 [Pipeline] node 00:00:03.027 Running on ME3 in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11 00:00:03.029 [Pipeline] { 00:00:03.051 [Pipeline] cleanWs 00:00:03.061 [WS-CLEANUP] Deleting project workspace... 00:00:03.061 [WS-CLEANUP] Deferred wipeout is used... 00:00:03.066 [WS-CLEANUP] done 00:00:03.084 [Pipeline] stage 00:00:03.090 [Pipeline] { (Prologue) 00:00:03.199 [Pipeline] withCredentials 00:00:03.232 > git --version # timeout=10 00:00:03.257 > git --version # 'git version 2.39.2' 00:00:03.272 Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD or $GIT_ASKPASS 00:00:03.274 [Pipeline] { 00:00:03.282 [Pipeline] retry 00:00:03.284 [Pipeline] { 00:00:03.483 [Pipeline] sh 00:00:03.758 + git ls-remote https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool refs/heads/master 00:00:05.671 [Pipeline] } 00:00:05.690 [Pipeline] // retry 00:00:05.694 [Pipeline] } 00:00:05.709 [Pipeline] // withCredentials 00:00:05.715 [Pipeline] nodesByLabel 00:00:05.717 Found a total of 1 nodes with the 'sorcerer' label 00:00:05.723 [Pipeline] httpRequest 00:00:05.726 HttpMethod: GET 00:00:05.727 URL: http://10.211.164.101/packages/jbp_10da8f6d99838e411e4e94523ded0bfebf3e7100.tar.gz 00:00:05.727 Sending request to url: http://10.211.164.101/packages/jbp_10da8f6d99838e411e4e94523ded0bfebf3e7100.tar.gz 00:00:05.728 Response Code: HTTP/1.1 200 OK 00:00:05.728 Success: Status code 200 is in the accepted range: 200,404 00:00:05.728 Saving response body to /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/jbp_10da8f6d99838e411e4e94523ded0bfebf3e7100.tar.gz 00:00:05.866 [Pipeline] sh 00:00:06.141 + tar --no-same-owner -xf jbp_10da8f6d99838e411e4e94523ded0bfebf3e7100.tar.gz 00:00:06.160 [Pipeline] withCredentials 00:00:06.170 > git --version # timeout=10 00:00:06.181 > git --version # 'git version 2.39.2' 00:00:06.197 Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD or $GIT_ASKPASS 00:00:06.199 [Pipeline] { 00:00:06.207 [Pipeline] retry 00:00:06.209 [Pipeline] { 00:00:06.224 [Pipeline] sh 00:00:06.502 + git ls-remote https://review.spdk.io/gerrit/a/spdk/spdk refs/heads/master 00:00:10.701 [Pipeline] } 00:00:10.730 [Pipeline] // retry 00:00:10.736 [Pipeline] } 00:00:10.759 [Pipeline] // withCredentials 00:00:10.771 [Pipeline] httpRequest 00:00:10.775 HttpMethod: GET 00:00:10.776 URL: http://10.211.164.101/packages/spdk_4506c0c368f63ba9b9b013ecff216cef6ee8d0a4.tar.gz 00:00:10.777 Sending request to url: http://10.211.164.101/packages/spdk_4506c0c368f63ba9b9b013ecff216cef6ee8d0a4.tar.gz 00:00:10.788 Response Code: HTTP/1.1 200 OK 00:00:10.789 Success: Status code 200 is in the accepted range: 200,404 00:00:10.789 Saving response body to /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk_4506c0c368f63ba9b9b013ecff216cef6ee8d0a4.tar.gz 00:00:35.954 [Pipeline] sh 00:00:36.229 + tar --no-same-owner -xf spdk_4506c0c368f63ba9b9b013ecff216cef6ee8d0a4.tar.gz 00:00:38.137 [Pipeline] sh 00:00:38.415 + git -C spdk log --oneline -n5 00:00:38.415 4506c0c36 test/common: Enable inherit_errexit 00:00:38.415 b24df7cfa test: Drop superfluous calls to print_backtrace() 00:00:38.415 7b52e4c17 test/scheduler: Meassure utime of $spdk_pid threads as a fallback 00:00:38.415 1dc065205 test/scheduler: Calculate median of the cpu load samples 00:00:38.415 b22f1b34d test/scheduler: Enhance lookup of the $old_cgroup in move_proc() 00:00:38.432 [Pipeline] sh 00:00:38.714 + git -C spdk show -s --format=%s 4506c0c368f63ba9b9b013ecff216cef6ee8d0a4 00:00:38.732 [Pipeline] setCustomBuildProperty 00:00:38.741 [Pipeline] setCustomBuildProperty 00:00:38.766 [Pipeline] withCredentials 00:00:38.776 > git --version # timeout=10 00:00:38.791 > git --version # 'git version 2.39.2' 00:00:38.807 Masking supported pattern matches of $GIT_PASSWORD or $GIT_ASKPASS 00:00:38.809 [Pipeline] { 00:00:38.819 [Pipeline] retry 00:00:38.820 [Pipeline] { 00:00:38.838 [Pipeline] sh 00:00:39.121 + git ls-remote http://dpdk.org/git/dpdk-stable v23.11 00:00:39.394 [Pipeline] } 00:00:39.413 [Pipeline] // retry 00:00:39.417 [Pipeline] } 00:00:39.435 [Pipeline] // withCredentials 00:00:39.449 [Pipeline] httpRequest 00:00:39.454 HttpMethod: GET 00:00:39.454 URL: http://10.211.164.101/packages/dpdk_d15625009dced269fcec27fc81dd74fd58d54cdb.tar.gz 00:00:39.455 Sending request to url: http://10.211.164.101/packages/dpdk_d15625009dced269fcec27fc81dd74fd58d54cdb.tar.gz 00:00:39.456 Response Code: HTTP/1.1 200 OK 00:00:39.457 Success: Status code 200 is in the accepted range: 200,404 00:00:39.457 Saving response body to /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dpdk_d15625009dced269fcec27fc81dd74fd58d54cdb.tar.gz 00:00:42.785 [Pipeline] sh 00:00:43.065 + tar --no-same-owner -xf dpdk_d15625009dced269fcec27fc81dd74fd58d54cdb.tar.gz 00:00:44.445 [Pipeline] sh 00:00:44.721 + git -C dpdk log --oneline -n5 00:00:44.721 eeb0605f11 version: 23.11.0 00:00:44.721 238778122a doc: update release notes for 23.11 00:00:44.721 46aa6b3cfc doc: fix description of RSS features 00:00:44.721 dd88f51a57 devtools: forbid DPDK API in cnxk base driver 00:00:44.721 7e421ae345 devtools: support skipping forbid rule check 00:00:44.733 [Pipeline] catchError 00:00:44.734 [Pipeline] { 00:00:44.753 [Pipeline] readJSON 00:00:44.766 [Pipeline] } 00:00:44.795 [Pipeline] // catchError 00:00:44.800 [Pipeline] } 00:00:44.831 [Pipeline] // stage 00:00:44.851 [Pipeline] catchError 00:00:44.853 [Pipeline] { 00:00:44.878 [Pipeline] stage 00:00:44.880 [Pipeline] { (Tests) 00:00:45.090 [Pipeline] parallel 00:00:45.211 [Pipeline] { (Branch: unittest-vg-autotest) 00:00:45.212 [Pipeline] { (Branch: valgrind-vg-autotest) 00:00:45.213 [Pipeline] { (Branch: autobuild-vg-autotest) 00:00:45.215 [Pipeline] { (Branch: scanbuild-vg-autotest) 00:00:45.216 [Pipeline] { (Branch: lvol-vg-autotest) 00:00:45.217 [Pipeline] { (Branch: vhost-initiator-vg-autotest) 00:00:45.218 [Pipeline] { (Branch: nvmf-tcp-vg-autotest) 00:00:45.220 [Pipeline] { (Branch: nvmf-tcp-uring-vg-autotest) 00:00:45.221 [Pipeline] { (Branch: nvme-vg-autotest) 00:00:45.222 [Pipeline] { (Branch: centos7-vg-autotest) 00:00:45.223 [Pipeline] { (Branch: rocky9-vg-autotest) 00:00:45.225 [Pipeline] { (Branch: ubuntu20-vg-autotest) 00:00:45.226 [Pipeline] { (Branch: ubuntu22-vg-autotest) 00:00:45.227 [Pipeline] { (Branch: iscsi-vg-autotest) 00:00:45.228 [Pipeline] { (Branch: iscsi-uring-vg-autotest) 00:00:45.230 [Pipeline] { (Branch: zns-vg-autotest) 00:00:45.231 [Pipeline] { (Branch: clang-vg-autotest) 00:00:45.232 [Pipeline] { (Branch: nvme-cmb-pmr-vg-autotest) 00:00:45.233 [Pipeline] { (Branch: BlobFS-phy-autotest) 00:00:45.235 [Pipeline] { (Branch: crypto-phy-autotest) 00:00:45.236 [Pipeline] { (Branch: vhost-phy-autotest) 00:00:45.238 [Pipeline] { (Branch: nvmf-phy-autotest) 00:00:45.239 [Pipeline] { (Branch: nvmf-tcp-phy-autotest) 00:00:45.240 [Pipeline] { (Branch: nvme-phy-autotest) 00:00:45.241 [Pipeline] { (Branch: vfio-user-phy-autotest) 00:00:45.243 [Pipeline] { (Branch: short-fuzz-phy-autotest) 00:00:45.244 [Pipeline] { (Branch: dsa-phy-autotest) 00:00:45.245 [Pipeline] { (Branch: vmd-phy-autotest) 00:00:45.288 [Pipeline] retry 00:00:45.291 [Pipeline] { 00:00:45.297 [Pipeline] retry 00:00:45.299 [Pipeline] { 00:00:45.306 [Pipeline] retry 00:00:45.308 [Pipeline] { 00:00:45.315 [Pipeline] retry 00:00:45.317 [Pipeline] { 00:00:45.323 [Pipeline] retry 00:00:45.326 [Pipeline] { 00:00:45.332 [Pipeline] retry 00:00:45.334 [Pipeline] { 00:00:45.340 [Pipeline] retry 00:00:45.342 [Pipeline] { 00:00:45.349 [Pipeline] retry 00:00:45.351 [Pipeline] { 00:00:45.357 [Pipeline] retry 00:00:45.359 [Pipeline] { 00:00:45.365 [Pipeline] retry 00:00:45.367 [Pipeline] { 00:00:45.373 [Pipeline] retry 00:00:45.374 [Pipeline] { 00:00:45.379 [Pipeline] retry 00:00:45.380 [Pipeline] { 00:00:45.384 [Pipeline] retry 00:00:45.386 [Pipeline] { 00:00:45.390 [Pipeline] retry 00:00:45.391 [Pipeline] { 00:00:45.395 [Pipeline] retry 00:00:45.397 [Pipeline] { 00:00:45.401 [Pipeline] retry 00:00:45.402 [Pipeline] { 00:00:45.408 [Pipeline] retry 00:00:45.427 [Pipeline] { 00:00:45.456 [Pipeline] retry 00:00:45.466 [Pipeline] { 00:00:45.475 [Pipeline] retry 00:00:45.476 [Pipeline] { 00:00:45.479 [Pipeline] retry 00:00:45.480 [Pipeline] { 00:00:45.483 [Pipeline] retry 00:00:45.484 [Pipeline] { 00:00:45.487 [Pipeline] retry 00:00:45.488 [Pipeline] { 00:00:45.491 [Pipeline] retry 00:00:45.492 [Pipeline] { 00:00:45.495 [Pipeline] retry 00:00:45.497 [Pipeline] { 00:00:45.500 [Pipeline] retry 00:00:45.501 [Pipeline] { 00:00:45.504 [Pipeline] retry 00:00:45.505 [Pipeline] { 00:00:45.508 [Pipeline] retry 00:00:45.509 [Pipeline] { 00:00:45.512 [Pipeline] retry 00:00:45.513 [Pipeline] { 00:00:45.533 [Pipeline] build 00:00:45.534 Scheduling project: unittest-vg-autotest 00:00:45.542 [Pipeline] build 00:00:45.544 Scheduling project: valgrind-vg-autotest 00:00:45.550 [Pipeline] build 00:00:45.552 Scheduling project: autobuild-vg-autotest 00:00:45.558 [Pipeline] build 00:00:45.560 Scheduling project: scanbuild-vg-autotest 00:00:45.565 [Pipeline] build 00:00:45.567 Scheduling project: lvol-vg-autotest 00:00:45.573 [Pipeline] build 00:00:45.575 Scheduling project: vhost-initiator-vg-autotest 00:00:45.580 [Pipeline] build 00:00:45.582 Scheduling project: nvmf-tcp-vg-autotest 00:00:45.588 [Pipeline] build 00:00:45.590 Scheduling project: nvmf-tcp-uring-vg-autotest 00:00:45.596 [Pipeline] build 00:00:45.598 Scheduling project: nvme-vg-autotest 00:00:45.603 [Pipeline] build 00:00:45.605 Scheduling project: centos7-vg-autotest 00:00:45.611 [Pipeline] build 00:00:45.613 Scheduling project: rocky9-vg-autotest 00:00:45.618 [Pipeline] build 00:00:45.620 Scheduling project: ubuntu20-vg-autotest 00:00:45.625 [Pipeline] build 00:00:45.627 Scheduling project: ubuntu22-vg-autotest 00:00:45.632 [Pipeline] build 00:00:45.634 Scheduling project: iscsi-vg-autotest 00:00:45.639 [Pipeline] build 00:00:45.640 Scheduling project: iscsi-uring-vg-autotest 00:00:45.645 [Pipeline] build 00:00:45.647 Scheduling project: zns-vg-autotest 00:00:45.652 [Pipeline] build 00:00:45.654 Scheduling project: clang-vg-autotest 00:00:45.659 [Pipeline] build 00:00:45.661 Scheduling project: nvme-cmb-pmr-vg-autotest 00:00:45.666 [Pipeline] build 00:00:45.668 Scheduling project: BlobFS-phy-autotest 00:00:45.673 [Pipeline] build 00:00:45.675 Scheduling project: crypto-phy-autotest 00:00:45.680 [Pipeline] build 00:00:45.682 Scheduling project: vhost-phy-autotest 00:00:45.687 [Pipeline] build 00:00:45.689 Scheduling project: nvmf-phy-autotest 00:00:45.694 [Pipeline] build 00:00:45.696 Scheduling project: nvmf-tcp-phy-autotest 00:00:45.701 [Pipeline] build 00:00:45.702 Scheduling project: nvme-phy-autotest 00:00:45.707 [Pipeline] build 00:00:45.709 Scheduling project: vfio-user-phy-autotest 00:00:45.714 [Pipeline] build 00:00:45.716 Scheduling project: short-fuzz-phy-autotest 00:00:45.721 [Pipeline] build 00:00:45.723 Scheduling project: dsa-phy-autotest 00:00:45.727 [Pipeline] build 00:00:45.729 Scheduling project: vmd-phy-autotest 00:00:55.179 Starting building: dsa-phy-autotest #4853 00:00:55.182 Starting building: vhost-phy-autotest #26178 00:00:55.185 Starting building: short-fuzz-phy-autotest #26150 00:00:55.187 Starting building: vfio-user-phy-autotest #26161 00:00:55.190 Starting building: vmd-phy-autotest #2614 00:00:55.192 Starting building: crypto-phy-autotest #26153 00:00:55.196 Starting building: BlobFS-phy-autotest #26159 00:00:55.198 Starting building: vhost-initiator-vg-autotest #77453 00:00:55.201 Starting building: rocky9-vg-autotest #16043 00:00:55.204 Starting building: centos7-vg-autotest #73612 00:00:55.207 Starting building: nvmf-phy-autotest #72397 00:00:55.210 Starting building: nvme-vg-autotest #77433 00:00:55.215 Starting building: iscsi-uring-vg-autotest #77338 00:00:55.218 Starting building: zns-vg-autotest #76539 00:00:55.221 Starting building: lvol-vg-autotest #77382 00:00:55.224 Starting building: nvmf-tcp-vg-autotest #77424 00:00:55.229 Starting building: nvmf-tcp-uring-vg-autotest #77358 00:00:55.232 Starting building: valgrind-vg-autotest #49187 00:00:55.236 Starting building: nvmf-tcp-phy-autotest #72284 00:00:55.238 Starting building: ubuntu22-vg-autotest #37766 00:00:55.241 Starting building: nvme-phy-autotest #72325 00:00:55.244 Starting building: nvme-cmb-pmr-vg-autotest #78556 00:00:55.247 Starting building: ubuntu20-vg-autotest #73752 00:00:55.250 Starting building: unittest-vg-autotest #63370 00:00:55.252 Starting building: iscsi-vg-autotest #77471 00:00:55.256 Starting building: scanbuild-vg-autotest #63372 00:00:55.259 Starting building: clang-vg-autotest #57621 00:00:55.262 Starting building: autobuild-vg-autotest #63413 00:08:33.075 Build vmd-phy-autotest #2614 completed: SUCCESS 00:08:33.079 [Pipeline] } 00:08:33.123 [Pipeline] // retry 00:08:33.129 [Pipeline] } 00:08:35.014 Build short-fuzz-phy-autotest #26150 completed: SUCCESS 00:08:35.016 [Pipeline] } 00:08:35.044 [Pipeline] // retry 00:08:35.050 [Pipeline] } 00:11:32.004 Build dsa-phy-autotest #4853 completed: SUCCESS 00:11:32.006 [Pipeline] } 00:11:32.049 [Pipeline] // retry 00:11:32.056 [Pipeline] } 00:18:35.798 Build vhost-initiator-vg-autotest #77453 completed: SUCCESS 00:18:35.800 [Pipeline] } 00:18:35.852 [Pipeline] // retry 00:18:35.858 [Pipeline] } 00:22:20.957 Build zns-vg-autotest #76539 completed: SUCCESS 00:22:20.959 [Pipeline] } 00:22:21.010 [Pipeline] // retry 00:22:21.017 [Pipeline] } 00:23:13.982 Build BlobFS-phy-autotest #26159 completed: SUCCESS 00:23:13.985 [Pipeline] } 00:23:14.027 [Pipeline] // retry 00:23:14.033 [Pipeline] } 00:23:27.269 Build vfio-user-phy-autotest #26161 completed: SUCCESS 00:23:27.271 [Pipeline] } 00:23:27.316 [Pipeline] // retry 00:23:27.323 [Pipeline] } 00:24:17.252 Build lvol-vg-autotest #77382 completed: SUCCESS 00:24:17.254 [Pipeline] } 00:24:17.300 [Pipeline] // retry 00:24:17.307 [Pipeline] } 00:25:02.281 Build nvme-phy-autotest #72325 completed: SUCCESS 00:25:02.284 [Pipeline] } 00:25:02.336 [Pipeline] // retry 00:25:02.343 [Pipeline] } 00:25:42.320 Build nvme-cmb-pmr-vg-autotest #78556 completed: SUCCESS 00:25:42.323 [Pipeline] } 00:25:42.369 [Pipeline] // retry 00:25:42.376 [Pipeline] } 00:25:57.100 Build nvme-vg-autotest #77433 completed: FAILURE 00:25:57.824 [Pipeline] echo 00:25:57.825 No retry patterns found. 00:25:57.826 [Pipeline] } 00:25:57.851 [Pipeline] // retry 00:25:57.860 [Pipeline] error 00:25:57.863 [Pipeline] } 00:25:57.866 Failed in branch nvme-vg-autotest 00:28:27.831 Build vhost-phy-autotest #26178 completed: SUCCESS 00:28:27.833 [Pipeline] } 00:28:27.879 [Pipeline] // retry 00:28:27.886 [Pipeline] } 00:30:06.073 Build scanbuild-vg-autotest #63372 completed: SUCCESS 00:30:06.074 [Pipeline] } 00:30:06.103 [Pipeline] // retry 00:30:06.109 [Pipeline] } 00:38:11.478 Build iscsi-uring-vg-autotest #77338 completed: SUCCESS 00:38:11.481 [Pipeline] } 00:38:11.528 [Pipeline] // retry 00:38:11.535 [Pipeline] } 00:39:55.582 Build autobuild-vg-autotest #63413 completed: SUCCESS 00:39:55.584 [Pipeline] } 00:39:55.622 [Pipeline] // retry 00:39:55.628 [Pipeline] } 00:39:56.514 Build unittest-vg-autotest #63370 completed: SUCCESS 00:39:56.517 [Pipeline] } 00:39:56.561 [Pipeline] // retry 00:39:56.570 [Pipeline] } 00:40:00.297 Build nvmf-phy-autotest #72397 completed: SUCCESS 00:40:00.303 [Pipeline] } 00:40:00.350 [Pipeline] // retry 00:40:00.357 [Pipeline] } 00:40:49.406 Build nvmf-tcp-uring-vg-autotest #77358 completed: SUCCESS 00:40:49.408 [Pipeline] } 00:40:49.460 [Pipeline] // retry 00:40:49.477 [Pipeline] } 00:41:28.830 Build centos7-vg-autotest #73612 completed: SUCCESS 00:41:28.836 [Pipeline] } 00:41:28.899 [Pipeline] // retry 00:41:28.906 [Pipeline] } 00:41:56.451 Build crypto-phy-autotest #26153 completed: ABORTED 00:41:58.468 [Pipeline] echo 00:41:58.469 No retry patterns found. 00:41:58.470 [Pipeline] } 00:41:58.499 [Pipeline] // retry 00:41:58.506 [Pipeline] error 00:41:58.512 [Pipeline] } 00:41:58.517 Failed in branch crypto-phy-autotest 00:42:13.060 Build nvmf-tcp-phy-autotest #72284 completed: ABORTED 00:42:14.945 [Pipeline] echo 00:42:14.946 No retry patterns found. 00:42:14.946 [Pipeline] } 00:42:14.983 [Pipeline] // retry 00:42:14.989 [Pipeline] error 00:42:14.993 [Pipeline] } 00:42:14.995 Failed in branch nvmf-tcp-phy-autotest 00:45:41.941 Build iscsi-vg-autotest #77471 completed: SUCCESS 00:45:41.943 [Pipeline] } 00:45:41.979 [Pipeline] // retry 00:45:41.984 [Pipeline] } 00:46:48.054 Build clang-vg-autotest #57621 completed: SUCCESS 00:46:48.056 [Pipeline] } 00:46:48.100 [Pipeline] // retry 00:46:48.104 [Pipeline] } 00:48:50.466 Build nvmf-tcp-vg-autotest #77424 completed: SUCCESS 00:48:50.469 [Pipeline] } 00:48:50.513 [Pipeline] // retry 00:48:50.520 [Pipeline] } 00:57:46.939 Build ubuntu20-vg-autotest #73752 completed: FAILURE 00:57:48.929 [Pipeline] echo 00:57:48.930 No retry patterns found. 00:57:48.931 [Pipeline] } 00:57:48.972 [Pipeline] // retry 00:57:48.979 [Pipeline] error 00:57:48.985 [Pipeline] } 00:57:48.991 Failed in branch ubuntu20-vg-autotest 00:58:23.192 Build valgrind-vg-autotest #49187 completed: SUCCESS 00:58:23.194 [Pipeline] } 00:58:23.230 [Pipeline] // retry 00:58:23.236 [Pipeline] } 01:01:34.268 Build rocky9-vg-autotest #16043 completed: SUCCESS 01:01:34.271 [Pipeline] } 01:01:34.323 [Pipeline] // retry 01:01:34.331 [Pipeline] } 01:10:25.964 Build ubuntu22-vg-autotest #37766 completed: SUCCESS 01:10:25.968 [Pipeline] } 01:10:26.018 [Pipeline] // retry 01:10:26.024 [Pipeline] } 01:10:26.202 [Pipeline] // parallel 01:10:26.210 [Pipeline] } 01:10:26.246 [Pipeline] // stage 01:10:26.258 [Pipeline] } 01:10:26.268 ERROR: Build nvme-vg-autotest #77433 failed 01:10:26.297 [Pipeline] // catchError 01:10:26.305 [Pipeline] stage 01:10:26.306 [Pipeline] { (Autorun Post and Coverage) 01:10:26.330 [Pipeline] setCustomBuildProperty 01:10:26.664 [Pipeline] dir 01:10:26.665 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vmd-phy-autotest_2614 01:10:26.666 [Pipeline] { 01:10:26.713 [Pipeline] copyArtifacts 01:10:26.899 Copied 12 artifacts from "vmd-phy-autotest" build number 2614 01:10:26.902 [Pipeline] writeFile 01:10:27.021 [Pipeline] } 01:10:27.103 [Pipeline] // dir 01:10:27.493 [Pipeline] dir 01:10:27.494 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/short-fuzz-phy-autotest_26150 01:10:27.495 [Pipeline] { 01:10:27.597 [Pipeline] copyArtifacts 01:10:27.690 Copied 10 artifacts from "short-fuzz-phy-autotest" build number 26150 01:10:27.693 [Pipeline] writeFile 01:10:27.874 [Pipeline] } 01:10:27.993 [Pipeline] // dir 01:10:28.286 [Pipeline] dir 01:10:28.287 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dsa-phy-autotest_4853 01:10:28.288 [Pipeline] { 01:10:28.325 [Pipeline] copyArtifacts 01:10:28.532 Copied 12 artifacts from "dsa-phy-autotest" build number 4853 01:10:28.537 [Pipeline] writeFile 01:10:28.649 [Pipeline] } 01:10:28.737 [Pipeline] // dir 01:10:29.041 [Pipeline] dir 01:10:29.041 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-initiator-vg-autotest_77453 01:10:29.042 [Pipeline] { 01:10:29.079 [Pipeline] copyArtifacts 01:10:29.309 Copied 15 artifacts from "vhost-initiator-vg-autotest" build number 77453 01:10:29.313 [Pipeline] writeFile 01:10:29.447 [Pipeline] } 01:10:29.543 [Pipeline] // dir 01:10:29.875 [Pipeline] dir 01:10:29.875 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/zns-vg-autotest_76539 01:10:29.876 [Pipeline] { 01:10:29.918 [Pipeline] copyArtifacts 01:10:30.117 Copied 12 artifacts from "zns-vg-autotest" build number 76539 01:10:30.121 [Pipeline] writeFile 01:10:30.263 [Pipeline] } 01:10:30.375 [Pipeline] // dir 01:10:30.675 [Pipeline] dir 01:10:30.675 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/BlobFS-phy-autotest_26159 01:10:30.677 [Pipeline] { 01:10:30.720 [Pipeline] copyArtifacts 01:10:31.050 Copied 28 artifacts from "BlobFS-phy-autotest" build number 26159 01:10:31.054 [Pipeline] writeFile 01:10:31.170 [Pipeline] } 01:10:31.261 [Pipeline] // dir 01:10:31.749 [Pipeline] dir 01:10:31.750 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vfio-user-phy-autotest_26161 01:10:31.751 [Pipeline] { 01:10:31.801 [Pipeline] copyArtifacts 01:10:32.017 Copied 12 artifacts from "vfio-user-phy-autotest" build number 26161 01:10:32.022 [Pipeline] writeFile 01:10:32.242 [Pipeline] } 01:10:32.386 [Pipeline] // dir 01:10:32.999 [Pipeline] dir 01:10:33.000 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/lvol-vg-autotest_77382 01:10:33.001 [Pipeline] { 01:10:33.049 [Pipeline] copyArtifacts 01:10:33.243 Copied 12 artifacts from "lvol-vg-autotest" build number 77382 01:10:33.247 [Pipeline] writeFile 01:10:33.502 [Pipeline] } 01:10:33.642 [Pipeline] // dir 01:10:34.218 [Pipeline] dir 01:10:34.218 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-phy-autotest_72325 01:10:34.219 [Pipeline] { 01:10:34.270 [Pipeline] copyArtifacts 01:10:34.527 Copied 12 artifacts from "nvme-phy-autotest" build number 72325 01:10:34.534 [Pipeline] writeFile 01:10:34.765 [Pipeline] } 01:10:34.882 [Pipeline] // dir 01:10:35.225 [Pipeline] dir 01:10:35.226 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-cmb-pmr-vg-autotest_78556 01:10:35.227 [Pipeline] { 01:10:35.269 [Pipeline] copyArtifacts 01:10:35.473 Copied 12 artifacts from "nvme-cmb-pmr-vg-autotest" build number 78556 01:10:35.478 [Pipeline] writeFile 01:10:35.673 [Pipeline] } 01:10:35.769 [Pipeline] // dir 01:10:36.224 [Pipeline] dir 01:10:36.225 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-vg-autotest_77433 01:10:36.226 [Pipeline] { 01:10:36.273 [Pipeline] copyArtifacts 01:10:36.506 Copied 12 artifacts from "nvme-vg-autotest" build number 77433 01:10:36.511 [Pipeline] writeFile 01:10:36.693 [Pipeline] } 01:10:36.826 [Pipeline] // dir 01:10:37.497 [Pipeline] dir 01:10:37.497 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-phy-autotest_26178 01:10:37.498 [Pipeline] { 01:10:37.551 [Pipeline] copyArtifacts 01:10:37.776 Copied 13 artifacts from "vhost-phy-autotest" build number 26178 01:10:37.781 [Pipeline] writeFile 01:10:38.028 [Pipeline] } 01:10:38.173 [Pipeline] // dir 01:10:38.262 [Pipeline] dir 01:10:38.263 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/scanbuild-vg-autotest_63372 01:10:38.264 [Pipeline] { 01:10:38.288 [Pipeline] copyArtifacts 01:10:38.415 Copied 12 artifacts from "scanbuild-vg-autotest" build number 63372 01:10:38.419 [Pipeline] writeFile 01:10:38.475 [Pipeline] } 01:10:38.515 [Pipeline] // dir 01:10:40.391 [Pipeline] dir 01:10:40.391 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-uring-vg-autotest_77338 01:10:40.392 [Pipeline] { 01:10:40.560 [Pipeline] copyArtifacts 01:10:43.325 Copied 16 artifacts from "iscsi-uring-vg-autotest" build number 77338 01:10:43.329 [Pipeline] writeFile 01:10:44.363 [Pipeline] } 01:10:44.795 [Pipeline] // dir 01:10:44.957 [Pipeline] dir 01:10:44.958 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/autobuild-vg-autotest_63413 01:10:44.959 [Pipeline] { 01:10:44.998 [Pipeline] copyArtifacts 01:10:45.314 Copied 9 artifacts from "autobuild-vg-autotest" build number 63413 01:10:45.319 [Pipeline] writeFile 01:10:45.385 [Pipeline] } 01:10:45.452 [Pipeline] // dir 01:10:45.806 [Pipeline] dir 01:10:45.806 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/unittest-vg-autotest_63370 01:10:45.807 [Pipeline] { 01:10:45.843 [Pipeline] copyArtifacts 01:10:46.214 Copied 13 artifacts from "unittest-vg-autotest" build number 63370 01:10:46.218 [Pipeline] writeFile 01:10:46.333 [Pipeline] } 01:10:46.410 [Pipeline] // dir 01:10:47.406 [Pipeline] dir 01:10:47.406 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-phy-autotest_72397 01:10:47.407 [Pipeline] { 01:10:47.485 [Pipeline] copyArtifacts 01:10:47.825 Copied 14 artifacts from "nvmf-phy-autotest" build number 72397 01:10:47.828 [Pipeline] writeFile 01:10:48.175 [Pipeline] } 01:10:48.356 [Pipeline] // dir 01:10:49.177 [Pipeline] dir 01:10:49.178 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-uring-vg-autotest_77358 01:10:49.179 [Pipeline] { 01:10:49.249 [Pipeline] copyArtifacts 01:10:49.478 Copied 15 artifacts from "nvmf-tcp-uring-vg-autotest" build number 77358 01:10:49.482 [Pipeline] writeFile 01:10:49.832 [Pipeline] } 01:10:50.000 [Pipeline] // dir 01:10:50.873 [Pipeline] dir 01:10:50.873 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/centos7-vg-autotest_73612 01:10:50.874 [Pipeline] { 01:10:50.935 [Pipeline] copyArtifacts 01:10:51.274 Copied 13 artifacts from "centos7-vg-autotest" build number 73612 01:10:51.279 [Pipeline] writeFile 01:10:51.584 [Pipeline] } 01:10:51.773 [Pipeline] // dir 01:10:52.943 [Pipeline] dir 01:10:52.943 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/crypto-phy-autotest_26153 01:10:52.944 [Pipeline] { 01:10:53.028 [Pipeline] copyArtifacts 01:10:53.289 Copied 10 artifacts from "crypto-phy-autotest" build number 26153 01:10:53.292 [Pipeline] writeFile 01:10:53.810 [Pipeline] } 01:10:54.013 [Pipeline] // dir 01:10:55.265 [Pipeline] dir 01:10:55.266 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-phy-autotest_72284 01:10:55.267 [Pipeline] { 01:10:55.367 [Pipeline] copyArtifacts 01:10:55.636 Copied 13 artifacts from "nvmf-tcp-phy-autotest" build number 72284 01:10:55.640 [Pipeline] writeFile 01:10:56.142 [Pipeline] } 01:10:56.401 [Pipeline] // dir 01:10:58.431 [Pipeline] dir 01:10:58.431 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-vg-autotest_77471 01:10:58.432 [Pipeline] { 01:10:58.600 [Pipeline] copyArtifacts 01:11:01.692 Copied 16 artifacts from "iscsi-vg-autotest" build number 77471 01:11:01.696 [Pipeline] writeFile 01:11:02.627 [Pipeline] } 01:11:03.068 [Pipeline] // dir 01:11:03.344 [Pipeline] dir 01:11:03.344 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/clang-vg-autotest_57621 01:11:03.345 [Pipeline] { 01:11:03.387 [Pipeline] copyArtifacts 01:11:03.473 Copied 9 artifacts from "clang-vg-autotest" build number 57621 01:11:03.476 [Pipeline] writeFile 01:11:03.605 [Pipeline] } 01:11:03.708 [Pipeline] // dir 01:11:04.594 [Pipeline] dir 01:11:04.594 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-vg-autotest_77424 01:11:04.595 [Pipeline] { 01:11:04.671 [Pipeline] copyArtifacts 01:11:04.884 Copied 14 artifacts from "nvmf-tcp-vg-autotest" build number 77424 01:11:04.888 [Pipeline] writeFile 01:11:05.289 [Pipeline] } 01:11:05.478 [Pipeline] // dir 01:11:06.688 [Pipeline] dir 01:11:06.689 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu20-vg-autotest_73752 01:11:06.690 [Pipeline] { 01:11:06.771 [Pipeline] copyArtifacts 01:11:07.279 Copied 12 artifacts from "ubuntu20-vg-autotest" build number 73752 01:11:07.284 [Pipeline] writeFile 01:11:07.714 [Pipeline] } 01:11:07.941 [Pipeline] // dir 01:11:08.810 [Pipeline] dir 01:11:08.810 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/valgrind-vg-autotest_49187 01:11:08.811 [Pipeline] { 01:11:08.875 [Pipeline] copyArtifacts 01:11:09.202 Copied 13 artifacts from "valgrind-vg-autotest" build number 49187 01:11:09.207 [Pipeline] writeFile 01:11:09.539 [Pipeline] } 01:11:09.690 [Pipeline] // dir 01:11:10.969 [Pipeline] dir 01:11:10.969 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/rocky9-vg-autotest_16043 01:11:10.970 [Pipeline] { 01:11:11.059 [Pipeline] copyArtifacts 01:11:11.419 Copied 13 artifacts from "rocky9-vg-autotest" build number 16043 01:11:11.424 [Pipeline] writeFile 01:11:11.918 [Pipeline] } 01:11:12.163 [Pipeline] // dir 01:11:13.499 [Pipeline] dir 01:11:13.500 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu22-vg-autotest_37766 01:11:13.501 [Pipeline] { 01:11:13.596 [Pipeline] copyArtifacts 01:11:13.956 Copied 13 artifacts from "ubuntu22-vg-autotest" build number 37766 01:11:13.961 [Pipeline] writeFile 01:11:14.413 [Pipeline] } 01:11:14.663 [Pipeline] // dir 01:11:14.672 [Pipeline] catchError 01:11:14.673 [Pipeline] { 01:11:14.690 [Pipeline] sh 01:11:14.968 + jbp/jenkins/jjb-config/jobs/scripts/post_gen_coverage.sh 01:11:14.968 + shopt -s globstar nullglob 01:11:14.968 + echo 'Start stage post_gen_coverage.sh' 01:11:14.968 Start stage post_gen_coverage.sh 01:11:14.968 + cd /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11 01:11:14.968 + rm -rf /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/doc 01:11:14.968 + trap 'compress_coverage_and_docs; remove_partial_coverage_files && echo '\''End stage post_gen_coverage.sh'\''' EXIT 01:11:14.968 + move_artifacts 01:11:14.968 + local out_dirs 01:11:14.968 + out_dirs=(./**/output/) 01:11:14.968 + for dir in "${out_dirs[@]}" 01:11:14.968 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:14.968 + [[ -f ./autobuild-vg-autotest_63413/output//doc.tar.xz ]] 01:11:14.968 + tar -C ./autobuild-vg-autotest_63413/output/ -xf ./autobuild-vg-autotest_63413/output//doc.tar.xz 01:11:15.225 + rm ./autobuild-vg-autotest_63413/output//doc.tar.xz 01:11:15.225 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.225 + [[ -f ./autobuild-vg-autotest_63413/output//ut_coverage.tar.xz ]] 01:11:15.225 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.225 + [[ -f ./autobuild-vg-autotest_63413/output//llvm.tar.xz ]] 01:11:15.225 + mv ./autobuild-vg-autotest_63413/output//build-pkg-manifest.txt ./autobuild-vg-autotest_63413/output//build-repo-manifest.txt ./autobuild-vg-autotest_63413/output//dmesg.init.log ./autobuild-vg-autotest_63413/output//dmesg.log ./autobuild-vg-autotest_63413/output//doc ./autobuild-vg-autotest_63413/output//power.tar.xz ./autobuild-vg-autotest_63413/output//test_completions.txt ./autobuild-vg-autotest_63413/output//timing.svg ./autobuild-vg-autotest_63413/output//timing.txt ./autobuild-vg-autotest_63413/output//.. 01:11:15.225 + rmdir ./autobuild-vg-autotest_63413/output/ 01:11:15.225 + for dir in "${out_dirs[@]}" 01:11:15.225 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.225 + [[ -f ./BlobFS-phy-autotest_26159/output//doc.tar.xz ]] 01:11:15.225 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.225 + [[ -f ./BlobFS-phy-autotest_26159/output//ut_coverage.tar.xz ]] 01:11:15.225 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.225 + [[ -f ./BlobFS-phy-autotest_26159/output//llvm.tar.xz ]] 01:11:15.225 + mv ./BlobFS-phy-autotest_26159/output//all_tests.txt ./BlobFS-phy-autotest_26159/output//build-pkg-manifest.txt ./BlobFS-phy-autotest_26159/output//build-repo-manifest.txt ./BlobFS-phy-autotest_26159/output//cov_base.info.xz ./BlobFS-phy-autotest_26159/output//cov_test.info.xz ./BlobFS-phy-autotest_26159/output//cov_total.info.xz ./BlobFS-phy-autotest_26159/output//dmesg.init.log ./BlobFS-phy-autotest_26159/output//dmesg.log ./BlobFS-phy-autotest_26159/output//power.tar.xz ./BlobFS-phy-autotest_26159/output//rocksdb ./BlobFS-phy-autotest_26159/output//test_completions.txt ./BlobFS-phy-autotest_26159/output//timing.svg ./BlobFS-phy-autotest_26159/output//timing.txt ./BlobFS-phy-autotest_26159/output//.. 01:11:15.225 + rmdir ./BlobFS-phy-autotest_26159/output/ 01:11:15.225 + for dir in "${out_dirs[@]}" 01:11:15.225 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.225 + [[ -f ./centos7-vg-autotest_73612/output//doc.tar.xz ]] 01:11:15.225 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.225 + [[ -f ./centos7-vg-autotest_73612/output//ut_coverage.tar.xz ]] 01:11:15.225 + tar -C ./centos7-vg-autotest_73612/output/ -xf ./centos7-vg-autotest_73612/output//ut_coverage.tar.xz 01:11:15.484 + rm ./centos7-vg-autotest_73612/output//ut_coverage.tar.xz 01:11:15.484 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.484 + [[ -f ./centos7-vg-autotest_73612/output//llvm.tar.xz ]] 01:11:15.484 + mv ./centos7-vg-autotest_73612/output//all_tests.txt ./centos7-vg-autotest_73612/output//build-pkg-manifest.txt ./centos7-vg-autotest_73612/output//build-repo-manifest.txt ./centos7-vg-autotest_73612/output//cov_base.info.xz ./centos7-vg-autotest_73612/output//cov_test.info.xz ./centos7-vg-autotest_73612/output//cov_total.info.xz ./centos7-vg-autotest_73612/output//dmesg.init.log ./centos7-vg-autotest_73612/output//dmesg.log ./centos7-vg-autotest_73612/output//power.tar.xz ./centos7-vg-autotest_73612/output//test_completions.txt ./centos7-vg-autotest_73612/output//timing.svg ./centos7-vg-autotest_73612/output//timing.txt ./centos7-vg-autotest_73612/output//ut_coverage ./centos7-vg-autotest_73612/output//.. 01:11:15.484 + rmdir ./centos7-vg-autotest_73612/output/ 01:11:15.484 + for dir in "${out_dirs[@]}" 01:11:15.484 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.484 + [[ -f ./clang-vg-autotest_57621/output//doc.tar.xz ]] 01:11:15.484 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.484 + [[ -f ./clang-vg-autotest_57621/output//ut_coverage.tar.xz ]] 01:11:15.484 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.484 + [[ -f ./clang-vg-autotest_57621/output//llvm.tar.xz ]] 01:11:15.484 + mv ./clang-vg-autotest_57621/output//all_tests.txt ./clang-vg-autotest_57621/output//build-pkg-manifest.txt ./clang-vg-autotest_57621/output//build-repo-manifest.txt ./clang-vg-autotest_57621/output//dmesg.init.log ./clang-vg-autotest_57621/output//dmesg.log ./clang-vg-autotest_57621/output//power.tar.xz ./clang-vg-autotest_57621/output//test_completions.txt ./clang-vg-autotest_57621/output//timing.svg ./clang-vg-autotest_57621/output//timing.txt ./clang-vg-autotest_57621/output//.. 01:11:15.484 + rmdir ./clang-vg-autotest_57621/output/ 01:11:15.484 + for dir in "${out_dirs[@]}" 01:11:15.484 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.484 + [[ -f ./crypto-phy-autotest_26153/output//doc.tar.xz ]] 01:11:15.484 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.484 + [[ -f ./crypto-phy-autotest_26153/output//ut_coverage.tar.xz ]] 01:11:15.484 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.484 + [[ -f ./crypto-phy-autotest_26153/output//llvm.tar.xz ]] 01:11:15.484 + mv ./crypto-phy-autotest_26153/output//all_tests.txt ./crypto-phy-autotest_26153/output//build-pkg-manifest.txt ./crypto-phy-autotest_26153/output//build-repo-manifest.txt ./crypto-phy-autotest_26153/output//cov_base.info.xz ./crypto-phy-autotest_26153/output//dmesg.init.log ./crypto-phy-autotest_26153/output//dmesg.log ./crypto-phy-autotest_26153/output//power.tar.xz ./crypto-phy-autotest_26153/output//test_completions.txt ./crypto-phy-autotest_26153/output//timing.txt ./crypto-phy-autotest_26153/output//udev.log ./crypto-phy-autotest_26153/output//.. 01:11:15.484 + rmdir ./crypto-phy-autotest_26153/output/ 01:11:15.484 + for dir in "${out_dirs[@]}" 01:11:15.484 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.484 + [[ -f ./dsa-phy-autotest_4853/output//doc.tar.xz ]] 01:11:15.484 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.484 + [[ -f ./dsa-phy-autotest_4853/output//ut_coverage.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./dsa-phy-autotest_4853/output//llvm.tar.xz ]] 01:11:15.485 + mv ./dsa-phy-autotest_4853/output//all_tests.txt ./dsa-phy-autotest_4853/output//build-pkg-manifest.txt ./dsa-phy-autotest_4853/output//build-repo-manifest.txt ./dsa-phy-autotest_4853/output//cov_base.info.xz ./dsa-phy-autotest_4853/output//cov_test.info.xz ./dsa-phy-autotest_4853/output//cov_total.info.xz ./dsa-phy-autotest_4853/output//dmesg.init.log ./dsa-phy-autotest_4853/output//dmesg.log ./dsa-phy-autotest_4853/output//power.tar.xz ./dsa-phy-autotest_4853/output//test_completions.txt ./dsa-phy-autotest_4853/output//timing.svg ./dsa-phy-autotest_4853/output//timing.txt ./dsa-phy-autotest_4853/output//.. 01:11:15.485 + rmdir ./dsa-phy-autotest_4853/output/ 01:11:15.485 + for dir in "${out_dirs[@]}" 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./iscsi-uring-vg-autotest_77338/output//doc.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./iscsi-uring-vg-autotest_77338/output//ut_coverage.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./iscsi-uring-vg-autotest_77338/output//llvm.tar.xz ]] 01:11:15.485 + mv ./iscsi-uring-vg-autotest_77338/output//all_tests.txt ./iscsi-uring-vg-autotest_77338/output//build-pkg-manifest.txt ./iscsi-uring-vg-autotest_77338/output//build-repo-manifest.txt ./iscsi-uring-vg-autotest_77338/output//calsoft.json ./iscsi-uring-vg-autotest_77338/output//calsoft.tar.xz ./iscsi-uring-vg-autotest_77338/output//cov_base.info.xz ./iscsi-uring-vg-autotest_77338/output//cov_test.info.xz ./iscsi-uring-vg-autotest_77338/output//cov_total.info.xz ./iscsi-uring-vg-autotest_77338/output//dmesg.init.log ./iscsi-uring-vg-autotest_77338/output//dmesg.log ./iscsi-uring-vg-autotest_77338/output//iscsi_autofuzz_logs.txt ./iscsi-uring-vg-autotest_77338/output//iscsi_autofuzz_tgt_output.txt ./iscsi-uring-vg-autotest_77338/output//power.tar.xz ./iscsi-uring-vg-autotest_77338/output//test_completions.txt ./iscsi-uring-vg-autotest_77338/output//timing.svg ./iscsi-uring-vg-autotest_77338/output//timing.txt ./iscsi-uring-vg-autotest_77338/output//.. 01:11:15.485 + rmdir ./iscsi-uring-vg-autotest_77338/output/ 01:11:15.485 + for dir in "${out_dirs[@]}" 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./iscsi-vg-autotest_77471/output//doc.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./iscsi-vg-autotest_77471/output//ut_coverage.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./iscsi-vg-autotest_77471/output//llvm.tar.xz ]] 01:11:15.485 + mv ./iscsi-vg-autotest_77471/output//all_tests.txt ./iscsi-vg-autotest_77471/output//build-pkg-manifest.txt ./iscsi-vg-autotest_77471/output//build-repo-manifest.txt ./iscsi-vg-autotest_77471/output//calsoft.json ./iscsi-vg-autotest_77471/output//calsoft.tar.xz ./iscsi-vg-autotest_77471/output//cov_base.info.xz ./iscsi-vg-autotest_77471/output//cov_test.info.xz ./iscsi-vg-autotest_77471/output//cov_total.info.xz ./iscsi-vg-autotest_77471/output//dmesg.init.log ./iscsi-vg-autotest_77471/output//dmesg.log ./iscsi-vg-autotest_77471/output//iscsi_autofuzz_logs.txt ./iscsi-vg-autotest_77471/output//iscsi_autofuzz_tgt_output.txt ./iscsi-vg-autotest_77471/output//power.tar.xz ./iscsi-vg-autotest_77471/output//test_completions.txt ./iscsi-vg-autotest_77471/output//timing.svg ./iscsi-vg-autotest_77471/output//timing.txt ./iscsi-vg-autotest_77471/output//.. 01:11:15.485 + rmdir ./iscsi-vg-autotest_77471/output/ 01:11:15.485 + for dir in "${out_dirs[@]}" 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./lvol-vg-autotest_77382/output//doc.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./lvol-vg-autotest_77382/output//ut_coverage.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./lvol-vg-autotest_77382/output//llvm.tar.xz ]] 01:11:15.485 + mv ./lvol-vg-autotest_77382/output//all_tests.txt ./lvol-vg-autotest_77382/output//build-pkg-manifest.txt ./lvol-vg-autotest_77382/output//build-repo-manifest.txt ./lvol-vg-autotest_77382/output//cov_base.info.xz ./lvol-vg-autotest_77382/output//cov_test.info.xz ./lvol-vg-autotest_77382/output//cov_total.info.xz ./lvol-vg-autotest_77382/output//dmesg.init.log ./lvol-vg-autotest_77382/output//dmesg.log ./lvol-vg-autotest_77382/output//power.tar.xz ./lvol-vg-autotest_77382/output//test_completions.txt ./lvol-vg-autotest_77382/output//timing.svg ./lvol-vg-autotest_77382/output//timing.txt ./lvol-vg-autotest_77382/output//.. 01:11:15.485 + rmdir ./lvol-vg-autotest_77382/output/ 01:11:15.485 + for dir in "${out_dirs[@]}" 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvme-cmb-pmr-vg-autotest_78556/output//doc.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvme-cmb-pmr-vg-autotest_78556/output//ut_coverage.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvme-cmb-pmr-vg-autotest_78556/output//llvm.tar.xz ]] 01:11:15.485 + mv ./nvme-cmb-pmr-vg-autotest_78556/output//all_tests.txt ./nvme-cmb-pmr-vg-autotest_78556/output//build-pkg-manifest.txt ./nvme-cmb-pmr-vg-autotest_78556/output//build-repo-manifest.txt ./nvme-cmb-pmr-vg-autotest_78556/output//cov_base.info.xz ./nvme-cmb-pmr-vg-autotest_78556/output//cov_test.info.xz ./nvme-cmb-pmr-vg-autotest_78556/output//cov_total.info.xz ./nvme-cmb-pmr-vg-autotest_78556/output//dmesg.init.log ./nvme-cmb-pmr-vg-autotest_78556/output//dmesg.log ./nvme-cmb-pmr-vg-autotest_78556/output//power.tar.xz ./nvme-cmb-pmr-vg-autotest_78556/output//test_completions.txt ./nvme-cmb-pmr-vg-autotest_78556/output//timing.svg ./nvme-cmb-pmr-vg-autotest_78556/output//timing.txt ./nvme-cmb-pmr-vg-autotest_78556/output//.. 01:11:15.485 + rmdir ./nvme-cmb-pmr-vg-autotest_78556/output/ 01:11:15.485 + for dir in "${out_dirs[@]}" 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvme-phy-autotest_72325/output//doc.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvme-phy-autotest_72325/output//ut_coverage.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvme-phy-autotest_72325/output//llvm.tar.xz ]] 01:11:15.485 + mv ./nvme-phy-autotest_72325/output//all_tests.txt ./nvme-phy-autotest_72325/output//build-pkg-manifest.txt ./nvme-phy-autotest_72325/output//build-repo-manifest.txt ./nvme-phy-autotest_72325/output//cov_base.info.xz ./nvme-phy-autotest_72325/output//cov_test.info.xz ./nvme-phy-autotest_72325/output//cov_total.info.xz ./nvme-phy-autotest_72325/output//dmesg.init.log ./nvme-phy-autotest_72325/output//dmesg.log ./nvme-phy-autotest_72325/output//power.tar.xz ./nvme-phy-autotest_72325/output//test_completions.txt ./nvme-phy-autotest_72325/output//timing.svg ./nvme-phy-autotest_72325/output//timing.txt ./nvme-phy-autotest_72325/output//.. 01:11:15.485 + rmdir ./nvme-phy-autotest_72325/output/ 01:11:15.485 + for dir in "${out_dirs[@]}" 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvme-vg-autotest_77433/output//doc.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvme-vg-autotest_77433/output//ut_coverage.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvme-vg-autotest_77433/output//llvm.tar.xz ]] 01:11:15.485 + mv ./nvme-vg-autotest_77433/output//all_tests.txt ./nvme-vg-autotest_77433/output//build-kernel-manifest.txt ./nvme-vg-autotest_77433/output//build-pkg-manifest.txt ./nvme-vg-autotest_77433/output//build-repo-manifest.txt ./nvme-vg-autotest_77433/output//cov_base.info.xz ./nvme-vg-autotest_77433/output//dmesg.init.log ./nvme-vg-autotest_77433/output//dmesg.log ./nvme-vg-autotest_77433/output//power.tar.xz ./nvme-vg-autotest_77433/output//proc_list.txt ./nvme-vg-autotest_77433/output//test_completions.txt ./nvme-vg-autotest_77433/output//timing.txt ./nvme-vg-autotest_77433/output//udev.log ./nvme-vg-autotest_77433/output//.. 01:11:15.485 + rmdir ./nvme-vg-autotest_77433/output/ 01:11:15.485 + for dir in "${out_dirs[@]}" 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvmf-phy-autotest_72397/output//doc.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvmf-phy-autotest_72397/output//ut_coverage.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvmf-phy-autotest_72397/output//llvm.tar.xz ]] 01:11:15.485 + mv ./nvmf-phy-autotest_72397/output//all_tests.txt ./nvmf-phy-autotest_72397/output//build-pkg-manifest.txt ./nvmf-phy-autotest_72397/output//build-repo-manifest.txt ./nvmf-phy-autotest_72397/output//cov_base.info.xz ./nvmf-phy-autotest_72397/output//cov_test.info.xz ./nvmf-phy-autotest_72397/output//cov_total.info.xz ./nvmf-phy-autotest_72397/output//dmesg.init.log ./nvmf-phy-autotest_72397/output//dmesg.log ./nvmf-phy-autotest_72397/output//nvmf_fuzz_tgt_output.txt ./nvmf-phy-autotest_72397/output//nvmf_trace.0_shm.tar.gz ./nvmf-phy-autotest_72397/output//power.tar.xz ./nvmf-phy-autotest_72397/output//test_completions.txt ./nvmf-phy-autotest_72397/output//timing.svg ./nvmf-phy-autotest_72397/output//timing.txt ./nvmf-phy-autotest_72397/output//.. 01:11:15.485 + rmdir ./nvmf-phy-autotest_72397/output/ 01:11:15.485 + for dir in "${out_dirs[@]}" 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvmf-tcp-phy-autotest_72284/output//doc.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvmf-tcp-phy-autotest_72284/output//ut_coverage.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvmf-tcp-phy-autotest_72284/output//llvm.tar.xz ]] 01:11:15.485 + mv ./nvmf-tcp-phy-autotest_72284/output//all_tests.txt ./nvmf-tcp-phy-autotest_72284/output//build-pkg-manifest.txt ./nvmf-tcp-phy-autotest_72284/output//build-repo-manifest.txt ./nvmf-tcp-phy-autotest_72284/output//cov_base.info.xz ./nvmf-tcp-phy-autotest_72284/output//cov_test.info.xz ./nvmf-tcp-phy-autotest_72284/output//cov_total.info.xz ./nvmf-tcp-phy-autotest_72284/output//dmesg.init.log ./nvmf-tcp-phy-autotest_72284/output//dmesg.log ./nvmf-tcp-phy-autotest_72284/output//nvmf_fuzz_tgt_output.txt ./nvmf-tcp-phy-autotest_72284/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-phy-autotest_72284/output//power.tar.xz ./nvmf-tcp-phy-autotest_72284/output//test_completions.txt ./nvmf-tcp-phy-autotest_72284/output//timing.txt ./nvmf-tcp-phy-autotest_72284/output//.. 01:11:15.485 + rmdir ./nvmf-tcp-phy-autotest_72284/output/ 01:11:15.485 + for dir in "${out_dirs[@]}" 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvmf-tcp-uring-vg-autotest_77358/output//doc.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvmf-tcp-uring-vg-autotest_77358/output//ut_coverage.tar.xz ]] 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.485 + [[ -f ./nvmf-tcp-uring-vg-autotest_77358/output//llvm.tar.xz ]] 01:11:15.485 + mv ./nvmf-tcp-uring-vg-autotest_77358/output//all_tests.txt ./nvmf-tcp-uring-vg-autotest_77358/output//build-kernel-manifest.txt ./nvmf-tcp-uring-vg-autotest_77358/output//build-pkg-manifest.txt ./nvmf-tcp-uring-vg-autotest_77358/output//build-repo-manifest.txt ./nvmf-tcp-uring-vg-autotest_77358/output//cov_base.info.xz ./nvmf-tcp-uring-vg-autotest_77358/output//cov_test.info.xz ./nvmf-tcp-uring-vg-autotest_77358/output//cov_total.info.xz ./nvmf-tcp-uring-vg-autotest_77358/output//dmesg.init.log ./nvmf-tcp-uring-vg-autotest_77358/output//dmesg.log ./nvmf-tcp-uring-vg-autotest_77358/output//nvmf_fuzz_tgt_output.txt ./nvmf-tcp-uring-vg-autotest_77358/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-uring-vg-autotest_77358/output//power.tar.xz ./nvmf-tcp-uring-vg-autotest_77358/output//test_completions.txt ./nvmf-tcp-uring-vg-autotest_77358/output//timing.svg ./nvmf-tcp-uring-vg-autotest_77358/output//timing.txt ./nvmf-tcp-uring-vg-autotest_77358/output//.. 01:11:15.485 + rmdir ./nvmf-tcp-uring-vg-autotest_77358/output/ 01:11:15.485 + for dir in "${out_dirs[@]}" 01:11:15.485 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.486 + [[ -f ./nvmf-tcp-vg-autotest_77424/output//doc.tar.xz ]] 01:11:15.486 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.486 + [[ -f ./nvmf-tcp-vg-autotest_77424/output//ut_coverage.tar.xz ]] 01:11:15.486 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.486 + [[ -f ./nvmf-tcp-vg-autotest_77424/output//llvm.tar.xz ]] 01:11:15.486 + mv ./nvmf-tcp-vg-autotest_77424/output//all_tests.txt ./nvmf-tcp-vg-autotest_77424/output//build-pkg-manifest.txt ./nvmf-tcp-vg-autotest_77424/output//build-repo-manifest.txt ./nvmf-tcp-vg-autotest_77424/output//cov_base.info.xz ./nvmf-tcp-vg-autotest_77424/output//cov_test.info.xz ./nvmf-tcp-vg-autotest_77424/output//cov_total.info.xz ./nvmf-tcp-vg-autotest_77424/output//dmesg.init.log ./nvmf-tcp-vg-autotest_77424/output//dmesg.log ./nvmf-tcp-vg-autotest_77424/output//nvmf_fuzz_tgt_output.txt ./nvmf-tcp-vg-autotest_77424/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-vg-autotest_77424/output//power.tar.xz ./nvmf-tcp-vg-autotest_77424/output//test_completions.txt ./nvmf-tcp-vg-autotest_77424/output//timing.svg ./nvmf-tcp-vg-autotest_77424/output//timing.txt ./nvmf-tcp-vg-autotest_77424/output//.. 01:11:15.486 + rmdir ./nvmf-tcp-vg-autotest_77424/output/ 01:11:15.486 + for dir in "${out_dirs[@]}" 01:11:15.486 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.486 + [[ -f ./rocky9-vg-autotest_16043/output//doc.tar.xz ]] 01:11:15.486 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.486 + [[ -f ./rocky9-vg-autotest_16043/output//ut_coverage.tar.xz ]] 01:11:15.486 + tar -C ./rocky9-vg-autotest_16043/output/ -xf ./rocky9-vg-autotest_16043/output//ut_coverage.tar.xz 01:11:15.743 + rm ./rocky9-vg-autotest_16043/output//ut_coverage.tar.xz 01:11:15.743 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.743 + [[ -f ./rocky9-vg-autotest_16043/output//llvm.tar.xz ]] 01:11:15.743 + mv ./rocky9-vg-autotest_16043/output//all_tests.txt ./rocky9-vg-autotest_16043/output//build-pkg-manifest.txt ./rocky9-vg-autotest_16043/output//build-repo-manifest.txt ./rocky9-vg-autotest_16043/output//cov_base.info.xz ./rocky9-vg-autotest_16043/output//cov_test.info.xz ./rocky9-vg-autotest_16043/output//cov_total.info.xz ./rocky9-vg-autotest_16043/output//dmesg.init.log ./rocky9-vg-autotest_16043/output//dmesg.log ./rocky9-vg-autotest_16043/output//power.tar.xz ./rocky9-vg-autotest_16043/output//test_completions.txt ./rocky9-vg-autotest_16043/output//timing.svg ./rocky9-vg-autotest_16043/output//timing.txt ./rocky9-vg-autotest_16043/output//ut_coverage ./rocky9-vg-autotest_16043/output//.. 01:11:15.743 + rmdir ./rocky9-vg-autotest_16043/output/ 01:11:15.743 + for dir in "${out_dirs[@]}" 01:11:15.743 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.743 + [[ -f ./scanbuild-vg-autotest_63372/output//doc.tar.xz ]] 01:11:15.743 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.743 + [[ -f ./scanbuild-vg-autotest_63372/output//ut_coverage.tar.xz ]] 01:11:15.743 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.743 + [[ -f ./scanbuild-vg-autotest_63372/output//llvm.tar.xz ]] 01:11:15.743 + mv ./scanbuild-vg-autotest_63372/output//all_c_files.txt ./scanbuild-vg-autotest_63372/output//build_output.txt ./scanbuild-vg-autotest_63372/output//build-pkg-manifest.txt ./scanbuild-vg-autotest_63372/output//build-repo-manifest.txt ./scanbuild-vg-autotest_63372/output//built_c_files.txt ./scanbuild-vg-autotest_63372/output//dmesg.init.log ./scanbuild-vg-autotest_63372/output//dmesg.log ./scanbuild-vg-autotest_63372/output//power.tar.xz ./scanbuild-vg-autotest_63372/output//test_completions.txt ./scanbuild-vg-autotest_63372/output//timing.svg ./scanbuild-vg-autotest_63372/output//timing.txt ./scanbuild-vg-autotest_63372/output//unbuilt_c_files.txt ./scanbuild-vg-autotest_63372/output//.. 01:11:15.743 + rmdir ./scanbuild-vg-autotest_63372/output/ 01:11:15.743 + for dir in "${out_dirs[@]}" 01:11:15.743 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.743 + [[ -f ./short-fuzz-phy-autotest_26150/output//doc.tar.xz ]] 01:11:15.743 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.743 + [[ -f ./short-fuzz-phy-autotest_26150/output//ut_coverage.tar.xz ]] 01:11:15.743 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.743 + [[ -f ./short-fuzz-phy-autotest_26150/output//llvm.tar.xz ]] 01:11:15.743 + tar -C ./short-fuzz-phy-autotest_26150/output/ -xf ./short-fuzz-phy-autotest_26150/output//llvm.tar.xz 01:11:15.743 + rm ./short-fuzz-phy-autotest_26150/output//llvm.tar.xz 01:11:15.743 + mv ./short-fuzz-phy-autotest_26150/output//all_tests.txt ./short-fuzz-phy-autotest_26150/output//build-pkg-manifest.txt ./short-fuzz-phy-autotest_26150/output//build-repo-manifest.txt ./short-fuzz-phy-autotest_26150/output//dmesg.init.log ./short-fuzz-phy-autotest_26150/output//dmesg.log ./short-fuzz-phy-autotest_26150/output//llvm ./short-fuzz-phy-autotest_26150/output//power.tar.xz ./short-fuzz-phy-autotest_26150/output//test_completions.txt ./short-fuzz-phy-autotest_26150/output//timing.svg ./short-fuzz-phy-autotest_26150/output//timing.txt ./short-fuzz-phy-autotest_26150/output//.. 01:11:15.743 + rmdir ./short-fuzz-phy-autotest_26150/output/ 01:11:15.743 + for dir in "${out_dirs[@]}" 01:11:15.743 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.743 + [[ -f ./ubuntu20-vg-autotest_73752/output//doc.tar.xz ]] 01:11:15.743 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:15.743 + [[ -f ./ubuntu20-vg-autotest_73752/output//ut_coverage.tar.xz ]] 01:11:15.743 + tar -C ./ubuntu20-vg-autotest_73752/output/ -xf ./ubuntu20-vg-autotest_73752/output//ut_coverage.tar.xz 01:11:16.001 + rm ./ubuntu20-vg-autotest_73752/output//ut_coverage.tar.xz 01:11:16.001 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.001 + [[ -f ./ubuntu20-vg-autotest_73752/output//llvm.tar.xz ]] 01:11:16.001 + mv ./ubuntu20-vg-autotest_73752/output//all_tests.txt ./ubuntu20-vg-autotest_73752/output//build-pkg-manifest.txt ./ubuntu20-vg-autotest_73752/output//build-repo-manifest.txt ./ubuntu20-vg-autotest_73752/output//cov_base.info.xz ./ubuntu20-vg-autotest_73752/output//dmesg.init.log ./ubuntu20-vg-autotest_73752/output//dmesg.log ./ubuntu20-vg-autotest_73752/output//power.tar.xz ./ubuntu20-vg-autotest_73752/output//proc_list.txt ./ubuntu20-vg-autotest_73752/output//test_completions.txt ./ubuntu20-vg-autotest_73752/output//timing.txt ./ubuntu20-vg-autotest_73752/output//udev.log ./ubuntu20-vg-autotest_73752/output//ut_coverage ./ubuntu20-vg-autotest_73752/output//.. 01:11:16.001 + rmdir ./ubuntu20-vg-autotest_73752/output/ 01:11:16.001 + for dir in "${out_dirs[@]}" 01:11:16.001 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.001 + [[ -f ./ubuntu22-vg-autotest_37766/output//doc.tar.xz ]] 01:11:16.001 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.001 + [[ -f ./ubuntu22-vg-autotest_37766/output//ut_coverage.tar.xz ]] 01:11:16.001 + tar -C ./ubuntu22-vg-autotest_37766/output/ -xf ./ubuntu22-vg-autotest_37766/output//ut_coverage.tar.xz 01:11:16.258 + rm ./ubuntu22-vg-autotest_37766/output//ut_coverage.tar.xz 01:11:16.258 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.258 + [[ -f ./ubuntu22-vg-autotest_37766/output//llvm.tar.xz ]] 01:11:16.258 + mv ./ubuntu22-vg-autotest_37766/output//all_tests.txt ./ubuntu22-vg-autotest_37766/output//build-pkg-manifest.txt ./ubuntu22-vg-autotest_37766/output//build-repo-manifest.txt ./ubuntu22-vg-autotest_37766/output//cov_base.info.xz ./ubuntu22-vg-autotest_37766/output//cov_test.info.xz ./ubuntu22-vg-autotest_37766/output//cov_total.info.xz ./ubuntu22-vg-autotest_37766/output//dmesg.init.log ./ubuntu22-vg-autotest_37766/output//dmesg.log ./ubuntu22-vg-autotest_37766/output//power.tar.xz ./ubuntu22-vg-autotest_37766/output//test_completions.txt ./ubuntu22-vg-autotest_37766/output//timing.svg ./ubuntu22-vg-autotest_37766/output//timing.txt ./ubuntu22-vg-autotest_37766/output//ut_coverage ./ubuntu22-vg-autotest_37766/output//.. 01:11:16.258 + rmdir ./ubuntu22-vg-autotest_37766/output/ 01:11:16.258 + for dir in "${out_dirs[@]}" 01:11:16.258 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.258 + [[ -f ./unittest-vg-autotest_63370/output//doc.tar.xz ]] 01:11:16.258 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.258 + [[ -f ./unittest-vg-autotest_63370/output//ut_coverage.tar.xz ]] 01:11:16.258 + tar -C ./unittest-vg-autotest_63370/output/ -xf ./unittest-vg-autotest_63370/output//ut_coverage.tar.xz 01:11:16.516 + rm ./unittest-vg-autotest_63370/output//ut_coverage.tar.xz 01:11:16.516 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.516 + [[ -f ./unittest-vg-autotest_63370/output//llvm.tar.xz ]] 01:11:16.516 + mv ./unittest-vg-autotest_63370/output//all_tests.txt ./unittest-vg-autotest_63370/output//build-pkg-manifest.txt ./unittest-vg-autotest_63370/output//build-repo-manifest.txt ./unittest-vg-autotest_63370/output//cov_base.info.xz ./unittest-vg-autotest_63370/output//cov_test.info.xz ./unittest-vg-autotest_63370/output//cov_total.info.xz ./unittest-vg-autotest_63370/output//dmesg.init.log ./unittest-vg-autotest_63370/output//dmesg.log ./unittest-vg-autotest_63370/output//power.tar.xz ./unittest-vg-autotest_63370/output//test_completions.txt ./unittest-vg-autotest_63370/output//timing.svg ./unittest-vg-autotest_63370/output//timing.txt ./unittest-vg-autotest_63370/output//ut_coverage ./unittest-vg-autotest_63370/output//.. 01:11:16.516 + rmdir ./unittest-vg-autotest_63370/output/ 01:11:16.516 + for dir in "${out_dirs[@]}" 01:11:16.516 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.516 + [[ -f ./valgrind-vg-autotest_49187/output//doc.tar.xz ]] 01:11:16.516 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.516 + [[ -f ./valgrind-vg-autotest_49187/output//ut_coverage.tar.xz ]] 01:11:16.516 + tar -C ./valgrind-vg-autotest_49187/output/ -xf ./valgrind-vg-autotest_49187/output//ut_coverage.tar.xz 01:11:16.774 + rm ./valgrind-vg-autotest_49187/output//ut_coverage.tar.xz 01:11:16.774 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.774 + [[ -f ./valgrind-vg-autotest_49187/output//llvm.tar.xz ]] 01:11:16.774 + mv ./valgrind-vg-autotest_49187/output//all_tests.txt ./valgrind-vg-autotest_49187/output//build-pkg-manifest.txt ./valgrind-vg-autotest_49187/output//build-repo-manifest.txt ./valgrind-vg-autotest_49187/output//cov_base.info.xz ./valgrind-vg-autotest_49187/output//cov_test.info.xz ./valgrind-vg-autotest_49187/output//cov_total.info.xz ./valgrind-vg-autotest_49187/output//dmesg.init.log ./valgrind-vg-autotest_49187/output//dmesg.log ./valgrind-vg-autotest_49187/output//power.tar.xz ./valgrind-vg-autotest_49187/output//test_completions.txt ./valgrind-vg-autotest_49187/output//timing.svg ./valgrind-vg-autotest_49187/output//timing.txt ./valgrind-vg-autotest_49187/output//ut_coverage ./valgrind-vg-autotest_49187/output//.. 01:11:16.774 + rmdir ./valgrind-vg-autotest_49187/output/ 01:11:16.774 + for dir in "${out_dirs[@]}" 01:11:16.774 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.774 + [[ -f ./vfio-user-phy-autotest_26161/output//doc.tar.xz ]] 01:11:16.774 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.774 + [[ -f ./vfio-user-phy-autotest_26161/output//ut_coverage.tar.xz ]] 01:11:16.774 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.774 + [[ -f ./vfio-user-phy-autotest_26161/output//llvm.tar.xz ]] 01:11:16.774 + mv ./vfio-user-phy-autotest_26161/output//all_tests.txt ./vfio-user-phy-autotest_26161/output//build-pkg-manifest.txt ./vfio-user-phy-autotest_26161/output//build-repo-manifest.txt ./vfio-user-phy-autotest_26161/output//cov_base.info.xz ./vfio-user-phy-autotest_26161/output//cov_test.info.xz ./vfio-user-phy-autotest_26161/output//cov_total.info.xz ./vfio-user-phy-autotest_26161/output//dmesg.init.log ./vfio-user-phy-autotest_26161/output//dmesg.log ./vfio-user-phy-autotest_26161/output//power.tar.xz ./vfio-user-phy-autotest_26161/output//test_completions.txt ./vfio-user-phy-autotest_26161/output//timing.svg ./vfio-user-phy-autotest_26161/output//timing.txt ./vfio-user-phy-autotest_26161/output//.. 01:11:16.774 + rmdir ./vfio-user-phy-autotest_26161/output/ 01:11:16.774 + for dir in "${out_dirs[@]}" 01:11:16.774 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.774 + [[ -f ./vhost-initiator-vg-autotest_77453/output//doc.tar.xz ]] 01:11:16.774 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.774 + [[ -f ./vhost-initiator-vg-autotest_77453/output//ut_coverage.tar.xz ]] 01:11:16.774 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.774 + [[ -f ./vhost-initiator-vg-autotest_77453/output//llvm.tar.xz ]] 01:11:16.774 + mv ./vhost-initiator-vg-autotest_77453/output//all_tests.txt ./vhost-initiator-vg-autotest_77453/output//build-pkg-manifest.txt ./vhost-initiator-vg-autotest_77453/output//build-repo-manifest.txt ./vhost-initiator-vg-autotest_77453/output//cov_base.info.xz ./vhost-initiator-vg-autotest_77453/output//cov_test.info.xz ./vhost-initiator-vg-autotest_77453/output//cov_total.info.xz ./vhost-initiator-vg-autotest_77453/output//dmesg.init.log ./vhost-initiator-vg-autotest_77453/output//dmesg.log ./vhost-initiator-vg-autotest_77453/output//power.tar.xz ./vhost-initiator-vg-autotest_77453/output//test_completions.txt ./vhost-initiator-vg-autotest_77453/output//timing.svg ./vhost-initiator-vg-autotest_77453/output//timing.txt ./vhost-initiator-vg-autotest_77453/output//vhost_fuzz_output1.txt.xz ./vhost-initiator-vg-autotest_77453/output//vhost_fuzz_output2.txt.xz ./vhost-initiator-vg-autotest_77453/output//vhost_fuzz_tgt_output.txt.xz ./vhost-initiator-vg-autotest_77453/output//.. 01:11:16.774 + rmdir ./vhost-initiator-vg-autotest_77453/output/ 01:11:16.774 + for dir in "${out_dirs[@]}" 01:11:16.774 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.774 + [[ -f ./vhost-phy-autotest_26178/output//doc.tar.xz ]] 01:11:16.774 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.774 + [[ -f ./vhost-phy-autotest_26178/output//ut_coverage.tar.xz ]] 01:11:16.774 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.774 + [[ -f ./vhost-phy-autotest_26178/output//llvm.tar.xz ]] 01:11:16.774 + mv ./vhost-phy-autotest_26178/output//all_tests.txt ./vhost-phy-autotest_26178/output//build-pkg-manifest.txt ./vhost-phy-autotest_26178/output//build-repo-manifest.txt ./vhost-phy-autotest_26178/output//cov_base.info.xz ./vhost-phy-autotest_26178/output//cov_test.info.xz ./vhost-phy-autotest_26178/output//cov_total.info.xz ./vhost-phy-autotest_26178/output//dmesg.init.log ./vhost-phy-autotest_26178/output//dmesg.log ./vhost-phy-autotest_26178/output//nvmf_trace.0_shm.tar.gz ./vhost-phy-autotest_26178/output//power.tar.xz ./vhost-phy-autotest_26178/output//test_completions.txt ./vhost-phy-autotest_26178/output//timing.svg ./vhost-phy-autotest_26178/output//timing.txt ./vhost-phy-autotest_26178/output//.. 01:11:16.774 + rmdir ./vhost-phy-autotest_26178/output/ 01:11:16.774 + for dir in "${out_dirs[@]}" 01:11:16.774 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.774 + [[ -f ./vmd-phy-autotest_2614/output//doc.tar.xz ]] 01:11:16.774 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.774 + [[ -f ./vmd-phy-autotest_2614/output//ut_coverage.tar.xz ]] 01:11:16.774 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.774 + [[ -f ./vmd-phy-autotest_2614/output//llvm.tar.xz ]] 01:11:16.775 + mv ./vmd-phy-autotest_2614/output//all_tests.txt ./vmd-phy-autotest_2614/output//build-pkg-manifest.txt ./vmd-phy-autotest_2614/output//build-repo-manifest.txt ./vmd-phy-autotest_2614/output//cov_base.info.xz ./vmd-phy-autotest_2614/output//cov_test.info.xz ./vmd-phy-autotest_2614/output//cov_total.info.xz ./vmd-phy-autotest_2614/output//dmesg.init.log ./vmd-phy-autotest_2614/output//dmesg.log ./vmd-phy-autotest_2614/output//power.tar.xz ./vmd-phy-autotest_2614/output//test_completions.txt ./vmd-phy-autotest_2614/output//timing.svg ./vmd-phy-autotest_2614/output//timing.txt ./vmd-phy-autotest_2614/output//.. 01:11:16.775 + rmdir ./vmd-phy-autotest_2614/output/ 01:11:16.775 + for dir in "${out_dirs[@]}" 01:11:16.775 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.775 + [[ -f ./zns-vg-autotest_76539/output//doc.tar.xz ]] 01:11:16.775 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.775 + [[ -f ./zns-vg-autotest_76539/output//ut_coverage.tar.xz ]] 01:11:16.775 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:11:16.775 + [[ -f ./zns-vg-autotest_76539/output//llvm.tar.xz ]] 01:11:16.775 + mv ./zns-vg-autotest_76539/output//all_tests.txt ./zns-vg-autotest_76539/output//build-pkg-manifest.txt ./zns-vg-autotest_76539/output//build-repo-manifest.txt ./zns-vg-autotest_76539/output//cov_base.info.xz ./zns-vg-autotest_76539/output//cov_test.info.xz ./zns-vg-autotest_76539/output//cov_total.info.xz ./zns-vg-autotest_76539/output//dmesg.init.log ./zns-vg-autotest_76539/output//dmesg.log ./zns-vg-autotest_76539/output//power.tar.xz ./zns-vg-autotest_76539/output//test_completions.txt ./zns-vg-autotest_76539/output//timing.svg ./zns-vg-autotest_76539/output//timing.txt ./zns-vg-autotest_76539/output//.. 01:11:16.775 + rmdir ./zns-vg-autotest_76539/output/ 01:11:16.775 + unpack_cov_files 01:11:16.775 + local info_files 01:11:16.775 + info_files=(*/cov_*.info.xz) 01:11:16.775 + xargs -P0 -r -n1 xz -d 01:11:16.775 + printf '%s\n' BlobFS-phy-autotest_26159/cov_base.info.xz BlobFS-phy-autotest_26159/cov_test.info.xz BlobFS-phy-autotest_26159/cov_total.info.xz centos7-vg-autotest_73612/cov_base.info.xz centos7-vg-autotest_73612/cov_test.info.xz centos7-vg-autotest_73612/cov_total.info.xz crypto-phy-autotest_26153/cov_base.info.xz dsa-phy-autotest_4853/cov_base.info.xz dsa-phy-autotest_4853/cov_test.info.xz dsa-phy-autotest_4853/cov_total.info.xz iscsi-uring-vg-autotest_77338/cov_base.info.xz iscsi-uring-vg-autotest_77338/cov_test.info.xz iscsi-uring-vg-autotest_77338/cov_total.info.xz iscsi-vg-autotest_77471/cov_base.info.xz iscsi-vg-autotest_77471/cov_test.info.xz iscsi-vg-autotest_77471/cov_total.info.xz lvol-vg-autotest_77382/cov_base.info.xz lvol-vg-autotest_77382/cov_test.info.xz lvol-vg-autotest_77382/cov_total.info.xz nvme-cmb-pmr-vg-autotest_78556/cov_base.info.xz nvme-cmb-pmr-vg-autotest_78556/cov_test.info.xz nvme-cmb-pmr-vg-autotest_78556/cov_total.info.xz nvme-phy-autotest_72325/cov_base.info.xz nvme-phy-autotest_72325/cov_test.info.xz nvme-phy-autotest_72325/cov_total.info.xz nvme-vg-autotest_77433/cov_base.info.xz nvmf-phy-autotest_72397/cov_base.info.xz nvmf-phy-autotest_72397/cov_test.info.xz nvmf-phy-autotest_72397/cov_total.info.xz nvmf-tcp-phy-autotest_72284/cov_base.info.xz nvmf-tcp-phy-autotest_72284/cov_test.info.xz nvmf-tcp-phy-autotest_72284/cov_total.info.xz nvmf-tcp-uring-vg-autotest_77358/cov_base.info.xz nvmf-tcp-uring-vg-autotest_77358/cov_test.info.xz nvmf-tcp-uring-vg-autotest_77358/cov_total.info.xz nvmf-tcp-vg-autotest_77424/cov_base.info.xz nvmf-tcp-vg-autotest_77424/cov_test.info.xz nvmf-tcp-vg-autotest_77424/cov_total.info.xz rocky9-vg-autotest_16043/cov_base.info.xz rocky9-vg-autotest_16043/cov_test.info.xz rocky9-vg-autotest_16043/cov_total.info.xz ubuntu20-vg-autotest_73752/cov_base.info.xz ubuntu22-vg-autotest_37766/cov_base.info.xz ubuntu22-vg-autotest_37766/cov_test.info.xz ubuntu22-vg-autotest_37766/cov_total.info.xz unittest-vg-autotest_63370/cov_base.info.xz unittest-vg-autotest_63370/cov_test.info.xz unittest-vg-autotest_63370/cov_total.info.xz valgrind-vg-autotest_49187/cov_base.info.xz valgrind-vg-autotest_49187/cov_test.info.xz valgrind-vg-autotest_49187/cov_total.info.xz vfio-user-phy-autotest_26161/cov_base.info.xz vfio-user-phy-autotest_26161/cov_test.info.xz vfio-user-phy-autotest_26161/cov_total.info.xz vhost-initiator-vg-autotest_77453/cov_base.info.xz vhost-initiator-vg-autotest_77453/cov_test.info.xz vhost-initiator-vg-autotest_77453/cov_total.info.xz vhost-phy-autotest_26178/cov_base.info.xz vhost-phy-autotest_26178/cov_test.info.xz vhost-phy-autotest_26178/cov_total.info.xz vmd-phy-autotest_2614/cov_base.info.xz vmd-phy-autotest_2614/cov_test.info.xz vmd-phy-autotest_2614/cov_total.info.xz zns-vg-autotest_76539/cov_base.info.xz zns-vg-autotest_76539/cov_test.info.xz zns-vg-autotest_76539/cov_total.info.xz 01:11:17.341 + fix_downstream_job_paths 01:11:17.341 + sed -i -e 's#^SF:/.\+/spdk/#SF:/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/#g' BlobFS-phy-autotest_26159/cov_total.info centos7-vg-autotest_73612/cov_total.info dsa-phy-autotest_4853/cov_total.info iscsi-uring-vg-autotest_77338/cov_total.info iscsi-vg-autotest_77471/cov_total.info lvol-vg-autotest_77382/cov_total.info nvme-cmb-pmr-vg-autotest_78556/cov_total.info nvme-phy-autotest_72325/cov_total.info nvmf-phy-autotest_72397/cov_total.info nvmf-tcp-phy-autotest_72284/cov_total.info nvmf-tcp-uring-vg-autotest_77358/cov_total.info nvmf-tcp-vg-autotest_77424/cov_total.info rocky9-vg-autotest_16043/cov_total.info ubuntu22-vg-autotest_37766/cov_total.info unittest-vg-autotest_63370/cov_total.info valgrind-vg-autotest_49187/cov_total.info vfio-user-phy-autotest_26161/cov_total.info vhost-initiator-vg-autotest_77453/cov_total.info vhost-phy-autotest_26178/cov_total.info vmd-phy-autotest_2614/cov_total.info zns-vg-autotest_76539/cov_total.info 01:11:18.273 + ignore_ocf_headers 01:11:18.273 + sed -i 's#--show-details#--show-details --ignore-errors source#g' spdk/autorun_post.py 01:11:18.273 + create_errtrace_files_for_all_builds 01:11:18.273 + local build_log_list 01:11:18.273 + build_log_list=($(find "$WORKSPACE" -type f -name 'build.log')) 01:11:18.273 ++ find /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11 -type f -name build.log 01:11:18.533 + for build in "${build_log_list[@]}" 01:11:18.533 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-cmb-pmr-vg-autotest_78556/build.log 01:11:18.533 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.533 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-cmb-pmr-vg-autotest_78556/build.log 01:11:18.533 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-cmb-pmr-vg-autotest_78556/build.log 01:11:18.533 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-cmb-pmr-vg-autotest_78556 01:11:18.533 + job_name=nvme-cmb-pmr-vg-autotest_78556 01:11:18.533 + start_pattern='Backtrace start' 01:11:18.533 + end_pattern='Backtrace end' 01:11:18.533 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-cmb-pmr-vg-autotest_78556/build.log ]] 01:11:18.533 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-cmb-pmr-vg-autotest_78556/build.log 01:11:18.533 + backtraces= 01:11:18.533 + [[ -n '' ]] 01:11:18.533 + for build in "${build_log_list[@]}" 01:11:18.533 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-uring-vg-autotest_77338/build.log 01:11:18.533 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.533 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-uring-vg-autotest_77338/build.log 01:11:18.533 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-uring-vg-autotest_77338/build.log 01:11:18.533 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-uring-vg-autotest_77338 01:11:18.533 + job_name=iscsi-uring-vg-autotest_77338 01:11:18.533 + start_pattern='Backtrace start' 01:11:18.533 + end_pattern='Backtrace end' 01:11:18.533 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-uring-vg-autotest_77338/build.log ]] 01:11:18.534 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-uring-vg-autotest_77338/build.log 01:11:18.534 + backtraces= 01:11:18.534 + [[ -n '' ]] 01:11:18.534 + for build in "${build_log_list[@]}" 01:11:18.534 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-phy-autotest_72397/build.log 01:11:18.534 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.534 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-phy-autotest_72397/build.log 01:11:18.534 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-phy-autotest_72397/build.log 01:11:18.534 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-phy-autotest_72397 01:11:18.534 + job_name=nvmf-phy-autotest_72397 01:11:18.534 + start_pattern='Backtrace start' 01:11:18.534 + end_pattern='Backtrace end' 01:11:18.534 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-phy-autotest_72397/build.log ]] 01:11:18.534 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-phy-autotest_72397/build.log 01:11:18.534 + backtraces= 01:11:18.534 + [[ -n '' ]] 01:11:18.534 + for build in "${build_log_list[@]}" 01:11:18.534 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/short-fuzz-phy-autotest_26150/build.log 01:11:18.534 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.534 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/short-fuzz-phy-autotest_26150/build.log 01:11:18.534 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/short-fuzz-phy-autotest_26150/build.log 01:11:18.534 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/short-fuzz-phy-autotest_26150 01:11:18.534 + job_name=short-fuzz-phy-autotest_26150 01:11:18.534 + start_pattern='Backtrace start' 01:11:18.534 + end_pattern='Backtrace end' 01:11:18.534 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/short-fuzz-phy-autotest_26150/build.log ]] 01:11:18.534 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/short-fuzz-phy-autotest_26150/build.log 01:11:18.534 + backtraces= 01:11:18.534 + [[ -n '' ]] 01:11:18.534 + for build in "${build_log_list[@]}" 01:11:18.534 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/rocky9-vg-autotest_16043/build.log 01:11:18.534 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.534 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/rocky9-vg-autotest_16043/build.log 01:11:18.534 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/rocky9-vg-autotest_16043/build.log 01:11:18.534 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/rocky9-vg-autotest_16043 01:11:18.534 + job_name=rocky9-vg-autotest_16043 01:11:18.534 + start_pattern='Backtrace start' 01:11:18.534 + end_pattern='Backtrace end' 01:11:18.534 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/rocky9-vg-autotest_16043/build.log ]] 01:11:18.534 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/rocky9-vg-autotest_16043/build.log 01:11:18.534 + backtraces= 01:11:18.534 + [[ -n '' ]] 01:11:18.534 + for build in "${build_log_list[@]}" 01:11:18.534 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-initiator-vg-autotest_77453/build.log 01:11:18.534 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.534 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-initiator-vg-autotest_77453/build.log 01:11:18.534 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-initiator-vg-autotest_77453/build.log 01:11:18.534 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-initiator-vg-autotest_77453 01:11:18.534 + job_name=vhost-initiator-vg-autotest_77453 01:11:18.534 + start_pattern='Backtrace start' 01:11:18.534 + end_pattern='Backtrace end' 01:11:18.534 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-initiator-vg-autotest_77453/build.log ]] 01:11:18.534 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-initiator-vg-autotest_77453/build.log 01:11:18.534 + backtraces= 01:11:18.534 + [[ -n '' ]] 01:11:18.534 + for build in "${build_log_list[@]}" 01:11:18.534 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/autobuild-vg-autotest_63413/build.log 01:11:18.534 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.534 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/autobuild-vg-autotest_63413/build.log 01:11:18.534 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/autobuild-vg-autotest_63413/build.log 01:11:18.534 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/autobuild-vg-autotest_63413 01:11:18.534 + job_name=autobuild-vg-autotest_63413 01:11:18.534 + start_pattern='Backtrace start' 01:11:18.534 + end_pattern='Backtrace end' 01:11:18.534 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/autobuild-vg-autotest_63413/build.log ]] 01:11:18.534 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/autobuild-vg-autotest_63413/build.log 01:11:18.534 + backtraces= 01:11:18.534 + [[ -n '' ]] 01:11:18.534 + for build in "${build_log_list[@]}" 01:11:18.534 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/centos7-vg-autotest_73612/build.log 01:11:18.534 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.534 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/centos7-vg-autotest_73612/build.log 01:11:18.534 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/centos7-vg-autotest_73612/build.log 01:11:18.534 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/centos7-vg-autotest_73612 01:11:18.534 + job_name=centos7-vg-autotest_73612 01:11:18.534 + start_pattern='Backtrace start' 01:11:18.534 + end_pattern='Backtrace end' 01:11:18.534 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/centos7-vg-autotest_73612/build.log ]] 01:11:18.534 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/centos7-vg-autotest_73612/build.log 01:11:18.534 + backtraces= 01:11:18.534 + [[ -n '' ]] 01:11:18.534 + for build in "${build_log_list[@]}" 01:11:18.534 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dsa-phy-autotest_4853/build.log 01:11:18.534 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.534 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dsa-phy-autotest_4853/build.log 01:11:18.534 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dsa-phy-autotest_4853/build.log 01:11:18.534 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dsa-phy-autotest_4853 01:11:18.534 + job_name=dsa-phy-autotest_4853 01:11:18.534 + start_pattern='Backtrace start' 01:11:18.534 + end_pattern='Backtrace end' 01:11:18.534 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dsa-phy-autotest_4853/build.log ]] 01:11:18.534 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dsa-phy-autotest_4853/build.log 01:11:18.534 + backtraces= 01:11:18.534 + [[ -n '' ]] 01:11:18.534 + for build in "${build_log_list[@]}" 01:11:18.534 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/scanbuild-vg-autotest_63372/build.log 01:11:18.534 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.534 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/scanbuild-vg-autotest_63372/build.log 01:11:18.534 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/scanbuild-vg-autotest_63372/build.log 01:11:18.534 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/scanbuild-vg-autotest_63372 01:11:18.534 + job_name=scanbuild-vg-autotest_63372 01:11:18.534 + start_pattern='Backtrace start' 01:11:18.534 + end_pattern='Backtrace end' 01:11:18.534 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/scanbuild-vg-autotest_63372/build.log ]] 01:11:18.534 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/scanbuild-vg-autotest_63372/build.log 01:11:18.534 + backtraces= 01:11:18.534 + [[ -n '' ]] 01:11:18.534 + for build in "${build_log_list[@]}" 01:11:18.534 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu22-vg-autotest_37766/build.log 01:11:18.534 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.534 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu22-vg-autotest_37766/build.log 01:11:18.534 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu22-vg-autotest_37766/build.log 01:11:18.534 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu22-vg-autotest_37766 01:11:18.534 + job_name=ubuntu22-vg-autotest_37766 01:11:18.534 + start_pattern='Backtrace start' 01:11:18.534 + end_pattern='Backtrace end' 01:11:18.534 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu22-vg-autotest_37766/build.log ]] 01:11:18.534 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu22-vg-autotest_37766/build.log 01:11:18.534 + backtraces= 01:11:18.534 + [[ -n '' ]] 01:11:18.534 + for build in "${build_log_list[@]}" 01:11:18.534 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-vg-autotest_77471/build.log 01:11:18.534 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.534 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-vg-autotest_77471/build.log 01:11:18.534 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-vg-autotest_77471/build.log 01:11:18.534 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-vg-autotest_77471 01:11:18.534 + job_name=iscsi-vg-autotest_77471 01:11:18.534 + start_pattern='Backtrace start' 01:11:18.534 + end_pattern='Backtrace end' 01:11:18.534 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-vg-autotest_77471/build.log ]] 01:11:18.534 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-vg-autotest_77471/build.log 01:11:18.534 + backtraces= 01:11:18.534 + [[ -n '' ]] 01:11:18.534 + for build in "${build_log_list[@]}" 01:11:18.534 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/zns-vg-autotest_76539/build.log 01:11:18.534 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.534 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/zns-vg-autotest_76539/build.log 01:11:18.534 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/zns-vg-autotest_76539/build.log 01:11:18.534 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/zns-vg-autotest_76539 01:11:18.534 + job_name=zns-vg-autotest_76539 01:11:18.534 + start_pattern='Backtrace start' 01:11:18.534 + end_pattern='Backtrace end' 01:11:18.534 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/zns-vg-autotest_76539/build.log ]] 01:11:18.534 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/zns-vg-autotest_76539/build.log 01:11:18.534 + backtraces= 01:11:18.534 + [[ -n '' ]] 01:11:18.534 + for build in "${build_log_list[@]}" 01:11:18.534 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/crypto-phy-autotest_26153/build.log 01:11:18.534 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.535 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/crypto-phy-autotest_26153/build.log 01:11:18.535 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/crypto-phy-autotest_26153/build.log 01:11:18.535 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/crypto-phy-autotest_26153 01:11:18.535 + job_name=crypto-phy-autotest_26153 01:11:18.535 + start_pattern='Backtrace start' 01:11:18.535 + end_pattern='Backtrace end' 01:11:18.535 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/crypto-phy-autotest_26153/build.log ]] 01:11:18.535 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/crypto-phy-autotest_26153/build.log 01:11:18.535 + backtraces= 01:11:18.535 + [[ -n '' ]] 01:11:18.535 + for build in "${build_log_list[@]}" 01:11:18.535 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/clang-vg-autotest_57621/build.log 01:11:18.535 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.535 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/clang-vg-autotest_57621/build.log 01:11:18.535 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/clang-vg-autotest_57621/build.log 01:11:18.535 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/clang-vg-autotest_57621 01:11:18.535 + job_name=clang-vg-autotest_57621 01:11:18.535 + start_pattern='Backtrace start' 01:11:18.535 + end_pattern='Backtrace end' 01:11:18.535 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/clang-vg-autotest_57621/build.log ]] 01:11:18.535 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/clang-vg-autotest_57621/build.log 01:11:18.535 + backtraces= 01:11:18.535 + [[ -n '' ]] 01:11:18.535 + for build in "${build_log_list[@]}" 01:11:18.535 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-vg-autotest_77433/build.log 01:11:18.535 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.535 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-vg-autotest_77433/build.log 01:11:18.535 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-vg-autotest_77433/build.log 01:11:18.535 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-vg-autotest_77433 01:11:18.535 + job_name=nvme-vg-autotest_77433 01:11:18.535 + start_pattern='Backtrace start' 01:11:18.535 + end_pattern='Backtrace end' 01:11:18.535 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-vg-autotest_77433/build.log ]] 01:11:18.535 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-vg-autotest_77433/build.log 01:11:18.535 + backtraces='00:24:17.392 00:50:19 sw_hotplug -- common/autotest_common.sh@715 -- # time='\''========== Backtrace start: ========== 01:11:18.535 00:24:17.392 01:11:18.535 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 710 01:11:18.535 00:24:17.392 711 # We redirect cmd'\''\'\'''\''s std{out,err} to a separate fd dup'\''\'\'''\''ed to stdin'\''\'\'''\''s device (or /dev/null) to 01:11:18.535 00:24:17.392 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:11:18.535 00:24:17.392 713 # but $() will return just the time'\''\'\'''\''s data, hence making it possible to just do: 01:11:18.535 00:24:17.392 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.535 00:24:17.392 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.535 00:24:17.392 716 01:11:18.535 00:24:17.392 717 echo "$time" 01:11:18.535 00:24:17.392 718 ) 01:11:18.535 00:24:17.392 719 01:11:18.535 00:24:17.392 720 function timing_enter() { 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 11 # Pci bus hotplug 01:11:18.535 00:24:17.392 12 # Helper function to remove/attach cotrollers 01:11:18.535 00:24:17.392 13 debug_remove_attach_helper() { 01:11:18.535 00:24:17.392 14 local helper_time=0 01:11:18.535 00:24:17.392 15 01:11:18.535 00:24:17.392 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.535 00:24:17.392 17 printf '\''\'\'''\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\''\'\'''\'' \ 01:11:18.535 00:24:17.392 18 "$helper_time" "$nvme_count" >&2 01:11:18.535 00:24:17.392 19 } 01:11:18.535 00:24:17.392 20 01:11:18.535 00:24:17.392 21 remove_attach_helper() { 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.535 00:24:17.392 106 done 01:11:18.535 00:24:17.392 107 01:11:18.535 00:24:17.392 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.535 00:24:17.392 109 01:11:18.535 00:24:17.392 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.535 00:24:17.392 111 # Verify reregistering hotplug poller 01:11:18.535 00:24:17.392 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.535 00:24:17.392 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.535 00:24:17.392 114 01:11:18.535 00:24:17.392 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 133 01:11:18.535 00:24:17.392 134 # Run pci bus hotplug test 01:11:18.535 00:24:17.392 135 run_hotplug 01:11:18.535 00:24:17.392 136 01:11:18.535 00:24:17.392 137 # Run SPDK target based hotplug 01:11:18.535 00:24:17.392 => 138 tgt_run_hotplug 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 01:11:18.535 00:24:17.392 ========== Backtrace end =========='\'' 01:11:18.535 00:24:17.392 ========== Backtrace start: ========== 01:11:18.535 00:24:17.392 01:11:18.535 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 710 01:11:18.535 00:24:17.392 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 01:11:18.535 00:24:17.392 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:11:18.535 00:24:17.392 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:11:18.535 00:24:17.392 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.535 00:24:17.392 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.535 00:24:17.392 716 01:11:18.535 00:24:17.392 717 echo "$time" 01:11:18.535 00:24:17.392 718 ) 01:11:18.535 00:24:17.392 719 01:11:18.535 00:24:17.392 720 function timing_enter() { 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 11 # Pci bus hotplug 01:11:18.535 00:24:17.392 12 # Helper function to remove/attach cotrollers 01:11:18.535 00:24:17.392 13 debug_remove_attach_helper() { 01:11:18.535 00:24:17.392 14 local helper_time=0 01:11:18.535 00:24:17.392 15 01:11:18.535 00:24:17.392 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.535 00:24:17.392 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.535 00:24:17.392 18 "$helper_time" "$nvme_count" >&2 01:11:18.535 00:24:17.392 19 } 01:11:18.535 00:24:17.392 20 01:11:18.535 00:24:17.392 21 remove_attach_helper() { 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.535 00:24:17.392 106 done 01:11:18.535 00:24:17.392 107 01:11:18.535 00:24:17.392 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.535 00:24:17.392 109 01:11:18.535 00:24:17.392 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.535 00:24:17.392 111 # Verify reregistering hotplug poller 01:11:18.535 00:24:17.392 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.535 00:24:17.392 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.535 00:24:17.392 114 01:11:18.535 00:24:17.392 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 133 01:11:18.535 00:24:17.392 134 # Run pci bus hotplug test 01:11:18.535 00:24:17.392 135 run_hotplug 01:11:18.535 00:24:17.392 136 01:11:18.535 00:24:17.392 137 # Run SPDK target based hotplug 01:11:18.535 00:24:17.392 => 138 tgt_run_hotplug 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 01:11:18.535 00:24:17.392 ========== Backtrace end ========== 01:11:18.535 00:24:17.392 ========== Backtrace start: ========== 01:11:18.535 00:24:17.392 01:11:18.535 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:11:18.535 00:24:17.392 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.535 00:24:17.392 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.535 00:24:17.392 716 01:11:18.535 00:24:17.392 717 echo "$time" 01:11:18.535 00:24:17.392 => 718 ) 01:11:18.535 00:24:17.392 719 01:11:18.535 00:24:17.392 720 function timing_enter() { 01:11:18.535 00:24:17.392 721 xtrace_disable 01:11:18.535 00:24:17.392 722 timing "enter" "$1" 01:11:18.535 00:24:17.392 723 xtrace_restore 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.535 00:24:17.392 ... 01:11:18.535 00:24:17.392 11 # Pci bus hotplug 01:11:18.535 00:24:17.392 12 # Helper function to remove/attach cotrollers 01:11:18.535 00:24:17.392 13 debug_remove_attach_helper() { 01:11:18.535 00:24:17.392 14 local helper_time=0 01:11:18.535 00:24:17.392 15 01:11:18.535 00:24:17.392 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.535 00:24:17.392 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.535 00:24:17.392 18 "$helper_time" "$nvme_count" >&2 01:11:18.535 00:24:17.392 19 } 01:11:18.535 00:24:17.392 20 01:11:18.536 00:24:17.392 21 remove_attach_helper() { 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.536 00:24:17.393 106 done 01:11:18.536 00:24:17.393 107 01:11:18.536 00:24:17.393 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.536 00:24:17.393 109 01:11:18.536 00:24:17.393 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.536 00:24:17.393 111 # Verify reregistering hotplug poller 01:11:18.536 00:24:17.393 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.536 00:24:17.393 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.536 00:24:17.393 114 01:11:18.536 00:24:17.393 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 133 01:11:18.536 00:24:17.393 134 # Run pci bus hotplug test 01:11:18.536 00:24:17.393 135 run_hotplug 01:11:18.536 00:24:17.393 136 01:11:18.536 00:24:17.393 137 # Run SPDK target based hotplug 01:11:18.536 00:24:17.393 => 138 tgt_run_hotplug 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 01:11:18.536 00:24:17.393 ========== Backtrace end ========== 01:11:18.536 00:24:17.393 ========== Backtrace start: ========== 01:11:18.536 00:24:17.393 01:11:18.536 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 11 # Pci bus hotplug 01:11:18.536 00:24:17.393 12 # Helper function to remove/attach cotrollers 01:11:18.536 00:24:17.393 13 debug_remove_attach_helper() { 01:11:18.536 00:24:17.393 14 local helper_time=0 01:11:18.536 00:24:17.393 15 01:11:18.536 00:24:17.393 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.536 00:24:17.393 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.536 00:24:17.393 18 "$helper_time" "$nvme_count" >&2 01:11:18.536 00:24:17.393 19 } 01:11:18.536 00:24:17.393 20 01:11:18.536 00:24:17.393 21 remove_attach_helper() { 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.536 00:24:17.393 106 done 01:11:18.536 00:24:17.393 107 01:11:18.536 00:24:17.393 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.536 00:24:17.393 109 01:11:18.536 00:24:17.393 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.536 00:24:17.393 111 # Verify reregistering hotplug poller 01:11:18.536 00:24:17.393 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.536 00:24:17.393 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.536 00:24:17.393 114 01:11:18.536 00:24:17.393 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 133 01:11:18.536 00:24:17.393 134 # Run pci bus hotplug test 01:11:18.536 00:24:17.393 135 run_hotplug 01:11:18.536 00:24:17.393 136 01:11:18.536 00:24:17.393 137 # Run SPDK target based hotplug 01:11:18.536 00:24:17.393 => 138 tgt_run_hotplug 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 01:11:18.536 00:24:17.393 ========== Backtrace end ========== 01:11:18.536 00:24:17.393 ========== Backtrace start: ========== 01:11:18.536 00:24:17.393 01:11:18.536 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 1117 timing_enter $test_name 01:11:18.536 00:24:17.393 1118 echo "************************************" 01:11:18.536 00:24:17.393 1119 echo "START TEST $test_name" 01:11:18.536 00:24:17.393 1120 echo "************************************" 01:11:18.536 00:24:17.393 1121 xtrace_restore 01:11:18.536 00:24:17.393 1122 time "$@" 01:11:18.536 00:24:17.393 1123 xtrace_disable 01:11:18.536 00:24:17.393 1124 echo "************************************" 01:11:18.536 00:24:17.393 1125 echo "END TEST $test_name" 01:11:18.536 00:24:17.393 1126 echo "************************************" 01:11:18.536 00:24:17.393 1127 timing_exit $test_name 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 235 01:11:18.536 00:24:17.393 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 01:11:18.536 00:24:17.393 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 01:11:18.536 00:24:17.393 238 01:11:18.536 00:24:17.393 239 if [ $(uname -s) = Linux ]; then 01:11:18.536 00:24:17.393 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 01:11:18.536 00:24:17.393 241 fi 01:11:18.536 00:24:17.393 242 01:11:18.536 00:24:17.393 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 01:11:18.536 00:24:17.393 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 01:11:18.536 00:24:17.393 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 01:11:18.536 00:24:17.393 ... 01:11:18.536 00:24:17.393 01:11:18.536 00:24:17.393 ========== Backtrace end ==========' 01:11:18.536 + [[ -n 00:24:17.392 00:50:19 sw_hotplug -- common/autotest_common.sh@715 -- # time='========== Backtrace start: ========== 01:11:18.536 00:24:17.392 01:11:18.536 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.536 00:24:17.392 ... 01:11:18.536 00:24:17.392 710 01:11:18.536 00:24:17.392 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 01:11:18.536 00:24:17.392 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:11:18.536 00:24:17.392 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:11:18.536 00:24:17.392 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.536 00:24:17.392 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.536 00:24:17.392 716 01:11:18.536 00:24:17.392 717 echo "$time" 01:11:18.536 00:24:17.392 718 ) 01:11:18.536 00:24:17.392 719 01:11:18.536 00:24:17.392 720 function timing_enter() { 01:11:18.536 00:24:17.392 ... 01:11:18.536 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.536 00:24:17.392 ... 01:11:18.536 00:24:17.392 11 # Pci bus hotplug 01:11:18.536 00:24:17.392 12 # Helper function to remove/attach cotrollers 01:11:18.536 00:24:17.392 13 debug_remove_attach_helper() { 01:11:18.536 00:24:17.392 14 local helper_time=0 01:11:18.536 00:24:17.392 15 01:11:18.536 00:24:17.392 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.536 00:24:17.392 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.536 00:24:17.392 18 "$helper_time" "$nvme_count" >&2 01:11:18.536 00:24:17.392 19 } 01:11:18.536 00:24:17.392 20 01:11:18.536 00:24:17.392 21 remove_attach_helper() { 01:11:18.536 00:24:17.392 ... 01:11:18.536 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.536 00:24:17.392 ... 01:11:18.536 00:24:17.392 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.536 00:24:17.392 106 done 01:11:18.536 00:24:17.392 107 01:11:18.536 00:24:17.392 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.536 00:24:17.392 109 01:11:18.536 00:24:17.392 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.536 00:24:17.392 111 # Verify reregistering hotplug poller 01:11:18.536 00:24:17.392 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.536 00:24:17.392 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.536 00:24:17.392 114 01:11:18.536 00:24:17.392 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.536 00:24:17.392 ... 01:11:18.536 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.536 00:24:17.392 ... 01:11:18.536 00:24:17.392 133 01:11:18.536 00:24:17.392 134 # Run pci bus hotplug test 01:11:18.536 00:24:17.392 135 run_hotplug 01:11:18.536 00:24:17.392 136 01:11:18.536 00:24:17.392 137 # Run SPDK target based hotplug 01:11:18.536 00:24:17.392 => 138 tgt_run_hotplug 01:11:18.536 00:24:17.392 ... 01:11:18.536 00:24:17.392 01:11:18.536 00:24:17.392 ========== Backtrace end ==========' 01:11:18.536 00:24:17.392 ========== Backtrace start: ========== 01:11:18.536 00:24:17.392 01:11:18.536 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.536 00:24:17.392 ... 01:11:18.536 00:24:17.392 710 01:11:18.536 00:24:17.392 711 # We redirect cmd's std{out,err} to a separate fd dup'ed to stdin's device (or /dev/null) to 01:11:18.536 00:24:17.392 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:11:18.536 00:24:17.392 713 # but $() will return just the time's data, hence making it possible to just do: 01:11:18.536 00:24:17.392 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.536 00:24:17.392 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.536 00:24:17.392 716 01:11:18.536 00:24:17.392 717 echo "$time" 01:11:18.536 00:24:17.392 718 ) 01:11:18.536 00:24:17.392 719 01:11:18.536 00:24:17.392 720 function timing_enter() { 01:11:18.536 00:24:17.392 ... 01:11:18.536 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.536 00:24:17.392 ... 01:11:18.536 00:24:17.392 11 # Pci bus hotplug 01:11:18.536 00:24:17.392 12 # Helper function to remove/attach cotrollers 01:11:18.536 00:24:17.392 13 debug_remove_attach_helper() { 01:11:18.536 00:24:17.392 14 local helper_time=0 01:11:18.536 00:24:17.392 15 01:11:18.536 00:24:17.392 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.536 00:24:17.392 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 01:11:18.536 00:24:17.392 18 "$helper_time" "$nvme_count" >&2 01:11:18.536 00:24:17.392 19 } 01:11:18.537 00:24:17.392 20 01:11:18.537 00:24:17.392 21 remove_attach_helper() { 01:11:18.537 00:24:17.392 ... 01:11:18.537 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.537 00:24:17.392 ... 01:11:18.537 00:24:17.392 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.537 00:24:17.392 106 done 01:11:18.537 00:24:17.392 107 01:11:18.537 00:24:17.392 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.537 00:24:17.392 109 01:11:18.537 00:24:17.392 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.537 00:24:17.392 111 # Verify reregistering hotplug poller 01:11:18.537 00:24:17.392 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.537 00:24:17.392 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.537 00:24:17.392 114 01:11:18.537 00:24:17.392 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.537 00:24:17.392 ... 01:11:18.537 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.537 00:24:17.392 ... 01:11:18.537 00:24:17.392 133 01:11:18.537 00:24:17.392 134 # Run pci bus hotplug test 01:11:18.537 00:24:17.392 135 run_hotplug 01:11:18.537 00:24:17.392 136 01:11:18.537 00:24:17.392 137 # Run SPDK target based hotplug 01:11:18.537 00:24:17.392 => 138 tgt_run_hotplug 01:11:18.537 00:24:17.392 ... 01:11:18.537 00:24:17.392 01:11:18.537 00:24:17.392 ========== Backtrace end ========== 01:11:18.537 00:24:17.392 ========== Backtrace start: ========== 01:11:18.537 00:24:17.392 01:11:18.537 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.537 00:24:17.392 ... 01:11:18.537 00:24:17.392 713 # but $() will return just the time's data, hence making it possible to just do: 01:11:18.537 00:24:17.392 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.537 00:24:17.392 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.537 00:24:17.392 716 01:11:18.537 00:24:17.392 717 echo "$time" 01:11:18.537 00:24:17.392 => 718 ) 01:11:18.537 00:24:17.392 719 01:11:18.537 00:24:17.392 720 function timing_enter() { 01:11:18.537 00:24:17.392 721 xtrace_disable 01:11:18.537 00:24:17.392 722 timing "enter" "$1" 01:11:18.537 00:24:17.392 723 xtrace_restore 01:11:18.537 00:24:17.392 ... 01:11:18.537 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.537 00:24:17.392 ... 01:11:18.537 00:24:17.392 11 # Pci bus hotplug 01:11:18.537 00:24:17.392 12 # Helper function to remove/attach cotrollers 01:11:18.537 00:24:17.392 13 debug_remove_attach_helper() { 01:11:18.537 00:24:17.392 14 local helper_time=0 01:11:18.537 00:24:17.392 15 01:11:18.537 00:24:17.392 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.537 00:24:17.392 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 01:11:18.537 00:24:17.392 18 "$helper_time" "$nvme_count" >&2 01:11:18.537 00:24:17.392 19 } 01:11:18.537 00:24:17.392 20 01:11:18.537 00:24:17.392 21 remove_attach_helper() { 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.537 00:24:17.393 106 done 01:11:18.537 00:24:17.393 107 01:11:18.537 00:24:17.393 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.537 00:24:17.393 109 01:11:18.537 00:24:17.393 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.537 00:24:17.393 111 # Verify reregistering hotplug poller 01:11:18.537 00:24:17.393 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.537 00:24:17.393 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.537 00:24:17.393 114 01:11:18.537 00:24:17.393 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 133 01:11:18.537 00:24:17.393 134 # Run pci bus hotplug test 01:11:18.537 00:24:17.393 135 run_hotplug 01:11:18.537 00:24:17.393 136 01:11:18.537 00:24:17.393 137 # Run SPDK target based hotplug 01:11:18.537 00:24:17.393 => 138 tgt_run_hotplug 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 01:11:18.537 00:24:17.393 ========== Backtrace end ========== 01:11:18.537 00:24:17.393 ========== Backtrace start: ========== 01:11:18.537 00:24:17.393 01:11:18.537 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 11 # Pci bus hotplug 01:11:18.537 00:24:17.393 12 # Helper function to remove/attach cotrollers 01:11:18.537 00:24:17.393 13 debug_remove_attach_helper() { 01:11:18.537 00:24:17.393 14 local helper_time=0 01:11:18.537 00:24:17.393 15 01:11:18.537 00:24:17.393 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.537 00:24:17.393 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 01:11:18.537 00:24:17.393 18 "$helper_time" "$nvme_count" >&2 01:11:18.537 00:24:17.393 19 } 01:11:18.537 00:24:17.393 20 01:11:18.537 00:24:17.393 21 remove_attach_helper() { 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.537 00:24:17.393 106 done 01:11:18.537 00:24:17.393 107 01:11:18.537 00:24:17.393 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.537 00:24:17.393 109 01:11:18.537 00:24:17.393 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.537 00:24:17.393 111 # Verify reregistering hotplug poller 01:11:18.537 00:24:17.393 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.537 00:24:17.393 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.537 00:24:17.393 114 01:11:18.537 00:24:17.393 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 133 01:11:18.537 00:24:17.393 134 # Run pci bus hotplug test 01:11:18.537 00:24:17.393 135 run_hotplug 01:11:18.537 00:24:17.393 136 01:11:18.537 00:24:17.393 137 # Run SPDK target based hotplug 01:11:18.537 00:24:17.393 => 138 tgt_run_hotplug 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 01:11:18.537 00:24:17.393 ========== Backtrace end ========== 01:11:18.537 00:24:17.393 ========== Backtrace start: ========== 01:11:18.537 00:24:17.393 01:11:18.537 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 1117 timing_enter $test_name 01:11:18.537 00:24:17.393 1118 echo "************************************" 01:11:18.537 00:24:17.393 1119 echo "START TEST $test_name" 01:11:18.537 00:24:17.393 1120 echo "************************************" 01:11:18.537 00:24:17.393 1121 xtrace_restore 01:11:18.537 00:24:17.393 1122 time "$@" 01:11:18.537 00:24:17.393 1123 xtrace_disable 01:11:18.537 00:24:17.393 1124 echo "************************************" 01:11:18.537 00:24:17.393 1125 echo "END TEST $test_name" 01:11:18.537 00:24:17.393 1126 echo "************************************" 01:11:18.537 00:24:17.393 1127 timing_exit $test_name 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 235 01:11:18.537 00:24:17.393 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 01:11:18.537 00:24:17.393 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 01:11:18.537 00:24:17.393 238 01:11:18.537 00:24:17.393 239 if [ $(uname -s) = Linux ]; then 01:11:18.537 00:24:17.393 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 01:11:18.537 00:24:17.393 241 fi 01:11:18.537 00:24:17.393 242 01:11:18.537 00:24:17.393 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 01:11:18.537 00:24:17.393 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 01:11:18.537 00:24:17.393 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 01:11:18.537 00:24:17.393 ... 01:11:18.537 00:24:17.393 01:11:18.537 00:24:17.393 ========== Backtrace end ========== ]] 01:11:18.537 + echo '00:24:17.392 00:50:19 sw_hotplug -- common/autotest_common.sh@715 -- # time='\''========== Backtrace start: ========== 01:11:18.537 00:24:17.392 01:11:18.537 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.537 00:24:17.392 ... 01:11:18.537 00:24:17.392 710 01:11:18.537 00:24:17.392 711 # We redirect cmd'\''\'\'''\''s std{out,err} to a separate fd dup'\''\'\'''\''ed to stdin'\''\'\'''\''s device (or /dev/null) to 01:11:18.537 00:24:17.392 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:11:18.537 00:24:17.392 713 # but $() will return just the time'\''\'\'''\''s data, hence making it possible to just do: 01:11:18.537 00:24:17.392 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.537 00:24:17.392 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.537 00:24:17.392 716 01:11:18.537 00:24:17.392 717 echo "$time" 01:11:18.537 00:24:17.392 718 ) 01:11:18.537 00:24:17.392 719 01:11:18.537 00:24:17.392 720 function timing_enter() { 01:11:18.537 00:24:17.392 ... 01:11:18.537 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.537 00:24:17.392 ... 01:11:18.537 00:24:17.392 11 # Pci bus hotplug 01:11:18.537 00:24:17.392 12 # Helper function to remove/attach cotrollers 01:11:18.537 00:24:17.392 13 debug_remove_attach_helper() { 01:11:18.537 00:24:17.392 14 local helper_time=0 01:11:18.537 00:24:17.392 15 01:11:18.537 00:24:17.392 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.537 00:24:17.392 17 printf '\''\'\'''\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\''\'\'''\'' \ 01:11:18.537 00:24:17.392 18 "$helper_time" "$nvme_count" >&2 01:11:18.538 00:24:17.392 19 } 01:11:18.538 00:24:17.392 20 01:11:18.538 00:24:17.392 21 remove_attach_helper() { 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.538 00:24:17.392 106 done 01:11:18.538 00:24:17.392 107 01:11:18.538 00:24:17.392 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.538 00:24:17.392 109 01:11:18.538 00:24:17.392 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.538 00:24:17.392 111 # Verify reregistering hotplug poller 01:11:18.538 00:24:17.392 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.538 00:24:17.392 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.538 00:24:17.392 114 01:11:18.538 00:24:17.392 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 133 01:11:18.538 00:24:17.392 134 # Run pci bus hotplug test 01:11:18.538 00:24:17.392 135 run_hotplug 01:11:18.538 00:24:17.392 136 01:11:18.538 00:24:17.392 137 # Run SPDK target based hotplug 01:11:18.538 00:24:17.392 => 138 tgt_run_hotplug 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 01:11:18.538 00:24:17.392 ========== Backtrace end =========='\'' 01:11:18.538 00:24:17.392 ========== Backtrace start: ========== 01:11:18.538 00:24:17.392 01:11:18.538 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 710 01:11:18.538 00:24:17.392 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 01:11:18.538 00:24:17.392 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:11:18.538 00:24:17.392 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:11:18.538 00:24:17.392 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.538 00:24:17.392 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.538 00:24:17.392 716 01:11:18.538 00:24:17.392 717 echo "$time" 01:11:18.538 00:24:17.392 718 ) 01:11:18.538 00:24:17.392 719 01:11:18.538 00:24:17.392 720 function timing_enter() { 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 11 # Pci bus hotplug 01:11:18.538 00:24:17.392 12 # Helper function to remove/attach cotrollers 01:11:18.538 00:24:17.392 13 debug_remove_attach_helper() { 01:11:18.538 00:24:17.392 14 local helper_time=0 01:11:18.538 00:24:17.392 15 01:11:18.538 00:24:17.392 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.538 00:24:17.392 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.538 00:24:17.392 18 "$helper_time" "$nvme_count" >&2 01:11:18.538 00:24:17.392 19 } 01:11:18.538 00:24:17.392 20 01:11:18.538 00:24:17.392 21 remove_attach_helper() { 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.538 00:24:17.392 106 done 01:11:18.538 00:24:17.392 107 01:11:18.538 00:24:17.392 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.538 00:24:17.392 109 01:11:18.538 00:24:17.392 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.538 00:24:17.392 111 # Verify reregistering hotplug poller 01:11:18.538 00:24:17.392 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.538 00:24:17.392 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.538 00:24:17.392 114 01:11:18.538 00:24:17.392 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 133 01:11:18.538 00:24:17.392 134 # Run pci bus hotplug test 01:11:18.538 00:24:17.392 135 run_hotplug 01:11:18.538 00:24:17.392 136 01:11:18.538 00:24:17.392 137 # Run SPDK target based hotplug 01:11:18.538 00:24:17.392 => 138 tgt_run_hotplug 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 01:11:18.538 00:24:17.392 ========== Backtrace end ========== 01:11:18.538 00:24:17.392 ========== Backtrace start: ========== 01:11:18.538 00:24:17.392 01:11:18.538 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:11:18.538 00:24:17.392 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.538 00:24:17.392 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.538 00:24:17.392 716 01:11:18.538 00:24:17.392 717 echo "$time" 01:11:18.538 00:24:17.392 => 718 ) 01:11:18.538 00:24:17.392 719 01:11:18.538 00:24:17.392 720 function timing_enter() { 01:11:18.538 00:24:17.392 721 xtrace_disable 01:11:18.538 00:24:17.392 722 timing "enter" "$1" 01:11:18.538 00:24:17.392 723 xtrace_restore 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.538 00:24:17.392 ... 01:11:18.538 00:24:17.392 11 # Pci bus hotplug 01:11:18.538 00:24:17.392 12 # Helper function to remove/attach cotrollers 01:11:18.538 00:24:17.392 13 debug_remove_attach_helper() { 01:11:18.538 00:24:17.392 14 local helper_time=0 01:11:18.538 00:24:17.392 15 01:11:18.538 00:24:17.392 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.538 00:24:17.392 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.538 00:24:17.392 18 "$helper_time" "$nvme_count" >&2 01:11:18.538 00:24:17.392 19 } 01:11:18.538 00:24:17.392 20 01:11:18.538 00:24:17.392 21 remove_attach_helper() { 01:11:18.538 00:24:17.393 ... 01:11:18.538 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.538 00:24:17.393 ... 01:11:18.538 00:24:17.393 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.538 00:24:17.393 106 done 01:11:18.538 00:24:17.393 107 01:11:18.538 00:24:17.393 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.538 00:24:17.393 109 01:11:18.538 00:24:17.393 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.538 00:24:17.393 111 # Verify reregistering hotplug poller 01:11:18.538 00:24:17.393 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.538 00:24:17.393 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.538 00:24:17.393 114 01:11:18.538 00:24:17.393 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.538 00:24:17.393 ... 01:11:18.538 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.538 00:24:17.393 ... 01:11:18.538 00:24:17.393 133 01:11:18.538 00:24:17.393 134 # Run pci bus hotplug test 01:11:18.538 00:24:17.393 135 run_hotplug 01:11:18.538 00:24:17.393 136 01:11:18.538 00:24:17.393 137 # Run SPDK target based hotplug 01:11:18.538 00:24:17.393 => 138 tgt_run_hotplug 01:11:18.538 00:24:17.393 ... 01:11:18.538 00:24:17.393 01:11:18.538 00:24:17.393 ========== Backtrace end ========== 01:11:18.538 00:24:17.393 ========== Backtrace start: ========== 01:11:18.538 00:24:17.393 01:11:18.538 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.538 00:24:17.393 ... 01:11:18.538 00:24:17.393 11 # Pci bus hotplug 01:11:18.538 00:24:17.393 12 # Helper function to remove/attach cotrollers 01:11:18.538 00:24:17.393 13 debug_remove_attach_helper() { 01:11:18.538 00:24:17.393 14 local helper_time=0 01:11:18.538 00:24:17.393 15 01:11:18.539 00:24:17.393 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.539 00:24:17.393 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.539 00:24:17.393 18 "$helper_time" "$nvme_count" >&2 01:11:18.539 00:24:17.393 19 } 01:11:18.539 00:24:17.393 20 01:11:18.539 00:24:17.393 21 remove_attach_helper() { 01:11:18.539 00:24:17.393 ... 01:11:18.539 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.539 00:24:17.393 ... 01:11:18.539 00:24:17.393 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.539 00:24:17.393 106 done 01:11:18.539 00:24:17.393 107 01:11:18.539 00:24:17.393 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.539 00:24:17.393 109 01:11:18.539 00:24:17.393 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.539 00:24:17.393 111 # Verify reregistering hotplug poller 01:11:18.539 00:24:17.393 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.539 00:24:17.393 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.539 00:24:17.393 114 01:11:18.539 00:24:17.393 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.539 00:24:17.393 ... 01:11:18.539 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.539 00:24:17.393 ... 01:11:18.539 00:24:17.393 133 01:11:18.539 00:24:17.393 134 # Run pci bus hotplug test 01:11:18.539 00:24:17.393 135 run_hotplug 01:11:18.539 00:24:17.393 136 01:11:18.539 00:24:17.393 137 # Run SPDK target based hotplug 01:11:18.539 00:24:17.393 => 138 tgt_run_hotplug 01:11:18.539 00:24:17.393 ... 01:11:18.539 00:24:17.393 01:11:18.539 00:24:17.393 ========== Backtrace end ========== 01:11:18.539 00:24:17.393 ========== Backtrace start: ========== 01:11:18.539 00:24:17.393 01:11:18.539 00:24:17.393 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 01:11:18.539 00:24:17.393 ... 01:11:18.539 00:24:17.393 1117 timing_enter $test_name 01:11:18.539 00:24:17.393 1118 echo "************************************" 01:11:18.539 00:24:17.393 1119 echo "START TEST $test_name" 01:11:18.539 00:24:17.393 1120 echo "************************************" 01:11:18.539 00:24:17.393 1121 xtrace_restore 01:11:18.539 00:24:17.393 1122 time "$@" 01:11:18.539 00:24:17.393 1123 xtrace_disable 01:11:18.539 00:24:17.393 1124 echo "************************************" 01:11:18.539 00:24:17.393 1125 echo "END TEST $test_name" 01:11:18.539 00:24:17.393 1126 echo "************************************" 01:11:18.539 00:24:17.393 1127 timing_exit $test_name 01:11:18.539 00:24:17.393 ... 01:11:18.539 00:24:17.393 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 01:11:18.539 00:24:17.393 ... 01:11:18.539 00:24:17.393 235 01:11:18.539 00:24:17.393 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 01:11:18.539 00:24:17.393 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 01:11:18.539 00:24:17.393 238 01:11:18.539 00:24:17.393 239 if [ $(uname -s) = Linux ]; then 01:11:18.539 00:24:17.393 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 01:11:18.539 00:24:17.393 241 fi 01:11:18.539 00:24:17.393 242 01:11:18.539 00:24:17.393 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 01:11:18.539 00:24:17.393 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 01:11:18.539 00:24:17.393 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 01:11:18.539 00:24:17.393 ... 01:11:18.539 00:24:17.393 01:11:18.539 00:24:17.393 ========== Backtrace end ==========' 01:11:18.539 + for build in "${build_log_list[@]}" 01:11:18.539 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-phy-autotest_26178/build.log 01:11:18.539 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.539 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-phy-autotest_26178/build.log 01:11:18.539 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-phy-autotest_26178/build.log 01:11:18.539 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-phy-autotest_26178 01:11:18.539 + job_name=vhost-phy-autotest_26178 01:11:18.539 + start_pattern='Backtrace start' 01:11:18.539 + end_pattern='Backtrace end' 01:11:18.539 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-phy-autotest_26178/build.log ]] 01:11:18.539 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-phy-autotest_26178/build.log 01:11:18.539 + backtraces= 01:11:18.539 + [[ -n '' ]] 01:11:18.539 + for build in "${build_log_list[@]}" 01:11:18.539 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-phy-autotest_72325/build.log 01:11:18.539 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.539 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-phy-autotest_72325/build.log 01:11:18.539 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-phy-autotest_72325/build.log 01:11:18.539 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-phy-autotest_72325 01:11:18.539 + job_name=nvme-phy-autotest_72325 01:11:18.539 + start_pattern='Backtrace start' 01:11:18.539 + end_pattern='Backtrace end' 01:11:18.539 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-phy-autotest_72325/build.log ]] 01:11:18.539 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-phy-autotest_72325/build.log 01:11:18.539 + backtraces= 01:11:18.539 + [[ -n '' ]] 01:11:18.539 + for build in "${build_log_list[@]}" 01:11:18.539 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-uring-vg-autotest_77358/build.log 01:11:18.539 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.539 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-uring-vg-autotest_77358/build.log 01:11:18.539 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-uring-vg-autotest_77358/build.log 01:11:18.539 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-uring-vg-autotest_77358 01:11:18.539 + job_name=nvmf-tcp-uring-vg-autotest_77358 01:11:18.539 + start_pattern='Backtrace start' 01:11:18.539 + end_pattern='Backtrace end' 01:11:18.539 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-uring-vg-autotest_77358/build.log ]] 01:11:18.539 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-uring-vg-autotest_77358/build.log 01:11:18.539 + backtraces= 01:11:18.539 + [[ -n '' ]] 01:11:18.539 + for build in "${build_log_list[@]}" 01:11:18.539 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/unittest-vg-autotest_63370/build.log 01:11:18.539 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.539 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/unittest-vg-autotest_63370/build.log 01:11:18.539 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/unittest-vg-autotest_63370/build.log 01:11:18.539 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/unittest-vg-autotest_63370 01:11:18.539 + job_name=unittest-vg-autotest_63370 01:11:18.539 + start_pattern='Backtrace start' 01:11:18.539 + end_pattern='Backtrace end' 01:11:18.539 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/unittest-vg-autotest_63370/build.log ]] 01:11:18.539 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/unittest-vg-autotest_63370/build.log 01:11:18.539 + backtraces= 01:11:18.539 + [[ -n '' ]] 01:11:18.539 + for build in "${build_log_list[@]}" 01:11:18.539 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-phy-autotest_72284/build.log 01:11:18.539 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.539 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-phy-autotest_72284/build.log 01:11:18.539 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-phy-autotest_72284/build.log 01:11:18.539 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-phy-autotest_72284 01:11:18.539 + job_name=nvmf-tcp-phy-autotest_72284 01:11:18.539 + start_pattern='Backtrace start' 01:11:18.539 + end_pattern='Backtrace end' 01:11:18.539 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-phy-autotest_72284/build.log ]] 01:11:18.539 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-phy-autotest_72284/build.log 01:11:18.801 + backtraces='00:40:48.914 ========== Backtrace start: ========== 01:11:18.801 00:40:48.914 01:11:18.801 00:40:48.914 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/scripts/perf/pm/common:25 -> start_monitor_resources([]) 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 20 ${SUDO[MONITOR_RESOURCES_SUDO["$monitor"]]} \ 01:11:18.801 00:40:48.914 21 "$_pmdir/$monitor" -d "$PM_OUTPUTDIR" \ 01:11:18.801 00:40:48.914 22 -l -p "monitor.${0##*/}.$(date +%s)" & 01:11:18.801 00:40:48.914 23 done 01:11:18.801 00:40:48.914 24 # Give our monitors some time to setup their traps 01:11:18.801 00:40:48.914 => 25 sleep 1 01:11:18.801 00:40:48.914 26 } 01:11:18.801 00:40:48.914 27 01:11:18.801 00:40:48.914 28 stop_monitor_resources() { 01:11:18.801 00:40:48.914 29 signal_monitor_resources TERM 01:11:18.801 00:40:48.914 30 } 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autobuild_common.sh:455 -> source(["/var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autobuild_common.sh"]) 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 450 scanbuild_exclude+=" --exclude $rootdir/xnvme --exclude /tmp" 01:11:18.801 00:40:48.914 451 01:11:18.801 00:40:48.914 452 scanbuild="scan-build -o $output_dir/scan-build-tmp $scanbuild_exclude --status-bugs" 01:11:18.801 00:40:48.914 453 config_params=$(get_config_params) 01:11:18.801 00:40:48.914 454 01:11:18.801 00:40:48.914 => 455 start_monitor_resources 01:11:18.801 00:40:48.914 456 trap '\''stop_monitor_resources'\'' EXIT 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/autopackage.sh:8 -> main(["/var/jenkins/workspace/nvmf-tcp-phy-autotest/autorun-spdk.conf"]) 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 3 # Copyright (C) 2015 Intel Corporation 01:11:18.801 00:40:48.914 4 # All rights reserved. 01:11:18.801 00:40:48.914 5 # 01:11:18.801 00:40:48.914 6 01:11:18.801 00:40:48.914 7 rootdir=$(readlink -f $(dirname $0)) 01:11:18.801 00:40:48.914 => 8 source "$rootdir/test/common/autobuild_common.sh" 01:11:18.801 00:40:48.914 9 01:11:18.801 00:40:48.914 10 MAKEFLAGS=${MAKEFLAGS:--j16} 01:11:18.801 00:40:48.914 11 cd $rootdir 01:11:18.801 00:40:48.914 12 01:11:18.801 00:40:48.914 13 if [[ $SPDK_TEST_RELEASE_BUILD -eq 1 ]]; then 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 01:11:18.801 00:40:48.914 ========== Backtrace end ==========' 01:11:18.801 + [[ -n 00:40:48.914 ========== Backtrace start: ========== 01:11:18.801 00:40:48.914 01:11:18.801 00:40:48.914 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/scripts/perf/pm/common:25 -> start_monitor_resources([]) 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 20 ${SUDO[MONITOR_RESOURCES_SUDO["$monitor"]]} \ 01:11:18.801 00:40:48.914 21 "$_pmdir/$monitor" -d "$PM_OUTPUTDIR" \ 01:11:18.801 00:40:48.914 22 -l -p "monitor.${0##*/}.$(date +%s)" & 01:11:18.801 00:40:48.914 23 done 01:11:18.801 00:40:48.914 24 # Give our monitors some time to setup their traps 01:11:18.801 00:40:48.914 => 25 sleep 1 01:11:18.801 00:40:48.914 26 } 01:11:18.801 00:40:48.914 27 01:11:18.801 00:40:48.914 28 stop_monitor_resources() { 01:11:18.801 00:40:48.914 29 signal_monitor_resources TERM 01:11:18.801 00:40:48.914 30 } 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autobuild_common.sh:455 -> source(["/var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autobuild_common.sh"]) 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 450 scanbuild_exclude+=" --exclude $rootdir/xnvme --exclude /tmp" 01:11:18.801 00:40:48.914 451 01:11:18.801 00:40:48.914 452 scanbuild="scan-build -o $output_dir/scan-build-tmp $scanbuild_exclude --status-bugs" 01:11:18.801 00:40:48.914 453 config_params=$(get_config_params) 01:11:18.801 00:40:48.914 454 01:11:18.801 00:40:48.914 => 455 start_monitor_resources 01:11:18.801 00:40:48.914 456 trap 'stop_monitor_resources' EXIT 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/autopackage.sh:8 -> main(["/var/jenkins/workspace/nvmf-tcp-phy-autotest/autorun-spdk.conf"]) 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 3 # Copyright (C) 2015 Intel Corporation 01:11:18.801 00:40:48.914 4 # All rights reserved. 01:11:18.801 00:40:48.914 5 # 01:11:18.801 00:40:48.914 6 01:11:18.801 00:40:48.914 7 rootdir=$(readlink -f $(dirname $0)) 01:11:18.801 00:40:48.914 => 8 source "$rootdir/test/common/autobuild_common.sh" 01:11:18.801 00:40:48.914 9 01:11:18.801 00:40:48.914 10 MAKEFLAGS=${MAKEFLAGS:--j16} 01:11:18.801 00:40:48.914 11 cd $rootdir 01:11:18.801 00:40:48.914 12 01:11:18.801 00:40:48.914 13 if [[ $SPDK_TEST_RELEASE_BUILD -eq 1 ]]; then 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 01:11:18.801 00:40:48.914 ========== Backtrace end ========== ]] 01:11:18.801 + echo '00:40:48.914 ========== Backtrace start: ========== 01:11:18.801 00:40:48.914 01:11:18.801 00:40:48.914 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/scripts/perf/pm/common:25 -> start_monitor_resources([]) 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 20 ${SUDO[MONITOR_RESOURCES_SUDO["$monitor"]]} \ 01:11:18.801 00:40:48.914 21 "$_pmdir/$monitor" -d "$PM_OUTPUTDIR" \ 01:11:18.801 00:40:48.914 22 -l -p "monitor.${0##*/}.$(date +%s)" & 01:11:18.801 00:40:48.914 23 done 01:11:18.801 00:40:48.914 24 # Give our monitors some time to setup their traps 01:11:18.801 00:40:48.914 => 25 sleep 1 01:11:18.801 00:40:48.914 26 } 01:11:18.801 00:40:48.914 27 01:11:18.801 00:40:48.914 28 stop_monitor_resources() { 01:11:18.801 00:40:48.914 29 signal_monitor_resources TERM 01:11:18.801 00:40:48.914 30 } 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autobuild_common.sh:455 -> source(["/var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autobuild_common.sh"]) 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 450 scanbuild_exclude+=" --exclude $rootdir/xnvme --exclude /tmp" 01:11:18.801 00:40:48.914 451 01:11:18.801 00:40:48.914 452 scanbuild="scan-build -o $output_dir/scan-build-tmp $scanbuild_exclude --status-bugs" 01:11:18.801 00:40:48.914 453 config_params=$(get_config_params) 01:11:18.801 00:40:48.914 454 01:11:18.801 00:40:48.914 => 455 start_monitor_resources 01:11:18.801 00:40:48.914 456 trap '\''stop_monitor_resources'\'' EXIT 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/autopackage.sh:8 -> main(["/var/jenkins/workspace/nvmf-tcp-phy-autotest/autorun-spdk.conf"]) 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 3 # Copyright (C) 2015 Intel Corporation 01:11:18.801 00:40:48.914 4 # All rights reserved. 01:11:18.801 00:40:48.914 5 # 01:11:18.801 00:40:48.914 6 01:11:18.801 00:40:48.914 7 rootdir=$(readlink -f $(dirname $0)) 01:11:18.801 00:40:48.914 => 8 source "$rootdir/test/common/autobuild_common.sh" 01:11:18.801 00:40:48.914 9 01:11:18.801 00:40:48.914 10 MAKEFLAGS=${MAKEFLAGS:--j16} 01:11:18.801 00:40:48.914 11 cd $rootdir 01:11:18.801 00:40:48.914 12 01:11:18.801 00:40:48.914 13 if [[ $SPDK_TEST_RELEASE_BUILD -eq 1 ]]; then 01:11:18.801 00:40:48.914 ... 01:11:18.801 00:40:48.914 01:11:18.801 00:40:48.914 ========== Backtrace end ==========' 01:11:18.801 + for build in "${build_log_list[@]}" 01:11:18.801 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-vg-autotest_77424/build.log 01:11:18.801 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.801 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-vg-autotest_77424/build.log 01:11:18.801 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-vg-autotest_77424/build.log 01:11:18.801 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-vg-autotest_77424 01:11:18.801 + job_name=nvmf-tcp-vg-autotest_77424 01:11:18.801 + start_pattern='Backtrace start' 01:11:18.801 + end_pattern='Backtrace end' 01:11:18.801 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-vg-autotest_77424/build.log ]] 01:11:18.801 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-vg-autotest_77424/build.log 01:11:18.801 + backtraces= 01:11:18.801 + [[ -n '' ]] 01:11:18.801 + for build in "${build_log_list[@]}" 01:11:18.801 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vfio-user-phy-autotest_26161/build.log 01:11:18.801 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.801 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vfio-user-phy-autotest_26161/build.log 01:11:18.801 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vfio-user-phy-autotest_26161/build.log 01:11:18.801 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vfio-user-phy-autotest_26161 01:11:18.801 + job_name=vfio-user-phy-autotest_26161 01:11:18.801 + start_pattern='Backtrace start' 01:11:18.801 + end_pattern='Backtrace end' 01:11:18.801 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vfio-user-phy-autotest_26161/build.log ]] 01:11:18.801 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vfio-user-phy-autotest_26161/build.log 01:11:18.801 + backtraces= 01:11:18.801 + [[ -n '' ]] 01:11:18.801 + for build in "${build_log_list[@]}" 01:11:18.801 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/valgrind-vg-autotest_49187/build.log 01:11:18.801 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.801 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/valgrind-vg-autotest_49187/build.log 01:11:18.801 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/valgrind-vg-autotest_49187/build.log 01:11:18.801 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/valgrind-vg-autotest_49187 01:11:18.801 + job_name=valgrind-vg-autotest_49187 01:11:18.801 + start_pattern='Backtrace start' 01:11:18.801 + end_pattern='Backtrace end' 01:11:18.801 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/valgrind-vg-autotest_49187/build.log ]] 01:11:18.801 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/valgrind-vg-autotest_49187/build.log 01:11:18.801 + backtraces= 01:11:18.801 + [[ -n '' ]] 01:11:18.801 + for build in "${build_log_list[@]}" 01:11:18.801 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/lvol-vg-autotest_77382/build.log 01:11:18.801 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.801 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/lvol-vg-autotest_77382/build.log 01:11:18.801 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/lvol-vg-autotest_77382/build.log 01:11:18.801 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/lvol-vg-autotest_77382 01:11:18.801 + job_name=lvol-vg-autotest_77382 01:11:18.801 + start_pattern='Backtrace start' 01:11:18.801 + end_pattern='Backtrace end' 01:11:18.801 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/lvol-vg-autotest_77382/build.log ]] 01:11:18.801 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/lvol-vg-autotest_77382/build.log 01:11:18.801 + backtraces= 01:11:18.801 + [[ -n '' ]] 01:11:18.801 + for build in "${build_log_list[@]}" 01:11:18.801 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vmd-phy-autotest_2614/build.log 01:11:18.801 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.801 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vmd-phy-autotest_2614/build.log 01:11:18.801 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vmd-phy-autotest_2614/build.log 01:11:18.801 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vmd-phy-autotest_2614 01:11:18.801 + job_name=vmd-phy-autotest_2614 01:11:18.801 + start_pattern='Backtrace start' 01:11:18.801 + end_pattern='Backtrace end' 01:11:18.801 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vmd-phy-autotest_2614/build.log ]] 01:11:18.801 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vmd-phy-autotest_2614/build.log 01:11:18.801 + backtraces= 01:11:18.801 + [[ -n '' ]] 01:11:18.801 + for build in "${build_log_list[@]}" 01:11:18.801 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/BlobFS-phy-autotest_26159/build.log 01:11:18.801 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.801 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/BlobFS-phy-autotest_26159/build.log 01:11:18.801 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/BlobFS-phy-autotest_26159/build.log 01:11:18.801 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/BlobFS-phy-autotest_26159 01:11:18.801 + job_name=BlobFS-phy-autotest_26159 01:11:18.801 + start_pattern='Backtrace start' 01:11:18.801 + end_pattern='Backtrace end' 01:11:18.801 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/BlobFS-phy-autotest_26159/build.log ]] 01:11:18.801 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/BlobFS-phy-autotest_26159/build.log 01:11:18.801 + backtraces= 01:11:18.801 + [[ -n '' ]] 01:11:18.801 + for build in "${build_log_list[@]}" 01:11:18.802 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu20-vg-autotest_73752/build.log 01:11:18.802 + local build_log job_name start_pattern end_pattern backtraces 01:11:18.802 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu20-vg-autotest_73752/build.log 01:11:18.802 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu20-vg-autotest_73752/build.log 01:11:18.802 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu20-vg-autotest_73752 01:11:18.802 + job_name=ubuntu20-vg-autotest_73752 01:11:18.802 + start_pattern='Backtrace start' 01:11:18.802 + end_pattern='Backtrace end' 01:11:18.802 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu20-vg-autotest_73752/build.log ]] 01:11:18.802 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu20-vg-autotest_73752/build.log 01:11:18.802 + backtraces='00:55:59.653 01:22:01 sw_hotplug -- common/autotest_common.sh@715 -- # time='\''========== Backtrace start: ========== 01:11:18.802 00:55:59.653 01:11:18.802 00:55:59.653 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.802 00:55:59.653 ... 01:11:18.802 00:55:59.653 710 01:11:18.802 00:55:59.653 711 # We redirect cmd'\''\'\'''\''s std{out,err} to a separate fd dup'\''\'\'''\''ed to stdin'\''\'\'''\''s device (or /dev/null) to 01:11:18.802 00:55:59.653 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:11:18.802 00:55:59.653 713 # but $() will return just the time'\''\'\'''\''s data, hence making it possible to just do: 01:11:18.802 00:55:59.653 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.802 00:55:59.653 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.802 00:55:59.653 716 01:11:18.802 00:55:59.653 717 echo "$time" 01:11:18.802 00:55:59.653 718 ) 01:11:18.802 00:55:59.653 719 01:11:18.802 00:55:59.653 720 function timing_enter() { 01:11:18.802 00:55:59.653 ... 01:11:18.802 00:55:59.653 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.802 00:55:59.653 ... 01:11:18.802 00:55:59.653 11 # Pci bus hotplug 01:11:18.802 00:55:59.653 12 # Helper function to remove/attach cotrollers 01:11:18.802 00:55:59.653 13 debug_remove_attach_helper() { 01:11:18.802 00:55:59.653 14 local helper_time=0 01:11:18.802 00:55:59.653 15 01:11:18.802 00:55:59.653 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.802 00:55:59.653 17 printf '\''\'\'''\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\''\'\'''\'' \ 01:11:18.802 00:55:59.653 18 "$helper_time" "$nvme_count" >&2 01:11:18.802 00:55:59.653 19 } 01:11:18.802 00:55:59.653 20 01:11:18.802 00:55:59.653 21 remove_attach_helper() { 01:11:18.802 00:55:59.653 ... 01:11:18.802 00:55:59.653 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.802 00:55:59.653 ... 01:11:18.802 00:55:59.653 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.802 00:55:59.653 106 done 01:11:18.802 00:55:59.653 107 01:11:18.802 00:55:59.653 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.802 00:55:59.653 109 01:11:18.802 00:55:59.653 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.802 00:55:59.653 111 # Verify reregistering hotplug poller 01:11:18.802 00:55:59.653 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.802 00:55:59.653 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.802 00:55:59.653 114 01:11:18.802 00:55:59.653 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.802 00:55:59.653 ... 01:11:18.802 00:55:59.653 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.802 00:55:59.653 ... 01:11:18.802 00:55:59.653 133 01:11:18.802 00:55:59.653 134 # Run pci bus hotplug test 01:11:18.802 00:55:59.653 135 run_hotplug 01:11:18.802 00:55:59.653 136 01:11:18.802 00:55:59.653 137 # Run SPDK target based hotplug 01:11:18.802 00:55:59.654 => 138 tgt_run_hotplug 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 01:11:18.802 00:55:59.654 ========== Backtrace end ========== 01:11:18.802 00:55:59.654 ========== Backtrace start: ========== 01:11:18.802 00:55:59.654 01:11:18.802 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 710 01:11:18.802 00:55:59.654 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 01:11:18.802 00:55:59.654 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:11:18.802 00:55:59.654 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:11:18.802 00:55:59.654 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.802 00:55:59.654 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.802 00:55:59.654 716 01:11:18.802 00:55:59.654 717 echo "$time" 01:11:18.802 00:55:59.654 718 ) 01:11:18.802 00:55:59.654 719 01:11:18.802 00:55:59.654 720 function timing_enter() { 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 11 # Pci bus hotplug 01:11:18.802 00:55:59.654 12 # Helper function to remove/attach cotrollers 01:11:18.802 00:55:59.654 13 debug_remove_attach_helper() { 01:11:18.802 00:55:59.654 14 local helper_time=0 01:11:18.802 00:55:59.654 15 01:11:18.802 00:55:59.654 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.802 00:55:59.654 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.802 00:55:59.654 18 "$helper_time" "$nvme_count" >&2 01:11:18.802 00:55:59.654 19 } 01:11:18.802 00:55:59.654 20 01:11:18.802 00:55:59.654 21 remove_attach_helper() { 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.802 00:55:59.654 106 done 01:11:18.802 00:55:59.654 107 01:11:18.802 00:55:59.654 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.802 00:55:59.654 109 01:11:18.802 00:55:59.654 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.802 00:55:59.654 111 # Verify reregistering hotplug poller 01:11:18.802 00:55:59.654 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.802 00:55:59.654 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.802 00:55:59.654 114 01:11:18.802 00:55:59.654 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 133 01:11:18.802 00:55:59.654 134 # Run pci bus hotplug test 01:11:18.802 00:55:59.654 135 run_hotplug 01:11:18.802 00:55:59.654 136 01:11:18.802 00:55:59.654 137 # Run SPDK target based hotplug 01:11:18.802 00:55:59.654 => 138 tgt_run_hotplug 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 01:11:18.802 00:55:59.654 ========== Backtrace end ========== 01:11:18.802 00:55:59.654 ========== Backtrace start: ========== 01:11:18.802 00:55:59.654 01:11:18.802 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:11:18.802 00:55:59.654 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.802 00:55:59.654 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.802 00:55:59.654 716 01:11:18.802 00:55:59.654 717 echo "$time" 01:11:18.802 00:55:59.654 => 718 ) 01:11:18.802 00:55:59.654 719 01:11:18.802 00:55:59.654 720 function timing_enter() { 01:11:18.802 00:55:59.654 721 xtrace_disable 01:11:18.802 00:55:59.654 722 timing "enter" "$1" 01:11:18.802 00:55:59.654 723 xtrace_restore 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 11 # Pci bus hotplug 01:11:18.802 00:55:59.654 12 # Helper function to remove/attach cotrollers 01:11:18.802 00:55:59.654 13 debug_remove_attach_helper() { 01:11:18.802 00:55:59.654 14 local helper_time=0 01:11:18.802 00:55:59.654 15 01:11:18.802 00:55:59.654 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.802 00:55:59.654 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.802 00:55:59.654 18 "$helper_time" "$nvme_count" >&2 01:11:18.802 00:55:59.654 19 } 01:11:18.802 00:55:59.654 20 01:11:18.802 00:55:59.654 21 remove_attach_helper() { 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.802 00:55:59.654 106 done 01:11:18.802 00:55:59.654 107 01:11:18.802 00:55:59.654 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.802 00:55:59.654 109 01:11:18.802 00:55:59.654 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.802 00:55:59.654 111 # Verify reregistering hotplug poller 01:11:18.802 00:55:59.654 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.802 00:55:59.654 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.802 00:55:59.654 114 01:11:18.802 00:55:59.654 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 133 01:11:18.802 00:55:59.654 134 # Run pci bus hotplug test 01:11:18.802 00:55:59.654 135 run_hotplug 01:11:18.802 00:55:59.654 136 01:11:18.802 00:55:59.654 137 # Run SPDK target based hotplug 01:11:18.802 00:55:59.654 => 138 tgt_run_hotplug 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 01:11:18.802 00:55:59.654 ========== Backtrace end ========== 01:11:18.802 00:55:59.654 ========== Backtrace start: ========== 01:11:18.802 00:55:59.654 01:11:18.802 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 11 # Pci bus hotplug 01:11:18.802 00:55:59.654 12 # Helper function to remove/attach cotrollers 01:11:18.802 00:55:59.654 13 debug_remove_attach_helper() { 01:11:18.802 00:55:59.654 14 local helper_time=0 01:11:18.802 00:55:59.654 15 01:11:18.802 00:55:59.654 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.802 00:55:59.654 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.802 00:55:59.654 18 "$helper_time" "$nvme_count" >&2 01:11:18.802 00:55:59.654 19 } 01:11:18.802 00:55:59.654 20 01:11:18.802 00:55:59.654 21 remove_attach_helper() { 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.802 00:55:59.654 106 done 01:11:18.802 00:55:59.654 107 01:11:18.802 00:55:59.654 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.802 00:55:59.654 109 01:11:18.802 00:55:59.654 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.802 00:55:59.654 111 # Verify reregistering hotplug poller 01:11:18.802 00:55:59.654 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.802 00:55:59.654 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.802 00:55:59.654 114 01:11:18.802 00:55:59.654 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 133 01:11:18.802 00:55:59.654 134 # Run pci bus hotplug test 01:11:18.802 00:55:59.654 135 run_hotplug 01:11:18.802 00:55:59.654 136 01:11:18.802 00:55:59.654 137 # Run SPDK target based hotplug 01:11:18.802 00:55:59.654 => 138 tgt_run_hotplug 01:11:18.802 00:55:59.654 ... 01:11:18.802 00:55:59.654 01:11:18.802 00:55:59.654 ========== Backtrace end ========== 01:11:18.802 00:55:59.655 ========== Backtrace start: ========== 01:11:18.802 00:55:59.655 01:11:18.802 00:55:59.655 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 01:11:18.802 00:55:59.655 ... 01:11:18.802 00:55:59.655 1117 timing_enter $test_name 01:11:18.802 00:55:59.655 1118 echo "************************************" 01:11:18.802 00:55:59.655 1119 echo "START TEST $test_name" 01:11:18.802 00:55:59.655 1120 echo "************************************" 01:11:18.802 00:55:59.655 1121 xtrace_restore 01:11:18.802 00:55:59.655 1122 time "$@" 01:11:18.802 00:55:59.655 1123 xtrace_disable 01:11:18.802 00:55:59.655 1124 echo "************************************" 01:11:18.802 00:55:59.655 1125 echo "END TEST $test_name" 01:11:18.802 00:55:59.655 1126 echo "************************************" 01:11:18.802 00:55:59.655 1127 timing_exit $test_name 01:11:18.802 00:55:59.655 ... 01:11:18.802 00:55:59.655 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 01:11:18.802 00:55:59.655 ... 01:11:18.802 00:55:59.655 235 01:11:18.802 00:55:59.655 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 01:11:18.802 00:55:59.655 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 01:11:18.802 00:55:59.655 238 01:11:18.802 00:55:59.655 239 if [ $(uname -s) = Linux ]; then 01:11:18.802 00:55:59.655 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 01:11:18.802 00:55:59.655 241 fi 01:11:18.802 00:55:59.655 242 01:11:18.802 00:55:59.655 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 01:11:18.802 00:55:59.655 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 01:11:18.802 00:55:59.655 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 01:11:18.802 00:55:59.655 ... 01:11:18.802 00:55:59.655 01:11:18.802 00:55:59.655 ========== Backtrace end ==========' 01:11:18.802 + [[ -n 00:55:59.653 01:22:01 sw_hotplug -- common/autotest_common.sh@715 -- # time='========== Backtrace start: ========== 01:11:18.802 00:55:59.653 01:11:18.802 00:55:59.653 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.802 00:55:59.653 ... 01:11:18.802 00:55:59.653 710 01:11:18.802 00:55:59.653 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 01:11:18.802 00:55:59.653 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:11:18.802 00:55:59.653 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:11:18.802 00:55:59.653 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.802 00:55:59.653 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.802 00:55:59.653 716 01:11:18.802 00:55:59.653 717 echo "$time" 01:11:18.802 00:55:59.653 718 ) 01:11:18.802 00:55:59.653 719 01:11:18.802 00:55:59.653 720 function timing_enter() { 01:11:18.802 00:55:59.653 ... 01:11:18.802 00:55:59.653 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.802 00:55:59.653 ... 01:11:18.802 00:55:59.653 11 # Pci bus hotplug 01:11:18.802 00:55:59.653 12 # Helper function to remove/attach cotrollers 01:11:18.802 00:55:59.653 13 debug_remove_attach_helper() { 01:11:18.803 00:55:59.653 14 local helper_time=0 01:11:18.803 00:55:59.653 15 01:11:18.803 00:55:59.653 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.803 00:55:59.653 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.803 00:55:59.653 18 "$helper_time" "$nvme_count" >&2 01:11:18.803 00:55:59.653 19 } 01:11:18.803 00:55:59.653 20 01:11:18.803 00:55:59.653 21 remove_attach_helper() { 01:11:18.803 00:55:59.653 ... 01:11:18.803 00:55:59.653 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.803 00:55:59.653 ... 01:11:18.803 00:55:59.653 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.803 00:55:59.653 106 done 01:11:18.803 00:55:59.653 107 01:11:18.803 00:55:59.653 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.803 00:55:59.653 109 01:11:18.803 00:55:59.653 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.803 00:55:59.653 111 # Verify reregistering hotplug poller 01:11:18.803 00:55:59.653 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.803 00:55:59.653 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.803 00:55:59.653 114 01:11:18.803 00:55:59.653 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.803 00:55:59.653 ... 01:11:18.803 00:55:59.653 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.803 00:55:59.653 ... 01:11:18.803 00:55:59.653 133 01:11:18.803 00:55:59.653 134 # Run pci bus hotplug test 01:11:18.803 00:55:59.653 135 run_hotplug 01:11:18.803 00:55:59.653 136 01:11:18.803 00:55:59.653 137 # Run SPDK target based hotplug 01:11:18.803 00:55:59.654 => 138 tgt_run_hotplug 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 01:11:18.803 00:55:59.654 ========== Backtrace end ========== 01:11:18.803 00:55:59.654 ========== Backtrace start: ========== 01:11:18.803 00:55:59.654 01:11:18.803 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 710 01:11:18.803 00:55:59.654 711 # We redirect cmd's std{out,err} to a separate fd dup'ed to stdin's device (or /dev/null) to 01:11:18.803 00:55:59.654 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:11:18.803 00:55:59.654 713 # but $() will return just the time's data, hence making it possible to just do: 01:11:18.803 00:55:59.654 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.803 00:55:59.654 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.803 00:55:59.654 716 01:11:18.803 00:55:59.654 717 echo "$time" 01:11:18.803 00:55:59.654 718 ) 01:11:18.803 00:55:59.654 719 01:11:18.803 00:55:59.654 720 function timing_enter() { 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 11 # Pci bus hotplug 01:11:18.803 00:55:59.654 12 # Helper function to remove/attach cotrollers 01:11:18.803 00:55:59.654 13 debug_remove_attach_helper() { 01:11:18.803 00:55:59.654 14 local helper_time=0 01:11:18.803 00:55:59.654 15 01:11:18.803 00:55:59.654 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.803 00:55:59.654 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 01:11:18.803 00:55:59.654 18 "$helper_time" "$nvme_count" >&2 01:11:18.803 00:55:59.654 19 } 01:11:18.803 00:55:59.654 20 01:11:18.803 00:55:59.654 21 remove_attach_helper() { 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.803 00:55:59.654 106 done 01:11:18.803 00:55:59.654 107 01:11:18.803 00:55:59.654 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.803 00:55:59.654 109 01:11:18.803 00:55:59.654 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.803 00:55:59.654 111 # Verify reregistering hotplug poller 01:11:18.803 00:55:59.654 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.803 00:55:59.654 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.803 00:55:59.654 114 01:11:18.803 00:55:59.654 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 133 01:11:18.803 00:55:59.654 134 # Run pci bus hotplug test 01:11:18.803 00:55:59.654 135 run_hotplug 01:11:18.803 00:55:59.654 136 01:11:18.803 00:55:59.654 137 # Run SPDK target based hotplug 01:11:18.803 00:55:59.654 => 138 tgt_run_hotplug 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 01:11:18.803 00:55:59.654 ========== Backtrace end ========== 01:11:18.803 00:55:59.654 ========== Backtrace start: ========== 01:11:18.803 00:55:59.654 01:11:18.803 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 713 # but $() will return just the time's data, hence making it possible to just do: 01:11:18.803 00:55:59.654 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.803 00:55:59.654 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.803 00:55:59.654 716 01:11:18.803 00:55:59.654 717 echo "$time" 01:11:18.803 00:55:59.654 => 718 ) 01:11:18.803 00:55:59.654 719 01:11:18.803 00:55:59.654 720 function timing_enter() { 01:11:18.803 00:55:59.654 721 xtrace_disable 01:11:18.803 00:55:59.654 722 timing "enter" "$1" 01:11:18.803 00:55:59.654 723 xtrace_restore 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 11 # Pci bus hotplug 01:11:18.803 00:55:59.654 12 # Helper function to remove/attach cotrollers 01:11:18.803 00:55:59.654 13 debug_remove_attach_helper() { 01:11:18.803 00:55:59.654 14 local helper_time=0 01:11:18.803 00:55:59.654 15 01:11:18.803 00:55:59.654 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.803 00:55:59.654 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 01:11:18.803 00:55:59.654 18 "$helper_time" "$nvme_count" >&2 01:11:18.803 00:55:59.654 19 } 01:11:18.803 00:55:59.654 20 01:11:18.803 00:55:59.654 21 remove_attach_helper() { 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.803 00:55:59.654 106 done 01:11:18.803 00:55:59.654 107 01:11:18.803 00:55:59.654 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.803 00:55:59.654 109 01:11:18.803 00:55:59.654 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.803 00:55:59.654 111 # Verify reregistering hotplug poller 01:11:18.803 00:55:59.654 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.803 00:55:59.654 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.803 00:55:59.654 114 01:11:18.803 00:55:59.654 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 133 01:11:18.803 00:55:59.654 134 # Run pci bus hotplug test 01:11:18.803 00:55:59.654 135 run_hotplug 01:11:18.803 00:55:59.654 136 01:11:18.803 00:55:59.654 137 # Run SPDK target based hotplug 01:11:18.803 00:55:59.654 => 138 tgt_run_hotplug 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 01:11:18.803 00:55:59.654 ========== Backtrace end ========== 01:11:18.803 00:55:59.654 ========== Backtrace start: ========== 01:11:18.803 00:55:59.654 01:11:18.803 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 11 # Pci bus hotplug 01:11:18.803 00:55:59.654 12 # Helper function to remove/attach cotrollers 01:11:18.803 00:55:59.654 13 debug_remove_attach_helper() { 01:11:18.803 00:55:59.654 14 local helper_time=0 01:11:18.803 00:55:59.654 15 01:11:18.803 00:55:59.654 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.803 00:55:59.654 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 01:11:18.803 00:55:59.654 18 "$helper_time" "$nvme_count" >&2 01:11:18.803 00:55:59.654 19 } 01:11:18.803 00:55:59.654 20 01:11:18.803 00:55:59.654 21 remove_attach_helper() { 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.803 00:55:59.654 106 done 01:11:18.803 00:55:59.654 107 01:11:18.803 00:55:59.654 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.803 00:55:59.654 109 01:11:18.803 00:55:59.654 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.803 00:55:59.654 111 # Verify reregistering hotplug poller 01:11:18.803 00:55:59.654 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.803 00:55:59.654 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.803 00:55:59.654 114 01:11:18.803 00:55:59.654 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 133 01:11:18.803 00:55:59.654 134 # Run pci bus hotplug test 01:11:18.803 00:55:59.654 135 run_hotplug 01:11:18.803 00:55:59.654 136 01:11:18.803 00:55:59.654 137 # Run SPDK target based hotplug 01:11:18.803 00:55:59.654 => 138 tgt_run_hotplug 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 01:11:18.803 00:55:59.654 ========== Backtrace end ========== 01:11:18.803 00:55:59.655 ========== Backtrace start: ========== 01:11:18.803 00:55:59.655 01:11:18.803 00:55:59.655 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 01:11:18.803 00:55:59.655 ... 01:11:18.803 00:55:59.655 1117 timing_enter $test_name 01:11:18.803 00:55:59.655 1118 echo "************************************" 01:11:18.803 00:55:59.655 1119 echo "START TEST $test_name" 01:11:18.803 00:55:59.655 1120 echo "************************************" 01:11:18.803 00:55:59.655 1121 xtrace_restore 01:11:18.803 00:55:59.655 1122 time "$@" 01:11:18.803 00:55:59.655 1123 xtrace_disable 01:11:18.803 00:55:59.655 1124 echo "************************************" 01:11:18.803 00:55:59.655 1125 echo "END TEST $test_name" 01:11:18.803 00:55:59.655 1126 echo "************************************" 01:11:18.803 00:55:59.655 1127 timing_exit $test_name 01:11:18.803 00:55:59.655 ... 01:11:18.803 00:55:59.655 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 01:11:18.803 00:55:59.655 ... 01:11:18.803 00:55:59.655 235 01:11:18.803 00:55:59.655 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 01:11:18.803 00:55:59.655 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 01:11:18.803 00:55:59.655 238 01:11:18.803 00:55:59.655 239 if [ $(uname -s) = Linux ]; then 01:11:18.803 00:55:59.655 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 01:11:18.803 00:55:59.655 241 fi 01:11:18.803 00:55:59.655 242 01:11:18.803 00:55:59.655 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 01:11:18.803 00:55:59.655 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 01:11:18.803 00:55:59.655 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 01:11:18.803 00:55:59.655 ... 01:11:18.803 00:55:59.655 01:11:18.803 00:55:59.655 ========== Backtrace end ========== ]] 01:11:18.803 + echo '00:55:59.653 01:22:01 sw_hotplug -- common/autotest_common.sh@715 -- # time='\''========== Backtrace start: ========== 01:11:18.803 00:55:59.653 01:11:18.803 00:55:59.653 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.803 00:55:59.653 ... 01:11:18.803 00:55:59.653 710 01:11:18.803 00:55:59.653 711 # We redirect cmd'\''\'\'''\''s std{out,err} to a separate fd dup'\''\'\'''\''ed to stdin'\''\'\'''\''s device (or /dev/null) to 01:11:18.803 00:55:59.653 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:11:18.803 00:55:59.653 713 # but $() will return just the time'\''\'\'''\''s data, hence making it possible to just do: 01:11:18.803 00:55:59.653 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.803 00:55:59.653 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.803 00:55:59.653 716 01:11:18.803 00:55:59.653 717 echo "$time" 01:11:18.803 00:55:59.653 718 ) 01:11:18.803 00:55:59.653 719 01:11:18.803 00:55:59.653 720 function timing_enter() { 01:11:18.803 00:55:59.653 ... 01:11:18.803 00:55:59.653 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.803 00:55:59.653 ... 01:11:18.803 00:55:59.653 11 # Pci bus hotplug 01:11:18.803 00:55:59.653 12 # Helper function to remove/attach cotrollers 01:11:18.803 00:55:59.653 13 debug_remove_attach_helper() { 01:11:18.803 00:55:59.653 14 local helper_time=0 01:11:18.803 00:55:59.653 15 01:11:18.803 00:55:59.653 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.803 00:55:59.653 17 printf '\''\'\'''\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\''\'\'''\'' \ 01:11:18.803 00:55:59.653 18 "$helper_time" "$nvme_count" >&2 01:11:18.803 00:55:59.653 19 } 01:11:18.803 00:55:59.653 20 01:11:18.803 00:55:59.653 21 remove_attach_helper() { 01:11:18.803 00:55:59.653 ... 01:11:18.803 00:55:59.653 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.803 00:55:59.653 ... 01:11:18.803 00:55:59.653 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.803 00:55:59.653 106 done 01:11:18.803 00:55:59.653 107 01:11:18.803 00:55:59.653 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.803 00:55:59.653 109 01:11:18.803 00:55:59.653 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.803 00:55:59.653 111 # Verify reregistering hotplug poller 01:11:18.803 00:55:59.653 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.803 00:55:59.653 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.803 00:55:59.653 114 01:11:18.803 00:55:59.653 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.803 00:55:59.653 ... 01:11:18.803 00:55:59.653 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.803 00:55:59.653 ... 01:11:18.803 00:55:59.653 133 01:11:18.803 00:55:59.653 134 # Run pci bus hotplug test 01:11:18.803 00:55:59.653 135 run_hotplug 01:11:18.803 00:55:59.653 136 01:11:18.803 00:55:59.653 137 # Run SPDK target based hotplug 01:11:18.803 00:55:59.654 => 138 tgt_run_hotplug 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 01:11:18.803 00:55:59.654 ========== Backtrace end ========== 01:11:18.803 00:55:59.654 ========== Backtrace start: ========== 01:11:18.803 00:55:59.654 01:11:18.803 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.803 00:55:59.654 ... 01:11:18.803 00:55:59.654 710 01:11:18.804 00:55:59.654 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 01:11:18.804 00:55:59.654 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:11:18.804 00:55:59.654 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:11:18.804 00:55:59.654 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.804 00:55:59.654 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.804 00:55:59.654 716 01:11:18.804 00:55:59.654 717 echo "$time" 01:11:18.804 00:55:59.654 718 ) 01:11:18.804 00:55:59.654 719 01:11:18.804 00:55:59.654 720 function timing_enter() { 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 11 # Pci bus hotplug 01:11:18.804 00:55:59.654 12 # Helper function to remove/attach cotrollers 01:11:18.804 00:55:59.654 13 debug_remove_attach_helper() { 01:11:18.804 00:55:59.654 14 local helper_time=0 01:11:18.804 00:55:59.654 15 01:11:18.804 00:55:59.654 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.804 00:55:59.654 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.804 00:55:59.654 18 "$helper_time" "$nvme_count" >&2 01:11:18.804 00:55:59.654 19 } 01:11:18.804 00:55:59.654 20 01:11:18.804 00:55:59.654 21 remove_attach_helper() { 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.804 00:55:59.654 106 done 01:11:18.804 00:55:59.654 107 01:11:18.804 00:55:59.654 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.804 00:55:59.654 109 01:11:18.804 00:55:59.654 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.804 00:55:59.654 111 # Verify reregistering hotplug poller 01:11:18.804 00:55:59.654 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.804 00:55:59.654 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.804 00:55:59.654 114 01:11:18.804 00:55:59.654 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 133 01:11:18.804 00:55:59.654 134 # Run pci bus hotplug test 01:11:18.804 00:55:59.654 135 run_hotplug 01:11:18.804 00:55:59.654 136 01:11:18.804 00:55:59.654 137 # Run SPDK target based hotplug 01:11:18.804 00:55:59.654 => 138 tgt_run_hotplug 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 01:11:18.804 00:55:59.654 ========== Backtrace end ========== 01:11:18.804 00:55:59.654 ========== Backtrace start: ========== 01:11:18.804 00:55:59.654 01:11:18.804 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:11:18.804 00:55:59.654 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:11:18.804 00:55:59.654 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:11:18.804 00:55:59.654 716 01:11:18.804 00:55:59.654 717 echo "$time" 01:11:18.804 00:55:59.654 => 718 ) 01:11:18.804 00:55:59.654 719 01:11:18.804 00:55:59.654 720 function timing_enter() { 01:11:18.804 00:55:59.654 721 xtrace_disable 01:11:18.804 00:55:59.654 722 timing "enter" "$1" 01:11:18.804 00:55:59.654 723 xtrace_restore 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 11 # Pci bus hotplug 01:11:18.804 00:55:59.654 12 # Helper function to remove/attach cotrollers 01:11:18.804 00:55:59.654 13 debug_remove_attach_helper() { 01:11:18.804 00:55:59.654 14 local helper_time=0 01:11:18.804 00:55:59.654 15 01:11:18.804 00:55:59.654 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.804 00:55:59.654 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.804 00:55:59.654 18 "$helper_time" "$nvme_count" >&2 01:11:18.804 00:55:59.654 19 } 01:11:18.804 00:55:59.654 20 01:11:18.804 00:55:59.654 21 remove_attach_helper() { 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.804 00:55:59.654 106 done 01:11:18.804 00:55:59.654 107 01:11:18.804 00:55:59.654 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.804 00:55:59.654 109 01:11:18.804 00:55:59.654 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.804 00:55:59.654 111 # Verify reregistering hotplug poller 01:11:18.804 00:55:59.654 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.804 00:55:59.654 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.804 00:55:59.654 114 01:11:18.804 00:55:59.654 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 133 01:11:18.804 00:55:59.654 134 # Run pci bus hotplug test 01:11:18.804 00:55:59.654 135 run_hotplug 01:11:18.804 00:55:59.654 136 01:11:18.804 00:55:59.654 137 # Run SPDK target based hotplug 01:11:18.804 00:55:59.654 => 138 tgt_run_hotplug 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 01:11:18.804 00:55:59.654 ========== Backtrace end ========== 01:11:18.804 00:55:59.654 ========== Backtrace start: ========== 01:11:18.804 00:55:59.654 01:11:18.804 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 11 # Pci bus hotplug 01:11:18.804 00:55:59.654 12 # Helper function to remove/attach cotrollers 01:11:18.804 00:55:59.654 13 debug_remove_attach_helper() { 01:11:18.804 00:55:59.654 14 local helper_time=0 01:11:18.804 00:55:59.654 15 01:11:18.804 00:55:59.654 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:11:18.804 00:55:59.654 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:11:18.804 00:55:59.654 18 "$helper_time" "$nvme_count" >&2 01:11:18.804 00:55:59.654 19 } 01:11:18.804 00:55:59.654 20 01:11:18.804 00:55:59.654 21 remove_attach_helper() { 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:11:18.804 00:55:59.654 106 done 01:11:18.804 00:55:59.654 107 01:11:18.804 00:55:59.654 108 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.804 00:55:59.654 109 01:11:18.804 00:55:59.654 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.804 00:55:59.654 111 # Verify reregistering hotplug poller 01:11:18.804 00:55:59.654 112 rpc_cmd bdev_nvme_set_hotplug -d 01:11:18.804 00:55:59.654 113 rpc_cmd bdev_nvme_set_hotplug -e 01:11:18.804 00:55:59.654 114 01:11:18.804 00:55:59.654 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 133 01:11:18.804 00:55:59.654 134 # Run pci bus hotplug test 01:11:18.804 00:55:59.654 135 run_hotplug 01:11:18.804 00:55:59.654 136 01:11:18.804 00:55:59.654 137 # Run SPDK target based hotplug 01:11:18.804 00:55:59.654 => 138 tgt_run_hotplug 01:11:18.804 00:55:59.654 ... 01:11:18.804 00:55:59.654 01:11:18.804 00:55:59.654 ========== Backtrace end ========== 01:11:18.804 00:55:59.655 ========== Backtrace start: ========== 01:11:18.804 00:55:59.655 01:11:18.804 00:55:59.655 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 01:11:18.804 00:55:59.655 ... 01:11:18.804 00:55:59.655 1117 timing_enter $test_name 01:11:18.804 00:55:59.655 1118 echo "************************************" 01:11:18.804 00:55:59.655 1119 echo "START TEST $test_name" 01:11:18.804 00:55:59.655 1120 echo "************************************" 01:11:18.804 00:55:59.655 1121 xtrace_restore 01:11:18.804 00:55:59.655 1122 time "$@" 01:11:18.804 00:55:59.655 1123 xtrace_disable 01:11:18.804 00:55:59.655 1124 echo "************************************" 01:11:18.804 00:55:59.655 1125 echo "END TEST $test_name" 01:11:18.804 00:55:59.655 1126 echo "************************************" 01:11:18.804 00:55:59.655 1127 timing_exit $test_name 01:11:18.804 00:55:59.655 ... 01:11:18.804 00:55:59.655 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 01:11:18.804 00:55:59.655 ... 01:11:18.804 00:55:59.655 235 01:11:18.804 00:55:59.655 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 01:11:18.804 00:55:59.655 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 01:11:18.804 00:55:59.655 238 01:11:18.804 00:55:59.655 239 if [ $(uname -s) = Linux ]; then 01:11:18.804 00:55:59.655 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 01:11:18.804 00:55:59.655 241 fi 01:11:18.804 00:55:59.655 242 01:11:18.804 00:55:59.655 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 01:11:18.804 00:55:59.655 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 01:11:18.804 00:55:59.655 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 01:11:18.804 00:55:59.655 ... 01:11:18.804 00:55:59.655 01:11:18.804 00:55:59.655 ========== Backtrace end ==========' 01:11:18.804 + run_flamegraph 01:11:18.804 + create_timing_file 01:11:18.804 + local array file 01:11:18.804 + array=($(find $WORKSPACE -type f -name 'timing.txt')) 01:11:18.804 ++ find /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11 -type f -name timing.txt 01:11:18.804 + timing_dir=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/post_process 01:11:18.804 + mkdir -p /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/post_process 01:11:18.804 + timing_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/post_process/total_timing.txt 01:11:18.804 + for file in "${array[@]}" 01:11:18.804 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-cmb-pmr-vg-autotest_78556/timing.txt 01:11:18.804 + local job_file job_name 01:11:18.804 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-cmb-pmr-vg-autotest_78556/timing.txt 01:11:18.804 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-cmb-pmr-vg-autotest_78556/timing.txt 01:11:18.804 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-cmb-pmr-vg-autotest_78556 01:11:18.804 + job_name=nvme-cmb-pmr-vg-autotest_78556 01:11:18.804 + sed 's/^/nvme-cmb-pmr-vg-autotest_78556;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-cmb-pmr-vg-autotest_78556/timing.txt 01:11:18.804 + for file in "${array[@]}" 01:11:18.804 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-uring-vg-autotest_77338/timing.txt 01:11:18.804 + local job_file job_name 01:11:18.804 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-uring-vg-autotest_77338/timing.txt 01:11:18.804 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-uring-vg-autotest_77338/timing.txt 01:11:18.804 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-uring-vg-autotest_77338 01:11:18.804 + job_name=iscsi-uring-vg-autotest_77338 01:11:18.804 + sed 's/^/iscsi-uring-vg-autotest_77338;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-uring-vg-autotest_77338/timing.txt 01:11:18.804 + for file in "${array[@]}" 01:11:18.804 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-phy-autotest_72397/timing.txt 01:11:18.804 + local job_file job_name 01:11:18.804 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-phy-autotest_72397/timing.txt 01:11:18.804 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-phy-autotest_72397/timing.txt 01:11:18.804 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-phy-autotest_72397 01:11:18.804 + job_name=nvmf-phy-autotest_72397 01:11:18.804 + sed 's/^/nvmf-phy-autotest_72397;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-phy-autotest_72397/timing.txt 01:11:18.804 + for file in "${array[@]}" 01:11:18.804 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/short-fuzz-phy-autotest_26150/timing.txt 01:11:18.804 + local job_file job_name 01:11:18.804 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/short-fuzz-phy-autotest_26150/timing.txt 01:11:18.804 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/short-fuzz-phy-autotest_26150/timing.txt 01:11:18.804 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/short-fuzz-phy-autotest_26150 01:11:18.804 + job_name=short-fuzz-phy-autotest_26150 01:11:18.804 + sed 's/^/short-fuzz-phy-autotest_26150;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/short-fuzz-phy-autotest_26150/timing.txt 01:11:18.804 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/rocky9-vg-autotest_16043/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/rocky9-vg-autotest_16043/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/rocky9-vg-autotest_16043/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/rocky9-vg-autotest_16043 01:11:18.805 + job_name=rocky9-vg-autotest_16043 01:11:18.805 + sed 's/^/rocky9-vg-autotest_16043;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/rocky9-vg-autotest_16043/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-initiator-vg-autotest_77453/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-initiator-vg-autotest_77453/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-initiator-vg-autotest_77453/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-initiator-vg-autotest_77453 01:11:18.805 + job_name=vhost-initiator-vg-autotest_77453 01:11:18.805 + sed 's/^/vhost-initiator-vg-autotest_77453;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-initiator-vg-autotest_77453/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/autobuild-vg-autotest_63413/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/autobuild-vg-autotest_63413/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/autobuild-vg-autotest_63413/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/autobuild-vg-autotest_63413 01:11:18.805 + job_name=autobuild-vg-autotest_63413 01:11:18.805 + sed 's/^/autobuild-vg-autotest_63413;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/autobuild-vg-autotest_63413/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/centos7-vg-autotest_73612/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/centos7-vg-autotest_73612/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/centos7-vg-autotest_73612/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/centos7-vg-autotest_73612 01:11:18.805 + job_name=centos7-vg-autotest_73612 01:11:18.805 + sed 's/^/centos7-vg-autotest_73612;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/centos7-vg-autotest_73612/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dsa-phy-autotest_4853/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dsa-phy-autotest_4853/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dsa-phy-autotest_4853/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dsa-phy-autotest_4853 01:11:18.805 + job_name=dsa-phy-autotest_4853 01:11:18.805 + sed 's/^/dsa-phy-autotest_4853;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dsa-phy-autotest_4853/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/scanbuild-vg-autotest_63372/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/scanbuild-vg-autotest_63372/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/scanbuild-vg-autotest_63372/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/scanbuild-vg-autotest_63372 01:11:18.805 + job_name=scanbuild-vg-autotest_63372 01:11:18.805 + sed 's/^/scanbuild-vg-autotest_63372;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/scanbuild-vg-autotest_63372/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu22-vg-autotest_37766/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu22-vg-autotest_37766/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu22-vg-autotest_37766/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu22-vg-autotest_37766 01:11:18.805 + job_name=ubuntu22-vg-autotest_37766 01:11:18.805 + sed 's/^/ubuntu22-vg-autotest_37766;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu22-vg-autotest_37766/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-vg-autotest_77471/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-vg-autotest_77471/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-vg-autotest_77471/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-vg-autotest_77471 01:11:18.805 + job_name=iscsi-vg-autotest_77471 01:11:18.805 + sed 's/^/iscsi-vg-autotest_77471;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-vg-autotest_77471/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/zns-vg-autotest_76539/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/zns-vg-autotest_76539/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/zns-vg-autotest_76539/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/zns-vg-autotest_76539 01:11:18.805 + job_name=zns-vg-autotest_76539 01:11:18.805 + sed 's/^/zns-vg-autotest_76539;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/zns-vg-autotest_76539/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/crypto-phy-autotest_26153/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/crypto-phy-autotest_26153/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/crypto-phy-autotest_26153/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/crypto-phy-autotest_26153 01:11:18.805 + job_name=crypto-phy-autotest_26153 01:11:18.805 + sed 's/^/crypto-phy-autotest_26153;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/crypto-phy-autotest_26153/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/clang-vg-autotest_57621/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/clang-vg-autotest_57621/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/clang-vg-autotest_57621/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/clang-vg-autotest_57621 01:11:18.805 + job_name=clang-vg-autotest_57621 01:11:18.805 + sed 's/^/clang-vg-autotest_57621;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/clang-vg-autotest_57621/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-vg-autotest_77433/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-vg-autotest_77433/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-vg-autotest_77433/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-vg-autotest_77433 01:11:18.805 + job_name=nvme-vg-autotest_77433 01:11:18.805 + sed 's/^/nvme-vg-autotest_77433;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-vg-autotest_77433/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-phy-autotest_26178/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-phy-autotest_26178/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-phy-autotest_26178/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-phy-autotest_26178 01:11:18.805 + job_name=vhost-phy-autotest_26178 01:11:18.805 + sed 's/^/vhost-phy-autotest_26178;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-phy-autotest_26178/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-phy-autotest_72325/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-phy-autotest_72325/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-phy-autotest_72325/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-phy-autotest_72325 01:11:18.805 + job_name=nvme-phy-autotest_72325 01:11:18.805 + sed 's/^/nvme-phy-autotest_72325;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-phy-autotest_72325/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-uring-vg-autotest_77358/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-uring-vg-autotest_77358/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-uring-vg-autotest_77358/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-uring-vg-autotest_77358 01:11:18.805 + job_name=nvmf-tcp-uring-vg-autotest_77358 01:11:18.805 + sed 's/^/nvmf-tcp-uring-vg-autotest_77358;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-uring-vg-autotest_77358/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/unittest-vg-autotest_63370/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/unittest-vg-autotest_63370/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/unittest-vg-autotest_63370/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/unittest-vg-autotest_63370 01:11:18.805 + job_name=unittest-vg-autotest_63370 01:11:18.805 + sed 's/^/unittest-vg-autotest_63370;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/unittest-vg-autotest_63370/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-phy-autotest_72284/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-phy-autotest_72284/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-phy-autotest_72284/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-phy-autotest_72284 01:11:18.805 + job_name=nvmf-tcp-phy-autotest_72284 01:11:18.805 + sed 's/^/nvmf-tcp-phy-autotest_72284;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-phy-autotest_72284/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-vg-autotest_77424/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-vg-autotest_77424/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-vg-autotest_77424/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-vg-autotest_77424 01:11:18.805 + job_name=nvmf-tcp-vg-autotest_77424 01:11:18.805 + sed 's/^/nvmf-tcp-vg-autotest_77424;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-vg-autotest_77424/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vfio-user-phy-autotest_26161/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vfio-user-phy-autotest_26161/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vfio-user-phy-autotest_26161/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vfio-user-phy-autotest_26161 01:11:18.805 + job_name=vfio-user-phy-autotest_26161 01:11:18.805 + sed 's/^/vfio-user-phy-autotest_26161;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vfio-user-phy-autotest_26161/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/valgrind-vg-autotest_49187/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/valgrind-vg-autotest_49187/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/valgrind-vg-autotest_49187/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/valgrind-vg-autotest_49187 01:11:18.805 + job_name=valgrind-vg-autotest_49187 01:11:18.805 + sed 's/^/valgrind-vg-autotest_49187;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/valgrind-vg-autotest_49187/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/lvol-vg-autotest_77382/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/lvol-vg-autotest_77382/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/lvol-vg-autotest_77382/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/lvol-vg-autotest_77382 01:11:18.805 + job_name=lvol-vg-autotest_77382 01:11:18.805 + sed 's/^/lvol-vg-autotest_77382;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/lvol-vg-autotest_77382/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vmd-phy-autotest_2614/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vmd-phy-autotest_2614/timing.txt 01:11:18.805 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vmd-phy-autotest_2614/timing.txt 01:11:18.805 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vmd-phy-autotest_2614 01:11:18.805 + job_name=vmd-phy-autotest_2614 01:11:18.805 + sed 's/^/vmd-phy-autotest_2614;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vmd-phy-autotest_2614/timing.txt 01:11:18.805 + for file in "${array[@]}" 01:11:18.805 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/BlobFS-phy-autotest_26159/timing.txt 01:11:18.805 + local job_file job_name 01:11:18.805 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/BlobFS-phy-autotest_26159/timing.txt 01:11:19.063 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/BlobFS-phy-autotest_26159/timing.txt 01:11:19.063 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/BlobFS-phy-autotest_26159 01:11:19.063 + job_name=BlobFS-phy-autotest_26159 01:11:19.063 + sed 's/^/BlobFS-phy-autotest_26159;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/BlobFS-phy-autotest_26159/timing.txt 01:11:19.063 + for file in "${array[@]}" 01:11:19.063 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu20-vg-autotest_73752/timing.txt 01:11:19.063 + local job_file job_name 01:11:19.063 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu20-vg-autotest_73752/timing.txt 01:11:19.063 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu20-vg-autotest_73752/timing.txt 01:11:19.063 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu20-vg-autotest_73752 01:11:19.063 + job_name=ubuntu20-vg-autotest_73752 01:11:19.063 + sed 's/^/ubuntu20-vg-autotest_73752;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu20-vg-autotest_73752/timing.txt 01:11:19.063 + /usr/local/FlameGraph/flamegraph.pl --title 'Build Timing' --nametype Step: --countname seconds /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/post_process/total_timing.txt 01:11:19.063 + run_autorun_post 01:11:19.063 + s_opt= 01:11:19.063 + [[ '' == true ]] 01:11:19.063 + spdk/autorun_post.py -d ./ -r ./spdk 01:11:33.920 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/lib/util/crc32c.c:21 01:11:33.920 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/lib/util/crc64.c:165 01:11:33.920 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/lib/util/crc16.c:628 01:11:33.920 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/lib/util/crc16.c:634 01:11:33.920 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/lib/util/xor.c:120 01:12:00.438 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/module/blobfs/bdev/blobfs_bdev.c:254 01:12:04.639 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/module/bdev/nvme/bdev_mdns_client.c:561 01:12:04.639 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/module/bdev/nvme/bdev_mdns_client.c:613 01:12:04.639 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/module/bdev/nvme/bdev_mdns_client.c:582 01:12:04.639 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/module/bdev/nvme/bdev_mdns_client.c:456 01:12:14.597 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/module/blobfs/bdev/blobfs_bdev.c:254 01:12:26.798 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/module/blobfs/bdev/blobfs_bdev.c:254 01:12:44.863 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/nvmf_transport.h! 01:12:44.863 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/util.h! 01:12:44.863 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/thread.h! 01:12:44.863 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/mmio.h! 01:12:44.863 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/nvme.h! 01:12:44.863 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/base64.h! 01:12:44.863 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/endian.h! 01:12:44.863 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/bdev_module.h! 01:12:44.863 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/trace.h! 01:12:44.863 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/nvme_spec.h! 01:12:44.863 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/histogram_data.h! 01:12:44.863 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/histogram_data.h/histogram_ut.c! 01:12:46.760 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/lib/env_ocf/include/ocf/ocf_io.h! 01:12:46.760 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/lib/env_ocf/include/ocf/ocf_core.h! 01:12:46.760 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/spdk/lib/env_ocf/include/ocf/ocf_mngt.h! 01:13:04.828 -----Begin Post Process Script------ 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-cmb-pmr-vg-autotest_78556/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-uring-vg-autotest_77338/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-phy-autotest_72397/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/rocky9-vg-autotest_16043/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-initiator-vg-autotest_77453/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/centos7-vg-autotest_73612/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/dsa-phy-autotest_4853/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/ubuntu22-vg-autotest_37766/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/iscsi-vg-autotest_77471/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/zns-vg-autotest_76539/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vhost-phy-autotest_26178/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvme-phy-autotest_72325/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-uring-vg-autotest_77358/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/unittest-vg-autotest_63370/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-phy-autotest_72284/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/nvmf-tcp-vg-autotest_77424/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vfio-user-phy-autotest_26161/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/valgrind-vg-autotest_49187/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/lvol-vg-autotest_77382/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/vmd-phy-autotest_2614/cov_total.info 01:13:04.828 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/BlobFS-phy-autotest_26159/cov_total.info 01:13:04.828 01:13:04.828 01:13:04.828 -----Tests Executed in Build------ 01:13:04.828 accel 01:13:04.828 accel_assign_opcode 01:13:04.828 accel_cdev_comp 01:13:04.828 accel_cdev_decmop_full 01:13:04.828 accel_cdev_decomp 01:13:04.828 accel_cdev_decomp_full_mcore 01:13:04.828 accel_cdev_decomp_full_mthread 01:13:04.828 accel_cdev_decomp_mcore 01:13:04.828 accel_cdev_decomp_mthread 01:13:04.828 accel_comp 01:13:04.828 accel_compare 01:13:04.828 accel_compress_verify 01:13:04.828 accel_copy 01:13:04.828 accel_copy_crc32c 01:13:04.828 accel_copy_crc32c_C2 01:13:04.828 accel_crc32c 01:13:04.828 accel_crc32c_C2 01:13:04.828 accel_decmop_full 01:13:04.828 accel_decomp 01:13:04.828 accel_decomp_full_mcore 01:13:04.828 accel_decomp_full_mthread 01:13:04.828 accel_decomp_mcore 01:13:04.828 accel_decomp_mthread 01:13:04.828 accel_dif_functional_tests 01:13:04.828 accel_dif_generate 01:13:04.828 accel_dif_generate_copy 01:13:04.828 accel_dif_verify 01:13:04.828 accel_dualcast 01:13:04.828 accel_fill 01:13:04.828 accel_help 01:13:04.828 accel_missing_filename 01:13:04.828 accel_negative_buffers 01:13:04.828 accel_rpc 01:13:04.828 accel_scan_dsa_modules 01:13:04.828 accel_scan_iaa_modules 01:13:04.828 accel_wrong_workload 01:13:04.828 accel_xor 01:13:04.829 acl 01:13:04.829 alias_rpc 01:13:04.829 allowed 01:13:04.829 app_cmdline 01:13:04.829 app_repeat 01:13:04.829 asan 01:13:04.829 autobuild 01:13:04.829 autobuild_build_doc 01:13:04.829 autobuild_check_dpdk_pci_api 01:13:04.829 autobuild_check_format 01:13:04.829 autobuild_check_so_deps 01:13:04.829 autobuild_external_code 01:13:04.829 autobuild_generated_files_check 01:13:04.829 autobuild_generated_files_check_post_clean 01:13:04.829 autobuild_header_dependency_check 01:13:04.829 autobuild_llvm_precompile 01:13:04.829 autobuild_make_install 01:13:04.829 autobuild_make_uninstall 01:13:04.829 autobuild_ocf_precompile 01:13:04.829 bdev_bounds 01:13:04.829 bdev_crypto_enomem 01:13:04.829 bdev_error 01:13:04.829 bdev_fio 01:13:04.829 bdev_fio_rw_verify 01:13:04.829 bdev_fio_trim 01:13:04.829 bdev_gpt_uuid 01:13:04.829 bdev_hello_world 01:13:04.829 bdev_json_nonarray 01:13:04.829 bdev_json_nonenclosed 01:13:04.829 bdev_nbd 01:13:04.829 bdev_nvme_reset_stuck_adm_cmd 01:13:04.829 bdev_qd_sampling 01:13:04.829 bdev_qos 01:13:04.829 bdev_qos_bw 01:13:04.829 bdev_qos_iops 01:13:04.829 bdev_qos_ro_bw 01:13:04.829 bdev_raid 01:13:04.829 bdev_stat 01:13:04.829 bdev_verify 01:13:04.829 bdev_verify_big_io 01:13:04.829 bdev_write_zeroes 01:13:04.829 bdevperf_config 01:13:04.829 blob_io_wait 01:13:04.829 blobfs 01:13:04.829 blobfs_mkfs 01:13:04.829 blobstore 01:13:04.829 blobstore_grow 01:13:04.829 blockdev_crypto_aesni 01:13:04.829 blockdev_crypto_sw 01:13:04.829 blockdev_general 01:13:04.829 blockdev_nvme 01:13:04.829 blockdev_nvme_gpt 01:13:04.829 blockdev_raid5f 01:13:04.829 blockdev_rbd 01:13:04.829 build_native_dpdk 01:13:04.829 build_rpm_from_gen_spec 01:13:04.829 build_shared_native_dpdk_rpm 01:13:04.829 build_shared_rpm 01:13:04.829 build_shared_rpm_with_rpmed_dpdk 01:13:04.829 check_header_filenames 01:13:04.829 cmb_copy 01:13:04.829 compress_compdev 01:13:04.829 compress_isal 01:13:04.829 confirm_abi_deps 01:13:04.829 confirm_makefile_deps 01:13:04.829 cpu_locks 01:13:04.829 custom_alloc 01:13:04.829 dd_bs_lt_native_bs 01:13:04.829 dd_copy_to_out_bdev 01:13:04.829 dd_double_input 01:13:04.829 dd_double_output 01:13:04.829 dd_flag_append 01:13:04.829 dd_flag_append_forced_aio 01:13:04.829 dd_flag_directory 01:13:04.829 dd_flag_directory_forced_aio 01:13:04.829 dd_flag_noatime 01:13:04.829 dd_flag_noatime_forced_aio 01:13:04.829 dd_flag_nofollow 01:13:04.829 dd_flag_nofollow_forced_aio 01:13:04.829 dd_flags_misc 01:13:04.829 dd_flags_misc_forced_aio 01:13:04.829 dd_inflate_file 01:13:04.829 dd_invalid_arguments 01:13:04.829 dd_invalid_count 01:13:04.829 dd_invalid_iflag 01:13:04.829 dd_invalid_json 01:13:04.829 dd_invalid_oflag 01:13:04.829 dd_malloc_copy 01:13:04.829 dd_no_input 01:13:04.829 dd_no_output 01:13:04.829 dd_offset_magic 01:13:04.829 dd_rw 01:13:04.829 dd_rw_offset 01:13:04.829 dd_smaller_blocksize 01:13:04.829 dd_sparse_bdev_to_file 01:13:04.829 dd_sparse_file_to_bdev 01:13:04.829 dd_sparse_file_to_file 01:13:04.829 dd_unknown_flag 01:13:04.829 dd_uring_copy 01:13:04.829 dd_wrong_blocksize 01:13:04.829 default_locks 01:13:04.829 default_locks_via_rpc 01:13:04.829 default_setup 01:13:04.829 denied 01:13:04.829 devices 01:13:04.829 dm_mount 01:13:04.829 dma 01:13:04.829 dpdk_governor 01:13:04.829 dpdk_mem_utility 01:13:04.829 driver 01:13:04.829 env 01:13:04.829 env_dpdk_post_init 01:13:04.829 env_mem_callbacks 01:13:04.829 env_memory 01:13:04.829 env_pci 01:13:04.829 env_vtophys 01:13:04.829 even_2G_alloc 01:13:04.829 event 01:13:04.829 event_perf 01:13:04.829 event_reactor 01:13:04.829 event_reactor_perf 01:13:04.829 event_scheduler 01:13:04.829 exit_on_failed_rpc_init 01:13:04.829 external_make_accel_driver_shared 01:13:04.829 external_make_accel_driver_static 01:13:04.829 external_make_accel_module_shared 01:13:04.829 external_make_accel_module_static 01:13:04.829 external_make_hello_bdev_shared_combo 01:13:04.829 external_make_hello_bdev_shared_iso 01:13:04.829 external_make_hello_bdev_static 01:13:04.829 external_make_hello_no_bdev_shared_combo 01:13:04.829 external_make_hello_no_bdev_shared_iso 01:13:04.829 external_make_hello_no_bdev_static 01:13:04.829 external_make_nvme_shared 01:13:04.829 external_make_nvme_static 01:13:04.829 external_run_accel_driver_shared 01:13:04.829 external_run_accel_driver_static 01:13:04.829 external_run_accel_module_shared 01:13:04.829 external_run_accel_module_static 01:13:04.829 external_run_hello_bdev_shared_combo 01:13:04.829 external_run_hello_bdev_shared_iso 01:13:04.829 external_run_hello_bdev_static 01:13:04.829 external_run_hello_no_bdev_shared_combo 01:13:04.829 external_run_hello_no_bdev_shared_iso 01:13:04.829 external_run_hello_no_bdev_static 01:13:04.829 external_run_nvme_shared 01:13:04.829 external_run_nvme_static 01:13:04.829 filesystem_btrfs 01:13:04.829 filesystem_ext4 01:13:04.829 filesystem_in_capsule_btrfs 01:13:04.829 filesystem_in_capsule_ext4 01:13:04.829 filesystem_in_capsule_xfs 01:13:04.829 filesystem_xfs 01:13:04.829 fio_dif_1_default 01:13:04.829 fio_dif_1_multi_subsystems 01:13:04.829 fio_dif_digest 01:13:04.829 fio_dif_rand_params 01:13:04.829 go_rpc 01:13:04.829 guess_driver 01:13:04.829 hello_blob 01:13:04.829 hugepages 01:13:04.829 idle 01:13:04.829 interrupt_mode 01:13:04.829 ioat 01:13:04.829 ioat_perf 01:13:04.829 ioat_verify 01:13:04.829 is_zoned 01:13:04.829 iscsi_tgt 01:13:04.829 iscsi_tgt_bdev_io_wait 01:13:04.829 iscsi_tgt_calsoft 01:13:04.829 iscsi_tgt_digest 01:13:04.829 iscsi_tgt_digests 01:13:04.829 iscsi_tgt_ext4test 01:13:04.829 iscsi_tgt_filesystem 01:13:04.829 iscsi_tgt_filesystem_btrfs 01:13:04.829 iscsi_tgt_filesystem_ext4 01:13:04.829 iscsi_tgt_filesystem_xfs 01:13:04.829 iscsi_tgt_fio 01:13:04.829 iscsi_tgt_fuzz 01:13:04.829 iscsi_tgt_initiator 01:13:04.829 iscsi_tgt_ip_migration 01:13:04.829 iscsi_tgt_iscsi_lvol 01:13:04.829 iscsi_tgt_login_redirection 01:13:04.829 iscsi_tgt_multiconnection 01:13:04.829 iscsi_tgt_qos 01:13:04.829 iscsi_tgt_rbd 01:13:04.829 iscsi_tgt_reset 01:13:04.829 iscsi_tgt_resize 01:13:04.829 iscsi_tgt_rpc_config 01:13:04.829 iscsi_tgt_sock 01:13:04.829 iscsi_tgt_trace_record 01:13:04.829 json_config 01:13:04.829 json_config_extra_key 01:13:04.829 kernel_target_abort 01:13:04.829 keyring_file 01:13:04.829 llvm_fuzz 01:13:04.829 locking_app_on_locked_coremask 01:13:04.829 locking_app_on_unlocked_coremask 01:13:04.829 locking_overlapped_coremask 01:13:04.829 locking_overlapped_coremask_via_rpc 01:13:04.829 lvol 01:13:04.829 lvol_basic 01:13:04.829 lvol_esnap 01:13:04.829 lvol_external_copy 01:13:04.829 lvol_external_snapshot 01:13:04.829 lvol_hotremove 01:13:04.829 lvol_provisioning 01:13:04.829 lvol_rename 01:13:04.829 lvol_resize 01:13:04.829 lvol_snapshot_clone 01:13:04.829 lvol_tasting 01:13:04.829 lvs_grow_clean 01:13:04.829 lvs_grow_dirty 01:13:04.829 make 01:13:04.829 no_shrink_alloc 01:13:04.829 non_locking_app_on_locked_coremask 01:13:04.829 nvme 01:13:04.829 nvme_arbitration 01:13:04.829 nvme_cli_cuse 01:13:04.829 nvme_cli_plugin 01:13:04.829 nvme_cmb 01:13:04.829 nvme_cuse 01:13:04.829 nvme_cuse_app 01:13:04.829 nvme_cuse_rpc 01:13:04.829 nvme_doorbell_aers 01:13:04.829 nvme_e2edp 01:13:04.829 nvme_err_injection 01:13:04.829 nvme_fdp 01:13:04.829 nvme_fio 01:13:04.829 nvme_flexible_data_placement 01:13:04.829 nvme_hello_world 01:13:04.829 nvme_identify 01:13:04.829 nvme_mount 01:13:04.829 nvme_multi_aen 01:13:04.829 nvme_multi_secondary 01:13:04.829 nvme_ns_manage_cuse 01:13:04.829 nvme_overhead 01:13:04.829 nvme_perf 01:13:04.829 nvme_pmr 01:13:04.829 nvme_pmr_persistence 01:13:04.829 nvme_reserve 01:13:04.829 nvme_reset 01:13:04.829 nvme_rpc 01:13:04.829 nvme_rpc_timeouts 01:13:04.829 nvme_scc 01:13:04.829 nvme_sgl 01:13:04.829 nvme_simple_copy 01:13:04.829 nvme_single_aen 01:13:04.829 nvme_smartctl_cuse 01:13:04.829 nvme_startup 01:13:04.829 nvme_zns 01:13:04.829 nvmf_abort 01:13:04.829 nvmf_abort_qd_sizes 01:13:04.829 nvmf_aer 01:13:04.829 nvmf_async_init 01:13:04.829 nvmf_auth_host 01:13:04.829 nvmf_auth_target 01:13:04.829 nvmf_bdev_io_wait 01:13:04.829 nvmf_bdevio 01:13:04.829 nvmf_bdevio_no_huge 01:13:04.829 nvmf_bdevperf 01:13:04.829 nvmf_connect_disconnect 01:13:04.829 nvmf_connect_stress 01:13:04.829 nvmf_delete_subsystem 01:13:04.829 nvmf_device_removal 01:13:04.829 nvmf_device_removal_pci_remove 01:13:04.829 nvmf_device_removal_pci_remove_no_srq 01:13:04.829 nvmf_dif 01:13:04.829 nvmf_digest 01:13:04.829 nvmf_digest_clean 01:13:04.829 nvmf_digest_error 01:13:04.829 nvmf_discovery_remove_ifc 01:13:04.829 nvmf_example 01:13:04.829 nvmf_failover 01:13:04.829 nvmf_filesystem 01:13:04.829 nvmf_filesystem_in_capsule 01:13:04.829 nvmf_filesystem_no_in_capsule 01:13:04.829 nvmf_fio_host 01:13:04.829 nvmf_fio_target 01:13:04.829 nvmf_fips 01:13:04.829 nvmf_fused_ordering 01:13:04.829 nvmf_fuzz 01:13:04.829 nvmf_host_discovery 01:13:04.829 nvmf_host_management 01:13:04.829 nvmf_host_multipath 01:13:04.829 nvmf_host_multipath_status 01:13:04.830 nvmf_identify 01:13:04.830 nvmf_identify_kernel_target 01:13:04.830 nvmf_identify_passthru 01:13:04.830 nvmf_initiator_timeout 01:13:04.830 nvmf_invalid 01:13:04.830 nvmf_lvol 01:13:04.830 nvmf_lvs_grow 01:13:04.830 nvmf_mdns_discovery 01:13:04.830 nvmf_multiconnection 01:13:04.830 nvmf_multicontroller 01:13:04.830 nvmf_multitarget 01:13:04.830 nvmf_nmic 01:13:04.830 nvmf_ns_hotplug_stress 01:13:04.830 nvmf_ns_masking 01:13:04.830 nvmf_nvme_cli 01:13:04.830 nvmf_perf 01:13:04.830 nvmf_perf_adq 01:13:04.830 nvmf_queue_depth 01:13:04.830 nvmf_rdma 01:13:04.830 nvmf_referrals 01:13:04.830 nvmf_rpc 01:13:04.830 nvmf_shutdown 01:13:04.830 nvmf_shutdown_tc1 01:13:04.830 nvmf_shutdown_tc2 01:13:04.830 nvmf_shutdown_tc3 01:13:04.830 nvmf_srq_overwhelm 01:13:04.830 nvmf_target_disconnect 01:13:04.830 nvmf_target_disconnect_tc1 01:13:04.830 nvmf_target_disconnect_tc2 01:13:04.830 nvmf_target_disconnect_tc3 01:13:04.830 nvmf_target_discovery 01:13:04.830 nvmf_target_multipath 01:13:04.830 nvmf_tcp 01:13:04.830 nvmf_timeout 01:13:04.830 nvmf_tls 01:13:04.830 nvmf_vfio_user 01:13:04.830 nvmf_vfio_user_fuzz 01:13:04.830 nvmf_vfio_user_nvme_compliance 01:13:04.830 nvmf_vhost 01:13:04.830 nvmf_zcopy 01:13:04.830 ocf 01:13:04.830 ocf_bdevperf_iotypes 01:13:04.830 ocf_configuration_change 01:13:04.830 ocf_create_destruct 01:13:04.830 ocf_fio_modes 01:13:04.830 ocf_flush 01:13:04.830 ocf_multicore 01:13:04.830 ocf_remove 01:13:04.830 ocf_stats 01:13:04.830 odd_alloc 01:13:04.830 packaging 01:13:04.830 per_node_1G_alloc 01:13:04.830 raid0_resize_test 01:13:04.830 raid5f_rebuild_test 01:13:04.830 raid5f_rebuild_test_sb 01:13:04.830 raid5f_state_function_test 01:13:04.830 raid5f_state_function_test_sb 01:13:04.830 raid5f_superblock_test 01:13:04.830 raid_function_test_concat 01:13:04.830 raid_function_test_raid0 01:13:04.830 raid_rebuild_test 01:13:04.830 raid_rebuild_test_io 01:13:04.830 raid_rebuild_test_sb 01:13:04.830 raid_rebuild_test_sb_4k 01:13:04.830 raid_rebuild_test_sb_io 01:13:04.830 raid_rebuild_test_sb_md_interleaved 01:13:04.830 raid_rebuild_test_sb_md_separate 01:13:04.830 raid_state_function_test 01:13:04.830 raid_state_function_test_sb 01:13:04.830 raid_state_function_test_sb_4k 01:13:04.830 raid_state_function_test_sb_md_interleaved 01:13:04.830 raid_state_function_test_sb_md_separate 01:13:04.830 raid_superblock_test 01:13:04.830 raid_superblock_test_4k 01:13:04.830 raid_superblock_test_md_interleaved 01:13:04.830 raid_superblock_test_md_separate 01:13:04.830 reactor_set_interrupt 01:13:04.830 reap_unregistered_poller 01:13:04.830 rocksdb 01:13:04.830 rocksdb_insert 01:13:04.830 rocksdb_overwrite 01:13:04.830 rocksdb_randread 01:13:04.830 rocksdb_readwrite 01:13:04.830 rocksdb_writesync 01:13:04.830 rpc 01:13:04.830 rpc_client 01:13:04.830 rpc_daemon_integrity 01:13:04.830 rpc_integrity 01:13:04.830 rpc_plugins 01:13:04.830 rpc_trace_cmd_test 01:13:04.830 rpm_packaging 01:13:04.830 scanbuild_make 01:13:04.830 scheduler 01:13:04.830 scheduler_create_thread 01:13:04.830 setup.sh 01:13:04.830 skip_rpc 01:13:04.830 skip_rpc_with_delay 01:13:04.830 skip_rpc_with_json 01:13:04.830 sma 01:13:04.830 sma_crypto 01:13:04.830 sma_discovery 01:13:04.830 sma_nvmf_tcp 01:13:04.830 sma_plugins 01:13:04.830 sma_qos 01:13:04.830 sma_vfiouser_qemu 01:13:04.830 sma_vhost 01:13:04.830 spdk_dd 01:13:04.830 spdk_dd_basic_rw 01:13:04.830 spdk_dd_bdev_to_bdev 01:13:04.830 spdk_dd_malloc 01:13:04.830 spdk_dd_negative 01:13:04.830 spdk_dd_posix 01:13:04.830 spdk_dd_sparse 01:13:04.830 spdk_dd_uring 01:13:04.830 spdk_target_abort 01:13:04.830 spdkcli_iscsi 01:13:04.830 spdkcli_nvmf_rdma 01:13:04.830 spdkcli_nvmf_tcp 01:13:04.830 spdkcli_raid 01:13:04.830 spdkcli_rbd 01:13:04.830 spdkcli_tcp 01:13:04.830 spdkcli_vhost 01:13:04.830 spdkcli_virtio 01:13:04.830 sw_hotplug 01:13:04.830 test_bdev_lvol_delete_lvstore_with_clones 01:13:04.830 test_bdev_lvol_delete_ordering 01:13:04.830 test_clone_decouple_parent 01:13:04.830 test_clone_inflate 01:13:04.830 test_clone_snapshot_relations 01:13:04.830 test_construct_lvol 01:13:04.830 test_construct_lvol_alias_conflict 01:13:04.830 test_construct_lvol_fio_clear_method_none 01:13:04.830 test_construct_lvol_fio_clear_method_unmap 01:13:04.830 test_construct_lvol_full_lvs 01:13:04.830 test_construct_lvol_inexistent_lvs 01:13:04.830 test_construct_lvols_conflict_alias 01:13:04.830 test_construct_lvs 01:13:04.830 test_construct_lvs_clear_methods 01:13:04.830 test_construct_lvs_conflict_alias 01:13:04.830 test_construct_lvs_different_cluster_size 01:13:04.830 test_construct_lvs_nonexistent_bdev 01:13:04.830 test_construct_multi_lvols 01:13:04.830 test_construct_nested_lvol 01:13:04.830 test_construct_two_lvs_on_the_same_bdev 01:13:04.830 test_create_snapshot_of_snapshot 01:13:04.830 test_create_snapshot_with_io 01:13:04.830 test_delete_lvol_store_persistent_positive 01:13:04.830 test_delete_snapshot_with_clone 01:13:04.830 test_delete_snapshot_with_snapshot 01:13:04.830 test_destroy_after_bdev_lvol_resize_positive 01:13:04.830 test_esnap_clones 01:13:04.830 test_esnap_late_arrival 01:13:04.830 test_esnap_reload 01:13:04.830 test_esnap_remove_degraded 01:13:04.830 test_hotremove_lvol_store 01:13:04.830 test_hotremove_lvol_store_base 01:13:04.830 test_hotremove_lvol_store_base_with_lvols 01:13:04.830 test_hotremove_lvol_store_multiple_lvols 01:13:04.830 test_lvol_bdev_readonly 01:13:04.830 test_lvol_list 01:13:04.830 test_lvol_rename_negative 01:13:04.830 test_rename_lvs_negative 01:13:04.830 test_rename_positive 01:13:04.830 test_resize_lvol 01:13:04.830 test_resize_lvol_negative 01:13:04.830 test_resize_lvol_with_io_traffic 01:13:04.830 test_shallow_copy_compare 01:13:04.830 test_sigterm 01:13:04.830 test_snapshot_compare_with_lvol_bdev 01:13:04.830 test_tasting 01:13:04.830 test_thin_lvol_check_integrity 01:13:04.830 test_thin_lvol_check_space 01:13:04.830 test_thin_lvol_check_zeroes 01:13:04.830 test_thin_lvol_resize 01:13:04.830 test_thin_overprovisioning 01:13:04.830 test_unregister_lvol_bdev 01:13:04.830 thread 01:13:04.830 thread_poller_perf 01:13:04.830 thread_spdk_lock 01:13:04.830 ubsan 01:13:04.830 unittest 01:13:04.830 unittest_accel 01:13:04.830 unittest_bdev 01:13:04.830 unittest_bdev_compress 01:13:04.830 unittest_bdev_crypto 01:13:04.830 unittest_bdev_raid5f 01:13:04.830 unittest_blob_blobfs 01:13:04.830 unittest_build 01:13:04.830 unittest_dma 01:13:04.830 unittest_dpdk_compressdev 01:13:04.830 unittest_event 01:13:04.830 unittest_ftl 01:13:04.830 unittest_idxd_user 01:13:04.830 unittest_include 01:13:04.830 unittest_init 01:13:04.830 unittest_ioat 01:13:04.830 unittest_iobuf 01:13:04.830 unittest_iscsi 01:13:04.830 unittest_json 01:13:04.830 unittest_keyring 01:13:04.830 unittest_lib_reduce 01:13:04.830 unittest_log 01:13:04.830 unittest_lvol 01:13:04.830 unittest_notify 01:13:04.830 unittest_nvme 01:13:04.830 unittest_nvme_cuse 01:13:04.830 unittest_nvme_rdma 01:13:04.830 unittest_nvmf 01:13:04.830 unittest_nvmf_rdma 01:13:04.830 unittest_nvmf_transport 01:13:04.830 unittest_nvmf_vfio_user 01:13:04.830 unittest_pci_event 01:13:04.830 unittest_rdma 01:13:04.830 unittest_rpc 01:13:04.830 unittest_scsi 01:13:04.830 unittest_sock 01:13:04.830 unittest_thread 01:13:04.830 unittest_util 01:13:04.830 unittest_vhost 01:13:04.830 valgrind 01:13:04.830 version 01:13:04.830 vfio_fuzz 01:13:04.830 vfio_user_nvme_fio 01:13:04.830 vfio_user_nvme_restart_vm 01:13:04.830 vfio_user_qemu 01:13:04.830 vfio_user_virtio_bdevperf 01:13:04.830 vfio_user_virtio_blk_restart_vm 01:13:04.830 vfio_user_virtio_scsi_restart_vm 01:13:04.830 vhost 01:13:04.830 vhost_blk_cores_2ctrl 01:13:04.830 vhost_blk_fs_integrity 01:13:04.830 vhost_blk_integrity 01:13:04.830 vhost_blk_lvol_integrity 01:13:04.830 vhost_blk_packed_ring_integrity 01:13:04.830 vhost_blockdev 01:13:04.830 vhost_boot 01:13:04.830 vhost_fuzz 01:13:04.830 vhost_migration 01:13:04.830 vhost_migration_tc1 01:13:04.830 vhost_migration_tc2 01:13:04.830 vhost_negative 01:13:04.830 vhost_readonly 01:13:04.830 vhost_scsi_cores_2ctrl 01:13:04.830 vhost_scsi_fs_integrity 01:13:04.830 vhost_scsi_integrity 01:13:04.830 vhost_scsi_lvol_integrity 01:13:04.830 vhost_shared 01:13:04.830 vmd 01:13:04.830 vmd_bdev_svc 01:13:04.830 vmd_fio 01:13:04.830 vmd_hello_world 01:13:04.830 vmd_identify 01:13:04.830 vmd_perf 01:13:04.830 zoned_fio 01:13:04.830 01:13:04.830 01:13:04.830 -----Tests Missing From Build------ 01:13:04.830 balanced 01:13:04.830 blockdev_crypto_qat 01:13:04.830 blockdev_xnvme 01:13:04.830 busy 01:13:04.830 chaining 01:13:04.830 core_load 01:13:04.830 ftl 01:13:04.830 ftl_bdevperf 01:13:04.830 ftl_dirty_shutdown 01:13:04.830 ftl_fio_basic 01:13:04.830 ftl_restore 01:13:04.830 ftl_restore_fast 01:13:04.830 ftl_trim 01:13:04.830 ftl_upgrade_shutdown 01:13:04.830 iscsi_tgt_data_digest 01:13:04.830 keyring_linux 01:13:04.830 nvme_boot_partition 01:13:04.830 nvme_bp 01:13:04.830 nvme_opal 01:13:04.830 nvme_opal_bdevio 01:13:04.830 nvme_opal_bdevperf 01:13:04.830 nvme_opal_spdk_tgt 01:13:04.830 nvme_xnvme 01:13:04.830 nvmf_digest_dsa_initiator 01:13:04.830 nvmf_digest_dsa_target 01:13:04.830 nvmf_fc 01:13:04.830 spdkcli_nvmf_fc 01:13:04.830 test_create_multi_ublk 01:13:04.830 test_create_ublk 01:13:04.830 test_save_ublk_config 01:13:04.830 ublk 01:13:04.830 ublk_recovery 01:13:04.830 unittest_nvmf_fc 01:13:04.830 unittest_nvmf_fc_ls 01:13:04.831 vhost_blk_hot_remove 01:13:04.831 vhost_hotplug 01:13:04.831 vhost_scsi_hot_remove 01:13:04.831 xnvme_bdevperf 01:13:04.831 xnvme_to_malloc_dd_copy 01:13:04.831 01:13:04.831 01:13:04.831 -----Tests Missing ASAN------ 01:13:04.831 accel_cdev_comp 01:13:04.831 accel_cdev_decmop_full 01:13:04.831 accel_cdev_decomp 01:13:04.831 accel_cdev_decomp_full_mcore 01:13:04.831 accel_cdev_decomp_full_mthread 01:13:04.831 accel_cdev_decomp_mcore 01:13:04.831 accel_cdev_decomp_mthread 01:13:04.831 autobuild_llvm_precompile 01:13:04.831 balanced 01:13:04.831 bdev_crypto_enomem 01:13:04.831 blockdev_crypto_aesni 01:13:04.831 blockdev_crypto_qat 01:13:04.831 blockdev_crypto_sw 01:13:04.831 blockdev_rbd 01:13:04.831 blockdev_xnvme 01:13:04.831 busy 01:13:04.831 chaining 01:13:04.831 cmb_copy 01:13:04.831 compress_compdev 01:13:04.831 compress_isal 01:13:04.831 core_load 01:13:04.831 dd_uring_copy 01:13:04.831 dma 01:13:04.831 dpdk_governor 01:13:04.831 filesystem_btrfs 01:13:04.831 filesystem_ext4 01:13:04.831 filesystem_in_capsule_btrfs 01:13:04.831 filesystem_in_capsule_ext4 01:13:04.831 filesystem_in_capsule_xfs 01:13:04.831 filesystem_xfs 01:13:04.831 fio_dif_1_default 01:13:04.831 fio_dif_1_multi_subsystems 01:13:04.831 fio_dif_digest 01:13:04.831 fio_dif_rand_params 01:13:04.831 ftl 01:13:04.831 ftl_bdevperf 01:13:04.831 ftl_dirty_shutdown 01:13:04.831 ftl_fio_basic 01:13:04.831 ftl_restore 01:13:04.831 ftl_restore_fast 01:13:04.831 ftl_trim 01:13:04.831 ftl_upgrade_shutdown 01:13:04.831 go_rpc 01:13:04.831 idle 01:13:04.831 interrupt_mode 01:13:04.831 ioat 01:13:04.831 ioat_perf 01:13:04.831 ioat_verify 01:13:04.831 is_zoned 01:13:04.831 iscsi_tgt 01:13:04.831 iscsi_tgt_bdev_io_wait 01:13:04.831 iscsi_tgt_calsoft 01:13:04.831 iscsi_tgt_data_digest 01:13:04.831 iscsi_tgt_digest 01:13:04.831 iscsi_tgt_digests 01:13:04.831 iscsi_tgt_ext4test 01:13:04.831 iscsi_tgt_filesystem 01:13:04.831 iscsi_tgt_filesystem_btrfs 01:13:04.831 iscsi_tgt_filesystem_ext4 01:13:04.831 iscsi_tgt_filesystem_xfs 01:13:04.831 iscsi_tgt_fio 01:13:04.831 iscsi_tgt_fuzz 01:13:04.831 iscsi_tgt_initiator 01:13:04.831 iscsi_tgt_ip_migration 01:13:04.831 iscsi_tgt_iscsi_lvol 01:13:04.831 iscsi_tgt_login_redirection 01:13:04.831 iscsi_tgt_multiconnection 01:13:04.831 iscsi_tgt_qos 01:13:04.831 iscsi_tgt_rbd 01:13:04.831 iscsi_tgt_reset 01:13:04.831 iscsi_tgt_resize 01:13:04.831 iscsi_tgt_rpc_config 01:13:04.831 iscsi_tgt_sock 01:13:04.831 iscsi_tgt_trace_record 01:13:04.831 kernel_target_abort 01:13:04.831 keyring_file 01:13:04.831 keyring_linux 01:13:04.831 llvm_fuzz 01:13:04.831 lvs_grow_clean 01:13:04.831 lvs_grow_dirty 01:13:04.831 nvme_boot_partition 01:13:04.831 nvme_bp 01:13:04.831 nvme_cli_cuse 01:13:04.831 nvme_cli_plugin 01:13:04.831 nvme_cmb 01:13:04.831 nvme_cuse 01:13:04.831 nvme_cuse_app 01:13:04.831 nvme_cuse_rpc 01:13:04.831 nvme_ns_manage_cuse 01:13:04.831 nvme_opal 01:13:04.831 nvme_opal_bdevio 01:13:04.831 nvme_opal_bdevperf 01:13:04.831 nvme_opal_spdk_tgt 01:13:04.831 nvme_pmr 01:13:04.831 nvme_pmr_persistence 01:13:04.831 nvme_smartctl_cuse 01:13:04.831 nvme_xnvme 01:13:04.831 nvme_zns 01:13:04.831 nvmf_abort 01:13:04.831 nvmf_abort_qd_sizes 01:13:04.831 nvmf_aer 01:13:04.831 nvmf_async_init 01:13:04.831 nvmf_auth_host 01:13:04.831 nvmf_auth_target 01:13:04.831 nvmf_bdev_io_wait 01:13:04.831 nvmf_bdevio 01:13:04.831 nvmf_bdevio_no_huge 01:13:04.831 nvmf_bdevperf 01:13:04.831 nvmf_connect_disconnect 01:13:04.831 nvmf_connect_stress 01:13:04.831 nvmf_delete_subsystem 01:13:04.831 nvmf_device_removal 01:13:04.831 nvmf_device_removal_pci_remove 01:13:04.831 nvmf_device_removal_pci_remove_no_srq 01:13:04.831 nvmf_dif 01:13:04.831 nvmf_digest 01:13:04.831 nvmf_digest_clean 01:13:04.831 nvmf_digest_dsa_initiator 01:13:04.831 nvmf_digest_dsa_target 01:13:04.831 nvmf_digest_error 01:13:04.831 nvmf_discovery_remove_ifc 01:13:04.831 nvmf_example 01:13:04.831 nvmf_failover 01:13:04.831 nvmf_fc 01:13:04.831 nvmf_filesystem 01:13:04.831 nvmf_filesystem_in_capsule 01:13:04.831 nvmf_filesystem_no_in_capsule 01:13:04.831 nvmf_fio_host 01:13:04.831 nvmf_fio_target 01:13:04.831 nvmf_fips 01:13:04.831 nvmf_fused_ordering 01:13:04.831 nvmf_fuzz 01:13:04.831 nvmf_host_discovery 01:13:04.831 nvmf_host_management 01:13:04.831 nvmf_host_multipath 01:13:04.831 nvmf_host_multipath_status 01:13:04.831 nvmf_identify 01:13:04.831 nvmf_identify_kernel_target 01:13:04.831 nvmf_identify_passthru 01:13:04.831 nvmf_initiator_timeout 01:13:04.831 nvmf_invalid 01:13:04.831 nvmf_lvol 01:13:04.831 nvmf_lvs_grow 01:13:04.831 nvmf_mdns_discovery 01:13:04.831 nvmf_multiconnection 01:13:04.831 nvmf_multicontroller 01:13:04.831 nvmf_multitarget 01:13:04.831 nvmf_nmic 01:13:04.831 nvmf_ns_hotplug_stress 01:13:04.831 nvmf_ns_masking 01:13:04.831 nvmf_nvme_cli 01:13:04.831 nvmf_perf 01:13:04.831 nvmf_perf_adq 01:13:04.831 nvmf_queue_depth 01:13:04.831 nvmf_rdma 01:13:04.831 nvmf_referrals 01:13:04.831 nvmf_rpc 01:13:04.831 nvmf_shutdown 01:13:04.831 nvmf_shutdown_tc1 01:13:04.831 nvmf_shutdown_tc2 01:13:04.831 nvmf_shutdown_tc3 01:13:04.831 nvmf_srq_overwhelm 01:13:04.831 nvmf_target_disconnect 01:13:04.831 nvmf_target_disconnect_tc1 01:13:04.831 nvmf_target_disconnect_tc2 01:13:04.831 nvmf_target_disconnect_tc3 01:13:04.831 nvmf_target_discovery 01:13:04.831 nvmf_target_multipath 01:13:04.831 nvmf_tcp 01:13:04.831 nvmf_timeout 01:13:04.831 nvmf_tls 01:13:04.831 nvmf_vfio_user 01:13:04.831 nvmf_vfio_user_fuzz 01:13:04.831 nvmf_vfio_user_nvme_compliance 01:13:04.831 nvmf_zcopy 01:13:04.831 ocf 01:13:04.831 ocf_bdevperf_iotypes 01:13:04.831 ocf_configuration_change 01:13:04.831 ocf_create_destruct 01:13:04.831 ocf_fio_modes 01:13:04.831 ocf_flush 01:13:04.831 ocf_multicore 01:13:04.831 ocf_remove 01:13:04.831 ocf_stats 01:13:04.831 scheduler 01:13:04.831 spdk_dd_uring 01:13:04.831 spdk_target_abort 01:13:04.831 spdkcli_iscsi 01:13:04.831 spdkcli_nvmf_fc 01:13:04.831 spdkcli_nvmf_rdma 01:13:04.831 spdkcli_nvmf_tcp 01:13:04.831 spdkcli_raid 01:13:04.831 spdkcli_rbd 01:13:04.831 spdkcli_virtio 01:13:04.831 test_create_multi_ublk 01:13:04.831 test_create_ublk 01:13:04.831 test_save_ublk_config 01:13:04.831 ublk 01:13:04.831 ublk_recovery 01:13:04.831 unittest_nvmf_fc 01:13:04.831 unittest_nvmf_fc_ls 01:13:04.831 valgrind 01:13:04.831 vfio_fuzz 01:13:04.831 vhost_blk_hot_remove 01:13:04.831 vhost_blockdev 01:13:04.831 vhost_fuzz 01:13:04.831 vhost_hotplug 01:13:04.831 vhost_scsi_hot_remove 01:13:04.831 vhost_shared 01:13:04.831 xnvme_bdevperf 01:13:04.831 xnvme_to_malloc_dd_copy 01:13:04.831 zoned_fio 01:13:04.831 01:13:04.831 01:13:04.831 -----Tests Missing UBSAN------ 01:13:04.831 balanced 01:13:04.831 blockdev_crypto_qat 01:13:04.831 blockdev_xnvme 01:13:04.831 busy 01:13:04.831 chaining 01:13:04.831 cmb_copy 01:13:04.831 core_load 01:13:04.831 ftl 01:13:04.831 ftl_bdevperf 01:13:04.831 ftl_dirty_shutdown 01:13:04.831 ftl_fio_basic 01:13:04.831 ftl_restore 01:13:04.831 ftl_restore_fast 01:13:04.831 ftl_trim 01:13:04.831 ftl_upgrade_shutdown 01:13:04.831 is_zoned 01:13:04.831 iscsi_tgt_data_digest 01:13:04.831 keyring_linux 01:13:04.831 nvme_boot_partition 01:13:04.831 nvme_bp 01:13:04.831 nvme_cmb 01:13:04.831 nvme_opal 01:13:04.831 nvme_opal_bdevio 01:13:04.831 nvme_opal_bdevperf 01:13:04.831 nvme_opal_spdk_tgt 01:13:04.831 nvme_pmr 01:13:04.831 nvme_pmr_persistence 01:13:04.831 nvme_xnvme 01:13:04.831 nvme_zns 01:13:04.831 nvmf_digest_dsa_initiator 01:13:04.831 nvmf_digest_dsa_target 01:13:04.831 nvmf_fc 01:13:04.831 spdkcli_nvmf_fc 01:13:04.831 test_create_multi_ublk 01:13:04.831 test_create_ublk 01:13:04.831 test_save_ublk_config 01:13:04.831 ublk 01:13:04.831 ublk_recovery 01:13:04.831 unittest_nvmf_fc 01:13:04.831 unittest_nvmf_fc_ls 01:13:04.831 vhost_blk_hot_remove 01:13:04.831 vhost_hotplug 01:13:04.831 vhost_scsi_hot_remove 01:13:04.831 xnvme_bdevperf 01:13:04.831 xnvme_to_malloc_dd_copy 01:13:04.831 zoned_fio 01:13:04.831 Not all tests were run. Failing the build. 01:13:04.831 ['blockdev_crypto_qat', 'blockdev_xnvme', 'chaining', 'ftl', 'ftl_bdevperf', 'ftl_dirty_shutdown', 'ftl_fio_basic', 'ftl_restore', 'ftl_trim', 'ftl_upgrade_shutdown', 'nvme_xnvme', 'test_create_multi_ublk', 'test_create_ublk', 'test_save_ublk_config', 'ublk', 'ublk_recovery', 'xnvme_bdevperf', 'xnvme_to_malloc_dd_copy'] 01:13:04.831 + compress_coverage_and_docs 01:13:04.831 + echo 'Start compress coverage and docs' 01:13:04.831 Start compress coverage and docs 01:13:04.831 + tar -C coverage -czf coverage_autotest-spdk-master-vs-dpdk-v23.11_424.tar.gz ./ --remove-files 01:13:04.831 + tar -C ut_coverage -czf ut_coverage_autotest-spdk-master-vs-dpdk-v23.11_424.tar.gz ./ --remove-files 01:13:05.396 + tar -C doc -czf doc_autotest-spdk-master-vs-dpdk-v23.11_424.tar.gz ./ --remove-files 01:13:05.654 + [[ -d llvm ]] 01:13:05.654 + true 01:13:05.654 + echo 'End compress coverage and docs' 01:13:05.654 End compress coverage and docs 01:13:05.654 + remove_partial_coverage_files 01:13:05.654 + find . -name 'cov_*.info' -delete 01:13:05.654 + echo 'End stage post_gen_coverage.sh' 01:13:05.654 End stage post_gen_coverage.sh 01:13:05.669 [Pipeline] } 01:13:05.672 ERROR: script returned exit code 1 01:13:05.698 [Pipeline] // catchError 01:13:05.705 [Pipeline] catchError 01:13:05.707 [Pipeline] { 01:13:05.724 [Pipeline] dir 01:13:05.724 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-v23.11/post_process 01:13:05.725 [Pipeline] {