00:00:00.001 Started by user sys_sgci 00:00:00.009 Checking out git https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool into /var/jenkins_home/workspace/autotest-per-patch_script/33b20b30f0a51e6b52980845e0f6aa336787973ad45e341fbbf98d1b65b265d4 to read jbp/jenkins/jjb-config/jobs/autotest-upstream/autotest.groovy 00:00:01.563 The recommended git tool is: git 00:00:01.563 using credential 00000000-0000-0000-0000-000000000002 00:00:01.566 > git rev-parse --resolve-git-dir /var/jenkins_home/workspace/autotest-per-patch_script/33b20b30f0a51e6b52980845e0f6aa336787973ad45e341fbbf98d1b65b265d4/jbp/.git # timeout=10 00:00:01.581 Fetching changes from the remote Git repository 00:00:01.583 > git config remote.origin.url https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool # timeout=10 00:00:01.598 Using shallow fetch with depth 1 00:00:01.598 Fetching upstream changes from https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool 00:00:01.598 > git --version # timeout=10 00:00:01.611 > git --version # 'git version 2.39.2' 00:00:01.611 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:01.628 Setting http proxy: proxy-dmz.intel.com:911 00:00:01.628 > git fetch --tags --force --progress --depth=1 -- https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool refs/heads/master # timeout=5 00:00:03.663 > git rev-parse origin/FETCH_HEAD^{commit} # timeout=10 00:00:03.675 > git rev-parse FETCH_HEAD^{commit} # timeout=10 00:00:03.687 Checking out Revision f95f9907808933a1db7196e15e13478e0f322ee7 (FETCH_HEAD) 00:00:03.687 > git config core.sparsecheckout # timeout=10 00:00:03.697 > git read-tree -mu HEAD # timeout=10 00:00:03.713 > git checkout -f f95f9907808933a1db7196e15e13478e0f322ee7 # timeout=5 00:00:03.731 Commit message: "Revert "autotest-phy: replace deprecated label for nvmf-cvl"" 00:00:03.731 > git rev-list --no-walk f95f9907808933a1db7196e15e13478e0f322ee7 # timeout=10 00:00:03.921 [Pipeline] Start of Pipeline 00:00:03.937 [Pipeline] library 00:00:03.939 Loading library shm_lib@master 00:00:03.939 Library shm_lib@master is cached. Copying from home. 00:00:03.961 [Pipeline] node 00:00:03.967 Running on ME3 in /var/jenkins/workspace/autotest-per-patch_3 00:00:03.972 [Pipeline] { 00:00:03.986 [Pipeline] cleanWs 00:00:03.995 [WS-CLEANUP] Deleting project workspace... 00:00:03.995 [WS-CLEANUP] Deferred wipeout is used... 00:00:04.001 [WS-CLEANUP] done 00:00:04.006 [Pipeline] stage 00:00:04.010 [Pipeline] { (Prologue) 00:00:04.138 [Pipeline] withCredentials 00:00:04.147 > git --version # timeout=10 00:00:04.160 > git --version # 'git version 2.39.2' 00:00:04.173 Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD or $GIT_ASKPASS 00:00:04.175 [Pipeline] { 00:00:04.183 [Pipeline] retry 00:00:04.184 [Pipeline] { 00:00:04.455 [Pipeline] sh 00:00:04.730 + git ls-remote https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool refs/heads/master 00:00:06.651 [Pipeline] } 00:00:06.675 [Pipeline] // retry 00:00:06.680 [Pipeline] } 00:00:06.700 [Pipeline] // withCredentials 00:00:06.710 [Pipeline] httpRequest 00:00:07.107 [Pipeline] echo 00:00:07.109 Sorcerer 10.211.164.101 is alive 00:00:07.118 [Pipeline] retry 00:00:07.120 [Pipeline] { 00:00:07.134 [Pipeline] httpRequest 00:00:07.139 HttpMethod: GET 00:00:07.139 URL: http://10.211.164.101/packages/jbp_f95f9907808933a1db7196e15e13478e0f322ee7.tar.gz 00:00:07.140 Sending request to url: http://10.211.164.101/packages/jbp_f95f9907808933a1db7196e15e13478e0f322ee7.tar.gz 00:00:07.141 Response Code: HTTP/1.1 200 OK 00:00:07.142 Success: Status code 200 is in the accepted range: 200,404 00:00:07.142 Saving response body to /var/jenkins/workspace/autotest-per-patch_3/jbp_f95f9907808933a1db7196e15e13478e0f322ee7.tar.gz 00:00:07.295 [Pipeline] } 00:00:07.313 [Pipeline] // retry 00:00:07.321 [Pipeline] sh 00:00:07.599 + tar --no-same-owner -xf jbp_f95f9907808933a1db7196e15e13478e0f322ee7.tar.gz 00:00:07.620 [Pipeline] httpRequest 00:00:08.016 [Pipeline] echo 00:00:08.018 Sorcerer 10.211.164.101 is alive 00:00:08.028 [Pipeline] retry 00:00:08.030 [Pipeline] { 00:00:08.045 [Pipeline] httpRequest 00:00:08.050 HttpMethod: GET 00:00:08.050 URL: http://10.211.164.101/packages/spdk_2a4f56c54d9adf89f0a3da3351edcad2c0a1ed33.tar.gz 00:00:08.051 Sending request to url: http://10.211.164.101/packages/spdk_2a4f56c54d9adf89f0a3da3351edcad2c0a1ed33.tar.gz 00:00:08.052 Response Code: HTTP/1.1 404 Not Found 00:00:08.053 Success: Status code 404 is in the accepted range: 200,404 00:00:08.053 Saving response body to /var/jenkins/workspace/autotest-per-patch_3/spdk_2a4f56c54d9adf89f0a3da3351edcad2c0a1ed33.tar.gz 00:00:08.059 [Pipeline] } 00:00:08.077 [Pipeline] // retry 00:00:08.084 [Pipeline] sh 00:00:08.360 + rm -f spdk_2a4f56c54d9adf89f0a3da3351edcad2c0a1ed33.tar.gz 00:00:08.368 [Pipeline] retry 00:00:08.370 [Pipeline] { 00:00:08.383 [Pipeline] checkout 00:00:08.388 The recommended git tool is: NONE 00:00:08.395 using credential 00000000-0000-0000-0000-000000000002 00:00:08.397 Wiping out workspace first. 00:00:08.402 Cloning the remote Git repository 00:00:08.404 Honoring refspec on initial clone 00:00:08.405 Cloning repository https://review.spdk.io/gerrit/a/spdk/spdk 00:00:08.405 > git init /var/jenkins/workspace/autotest-per-patch_3/spdk # timeout=10 00:00:08.409 Using reference repository: /var/ci_repos/spdk_multi 00:00:08.409 Fetching upstream changes from https://review.spdk.io/gerrit/a/spdk/spdk 00:00:08.409 > git --version # timeout=10 00:00:08.410 > git --version # 'git version 2.25.1' 00:00:08.410 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:08.412 Setting http proxy: proxy-dmz.intel.com:911 00:00:08.412 > git fetch --tags --force --progress -- https://review.spdk.io/gerrit/a/spdk/spdk refs/changes/30/25030/14 +refs/heads/master:refs/remotes/origin/master # timeout=10 00:00:18.385 Avoid second fetch 00:00:18.476 Checking out Revision 2a4f56c54d9adf89f0a3da3351edcad2c0a1ed33 (FETCH_HEAD) 00:00:18.906 Commit message: "bdev/nvme: controller failover/multipath doc change" 00:00:18.946 First time build. Skipping changelog. 00:00:18.256 > git config remote.origin.url https://review.spdk.io/gerrit/a/spdk/spdk # timeout=10 00:00:18.289 > git config --add remote.origin.fetch refs/changes/30/25030/14 # timeout=10 00:00:18.330 > git config --add remote.origin.fetch +refs/heads/master:refs/remotes/origin/master # timeout=10 00:00:18.385 > git rev-parse origin/FETCH_HEAD^{commit} # timeout=10 00:00:18.437 > git rev-parse FETCH_HEAD^{commit} # timeout=10 00:00:18.477 > git config core.sparsecheckout # timeout=10 00:00:18.518 > git checkout -f 2a4f56c54d9adf89f0a3da3351edcad2c0a1ed33 # timeout=10 00:00:18.907 > git rev-list --no-walk 997975c2e2c41ee2ed90318a9356ab5b553b8e7f # timeout=10 00:00:18.949 > git remote # timeout=10 00:00:18.988 > git submodule init # timeout=10 00:00:19.331 > git submodule sync # timeout=10 00:00:19.595 > git config --get remote.origin.url # timeout=10 00:00:19.634 > git submodule init # timeout=10 00:00:19.760 > git config -f .gitmodules --get-regexp ^submodule\.(.+)\.url # timeout=10 00:00:19.789 > git config --get submodule.dpdk.url # timeout=10 00:00:19.801 > git remote # timeout=10 00:00:19.805 > git config --get remote.origin.url # timeout=10 00:00:19.811 > git config -f .gitmodules --get submodule.dpdk.path # timeout=10 00:00:19.813 > git config --get submodule.intel-ipsec-mb.url # timeout=10 00:00:19.817 > git remote # timeout=10 00:00:19.821 > git config --get remote.origin.url # timeout=10 00:00:19.825 > git config -f .gitmodules --get submodule.intel-ipsec-mb.path # timeout=10 00:00:19.827 > git config --get submodule.isa-l.url # timeout=10 00:00:19.829 > git remote # timeout=10 00:00:19.833 > git config --get remote.origin.url # timeout=10 00:00:19.837 > git config -f .gitmodules --get submodule.isa-l.path # timeout=10 00:00:19.839 > git config --get submodule.ocf.url # timeout=10 00:00:19.841 > git remote # timeout=10 00:00:19.843 > git config --get remote.origin.url # timeout=10 00:00:19.844 > git config -f .gitmodules --get submodule.ocf.path # timeout=10 00:00:19.854 > git config --get submodule.libvfio-user.url # timeout=10 00:00:19.864 > git remote # timeout=10 00:00:19.869 > git config --get remote.origin.url # timeout=10 00:00:19.873 > git config -f .gitmodules --get submodule.libvfio-user.path # timeout=10 00:00:19.877 > git config --get submodule.xnvme.url # timeout=10 00:00:19.879 > git remote # timeout=10 00:00:19.881 > git config --get remote.origin.url # timeout=10 00:00:19.883 > git config -f .gitmodules --get submodule.xnvme.path # timeout=10 00:00:19.884 > git config --get submodule.isa-l-crypto.url # timeout=10 00:00:19.889 > git remote # timeout=10 00:00:19.897 > git config --get remote.origin.url # timeout=10 00:00:19.903 > git config -f .gitmodules --get submodule.isa-l-crypto.path # timeout=10 00:00:19.914 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:19.914 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:19.914 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:19.915 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:19.915 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:19.915 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:19.917 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:19.920 Setting http proxy: proxy-dmz.intel.com:911 00:00:19.920 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi isa-l-crypto # timeout=10 00:00:19.925 Setting http proxy: proxy-dmz.intel.com:911 00:00:19.925 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi xnvme # timeout=10 00:00:19.926 Setting http proxy: proxy-dmz.intel.com:911 00:00:19.926 Setting http proxy: proxy-dmz.intel.com:911 00:00:19.926 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi libvfio-user # timeout=10 00:00:19.926 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi intel-ipsec-mb # timeout=10 00:00:19.927 Setting http proxy: proxy-dmz.intel.com:911 00:00:19.927 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi dpdk # timeout=10 00:00:19.930 Setting http proxy: proxy-dmz.intel.com:911 00:00:19.930 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi ocf # timeout=10 00:00:19.951 Setting http proxy: proxy-dmz.intel.com:911 00:00:19.951 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi isa-l # timeout=10 00:00:50.957 [Pipeline] dir 00:00:50.958 Running in /var/jenkins/workspace/autotest-per-patch_3/spdk 00:00:50.959 [Pipeline] { 00:00:50.973 [Pipeline] sh 00:00:51.250 ++ nproc 00:00:51.250 + threads=8 00:00:51.250 + git repack -a -d --threads=8 00:00:56.519 + git submodule foreach git repack -a -d --threads=8 00:00:56.777 Entering 'dpdk' 00:01:00.960 Entering 'intel-ipsec-mb' 00:01:00.960 Entering 'isa-l' 00:01:01.218 Entering 'isa-l-crypto' 00:01:01.218 Entering 'libvfio-user' 00:01:01.218 Entering 'ocf' 00:01:01.475 Entering 'xnvme' 00:01:01.733 + find .git -type f -name alternates -print -delete 00:01:01.733 .git/objects/info/alternates 00:01:01.733 .git/modules/ocf/objects/info/alternates 00:01:01.733 .git/modules/intel-ipsec-mb/objects/info/alternates 00:01:01.733 .git/modules/isa-l-crypto/objects/info/alternates 00:01:01.733 .git/modules/xnvme/objects/info/alternates 00:01:01.733 .git/modules/libvfio-user/objects/info/alternates 00:01:01.733 .git/modules/dpdk/objects/info/alternates 00:01:01.733 .git/modules/isa-l/objects/info/alternates 00:01:01.742 [Pipeline] } 00:01:01.762 [Pipeline] // dir 00:01:01.768 [Pipeline] } 00:01:01.784 [Pipeline] // retry 00:01:01.792 [Pipeline] sh 00:01:02.073 + hash pigz 00:01:02.073 + tar -cf spdk_2a4f56c54d9adf89f0a3da3351edcad2c0a1ed33.tar.gz -I pigz spdk 00:01:04.617 [Pipeline] retry 00:01:04.619 [Pipeline] { 00:01:04.634 [Pipeline] httpRequest 00:01:04.642 HttpMethod: PUT 00:01:04.642 URL: http://10.211.164.101/cgi-bin/sorcerer.py?group=packages&filename=spdk_2a4f56c54d9adf89f0a3da3351edcad2c0a1ed33.tar.gz 00:01:04.643 Sending request to url: http://10.211.164.101/cgi-bin/sorcerer.py?group=packages&filename=spdk_2a4f56c54d9adf89f0a3da3351edcad2c0a1ed33.tar.gz 00:01:07.158 Response Code: HTTP/1.1 200 OK 00:01:07.163 Success: Status code 200 is in the accepted range: 200 00:01:07.166 [Pipeline] } 00:01:07.186 [Pipeline] // retry 00:01:07.192 [Pipeline] echo 00:01:07.193 00:01:07.193 Locking 00:01:07.193 Waited 0s for lock 00:01:07.193 Everything Fine. Saved: /storage/packages/spdk_2a4f56c54d9adf89f0a3da3351edcad2c0a1ed33.tar.gz 00:01:07.193 00:01:07.196 [Pipeline] sh 00:01:07.479 + git -C spdk log --oneline -n5 00:01:07.479 2a4f56c54 bdev/nvme: controller failover/multipath doc change 00:01:07.479 d16db39ee bdev/nvme: removed 'multipath' param from spdk_bdev_nvme_create() 00:01:07.479 32fb30b70 bdev/nvme: changed default config to multipath 00:01:07.479 397c5fc31 bdev/nvme: ctrl config consistency check 00:01:07.479 3950cd1bb bdev/nvme: Change spdk_bdev_reset() to succeed if at least one nvme_ctrlr is reconnected 00:01:07.496 [Pipeline] setCustomBuildProperty 00:01:07.504 [Pipeline] setCustomBuildProperty 00:01:07.513 [Pipeline] catchError 00:01:07.514 [Pipeline] { 00:01:07.530 [Pipeline] sh 00:01:07.813 + git -C spdk describe --tags --abbrev=0 origin/master 00:01:07.828 [Pipeline] sh 00:01:08.110 + git -C spdk describe --tags --abbrev=0 --exclude=LTS HEAD 00:01:08.125 [Pipeline] echo 00:01:08.127 Branch: master 00:01:08.131 [Pipeline] fileExists 00:01:08.148 [Pipeline] readJSON 00:01:08.162 [Pipeline] } 00:01:08.184 [Pipeline] // catchError 00:01:08.195 [Pipeline] sh 00:01:08.479 + /var/jenkins/workspace/autotest-per-patch_3/jbp/jenkins/jjb-config/jobs/scripts/get-pkgdep-jobs.sh /var/jenkins/workspace/autotest-per-patch_3/spdk 00:01:08.495 [Pipeline] } 00:01:08.512 [Pipeline] // stage 00:01:08.528 [Pipeline] catchError 00:01:08.529 [Pipeline] { 00:01:08.545 [Pipeline] stage 00:01:08.547 [Pipeline] { (Pre tests) 00:01:08.580 [Pipeline] parallel 00:01:08.591 [Pipeline] { (Branch: check-format-docker-autotest) 00:01:08.593 [Pipeline] { (Branch: check-so-deps-docker-autotest) 00:01:08.595 [Pipeline] { (Branch: doc-docker-autotest) 00:01:08.596 [Pipeline] { (Branch: build-files-docker-autotest) 00:01:08.615 [Pipeline] retry 00:01:08.617 [Pipeline] { 00:01:08.622 [Pipeline] retry 00:01:08.623 [Pipeline] { 00:01:08.628 [Pipeline] retry 00:01:08.630 [Pipeline] { 00:01:08.634 [Pipeline] retry 00:01:08.636 [Pipeline] { 00:01:08.652 [Pipeline] build 00:01:08.655 Scheduling project: check-format-docker-autotest 00:01:08.665 [Pipeline] build 00:01:08.669 Scheduling project: check-so-deps-docker-autotest 00:01:08.677 [Pipeline] build 00:01:08.679 Scheduling project: doc-docker-autotest 00:01:08.685 [Pipeline] build 00:01:08.688 Scheduling project: build-files-docker-autotest 00:01:14.106 Starting building: check-format-docker-autotest #33194 00:01:14.109 Starting building: build-files-docker-autotest #33168 00:01:14.112 Starting building: check-so-deps-docker-autotest #33415 00:01:14.115 Starting building: doc-docker-autotest #33497 00:02:07.031 Build check-format-docker-autotest #33194 completed: SUCCESS 00:02:07.033 [Pipeline] } 00:02:07.061 [Pipeline] // retry 00:02:07.067 [Pipeline] } 00:02:12.019 Build doc-docker-autotest #33497 completed: SUCCESS 00:02:12.022 [Pipeline] } 00:02:12.049 [Pipeline] // retry 00:02:12.055 [Pipeline] } 00:03:55.028 Build build-files-docker-autotest #33168 completed: SUCCESS 00:03:55.030 [Pipeline] } 00:03:55.057 [Pipeline] // retry 00:03:55.063 [Pipeline] } 00:04:19.763 Build check-so-deps-docker-autotest #33415 completed: SUCCESS 00:04:19.764 [Pipeline] } 00:04:19.786 [Pipeline] // retry 00:04:19.790 [Pipeline] } 00:04:19.827 [Pipeline] // parallel 00:04:19.833 [Pipeline] } 00:04:19.852 [Pipeline] // stage 00:04:19.857 [Pipeline] } 00:04:19.876 [Pipeline] // catchError 00:04:19.921 [Pipeline] catchError 00:04:19.923 [Pipeline] { 00:04:19.938 [Pipeline] stage 00:04:19.940 [Pipeline] { (Tests) 00:04:20.056 [Pipeline] parallel 00:04:20.113 [Pipeline] { (Branch: unittest-clang-docker-autotest) 00:04:20.114 [Pipeline] { (Branch: unittest-main-docker-autotest) 00:04:20.115 [Pipeline] { (Branch: autobuild-external-vg-autotest) 00:04:20.116 [Pipeline] { (Branch: scan-build-docker-autotest) 00:04:20.117 [Pipeline] { (Branch: unittest-valgrind-docker-autotest) 00:04:20.118 [Pipeline] { (Branch: lvol-vg-autotest) 00:04:20.118 [Pipeline] { (Branch: vhost-initiator-vg-autotest) 00:04:20.119 [Pipeline] { (Branch: nvmf-tcp-vg-autotest) 00:04:20.120 [Pipeline] { (Branch: nvmf-tcp-uring-vg-autotest) 00:04:20.121 [Pipeline] { (Branch: nvme-vg-autotest) 00:04:20.122 [Pipeline] { (Branch: rocky9-vg-autotest) 00:04:20.123 [Pipeline] { (Branch: ubuntu22-vg-autotest) 00:04:20.124 [Pipeline] { (Branch: ubuntu24-vg-autotest) 00:04:20.125 [Pipeline] { (Branch: freebsd-vg-autotest) 00:04:20.126 [Pipeline] { (Branch: iscsi-vg-autotest) 00:04:20.126 [Pipeline] { (Branch: iscsi-uring-vg-autotest) 00:04:20.127 [Pipeline] { (Branch: zns-vg-autotest) 00:04:20.128 [Pipeline] { (Branch: nvme-cmb-pmr-vg-autotest) 00:04:20.129 [Pipeline] { (Branch: raid-vg-autotest) 00:04:20.130 [Pipeline] { (Branch: BlobFS-phy-autotest) 00:04:20.131 [Pipeline] { (Branch: crypto-phy-autotest) 00:04:20.132 [Pipeline] { (Branch: vhost-phy-autotest) 00:04:20.133 [Pipeline] { (Branch: nvmf-phy-autotest) 00:04:20.134 [Pipeline] { (Branch: nvmf-tcp-phy-autotest) 00:04:20.134 [Pipeline] { (Branch: nvme-phy-autotest) 00:04:20.135 [Pipeline] { (Branch: vfio-user-phy-autotest) 00:04:20.136 [Pipeline] { (Branch: short-fuzz-phy-autotest) 00:04:20.137 [Pipeline] { (Branch: packaging-docker-autotest) 00:04:20.138 [Pipeline] { (Branch: release-build-main-docker-autotest) 00:04:20.139 [Pipeline] { (Branch: release-build-clang-docker-autotest) 00:04:20.158 [Pipeline] retry 00:04:20.159 [Pipeline] { 00:04:20.163 [Pipeline] retry 00:04:20.164 [Pipeline] { 00:04:20.167 [Pipeline] retry 00:04:20.168 [Pipeline] { 00:04:20.172 [Pipeline] retry 00:04:20.174 [Pipeline] { 00:04:20.177 [Pipeline] retry 00:04:20.179 [Pipeline] { 00:04:20.183 [Pipeline] retry 00:04:20.184 [Pipeline] { 00:04:20.187 [Pipeline] retry 00:04:20.188 [Pipeline] { 00:04:20.191 [Pipeline] retry 00:04:20.192 [Pipeline] { 00:04:20.196 [Pipeline] retry 00:04:20.197 [Pipeline] { 00:04:20.200 [Pipeline] retry 00:04:20.201 [Pipeline] { 00:04:20.204 [Pipeline] retry 00:04:20.206 [Pipeline] { 00:04:20.209 [Pipeline] retry 00:04:20.210 [Pipeline] { 00:04:20.212 [Pipeline] retry 00:04:20.215 [Pipeline] { 00:04:20.219 [Pipeline] retry 00:04:20.220 [Pipeline] { 00:04:20.223 [Pipeline] retry 00:04:20.224 [Pipeline] { 00:04:20.227 [Pipeline] retry 00:04:20.228 [Pipeline] { 00:04:20.231 [Pipeline] retry 00:04:20.232 [Pipeline] { 00:04:20.235 [Pipeline] retry 00:04:20.236 [Pipeline] { 00:04:20.240 [Pipeline] retry 00:04:20.241 [Pipeline] { 00:04:20.244 [Pipeline] retry 00:04:20.245 [Pipeline] { 00:04:20.248 [Pipeline] retry 00:04:20.249 [Pipeline] { 00:04:20.252 [Pipeline] retry 00:04:20.253 [Pipeline] { 00:04:20.256 [Pipeline] retry 00:04:20.257 [Pipeline] { 00:04:20.259 [Pipeline] retry 00:04:20.260 [Pipeline] { 00:04:20.263 [Pipeline] retry 00:04:20.264 [Pipeline] { 00:04:20.267 [Pipeline] retry 00:04:20.268 [Pipeline] { 00:04:20.271 [Pipeline] retry 00:04:20.272 [Pipeline] { 00:04:20.275 [Pipeline] retry 00:04:20.276 [Pipeline] { 00:04:20.279 [Pipeline] retry 00:04:20.280 [Pipeline] { 00:04:20.283 [Pipeline] retry 00:04:20.284 [Pipeline] { 00:04:20.299 [Pipeline] build 00:04:20.301 Scheduling project: unittest-clang-docker-autotest 00:04:20.306 [Pipeline] build 00:04:20.307 Scheduling project: unittest-main-docker-autotest 00:04:20.311 [Pipeline] build 00:04:20.313 Scheduling project: autobuild-external-vg-autotest 00:04:20.317 [Pipeline] build 00:04:20.319 Scheduling project: scan-build-docker-autotest 00:04:20.323 [Pipeline] build 00:04:20.325 Scheduling project: unittest-valgrind-docker-autotest 00:04:20.329 [Pipeline] build 00:04:20.330 Scheduling project: lvol-vg-autotest 00:04:20.335 [Pipeline] build 00:04:20.337 Scheduling project: vhost-initiator-vg-autotest 00:04:20.341 [Pipeline] build 00:04:20.343 Scheduling project: nvmf-tcp-vg-autotest 00:04:20.347 [Pipeline] build 00:04:20.349 Scheduling project: nvmf-tcp-uring-vg-autotest 00:04:20.354 [Pipeline] build 00:04:20.355 Scheduling project: nvme-vg-autotest 00:04:20.360 [Pipeline] build 00:04:20.362 Scheduling project: rocky9-vg-autotest 00:04:20.366 [Pipeline] build 00:04:20.368 Scheduling project: ubuntu22-vg-autotest 00:04:20.372 [Pipeline] build 00:04:20.374 Scheduling project: ubuntu24-vg-autotest 00:04:20.379 [Pipeline] build 00:04:20.381 Scheduling project: freebsd-vg-autotest 00:04:20.386 [Pipeline] build 00:04:20.387 Scheduling project: iscsi-vg-autotest 00:04:20.392 [Pipeline] build 00:04:20.394 Scheduling project: iscsi-uring-vg-autotest 00:04:20.398 [Pipeline] build 00:04:20.400 Scheduling project: zns-vg-autotest 00:04:20.404 [Pipeline] build 00:04:20.405 Scheduling project: nvme-cmb-pmr-vg-autotest 00:04:20.410 [Pipeline] build 00:04:20.411 Scheduling project: raid-vg-autotest 00:04:20.417 [Pipeline] build 00:04:20.419 Scheduling project: BlobFS-phy-autotest 00:04:20.424 [Pipeline] build 00:04:20.426 Scheduling project: crypto-phy-autotest 00:04:20.430 [Pipeline] build 00:04:20.432 Scheduling project: vhost-phy-autotest 00:04:20.436 [Pipeline] build 00:04:20.438 Scheduling project: nvmf-phy-autotest 00:04:20.443 [Pipeline] build 00:04:20.445 Scheduling project: nvmf-tcp-phy-autotest 00:04:20.450 [Pipeline] build 00:04:20.452 Scheduling project: nvme-phy-autotest 00:04:20.456 [Pipeline] build 00:04:20.457 Scheduling project: vfio-user-phy-autotest 00:04:20.462 [Pipeline] build 00:04:20.463 Scheduling project: short-fuzz-phy-autotest 00:04:20.476 [Pipeline] build 00:04:20.477 Scheduling project: packaging-docker-autotest 00:04:20.485 [Pipeline] build 00:04:20.486 Scheduling project: release-build-main-docker-autotest 00:04:20.491 [Pipeline] build 00:04:20.492 Scheduling project: release-build-clang-docker-autotest 00:04:25.398 Starting building: rocky9-vg-autotest #27871 00:04:25.401 Starting building: scan-build-docker-autotest #29948 00:04:25.404 Starting building: unittest-main-docker-autotest #29944 00:04:25.407 Starting building: unittest-clang-docker-autotest #29942 00:04:25.411 Starting building: autobuild-external-vg-autotest #30579 00:04:25.414 Starting building: ubuntu22-vg-autotest #49538 00:04:25.418 Starting building: vhost-initiator-vg-autotest #90686 00:04:25.422 Starting building: nvmf-tcp-vg-autotest #90650 00:04:25.425 Starting building: unittest-valgrind-docker-autotest #29946 00:04:25.429 Starting building: lvol-vg-autotest #90617 00:04:25.432 Starting building: nvme-vg-autotest #90668 00:04:25.435 Starting building: nvmf-tcp-uring-vg-autotest #90611 00:04:25.458 Starting building: ubuntu24-vg-autotest #7099 00:04:25.461 Starting building: raid-vg-autotest #5263 00:04:25.463 Starting building: vhost-phy-autotest #38373 00:04:25.466 Starting building: crypto-phy-autotest #38305 00:04:25.469 Starting building: iscsi-uring-vg-autotest #90569 00:04:25.471 Starting building: BlobFS-phy-autotest #38316 00:04:25.474 Starting building: nvme-cmb-pmr-vg-autotest #91797 00:04:25.477 Starting building: iscsi-vg-autotest #90705 00:04:25.480 Starting building: zns-vg-autotest #89770 00:04:25.483 Starting building: freebsd-vg-autotest #68599 00:04:25.803 Starting building: nvmf-phy-autotest #84548 00:04:25.805 Starting building: release-build-main-docker-autotest #3119 00:04:25.807 Starting building: vfio-user-phy-autotest #38306 00:04:25.810 Starting building: release-build-clang-docker-autotest #3119 00:04:25.812 Starting building: short-fuzz-phy-autotest #38301 00:04:25.815 Starting building: nvmf-tcp-phy-autotest #84430 00:04:25.818 Starting building: packaging-docker-autotest #29941 00:04:25.821 Starting building: nvme-phy-autotest #84479 00:10:40.054 Build release-build-clang-docker-autotest #3119 completed: SUCCESS 00:10:40.058 [Pipeline] } 00:10:40.099 [Pipeline] // retry 00:10:40.105 [Pipeline] } 00:12:02.565 Build packaging-docker-autotest #29941 completed: SUCCESS 00:12:02.567 [Pipeline] } 00:12:02.595 [Pipeline] // retry 00:12:02.600 [Pipeline] } 00:12:02.702 Build release-build-main-docker-autotest #3119 completed: SUCCESS 00:12:02.704 [Pipeline] } 00:12:02.741 [Pipeline] // retry 00:12:02.748 [Pipeline] } 00:12:05.348 Build scan-build-docker-autotest #29948 completed: SUCCESS 00:12:05.351 [Pipeline] } 00:12:05.390 [Pipeline] // retry 00:12:05.397 [Pipeline] } 00:12:27.352 Build unittest-clang-docker-autotest #29942 completed: SUCCESS 00:12:27.354 [Pipeline] } 00:12:27.391 [Pipeline] // retry 00:12:27.397 [Pipeline] } 00:15:12.160 Build short-fuzz-phy-autotest #38301 completed: SUCCESS 00:15:12.163 [Pipeline] } 00:15:12.203 [Pipeline] // retry 00:15:12.210 [Pipeline] } 00:15:15.554 Build unittest-main-docker-autotest #29944 completed: SUCCESS 00:15:15.556 [Pipeline] } 00:15:15.594 [Pipeline] // retry 00:15:15.600 [Pipeline] } 00:16:56.359 Build zns-vg-autotest #89770 completed: SUCCESS 00:16:56.361 [Pipeline] } 00:16:56.396 [Pipeline] // retry 00:16:56.402 [Pipeline] } 00:17:02.905 Build autobuild-external-vg-autotest #30579 completed: SUCCESS 00:17:02.907 [Pipeline] } 00:17:02.942 [Pipeline] // retry 00:17:02.947 [Pipeline] } 00:17:44.521 Build vhost-initiator-vg-autotest #90686 completed: SUCCESS 00:17:44.524 [Pipeline] } 00:17:44.562 [Pipeline] // retry 00:17:44.569 [Pipeline] } 00:17:48.921 Build unittest-valgrind-docker-autotest #29946 completed: SUCCESS 00:17:48.923 [Pipeline] } 00:17:48.969 [Pipeline] // retry 00:17:48.977 [Pipeline] } 00:18:53.722 Build nvme-cmb-pmr-vg-autotest #91797 completed: SUCCESS 00:18:53.723 [Pipeline] } 00:18:53.756 [Pipeline] // retry 00:18:53.762 [Pipeline] } 00:19:14.739 Build freebsd-vg-autotest #68599 completed: SUCCESS 00:19:14.742 [Pipeline] } 00:19:14.778 [Pipeline] // retry 00:19:14.781 [Pipeline] } 00:22:50.494 Build lvol-vg-autotest #90617 completed: SUCCESS 00:22:50.497 [Pipeline] } 00:22:50.532 [Pipeline] // retry 00:22:50.539 [Pipeline] } 00:22:52.002 Build BlobFS-phy-autotest #38316 completed: SUCCESS 00:22:52.004 [Pipeline] } 00:22:52.034 [Pipeline] // retry 00:22:52.041 [Pipeline] } 00:23:36.651 Build crypto-phy-autotest #38305 completed: SUCCESS 00:23:36.653 [Pipeline] } 00:23:36.686 [Pipeline] // retry 00:23:36.692 [Pipeline] } 00:23:36.934 Build iscsi-uring-vg-autotest #90569 completed: SUCCESS 00:23:36.936 [Pipeline] } 00:23:36.971 [Pipeline] // retry 00:23:36.977 [Pipeline] } 00:23:41.903 Build vhost-phy-autotest #38373 completed: SUCCESS 00:23:41.906 [Pipeline] } 00:23:41.942 [Pipeline] // retry 00:23:41.948 [Pipeline] } 00:25:12.714 Build vfio-user-phy-autotest #38306 completed: SUCCESS 00:25:12.716 [Pipeline] } 00:25:12.750 [Pipeline] // retry 00:25:12.755 [Pipeline] } 00:26:01.365 Build nvmf-tcp-uring-vg-autotest #90611 completed: FAILURE 00:26:03.012 [Pipeline] echo 00:26:03.013 No retry patterns found. 00:26:03.014 [Pipeline] } 00:26:03.041 [Pipeline] // retry 00:26:03.048 [Pipeline] error 00:26:03.052 [Pipeline] } 00:26:03.055 Failed in branch nvmf-tcp-uring-vg-autotest 00:26:16.577 Build iscsi-vg-autotest #90705 completed: SUCCESS 00:26:16.579 [Pipeline] } 00:26:16.615 [Pipeline] // retry 00:26:16.620 [Pipeline] } 00:27:41.051 Build nvmf-tcp-vg-autotest #90650 completed: FAILURE 00:27:42.654 [Pipeline] echo 00:27:42.655 No retry patterns found. 00:27:42.656 [Pipeline] } 00:27:42.685 [Pipeline] // retry 00:27:42.692 [Pipeline] error 00:27:42.698 [Pipeline] } 00:27:42.704 Failed in branch nvmf-tcp-vg-autotest 00:29:09.008 Build raid-vg-autotest #5263 completed: SUCCESS 00:29:09.010 [Pipeline] } 00:29:09.047 [Pipeline] // retry 00:29:09.053 [Pipeline] } 00:29:50.456 Build rocky9-vg-autotest #27871 completed: SUCCESS 00:29:50.459 [Pipeline] } 00:29:50.495 [Pipeline] // retry 00:29:50.502 [Pipeline] } 00:30:24.271 Build ubuntu24-vg-autotest #7099 completed: SUCCESS 00:30:24.273 [Pipeline] } 00:30:24.306 [Pipeline] // retry 00:30:24.313 [Pipeline] } 00:30:34.927 Build nvmf-phy-autotest #84548 completed: FAILURE 00:30:36.178 [Pipeline] echo 00:30:36.179 No retry patterns found. 00:30:36.180 [Pipeline] } 00:30:36.201 [Pipeline] // retry 00:30:36.209 [Pipeline] error 00:30:36.214 [Pipeline] } 00:30:36.219 Failed in branch nvmf-phy-autotest 00:30:42.221 Build nvme-phy-autotest #84479 completed: SUCCESS 00:30:42.223 [Pipeline] } 00:30:42.257 [Pipeline] // retry 00:30:42.263 [Pipeline] } 00:30:58.002 Build ubuntu22-vg-autotest #49538 completed: SUCCESS 00:30:58.005 [Pipeline] } 00:30:58.034 [Pipeline] // retry 00:30:58.037 [Pipeline] } 00:31:35.493 Build nvmf-tcp-phy-autotest #84430 completed: FAILURE 00:31:37.372 [Pipeline] echo 00:31:37.373 No retry patterns found. 00:31:37.373 [Pipeline] } 00:31:37.391 [Pipeline] // retry 00:31:37.398 [Pipeline] error 00:31:37.404 [Pipeline] } 00:31:37.409 Failed in branch nvmf-tcp-phy-autotest 00:31:57.120 Build nvme-vg-autotest #90668 completed: SUCCESS 00:31:57.123 [Pipeline] } 00:31:57.157 [Pipeline] // retry 00:31:57.163 [Pipeline] } 00:31:57.407 [Pipeline] // parallel 00:31:57.419 [Pipeline] } 00:31:57.450 [Pipeline] // stage 00:31:57.463 [Pipeline] } 00:31:57.475 ERROR: Build nvmf-tcp-uring-vg-autotest #90611 failed 00:31:57.475 Setting overall build result to FAILURE 00:31:57.498 [Pipeline] // catchError 00:31:57.509 [Pipeline] stage 00:31:57.511 [Pipeline] { (Autorun Post and Coverage) 00:31:57.537 [Pipeline] setCustomBuildProperty 00:31:57.559 [Pipeline] dir 00:31:57.560 Running in /var/jenkins/workspace/autotest-per-patch_3/check-format-docker-autotest_33194 00:31:57.562 [Pipeline] { 00:31:57.599 [Pipeline] copyArtifacts 00:31:57.659 Copied 5 artifacts from "check-format-docker-autotest" build number 33194 00:31:57.663 [Pipeline] writeFile 00:31:57.684 [Pipeline] } 00:31:57.710 [Pipeline] // dir 00:31:57.724 [Pipeline] dir 00:31:57.724 Running in /var/jenkins/workspace/autotest-per-patch_3/doc-docker-autotest_33497 00:31:57.726 [Pipeline] { 00:31:57.744 [Pipeline] copyArtifacts 00:31:58.020 Copied 6 artifacts from "doc-docker-autotest" build number 33497 00:31:58.025 [Pipeline] writeFile 00:31:58.048 [Pipeline] } 00:31:58.072 [Pipeline] // dir 00:31:58.168 [Pipeline] dir 00:31:58.168 Running in /var/jenkins/workspace/autotest-per-patch_3/build-files-docker-autotest_33168 00:31:58.170 [Pipeline] { 00:31:58.186 [Pipeline] copyArtifacts 00:31:58.233 Copied 5 artifacts from "build-files-docker-autotest" build number 33168 00:31:58.238 [Pipeline] writeFile 00:31:58.281 [Pipeline] } 00:31:58.357 [Pipeline] // dir 00:31:58.410 [Pipeline] dir 00:31:58.411 Running in /var/jenkins/workspace/autotest-per-patch_3/check-so-deps-docker-autotest_33415 00:31:58.412 [Pipeline] { 00:31:58.428 [Pipeline] copyArtifacts 00:31:58.478 Copied 5 artifacts from "check-so-deps-docker-autotest" build number 33415 00:31:58.482 [Pipeline] writeFile 00:31:58.509 [Pipeline] } 00:31:58.537 [Pipeline] // dir 00:31:58.593 [Pipeline] dir 00:31:58.593 Running in /var/jenkins/workspace/autotest-per-patch_3/release-build-clang-docker-autotest_3119 00:31:58.594 [Pipeline] { 00:31:58.613 [Pipeline] copyArtifacts 00:31:58.667 Copied 5 artifacts from "release-build-clang-docker-autotest" build number 3119 00:31:58.669 [Pipeline] writeFile 00:31:58.700 [Pipeline] } 00:31:58.729 [Pipeline] // dir 00:31:59.017 [Pipeline] dir 00:31:59.017 Running in /var/jenkins/workspace/autotest-per-patch_3/packaging-docker-autotest_29941 00:31:59.018 [Pipeline] { 00:31:59.051 [Pipeline] copyArtifacts 00:31:59.107 Copied 5 artifacts from "packaging-docker-autotest" build number 29941 00:31:59.111 [Pipeline] writeFile 00:31:59.281 [Pipeline] } 00:31:59.345 [Pipeline] // dir 00:31:59.397 [Pipeline] dir 00:31:59.398 Running in /var/jenkins/workspace/autotest-per-patch_3/release-build-main-docker-autotest_3119 00:31:59.403 [Pipeline] { 00:31:59.459 [Pipeline] copyArtifacts 00:31:59.592 Copied 5 artifacts from "release-build-main-docker-autotest" build number 3119 00:31:59.596 [Pipeline] writeFile 00:31:59.619 [Pipeline] } 00:31:59.643 [Pipeline] // dir 00:31:59.671 [Pipeline] dir 00:31:59.671 Running in /var/jenkins/workspace/autotest-per-patch_3/scan-build-docker-autotest_29948 00:31:59.673 [Pipeline] { 00:31:59.696 [Pipeline] copyArtifacts 00:31:59.804 Copied 9 artifacts from "scan-build-docker-autotest" build number 29948 00:31:59.809 [Pipeline] writeFile 00:31:59.830 [Pipeline] } 00:31:59.855 [Pipeline] // dir 00:31:59.993 [Pipeline] dir 00:31:59.993 Running in /var/jenkins/workspace/autotest-per-patch_3/unittest-clang-docker-autotest_29942 00:31:59.994 [Pipeline] { 00:32:00.016 [Pipeline] copyArtifacts 00:32:00.334 Copied 6 artifacts from "unittest-clang-docker-autotest" build number 29942 00:32:00.338 [Pipeline] writeFile 00:32:00.386 [Pipeline] } 00:32:00.434 [Pipeline] // dir 00:32:00.830 [Pipeline] dir 00:32:00.830 Running in /var/jenkins/workspace/autotest-per-patch_3/short-fuzz-phy-autotest_38301 00:32:00.832 [Pipeline] { 00:32:00.868 [Pipeline] copyArtifacts 00:32:01.121 Copied 14 artifacts from "short-fuzz-phy-autotest" build number 38301 00:32:01.125 [Pipeline] writeFile 00:32:01.283 [Pipeline] } 00:32:01.373 [Pipeline] // dir 00:32:01.539 [Pipeline] dir 00:32:01.539 Running in /var/jenkins/workspace/autotest-per-patch_3/unittest-main-docker-autotest_29944 00:32:01.541 [Pipeline] { 00:32:01.565 [Pipeline] copyArtifacts 00:32:01.863 Copied 6 artifacts from "unittest-main-docker-autotest" build number 29944 00:32:01.868 [Pipeline] writeFile 00:32:01.930 [Pipeline] } 00:32:01.993 [Pipeline] // dir 00:32:02.338 [Pipeline] dir 00:32:02.339 Running in /var/jenkins/workspace/autotest-per-patch_3/zns-vg-autotest_89770 00:32:02.340 [Pipeline] { 00:32:02.366 [Pipeline] copyArtifacts 00:32:02.584 Copied 13 artifacts from "zns-vg-autotest" build number 89770 00:32:02.589 [Pipeline] writeFile 00:32:02.667 [Pipeline] } 00:32:02.721 [Pipeline] // dir 00:32:02.873 [Pipeline] dir 00:32:02.874 Running in /var/jenkins/workspace/autotest-per-patch_3/autobuild-external-vg-autotest_30579 00:32:02.875 [Pipeline] { 00:32:02.899 [Pipeline] copyArtifacts 00:32:03.011 Copied 9 artifacts from "autobuild-external-vg-autotest" build number 30579 00:32:03.016 [Pipeline] writeFile 00:32:03.095 [Pipeline] } 00:32:03.136 [Pipeline] // dir 00:32:03.467 [Pipeline] dir 00:32:03.468 Running in /var/jenkins/workspace/autotest-per-patch_3/vhost-initiator-vg-autotest_90686 00:32:03.469 [Pipeline] { 00:32:03.502 [Pipeline] copyArtifacts 00:32:03.765 Copied 16 artifacts from "vhost-initiator-vg-autotest" build number 90686 00:32:03.769 [Pipeline] writeFile 00:32:03.879 [Pipeline] } 00:32:03.938 [Pipeline] // dir 00:32:04.398 [Pipeline] dir 00:32:04.398 Running in /var/jenkins/workspace/autotest-per-patch_3/unittest-valgrind-docker-autotest_29946 00:32:04.399 [Pipeline] { 00:32:04.440 [Pipeline] copyArtifacts 00:32:04.762 Copied 6 artifacts from "unittest-valgrind-docker-autotest" build number 29946 00:32:04.766 [Pipeline] writeFile 00:32:04.896 [Pipeline] } 00:32:04.982 [Pipeline] // dir 00:32:05.285 [Pipeline] dir 00:32:05.286 Running in /var/jenkins/workspace/autotest-per-patch_3/nvme-cmb-pmr-vg-autotest_91797 00:32:05.287 [Pipeline] { 00:32:05.312 [Pipeline] copyArtifacts 00:32:05.503 Copied 13 artifacts from "nvme-cmb-pmr-vg-autotest" build number 91797 00:32:05.507 [Pipeline] writeFile 00:32:05.592 [Pipeline] } 00:32:05.647 [Pipeline] // dir 00:32:05.913 [Pipeline] dir 00:32:05.914 Running in /var/jenkins/workspace/autotest-per-patch_3/freebsd-vg-autotest_68599 00:32:05.915 [Pipeline] { 00:32:05.943 [Pipeline] copyArtifacts 00:32:06.280 Copied 12 artifacts from "freebsd-vg-autotest" build number 68599 00:32:06.284 [Pipeline] writeFile 00:32:06.371 [Pipeline] } 00:32:06.432 [Pipeline] // dir 00:32:06.978 [Pipeline] dir 00:32:06.978 Running in /var/jenkins/workspace/autotest-per-patch_3/lvol-vg-autotest_90617 00:32:06.979 [Pipeline] { 00:32:07.018 [Pipeline] copyArtifacts 00:32:07.206 Copied 13 artifacts from "lvol-vg-autotest" build number 90617 00:32:07.210 [Pipeline] writeFile 00:32:07.401 [Pipeline] } 00:32:07.497 [Pipeline] // dir 00:32:07.644 [Pipeline] dir 00:32:07.645 Running in /var/jenkins/workspace/autotest-per-patch_3/BlobFS-phy-autotest_38316 00:32:07.645 [Pipeline] { 00:32:07.667 [Pipeline] copyArtifacts 00:32:08.007 Copied 29 artifacts from "BlobFS-phy-autotest" build number 38316 00:32:08.011 [Pipeline] writeFile 00:32:08.072 [Pipeline] } 00:32:08.103 [Pipeline] // dir 00:32:08.666 [Pipeline] dir 00:32:08.666 Running in /var/jenkins/workspace/autotest-per-patch_3/crypto-phy-autotest_38305 00:32:08.667 [Pipeline] { 00:32:08.712 [Pipeline] copyArtifacts 00:32:08.899 Copied 13 artifacts from "crypto-phy-autotest" build number 38305 00:32:08.903 [Pipeline] writeFile 00:32:09.146 [Pipeline] } 00:32:09.252 [Pipeline] // dir 00:32:09.835 [Pipeline] dir 00:32:09.835 Running in /var/jenkins/workspace/autotest-per-patch_3/iscsi-uring-vg-autotest_90569 00:32:09.836 [Pipeline] { 00:32:09.866 [Pipeline] copyArtifacts 00:32:12.450 Copied 17 artifacts from "iscsi-uring-vg-autotest" build number 90569 00:32:12.455 [Pipeline] writeFile 00:32:12.546 [Pipeline] } 00:32:12.622 [Pipeline] // dir 00:32:12.857 [Pipeline] dir 00:32:12.857 Running in /var/jenkins/workspace/autotest-per-patch_3/vhost-phy-autotest_38373 00:32:12.859 [Pipeline] { 00:32:12.887 [Pipeline] copyArtifacts 00:32:13.094 Copied 13 artifacts from "vhost-phy-autotest" build number 38373 00:32:13.098 [Pipeline] writeFile 00:32:13.182 [Pipeline] } 00:32:13.236 [Pipeline] // dir 00:32:13.543 [Pipeline] dir 00:32:13.544 Running in /var/jenkins/workspace/autotest-per-patch_3/vfio-user-phy-autotest_38306 00:32:13.545 [Pipeline] { 00:32:13.575 [Pipeline] copyArtifacts 00:32:13.776 Copied 13 artifacts from "vfio-user-phy-autotest" build number 38306 00:32:13.778 [Pipeline] writeFile 00:32:13.911 [Pipeline] } 00:32:13.985 [Pipeline] // dir 00:32:14.982 [Pipeline] dir 00:32:14.982 Running in /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-uring-vg-autotest_90611 00:32:14.983 [Pipeline] { 00:32:15.051 [Pipeline] copyArtifacts 00:32:15.504 Copied 13 artifacts from "nvmf-tcp-uring-vg-autotest" build number 90611 00:32:15.509 [Pipeline] writeFile 00:32:16.037 [Pipeline] } 00:32:16.235 [Pipeline] // dir 00:32:16.545 [Pipeline] dir 00:32:16.545 Running in /var/jenkins/workspace/autotest-per-patch_3/iscsi-vg-autotest_90705 00:32:16.546 [Pipeline] { 00:32:16.574 [Pipeline] copyArtifacts 00:32:18.904 Copied 17 artifacts from "iscsi-vg-autotest" build number 90705 00:32:18.908 [Pipeline] writeFile 00:32:19.008 [Pipeline] } 00:32:19.068 [Pipeline] // dir 00:32:20.109 [Pipeline] dir 00:32:20.110 Running in /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-vg-autotest_90650 00:32:20.111 [Pipeline] { 00:32:20.190 [Pipeline] copyArtifacts 00:32:20.683 Copied 13 artifacts from "nvmf-tcp-vg-autotest" build number 90650 00:32:20.687 [Pipeline] writeFile 00:32:21.171 [Pipeline] } 00:32:21.374 [Pipeline] // dir 00:32:22.333 [Pipeline] dir 00:32:22.334 Running in /var/jenkins/workspace/autotest-per-patch_3/raid-vg-autotest_5263 00:32:22.335 [Pipeline] { 00:32:22.389 [Pipeline] copyArtifacts 00:32:22.596 Copied 13 artifacts from "raid-vg-autotest" build number 5263 00:32:22.600 [Pipeline] writeFile 00:32:23.004 [Pipeline] } 00:32:23.165 [Pipeline] // dir 00:32:23.518 [Pipeline] dir 00:32:23.519 Running in /var/jenkins/workspace/autotest-per-patch_3/rocky9-vg-autotest_27871 00:32:23.520 [Pipeline] { 00:32:23.552 [Pipeline] copyArtifacts 00:32:23.963 Copied 13 artifacts from "rocky9-vg-autotest" build number 27871 00:32:23.968 [Pipeline] writeFile 00:32:24.096 [Pipeline] } 00:32:24.164 [Pipeline] // dir 00:32:24.579 [Pipeline] dir 00:32:24.580 Running in /var/jenkins/workspace/autotest-per-patch_3/ubuntu24-vg-autotest_7099 00:32:24.581 [Pipeline] { 00:32:24.613 [Pipeline] copyArtifacts 00:32:24.990 Copied 13 artifacts from "ubuntu24-vg-autotest" build number 7099 00:32:24.994 [Pipeline] writeFile 00:32:25.178 [Pipeline] } 00:32:25.266 [Pipeline] // dir 00:32:25.921 [Pipeline] dir 00:32:25.922 Running in /var/jenkins/workspace/autotest-per-patch_3/nvmf-phy-autotest_84548 00:32:25.923 [Pipeline] { 00:32:25.972 [Pipeline] copyArtifacts 00:32:26.407 Copied 13 artifacts from "nvmf-phy-autotest" build number 84548 00:32:26.412 [Pipeline] writeFile 00:32:26.661 [Pipeline] } 00:32:26.768 [Pipeline] // dir 00:32:27.337 [Pipeline] dir 00:32:27.338 Running in /var/jenkins/workspace/autotest-per-patch_3/nvme-phy-autotest_84479 00:32:27.339 [Pipeline] { 00:32:27.380 [Pipeline] copyArtifacts 00:32:27.616 Copied 13 artifacts from "nvme-phy-autotest" build number 84479 00:32:27.621 [Pipeline] writeFile 00:32:27.828 [Pipeline] } 00:32:27.933 [Pipeline] // dir 00:32:28.359 [Pipeline] dir 00:32:28.360 Running in /var/jenkins/workspace/autotest-per-patch_3/ubuntu22-vg-autotest_49538 00:32:28.361 [Pipeline] { 00:32:28.454 [Pipeline] copyArtifacts 00:32:28.783 Copied 13 artifacts from "ubuntu22-vg-autotest" build number 49538 00:32:28.787 [Pipeline] writeFile 00:32:28.942 [Pipeline] } 00:32:29.037 [Pipeline] // dir 00:32:30.147 [Pipeline] dir 00:32:30.147 Running in /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-phy-autotest_84430 00:32:30.148 [Pipeline] { 00:32:30.231 [Pipeline] copyArtifacts 00:32:30.735 Copied 13 artifacts from "nvmf-tcp-phy-autotest" build number 84430 00:32:30.739 [Pipeline] writeFile 00:32:31.237 [Pipeline] } 00:32:31.417 [Pipeline] // dir 00:32:31.987 [Pipeline] dir 00:32:31.988 Running in /var/jenkins/workspace/autotest-per-patch_3/nvme-vg-autotest_90668 00:32:31.989 [Pipeline] { 00:32:32.027 [Pipeline] copyArtifacts 00:32:32.235 Copied 13 artifacts from "nvme-vg-autotest" build number 90668 00:32:32.238 [Pipeline] writeFile 00:32:32.466 [Pipeline] } 00:32:32.583 [Pipeline] // dir 00:32:32.594 [Pipeline] catchError 00:32:32.596 [Pipeline] { 00:32:32.615 [Pipeline] sh 00:32:32.893 + jbp/jenkins/jjb-config/jobs/scripts/post_gen_coverage.sh 00:32:32.893 + shopt -s globstar nullglob 00:32:32.893 + echo 'Start stage post_gen_coverage.sh' 00:32:32.893 Start stage post_gen_coverage.sh 00:32:32.893 + cd /var/jenkins/workspace/autotest-per-patch_3 00:32:32.893 + rm -rf /var/jenkins/workspace/autotest-per-patch_3/spdk/doc 00:32:32.893 + trap 'compress_coverage_and_docs; remove_partial_coverage_files && echo '\''End stage post_gen_coverage.sh'\''' EXIT 00:32:32.893 + move_artifacts 00:32:32.893 + local out_dirs 00:32:32.893 + out_dirs=(./**/output/) 00:32:32.893 + for dir in "${out_dirs[@]}" 00:32:32.893 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.893 + [[ -f ./autobuild-external-vg-autotest_30579/output//doc.tar.xz ]] 00:32:32.893 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.893 + [[ -f ./autobuild-external-vg-autotest_30579/output//ut_coverage.tar.xz ]] 00:32:32.893 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.893 + [[ -f ./autobuild-external-vg-autotest_30579/output//llvm.tar.xz ]] 00:32:32.893 + mv ./autobuild-external-vg-autotest_30579/output//build-kernel-manifest.txt ./autobuild-external-vg-autotest_30579/output//build-pkg-manifest.txt ./autobuild-external-vg-autotest_30579/output//build-repo-manifest.txt ./autobuild-external-vg-autotest_30579/output//dmesg.init.log ./autobuild-external-vg-autotest_30579/output//dmesg.log ./autobuild-external-vg-autotest_30579/output//power.tar.xz ./autobuild-external-vg-autotest_30579/output//test_completions.txt ./autobuild-external-vg-autotest_30579/output//timing.svg ./autobuild-external-vg-autotest_30579/output//timing.txt ./autobuild-external-vg-autotest_30579/output//.. 00:32:32.893 + rmdir ./autobuild-external-vg-autotest_30579/output/ 00:32:32.893 + for dir in "${out_dirs[@]}" 00:32:32.893 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.893 + [[ -f ./BlobFS-phy-autotest_38316/output//doc.tar.xz ]] 00:32:32.893 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.893 + [[ -f ./BlobFS-phy-autotest_38316/output//ut_coverage.tar.xz ]] 00:32:32.893 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.893 + [[ -f ./BlobFS-phy-autotest_38316/output//llvm.tar.xz ]] 00:32:32.893 + mv ./BlobFS-phy-autotest_38316/output//all_tests.txt ./BlobFS-phy-autotest_38316/output//build-kernel-manifest.txt ./BlobFS-phy-autotest_38316/output//build-pkg-manifest.txt ./BlobFS-phy-autotest_38316/output//build-repo-manifest.txt ./BlobFS-phy-autotest_38316/output//cov_base.info.xz ./BlobFS-phy-autotest_38316/output//cov_test.info.xz ./BlobFS-phy-autotest_38316/output//cov_total.info.xz ./BlobFS-phy-autotest_38316/output//dmesg.init.log ./BlobFS-phy-autotest_38316/output//dmesg.log ./BlobFS-phy-autotest_38316/output//power.tar.xz ./BlobFS-phy-autotest_38316/output//rocksdb ./BlobFS-phy-autotest_38316/output//test_completions.txt ./BlobFS-phy-autotest_38316/output//timing.svg ./BlobFS-phy-autotest_38316/output//timing.txt ./BlobFS-phy-autotest_38316/output//.. 00:32:32.893 + rmdir ./BlobFS-phy-autotest_38316/output/ 00:32:32.893 + for dir in "${out_dirs[@]}" 00:32:32.893 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.893 + [[ -f ./build-files-docker-autotest_33168/output//doc.tar.xz ]] 00:32:32.893 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.893 + [[ -f ./build-files-docker-autotest_33168/output//ut_coverage.tar.xz ]] 00:32:32.894 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.894 + [[ -f ./build-files-docker-autotest_33168/output//llvm.tar.xz ]] 00:32:32.894 + mv ./build-files-docker-autotest_33168/output//build-repo-manifest.txt ./build-files-docker-autotest_33168/output//power.tar.xz ./build-files-docker-autotest_33168/output//test_completions.txt ./build-files-docker-autotest_33168/output//timing.svg ./build-files-docker-autotest_33168/output//timing.txt ./build-files-docker-autotest_33168/output//.. 00:32:32.894 + rmdir ./build-files-docker-autotest_33168/output/ 00:32:32.894 + for dir in "${out_dirs[@]}" 00:32:32.894 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.894 + [[ -f ./check-format-docker-autotest_33194/output//doc.tar.xz ]] 00:32:32.894 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.894 + [[ -f ./check-format-docker-autotest_33194/output//ut_coverage.tar.xz ]] 00:32:32.894 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.894 + [[ -f ./check-format-docker-autotest_33194/output//llvm.tar.xz ]] 00:32:32.894 + mv ./check-format-docker-autotest_33194/output//build-repo-manifest.txt ./check-format-docker-autotest_33194/output//power.tar.xz ./check-format-docker-autotest_33194/output//test_completions.txt ./check-format-docker-autotest_33194/output//timing.svg ./check-format-docker-autotest_33194/output//timing.txt ./check-format-docker-autotest_33194/output//.. 00:32:32.894 + rmdir ./check-format-docker-autotest_33194/output/ 00:32:32.894 + for dir in "${out_dirs[@]}" 00:32:32.894 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.894 + [[ -f ./check-so-deps-docker-autotest_33415/output//doc.tar.xz ]] 00:32:32.894 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.894 + [[ -f ./check-so-deps-docker-autotest_33415/output//ut_coverage.tar.xz ]] 00:32:32.894 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.894 + [[ -f ./check-so-deps-docker-autotest_33415/output//llvm.tar.xz ]] 00:32:32.894 + mv ./check-so-deps-docker-autotest_33415/output//build-repo-manifest.txt ./check-so-deps-docker-autotest_33415/output//power.tar.xz ./check-so-deps-docker-autotest_33415/output//test_completions.txt ./check-so-deps-docker-autotest_33415/output//timing.svg ./check-so-deps-docker-autotest_33415/output//timing.txt ./check-so-deps-docker-autotest_33415/output//.. 00:32:32.894 + rmdir ./check-so-deps-docker-autotest_33415/output/ 00:32:32.894 + for dir in "${out_dirs[@]}" 00:32:32.894 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.894 + [[ -f ./crypto-phy-autotest_38305/output//doc.tar.xz ]] 00:32:32.894 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.894 + [[ -f ./crypto-phy-autotest_38305/output//ut_coverage.tar.xz ]] 00:32:32.894 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.894 + [[ -f ./crypto-phy-autotest_38305/output//llvm.tar.xz ]] 00:32:32.894 + mv ./crypto-phy-autotest_38305/output//all_tests.txt ./crypto-phy-autotest_38305/output//build-kernel-manifest.txt ./crypto-phy-autotest_38305/output//build-pkg-manifest.txt ./crypto-phy-autotest_38305/output//build-repo-manifest.txt ./crypto-phy-autotest_38305/output//cov_base.info.xz ./crypto-phy-autotest_38305/output//cov_test.info.xz ./crypto-phy-autotest_38305/output//cov_total.info.xz ./crypto-phy-autotest_38305/output//dmesg.init.log ./crypto-phy-autotest_38305/output//dmesg.log ./crypto-phy-autotest_38305/output//power.tar.xz ./crypto-phy-autotest_38305/output//test_completions.txt ./crypto-phy-autotest_38305/output//timing.svg ./crypto-phy-autotest_38305/output//timing.txt ./crypto-phy-autotest_38305/output//.. 00:32:32.894 + rmdir ./crypto-phy-autotest_38305/output/ 00:32:32.894 + for dir in "${out_dirs[@]}" 00:32:32.894 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:32.894 + [[ -f ./doc-docker-autotest_33497/output//doc.tar.xz ]] 00:32:32.894 + tar -C ./doc-docker-autotest_33497/output/ -xf ./doc-docker-autotest_33497/output//doc.tar.xz 00:32:33.150 + rm ./doc-docker-autotest_33497/output//doc.tar.xz 00:32:33.150 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.151 + [[ -f ./doc-docker-autotest_33497/output//ut_coverage.tar.xz ]] 00:32:33.151 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.151 + [[ -f ./doc-docker-autotest_33497/output//llvm.tar.xz ]] 00:32:33.151 + mv ./doc-docker-autotest_33497/output//build-repo-manifest.txt ./doc-docker-autotest_33497/output//doc ./doc-docker-autotest_33497/output//power.tar.xz ./doc-docker-autotest_33497/output//test_completions.txt ./doc-docker-autotest_33497/output//timing.svg ./doc-docker-autotest_33497/output//timing.txt ./doc-docker-autotest_33497/output//.. 00:32:33.151 + rmdir ./doc-docker-autotest_33497/output/ 00:32:33.151 + for dir in "${out_dirs[@]}" 00:32:33.151 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.151 + [[ -f ./freebsd-vg-autotest_68599/output//doc.tar.xz ]] 00:32:33.151 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.151 + [[ -f ./freebsd-vg-autotest_68599/output//ut_coverage.tar.xz ]] 00:32:33.151 + tar -C ./freebsd-vg-autotest_68599/output/ -xf ./freebsd-vg-autotest_68599/output//ut_coverage.tar.xz 00:32:33.409 + rm ./freebsd-vg-autotest_68599/output//ut_coverage.tar.xz 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./freebsd-vg-autotest_68599/output//llvm.tar.xz ]] 00:32:33.409 + mv ./freebsd-vg-autotest_68599/output//all_tests.txt ./freebsd-vg-autotest_68599/output//build-pkg-manifest.txt ./freebsd-vg-autotest_68599/output//build-repo-manifest.txt ./freebsd-vg-autotest_68599/output//cov_base.info.xz ./freebsd-vg-autotest_68599/output//cov_test.info.xz ./freebsd-vg-autotest_68599/output//cov_total.info.xz ./freebsd-vg-autotest_68599/output//messages ./freebsd-vg-autotest_68599/output//power.tar.xz ./freebsd-vg-autotest_68599/output//test_completions.txt ./freebsd-vg-autotest_68599/output//timing.svg ./freebsd-vg-autotest_68599/output//timing.txt ./freebsd-vg-autotest_68599/output//ut_coverage ./freebsd-vg-autotest_68599/output//.. 00:32:33.409 + rmdir ./freebsd-vg-autotest_68599/output/ 00:32:33.409 + for dir in "${out_dirs[@]}" 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./iscsi-uring-vg-autotest_90569/output//doc.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./iscsi-uring-vg-autotest_90569/output//ut_coverage.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./iscsi-uring-vg-autotest_90569/output//llvm.tar.xz ]] 00:32:33.409 + mv ./iscsi-uring-vg-autotest_90569/output//all_tests.txt ./iscsi-uring-vg-autotest_90569/output//build-kernel-manifest.txt ./iscsi-uring-vg-autotest_90569/output//build-pkg-manifest.txt ./iscsi-uring-vg-autotest_90569/output//build-repo-manifest.txt ./iscsi-uring-vg-autotest_90569/output//calsoft.json ./iscsi-uring-vg-autotest_90569/output//calsoft.tar.xz ./iscsi-uring-vg-autotest_90569/output//cov_base.info.xz ./iscsi-uring-vg-autotest_90569/output//cov_test.info.xz ./iscsi-uring-vg-autotest_90569/output//cov_total.info.xz ./iscsi-uring-vg-autotest_90569/output//dmesg.init.log ./iscsi-uring-vg-autotest_90569/output//dmesg.log ./iscsi-uring-vg-autotest_90569/output//iscsi_autofuzz_logs.txt ./iscsi-uring-vg-autotest_90569/output//iscsi_autofuzz_tgt_output.txt ./iscsi-uring-vg-autotest_90569/output//power.tar.xz ./iscsi-uring-vg-autotest_90569/output//test_completions.txt ./iscsi-uring-vg-autotest_90569/output//timing.svg ./iscsi-uring-vg-autotest_90569/output//timing.txt ./iscsi-uring-vg-autotest_90569/output//.. 00:32:33.409 + rmdir ./iscsi-uring-vg-autotest_90569/output/ 00:32:33.409 + for dir in "${out_dirs[@]}" 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./iscsi-vg-autotest_90705/output//doc.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./iscsi-vg-autotest_90705/output//ut_coverage.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./iscsi-vg-autotest_90705/output//llvm.tar.xz ]] 00:32:33.409 + mv ./iscsi-vg-autotest_90705/output//all_tests.txt ./iscsi-vg-autotest_90705/output//build-kernel-manifest.txt ./iscsi-vg-autotest_90705/output//build-pkg-manifest.txt ./iscsi-vg-autotest_90705/output//build-repo-manifest.txt ./iscsi-vg-autotest_90705/output//calsoft.json ./iscsi-vg-autotest_90705/output//calsoft.tar.xz ./iscsi-vg-autotest_90705/output//cov_base.info.xz ./iscsi-vg-autotest_90705/output//cov_test.info.xz ./iscsi-vg-autotest_90705/output//cov_total.info.xz ./iscsi-vg-autotest_90705/output//dmesg.init.log ./iscsi-vg-autotest_90705/output//dmesg.log ./iscsi-vg-autotest_90705/output//iscsi_autofuzz_logs.txt ./iscsi-vg-autotest_90705/output//iscsi_autofuzz_tgt_output.txt ./iscsi-vg-autotest_90705/output//power.tar.xz ./iscsi-vg-autotest_90705/output//test_completions.txt ./iscsi-vg-autotest_90705/output//timing.svg ./iscsi-vg-autotest_90705/output//timing.txt ./iscsi-vg-autotest_90705/output//.. 00:32:33.409 + rmdir ./iscsi-vg-autotest_90705/output/ 00:32:33.409 + for dir in "${out_dirs[@]}" 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./lvol-vg-autotest_90617/output//doc.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./lvol-vg-autotest_90617/output//ut_coverage.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./lvol-vg-autotest_90617/output//llvm.tar.xz ]] 00:32:33.409 + mv ./lvol-vg-autotest_90617/output//all_tests.txt ./lvol-vg-autotest_90617/output//build-kernel-manifest.txt ./lvol-vg-autotest_90617/output//build-pkg-manifest.txt ./lvol-vg-autotest_90617/output//build-repo-manifest.txt ./lvol-vg-autotest_90617/output//cov_base.info.xz ./lvol-vg-autotest_90617/output//cov_test.info.xz ./lvol-vg-autotest_90617/output//cov_total.info.xz ./lvol-vg-autotest_90617/output//dmesg.init.log ./lvol-vg-autotest_90617/output//dmesg.log ./lvol-vg-autotest_90617/output//power.tar.xz ./lvol-vg-autotest_90617/output//test_completions.txt ./lvol-vg-autotest_90617/output//timing.svg ./lvol-vg-autotest_90617/output//timing.txt ./lvol-vg-autotest_90617/output//.. 00:32:33.409 + rmdir ./lvol-vg-autotest_90617/output/ 00:32:33.409 + for dir in "${out_dirs[@]}" 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./nvme-cmb-pmr-vg-autotest_91797/output//doc.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./nvme-cmb-pmr-vg-autotest_91797/output//ut_coverage.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./nvme-cmb-pmr-vg-autotest_91797/output//llvm.tar.xz ]] 00:32:33.409 + mv ./nvme-cmb-pmr-vg-autotest_91797/output//all_tests.txt ./nvme-cmb-pmr-vg-autotest_91797/output//build-kernel-manifest.txt ./nvme-cmb-pmr-vg-autotest_91797/output//build-pkg-manifest.txt ./nvme-cmb-pmr-vg-autotest_91797/output//build-repo-manifest.txt ./nvme-cmb-pmr-vg-autotest_91797/output//cov_base.info.xz ./nvme-cmb-pmr-vg-autotest_91797/output//cov_test.info.xz ./nvme-cmb-pmr-vg-autotest_91797/output//cov_total.info.xz ./nvme-cmb-pmr-vg-autotest_91797/output//dmesg.init.log ./nvme-cmb-pmr-vg-autotest_91797/output//dmesg.log ./nvme-cmb-pmr-vg-autotest_91797/output//power.tar.xz ./nvme-cmb-pmr-vg-autotest_91797/output//test_completions.txt ./nvme-cmb-pmr-vg-autotest_91797/output//timing.svg ./nvme-cmb-pmr-vg-autotest_91797/output//timing.txt ./nvme-cmb-pmr-vg-autotest_91797/output//.. 00:32:33.409 + rmdir ./nvme-cmb-pmr-vg-autotest_91797/output/ 00:32:33.409 + for dir in "${out_dirs[@]}" 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./nvme-phy-autotest_84479/output//doc.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./nvme-phy-autotest_84479/output//ut_coverage.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./nvme-phy-autotest_84479/output//llvm.tar.xz ]] 00:32:33.409 + mv ./nvme-phy-autotest_84479/output//all_tests.txt ./nvme-phy-autotest_84479/output//build-kernel-manifest.txt ./nvme-phy-autotest_84479/output//build-pkg-manifest.txt ./nvme-phy-autotest_84479/output//build-repo-manifest.txt ./nvme-phy-autotest_84479/output//cov_base.info.xz ./nvme-phy-autotest_84479/output//cov_test.info.xz ./nvme-phy-autotest_84479/output//cov_total.info.xz ./nvme-phy-autotest_84479/output//dmesg.init.log ./nvme-phy-autotest_84479/output//dmesg.log ./nvme-phy-autotest_84479/output//power.tar.xz ./nvme-phy-autotest_84479/output//test_completions.txt ./nvme-phy-autotest_84479/output//timing.svg ./nvme-phy-autotest_84479/output//timing.txt ./nvme-phy-autotest_84479/output//.. 00:32:33.409 + rmdir ./nvme-phy-autotest_84479/output/ 00:32:33.409 + for dir in "${out_dirs[@]}" 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./nvme-vg-autotest_90668/output//doc.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./nvme-vg-autotest_90668/output//ut_coverage.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./nvme-vg-autotest_90668/output//llvm.tar.xz ]] 00:32:33.409 + mv ./nvme-vg-autotest_90668/output//all_tests.txt ./nvme-vg-autotest_90668/output//build-kernel-manifest.txt ./nvme-vg-autotest_90668/output//build-pkg-manifest.txt ./nvme-vg-autotest_90668/output//build-repo-manifest.txt ./nvme-vg-autotest_90668/output//cov_base.info.xz ./nvme-vg-autotest_90668/output//cov_test.info.xz ./nvme-vg-autotest_90668/output//cov_total.info.xz ./nvme-vg-autotest_90668/output//dmesg.init.log ./nvme-vg-autotest_90668/output//dmesg.log ./nvme-vg-autotest_90668/output//power.tar.xz ./nvme-vg-autotest_90668/output//test_completions.txt ./nvme-vg-autotest_90668/output//timing.svg ./nvme-vg-autotest_90668/output//timing.txt ./nvme-vg-autotest_90668/output//.. 00:32:33.409 + rmdir ./nvme-vg-autotest_90668/output/ 00:32:33.409 + for dir in "${out_dirs[@]}" 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./nvmf-phy-autotest_84548/output//doc.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./nvmf-phy-autotest_84548/output//ut_coverage.tar.xz ]] 00:32:33.409 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.409 + [[ -f ./nvmf-phy-autotest_84548/output//llvm.tar.xz ]] 00:32:33.409 + mv ./nvmf-phy-autotest_84548/output//all_tests.txt ./nvmf-phy-autotest_84548/output//build-kernel-manifest.txt ./nvmf-phy-autotest_84548/output//build-pkg-manifest.txt ./nvmf-phy-autotest_84548/output//build-repo-manifest.txt ./nvmf-phy-autotest_84548/output//cov_base.info.xz ./nvmf-phy-autotest_84548/output//dmesg.init.log ./nvmf-phy-autotest_84548/output//dmesg.log ./nvmf-phy-autotest_84548/output//nvmf_trace.0_shm.tar.gz ./nvmf-phy-autotest_84548/output//power.tar.xz ./nvmf-phy-autotest_84548/output//proc_list.txt ./nvmf-phy-autotest_84548/output//test_completions.txt ./nvmf-phy-autotest_84548/output//timing.txt ./nvmf-phy-autotest_84548/output//udev.log ./nvmf-phy-autotest_84548/output//.. 00:32:33.410 + rmdir ./nvmf-phy-autotest_84548/output/ 00:32:33.410 + for dir in "${out_dirs[@]}" 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./nvmf-tcp-phy-autotest_84430/output//doc.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./nvmf-tcp-phy-autotest_84430/output//ut_coverage.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./nvmf-tcp-phy-autotest_84430/output//llvm.tar.xz ]] 00:32:33.410 + mv ./nvmf-tcp-phy-autotest_84430/output//all_tests.txt ./nvmf-tcp-phy-autotest_84430/output//build-kernel-manifest.txt ./nvmf-tcp-phy-autotest_84430/output//build-pkg-manifest.txt ./nvmf-tcp-phy-autotest_84430/output//build-repo-manifest.txt ./nvmf-tcp-phy-autotest_84430/output//cov_base.info.xz ./nvmf-tcp-phy-autotest_84430/output//dmesg.init.log ./nvmf-tcp-phy-autotest_84430/output//dmesg.log ./nvmf-tcp-phy-autotest_84430/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-phy-autotest_84430/output//power.tar.xz ./nvmf-tcp-phy-autotest_84430/output//proc_list.txt ./nvmf-tcp-phy-autotest_84430/output//test_completions.txt ./nvmf-tcp-phy-autotest_84430/output//timing.txt ./nvmf-tcp-phy-autotest_84430/output//udev.log ./nvmf-tcp-phy-autotest_84430/output//.. 00:32:33.410 + rmdir ./nvmf-tcp-phy-autotest_84430/output/ 00:32:33.410 + for dir in "${out_dirs[@]}" 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./nvmf-tcp-uring-vg-autotest_90611/output//doc.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./nvmf-tcp-uring-vg-autotest_90611/output//ut_coverage.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./nvmf-tcp-uring-vg-autotest_90611/output//llvm.tar.xz ]] 00:32:33.410 + mv ./nvmf-tcp-uring-vg-autotest_90611/output//all_tests.txt ./nvmf-tcp-uring-vg-autotest_90611/output//build-kernel-manifest.txt ./nvmf-tcp-uring-vg-autotest_90611/output//build-pkg-manifest.txt ./nvmf-tcp-uring-vg-autotest_90611/output//build-repo-manifest.txt ./nvmf-tcp-uring-vg-autotest_90611/output//cov_base.info.xz ./nvmf-tcp-uring-vg-autotest_90611/output//dmesg.init.log ./nvmf-tcp-uring-vg-autotest_90611/output//dmesg.log ./nvmf-tcp-uring-vg-autotest_90611/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-uring-vg-autotest_90611/output//power.tar.xz ./nvmf-tcp-uring-vg-autotest_90611/output//proc_list.txt ./nvmf-tcp-uring-vg-autotest_90611/output//test_completions.txt ./nvmf-tcp-uring-vg-autotest_90611/output//timing.txt ./nvmf-tcp-uring-vg-autotest_90611/output//udev.log ./nvmf-tcp-uring-vg-autotest_90611/output//.. 00:32:33.410 + rmdir ./nvmf-tcp-uring-vg-autotest_90611/output/ 00:32:33.410 + for dir in "${out_dirs[@]}" 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./nvmf-tcp-vg-autotest_90650/output//doc.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./nvmf-tcp-vg-autotest_90650/output//ut_coverage.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./nvmf-tcp-vg-autotest_90650/output//llvm.tar.xz ]] 00:32:33.410 + mv ./nvmf-tcp-vg-autotest_90650/output//all_tests.txt ./nvmf-tcp-vg-autotest_90650/output//build-kernel-manifest.txt ./nvmf-tcp-vg-autotest_90650/output//build-pkg-manifest.txt ./nvmf-tcp-vg-autotest_90650/output//build-repo-manifest.txt ./nvmf-tcp-vg-autotest_90650/output//cov_base.info.xz ./nvmf-tcp-vg-autotest_90650/output//dmesg.init.log ./nvmf-tcp-vg-autotest_90650/output//dmesg.log ./nvmf-tcp-vg-autotest_90650/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-vg-autotest_90650/output//power.tar.xz ./nvmf-tcp-vg-autotest_90650/output//proc_list.txt ./nvmf-tcp-vg-autotest_90650/output//test_completions.txt ./nvmf-tcp-vg-autotest_90650/output//timing.txt ./nvmf-tcp-vg-autotest_90650/output//udev.log ./nvmf-tcp-vg-autotest_90650/output//.. 00:32:33.410 + rmdir ./nvmf-tcp-vg-autotest_90650/output/ 00:32:33.410 + for dir in "${out_dirs[@]}" 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./packaging-docker-autotest_29941/output//doc.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./packaging-docker-autotest_29941/output//ut_coverage.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./packaging-docker-autotest_29941/output//llvm.tar.xz ]] 00:32:33.410 + mv ./packaging-docker-autotest_29941/output//build-repo-manifest.txt ./packaging-docker-autotest_29941/output//power.tar.xz ./packaging-docker-autotest_29941/output//test_completions.txt ./packaging-docker-autotest_29941/output//timing.svg ./packaging-docker-autotest_29941/output//timing.txt ./packaging-docker-autotest_29941/output//.. 00:32:33.410 + rmdir ./packaging-docker-autotest_29941/output/ 00:32:33.410 + for dir in "${out_dirs[@]}" 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./raid-vg-autotest_5263/output//doc.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./raid-vg-autotest_5263/output//ut_coverage.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./raid-vg-autotest_5263/output//llvm.tar.xz ]] 00:32:33.410 + mv ./raid-vg-autotest_5263/output//all_tests.txt ./raid-vg-autotest_5263/output//build-kernel-manifest.txt ./raid-vg-autotest_5263/output//build-pkg-manifest.txt ./raid-vg-autotest_5263/output//build-repo-manifest.txt ./raid-vg-autotest_5263/output//cov_base.info.xz ./raid-vg-autotest_5263/output//cov_test.info.xz ./raid-vg-autotest_5263/output//cov_total.info.xz ./raid-vg-autotest_5263/output//dmesg.init.log ./raid-vg-autotest_5263/output//dmesg.log ./raid-vg-autotest_5263/output//power.tar.xz ./raid-vg-autotest_5263/output//test_completions.txt ./raid-vg-autotest_5263/output//timing.svg ./raid-vg-autotest_5263/output//timing.txt ./raid-vg-autotest_5263/output//.. 00:32:33.410 + rmdir ./raid-vg-autotest_5263/output/ 00:32:33.410 + for dir in "${out_dirs[@]}" 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./release-build-clang-docker-autotest_3119/output//doc.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./release-build-clang-docker-autotest_3119/output//ut_coverage.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./release-build-clang-docker-autotest_3119/output//llvm.tar.xz ]] 00:32:33.410 + mv ./release-build-clang-docker-autotest_3119/output//build-repo-manifest.txt ./release-build-clang-docker-autotest_3119/output//power.tar.xz ./release-build-clang-docker-autotest_3119/output//test_completions.txt ./release-build-clang-docker-autotest_3119/output//timing.svg ./release-build-clang-docker-autotest_3119/output//timing.txt ./release-build-clang-docker-autotest_3119/output//.. 00:32:33.410 + rmdir ./release-build-clang-docker-autotest_3119/output/ 00:32:33.410 + for dir in "${out_dirs[@]}" 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./release-build-main-docker-autotest_3119/output//doc.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./release-build-main-docker-autotest_3119/output//ut_coverage.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./release-build-main-docker-autotest_3119/output//llvm.tar.xz ]] 00:32:33.410 + mv ./release-build-main-docker-autotest_3119/output//build-repo-manifest.txt ./release-build-main-docker-autotest_3119/output//power.tar.xz ./release-build-main-docker-autotest_3119/output//test_completions.txt ./release-build-main-docker-autotest_3119/output//timing.svg ./release-build-main-docker-autotest_3119/output//timing.txt ./release-build-main-docker-autotest_3119/output//.. 00:32:33.410 + rmdir ./release-build-main-docker-autotest_3119/output/ 00:32:33.410 + for dir in "${out_dirs[@]}" 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./rocky9-vg-autotest_27871/output//doc.tar.xz ]] 00:32:33.410 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.410 + [[ -f ./rocky9-vg-autotest_27871/output//ut_coverage.tar.xz ]] 00:32:33.410 + tar -C ./rocky9-vg-autotest_27871/output/ -xf ./rocky9-vg-autotest_27871/output//ut_coverage.tar.xz 00:32:33.668 + rm ./rocky9-vg-autotest_27871/output//ut_coverage.tar.xz 00:32:33.668 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.668 + [[ -f ./rocky9-vg-autotest_27871/output//llvm.tar.xz ]] 00:32:33.668 + mv ./rocky9-vg-autotest_27871/output//all_tests.txt ./rocky9-vg-autotest_27871/output//build-pkg-manifest.txt ./rocky9-vg-autotest_27871/output//build-repo-manifest.txt ./rocky9-vg-autotest_27871/output//cov_base.info.xz ./rocky9-vg-autotest_27871/output//cov_test.info.xz ./rocky9-vg-autotest_27871/output//cov_total.info.xz ./rocky9-vg-autotest_27871/output//dmesg.init.log ./rocky9-vg-autotest_27871/output//dmesg.log ./rocky9-vg-autotest_27871/output//power.tar.xz ./rocky9-vg-autotest_27871/output//test_completions.txt ./rocky9-vg-autotest_27871/output//timing.svg ./rocky9-vg-autotest_27871/output//timing.txt ./rocky9-vg-autotest_27871/output//ut_coverage ./rocky9-vg-autotest_27871/output//.. 00:32:33.668 + rmdir ./rocky9-vg-autotest_27871/output/ 00:32:33.668 + for dir in "${out_dirs[@]}" 00:32:33.668 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.668 + [[ -f ./scan-build-docker-autotest_29948/output//doc.tar.xz ]] 00:32:33.668 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.668 + [[ -f ./scan-build-docker-autotest_29948/output//ut_coverage.tar.xz ]] 00:32:33.668 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.668 + [[ -f ./scan-build-docker-autotest_29948/output//llvm.tar.xz ]] 00:32:33.668 + mv ./scan-build-docker-autotest_29948/output//all_c_files.txt ./scan-build-docker-autotest_29948/output//build_output.txt ./scan-build-docker-autotest_29948/output//build-repo-manifest.txt ./scan-build-docker-autotest_29948/output//built_c_files.txt ./scan-build-docker-autotest_29948/output//power.tar.xz ./scan-build-docker-autotest_29948/output//test_completions.txt ./scan-build-docker-autotest_29948/output//timing.svg ./scan-build-docker-autotest_29948/output//timing.txt ./scan-build-docker-autotest_29948/output//unbuilt_c_files.txt ./scan-build-docker-autotest_29948/output//.. 00:32:33.668 + rmdir ./scan-build-docker-autotest_29948/output/ 00:32:33.668 + for dir in "${out_dirs[@]}" 00:32:33.668 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.668 + [[ -f ./short-fuzz-phy-autotest_38301/output//doc.tar.xz ]] 00:32:33.668 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.668 + [[ -f ./short-fuzz-phy-autotest_38301/output//ut_coverage.tar.xz ]] 00:32:33.668 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.668 + [[ -f ./short-fuzz-phy-autotest_38301/output//llvm.tar.xz ]] 00:32:33.668 + tar -C ./short-fuzz-phy-autotest_38301/output/ -xf ./short-fuzz-phy-autotest_38301/output//llvm.tar.xz 00:32:33.668 + rm ./short-fuzz-phy-autotest_38301/output//llvm.tar.xz 00:32:33.668 + mv ./short-fuzz-phy-autotest_38301/output//all_tests.txt ./short-fuzz-phy-autotest_38301/output//build-kernel-manifest.txt ./short-fuzz-phy-autotest_38301/output//build-pkg-manifest.txt ./short-fuzz-phy-autotest_38301/output//build-repo-manifest.txt ./short-fuzz-phy-autotest_38301/output//cov_base.info.xz ./short-fuzz-phy-autotest_38301/output//cov_test.info.xz ./short-fuzz-phy-autotest_38301/output//cov_total.info.xz ./short-fuzz-phy-autotest_38301/output//dmesg.init.log ./short-fuzz-phy-autotest_38301/output//dmesg.log ./short-fuzz-phy-autotest_38301/output//llvm ./short-fuzz-phy-autotest_38301/output//power.tar.xz ./short-fuzz-phy-autotest_38301/output//test_completions.txt ./short-fuzz-phy-autotest_38301/output//timing.svg ./short-fuzz-phy-autotest_38301/output//timing.txt ./short-fuzz-phy-autotest_38301/output//.. 00:32:33.668 + rmdir ./short-fuzz-phy-autotest_38301/output/ 00:32:33.668 + for dir in "${out_dirs[@]}" 00:32:33.668 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.668 + [[ -f ./ubuntu22-vg-autotest_49538/output//doc.tar.xz ]] 00:32:33.668 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.668 + [[ -f ./ubuntu22-vg-autotest_49538/output//ut_coverage.tar.xz ]] 00:32:33.668 + tar -C ./ubuntu22-vg-autotest_49538/output/ -xf ./ubuntu22-vg-autotest_49538/output//ut_coverage.tar.xz 00:32:33.925 + rm ./ubuntu22-vg-autotest_49538/output//ut_coverage.tar.xz 00:32:33.925 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.925 + [[ -f ./ubuntu22-vg-autotest_49538/output//llvm.tar.xz ]] 00:32:33.925 + mv ./ubuntu22-vg-autotest_49538/output//all_tests.txt ./ubuntu22-vg-autotest_49538/output//build-pkg-manifest.txt ./ubuntu22-vg-autotest_49538/output//build-repo-manifest.txt ./ubuntu22-vg-autotest_49538/output//cov_base.info.xz ./ubuntu22-vg-autotest_49538/output//cov_test.info.xz ./ubuntu22-vg-autotest_49538/output//cov_total.info.xz ./ubuntu22-vg-autotest_49538/output//dmesg.init.log ./ubuntu22-vg-autotest_49538/output//dmesg.log ./ubuntu22-vg-autotest_49538/output//power.tar.xz ./ubuntu22-vg-autotest_49538/output//test_completions.txt ./ubuntu22-vg-autotest_49538/output//timing.svg ./ubuntu22-vg-autotest_49538/output//timing.txt ./ubuntu22-vg-autotest_49538/output//ut_coverage ./ubuntu22-vg-autotest_49538/output//.. 00:32:33.925 + rmdir ./ubuntu22-vg-autotest_49538/output/ 00:32:33.925 + for dir in "${out_dirs[@]}" 00:32:33.925 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.925 + [[ -f ./ubuntu24-vg-autotest_7099/output//doc.tar.xz ]] 00:32:33.925 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:33.925 + [[ -f ./ubuntu24-vg-autotest_7099/output//ut_coverage.tar.xz ]] 00:32:33.925 + tar -C ./ubuntu24-vg-autotest_7099/output/ -xf ./ubuntu24-vg-autotest_7099/output//ut_coverage.tar.xz 00:32:34.182 + rm ./ubuntu24-vg-autotest_7099/output//ut_coverage.tar.xz 00:32:34.182 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:34.182 + [[ -f ./ubuntu24-vg-autotest_7099/output//llvm.tar.xz ]] 00:32:34.182 + mv ./ubuntu24-vg-autotest_7099/output//all_tests.txt ./ubuntu24-vg-autotest_7099/output//build-pkg-manifest.txt ./ubuntu24-vg-autotest_7099/output//build-repo-manifest.txt ./ubuntu24-vg-autotest_7099/output//cov_base.info.xz ./ubuntu24-vg-autotest_7099/output//cov_test.info.xz ./ubuntu24-vg-autotest_7099/output//cov_total.info.xz ./ubuntu24-vg-autotest_7099/output//dmesg.init.log ./ubuntu24-vg-autotest_7099/output//dmesg.log ./ubuntu24-vg-autotest_7099/output//power.tar.xz ./ubuntu24-vg-autotest_7099/output//test_completions.txt ./ubuntu24-vg-autotest_7099/output//timing.svg ./ubuntu24-vg-autotest_7099/output//timing.txt ./ubuntu24-vg-autotest_7099/output//ut_coverage ./ubuntu24-vg-autotest_7099/output//.. 00:32:34.182 + rmdir ./ubuntu24-vg-autotest_7099/output/ 00:32:34.182 + for dir in "${out_dirs[@]}" 00:32:34.182 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:34.182 + [[ -f ./unittest-clang-docker-autotest_29942/output//doc.tar.xz ]] 00:32:34.182 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:34.182 + [[ -f ./unittest-clang-docker-autotest_29942/output//ut_coverage.tar.xz ]] 00:32:34.182 + tar -C ./unittest-clang-docker-autotest_29942/output/ -xf ./unittest-clang-docker-autotest_29942/output//ut_coverage.tar.xz 00:32:34.747 + rm ./unittest-clang-docker-autotest_29942/output//ut_coverage.tar.xz 00:32:34.747 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:34.747 + [[ -f ./unittest-clang-docker-autotest_29942/output//llvm.tar.xz ]] 00:32:34.747 + mv ./unittest-clang-docker-autotest_29942/output//build-repo-manifest.txt ./unittest-clang-docker-autotest_29942/output//power.tar.xz ./unittest-clang-docker-autotest_29942/output//test_completions.txt ./unittest-clang-docker-autotest_29942/output//timing.svg ./unittest-clang-docker-autotest_29942/output//timing.txt ./unittest-clang-docker-autotest_29942/output//ut_coverage ./unittest-clang-docker-autotest_29942/output//.. 00:32:34.747 + rmdir ./unittest-clang-docker-autotest_29942/output/ 00:32:34.747 + for dir in "${out_dirs[@]}" 00:32:34.747 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:34.747 + [[ -f ./unittest-main-docker-autotest_29944/output//doc.tar.xz ]] 00:32:34.747 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:34.747 + [[ -f ./unittest-main-docker-autotest_29944/output//ut_coverage.tar.xz ]] 00:32:34.747 + tar -C ./unittest-main-docker-autotest_29944/output/ -xf ./unittest-main-docker-autotest_29944/output//ut_coverage.tar.xz 00:32:35.004 + rm ./unittest-main-docker-autotest_29944/output//ut_coverage.tar.xz 00:32:35.004 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.004 + [[ -f ./unittest-main-docker-autotest_29944/output//llvm.tar.xz ]] 00:32:35.004 + mv ./unittest-main-docker-autotest_29944/output//build-repo-manifest.txt ./unittest-main-docker-autotest_29944/output//power.tar.xz ./unittest-main-docker-autotest_29944/output//test_completions.txt ./unittest-main-docker-autotest_29944/output//timing.svg ./unittest-main-docker-autotest_29944/output//timing.txt ./unittest-main-docker-autotest_29944/output//ut_coverage ./unittest-main-docker-autotest_29944/output//.. 00:32:35.004 + rmdir ./unittest-main-docker-autotest_29944/output/ 00:32:35.004 + for dir in "${out_dirs[@]}" 00:32:35.004 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.004 + [[ -f ./unittest-valgrind-docker-autotest_29946/output//doc.tar.xz ]] 00:32:35.004 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.004 + [[ -f ./unittest-valgrind-docker-autotest_29946/output//ut_coverage.tar.xz ]] 00:32:35.004 + tar -C ./unittest-valgrind-docker-autotest_29946/output/ -xf ./unittest-valgrind-docker-autotest_29946/output//ut_coverage.tar.xz 00:32:35.261 + rm ./unittest-valgrind-docker-autotest_29946/output//ut_coverage.tar.xz 00:32:35.261 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.262 + [[ -f ./unittest-valgrind-docker-autotest_29946/output//llvm.tar.xz ]] 00:32:35.262 + mv ./unittest-valgrind-docker-autotest_29946/output//build-repo-manifest.txt ./unittest-valgrind-docker-autotest_29946/output//power.tar.xz ./unittest-valgrind-docker-autotest_29946/output//test_completions.txt ./unittest-valgrind-docker-autotest_29946/output//timing.svg ./unittest-valgrind-docker-autotest_29946/output//timing.txt ./unittest-valgrind-docker-autotest_29946/output//ut_coverage ./unittest-valgrind-docker-autotest_29946/output//.. 00:32:35.262 + rmdir ./unittest-valgrind-docker-autotest_29946/output/ 00:32:35.262 + for dir in "${out_dirs[@]}" 00:32:35.262 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.262 + [[ -f ./vfio-user-phy-autotest_38306/output//doc.tar.xz ]] 00:32:35.262 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.262 + [[ -f ./vfio-user-phy-autotest_38306/output//ut_coverage.tar.xz ]] 00:32:35.262 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.262 + [[ -f ./vfio-user-phy-autotest_38306/output//llvm.tar.xz ]] 00:32:35.262 + mv ./vfio-user-phy-autotest_38306/output//all_tests.txt ./vfio-user-phy-autotest_38306/output//build-kernel-manifest.txt ./vfio-user-phy-autotest_38306/output//build-pkg-manifest.txt ./vfio-user-phy-autotest_38306/output//build-repo-manifest.txt ./vfio-user-phy-autotest_38306/output//cov_base.info.xz ./vfio-user-phy-autotest_38306/output//cov_test.info.xz ./vfio-user-phy-autotest_38306/output//cov_total.info.xz ./vfio-user-phy-autotest_38306/output//dmesg.init.log ./vfio-user-phy-autotest_38306/output//dmesg.log ./vfio-user-phy-autotest_38306/output//power.tar.xz ./vfio-user-phy-autotest_38306/output//test_completions.txt ./vfio-user-phy-autotest_38306/output//timing.svg ./vfio-user-phy-autotest_38306/output//timing.txt ./vfio-user-phy-autotest_38306/output//.. 00:32:35.262 + rmdir ./vfio-user-phy-autotest_38306/output/ 00:32:35.262 + for dir in "${out_dirs[@]}" 00:32:35.262 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.262 + [[ -f ./vhost-initiator-vg-autotest_90686/output//doc.tar.xz ]] 00:32:35.262 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.262 + [[ -f ./vhost-initiator-vg-autotest_90686/output//ut_coverage.tar.xz ]] 00:32:35.262 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.262 + [[ -f ./vhost-initiator-vg-autotest_90686/output//llvm.tar.xz ]] 00:32:35.262 + mv ./vhost-initiator-vg-autotest_90686/output//all_tests.txt ./vhost-initiator-vg-autotest_90686/output//build-kernel-manifest.txt ./vhost-initiator-vg-autotest_90686/output//build-pkg-manifest.txt ./vhost-initiator-vg-autotest_90686/output//build-repo-manifest.txt ./vhost-initiator-vg-autotest_90686/output//cov_base.info.xz ./vhost-initiator-vg-autotest_90686/output//cov_test.info.xz ./vhost-initiator-vg-autotest_90686/output//cov_total.info.xz ./vhost-initiator-vg-autotest_90686/output//dmesg.init.log ./vhost-initiator-vg-autotest_90686/output//dmesg.log ./vhost-initiator-vg-autotest_90686/output//power.tar.xz ./vhost-initiator-vg-autotest_90686/output//test_completions.txt ./vhost-initiator-vg-autotest_90686/output//timing.svg ./vhost-initiator-vg-autotest_90686/output//timing.txt ./vhost-initiator-vg-autotest_90686/output//vhost_fuzz_output1.txt.xz ./vhost-initiator-vg-autotest_90686/output//vhost_fuzz_output2.txt.xz ./vhost-initiator-vg-autotest_90686/output//vhost_fuzz_tgt_output.txt.xz ./vhost-initiator-vg-autotest_90686/output//.. 00:32:35.262 + rmdir ./vhost-initiator-vg-autotest_90686/output/ 00:32:35.262 + for dir in "${out_dirs[@]}" 00:32:35.262 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.262 + [[ -f ./vhost-phy-autotest_38373/output//doc.tar.xz ]] 00:32:35.262 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.262 + [[ -f ./vhost-phy-autotest_38373/output//ut_coverage.tar.xz ]] 00:32:35.262 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.262 + [[ -f ./vhost-phy-autotest_38373/output//llvm.tar.xz ]] 00:32:35.262 + mv ./vhost-phy-autotest_38373/output//all_tests.txt ./vhost-phy-autotest_38373/output//build-kernel-manifest.txt ./vhost-phy-autotest_38373/output//build-pkg-manifest.txt ./vhost-phy-autotest_38373/output//build-repo-manifest.txt ./vhost-phy-autotest_38373/output//cov_base.info.xz ./vhost-phy-autotest_38373/output//cov_test.info.xz ./vhost-phy-autotest_38373/output//cov_total.info.xz ./vhost-phy-autotest_38373/output//dmesg.init.log ./vhost-phy-autotest_38373/output//dmesg.log ./vhost-phy-autotest_38373/output//power.tar.xz ./vhost-phy-autotest_38373/output//test_completions.txt ./vhost-phy-autotest_38373/output//timing.svg ./vhost-phy-autotest_38373/output//timing.txt ./vhost-phy-autotest_38373/output//.. 00:32:35.262 + rmdir ./vhost-phy-autotest_38373/output/ 00:32:35.262 + for dir in "${out_dirs[@]}" 00:32:35.262 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.262 + [[ -f ./zns-vg-autotest_89770/output//doc.tar.xz ]] 00:32:35.262 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.262 + [[ -f ./zns-vg-autotest_89770/output//ut_coverage.tar.xz ]] 00:32:35.262 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:32:35.262 + [[ -f ./zns-vg-autotest_89770/output//llvm.tar.xz ]] 00:32:35.262 + mv ./zns-vg-autotest_89770/output//all_tests.txt ./zns-vg-autotest_89770/output//build-kernel-manifest.txt ./zns-vg-autotest_89770/output//build-pkg-manifest.txt ./zns-vg-autotest_89770/output//build-repo-manifest.txt ./zns-vg-autotest_89770/output//cov_base.info.xz ./zns-vg-autotest_89770/output//cov_test.info.xz ./zns-vg-autotest_89770/output//cov_total.info.xz ./zns-vg-autotest_89770/output//dmesg.init.log ./zns-vg-autotest_89770/output//dmesg.log ./zns-vg-autotest_89770/output//power.tar.xz ./zns-vg-autotest_89770/output//test_completions.txt ./zns-vg-autotest_89770/output//timing.svg ./zns-vg-autotest_89770/output//timing.txt ./zns-vg-autotest_89770/output//.. 00:32:35.262 + rmdir ./zns-vg-autotest_89770/output/ 00:32:35.262 + unpack_cov_files 00:32:35.262 + local info_files 00:32:35.262 + info_files=(*/cov_*.info.xz) 00:32:35.262 + xargs -P0 -r -n1 xz -d 00:32:35.262 + printf '%s\n' BlobFS-phy-autotest_38316/cov_base.info.xz BlobFS-phy-autotest_38316/cov_test.info.xz BlobFS-phy-autotest_38316/cov_total.info.xz crypto-phy-autotest_38305/cov_base.info.xz crypto-phy-autotest_38305/cov_test.info.xz crypto-phy-autotest_38305/cov_total.info.xz freebsd-vg-autotest_68599/cov_base.info.xz freebsd-vg-autotest_68599/cov_test.info.xz freebsd-vg-autotest_68599/cov_total.info.xz iscsi-uring-vg-autotest_90569/cov_base.info.xz iscsi-uring-vg-autotest_90569/cov_test.info.xz iscsi-uring-vg-autotest_90569/cov_total.info.xz iscsi-vg-autotest_90705/cov_base.info.xz iscsi-vg-autotest_90705/cov_test.info.xz iscsi-vg-autotest_90705/cov_total.info.xz lvol-vg-autotest_90617/cov_base.info.xz lvol-vg-autotest_90617/cov_test.info.xz lvol-vg-autotest_90617/cov_total.info.xz nvme-cmb-pmr-vg-autotest_91797/cov_base.info.xz nvme-cmb-pmr-vg-autotest_91797/cov_test.info.xz nvme-cmb-pmr-vg-autotest_91797/cov_total.info.xz nvme-phy-autotest_84479/cov_base.info.xz nvme-phy-autotest_84479/cov_test.info.xz nvme-phy-autotest_84479/cov_total.info.xz nvme-vg-autotest_90668/cov_base.info.xz nvme-vg-autotest_90668/cov_test.info.xz nvme-vg-autotest_90668/cov_total.info.xz nvmf-phy-autotest_84548/cov_base.info.xz nvmf-tcp-phy-autotest_84430/cov_base.info.xz nvmf-tcp-uring-vg-autotest_90611/cov_base.info.xz nvmf-tcp-vg-autotest_90650/cov_base.info.xz raid-vg-autotest_5263/cov_base.info.xz raid-vg-autotest_5263/cov_test.info.xz raid-vg-autotest_5263/cov_total.info.xz rocky9-vg-autotest_27871/cov_base.info.xz rocky9-vg-autotest_27871/cov_test.info.xz rocky9-vg-autotest_27871/cov_total.info.xz short-fuzz-phy-autotest_38301/cov_base.info.xz short-fuzz-phy-autotest_38301/cov_test.info.xz short-fuzz-phy-autotest_38301/cov_total.info.xz ubuntu22-vg-autotest_49538/cov_base.info.xz ubuntu22-vg-autotest_49538/cov_test.info.xz ubuntu22-vg-autotest_49538/cov_total.info.xz ubuntu24-vg-autotest_7099/cov_base.info.xz ubuntu24-vg-autotest_7099/cov_test.info.xz ubuntu24-vg-autotest_7099/cov_total.info.xz vfio-user-phy-autotest_38306/cov_base.info.xz vfio-user-phy-autotest_38306/cov_test.info.xz vfio-user-phy-autotest_38306/cov_total.info.xz vhost-initiator-vg-autotest_90686/cov_base.info.xz vhost-initiator-vg-autotest_90686/cov_test.info.xz vhost-initiator-vg-autotest_90686/cov_total.info.xz vhost-phy-autotest_38373/cov_base.info.xz vhost-phy-autotest_38373/cov_test.info.xz vhost-phy-autotest_38373/cov_total.info.xz zns-vg-autotest_89770/cov_base.info.xz zns-vg-autotest_89770/cov_test.info.xz zns-vg-autotest_89770/cov_total.info.xz 00:32:35.826 + fix_downstream_job_paths 00:32:35.826 + sed -i -e 's#^SF:/.\+/spdk/#SF:/var/jenkins/workspace/autotest-per-patch_3/spdk/#g' BlobFS-phy-autotest_38316/cov_total.info crypto-phy-autotest_38305/cov_total.info freebsd-vg-autotest_68599/cov_total.info iscsi-uring-vg-autotest_90569/cov_total.info iscsi-vg-autotest_90705/cov_total.info lvol-vg-autotest_90617/cov_total.info nvme-cmb-pmr-vg-autotest_91797/cov_total.info nvme-phy-autotest_84479/cov_total.info nvme-vg-autotest_90668/cov_total.info raid-vg-autotest_5263/cov_total.info rocky9-vg-autotest_27871/cov_total.info short-fuzz-phy-autotest_38301/cov_total.info ubuntu22-vg-autotest_49538/cov_total.info ubuntu24-vg-autotest_7099/cov_total.info vfio-user-phy-autotest_38306/cov_total.info vhost-initiator-vg-autotest_90686/cov_total.info vhost-phy-autotest_38373/cov_total.info zns-vg-autotest_89770/cov_total.info 00:32:36.758 + ignore_ocf_headers 00:32:36.758 + sed -i 's#--show-details#--show-details --ignore-errors source#g' spdk/autorun_post.py 00:32:36.758 + create_errtrace_files_for_all_builds 00:32:36.758 + local build_log_list 00:32:36.758 + build_log_list=($(find "$WORKSPACE" -type f -name 'build.log')) 00:32:36.758 ++ find /var/jenkins/workspace/autotest-per-patch_3 -type f -name build.log 00:32:36.758 + for build in "${build_log_list[@]}" 00:32:36.758 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/lvol-vg-autotest_90617/build.log 00:32:36.758 + local build_log job_name start_pattern end_pattern backtraces 00:32:36.758 + build_log=/var/jenkins/workspace/autotest-per-patch_3/lvol-vg-autotest_90617/build.log 00:32:36.758 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/lvol-vg-autotest_90617/build.log 00:32:36.758 ++ basename /var/jenkins/workspace/autotest-per-patch_3/lvol-vg-autotest_90617 00:32:36.758 + job_name=lvol-vg-autotest_90617 00:32:36.758 + start_pattern='Backtrace start' 00:32:36.758 + end_pattern='Backtrace end' 00:32:36.758 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/lvol-vg-autotest_90617/build.log ]] 00:32:36.758 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/lvol-vg-autotest_90617/build.log 00:32:36.758 + backtraces= 00:32:36.758 + [[ -n '' ]] 00:32:36.758 + for build in "${build_log_list[@]}" 00:32:36.758 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/nvme-vg-autotest_90668/build.log 00:32:36.758 + local build_log job_name start_pattern end_pattern backtraces 00:32:36.758 + build_log=/var/jenkins/workspace/autotest-per-patch_3/nvme-vg-autotest_90668/build.log 00:32:36.758 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvme-vg-autotest_90668/build.log 00:32:36.758 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvme-vg-autotest_90668 00:32:36.758 + job_name=nvme-vg-autotest_90668 00:32:36.758 + start_pattern='Backtrace start' 00:32:36.758 + end_pattern='Backtrace end' 00:32:36.758 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/nvme-vg-autotest_90668/build.log ]] 00:32:36.758 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/nvme-vg-autotest_90668/build.log 00:32:36.758 + backtraces= 00:32:36.758 + [[ -n '' ]] 00:32:36.758 + for build in "${build_log_list[@]}" 00:32:36.758 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/raid-vg-autotest_5263/build.log 00:32:36.758 + local build_log job_name start_pattern end_pattern backtraces 00:32:36.758 + build_log=/var/jenkins/workspace/autotest-per-patch_3/raid-vg-autotest_5263/build.log 00:32:36.758 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/raid-vg-autotest_5263/build.log 00:32:36.758 ++ basename /var/jenkins/workspace/autotest-per-patch_3/raid-vg-autotest_5263 00:32:36.758 + job_name=raid-vg-autotest_5263 00:32:36.758 + start_pattern='Backtrace start' 00:32:36.758 + end_pattern='Backtrace end' 00:32:36.758 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/raid-vg-autotest_5263/build.log ]] 00:32:36.758 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/raid-vg-autotest_5263/build.log 00:32:37.021 + backtraces= 00:32:37.021 + [[ -n '' ]] 00:32:37.021 + for build in "${build_log_list[@]}" 00:32:37.021 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/check-format-docker-autotest_33194/build.log 00:32:37.021 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.021 + build_log=/var/jenkins/workspace/autotest-per-patch_3/check-format-docker-autotest_33194/build.log 00:32:37.021 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/check-format-docker-autotest_33194/build.log 00:32:37.021 ++ basename /var/jenkins/workspace/autotest-per-patch_3/check-format-docker-autotest_33194 00:32:37.021 + job_name=check-format-docker-autotest_33194 00:32:37.021 + start_pattern='Backtrace start' 00:32:37.021 + end_pattern='Backtrace end' 00:32:37.021 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/check-format-docker-autotest_33194/build.log ]] 00:32:37.021 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/check-format-docker-autotest_33194/build.log 00:32:37.021 + backtraces= 00:32:37.021 + [[ -n '' ]] 00:32:37.021 + for build in "${build_log_list[@]}" 00:32:37.021 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-uring-vg-autotest_90611/build.log 00:32:37.021 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.021 + build_log=/var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-uring-vg-autotest_90611/build.log 00:32:37.021 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-uring-vg-autotest_90611/build.log 00:32:37.021 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-uring-vg-autotest_90611 00:32:37.021 + job_name=nvmf-tcp-uring-vg-autotest_90611 00:32:37.021 + start_pattern='Backtrace start' 00:32:37.021 + end_pattern='Backtrace end' 00:32:37.021 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-uring-vg-autotest_90611/build.log ]] 00:32:37.021 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-uring-vg-autotest_90611/build.log 00:32:37.021 + backtraces='00:20:57.450 ========== Backtrace start: ========== 00:32:37.021 00:20:57.450 00:32:37.021 00:20:57.450 in /home/vagrant/spdk_repo/spdk/test/nvmf/host/failover.sh:68 -> main(["--transport=tcp"]) 00:32:37.021 00:20:57.450 ... 00:32:37.021 00:20:57.450 63 cat $testdir/try.txt 00:32:37.021 00:20:57.450 64 # if this test fails it means we didn'\''t fail over to the second 00:32:37.021 00:20:57.450 65 count="$(grep -c "Resetting controller successful" < $testdir/try.txt)" 00:32:37.021 00:20:57.450 66 00:32:37.021 00:20:57.450 67 if ((count != 3)); then 00:32:37.021 00:20:57.450 => 68 false 00:32:37.021 00:20:57.450 69 fi 00:32:37.021 00:20:57.450 70 00:32:37.021 00:20:57.450 71 # Part 2 of the test. Start removing ports, starting with the one we are connected to, confirm that the ctrlr remains active until the final trid is removed. 00:32:37.021 00:20:57.450 72 $rootdir/build/examples/bdevperf -z -r $bdevperf_rpc_sock -q 128 -o 4096 -w verify -t 1 -f &> $testdir/try.txt & 00:32:37.021 00:20:57.450 73 bdevperf_pid=$! 00:32:37.021 00:20:57.450 ... 00:32:37.021 00:20:57.451 00:32:37.021 00:20:57.451 ========== Backtrace end ========== 00:32:37.021 00:20:57.876 ========== Backtrace start: ========== 00:32:37.021 00:20:57.876 00:32:37.021 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_failover"],["/home/vagrant/spdk_repo/spdk/test/nvmf/host/failover.sh"],["--transport=tcp"]) 00:32:37.021 00:20:57.876 ... 00:32:37.021 00:20:57.876 1120 timing_enter $test_name 00:32:37.021 00:20:57.876 1121 echo "************************************" 00:32:37.021 00:20:57.876 1122 echo "START TEST $test_name" 00:32:37.021 00:20:57.876 1123 echo "************************************" 00:32:37.021 00:20:57.876 1124 xtrace_restore 00:32:37.021 00:20:57.876 1125 time "$@" 00:32:37.021 00:20:57.876 1126 xtrace_disable 00:32:37.021 00:20:57.876 1127 echo "************************************" 00:32:37.021 00:20:57.876 1128 echo "END TEST $test_name" 00:32:37.021 00:20:57.876 1129 echo "************************************" 00:32:37.021 00:20:57.876 1130 timing_exit $test_name 00:32:37.021 00:20:57.876 ... 00:32:37.021 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/nvmf/nvmf_host.sh:25 -> main(["--transport=tcp"]) 00:32:37.021 00:20:57.876 ... 00:32:37.021 00:20:57.876 20 fi 00:32:37.021 00:20:57.876 21 00:32:37.021 00:20:57.876 22 run_test "nvmf_identify" $rootdir/test/nvmf/host/identify.sh "${TEST_ARGS[@]}" 00:32:37.021 00:20:57.876 23 run_test "nvmf_perf" $rootdir/test/nvmf/host/perf.sh "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 24 run_test "nvmf_fio_host" $rootdir/test/nvmf/host/fio.sh "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 => 25 run_test "nvmf_failover" $rootdir/test/nvmf/host/failover.sh "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 26 run_test "nvmf_host_discovery" $rootdir/test/nvmf/host/discovery.sh "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 27 run_test "nvmf_host_multipath_status" $rootdir/test/nvmf/host/multipath_status.sh "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 28 run_test "nvmf_discovery_remove_ifc" $rootdir/test/nvmf/host/discovery_remove_ifc.sh "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 29 run_test "nvmf_identify_kernel_target" "$rootdir/test/nvmf/host/identify_kernel_nvmf.sh" "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 30 run_test "nvmf_auth_host" "$rootdir/test/nvmf/host/auth.sh" "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 00:32:37.022 00:20:57.876 ========== Backtrace end ========== 00:32:37.022 00:20:57.876 ========== Backtrace start: ========== 00:32:37.022 00:20:57.876 00:32:37.022 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_host"],["/home/vagrant/spdk_repo/spdk/test/nvmf/nvmf_host.sh"],["--transport=tcp"]) 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 1120 timing_enter $test_name 00:32:37.022 00:20:57.876 1121 echo "************************************" 00:32:37.022 00:20:57.876 1122 echo "START TEST $test_name" 00:32:37.022 00:20:57.876 1123 echo "************************************" 00:32:37.022 00:20:57.876 1124 xtrace_restore 00:32:37.022 00:20:57.876 1125 time "$@" 00:32:37.022 00:20:57.876 1126 xtrace_disable 00:32:37.022 00:20:57.876 1127 echo "************************************" 00:32:37.022 00:20:57.876 1128 echo "END TEST $test_name" 00:32:37.022 00:20:57.876 1129 echo "************************************" 00:32:37.022 00:20:57.876 1130 timing_exit $test_name 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/nvmf/nvmf.sh:16 -> main(["--transport=tcp"]) 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 11 exit 0 00:32:37.022 00:20:57.876 12 fi 00:32:37.022 00:20:57.876 13 00:32:37.022 00:20:57.876 14 run_test "nvmf_target_core" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.022 00:20:57.876 15 run_test "nvmf_target_extra" $rootdir/test/nvmf/nvmf_target_extra.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.022 00:20:57.876 => 16 run_test "nvmf_host" $rootdir/test/nvmf/nvmf_host.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.022 00:20:57.876 17 00:32:37.022 00:20:57.876 18 # Interrupt mode for now is supported only on the target, with the TCP transport and posix or ssl socket implementations. 00:32:37.022 00:20:57.876 19 if [[ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" && $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.022 00:20:57.876 20 run_test "nvmf_target_core_interrupt_mode" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.022 00:20:57.876 21 run_test "nvmf_interrupt" $rootdir/test/nvmf/target/interrupt.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 00:32:37.022 00:20:57.876 ========== Backtrace end ========== 00:32:37.022 00:20:57.876 ========== Backtrace start: ========== 00:32:37.022 00:20:57.876 00:32:37.022 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_tcp"],["/home/vagrant/spdk_repo/spdk/test/nvmf/nvmf.sh"],["--transport=tcp"]) 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 1120 timing_enter $test_name 00:32:37.022 00:20:57.876 1121 echo "************************************" 00:32:37.022 00:20:57.876 1122 echo "START TEST $test_name" 00:32:37.022 00:20:57.876 1123 echo "************************************" 00:32:37.022 00:20:57.876 1124 xtrace_restore 00:32:37.022 00:20:57.876 1125 time "$@" 00:32:37.022 00:20:57.876 1126 xtrace_disable 00:32:37.022 00:20:57.876 1127 echo "************************************" 00:32:37.022 00:20:57.876 1128 echo "END TEST $test_name" 00:32:37.022 00:20:57.876 1129 echo "************************************" 00:32:37.022 00:20:57.876 1130 timing_exit $test_name 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 in /home/vagrant/spdk_repo/spdk/autotest.sh:280 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 275 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:32:37.022 00:20:57.876 276 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:32:37.022 00:20:57.876 277 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.022 00:20:57.876 278 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.022 00:20:57.876 279 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:32:37.022 00:20:57.876 => 280 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.022 00:20:57.876 281 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.022 00:20:57.876 282 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.022 00:20:57.876 283 run_test "nvmf_identify_passthru" $rootdir/test/nvmf/target/identify_passthru.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.022 00:20:57.876 284 fi 00:32:37.022 00:20:57.876 285 run_test "nvmf_dif" $rootdir/test/nvmf/target/dif.sh 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 00:32:37.022 00:20:57.876 ========== Backtrace end ==========' 00:32:37.022 + [[ -n 00:20:57.450 ========== Backtrace start: ========== 00:32:37.022 00:20:57.450 00:32:37.022 00:20:57.450 in /home/vagrant/spdk_repo/spdk/test/nvmf/host/failover.sh:68 -> main(["--transport=tcp"]) 00:32:37.022 00:20:57.450 ... 00:32:37.022 00:20:57.450 63 cat $testdir/try.txt 00:32:37.022 00:20:57.450 64 # if this test fails it means we didn't fail over to the second 00:32:37.022 00:20:57.450 65 count="$(grep -c "Resetting controller successful" < $testdir/try.txt)" 00:32:37.022 00:20:57.450 66 00:32:37.022 00:20:57.450 67 if ((count != 3)); then 00:32:37.022 00:20:57.450 => 68 false 00:32:37.022 00:20:57.450 69 fi 00:32:37.022 00:20:57.450 70 00:32:37.022 00:20:57.450 71 # Part 2 of the test. Start removing ports, starting with the one we are connected to, confirm that the ctrlr remains active until the final trid is removed. 00:32:37.022 00:20:57.450 72 $rootdir/build/examples/bdevperf -z -r $bdevperf_rpc_sock -q 128 -o 4096 -w verify -t 1 -f &> $testdir/try.txt & 00:32:37.022 00:20:57.450 73 bdevperf_pid=$! 00:32:37.022 00:20:57.450 ... 00:32:37.022 00:20:57.451 00:32:37.022 00:20:57.451 ========== Backtrace end ========== 00:32:37.022 00:20:57.876 ========== Backtrace start: ========== 00:32:37.022 00:20:57.876 00:32:37.022 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_failover"],["/home/vagrant/spdk_repo/spdk/test/nvmf/host/failover.sh"],["--transport=tcp"]) 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 1120 timing_enter $test_name 00:32:37.022 00:20:57.876 1121 echo "************************************" 00:32:37.022 00:20:57.876 1122 echo "START TEST $test_name" 00:32:37.022 00:20:57.876 1123 echo "************************************" 00:32:37.022 00:20:57.876 1124 xtrace_restore 00:32:37.022 00:20:57.876 1125 time "$@" 00:32:37.022 00:20:57.876 1126 xtrace_disable 00:32:37.022 00:20:57.876 1127 echo "************************************" 00:32:37.022 00:20:57.876 1128 echo "END TEST $test_name" 00:32:37.022 00:20:57.876 1129 echo "************************************" 00:32:37.022 00:20:57.876 1130 timing_exit $test_name 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/nvmf/nvmf_host.sh:25 -> main(["--transport=tcp"]) 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 20 fi 00:32:37.022 00:20:57.876 21 00:32:37.022 00:20:57.876 22 run_test "nvmf_identify" $rootdir/test/nvmf/host/identify.sh "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 23 run_test "nvmf_perf" $rootdir/test/nvmf/host/perf.sh "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 24 run_test "nvmf_fio_host" $rootdir/test/nvmf/host/fio.sh "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 => 25 run_test "nvmf_failover" $rootdir/test/nvmf/host/failover.sh "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 26 run_test "nvmf_host_discovery" $rootdir/test/nvmf/host/discovery.sh "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 27 run_test "nvmf_host_multipath_status" $rootdir/test/nvmf/host/multipath_status.sh "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 28 run_test "nvmf_discovery_remove_ifc" $rootdir/test/nvmf/host/discovery_remove_ifc.sh "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 29 run_test "nvmf_identify_kernel_target" "$rootdir/test/nvmf/host/identify_kernel_nvmf.sh" "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 30 run_test "nvmf_auth_host" "$rootdir/test/nvmf/host/auth.sh" "${TEST_ARGS[@]}" 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 00:32:37.022 00:20:57.876 ========== Backtrace end ========== 00:32:37.022 00:20:57.876 ========== Backtrace start: ========== 00:32:37.022 00:20:57.876 00:32:37.022 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_host"],["/home/vagrant/spdk_repo/spdk/test/nvmf/nvmf_host.sh"],["--transport=tcp"]) 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 1120 timing_enter $test_name 00:32:37.022 00:20:57.876 1121 echo "************************************" 00:32:37.022 00:20:57.876 1122 echo "START TEST $test_name" 00:32:37.022 00:20:57.876 1123 echo "************************************" 00:32:37.022 00:20:57.876 1124 xtrace_restore 00:32:37.022 00:20:57.876 1125 time "$@" 00:32:37.022 00:20:57.876 1126 xtrace_disable 00:32:37.022 00:20:57.876 1127 echo "************************************" 00:32:37.022 00:20:57.876 1128 echo "END TEST $test_name" 00:32:37.022 00:20:57.876 1129 echo "************************************" 00:32:37.022 00:20:57.876 1130 timing_exit $test_name 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/nvmf/nvmf.sh:16 -> main(["--transport=tcp"]) 00:32:37.022 00:20:57.876 ... 00:32:37.022 00:20:57.876 11 exit 0 00:32:37.022 00:20:57.876 12 fi 00:32:37.022 00:20:57.876 13 00:32:37.022 00:20:57.876 14 run_test "nvmf_target_core" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.022 00:20:57.876 15 run_test "nvmf_target_extra" $rootdir/test/nvmf/nvmf_target_extra.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.022 00:20:57.876 => 16 run_test "nvmf_host" $rootdir/test/nvmf/nvmf_host.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.022 00:20:57.876 17 00:32:37.022 00:20:57.876 18 # Interrupt mode for now is supported only on the target, with the TCP transport and posix or ssl socket implementations. 00:32:37.022 00:20:57.876 19 if [[ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" && $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.022 00:20:57.876 20 run_test "nvmf_target_core_interrupt_mode" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.022 00:20:57.876 21 run_test "nvmf_interrupt" $rootdir/test/nvmf/target/interrupt.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 00:32:37.023 00:20:57.876 ========== Backtrace end ========== 00:32:37.023 00:20:57.876 ========== Backtrace start: ========== 00:32:37.023 00:20:57.876 00:32:37.023 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_tcp"],["/home/vagrant/spdk_repo/spdk/test/nvmf/nvmf.sh"],["--transport=tcp"]) 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 1120 timing_enter $test_name 00:32:37.023 00:20:57.876 1121 echo "************************************" 00:32:37.023 00:20:57.876 1122 echo "START TEST $test_name" 00:32:37.023 00:20:57.876 1123 echo "************************************" 00:32:37.023 00:20:57.876 1124 xtrace_restore 00:32:37.023 00:20:57.876 1125 time "$@" 00:32:37.023 00:20:57.876 1126 xtrace_disable 00:32:37.023 00:20:57.876 1127 echo "************************************" 00:32:37.023 00:20:57.876 1128 echo "END TEST $test_name" 00:32:37.023 00:20:57.876 1129 echo "************************************" 00:32:37.023 00:20:57.876 1130 timing_exit $test_name 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 in /home/vagrant/spdk_repo/spdk/autotest.sh:280 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 275 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:32:37.023 00:20:57.876 276 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:32:37.023 00:20:57.876 277 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.023 00:20:57.876 278 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.023 00:20:57.876 279 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:32:37.023 00:20:57.876 => 280 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.023 00:20:57.876 281 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.023 00:20:57.876 282 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.023 00:20:57.876 283 run_test "nvmf_identify_passthru" $rootdir/test/nvmf/target/identify_passthru.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.023 00:20:57.876 284 fi 00:32:37.023 00:20:57.876 285 run_test "nvmf_dif" $rootdir/test/nvmf/target/dif.sh 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 00:32:37.023 00:20:57.876 ========== Backtrace end ========== ]] 00:32:37.023 + echo '00:20:57.450 ========== Backtrace start: ========== 00:32:37.023 00:20:57.450 00:32:37.023 00:20:57.450 in /home/vagrant/spdk_repo/spdk/test/nvmf/host/failover.sh:68 -> main(["--transport=tcp"]) 00:32:37.023 00:20:57.450 ... 00:32:37.023 00:20:57.450 63 cat $testdir/try.txt 00:32:37.023 00:20:57.450 64 # if this test fails it means we didn'\''t fail over to the second 00:32:37.023 00:20:57.450 65 count="$(grep -c "Resetting controller successful" < $testdir/try.txt)" 00:32:37.023 00:20:57.450 66 00:32:37.023 00:20:57.450 67 if ((count != 3)); then 00:32:37.023 00:20:57.450 => 68 false 00:32:37.023 00:20:57.450 69 fi 00:32:37.023 00:20:57.450 70 00:32:37.023 00:20:57.450 71 # Part 2 of the test. Start removing ports, starting with the one we are connected to, confirm that the ctrlr remains active until the final trid is removed. 00:32:37.023 00:20:57.450 72 $rootdir/build/examples/bdevperf -z -r $bdevperf_rpc_sock -q 128 -o 4096 -w verify -t 1 -f &> $testdir/try.txt & 00:32:37.023 00:20:57.450 73 bdevperf_pid=$! 00:32:37.023 00:20:57.450 ... 00:32:37.023 00:20:57.451 00:32:37.023 00:20:57.451 ========== Backtrace end ========== 00:32:37.023 00:20:57.876 ========== Backtrace start: ========== 00:32:37.023 00:20:57.876 00:32:37.023 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_failover"],["/home/vagrant/spdk_repo/spdk/test/nvmf/host/failover.sh"],["--transport=tcp"]) 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 1120 timing_enter $test_name 00:32:37.023 00:20:57.876 1121 echo "************************************" 00:32:37.023 00:20:57.876 1122 echo "START TEST $test_name" 00:32:37.023 00:20:57.876 1123 echo "************************************" 00:32:37.023 00:20:57.876 1124 xtrace_restore 00:32:37.023 00:20:57.876 1125 time "$@" 00:32:37.023 00:20:57.876 1126 xtrace_disable 00:32:37.023 00:20:57.876 1127 echo "************************************" 00:32:37.023 00:20:57.876 1128 echo "END TEST $test_name" 00:32:37.023 00:20:57.876 1129 echo "************************************" 00:32:37.023 00:20:57.876 1130 timing_exit $test_name 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/nvmf/nvmf_host.sh:25 -> main(["--transport=tcp"]) 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 20 fi 00:32:37.023 00:20:57.876 21 00:32:37.023 00:20:57.876 22 run_test "nvmf_identify" $rootdir/test/nvmf/host/identify.sh "${TEST_ARGS[@]}" 00:32:37.023 00:20:57.876 23 run_test "nvmf_perf" $rootdir/test/nvmf/host/perf.sh "${TEST_ARGS[@]}" 00:32:37.023 00:20:57.876 24 run_test "nvmf_fio_host" $rootdir/test/nvmf/host/fio.sh "${TEST_ARGS[@]}" 00:32:37.023 00:20:57.876 => 25 run_test "nvmf_failover" $rootdir/test/nvmf/host/failover.sh "${TEST_ARGS[@]}" 00:32:37.023 00:20:57.876 26 run_test "nvmf_host_discovery" $rootdir/test/nvmf/host/discovery.sh "${TEST_ARGS[@]}" 00:32:37.023 00:20:57.876 27 run_test "nvmf_host_multipath_status" $rootdir/test/nvmf/host/multipath_status.sh "${TEST_ARGS[@]}" 00:32:37.023 00:20:57.876 28 run_test "nvmf_discovery_remove_ifc" $rootdir/test/nvmf/host/discovery_remove_ifc.sh "${TEST_ARGS[@]}" 00:32:37.023 00:20:57.876 29 run_test "nvmf_identify_kernel_target" "$rootdir/test/nvmf/host/identify_kernel_nvmf.sh" "${TEST_ARGS[@]}" 00:32:37.023 00:20:57.876 30 run_test "nvmf_auth_host" "$rootdir/test/nvmf/host/auth.sh" "${TEST_ARGS[@]}" 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 00:32:37.023 00:20:57.876 ========== Backtrace end ========== 00:32:37.023 00:20:57.876 ========== Backtrace start: ========== 00:32:37.023 00:20:57.876 00:32:37.023 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_host"],["/home/vagrant/spdk_repo/spdk/test/nvmf/nvmf_host.sh"],["--transport=tcp"]) 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 1120 timing_enter $test_name 00:32:37.023 00:20:57.876 1121 echo "************************************" 00:32:37.023 00:20:57.876 1122 echo "START TEST $test_name" 00:32:37.023 00:20:57.876 1123 echo "************************************" 00:32:37.023 00:20:57.876 1124 xtrace_restore 00:32:37.023 00:20:57.876 1125 time "$@" 00:32:37.023 00:20:57.876 1126 xtrace_disable 00:32:37.023 00:20:57.876 1127 echo "************************************" 00:32:37.023 00:20:57.876 1128 echo "END TEST $test_name" 00:32:37.023 00:20:57.876 1129 echo "************************************" 00:32:37.023 00:20:57.876 1130 timing_exit $test_name 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/nvmf/nvmf.sh:16 -> main(["--transport=tcp"]) 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 11 exit 0 00:32:37.023 00:20:57.876 12 fi 00:32:37.023 00:20:57.876 13 00:32:37.023 00:20:57.876 14 run_test "nvmf_target_core" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.023 00:20:57.876 15 run_test "nvmf_target_extra" $rootdir/test/nvmf/nvmf_target_extra.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.023 00:20:57.876 => 16 run_test "nvmf_host" $rootdir/test/nvmf/nvmf_host.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.023 00:20:57.876 17 00:32:37.023 00:20:57.876 18 # Interrupt mode for now is supported only on the target, with the TCP transport and posix or ssl socket implementations. 00:32:37.023 00:20:57.876 19 if [[ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" && $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.023 00:20:57.876 20 run_test "nvmf_target_core_interrupt_mode" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.023 00:20:57.876 21 run_test "nvmf_interrupt" $rootdir/test/nvmf/target/interrupt.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 00:32:37.023 00:20:57.876 ========== Backtrace end ========== 00:32:37.023 00:20:57.876 ========== Backtrace start: ========== 00:32:37.023 00:20:57.876 00:32:37.023 00:20:57.876 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_tcp"],["/home/vagrant/spdk_repo/spdk/test/nvmf/nvmf.sh"],["--transport=tcp"]) 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 1120 timing_enter $test_name 00:32:37.023 00:20:57.876 1121 echo "************************************" 00:32:37.023 00:20:57.876 1122 echo "START TEST $test_name" 00:32:37.023 00:20:57.876 1123 echo "************************************" 00:32:37.023 00:20:57.876 1124 xtrace_restore 00:32:37.023 00:20:57.876 1125 time "$@" 00:32:37.023 00:20:57.876 1126 xtrace_disable 00:32:37.023 00:20:57.876 1127 echo "************************************" 00:32:37.023 00:20:57.876 1128 echo "END TEST $test_name" 00:32:37.023 00:20:57.876 1129 echo "************************************" 00:32:37.023 00:20:57.876 1130 timing_exit $test_name 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 in /home/vagrant/spdk_repo/spdk/autotest.sh:280 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 275 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:32:37.023 00:20:57.876 276 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:32:37.023 00:20:57.876 277 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.023 00:20:57.876 278 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.023 00:20:57.876 279 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:32:37.023 00:20:57.876 => 280 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.023 00:20:57.876 281 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.023 00:20:57.876 282 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.023 00:20:57.876 283 run_test "nvmf_identify_passthru" $rootdir/test/nvmf/target/identify_passthru.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.023 00:20:57.876 284 fi 00:32:37.023 00:20:57.876 285 run_test "nvmf_dif" $rootdir/test/nvmf/target/dif.sh 00:32:37.023 00:20:57.876 ... 00:32:37.023 00:20:57.876 00:32:37.023 00:20:57.876 ========== Backtrace end ==========' 00:32:37.023 + for build in "${build_log_list[@]}" 00:32:37.024 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/vhost-phy-autotest_38373/build.log 00:32:37.024 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.024 + build_log=/var/jenkins/workspace/autotest-per-patch_3/vhost-phy-autotest_38373/build.log 00:32:37.024 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/vhost-phy-autotest_38373/build.log 00:32:37.024 ++ basename /var/jenkins/workspace/autotest-per-patch_3/vhost-phy-autotest_38373 00:32:37.024 + job_name=vhost-phy-autotest_38373 00:32:37.024 + start_pattern='Backtrace start' 00:32:37.024 + end_pattern='Backtrace end' 00:32:37.024 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/vhost-phy-autotest_38373/build.log ]] 00:32:37.024 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/vhost-phy-autotest_38373/build.log 00:32:37.024 + backtraces= 00:32:37.024 + [[ -n '' ]] 00:32:37.024 + for build in "${build_log_list[@]}" 00:32:37.024 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/iscsi-uring-vg-autotest_90569/build.log 00:32:37.024 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.024 + build_log=/var/jenkins/workspace/autotest-per-patch_3/iscsi-uring-vg-autotest_90569/build.log 00:32:37.024 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/iscsi-uring-vg-autotest_90569/build.log 00:32:37.024 ++ basename /var/jenkins/workspace/autotest-per-patch_3/iscsi-uring-vg-autotest_90569 00:32:37.024 + job_name=iscsi-uring-vg-autotest_90569 00:32:37.024 + start_pattern='Backtrace start' 00:32:37.024 + end_pattern='Backtrace end' 00:32:37.024 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/iscsi-uring-vg-autotest_90569/build.log ]] 00:32:37.024 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/iscsi-uring-vg-autotest_90569/build.log 00:32:37.024 + backtraces= 00:32:37.024 + [[ -n '' ]] 00:32:37.024 + for build in "${build_log_list[@]}" 00:32:37.024 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-phy-autotest_84430/build.log 00:32:37.024 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.024 + build_log=/var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-phy-autotest_84430/build.log 00:32:37.024 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-phy-autotest_84430/build.log 00:32:37.024 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-phy-autotest_84430 00:32:37.024 + job_name=nvmf-tcp-phy-autotest_84430 00:32:37.024 + start_pattern='Backtrace start' 00:32:37.024 + end_pattern='Backtrace end' 00:32:37.024 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-phy-autotest_84430/build.log ]] 00:32:37.024 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-phy-autotest_84430/build.log 00:32:37.024 + backtraces='00:24:23.360 ========== Backtrace start: ========== 00:32:37.024 00:24:23.360 00:32:37.024 00:24:23.360 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/host/failover.sh:68 -> main(["--transport=tcp"]) 00:32:37.024 00:24:23.360 ... 00:32:37.024 00:24:23.360 63 cat $testdir/try.txt 00:32:37.024 00:24:23.360 64 # if this test fails it means we didn'\''t fail over to the second 00:32:37.024 00:24:23.360 65 count="$(grep -c "Resetting controller successful" < $testdir/try.txt)" 00:32:37.024 00:24:23.360 66 00:32:37.024 00:24:23.360 67 if ((count != 3)); then 00:32:37.024 00:24:23.360 => 68 false 00:32:37.024 00:24:23.360 69 fi 00:32:37.024 00:24:23.360 70 00:32:37.024 00:24:23.360 71 # Part 2 of the test. Start removing ports, starting with the one we are connected to, confirm that the ctrlr remains active until the final trid is removed. 00:32:37.024 00:24:23.360 72 $rootdir/build/examples/bdevperf -z -r $bdevperf_rpc_sock -q 128 -o 4096 -w verify -t 1 -f &> $testdir/try.txt & 00:32:37.024 00:24:23.360 73 bdevperf_pid=$! 00:32:37.024 00:24:23.360 ... 00:32:37.024 00:24:23.360 00:32:37.024 00:24:23.360 ========== Backtrace end ========== 00:32:37.024 00:24:24.967 ========== Backtrace start: ========== 00:32:37.024 00:24:24.967 00:32:37.024 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_failover"],["/var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/host/failover.sh"],["--transport=tcp"]) 00:32:37.024 00:24:24.967 ... 00:32:37.024 00:24:24.967 1120 timing_enter $test_name 00:32:37.024 00:24:24.967 1121 echo "************************************" 00:32:37.024 00:24:24.967 1122 echo "START TEST $test_name" 00:32:37.024 00:24:24.967 1123 echo "************************************" 00:32:37.024 00:24:24.967 1124 xtrace_restore 00:32:37.024 00:24:24.967 1125 time "$@" 00:32:37.024 00:24:24.967 1126 xtrace_disable 00:32:37.024 00:24:24.967 1127 echo "************************************" 00:32:37.024 00:24:24.967 1128 echo "END TEST $test_name" 00:32:37.024 00:24:24.967 1129 echo "************************************" 00:32:37.024 00:24:24.967 1130 timing_exit $test_name 00:32:37.024 00:24:24.967 ... 00:32:37.024 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/nvmf_host.sh:25 -> main(["--transport=tcp"]) 00:32:37.024 00:24:24.967 ... 00:32:37.024 00:24:24.967 20 fi 00:32:37.024 00:24:24.967 21 00:32:37.024 00:24:24.967 22 run_test "nvmf_identify" $rootdir/test/nvmf/host/identify.sh "${TEST_ARGS[@]}" 00:32:37.024 00:24:24.967 23 run_test "nvmf_perf" $rootdir/test/nvmf/host/perf.sh "${TEST_ARGS[@]}" 00:32:37.024 00:24:24.967 24 run_test "nvmf_fio_host" $rootdir/test/nvmf/host/fio.sh "${TEST_ARGS[@]}" 00:32:37.024 00:24:24.967 => 25 run_test "nvmf_failover" $rootdir/test/nvmf/host/failover.sh "${TEST_ARGS[@]}" 00:32:37.024 00:24:24.967 26 run_test "nvmf_host_discovery" $rootdir/test/nvmf/host/discovery.sh "${TEST_ARGS[@]}" 00:32:37.024 00:24:24.967 27 run_test "nvmf_host_multipath_status" $rootdir/test/nvmf/host/multipath_status.sh "${TEST_ARGS[@]}" 00:32:37.024 00:24:24.967 28 run_test "nvmf_discovery_remove_ifc" $rootdir/test/nvmf/host/discovery_remove_ifc.sh "${TEST_ARGS[@]}" 00:32:37.024 00:24:24.967 29 run_test "nvmf_identify_kernel_target" "$rootdir/test/nvmf/host/identify_kernel_nvmf.sh" "${TEST_ARGS[@]}" 00:32:37.024 00:24:24.967 30 run_test "nvmf_auth_host" "$rootdir/test/nvmf/host/auth.sh" "${TEST_ARGS[@]}" 00:32:37.024 00:24:24.967 ... 00:32:37.024 00:24:24.967 00:32:37.024 00:24:24.967 ========== Backtrace end ========== 00:32:37.024 00:24:24.967 ========== Backtrace start: ========== 00:32:37.024 00:24:24.967 00:32:37.024 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_host"],["/var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/nvmf_host.sh"],["--transport=tcp"]) 00:32:37.024 00:24:24.967 ... 00:32:37.024 00:24:24.967 1120 timing_enter $test_name 00:32:37.024 00:24:24.967 1121 echo "************************************" 00:32:37.024 00:24:24.967 1122 echo "START TEST $test_name" 00:32:37.024 00:24:24.967 1123 echo "************************************" 00:32:37.024 00:24:24.967 1124 xtrace_restore 00:32:37.024 00:24:24.967 1125 time "$@" 00:32:37.024 00:24:24.967 1126 xtrace_disable 00:32:37.024 00:24:24.967 1127 echo "************************************" 00:32:37.024 00:24:24.967 1128 echo "END TEST $test_name" 00:32:37.024 00:24:24.967 1129 echo "************************************" 00:32:37.024 00:24:24.967 1130 timing_exit $test_name 00:32:37.024 00:24:24.967 ... 00:32:37.024 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/nvmf.sh:16 -> main(["--transport=tcp"]) 00:32:37.024 00:24:24.967 ... 00:32:37.024 00:24:24.967 11 exit 0 00:32:37.024 00:24:24.967 12 fi 00:32:37.024 00:24:24.967 13 00:32:37.024 00:24:24.967 14 run_test "nvmf_target_core" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.024 00:24:24.967 15 run_test "nvmf_target_extra" $rootdir/test/nvmf/nvmf_target_extra.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.024 00:24:24.967 => 16 run_test "nvmf_host" $rootdir/test/nvmf/nvmf_host.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.024 00:24:24.967 17 00:32:37.024 00:24:24.967 18 # Interrupt mode for now is supported only on the target, with the TCP transport and posix or ssl socket implementations. 00:32:37.024 00:24:24.967 19 if [[ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" && $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.024 00:24:24.967 20 run_test "nvmf_target_core_interrupt_mode" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.024 00:24:24.967 21 run_test "nvmf_interrupt" $rootdir/test/nvmf/target/interrupt.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.024 00:24:24.967 ... 00:32:37.024 00:24:24.967 00:32:37.024 00:24:24.967 ========== Backtrace end ========== 00:32:37.024 00:24:24.967 ========== Backtrace start: ========== 00:32:37.024 00:24:24.967 00:32:37.024 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_tcp"],["/var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/nvmf.sh"],["--transport=tcp"]) 00:32:37.024 00:24:24.967 ... 00:32:37.024 00:24:24.968 1120 timing_enter $test_name 00:32:37.024 00:24:24.968 1121 echo "************************************" 00:32:37.024 00:24:24.968 1122 echo "START TEST $test_name" 00:32:37.024 00:24:24.968 1123 echo "************************************" 00:32:37.024 00:24:24.968 1124 xtrace_restore 00:32:37.024 00:24:24.968 1125 time "$@" 00:32:37.024 00:24:24.968 1126 xtrace_disable 00:32:37.024 00:24:24.968 1127 echo "************************************" 00:32:37.024 00:24:24.968 1128 echo "END TEST $test_name" 00:32:37.024 00:24:24.968 1129 echo "************************************" 00:32:37.024 00:24:24.968 1130 timing_exit $test_name 00:32:37.024 00:24:24.968 ... 00:32:37.024 00:24:24.968 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/autotest.sh:280 -> main(["/var/jenkins/workspace/nvmf-tcp-phy-autotest/autorun-spdk.conf"]) 00:32:37.024 00:24:24.968 ... 00:32:37.024 00:24:24.968 275 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:32:37.024 00:24:24.968 276 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:32:37.024 00:24:24.968 277 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.024 00:24:24.968 278 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.024 00:24:24.968 279 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:32:37.024 00:24:24.968 => 280 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.024 00:24:24.968 281 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.024 00:24:24.968 282 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.025 00:24:24.968 283 run_test "nvmf_identify_passthru" $rootdir/test/nvmf/target/identify_passthru.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.025 00:24:24.968 284 fi 00:32:37.025 00:24:24.968 285 run_test "nvmf_dif" $rootdir/test/nvmf/target/dif.sh 00:32:37.025 00:24:24.968 ... 00:32:37.025 00:24:24.968 00:32:37.025 00:24:24.968 ========== Backtrace end ==========' 00:32:37.025 + [[ -n 00:24:23.360 ========== Backtrace start: ========== 00:32:37.025 00:24:23.360 00:32:37.025 00:24:23.360 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/host/failover.sh:68 -> main(["--transport=tcp"]) 00:32:37.025 00:24:23.360 ... 00:32:37.025 00:24:23.360 63 cat $testdir/try.txt 00:32:37.025 00:24:23.360 64 # if this test fails it means we didn't fail over to the second 00:32:37.025 00:24:23.360 65 count="$(grep -c "Resetting controller successful" < $testdir/try.txt)" 00:32:37.025 00:24:23.360 66 00:32:37.025 00:24:23.360 67 if ((count != 3)); then 00:32:37.025 00:24:23.360 => 68 false 00:32:37.025 00:24:23.360 69 fi 00:32:37.025 00:24:23.360 70 00:32:37.025 00:24:23.360 71 # Part 2 of the test. Start removing ports, starting with the one we are connected to, confirm that the ctrlr remains active until the final trid is removed. 00:32:37.025 00:24:23.360 72 $rootdir/build/examples/bdevperf -z -r $bdevperf_rpc_sock -q 128 -o 4096 -w verify -t 1 -f &> $testdir/try.txt & 00:32:37.025 00:24:23.360 73 bdevperf_pid=$! 00:32:37.025 00:24:23.360 ... 00:32:37.025 00:24:23.360 00:32:37.025 00:24:23.360 ========== Backtrace end ========== 00:32:37.025 00:24:24.967 ========== Backtrace start: ========== 00:32:37.025 00:24:24.967 00:32:37.025 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_failover"],["/var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/host/failover.sh"],["--transport=tcp"]) 00:32:37.025 00:24:24.967 ... 00:32:37.025 00:24:24.967 1120 timing_enter $test_name 00:32:37.025 00:24:24.967 1121 echo "************************************" 00:32:37.025 00:24:24.967 1122 echo "START TEST $test_name" 00:32:37.025 00:24:24.967 1123 echo "************************************" 00:32:37.025 00:24:24.967 1124 xtrace_restore 00:32:37.025 00:24:24.967 1125 time "$@" 00:32:37.025 00:24:24.967 1126 xtrace_disable 00:32:37.025 00:24:24.967 1127 echo "************************************" 00:32:37.025 00:24:24.967 1128 echo "END TEST $test_name" 00:32:37.025 00:24:24.967 1129 echo "************************************" 00:32:37.025 00:24:24.967 1130 timing_exit $test_name 00:32:37.025 00:24:24.967 ... 00:32:37.025 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/nvmf_host.sh:25 -> main(["--transport=tcp"]) 00:32:37.025 00:24:24.967 ... 00:32:37.025 00:24:24.967 20 fi 00:32:37.025 00:24:24.967 21 00:32:37.025 00:24:24.967 22 run_test "nvmf_identify" $rootdir/test/nvmf/host/identify.sh "${TEST_ARGS[@]}" 00:32:37.025 00:24:24.967 23 run_test "nvmf_perf" $rootdir/test/nvmf/host/perf.sh "${TEST_ARGS[@]}" 00:32:37.025 00:24:24.967 24 run_test "nvmf_fio_host" $rootdir/test/nvmf/host/fio.sh "${TEST_ARGS[@]}" 00:32:37.025 00:24:24.967 => 25 run_test "nvmf_failover" $rootdir/test/nvmf/host/failover.sh "${TEST_ARGS[@]}" 00:32:37.025 00:24:24.967 26 run_test "nvmf_host_discovery" $rootdir/test/nvmf/host/discovery.sh "${TEST_ARGS[@]}" 00:32:37.025 00:24:24.967 27 run_test "nvmf_host_multipath_status" $rootdir/test/nvmf/host/multipath_status.sh "${TEST_ARGS[@]}" 00:32:37.025 00:24:24.967 28 run_test "nvmf_discovery_remove_ifc" $rootdir/test/nvmf/host/discovery_remove_ifc.sh "${TEST_ARGS[@]}" 00:32:37.025 00:24:24.967 29 run_test "nvmf_identify_kernel_target" "$rootdir/test/nvmf/host/identify_kernel_nvmf.sh" "${TEST_ARGS[@]}" 00:32:37.025 00:24:24.967 30 run_test "nvmf_auth_host" "$rootdir/test/nvmf/host/auth.sh" "${TEST_ARGS[@]}" 00:32:37.025 00:24:24.967 ... 00:32:37.025 00:24:24.967 00:32:37.025 00:24:24.967 ========== Backtrace end ========== 00:32:37.025 00:24:24.967 ========== Backtrace start: ========== 00:32:37.025 00:24:24.967 00:32:37.025 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_host"],["/var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/nvmf_host.sh"],["--transport=tcp"]) 00:32:37.025 00:24:24.967 ... 00:32:37.025 00:24:24.967 1120 timing_enter $test_name 00:32:37.025 00:24:24.967 1121 echo "************************************" 00:32:37.025 00:24:24.967 1122 echo "START TEST $test_name" 00:32:37.025 00:24:24.967 1123 echo "************************************" 00:32:37.025 00:24:24.967 1124 xtrace_restore 00:32:37.025 00:24:24.967 1125 time "$@" 00:32:37.025 00:24:24.967 1126 xtrace_disable 00:32:37.025 00:24:24.967 1127 echo "************************************" 00:32:37.025 00:24:24.967 1128 echo "END TEST $test_name" 00:32:37.025 00:24:24.967 1129 echo "************************************" 00:32:37.025 00:24:24.967 1130 timing_exit $test_name 00:32:37.025 00:24:24.967 ... 00:32:37.025 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/nvmf.sh:16 -> main(["--transport=tcp"]) 00:32:37.025 00:24:24.967 ... 00:32:37.025 00:24:24.967 11 exit 0 00:32:37.025 00:24:24.967 12 fi 00:32:37.025 00:24:24.967 13 00:32:37.025 00:24:24.967 14 run_test "nvmf_target_core" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.025 00:24:24.967 15 run_test "nvmf_target_extra" $rootdir/test/nvmf/nvmf_target_extra.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.025 00:24:24.967 => 16 run_test "nvmf_host" $rootdir/test/nvmf/nvmf_host.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.025 00:24:24.967 17 00:32:37.025 00:24:24.967 18 # Interrupt mode for now is supported only on the target, with the TCP transport and posix or ssl socket implementations. 00:32:37.025 00:24:24.967 19 if [[ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" && $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.025 00:24:24.967 20 run_test "nvmf_target_core_interrupt_mode" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.025 00:24:24.967 21 run_test "nvmf_interrupt" $rootdir/test/nvmf/target/interrupt.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.025 00:24:24.967 ... 00:32:37.025 00:24:24.967 00:32:37.025 00:24:24.967 ========== Backtrace end ========== 00:32:37.025 00:24:24.967 ========== Backtrace start: ========== 00:32:37.025 00:24:24.967 00:32:37.025 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_tcp"],["/var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/nvmf.sh"],["--transport=tcp"]) 00:32:37.025 00:24:24.967 ... 00:32:37.025 00:24:24.968 1120 timing_enter $test_name 00:32:37.025 00:24:24.968 1121 echo "************************************" 00:32:37.025 00:24:24.968 1122 echo "START TEST $test_name" 00:32:37.025 00:24:24.968 1123 echo "************************************" 00:32:37.025 00:24:24.968 1124 xtrace_restore 00:32:37.025 00:24:24.968 1125 time "$@" 00:32:37.025 00:24:24.968 1126 xtrace_disable 00:32:37.025 00:24:24.968 1127 echo "************************************" 00:32:37.025 00:24:24.968 1128 echo "END TEST $test_name" 00:32:37.025 00:24:24.968 1129 echo "************************************" 00:32:37.025 00:24:24.968 1130 timing_exit $test_name 00:32:37.025 00:24:24.968 ... 00:32:37.025 00:24:24.968 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/autotest.sh:280 -> main(["/var/jenkins/workspace/nvmf-tcp-phy-autotest/autorun-spdk.conf"]) 00:32:37.025 00:24:24.968 ... 00:32:37.025 00:24:24.968 275 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:32:37.025 00:24:24.968 276 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:32:37.025 00:24:24.968 277 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.025 00:24:24.968 278 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.025 00:24:24.968 279 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:32:37.025 00:24:24.968 => 280 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.025 00:24:24.968 281 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.025 00:24:24.968 282 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.025 00:24:24.968 283 run_test "nvmf_identify_passthru" $rootdir/test/nvmf/target/identify_passthru.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.025 00:24:24.968 284 fi 00:32:37.025 00:24:24.968 285 run_test "nvmf_dif" $rootdir/test/nvmf/target/dif.sh 00:32:37.025 00:24:24.968 ... 00:32:37.025 00:24:24.968 00:32:37.025 00:24:24.968 ========== Backtrace end ========== ]] 00:32:37.025 + echo '00:24:23.360 ========== Backtrace start: ========== 00:32:37.025 00:24:23.360 00:32:37.025 00:24:23.360 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/host/failover.sh:68 -> main(["--transport=tcp"]) 00:32:37.025 00:24:23.360 ... 00:32:37.025 00:24:23.360 63 cat $testdir/try.txt 00:32:37.025 00:24:23.360 64 # if this test fails it means we didn'\''t fail over to the second 00:32:37.025 00:24:23.360 65 count="$(grep -c "Resetting controller successful" < $testdir/try.txt)" 00:32:37.025 00:24:23.360 66 00:32:37.025 00:24:23.360 67 if ((count != 3)); then 00:32:37.025 00:24:23.360 => 68 false 00:32:37.025 00:24:23.360 69 fi 00:32:37.025 00:24:23.360 70 00:32:37.025 00:24:23.360 71 # Part 2 of the test. Start removing ports, starting with the one we are connected to, confirm that the ctrlr remains active until the final trid is removed. 00:32:37.025 00:24:23.360 72 $rootdir/build/examples/bdevperf -z -r $bdevperf_rpc_sock -q 128 -o 4096 -w verify -t 1 -f &> $testdir/try.txt & 00:32:37.025 00:24:23.360 73 bdevperf_pid=$! 00:32:37.025 00:24:23.360 ... 00:32:37.025 00:24:23.360 00:32:37.025 00:24:23.360 ========== Backtrace end ========== 00:32:37.025 00:24:24.967 ========== Backtrace start: ========== 00:32:37.025 00:24:24.967 00:32:37.025 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_failover"],["/var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/host/failover.sh"],["--transport=tcp"]) 00:32:37.025 00:24:24.967 ... 00:32:37.025 00:24:24.967 1120 timing_enter $test_name 00:32:37.025 00:24:24.967 1121 echo "************************************" 00:32:37.025 00:24:24.967 1122 echo "START TEST $test_name" 00:32:37.025 00:24:24.967 1123 echo "************************************" 00:32:37.025 00:24:24.967 1124 xtrace_restore 00:32:37.025 00:24:24.967 1125 time "$@" 00:32:37.025 00:24:24.967 1126 xtrace_disable 00:32:37.025 00:24:24.967 1127 echo "************************************" 00:32:37.025 00:24:24.967 1128 echo "END TEST $test_name" 00:32:37.025 00:24:24.967 1129 echo "************************************" 00:32:37.025 00:24:24.967 1130 timing_exit $test_name 00:32:37.025 00:24:24.967 ... 00:32:37.026 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/nvmf_host.sh:25 -> main(["--transport=tcp"]) 00:32:37.026 00:24:24.967 ... 00:32:37.026 00:24:24.967 20 fi 00:32:37.026 00:24:24.967 21 00:32:37.026 00:24:24.967 22 run_test "nvmf_identify" $rootdir/test/nvmf/host/identify.sh "${TEST_ARGS[@]}" 00:32:37.026 00:24:24.967 23 run_test "nvmf_perf" $rootdir/test/nvmf/host/perf.sh "${TEST_ARGS[@]}" 00:32:37.026 00:24:24.967 24 run_test "nvmf_fio_host" $rootdir/test/nvmf/host/fio.sh "${TEST_ARGS[@]}" 00:32:37.026 00:24:24.967 => 25 run_test "nvmf_failover" $rootdir/test/nvmf/host/failover.sh "${TEST_ARGS[@]}" 00:32:37.026 00:24:24.967 26 run_test "nvmf_host_discovery" $rootdir/test/nvmf/host/discovery.sh "${TEST_ARGS[@]}" 00:32:37.026 00:24:24.967 27 run_test "nvmf_host_multipath_status" $rootdir/test/nvmf/host/multipath_status.sh "${TEST_ARGS[@]}" 00:32:37.026 00:24:24.967 28 run_test "nvmf_discovery_remove_ifc" $rootdir/test/nvmf/host/discovery_remove_ifc.sh "${TEST_ARGS[@]}" 00:32:37.026 00:24:24.967 29 run_test "nvmf_identify_kernel_target" "$rootdir/test/nvmf/host/identify_kernel_nvmf.sh" "${TEST_ARGS[@]}" 00:32:37.026 00:24:24.967 30 run_test "nvmf_auth_host" "$rootdir/test/nvmf/host/auth.sh" "${TEST_ARGS[@]}" 00:32:37.026 00:24:24.967 ... 00:32:37.026 00:24:24.967 00:32:37.026 00:24:24.967 ========== Backtrace end ========== 00:32:37.026 00:24:24.967 ========== Backtrace start: ========== 00:32:37.026 00:24:24.967 00:32:37.026 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_host"],["/var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/nvmf_host.sh"],["--transport=tcp"]) 00:32:37.026 00:24:24.967 ... 00:32:37.026 00:24:24.967 1120 timing_enter $test_name 00:32:37.026 00:24:24.967 1121 echo "************************************" 00:32:37.026 00:24:24.967 1122 echo "START TEST $test_name" 00:32:37.026 00:24:24.967 1123 echo "************************************" 00:32:37.026 00:24:24.967 1124 xtrace_restore 00:32:37.026 00:24:24.967 1125 time "$@" 00:32:37.026 00:24:24.967 1126 xtrace_disable 00:32:37.026 00:24:24.967 1127 echo "************************************" 00:32:37.026 00:24:24.967 1128 echo "END TEST $test_name" 00:32:37.026 00:24:24.967 1129 echo "************************************" 00:32:37.026 00:24:24.967 1130 timing_exit $test_name 00:32:37.026 00:24:24.967 ... 00:32:37.026 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/nvmf.sh:16 -> main(["--transport=tcp"]) 00:32:37.026 00:24:24.967 ... 00:32:37.026 00:24:24.967 11 exit 0 00:32:37.026 00:24:24.967 12 fi 00:32:37.026 00:24:24.967 13 00:32:37.026 00:24:24.967 14 run_test "nvmf_target_core" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.026 00:24:24.967 15 run_test "nvmf_target_extra" $rootdir/test/nvmf/nvmf_target_extra.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.026 00:24:24.967 => 16 run_test "nvmf_host" $rootdir/test/nvmf/nvmf_host.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.026 00:24:24.967 17 00:32:37.026 00:24:24.967 18 # Interrupt mode for now is supported only on the target, with the TCP transport and posix or ssl socket implementations. 00:32:37.026 00:24:24.967 19 if [[ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" && $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.026 00:24:24.967 20 run_test "nvmf_target_core_interrupt_mode" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.026 00:24:24.967 21 run_test "nvmf_interrupt" $rootdir/test/nvmf/target/interrupt.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.026 00:24:24.967 ... 00:32:37.026 00:24:24.967 00:32:37.026 00:24:24.967 ========== Backtrace end ========== 00:32:37.026 00:24:24.967 ========== Backtrace start: ========== 00:32:37.026 00:24:24.967 00:32:37.026 00:24:24.967 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_tcp"],["/var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/test/nvmf/nvmf.sh"],["--transport=tcp"]) 00:32:37.026 00:24:24.967 ... 00:32:37.026 00:24:24.968 1120 timing_enter $test_name 00:32:37.026 00:24:24.968 1121 echo "************************************" 00:32:37.026 00:24:24.968 1122 echo "START TEST $test_name" 00:32:37.026 00:24:24.968 1123 echo "************************************" 00:32:37.026 00:24:24.968 1124 xtrace_restore 00:32:37.026 00:24:24.968 1125 time "$@" 00:32:37.026 00:24:24.968 1126 xtrace_disable 00:32:37.026 00:24:24.968 1127 echo "************************************" 00:32:37.026 00:24:24.968 1128 echo "END TEST $test_name" 00:32:37.026 00:24:24.968 1129 echo "************************************" 00:32:37.026 00:24:24.968 1130 timing_exit $test_name 00:32:37.026 00:24:24.968 ... 00:32:37.026 00:24:24.968 in /var/jenkins/workspace/nvmf-tcp-phy-autotest/spdk/autotest.sh:280 -> main(["/var/jenkins/workspace/nvmf-tcp-phy-autotest/autorun-spdk.conf"]) 00:32:37.026 00:24:24.968 ... 00:32:37.026 00:24:24.968 275 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:32:37.026 00:24:24.968 276 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:32:37.026 00:24:24.968 277 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.026 00:24:24.968 278 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.026 00:24:24.968 279 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:32:37.026 00:24:24.968 => 280 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.026 00:24:24.968 281 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.026 00:24:24.968 282 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.026 00:24:24.968 283 run_test "nvmf_identify_passthru" $rootdir/test/nvmf/target/identify_passthru.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.026 00:24:24.968 284 fi 00:32:37.026 00:24:24.968 285 run_test "nvmf_dif" $rootdir/test/nvmf/target/dif.sh 00:32:37.026 00:24:24.968 ... 00:32:37.026 00:24:24.968 00:32:37.026 00:24:24.968 ========== Backtrace end ==========' 00:32:37.026 + for build in "${build_log_list[@]}" 00:32:37.026 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/nvme-cmb-pmr-vg-autotest_91797/build.log 00:32:37.026 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.026 + build_log=/var/jenkins/workspace/autotest-per-patch_3/nvme-cmb-pmr-vg-autotest_91797/build.log 00:32:37.026 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvme-cmb-pmr-vg-autotest_91797/build.log 00:32:37.026 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvme-cmb-pmr-vg-autotest_91797 00:32:37.026 + job_name=nvme-cmb-pmr-vg-autotest_91797 00:32:37.026 + start_pattern='Backtrace start' 00:32:37.026 + end_pattern='Backtrace end' 00:32:37.026 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/nvme-cmb-pmr-vg-autotest_91797/build.log ]] 00:32:37.026 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/nvme-cmb-pmr-vg-autotest_91797/build.log 00:32:37.026 + backtraces= 00:32:37.026 + [[ -n '' ]] 00:32:37.026 + for build in "${build_log_list[@]}" 00:32:37.026 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/vhost-initiator-vg-autotest_90686/build.log 00:32:37.026 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.026 + build_log=/var/jenkins/workspace/autotest-per-patch_3/vhost-initiator-vg-autotest_90686/build.log 00:32:37.026 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/vhost-initiator-vg-autotest_90686/build.log 00:32:37.026 ++ basename /var/jenkins/workspace/autotest-per-patch_3/vhost-initiator-vg-autotest_90686 00:32:37.026 + job_name=vhost-initiator-vg-autotest_90686 00:32:37.026 + start_pattern='Backtrace start' 00:32:37.026 + end_pattern='Backtrace end' 00:32:37.026 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/vhost-initiator-vg-autotest_90686/build.log ]] 00:32:37.026 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/vhost-initiator-vg-autotest_90686/build.log 00:32:37.026 + backtraces= 00:32:37.026 + [[ -n '' ]] 00:32:37.026 + for build in "${build_log_list[@]}" 00:32:37.026 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/short-fuzz-phy-autotest_38301/build.log 00:32:37.026 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.026 + build_log=/var/jenkins/workspace/autotest-per-patch_3/short-fuzz-phy-autotest_38301/build.log 00:32:37.026 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/short-fuzz-phy-autotest_38301/build.log 00:32:37.026 ++ basename /var/jenkins/workspace/autotest-per-patch_3/short-fuzz-phy-autotest_38301 00:32:37.026 + job_name=short-fuzz-phy-autotest_38301 00:32:37.026 + start_pattern='Backtrace start' 00:32:37.026 + end_pattern='Backtrace end' 00:32:37.026 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/short-fuzz-phy-autotest_38301/build.log ]] 00:32:37.026 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/short-fuzz-phy-autotest_38301/build.log 00:32:37.026 + backtraces= 00:32:37.026 + [[ -n '' ]] 00:32:37.026 + for build in "${build_log_list[@]}" 00:32:37.026 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/packaging-docker-autotest_29941/build.log 00:32:37.026 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.026 + build_log=/var/jenkins/workspace/autotest-per-patch_3/packaging-docker-autotest_29941/build.log 00:32:37.026 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/packaging-docker-autotest_29941/build.log 00:32:37.026 ++ basename /var/jenkins/workspace/autotest-per-patch_3/packaging-docker-autotest_29941 00:32:37.026 + job_name=packaging-docker-autotest_29941 00:32:37.026 + start_pattern='Backtrace start' 00:32:37.026 + end_pattern='Backtrace end' 00:32:37.027 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/packaging-docker-autotest_29941/build.log ]] 00:32:37.027 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/packaging-docker-autotest_29941/build.log 00:32:37.027 + backtraces= 00:32:37.027 + [[ -n '' ]] 00:32:37.027 + for build in "${build_log_list[@]}" 00:32:37.027 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/vfio-user-phy-autotest_38306/build.log 00:32:37.027 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.027 + build_log=/var/jenkins/workspace/autotest-per-patch_3/vfio-user-phy-autotest_38306/build.log 00:32:37.027 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/vfio-user-phy-autotest_38306/build.log 00:32:37.027 ++ basename /var/jenkins/workspace/autotest-per-patch_3/vfio-user-phy-autotest_38306 00:32:37.027 + job_name=vfio-user-phy-autotest_38306 00:32:37.027 + start_pattern='Backtrace start' 00:32:37.027 + end_pattern='Backtrace end' 00:32:37.027 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/vfio-user-phy-autotest_38306/build.log ]] 00:32:37.027 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/vfio-user-phy-autotest_38306/build.log 00:32:37.027 + backtraces= 00:32:37.027 + [[ -n '' ]] 00:32:37.027 + for build in "${build_log_list[@]}" 00:32:37.027 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/check-so-deps-docker-autotest_33415/build.log 00:32:37.027 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.027 + build_log=/var/jenkins/workspace/autotest-per-patch_3/check-so-deps-docker-autotest_33415/build.log 00:32:37.027 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/check-so-deps-docker-autotest_33415/build.log 00:32:37.027 ++ basename /var/jenkins/workspace/autotest-per-patch_3/check-so-deps-docker-autotest_33415 00:32:37.027 + job_name=check-so-deps-docker-autotest_33415 00:32:37.027 + start_pattern='Backtrace start' 00:32:37.027 + end_pattern='Backtrace end' 00:32:37.027 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/check-so-deps-docker-autotest_33415/build.log ]] 00:32:37.027 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/check-so-deps-docker-autotest_33415/build.log 00:32:37.027 + backtraces= 00:32:37.027 + [[ -n '' ]] 00:32:37.027 + for build in "${build_log_list[@]}" 00:32:37.027 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/unittest-clang-docker-autotest_29942/build.log 00:32:37.027 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.027 + build_log=/var/jenkins/workspace/autotest-per-patch_3/unittest-clang-docker-autotest_29942/build.log 00:32:37.027 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/unittest-clang-docker-autotest_29942/build.log 00:32:37.027 ++ basename /var/jenkins/workspace/autotest-per-patch_3/unittest-clang-docker-autotest_29942 00:32:37.027 + job_name=unittest-clang-docker-autotest_29942 00:32:37.027 + start_pattern='Backtrace start' 00:32:37.027 + end_pattern='Backtrace end' 00:32:37.027 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/unittest-clang-docker-autotest_29942/build.log ]] 00:32:37.027 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/unittest-clang-docker-autotest_29942/build.log 00:32:37.027 + backtraces= 00:32:37.027 + [[ -n '' ]] 00:32:37.027 + for build in "${build_log_list[@]}" 00:32:37.027 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/BlobFS-phy-autotest_38316/build.log 00:32:37.027 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.027 + build_log=/var/jenkins/workspace/autotest-per-patch_3/BlobFS-phy-autotest_38316/build.log 00:32:37.027 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/BlobFS-phy-autotest_38316/build.log 00:32:37.027 ++ basename /var/jenkins/workspace/autotest-per-patch_3/BlobFS-phy-autotest_38316 00:32:37.027 + job_name=BlobFS-phy-autotest_38316 00:32:37.027 + start_pattern='Backtrace start' 00:32:37.027 + end_pattern='Backtrace end' 00:32:37.027 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/BlobFS-phy-autotest_38316/build.log ]] 00:32:37.027 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/BlobFS-phy-autotest_38316/build.log 00:32:37.027 + backtraces= 00:32:37.027 + [[ -n '' ]] 00:32:37.027 + for build in "${build_log_list[@]}" 00:32:37.027 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/unittest-main-docker-autotest_29944/build.log 00:32:37.027 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.027 + build_log=/var/jenkins/workspace/autotest-per-patch_3/unittest-main-docker-autotest_29944/build.log 00:32:37.027 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/unittest-main-docker-autotest_29944/build.log 00:32:37.027 ++ basename /var/jenkins/workspace/autotest-per-patch_3/unittest-main-docker-autotest_29944 00:32:37.027 + job_name=unittest-main-docker-autotest_29944 00:32:37.027 + start_pattern='Backtrace start' 00:32:37.027 + end_pattern='Backtrace end' 00:32:37.027 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/unittest-main-docker-autotest_29944/build.log ]] 00:32:37.027 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/unittest-main-docker-autotest_29944/build.log 00:32:37.027 + backtraces= 00:32:37.027 + [[ -n '' ]] 00:32:37.027 + for build in "${build_log_list[@]}" 00:32:37.027 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/ubuntu24-vg-autotest_7099/build.log 00:32:37.027 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.027 + build_log=/var/jenkins/workspace/autotest-per-patch_3/ubuntu24-vg-autotest_7099/build.log 00:32:37.027 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/ubuntu24-vg-autotest_7099/build.log 00:32:37.027 ++ basename /var/jenkins/workspace/autotest-per-patch_3/ubuntu24-vg-autotest_7099 00:32:37.027 + job_name=ubuntu24-vg-autotest_7099 00:32:37.027 + start_pattern='Backtrace start' 00:32:37.027 + end_pattern='Backtrace end' 00:32:37.027 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/ubuntu24-vg-autotest_7099/build.log ]] 00:32:37.027 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/ubuntu24-vg-autotest_7099/build.log 00:32:37.027 + backtraces= 00:32:37.027 + [[ -n '' ]] 00:32:37.027 + for build in "${build_log_list[@]}" 00:32:37.027 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/autobuild-external-vg-autotest_30579/build.log 00:32:37.027 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.027 + build_log=/var/jenkins/workspace/autotest-per-patch_3/autobuild-external-vg-autotest_30579/build.log 00:32:37.027 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/autobuild-external-vg-autotest_30579/build.log 00:32:37.027 ++ basename /var/jenkins/workspace/autotest-per-patch_3/autobuild-external-vg-autotest_30579 00:32:37.027 + job_name=autobuild-external-vg-autotest_30579 00:32:37.027 + start_pattern='Backtrace start' 00:32:37.027 + end_pattern='Backtrace end' 00:32:37.027 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/autobuild-external-vg-autotest_30579/build.log ]] 00:32:37.027 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/autobuild-external-vg-autotest_30579/build.log 00:32:37.027 + backtraces= 00:32:37.027 + [[ -n '' ]] 00:32:37.027 + for build in "${build_log_list[@]}" 00:32:37.027 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/rocky9-vg-autotest_27871/build.log 00:32:37.027 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.027 + build_log=/var/jenkins/workspace/autotest-per-patch_3/rocky9-vg-autotest_27871/build.log 00:32:37.027 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/rocky9-vg-autotest_27871/build.log 00:32:37.027 ++ basename /var/jenkins/workspace/autotest-per-patch_3/rocky9-vg-autotest_27871 00:32:37.027 + job_name=rocky9-vg-autotest_27871 00:32:37.027 + start_pattern='Backtrace start' 00:32:37.027 + end_pattern='Backtrace end' 00:32:37.027 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/rocky9-vg-autotest_27871/build.log ]] 00:32:37.027 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/rocky9-vg-autotest_27871/build.log 00:32:37.027 + backtraces= 00:32:37.027 + [[ -n '' ]] 00:32:37.027 + for build in "${build_log_list[@]}" 00:32:37.027 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/freebsd-vg-autotest_68599/build.log 00:32:37.027 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.027 + build_log=/var/jenkins/workspace/autotest-per-patch_3/freebsd-vg-autotest_68599/build.log 00:32:37.027 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/freebsd-vg-autotest_68599/build.log 00:32:37.027 ++ basename /var/jenkins/workspace/autotest-per-patch_3/freebsd-vg-autotest_68599 00:32:37.027 + job_name=freebsd-vg-autotest_68599 00:32:37.027 + start_pattern='Backtrace start' 00:32:37.027 + end_pattern='Backtrace end' 00:32:37.027 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/freebsd-vg-autotest_68599/build.log ]] 00:32:37.027 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/freebsd-vg-autotest_68599/build.log 00:32:37.027 + backtraces= 00:32:37.027 + [[ -n '' ]] 00:32:37.027 + for build in "${build_log_list[@]}" 00:32:37.027 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/ubuntu22-vg-autotest_49538/build.log 00:32:37.027 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.027 + build_log=/var/jenkins/workspace/autotest-per-patch_3/ubuntu22-vg-autotest_49538/build.log 00:32:37.027 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/ubuntu22-vg-autotest_49538/build.log 00:32:37.027 ++ basename /var/jenkins/workspace/autotest-per-patch_3/ubuntu22-vg-autotest_49538 00:32:37.027 + job_name=ubuntu22-vg-autotest_49538 00:32:37.027 + start_pattern='Backtrace start' 00:32:37.027 + end_pattern='Backtrace end' 00:32:37.027 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/ubuntu22-vg-autotest_49538/build.log ]] 00:32:37.027 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/ubuntu22-vg-autotest_49538/build.log 00:32:37.027 + backtraces= 00:32:37.027 + [[ -n '' ]] 00:32:37.027 + for build in "${build_log_list[@]}" 00:32:37.027 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/scan-build-docker-autotest_29948/build.log 00:32:37.027 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.027 + build_log=/var/jenkins/workspace/autotest-per-patch_3/scan-build-docker-autotest_29948/build.log 00:32:37.027 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/scan-build-docker-autotest_29948/build.log 00:32:37.027 ++ basename /var/jenkins/workspace/autotest-per-patch_3/scan-build-docker-autotest_29948 00:32:37.027 + job_name=scan-build-docker-autotest_29948 00:32:37.027 + start_pattern='Backtrace start' 00:32:37.027 + end_pattern='Backtrace end' 00:32:37.027 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/scan-build-docker-autotest_29948/build.log ]] 00:32:37.027 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/scan-build-docker-autotest_29948/build.log 00:32:37.027 + backtraces= 00:32:37.027 + [[ -n '' ]] 00:32:37.027 + for build in "${build_log_list[@]}" 00:32:37.027 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/zns-vg-autotest_89770/build.log 00:32:37.027 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.027 + build_log=/var/jenkins/workspace/autotest-per-patch_3/zns-vg-autotest_89770/build.log 00:32:37.027 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/zns-vg-autotest_89770/build.log 00:32:37.027 ++ basename /var/jenkins/workspace/autotest-per-patch_3/zns-vg-autotest_89770 00:32:37.027 + job_name=zns-vg-autotest_89770 00:32:37.027 + start_pattern='Backtrace start' 00:32:37.027 + end_pattern='Backtrace end' 00:32:37.027 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/zns-vg-autotest_89770/build.log ]] 00:32:37.027 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/zns-vg-autotest_89770/build.log 00:32:37.027 + backtraces= 00:32:37.027 + [[ -n '' ]] 00:32:37.028 + for build in "${build_log_list[@]}" 00:32:37.028 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/nvme-phy-autotest_84479/build.log 00:32:37.028 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.028 + build_log=/var/jenkins/workspace/autotest-per-patch_3/nvme-phy-autotest_84479/build.log 00:32:37.028 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvme-phy-autotest_84479/build.log 00:32:37.028 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvme-phy-autotest_84479 00:32:37.028 + job_name=nvme-phy-autotest_84479 00:32:37.028 + start_pattern='Backtrace start' 00:32:37.028 + end_pattern='Backtrace end' 00:32:37.028 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/nvme-phy-autotest_84479/build.log ]] 00:32:37.028 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/nvme-phy-autotest_84479/build.log 00:32:37.028 + backtraces= 00:32:37.028 + [[ -n '' ]] 00:32:37.028 + for build in "${build_log_list[@]}" 00:32:37.028 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/doc-docker-autotest_33497/build.log 00:32:37.028 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.028 + build_log=/var/jenkins/workspace/autotest-per-patch_3/doc-docker-autotest_33497/build.log 00:32:37.028 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/doc-docker-autotest_33497/build.log 00:32:37.028 ++ basename /var/jenkins/workspace/autotest-per-patch_3/doc-docker-autotest_33497 00:32:37.028 + job_name=doc-docker-autotest_33497 00:32:37.028 + start_pattern='Backtrace start' 00:32:37.028 + end_pattern='Backtrace end' 00:32:37.028 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/doc-docker-autotest_33497/build.log ]] 00:32:37.028 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/doc-docker-autotest_33497/build.log 00:32:37.028 + backtraces= 00:32:37.028 + [[ -n '' ]] 00:32:37.028 + for build in "${build_log_list[@]}" 00:32:37.028 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/unittest-valgrind-docker-autotest_29946/build.log 00:32:37.028 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.028 + build_log=/var/jenkins/workspace/autotest-per-patch_3/unittest-valgrind-docker-autotest_29946/build.log 00:32:37.028 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/unittest-valgrind-docker-autotest_29946/build.log 00:32:37.028 ++ basename /var/jenkins/workspace/autotest-per-patch_3/unittest-valgrind-docker-autotest_29946 00:32:37.028 + job_name=unittest-valgrind-docker-autotest_29946 00:32:37.028 + start_pattern='Backtrace start' 00:32:37.028 + end_pattern='Backtrace end' 00:32:37.028 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/unittest-valgrind-docker-autotest_29946/build.log ]] 00:32:37.028 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/unittest-valgrind-docker-autotest_29946/build.log 00:32:37.028 + backtraces= 00:32:37.028 + [[ -n '' ]] 00:32:37.028 + for build in "${build_log_list[@]}" 00:32:37.028 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-vg-autotest_90650/build.log 00:32:37.028 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.028 + build_log=/var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-vg-autotest_90650/build.log 00:32:37.028 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-vg-autotest_90650/build.log 00:32:37.028 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-vg-autotest_90650 00:32:37.028 + job_name=nvmf-tcp-vg-autotest_90650 00:32:37.028 + start_pattern='Backtrace start' 00:32:37.028 + end_pattern='Backtrace end' 00:32:37.028 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-vg-autotest_90650/build.log ]] 00:32:37.028 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-vg-autotest_90650/build.log 00:32:37.028 + backtraces='00:22:37.830 ========== Backtrace start: ========== 00:32:37.028 00:22:37.830 00:32:37.028 00:22:37.830 in /home/vagrant/spdk_repo/spdk/test/nvmf/host/failover.sh:68 -> main(["--transport=tcp"]) 00:32:37.028 00:22:37.830 ... 00:32:37.028 00:22:37.830 63 cat $testdir/try.txt 00:32:37.028 00:22:37.830 64 # if this test fails it means we didn'\''t fail over to the second 00:32:37.028 00:22:37.830 65 count="$(grep -c "Resetting controller successful" < $testdir/try.txt)" 00:32:37.028 00:22:37.830 66 00:32:37.028 00:22:37.830 67 if ((count != 3)); then 00:32:37.028 00:22:37.830 => 68 false 00:32:37.028 00:22:37.830 69 fi 00:32:37.028 00:22:37.830 70 00:32:37.028 00:22:37.830 71 # Part 2 of the test. Start removing ports, starting with the one we are connected to, confirm that the ctrlr remains active until the final trid is removed. 00:32:37.028 00:22:37.830 72 $rootdir/build/examples/bdevperf -z -r $bdevperf_rpc_sock -q 128 -o 4096 -w verify -t 1 -f &> $testdir/try.txt & 00:32:37.028 00:22:37.830 73 bdevperf_pid=$! 00:32:37.028 00:22:37.830 ... 00:32:37.028 00:22:37.830 00:32:37.028 00:22:37.830 ========== Backtrace end ========== 00:32:37.028 00:22:38.013 ========== Backtrace start: ========== 00:32:37.028 00:22:38.013 00:32:37.028 00:22:38.013 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_failover"],["/home/vagrant/spdk_repo/spdk/test/nvmf/host/failover.sh"],["--transport=tcp"]) 00:32:37.028 00:22:38.013 ... 00:32:37.028 00:22:38.013 1120 timing_enter $test_name 00:32:37.028 00:22:38.013 1121 echo "************************************" 00:32:37.028 00:22:38.013 1122 echo "START TEST $test_name" 00:32:37.028 00:22:38.013 1123 echo "************************************" 00:32:37.028 00:22:38.013 1124 xtrace_restore 00:32:37.028 00:22:38.013 1125 time "$@" 00:32:37.028 00:22:38.013 1126 xtrace_disable 00:32:37.028 00:22:38.013 1127 echo "************************************" 00:32:37.028 00:22:38.013 1128 echo "END TEST $test_name" 00:32:37.028 00:22:38.013 1129 echo "************************************" 00:32:37.028 00:22:38.013 1130 timing_exit $test_name 00:32:37.028 00:22:38.013 ... 00:32:37.028 00:22:38.013 in /home/vagrant/spdk_repo/spdk/test/nvmf/nvmf_host.sh:25 -> main(["--transport=tcp"]) 00:32:37.028 00:22:38.013 ... 00:32:37.028 00:22:38.013 20 fi 00:32:37.028 00:22:38.013 21 00:32:37.028 00:22:38.013 22 run_test "nvmf_identify" $rootdir/test/nvmf/host/identify.sh "${TEST_ARGS[@]}" 00:32:37.028 00:22:38.013 23 run_test "nvmf_perf" $rootdir/test/nvmf/host/perf.sh "${TEST_ARGS[@]}" 00:32:37.028 00:22:38.013 24 run_test "nvmf_fio_host" $rootdir/test/nvmf/host/fio.sh "${TEST_ARGS[@]}" 00:32:37.028 00:22:38.013 => 25 run_test "nvmf_failover" $rootdir/test/nvmf/host/failover.sh "${TEST_ARGS[@]}" 00:32:37.028 00:22:38.013 26 run_test "nvmf_host_discovery" $rootdir/test/nvmf/host/discovery.sh "${TEST_ARGS[@]}" 00:32:37.028 00:22:38.013 27 run_test "nvmf_host_multipath_status" $rootdir/test/nvmf/host/multipath_status.sh "${TEST_ARGS[@]}" 00:32:37.028 00:22:38.013 28 run_test "nvmf_discovery_remove_ifc" $rootdir/test/nvmf/host/discovery_remove_ifc.sh "${TEST_ARGS[@]}" 00:32:37.028 00:22:38.013 29 run_test "nvmf_identify_kernel_target" "$rootdir/test/nvmf/host/identify_kernel_nvmf.sh" "${TEST_ARGS[@]}" 00:32:37.028 00:22:38.013 30 run_test "nvmf_auth_host" "$rootdir/test/nvmf/host/auth.sh" "${TEST_ARGS[@]}" 00:32:37.028 00:22:38.013 ... 00:32:37.028 00:22:38.013 00:32:37.028 00:22:38.013 ========== Backtrace end ========== 00:32:37.028 00:22:38.013 ========== Backtrace start: ========== 00:32:37.028 00:22:38.013 00:32:37.028 00:22:38.013 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_host"],["/home/vagrant/spdk_repo/spdk/test/nvmf/nvmf_host.sh"],["--transport=tcp"]) 00:32:37.028 00:22:38.013 ... 00:32:37.028 00:22:38.013 1120 timing_enter $test_name 00:32:37.028 00:22:38.013 1121 echo "************************************" 00:32:37.028 00:22:38.013 1122 echo "START TEST $test_name" 00:32:37.028 00:22:38.013 1123 echo "************************************" 00:32:37.028 00:22:38.013 1124 xtrace_restore 00:32:37.028 00:22:38.013 1125 time "$@" 00:32:37.028 00:22:38.013 1126 xtrace_disable 00:32:37.028 00:22:38.013 1127 echo "************************************" 00:32:37.028 00:22:38.013 1128 echo "END TEST $test_name" 00:32:37.028 00:22:38.013 1129 echo "************************************" 00:32:37.028 00:22:38.013 1130 timing_exit $test_name 00:32:37.028 00:22:38.013 ... 00:32:37.028 00:22:38.272 in /home/vagrant/spdk_repo/spdk/test/nvmf/nvmf.sh:16 -> main(["--transport=tcp"]) 00:32:37.028 00:22:38.272 ... 00:32:37.028 00:22:38.272 11 exit 0 00:32:37.028 00:22:38.272 12 fi 00:32:37.028 00:22:38.272 13 00:32:37.028 00:22:38.272 14 run_test "nvmf_target_core" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.028 00:22:38.272 15 run_test "nvmf_target_extra" $rootdir/test/nvmf/nvmf_target_extra.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.028 00:22:38.272 => 16 run_test "nvmf_host" $rootdir/test/nvmf/nvmf_host.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.028 00:22:38.272 17 00:32:37.028 00:22:38.272 18 # Interrupt mode for now is supported only on the target, with the TCP transport and posix or ssl socket implementations. 00:32:37.028 00:22:38.272 19 if [[ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" && $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.028 00:22:38.272 20 run_test "nvmf_target_core_interrupt_mode" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.028 00:22:38.272 21 run_test "nvmf_interrupt" $rootdir/test/nvmf/target/interrupt.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.028 00:22:38.272 ... 00:32:37.028 00:22:38.272 00:32:37.028 00:22:38.272 ========== Backtrace end ========== 00:32:37.028 00:22:38.272 ========== Backtrace start: ========== 00:32:37.028 00:22:38.272 00:32:37.028 00:22:38.272 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_tcp"],["/home/vagrant/spdk_repo/spdk/test/nvmf/nvmf.sh"],["--transport=tcp"]) 00:32:37.028 00:22:38.272 ... 00:32:37.028 00:22:38.272 1120 timing_enter $test_name 00:32:37.028 00:22:38.272 1121 echo "************************************" 00:32:37.028 00:22:38.272 1122 echo "START TEST $test_name" 00:32:37.028 00:22:38.272 1123 echo "************************************" 00:32:37.028 00:22:38.272 1124 xtrace_restore 00:32:37.028 00:22:38.272 1125 time "$@" 00:32:37.028 00:22:38.272 1126 xtrace_disable 00:32:37.028 00:22:38.272 1127 echo "************************************" 00:32:37.028 00:22:38.272 1128 echo "END TEST $test_name" 00:32:37.028 00:22:38.272 1129 echo "************************************" 00:32:37.028 00:22:38.272 1130 timing_exit $test_name 00:32:37.028 00:22:38.272 ... 00:32:37.028 00:22:38.272 in /home/vagrant/spdk_repo/spdk/autotest.sh:280 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:32:37.028 00:22:38.272 ... 00:32:37.028 00:22:38.272 275 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:32:37.028 00:22:38.272 276 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:32:37.028 00:22:38.272 277 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.028 00:22:38.272 278 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.028 00:22:38.272 279 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:32:37.028 00:22:38.272 => 280 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.028 00:22:38.272 281 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.028 00:22:38.272 282 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.029 00:22:38.272 283 run_test "nvmf_identify_passthru" $rootdir/test/nvmf/target/identify_passthru.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.029 00:22:38.272 284 fi 00:32:37.029 00:22:38.272 285 run_test "nvmf_dif" $rootdir/test/nvmf/target/dif.sh 00:32:37.029 00:22:38.272 ... 00:32:37.029 00:22:38.272 00:32:37.029 00:22:38.272 ========== Backtrace end ==========' 00:32:37.029 + [[ -n 00:22:37.830 ========== Backtrace start: ========== 00:32:37.029 00:22:37.830 00:32:37.029 00:22:37.830 in /home/vagrant/spdk_repo/spdk/test/nvmf/host/failover.sh:68 -> main(["--transport=tcp"]) 00:32:37.029 00:22:37.830 ... 00:32:37.029 00:22:37.830 63 cat $testdir/try.txt 00:32:37.029 00:22:37.830 64 # if this test fails it means we didn't fail over to the second 00:32:37.029 00:22:37.830 65 count="$(grep -c "Resetting controller successful" < $testdir/try.txt)" 00:32:37.029 00:22:37.830 66 00:32:37.029 00:22:37.830 67 if ((count != 3)); then 00:32:37.029 00:22:37.830 => 68 false 00:32:37.029 00:22:37.830 69 fi 00:32:37.029 00:22:37.830 70 00:32:37.029 00:22:37.830 71 # Part 2 of the test. Start removing ports, starting with the one we are connected to, confirm that the ctrlr remains active until the final trid is removed. 00:32:37.029 00:22:37.830 72 $rootdir/build/examples/bdevperf -z -r $bdevperf_rpc_sock -q 128 -o 4096 -w verify -t 1 -f &> $testdir/try.txt & 00:32:37.029 00:22:37.830 73 bdevperf_pid=$! 00:32:37.029 00:22:37.830 ... 00:32:37.029 00:22:37.830 00:32:37.029 00:22:37.830 ========== Backtrace end ========== 00:32:37.029 00:22:38.013 ========== Backtrace start: ========== 00:32:37.029 00:22:38.013 00:32:37.029 00:22:38.013 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_failover"],["/home/vagrant/spdk_repo/spdk/test/nvmf/host/failover.sh"],["--transport=tcp"]) 00:32:37.029 00:22:38.013 ... 00:32:37.029 00:22:38.013 1120 timing_enter $test_name 00:32:37.029 00:22:38.013 1121 echo "************************************" 00:32:37.029 00:22:38.013 1122 echo "START TEST $test_name" 00:32:37.029 00:22:38.013 1123 echo "************************************" 00:32:37.029 00:22:38.013 1124 xtrace_restore 00:32:37.029 00:22:38.013 1125 time "$@" 00:32:37.029 00:22:38.013 1126 xtrace_disable 00:32:37.029 00:22:38.013 1127 echo "************************************" 00:32:37.029 00:22:38.013 1128 echo "END TEST $test_name" 00:32:37.029 00:22:38.013 1129 echo "************************************" 00:32:37.029 00:22:38.013 1130 timing_exit $test_name 00:32:37.029 00:22:38.013 ... 00:32:37.029 00:22:38.013 in /home/vagrant/spdk_repo/spdk/test/nvmf/nvmf_host.sh:25 -> main(["--transport=tcp"]) 00:32:37.029 00:22:38.013 ... 00:32:37.029 00:22:38.013 20 fi 00:32:37.029 00:22:38.013 21 00:32:37.029 00:22:38.013 22 run_test "nvmf_identify" $rootdir/test/nvmf/host/identify.sh "${TEST_ARGS[@]}" 00:32:37.029 00:22:38.013 23 run_test "nvmf_perf" $rootdir/test/nvmf/host/perf.sh "${TEST_ARGS[@]}" 00:32:37.029 00:22:38.013 24 run_test "nvmf_fio_host" $rootdir/test/nvmf/host/fio.sh "${TEST_ARGS[@]}" 00:32:37.029 00:22:38.013 => 25 run_test "nvmf_failover" $rootdir/test/nvmf/host/failover.sh "${TEST_ARGS[@]}" 00:32:37.029 00:22:38.013 26 run_test "nvmf_host_discovery" $rootdir/test/nvmf/host/discovery.sh "${TEST_ARGS[@]}" 00:32:37.029 00:22:38.013 27 run_test "nvmf_host_multipath_status" $rootdir/test/nvmf/host/multipath_status.sh "${TEST_ARGS[@]}" 00:32:37.029 00:22:38.013 28 run_test "nvmf_discovery_remove_ifc" $rootdir/test/nvmf/host/discovery_remove_ifc.sh "${TEST_ARGS[@]}" 00:32:37.029 00:22:38.013 29 run_test "nvmf_identify_kernel_target" "$rootdir/test/nvmf/host/identify_kernel_nvmf.sh" "${TEST_ARGS[@]}" 00:32:37.029 00:22:38.013 30 run_test "nvmf_auth_host" "$rootdir/test/nvmf/host/auth.sh" "${TEST_ARGS[@]}" 00:32:37.029 00:22:38.013 ... 00:32:37.029 00:22:38.013 00:32:37.029 00:22:38.013 ========== Backtrace end ========== 00:32:37.029 00:22:38.013 ========== Backtrace start: ========== 00:32:37.029 00:22:38.013 00:32:37.029 00:22:38.013 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_host"],["/home/vagrant/spdk_repo/spdk/test/nvmf/nvmf_host.sh"],["--transport=tcp"]) 00:32:37.029 00:22:38.013 ... 00:32:37.029 00:22:38.013 1120 timing_enter $test_name 00:32:37.029 00:22:38.013 1121 echo "************************************" 00:32:37.029 00:22:38.013 1122 echo "START TEST $test_name" 00:32:37.029 00:22:38.013 1123 echo "************************************" 00:32:37.029 00:22:38.013 1124 xtrace_restore 00:32:37.029 00:22:38.013 1125 time "$@" 00:32:37.029 00:22:38.013 1126 xtrace_disable 00:32:37.029 00:22:38.013 1127 echo "************************************" 00:32:37.029 00:22:38.013 1128 echo "END TEST $test_name" 00:32:37.029 00:22:38.013 1129 echo "************************************" 00:32:37.029 00:22:38.013 1130 timing_exit $test_name 00:32:37.029 00:22:38.013 ... 00:32:37.029 00:22:38.272 in /home/vagrant/spdk_repo/spdk/test/nvmf/nvmf.sh:16 -> main(["--transport=tcp"]) 00:32:37.029 00:22:38.272 ... 00:32:37.029 00:22:38.272 11 exit 0 00:32:37.029 00:22:38.272 12 fi 00:32:37.029 00:22:38.272 13 00:32:37.029 00:22:38.272 14 run_test "nvmf_target_core" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.029 00:22:38.272 15 run_test "nvmf_target_extra" $rootdir/test/nvmf/nvmf_target_extra.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.029 00:22:38.272 => 16 run_test "nvmf_host" $rootdir/test/nvmf/nvmf_host.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.029 00:22:38.272 17 00:32:37.029 00:22:38.272 18 # Interrupt mode for now is supported only on the target, with the TCP transport and posix or ssl socket implementations. 00:32:37.029 00:22:38.272 19 if [[ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" && $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.029 00:22:38.272 20 run_test "nvmf_target_core_interrupt_mode" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.029 00:22:38.272 21 run_test "nvmf_interrupt" $rootdir/test/nvmf/target/interrupt.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.029 00:22:38.272 ... 00:32:37.029 00:22:38.272 00:32:37.029 00:22:38.272 ========== Backtrace end ========== 00:32:37.029 00:22:38.272 ========== Backtrace start: ========== 00:32:37.029 00:22:38.272 00:32:37.029 00:22:38.272 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_tcp"],["/home/vagrant/spdk_repo/spdk/test/nvmf/nvmf.sh"],["--transport=tcp"]) 00:32:37.029 00:22:38.272 ... 00:32:37.029 00:22:38.272 1120 timing_enter $test_name 00:32:37.029 00:22:38.272 1121 echo "************************************" 00:32:37.029 00:22:38.272 1122 echo "START TEST $test_name" 00:32:37.029 00:22:38.272 1123 echo "************************************" 00:32:37.029 00:22:38.272 1124 xtrace_restore 00:32:37.029 00:22:38.272 1125 time "$@" 00:32:37.029 00:22:38.272 1126 xtrace_disable 00:32:37.029 00:22:38.272 1127 echo "************************************" 00:32:37.029 00:22:38.272 1128 echo "END TEST $test_name" 00:32:37.029 00:22:38.272 1129 echo "************************************" 00:32:37.029 00:22:38.272 1130 timing_exit $test_name 00:32:37.029 00:22:38.272 ... 00:32:37.029 00:22:38.272 in /home/vagrant/spdk_repo/spdk/autotest.sh:280 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:32:37.029 00:22:38.272 ... 00:32:37.029 00:22:38.272 275 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:32:37.029 00:22:38.272 276 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:32:37.029 00:22:38.272 277 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.029 00:22:38.272 278 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.029 00:22:38.272 279 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:32:37.029 00:22:38.272 => 280 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.029 00:22:38.272 281 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.029 00:22:38.272 282 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.029 00:22:38.272 283 run_test "nvmf_identify_passthru" $rootdir/test/nvmf/target/identify_passthru.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.029 00:22:38.272 284 fi 00:32:37.029 00:22:38.272 285 run_test "nvmf_dif" $rootdir/test/nvmf/target/dif.sh 00:32:37.029 00:22:38.272 ... 00:32:37.029 00:22:38.272 00:32:37.029 00:22:38.272 ========== Backtrace end ========== ]] 00:32:37.029 + echo '00:22:37.830 ========== Backtrace start: ========== 00:32:37.029 00:22:37.830 00:32:37.029 00:22:37.830 in /home/vagrant/spdk_repo/spdk/test/nvmf/host/failover.sh:68 -> main(["--transport=tcp"]) 00:32:37.029 00:22:37.830 ... 00:32:37.029 00:22:37.830 63 cat $testdir/try.txt 00:32:37.029 00:22:37.830 64 # if this test fails it means we didn'\''t fail over to the second 00:32:37.029 00:22:37.830 65 count="$(grep -c "Resetting controller successful" < $testdir/try.txt)" 00:32:37.029 00:22:37.830 66 00:32:37.029 00:22:37.830 67 if ((count != 3)); then 00:32:37.029 00:22:37.830 => 68 false 00:32:37.029 00:22:37.830 69 fi 00:32:37.029 00:22:37.830 70 00:32:37.029 00:22:37.830 71 # Part 2 of the test. Start removing ports, starting with the one we are connected to, confirm that the ctrlr remains active until the final trid is removed. 00:32:37.029 00:22:37.830 72 $rootdir/build/examples/bdevperf -z -r $bdevperf_rpc_sock -q 128 -o 4096 -w verify -t 1 -f &> $testdir/try.txt & 00:32:37.029 00:22:37.830 73 bdevperf_pid=$! 00:32:37.029 00:22:37.830 ... 00:32:37.029 00:22:37.830 00:32:37.029 00:22:37.830 ========== Backtrace end ========== 00:32:37.029 00:22:38.013 ========== Backtrace start: ========== 00:32:37.029 00:22:38.013 00:32:37.029 00:22:38.013 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_failover"],["/home/vagrant/spdk_repo/spdk/test/nvmf/host/failover.sh"],["--transport=tcp"]) 00:32:37.029 00:22:38.013 ... 00:32:37.029 00:22:38.013 1120 timing_enter $test_name 00:32:37.029 00:22:38.013 1121 echo "************************************" 00:32:37.029 00:22:38.013 1122 echo "START TEST $test_name" 00:32:37.029 00:22:38.013 1123 echo "************************************" 00:32:37.029 00:22:38.013 1124 xtrace_restore 00:32:37.029 00:22:38.013 1125 time "$@" 00:32:37.029 00:22:38.013 1126 xtrace_disable 00:32:37.029 00:22:38.013 1127 echo "************************************" 00:32:37.029 00:22:38.013 1128 echo "END TEST $test_name" 00:32:37.029 00:22:38.013 1129 echo "************************************" 00:32:37.029 00:22:38.013 1130 timing_exit $test_name 00:32:37.029 00:22:38.013 ... 00:32:37.029 00:22:38.013 in /home/vagrant/spdk_repo/spdk/test/nvmf/nvmf_host.sh:25 -> main(["--transport=tcp"]) 00:32:37.029 00:22:38.013 ... 00:32:37.029 00:22:38.013 20 fi 00:32:37.029 00:22:38.013 21 00:32:37.029 00:22:38.013 22 run_test "nvmf_identify" $rootdir/test/nvmf/host/identify.sh "${TEST_ARGS[@]}" 00:32:37.029 00:22:38.013 23 run_test "nvmf_perf" $rootdir/test/nvmf/host/perf.sh "${TEST_ARGS[@]}" 00:32:37.029 00:22:38.013 24 run_test "nvmf_fio_host" $rootdir/test/nvmf/host/fio.sh "${TEST_ARGS[@]}" 00:32:37.029 00:22:38.013 => 25 run_test "nvmf_failover" $rootdir/test/nvmf/host/failover.sh "${TEST_ARGS[@]}" 00:32:37.029 00:22:38.013 26 run_test "nvmf_host_discovery" $rootdir/test/nvmf/host/discovery.sh "${TEST_ARGS[@]}" 00:32:37.030 00:22:38.013 27 run_test "nvmf_host_multipath_status" $rootdir/test/nvmf/host/multipath_status.sh "${TEST_ARGS[@]}" 00:32:37.030 00:22:38.013 28 run_test "nvmf_discovery_remove_ifc" $rootdir/test/nvmf/host/discovery_remove_ifc.sh "${TEST_ARGS[@]}" 00:32:37.030 00:22:38.013 29 run_test "nvmf_identify_kernel_target" "$rootdir/test/nvmf/host/identify_kernel_nvmf.sh" "${TEST_ARGS[@]}" 00:32:37.030 00:22:38.013 30 run_test "nvmf_auth_host" "$rootdir/test/nvmf/host/auth.sh" "${TEST_ARGS[@]}" 00:32:37.030 00:22:38.013 ... 00:32:37.030 00:22:38.013 00:32:37.030 00:22:38.013 ========== Backtrace end ========== 00:32:37.030 00:22:38.013 ========== Backtrace start: ========== 00:32:37.030 00:22:38.013 00:32:37.030 00:22:38.013 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_host"],["/home/vagrant/spdk_repo/spdk/test/nvmf/nvmf_host.sh"],["--transport=tcp"]) 00:32:37.030 00:22:38.013 ... 00:32:37.030 00:22:38.013 1120 timing_enter $test_name 00:32:37.030 00:22:38.013 1121 echo "************************************" 00:32:37.030 00:22:38.013 1122 echo "START TEST $test_name" 00:32:37.030 00:22:38.013 1123 echo "************************************" 00:32:37.030 00:22:38.013 1124 xtrace_restore 00:32:37.030 00:22:38.013 1125 time "$@" 00:32:37.030 00:22:38.013 1126 xtrace_disable 00:32:37.030 00:22:38.013 1127 echo "************************************" 00:32:37.030 00:22:38.013 1128 echo "END TEST $test_name" 00:32:37.030 00:22:38.013 1129 echo "************************************" 00:32:37.030 00:22:38.013 1130 timing_exit $test_name 00:32:37.030 00:22:38.013 ... 00:32:37.030 00:22:38.272 in /home/vagrant/spdk_repo/spdk/test/nvmf/nvmf.sh:16 -> main(["--transport=tcp"]) 00:32:37.030 00:22:38.272 ... 00:32:37.030 00:22:38.272 11 exit 0 00:32:37.030 00:22:38.272 12 fi 00:32:37.030 00:22:38.272 13 00:32:37.030 00:22:38.272 14 run_test "nvmf_target_core" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.030 00:22:38.272 15 run_test "nvmf_target_extra" $rootdir/test/nvmf/nvmf_target_extra.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.030 00:22:38.272 => 16 run_test "nvmf_host" $rootdir/test/nvmf/nvmf_host.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.030 00:22:38.272 17 00:32:37.030 00:22:38.272 18 # Interrupt mode for now is supported only on the target, with the TCP transport and posix or ssl socket implementations. 00:32:37.030 00:22:38.272 19 if [[ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" && $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.030 00:22:38.272 20 run_test "nvmf_target_core_interrupt_mode" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.030 00:22:38.272 21 run_test "nvmf_interrupt" $rootdir/test/nvmf/target/interrupt.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.030 00:22:38.272 ... 00:32:37.030 00:22:38.272 00:32:37.030 00:22:38.272 ========== Backtrace end ========== 00:32:37.030 00:22:38.272 ========== Backtrace start: ========== 00:32:37.030 00:22:38.272 00:32:37.030 00:22:38.272 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_tcp"],["/home/vagrant/spdk_repo/spdk/test/nvmf/nvmf.sh"],["--transport=tcp"]) 00:32:37.030 00:22:38.272 ... 00:32:37.030 00:22:38.272 1120 timing_enter $test_name 00:32:37.030 00:22:38.272 1121 echo "************************************" 00:32:37.030 00:22:38.272 1122 echo "START TEST $test_name" 00:32:37.030 00:22:38.272 1123 echo "************************************" 00:32:37.030 00:22:38.272 1124 xtrace_restore 00:32:37.030 00:22:38.272 1125 time "$@" 00:32:37.030 00:22:38.272 1126 xtrace_disable 00:32:37.030 00:22:38.272 1127 echo "************************************" 00:32:37.030 00:22:38.272 1128 echo "END TEST $test_name" 00:32:37.030 00:22:38.272 1129 echo "************************************" 00:32:37.030 00:22:38.272 1130 timing_exit $test_name 00:32:37.030 00:22:38.272 ... 00:32:37.030 00:22:38.272 in /home/vagrant/spdk_repo/spdk/autotest.sh:280 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:32:37.030 00:22:38.272 ... 00:32:37.030 00:22:38.272 275 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:32:37.030 00:22:38.272 276 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:32:37.030 00:22:38.272 277 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.030 00:22:38.272 278 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.030 00:22:38.272 279 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:32:37.030 00:22:38.272 => 280 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.030 00:22:38.272 281 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.030 00:22:38.272 282 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.030 00:22:38.272 283 run_test "nvmf_identify_passthru" $rootdir/test/nvmf/target/identify_passthru.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.030 00:22:38.272 284 fi 00:32:37.030 00:22:38.272 285 run_test "nvmf_dif" $rootdir/test/nvmf/target/dif.sh 00:32:37.030 00:22:38.272 ... 00:32:37.030 00:22:38.272 00:32:37.030 00:22:38.272 ========== Backtrace end ==========' 00:32:37.030 + for build in "${build_log_list[@]}" 00:32:37.030 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/release-build-main-docker-autotest_3119/build.log 00:32:37.030 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.030 + build_log=/var/jenkins/workspace/autotest-per-patch_3/release-build-main-docker-autotest_3119/build.log 00:32:37.030 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/release-build-main-docker-autotest_3119/build.log 00:32:37.030 ++ basename /var/jenkins/workspace/autotest-per-patch_3/release-build-main-docker-autotest_3119 00:32:37.030 + job_name=release-build-main-docker-autotest_3119 00:32:37.030 + start_pattern='Backtrace start' 00:32:37.030 + end_pattern='Backtrace end' 00:32:37.030 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/release-build-main-docker-autotest_3119/build.log ]] 00:32:37.030 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/release-build-main-docker-autotest_3119/build.log 00:32:37.030 + backtraces= 00:32:37.030 + [[ -n '' ]] 00:32:37.030 + for build in "${build_log_list[@]}" 00:32:37.030 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/release-build-clang-docker-autotest_3119/build.log 00:32:37.030 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.030 + build_log=/var/jenkins/workspace/autotest-per-patch_3/release-build-clang-docker-autotest_3119/build.log 00:32:37.030 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/release-build-clang-docker-autotest_3119/build.log 00:32:37.030 ++ basename /var/jenkins/workspace/autotest-per-patch_3/release-build-clang-docker-autotest_3119 00:32:37.030 + job_name=release-build-clang-docker-autotest_3119 00:32:37.030 + start_pattern='Backtrace start' 00:32:37.030 + end_pattern='Backtrace end' 00:32:37.030 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/release-build-clang-docker-autotest_3119/build.log ]] 00:32:37.030 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/release-build-clang-docker-autotest_3119/build.log 00:32:37.030 + backtraces= 00:32:37.030 + [[ -n '' ]] 00:32:37.030 + for build in "${build_log_list[@]}" 00:32:37.030 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/build-files-docker-autotest_33168/build.log 00:32:37.030 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.030 + build_log=/var/jenkins/workspace/autotest-per-patch_3/build-files-docker-autotest_33168/build.log 00:32:37.030 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/build-files-docker-autotest_33168/build.log 00:32:37.030 ++ basename /var/jenkins/workspace/autotest-per-patch_3/build-files-docker-autotest_33168 00:32:37.030 + job_name=build-files-docker-autotest_33168 00:32:37.030 + start_pattern='Backtrace start' 00:32:37.030 + end_pattern='Backtrace end' 00:32:37.030 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/build-files-docker-autotest_33168/build.log ]] 00:32:37.030 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/build-files-docker-autotest_33168/build.log 00:32:37.030 + backtraces= 00:32:37.030 + [[ -n '' ]] 00:32:37.030 + for build in "${build_log_list[@]}" 00:32:37.030 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/crypto-phy-autotest_38305/build.log 00:32:37.030 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.030 + build_log=/var/jenkins/workspace/autotest-per-patch_3/crypto-phy-autotest_38305/build.log 00:32:37.030 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/crypto-phy-autotest_38305/build.log 00:32:37.030 ++ basename /var/jenkins/workspace/autotest-per-patch_3/crypto-phy-autotest_38305 00:32:37.030 + job_name=crypto-phy-autotest_38305 00:32:37.030 + start_pattern='Backtrace start' 00:32:37.030 + end_pattern='Backtrace end' 00:32:37.030 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/crypto-phy-autotest_38305/build.log ]] 00:32:37.030 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/crypto-phy-autotest_38305/build.log 00:32:37.030 + backtraces= 00:32:37.030 + [[ -n '' ]] 00:32:37.030 + for build in "${build_log_list[@]}" 00:32:37.030 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/nvmf-phy-autotest_84548/build.log 00:32:37.030 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.030 + build_log=/var/jenkins/workspace/autotest-per-patch_3/nvmf-phy-autotest_84548/build.log 00:32:37.030 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvmf-phy-autotest_84548/build.log 00:32:37.030 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvmf-phy-autotest_84548 00:32:37.030 + job_name=nvmf-phy-autotest_84548 00:32:37.030 + start_pattern='Backtrace start' 00:32:37.030 + end_pattern='Backtrace end' 00:32:37.030 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/nvmf-phy-autotest_84548/build.log ]] 00:32:37.030 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/nvmf-phy-autotest_84548/build.log 00:32:37.030 + backtraces='00:23:03.678 ========== Backtrace start: ========== 00:32:37.030 00:23:03.678 00:32:37.030 00:23:03.678 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/host/failover.sh:68 -> main(["--transport=rdma"]) 00:32:37.030 00:23:03.678 ... 00:32:37.030 00:23:03.678 63 cat $testdir/try.txt 00:32:37.030 00:23:03.678 64 # if this test fails it means we didn'\''t fail over to the second 00:32:37.030 00:23:03.678 65 count="$(grep -c "Resetting controller successful" < $testdir/try.txt)" 00:32:37.031 00:23:03.678 66 00:32:37.031 00:23:03.678 67 if ((count != 3)); then 00:32:37.031 00:23:03.678 => 68 false 00:32:37.031 00:23:03.678 69 fi 00:32:37.031 00:23:03.678 70 00:32:37.031 00:23:03.678 71 # Part 2 of the test. Start removing ports, starting with the one we are connected to, confirm that the ctrlr remains active until the final trid is removed. 00:32:37.031 00:23:03.678 72 $rootdir/build/examples/bdevperf -z -r $bdevperf_rpc_sock -q 128 -o 4096 -w verify -t 1 -f &> $testdir/try.txt & 00:32:37.031 00:23:03.678 73 bdevperf_pid=$! 00:32:37.031 00:23:03.678 ... 00:32:37.031 00:23:03.678 00:32:37.031 00:23:03.678 ========== Backtrace end ========== 00:32:37.031 00:23:03.948 ========== Backtrace start: ========== 00:32:37.031 00:23:03.948 00:32:37.031 00:23:03.948 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_failover"],["/var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/host/failover.sh"],["--transport=rdma"]) 00:32:37.031 00:23:03.948 ... 00:32:37.031 00:23:03.948 1120 timing_enter $test_name 00:32:37.031 00:23:03.948 1121 echo "************************************" 00:32:37.031 00:23:03.948 1122 echo "START TEST $test_name" 00:32:37.031 00:23:03.948 1123 echo "************************************" 00:32:37.031 00:23:03.948 1124 xtrace_restore 00:32:37.031 00:23:03.948 1125 time "$@" 00:32:37.031 00:23:03.948 1126 xtrace_disable 00:32:37.031 00:23:03.948 1127 echo "************************************" 00:32:37.031 00:23:03.948 1128 echo "END TEST $test_name" 00:32:37.031 00:23:03.948 1129 echo "************************************" 00:32:37.031 00:23:03.948 1130 timing_exit $test_name 00:32:37.031 00:23:03.948 ... 00:32:37.031 00:23:03.948 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf_host.sh:25 -> main(["--transport=rdma"]) 00:32:37.031 00:23:03.948 ... 00:32:37.031 00:23:03.948 20 fi 00:32:37.031 00:23:03.948 21 00:32:37.031 00:23:03.948 22 run_test "nvmf_identify" $rootdir/test/nvmf/host/identify.sh "${TEST_ARGS[@]}" 00:32:37.031 00:23:03.948 23 run_test "nvmf_perf" $rootdir/test/nvmf/host/perf.sh "${TEST_ARGS[@]}" 00:32:37.031 00:23:03.948 24 run_test "nvmf_fio_host" $rootdir/test/nvmf/host/fio.sh "${TEST_ARGS[@]}" 00:32:37.031 00:23:03.948 => 25 run_test "nvmf_failover" $rootdir/test/nvmf/host/failover.sh "${TEST_ARGS[@]}" 00:32:37.031 00:23:03.948 26 run_test "nvmf_host_discovery" $rootdir/test/nvmf/host/discovery.sh "${TEST_ARGS[@]}" 00:32:37.031 00:23:03.948 27 run_test "nvmf_host_multipath_status" $rootdir/test/nvmf/host/multipath_status.sh "${TEST_ARGS[@]}" 00:32:37.031 00:23:03.948 28 run_test "nvmf_discovery_remove_ifc" $rootdir/test/nvmf/host/discovery_remove_ifc.sh "${TEST_ARGS[@]}" 00:32:37.031 00:23:03.948 29 run_test "nvmf_identify_kernel_target" "$rootdir/test/nvmf/host/identify_kernel_nvmf.sh" "${TEST_ARGS[@]}" 00:32:37.031 00:23:03.948 30 run_test "nvmf_auth_host" "$rootdir/test/nvmf/host/auth.sh" "${TEST_ARGS[@]}" 00:32:37.031 00:23:03.948 ... 00:32:37.031 00:23:03.948 00:32:37.031 00:23:03.948 ========== Backtrace end ========== 00:32:37.031 00:23:03.948 ========== Backtrace start: ========== 00:32:37.031 00:23:03.948 00:32:37.031 00:23:03.948 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_host"],["/var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf_host.sh"],["--transport=rdma"]) 00:32:37.031 00:23:03.948 ... 00:32:37.031 00:23:03.948 1120 timing_enter $test_name 00:32:37.031 00:23:03.948 1121 echo "************************************" 00:32:37.031 00:23:03.948 1122 echo "START TEST $test_name" 00:32:37.031 00:23:03.948 1123 echo "************************************" 00:32:37.031 00:23:03.948 1124 xtrace_restore 00:32:37.031 00:23:03.948 1125 time "$@" 00:32:37.031 00:23:03.948 1126 xtrace_disable 00:32:37.031 00:23:03.948 1127 echo "************************************" 00:32:37.031 00:23:03.948 1128 echo "END TEST $test_name" 00:32:37.031 00:23:03.948 1129 echo "************************************" 00:32:37.031 00:23:03.948 1130 timing_exit $test_name 00:32:37.031 00:23:03.948 ... 00:32:37.031 00:23:03.948 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf.sh:16 -> main(["--transport=rdma"]) 00:32:37.031 00:23:03.948 ... 00:32:37.031 00:23:03.948 11 exit 0 00:32:37.031 00:23:03.948 12 fi 00:32:37.031 00:23:03.948 13 00:32:37.031 00:23:03.948 14 run_test "nvmf_target_core" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.031 00:23:03.948 15 run_test "nvmf_target_extra" $rootdir/test/nvmf/nvmf_target_extra.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.031 00:23:03.948 => 16 run_test "nvmf_host" $rootdir/test/nvmf/nvmf_host.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.031 00:23:03.948 17 00:32:37.031 00:23:03.948 18 # Interrupt mode for now is supported only on the target, with the TCP transport and posix or ssl socket implementations. 00:32:37.031 00:23:03.948 19 if [[ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" && $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.031 00:23:03.949 20 run_test "nvmf_target_core_interrupt_mode" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.031 00:23:03.949 21 run_test "nvmf_interrupt" $rootdir/test/nvmf/target/interrupt.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.031 00:23:03.949 ... 00:32:37.031 00:23:03.949 00:32:37.031 00:23:03.949 ========== Backtrace end ========== 00:32:37.031 00:23:03.949 ========== Backtrace start: ========== 00:32:37.031 00:23:03.949 00:32:37.031 00:23:03.949 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_rdma"],["/var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf.sh"],["--transport=rdma"]) 00:32:37.031 00:23:03.949 ... 00:32:37.031 00:23:03.949 1120 timing_enter $test_name 00:32:37.031 00:23:03.949 1121 echo "************************************" 00:32:37.031 00:23:03.949 1122 echo "START TEST $test_name" 00:32:37.031 00:23:03.949 1123 echo "************************************" 00:32:37.031 00:23:03.949 1124 xtrace_restore 00:32:37.031 00:23:03.949 1125 time "$@" 00:32:37.031 00:23:03.949 1126 xtrace_disable 00:32:37.031 00:23:03.949 1127 echo "************************************" 00:32:37.031 00:23:03.949 1128 echo "END TEST $test_name" 00:32:37.031 00:23:03.949 1129 echo "************************************" 00:32:37.031 00:23:03.949 1130 timing_exit $test_name 00:32:37.031 00:23:03.949 ... 00:32:37.031 00:23:03.949 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/autotest.sh:277 -> main(["/var/jenkins/workspace/nvmf-phy-autotest/autorun-spdk.conf"]) 00:32:37.031 00:23:03.949 ... 00:32:37.031 00:23:03.949 272 if [ $SPDK_TEST_NVMF -eq 1 ]; then 00:32:37.031 00:23:03.949 273 export NET_TYPE 00:32:37.031 00:23:03.949 274 # The NVMe-oF run test cases are split out like this so that the parser that compiles the 00:32:37.031 00:23:03.949 275 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:32:37.031 00:23:03.949 276 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:32:37.031 00:23:03.949 => 277 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.031 00:23:03.949 278 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.031 00:23:03.949 279 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:32:37.031 00:23:03.949 280 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.031 00:23:03.949 281 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.031 00:23:03.949 282 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.031 00:23:03.949 ... 00:32:37.031 00:23:03.949 00:32:37.031 00:23:03.949 ========== Backtrace end ==========' 00:32:37.031 + [[ -n 00:23:03.678 ========== Backtrace start: ========== 00:32:37.031 00:23:03.678 00:32:37.031 00:23:03.678 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/host/failover.sh:68 -> main(["--transport=rdma"]) 00:32:37.031 00:23:03.678 ... 00:32:37.031 00:23:03.678 63 cat $testdir/try.txt 00:32:37.031 00:23:03.678 64 # if this test fails it means we didn't fail over to the second 00:32:37.031 00:23:03.678 65 count="$(grep -c "Resetting controller successful" < $testdir/try.txt)" 00:32:37.031 00:23:03.678 66 00:32:37.031 00:23:03.678 67 if ((count != 3)); then 00:32:37.031 00:23:03.678 => 68 false 00:32:37.031 00:23:03.678 69 fi 00:32:37.031 00:23:03.678 70 00:32:37.031 00:23:03.678 71 # Part 2 of the test. Start removing ports, starting with the one we are connected to, confirm that the ctrlr remains active until the final trid is removed. 00:32:37.031 00:23:03.678 72 $rootdir/build/examples/bdevperf -z -r $bdevperf_rpc_sock -q 128 -o 4096 -w verify -t 1 -f &> $testdir/try.txt & 00:32:37.031 00:23:03.678 73 bdevperf_pid=$! 00:32:37.031 00:23:03.678 ... 00:32:37.031 00:23:03.678 00:32:37.031 00:23:03.678 ========== Backtrace end ========== 00:32:37.031 00:23:03.948 ========== Backtrace start: ========== 00:32:37.031 00:23:03.948 00:32:37.031 00:23:03.948 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_failover"],["/var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/host/failover.sh"],["--transport=rdma"]) 00:32:37.031 00:23:03.948 ... 00:32:37.031 00:23:03.948 1120 timing_enter $test_name 00:32:37.031 00:23:03.948 1121 echo "************************************" 00:32:37.031 00:23:03.948 1122 echo "START TEST $test_name" 00:32:37.031 00:23:03.948 1123 echo "************************************" 00:32:37.031 00:23:03.948 1124 xtrace_restore 00:32:37.031 00:23:03.948 1125 time "$@" 00:32:37.031 00:23:03.948 1126 xtrace_disable 00:32:37.031 00:23:03.948 1127 echo "************************************" 00:32:37.031 00:23:03.948 1128 echo "END TEST $test_name" 00:32:37.031 00:23:03.948 1129 echo "************************************" 00:32:37.031 00:23:03.948 1130 timing_exit $test_name 00:32:37.031 00:23:03.948 ... 00:32:37.031 00:23:03.948 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf_host.sh:25 -> main(["--transport=rdma"]) 00:32:37.031 00:23:03.948 ... 00:32:37.031 00:23:03.948 20 fi 00:32:37.031 00:23:03.948 21 00:32:37.031 00:23:03.948 22 run_test "nvmf_identify" $rootdir/test/nvmf/host/identify.sh "${TEST_ARGS[@]}" 00:32:37.031 00:23:03.948 23 run_test "nvmf_perf" $rootdir/test/nvmf/host/perf.sh "${TEST_ARGS[@]}" 00:32:37.031 00:23:03.948 24 run_test "nvmf_fio_host" $rootdir/test/nvmf/host/fio.sh "${TEST_ARGS[@]}" 00:32:37.031 00:23:03.948 => 25 run_test "nvmf_failover" $rootdir/test/nvmf/host/failover.sh "${TEST_ARGS[@]}" 00:32:37.031 00:23:03.948 26 run_test "nvmf_host_discovery" $rootdir/test/nvmf/host/discovery.sh "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 27 run_test "nvmf_host_multipath_status" $rootdir/test/nvmf/host/multipath_status.sh "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 28 run_test "nvmf_discovery_remove_ifc" $rootdir/test/nvmf/host/discovery_remove_ifc.sh "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 29 run_test "nvmf_identify_kernel_target" "$rootdir/test/nvmf/host/identify_kernel_nvmf.sh" "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 30 run_test "nvmf_auth_host" "$rootdir/test/nvmf/host/auth.sh" "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 ... 00:32:37.032 00:23:03.948 00:32:37.032 00:23:03.948 ========== Backtrace end ========== 00:32:37.032 00:23:03.948 ========== Backtrace start: ========== 00:32:37.032 00:23:03.948 00:32:37.032 00:23:03.948 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_host"],["/var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf_host.sh"],["--transport=rdma"]) 00:32:37.032 00:23:03.948 ... 00:32:37.032 00:23:03.948 1120 timing_enter $test_name 00:32:37.032 00:23:03.948 1121 echo "************************************" 00:32:37.032 00:23:03.948 1122 echo "START TEST $test_name" 00:32:37.032 00:23:03.948 1123 echo "************************************" 00:32:37.032 00:23:03.948 1124 xtrace_restore 00:32:37.032 00:23:03.948 1125 time "$@" 00:32:37.032 00:23:03.948 1126 xtrace_disable 00:32:37.032 00:23:03.948 1127 echo "************************************" 00:32:37.032 00:23:03.948 1128 echo "END TEST $test_name" 00:32:37.032 00:23:03.948 1129 echo "************************************" 00:32:37.032 00:23:03.948 1130 timing_exit $test_name 00:32:37.032 00:23:03.948 ... 00:32:37.032 00:23:03.948 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf.sh:16 -> main(["--transport=rdma"]) 00:32:37.032 00:23:03.948 ... 00:32:37.032 00:23:03.948 11 exit 0 00:32:37.032 00:23:03.948 12 fi 00:32:37.032 00:23:03.948 13 00:32:37.032 00:23:03.948 14 run_test "nvmf_target_core" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.032 00:23:03.948 15 run_test "nvmf_target_extra" $rootdir/test/nvmf/nvmf_target_extra.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.032 00:23:03.948 => 16 run_test "nvmf_host" $rootdir/test/nvmf/nvmf_host.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.032 00:23:03.948 17 00:32:37.032 00:23:03.948 18 # Interrupt mode for now is supported only on the target, with the TCP transport and posix or ssl socket implementations. 00:32:37.032 00:23:03.948 19 if [[ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" && $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.032 00:23:03.949 20 run_test "nvmf_target_core_interrupt_mode" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.032 00:23:03.949 21 run_test "nvmf_interrupt" $rootdir/test/nvmf/target/interrupt.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.032 00:23:03.949 ... 00:32:37.032 00:23:03.949 00:32:37.032 00:23:03.949 ========== Backtrace end ========== 00:32:37.032 00:23:03.949 ========== Backtrace start: ========== 00:32:37.032 00:23:03.949 00:32:37.032 00:23:03.949 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_rdma"],["/var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf.sh"],["--transport=rdma"]) 00:32:37.032 00:23:03.949 ... 00:32:37.032 00:23:03.949 1120 timing_enter $test_name 00:32:37.032 00:23:03.949 1121 echo "************************************" 00:32:37.032 00:23:03.949 1122 echo "START TEST $test_name" 00:32:37.032 00:23:03.949 1123 echo "************************************" 00:32:37.032 00:23:03.949 1124 xtrace_restore 00:32:37.032 00:23:03.949 1125 time "$@" 00:32:37.032 00:23:03.949 1126 xtrace_disable 00:32:37.032 00:23:03.949 1127 echo "************************************" 00:32:37.032 00:23:03.949 1128 echo "END TEST $test_name" 00:32:37.032 00:23:03.949 1129 echo "************************************" 00:32:37.032 00:23:03.949 1130 timing_exit $test_name 00:32:37.032 00:23:03.949 ... 00:32:37.032 00:23:03.949 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/autotest.sh:277 -> main(["/var/jenkins/workspace/nvmf-phy-autotest/autorun-spdk.conf"]) 00:32:37.032 00:23:03.949 ... 00:32:37.032 00:23:03.949 272 if [ $SPDK_TEST_NVMF -eq 1 ]; then 00:32:37.032 00:23:03.949 273 export NET_TYPE 00:32:37.032 00:23:03.949 274 # The NVMe-oF run test cases are split out like this so that the parser that compiles the 00:32:37.032 00:23:03.949 275 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:32:37.032 00:23:03.949 276 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:32:37.032 00:23:03.949 => 277 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.032 00:23:03.949 278 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.032 00:23:03.949 279 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:32:37.032 00:23:03.949 280 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.032 00:23:03.949 281 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.032 00:23:03.949 282 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.032 00:23:03.949 ... 00:32:37.032 00:23:03.949 00:32:37.032 00:23:03.949 ========== Backtrace end ========== ]] 00:32:37.032 + echo '00:23:03.678 ========== Backtrace start: ========== 00:32:37.032 00:23:03.678 00:32:37.032 00:23:03.678 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/host/failover.sh:68 -> main(["--transport=rdma"]) 00:32:37.032 00:23:03.678 ... 00:32:37.032 00:23:03.678 63 cat $testdir/try.txt 00:32:37.032 00:23:03.678 64 # if this test fails it means we didn'\''t fail over to the second 00:32:37.032 00:23:03.678 65 count="$(grep -c "Resetting controller successful" < $testdir/try.txt)" 00:32:37.032 00:23:03.678 66 00:32:37.032 00:23:03.678 67 if ((count != 3)); then 00:32:37.032 00:23:03.678 => 68 false 00:32:37.032 00:23:03.678 69 fi 00:32:37.032 00:23:03.678 70 00:32:37.032 00:23:03.678 71 # Part 2 of the test. Start removing ports, starting with the one we are connected to, confirm that the ctrlr remains active until the final trid is removed. 00:32:37.032 00:23:03.678 72 $rootdir/build/examples/bdevperf -z -r $bdevperf_rpc_sock -q 128 -o 4096 -w verify -t 1 -f &> $testdir/try.txt & 00:32:37.032 00:23:03.678 73 bdevperf_pid=$! 00:32:37.032 00:23:03.678 ... 00:32:37.032 00:23:03.678 00:32:37.032 00:23:03.678 ========== Backtrace end ========== 00:32:37.032 00:23:03.948 ========== Backtrace start: ========== 00:32:37.032 00:23:03.948 00:32:37.032 00:23:03.948 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_failover"],["/var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/host/failover.sh"],["--transport=rdma"]) 00:32:37.032 00:23:03.948 ... 00:32:37.032 00:23:03.948 1120 timing_enter $test_name 00:32:37.032 00:23:03.948 1121 echo "************************************" 00:32:37.032 00:23:03.948 1122 echo "START TEST $test_name" 00:32:37.032 00:23:03.948 1123 echo "************************************" 00:32:37.032 00:23:03.948 1124 xtrace_restore 00:32:37.032 00:23:03.948 1125 time "$@" 00:32:37.032 00:23:03.948 1126 xtrace_disable 00:32:37.032 00:23:03.948 1127 echo "************************************" 00:32:37.032 00:23:03.948 1128 echo "END TEST $test_name" 00:32:37.032 00:23:03.948 1129 echo "************************************" 00:32:37.032 00:23:03.948 1130 timing_exit $test_name 00:32:37.032 00:23:03.948 ... 00:32:37.032 00:23:03.948 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf_host.sh:25 -> main(["--transport=rdma"]) 00:32:37.032 00:23:03.948 ... 00:32:37.032 00:23:03.948 20 fi 00:32:37.032 00:23:03.948 21 00:32:37.032 00:23:03.948 22 run_test "nvmf_identify" $rootdir/test/nvmf/host/identify.sh "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 23 run_test "nvmf_perf" $rootdir/test/nvmf/host/perf.sh "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 24 run_test "nvmf_fio_host" $rootdir/test/nvmf/host/fio.sh "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 => 25 run_test "nvmf_failover" $rootdir/test/nvmf/host/failover.sh "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 26 run_test "nvmf_host_discovery" $rootdir/test/nvmf/host/discovery.sh "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 27 run_test "nvmf_host_multipath_status" $rootdir/test/nvmf/host/multipath_status.sh "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 28 run_test "nvmf_discovery_remove_ifc" $rootdir/test/nvmf/host/discovery_remove_ifc.sh "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 29 run_test "nvmf_identify_kernel_target" "$rootdir/test/nvmf/host/identify_kernel_nvmf.sh" "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 30 run_test "nvmf_auth_host" "$rootdir/test/nvmf/host/auth.sh" "${TEST_ARGS[@]}" 00:32:37.032 00:23:03.948 ... 00:32:37.032 00:23:03.948 00:32:37.032 00:23:03.948 ========== Backtrace end ========== 00:32:37.032 00:23:03.948 ========== Backtrace start: ========== 00:32:37.032 00:23:03.948 00:32:37.032 00:23:03.948 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_host"],["/var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf_host.sh"],["--transport=rdma"]) 00:32:37.032 00:23:03.948 ... 00:32:37.032 00:23:03.948 1120 timing_enter $test_name 00:32:37.032 00:23:03.948 1121 echo "************************************" 00:32:37.032 00:23:03.948 1122 echo "START TEST $test_name" 00:32:37.032 00:23:03.948 1123 echo "************************************" 00:32:37.032 00:23:03.948 1124 xtrace_restore 00:32:37.032 00:23:03.948 1125 time "$@" 00:32:37.032 00:23:03.948 1126 xtrace_disable 00:32:37.032 00:23:03.948 1127 echo "************************************" 00:32:37.032 00:23:03.948 1128 echo "END TEST $test_name" 00:32:37.032 00:23:03.948 1129 echo "************************************" 00:32:37.032 00:23:03.948 1130 timing_exit $test_name 00:32:37.032 00:23:03.948 ... 00:32:37.032 00:23:03.948 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf.sh:16 -> main(["--transport=rdma"]) 00:32:37.032 00:23:03.948 ... 00:32:37.032 00:23:03.948 11 exit 0 00:32:37.032 00:23:03.948 12 fi 00:32:37.032 00:23:03.948 13 00:32:37.032 00:23:03.948 14 run_test "nvmf_target_core" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.032 00:23:03.948 15 run_test "nvmf_target_extra" $rootdir/test/nvmf/nvmf_target_extra.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.032 00:23:03.948 => 16 run_test "nvmf_host" $rootdir/test/nvmf/nvmf_host.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.032 00:23:03.948 17 00:32:37.032 00:23:03.948 18 # Interrupt mode for now is supported only on the target, with the TCP transport and posix or ssl socket implementations. 00:32:37.032 00:23:03.948 19 if [[ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" && $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.032 00:23:03.949 20 run_test "nvmf_target_core_interrupt_mode" $rootdir/test/nvmf/nvmf_target_core.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.032 00:23:03.949 21 run_test "nvmf_interrupt" $rootdir/test/nvmf/target/interrupt.sh --transport=$SPDK_TEST_NVMF_TRANSPORT --interrupt-mode 00:32:37.032 00:23:03.949 ... 00:32:37.032 00:23:03.949 00:32:37.032 00:23:03.949 ========== Backtrace end ========== 00:32:37.033 00:23:03.949 ========== Backtrace start: ========== 00:32:37.033 00:23:03.949 00:32:37.033 00:23:03.949 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1125 -> run_test(["nvmf_rdma"],["/var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf.sh"],["--transport=rdma"]) 00:32:37.033 00:23:03.949 ... 00:32:37.033 00:23:03.949 1120 timing_enter $test_name 00:32:37.033 00:23:03.949 1121 echo "************************************" 00:32:37.033 00:23:03.949 1122 echo "START TEST $test_name" 00:32:37.033 00:23:03.949 1123 echo "************************************" 00:32:37.033 00:23:03.949 1124 xtrace_restore 00:32:37.033 00:23:03.949 1125 time "$@" 00:32:37.033 00:23:03.949 1126 xtrace_disable 00:32:37.033 00:23:03.949 1127 echo "************************************" 00:32:37.033 00:23:03.949 1128 echo "END TEST $test_name" 00:32:37.033 00:23:03.949 1129 echo "************************************" 00:32:37.033 00:23:03.949 1130 timing_exit $test_name 00:32:37.033 00:23:03.949 ... 00:32:37.033 00:23:03.949 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/autotest.sh:277 -> main(["/var/jenkins/workspace/nvmf-phy-autotest/autorun-spdk.conf"]) 00:32:37.033 00:23:03.949 ... 00:32:37.033 00:23:03.949 272 if [ $SPDK_TEST_NVMF -eq 1 ]; then 00:32:37.033 00:23:03.949 273 export NET_TYPE 00:32:37.033 00:23:03.949 274 # The NVMe-oF run test cases are split out like this so that the parser that compiles the 00:32:37.033 00:23:03.949 275 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:32:37.033 00:23:03.949 276 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:32:37.033 00:23:03.949 => 277 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.033 00:23:03.949 278 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.033 00:23:03.949 279 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:32:37.033 00:23:03.949 280 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.033 00:23:03.949 281 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:32:37.033 00:23:03.949 282 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:32:37.033 00:23:03.949 ... 00:32:37.033 00:23:03.949 00:32:37.033 00:23:03.949 ========== Backtrace end ==========' 00:32:37.033 + for build in "${build_log_list[@]}" 00:32:37.033 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_3/iscsi-vg-autotest_90705/build.log 00:32:37.033 + local build_log job_name start_pattern end_pattern backtraces 00:32:37.033 + build_log=/var/jenkins/workspace/autotest-per-patch_3/iscsi-vg-autotest_90705/build.log 00:32:37.033 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/iscsi-vg-autotest_90705/build.log 00:32:37.033 ++ basename /var/jenkins/workspace/autotest-per-patch_3/iscsi-vg-autotest_90705 00:32:37.033 + job_name=iscsi-vg-autotest_90705 00:32:37.033 + start_pattern='Backtrace start' 00:32:37.033 + end_pattern='Backtrace end' 00:32:37.033 + [[ -f /var/jenkins/workspace/autotest-per-patch_3/iscsi-vg-autotest_90705/build.log ]] 00:32:37.033 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_3/iscsi-vg-autotest_90705/build.log 00:32:37.033 + backtraces= 00:32:37.033 + [[ -n '' ]] 00:32:37.033 + run_flamegraph 00:32:37.033 + create_timing_file 00:32:37.033 + local array file 00:32:37.033 + array=($(find $WORKSPACE -type f -name 'timing.txt')) 00:32:37.033 ++ find /var/jenkins/workspace/autotest-per-patch_3 -type f -name timing.txt 00:32:37.291 + timing_dir=/var/jenkins/workspace/autotest-per-patch_3/post_process 00:32:37.291 + mkdir -p /var/jenkins/workspace/autotest-per-patch_3/post_process 00:32:37.291 + timing_file=/var/jenkins/workspace/autotest-per-patch_3/post_process/total_timing.txt 00:32:37.291 + for file in "${array[@]}" 00:32:37.291 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/lvol-vg-autotest_90617/timing.txt 00:32:37.291 + local job_file job_name 00:32:37.291 + job_file=/var/jenkins/workspace/autotest-per-patch_3/lvol-vg-autotest_90617/timing.txt 00:32:37.291 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/lvol-vg-autotest_90617/timing.txt 00:32:37.291 ++ basename /var/jenkins/workspace/autotest-per-patch_3/lvol-vg-autotest_90617 00:32:37.291 + job_name=lvol-vg-autotest_90617 00:32:37.291 + sed 's/^/lvol-vg-autotest_90617;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/lvol-vg-autotest_90617/timing.txt 00:32:37.291 + for file in "${array[@]}" 00:32:37.291 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/nvme-vg-autotest_90668/timing.txt 00:32:37.291 + local job_file job_name 00:32:37.291 + job_file=/var/jenkins/workspace/autotest-per-patch_3/nvme-vg-autotest_90668/timing.txt 00:32:37.291 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvme-vg-autotest_90668/timing.txt 00:32:37.291 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvme-vg-autotest_90668 00:32:37.291 + job_name=nvme-vg-autotest_90668 00:32:37.291 + sed 's/^/nvme-vg-autotest_90668;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/nvme-vg-autotest_90668/timing.txt 00:32:37.291 + for file in "${array[@]}" 00:32:37.291 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/raid-vg-autotest_5263/timing.txt 00:32:37.291 + local job_file job_name 00:32:37.291 + job_file=/var/jenkins/workspace/autotest-per-patch_3/raid-vg-autotest_5263/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/raid-vg-autotest_5263/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/raid-vg-autotest_5263 00:32:37.292 + job_name=raid-vg-autotest_5263 00:32:37.292 + sed 's/^/raid-vg-autotest_5263;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/raid-vg-autotest_5263/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/check-format-docker-autotest_33194/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/check-format-docker-autotest_33194/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/check-format-docker-autotest_33194/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/check-format-docker-autotest_33194 00:32:37.292 + job_name=check-format-docker-autotest_33194 00:32:37.292 + sed 's/^/check-format-docker-autotest_33194;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/check-format-docker-autotest_33194/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-uring-vg-autotest_90611/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-uring-vg-autotest_90611/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-uring-vg-autotest_90611/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-uring-vg-autotest_90611 00:32:37.292 + job_name=nvmf-tcp-uring-vg-autotest_90611 00:32:37.292 + sed 's/^/nvmf-tcp-uring-vg-autotest_90611;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-uring-vg-autotest_90611/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/vhost-phy-autotest_38373/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/vhost-phy-autotest_38373/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/vhost-phy-autotest_38373/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/vhost-phy-autotest_38373 00:32:37.292 + job_name=vhost-phy-autotest_38373 00:32:37.292 + sed 's/^/vhost-phy-autotest_38373;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/vhost-phy-autotest_38373/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/iscsi-uring-vg-autotest_90569/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/iscsi-uring-vg-autotest_90569/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/iscsi-uring-vg-autotest_90569/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/iscsi-uring-vg-autotest_90569 00:32:37.292 + job_name=iscsi-uring-vg-autotest_90569 00:32:37.292 + sed 's/^/iscsi-uring-vg-autotest_90569;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/iscsi-uring-vg-autotest_90569/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-phy-autotest_84430/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-phy-autotest_84430/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-phy-autotest_84430/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-phy-autotest_84430 00:32:37.292 + job_name=nvmf-tcp-phy-autotest_84430 00:32:37.292 + sed 's/^/nvmf-tcp-phy-autotest_84430;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-phy-autotest_84430/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/nvme-cmb-pmr-vg-autotest_91797/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/nvme-cmb-pmr-vg-autotest_91797/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvme-cmb-pmr-vg-autotest_91797/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvme-cmb-pmr-vg-autotest_91797 00:32:37.292 + job_name=nvme-cmb-pmr-vg-autotest_91797 00:32:37.292 + sed 's/^/nvme-cmb-pmr-vg-autotest_91797;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/nvme-cmb-pmr-vg-autotest_91797/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/vhost-initiator-vg-autotest_90686/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/vhost-initiator-vg-autotest_90686/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/vhost-initiator-vg-autotest_90686/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/vhost-initiator-vg-autotest_90686 00:32:37.292 + job_name=vhost-initiator-vg-autotest_90686 00:32:37.292 + sed 's/^/vhost-initiator-vg-autotest_90686;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/vhost-initiator-vg-autotest_90686/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/short-fuzz-phy-autotest_38301/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/short-fuzz-phy-autotest_38301/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/short-fuzz-phy-autotest_38301/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/short-fuzz-phy-autotest_38301 00:32:37.292 + job_name=short-fuzz-phy-autotest_38301 00:32:37.292 + sed 's/^/short-fuzz-phy-autotest_38301;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/short-fuzz-phy-autotest_38301/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/packaging-docker-autotest_29941/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/packaging-docker-autotest_29941/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/packaging-docker-autotest_29941/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/packaging-docker-autotest_29941 00:32:37.292 + job_name=packaging-docker-autotest_29941 00:32:37.292 + sed 's/^/packaging-docker-autotest_29941;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/packaging-docker-autotest_29941/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/vfio-user-phy-autotest_38306/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/vfio-user-phy-autotest_38306/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/vfio-user-phy-autotest_38306/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/vfio-user-phy-autotest_38306 00:32:37.292 + job_name=vfio-user-phy-autotest_38306 00:32:37.292 + sed 's/^/vfio-user-phy-autotest_38306;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/vfio-user-phy-autotest_38306/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/check-so-deps-docker-autotest_33415/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/check-so-deps-docker-autotest_33415/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/check-so-deps-docker-autotest_33415/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/check-so-deps-docker-autotest_33415 00:32:37.292 + job_name=check-so-deps-docker-autotest_33415 00:32:37.292 + sed 's/^/check-so-deps-docker-autotest_33415;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/check-so-deps-docker-autotest_33415/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/unittest-clang-docker-autotest_29942/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/unittest-clang-docker-autotest_29942/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/unittest-clang-docker-autotest_29942/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/unittest-clang-docker-autotest_29942 00:32:37.292 + job_name=unittest-clang-docker-autotest_29942 00:32:37.292 + sed 's/^/unittest-clang-docker-autotest_29942;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/unittest-clang-docker-autotest_29942/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/BlobFS-phy-autotest_38316/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/BlobFS-phy-autotest_38316/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/BlobFS-phy-autotest_38316/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/BlobFS-phy-autotest_38316 00:32:37.292 + job_name=BlobFS-phy-autotest_38316 00:32:37.292 + sed 's/^/BlobFS-phy-autotest_38316;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/BlobFS-phy-autotest_38316/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/unittest-main-docker-autotest_29944/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/unittest-main-docker-autotest_29944/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/unittest-main-docker-autotest_29944/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/unittest-main-docker-autotest_29944 00:32:37.292 + job_name=unittest-main-docker-autotest_29944 00:32:37.292 + sed 's/^/unittest-main-docker-autotest_29944;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/unittest-main-docker-autotest_29944/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/ubuntu24-vg-autotest_7099/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/ubuntu24-vg-autotest_7099/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/ubuntu24-vg-autotest_7099/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/ubuntu24-vg-autotest_7099 00:32:37.292 + job_name=ubuntu24-vg-autotest_7099 00:32:37.292 + sed 's/^/ubuntu24-vg-autotest_7099;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/ubuntu24-vg-autotest_7099/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/autobuild-external-vg-autotest_30579/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/autobuild-external-vg-autotest_30579/timing.txt 00:32:37.292 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/autobuild-external-vg-autotest_30579/timing.txt 00:32:37.292 ++ basename /var/jenkins/workspace/autotest-per-patch_3/autobuild-external-vg-autotest_30579 00:32:37.292 + job_name=autobuild-external-vg-autotest_30579 00:32:37.292 + sed 's/^/autobuild-external-vg-autotest_30579;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/autobuild-external-vg-autotest_30579/timing.txt 00:32:37.292 + for file in "${array[@]}" 00:32:37.292 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/rocky9-vg-autotest_27871/timing.txt 00:32:37.292 + local job_file job_name 00:32:37.292 + job_file=/var/jenkins/workspace/autotest-per-patch_3/rocky9-vg-autotest_27871/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/rocky9-vg-autotest_27871/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/rocky9-vg-autotest_27871 00:32:37.293 + job_name=rocky9-vg-autotest_27871 00:32:37.293 + sed 's/^/rocky9-vg-autotest_27871;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/rocky9-vg-autotest_27871/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/freebsd-vg-autotest_68599/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/freebsd-vg-autotest_68599/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/freebsd-vg-autotest_68599/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/freebsd-vg-autotest_68599 00:32:37.293 + job_name=freebsd-vg-autotest_68599 00:32:37.293 + sed 's/^/freebsd-vg-autotest_68599;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/freebsd-vg-autotest_68599/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/ubuntu22-vg-autotest_49538/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/ubuntu22-vg-autotest_49538/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/ubuntu22-vg-autotest_49538/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/ubuntu22-vg-autotest_49538 00:32:37.293 + job_name=ubuntu22-vg-autotest_49538 00:32:37.293 + sed 's/^/ubuntu22-vg-autotest_49538;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/ubuntu22-vg-autotest_49538/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/scan-build-docker-autotest_29948/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/scan-build-docker-autotest_29948/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/scan-build-docker-autotest_29948/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/scan-build-docker-autotest_29948 00:32:37.293 + job_name=scan-build-docker-autotest_29948 00:32:37.293 + sed 's/^/scan-build-docker-autotest_29948;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/scan-build-docker-autotest_29948/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/zns-vg-autotest_89770/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/zns-vg-autotest_89770/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/zns-vg-autotest_89770/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/zns-vg-autotest_89770 00:32:37.293 + job_name=zns-vg-autotest_89770 00:32:37.293 + sed 's/^/zns-vg-autotest_89770;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/zns-vg-autotest_89770/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/nvme-phy-autotest_84479/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/nvme-phy-autotest_84479/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvme-phy-autotest_84479/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvme-phy-autotest_84479 00:32:37.293 + job_name=nvme-phy-autotest_84479 00:32:37.293 + sed 's/^/nvme-phy-autotest_84479;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/nvme-phy-autotest_84479/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/doc-docker-autotest_33497/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/doc-docker-autotest_33497/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/doc-docker-autotest_33497/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/doc-docker-autotest_33497 00:32:37.293 + job_name=doc-docker-autotest_33497 00:32:37.293 + sed 's/^/doc-docker-autotest_33497;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/doc-docker-autotest_33497/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/unittest-valgrind-docker-autotest_29946/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/unittest-valgrind-docker-autotest_29946/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/unittest-valgrind-docker-autotest_29946/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/unittest-valgrind-docker-autotest_29946 00:32:37.293 + job_name=unittest-valgrind-docker-autotest_29946 00:32:37.293 + sed 's/^/unittest-valgrind-docker-autotest_29946;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/unittest-valgrind-docker-autotest_29946/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-vg-autotest_90650/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-vg-autotest_90650/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-vg-autotest_90650/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-vg-autotest_90650 00:32:37.293 + job_name=nvmf-tcp-vg-autotest_90650 00:32:37.293 + sed 's/^/nvmf-tcp-vg-autotest_90650;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/nvmf-tcp-vg-autotest_90650/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/release-build-main-docker-autotest_3119/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/release-build-main-docker-autotest_3119/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/release-build-main-docker-autotest_3119/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/release-build-main-docker-autotest_3119 00:32:37.293 + job_name=release-build-main-docker-autotest_3119 00:32:37.293 + sed 's/^/release-build-main-docker-autotest_3119;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/release-build-main-docker-autotest_3119/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/release-build-clang-docker-autotest_3119/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/release-build-clang-docker-autotest_3119/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/release-build-clang-docker-autotest_3119/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/release-build-clang-docker-autotest_3119 00:32:37.293 + job_name=release-build-clang-docker-autotest_3119 00:32:37.293 + sed 's/^/release-build-clang-docker-autotest_3119;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/release-build-clang-docker-autotest_3119/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/build-files-docker-autotest_33168/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/build-files-docker-autotest_33168/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/build-files-docker-autotest_33168/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/build-files-docker-autotest_33168 00:32:37.293 + job_name=build-files-docker-autotest_33168 00:32:37.293 + sed 's/^/build-files-docker-autotest_33168;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/build-files-docker-autotest_33168/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/crypto-phy-autotest_38305/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/crypto-phy-autotest_38305/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/crypto-phy-autotest_38305/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/crypto-phy-autotest_38305 00:32:37.293 + job_name=crypto-phy-autotest_38305 00:32:37.293 + sed 's/^/crypto-phy-autotest_38305;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/crypto-phy-autotest_38305/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/nvmf-phy-autotest_84548/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/nvmf-phy-autotest_84548/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/nvmf-phy-autotest_84548/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/nvmf-phy-autotest_84548 00:32:37.293 + job_name=nvmf-phy-autotest_84548 00:32:37.293 + sed 's/^/nvmf-phy-autotest_84548;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/nvmf-phy-autotest_84548/timing.txt 00:32:37.293 + for file in "${array[@]}" 00:32:37.293 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_3/iscsi-vg-autotest_90705/timing.txt 00:32:37.293 + local job_file job_name 00:32:37.293 + job_file=/var/jenkins/workspace/autotest-per-patch_3/iscsi-vg-autotest_90705/timing.txt 00:32:37.293 +++ dirname /var/jenkins/workspace/autotest-per-patch_3/iscsi-vg-autotest_90705/timing.txt 00:32:37.293 ++ basename /var/jenkins/workspace/autotest-per-patch_3/iscsi-vg-autotest_90705 00:32:37.293 + job_name=iscsi-vg-autotest_90705 00:32:37.293 + sed 's/^/iscsi-vg-autotest_90705;autorun;/' /var/jenkins/workspace/autotest-per-patch_3/iscsi-vg-autotest_90705/timing.txt 00:32:37.293 + /usr/local/FlameGraph/flamegraph.pl --title 'Build Timing' --nametype Step: --countname seconds /var/jenkins/workspace/autotest-per-patch_3/post_process/total_timing.txt 00:32:37.551 + run_autorun_post 00:32:37.551 + s_opt= 00:32:37.551 + [[ '' == true ]] 00:32:37.551 + spdk/autorun_post.py -d ./ -r ./spdk 00:33:16.235 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/module/blobfs/bdev/blobfs_bdev.c:254 00:33:26.189 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/thread/thread.c:1613 00:33:26.189 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/thread/thread.c:1602 00:33:26.189 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/thread/thread.c:1597 00:33:26.189 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/thread/thread.c:1607 00:33:26.189 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/thread/thread.c:1591 00:33:26.189 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/thread/thread.c:2805 00:33:26.189 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/env_dpdk/pci.c:1078 00:33:26.189 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/env_dpdk/pci.c:1085 00:33:26.445 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/test/unit/lib/env_dpdk/pci_event.c/pci_event_ut.c:111 00:33:26.702 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/module/bdev/aio/bdev_aio.c:696 00:33:26.702 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/module/bdev/aio/bdev_aio.c:326 00:33:26.702 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/module/bdev/aio/bdev_aio.c:684 00:33:26.702 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/module/bdev/aio/bdev_aio.c:344 00:33:26.702 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/module/bdev/aio/bdev_aio.c:164 00:33:26.959 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/event/reactor.c:1530 00:33:27.217 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/env_dpdk/pci_event.c:233 00:33:27.217 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/env_dpdk/pci_event.c:226 00:33:27.474 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/uuid.c:139 00:33:27.474 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/uuid.c:127 00:33:27.474 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/uuid.c:74 00:33:27.474 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/uuid.c:133 00:33:27.474 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/uuid.c:115 00:33:27.474 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/uuid.c:93 00:33:27.474 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/uuid.c:121 00:33:27.731 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/fd_group.c:559 00:33:27.731 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/fd_group.c:517 00:33:27.731 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/fd_group.c:529 00:33:27.731 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/fd_group.c:553 00:33:27.731 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/fd_group.c:547 00:33:27.731 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/fd_group.c:510 00:33:27.731 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/fd_group.c:542 00:33:27.731 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/fd_group.c:536 00:33:27.731 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/fd_group.c:504 00:33:27.731 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/util/fd_group.c:524 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/endian.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/base64.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/nvme_spec.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/trace.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/nvmf_transport.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/histogram_data.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/bdev_module.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/mmio.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/util.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/fsdev_module.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/thread.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/nvme.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/histogram_data.h/histogram_ut.c! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/env_ocf/include/ocf/ocf_mngt.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/env_ocf/include/ocf/ocf_core.h! 00:34:23.924 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_3/spdk/lib/env_ocf/include/ocf/ocf_io.h! 00:34:41.990 -----Begin Post Process Script------ 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/lvol-vg-autotest_90617/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/nvme-vg-autotest_90668/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/raid-vg-autotest_5263/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/vhost-phy-autotest_38373/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/iscsi-uring-vg-autotest_90569/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/nvme-cmb-pmr-vg-autotest_91797/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/vhost-initiator-vg-autotest_90686/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/short-fuzz-phy-autotest_38301/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/vfio-user-phy-autotest_38306/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/BlobFS-phy-autotest_38316/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/ubuntu24-vg-autotest_7099/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/rocky9-vg-autotest_27871/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/freebsd-vg-autotest_68599/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/ubuntu22-vg-autotest_49538/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/zns-vg-autotest_89770/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/nvme-phy-autotest_84479/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/crypto-phy-autotest_38305/cov_total.info 00:34:41.990 /var/jenkins/workspace/autotest-per-patch_3/iscsi-vg-autotest_90705/cov_total.info 00:34:41.990 00:34:41.990 00:34:41.990 -----Tests Executed in Build------ 00:34:41.990 accel 00:34:41.990 accel_assign_opcode 00:34:41.990 accel_cdev_comp 00:34:41.990 accel_cdev_decomp 00:34:41.990 accel_cdev_decomp_full 00:34:41.991 accel_cdev_decomp_full_mcore 00:34:41.991 accel_cdev_decomp_full_mthread 00:34:41.991 accel_cdev_decomp_mcore 00:34:41.991 accel_cdev_decomp_mthread 00:34:41.991 accel_comp 00:34:41.991 accel_compare 00:34:41.991 accel_compress_verify 00:34:41.991 accel_copy 00:34:41.991 accel_copy_crc32c 00:34:41.991 accel_copy_crc32c_C2 00:34:41.991 accel_crc32c 00:34:41.991 accel_crc32c_C2 00:34:41.991 accel_decomp 00:34:41.991 accel_decomp_full 00:34:41.991 accel_decomp_full_mcore 00:34:41.991 accel_decomp_full_mthread 00:34:41.991 accel_decomp_mcore 00:34:41.991 accel_decomp_mthread 00:34:41.991 accel_dif_functional_tests 00:34:41.991 accel_dif_generate 00:34:41.991 accel_dif_generate_copy 00:34:41.991 accel_dif_verify 00:34:41.991 accel_dix_generate 00:34:41.991 accel_dix_verify 00:34:41.991 accel_dualcast 00:34:41.991 accel_fill 00:34:41.991 accel_help 00:34:41.991 accel_missing_filename 00:34:41.991 accel_negative_buffers 00:34:41.991 accel_rpc 00:34:41.991 accel_wrong_workload 00:34:41.991 accel_xor 00:34:41.991 acl 00:34:41.991 alias_rpc 00:34:41.991 allowed 00:34:41.991 app_cmdline 00:34:41.991 app_repeat 00:34:41.991 asan 00:34:41.991 autobuild 00:34:41.991 autobuild_build_doc 00:34:41.991 autobuild_check_dpdk_pci_api 00:34:41.991 autobuild_check_format 00:34:41.991 autobuild_check_so_deps 00:34:41.991 autobuild_external_code 00:34:41.991 autobuild_generated_files_check 00:34:41.991 autobuild_generated_files_check_post_clean 00:34:41.991 autobuild_header_dependency_check 00:34:41.991 autobuild_llvm_precompile 00:34:41.991 autobuild_make_install 00:34:41.991 autobuild_make_uninstall 00:34:41.991 autobuild_ocf_precompile 00:34:41.991 bdev_bounds 00:34:41.991 bdev_crypto_enomem 00:34:41.991 bdev_error 00:34:41.991 bdev_fio 00:34:41.991 bdev_fio_rw_verify 00:34:41.991 bdev_fio_trim 00:34:41.991 bdev_gpt_uuid 00:34:41.991 bdev_hello_world 00:34:41.991 bdev_json_nonarray 00:34:41.991 bdev_json_nonenclosed 00:34:41.991 bdev_nbd 00:34:41.991 bdev_nvme_reset_stuck_adm_cmd 00:34:41.991 bdev_qd_sampling 00:34:41.991 bdev_qos 00:34:41.991 bdev_qos_bw 00:34:41.991 bdev_qos_iops 00:34:41.991 bdev_qos_ro_bw 00:34:41.991 bdev_raid 00:34:41.991 bdev_stat 00:34:41.991 bdev_verify 00:34:41.991 bdev_verify_big_io 00:34:41.991 bdev_write_zeroes 00:34:41.991 bdevperf_config 00:34:41.991 blob_io_wait 00:34:41.991 blobfs 00:34:41.991 blobfs_mkfs 00:34:41.991 blobstore 00:34:41.991 blobstore_grow 00:34:41.991 blockdev_crypto_aesni 00:34:41.991 blockdev_crypto_qat 00:34:41.991 blockdev_crypto_sw 00:34:41.991 blockdev_general 00:34:41.991 blockdev_nvme 00:34:41.991 blockdev_nvme_gpt 00:34:41.991 blockdev_raid5f 00:34:41.991 blockdev_rbd 00:34:41.991 blockdev_xnvme 00:34:41.991 build_release 00:34:41.991 build_rpm_from_gen_spec 00:34:41.991 build_shared_rpm 00:34:41.991 chaining 00:34:41.991 chap_during_discovery 00:34:41.991 chap_mutual_auth 00:34:41.991 check_header_filenames 00:34:41.991 cmb_copy 00:34:41.991 compress_compdev 00:34:41.991 compress_isal 00:34:41.991 confirm_abi_deps 00:34:41.991 confirm_makefile_deps 00:34:41.991 core_isolating 00:34:41.991 cpu_locks 00:34:41.991 custom_alloc 00:34:41.991 dd_bs_lt_native_bs 00:34:41.991 dd_bs_not_multiple 00:34:41.991 dd_copy_to_out_bdev 00:34:41.991 dd_double_input 00:34:41.991 dd_double_output 00:34:41.991 dd_flag_append 00:34:41.991 dd_flag_append_forced_aio 00:34:41.991 dd_flag_directory 00:34:41.991 dd_flag_directory_forced_aio 00:34:41.991 dd_flag_noatime 00:34:41.991 dd_flag_noatime_forced_aio 00:34:41.991 dd_flag_nofollow 00:34:41.991 dd_flag_nofollow_forced_aio 00:34:41.991 dd_flags_misc 00:34:41.991 dd_flags_misc_forced_aio 00:34:41.991 dd_inflate_file 00:34:41.991 dd_invalid_arguments 00:34:41.991 dd_invalid_count 00:34:41.991 dd_invalid_iflag 00:34:41.991 dd_invalid_input_count 00:34:41.991 dd_invalid_json 00:34:41.991 dd_invalid_oflag 00:34:41.991 dd_invalid_output_count 00:34:41.991 dd_invalid_seek 00:34:41.991 dd_invalid_skip 00:34:41.991 dd_malloc_copy 00:34:41.991 dd_no_input 00:34:41.991 dd_no_output 00:34:41.991 dd_offset_magic 00:34:41.991 dd_rw 00:34:41.991 dd_rw_offset 00:34:41.991 dd_smaller_blocksize 00:34:41.991 dd_sparse_bdev_to_file 00:34:41.991 dd_sparse_file_to_bdev 00:34:41.991 dd_sparse_file_to_file 00:34:41.991 dd_unknown_flag 00:34:41.991 dd_uring_copy 00:34:41.991 dd_wrong_blocksize 00:34:41.991 default_locks 00:34:41.991 default_locks_via_rpc 00:34:41.991 denied 00:34:41.991 devices 00:34:41.991 dm_mount 00:34:41.991 dma 00:34:41.991 dpdk_governor 00:34:41.991 dpdk_mem_utility 00:34:41.991 driver 00:34:41.991 env 00:34:41.991 env_dpdk_post_init 00:34:41.991 env_mem_callbacks 00:34:41.991 env_memory 00:34:41.991 env_pci 00:34:41.991 env_vtophys 00:34:41.991 even_2G_alloc 00:34:41.991 event 00:34:41.991 event_perf 00:34:41.991 event_reactor 00:34:41.991 event_reactor_perf 00:34:41.991 event_scheduler 00:34:41.991 exit_on_failed_rpc_init 00:34:41.991 external_make_accel_driver_shared 00:34:41.991 external_make_accel_driver_static 00:34:41.991 external_make_accel_module_shared 00:34:41.991 external_make_accel_module_static 00:34:41.991 external_make_hello_bdev_shared_combo 00:34:41.991 external_make_hello_bdev_shared_iso 00:34:41.991 external_make_hello_bdev_static 00:34:41.991 external_make_hello_no_bdev_shared_combo 00:34:41.991 external_make_hello_no_bdev_shared_iso 00:34:41.991 external_make_hello_no_bdev_static 00:34:41.991 external_make_nvme_shared 00:34:41.991 external_make_nvme_static 00:34:41.991 external_run_accel_driver_shared 00:34:41.991 external_run_accel_driver_static 00:34:41.991 external_run_accel_module_shared 00:34:41.991 external_run_accel_module_static 00:34:41.991 external_run_hello_bdev_shared_combo 00:34:41.991 external_run_hello_bdev_shared_iso 00:34:41.991 external_run_hello_bdev_static 00:34:41.991 external_run_hello_no_bdev_shared_combo 00:34:41.991 external_run_hello_no_bdev_shared_iso 00:34:41.991 external_run_hello_no_bdev_static 00:34:41.991 external_run_nvme_shared 00:34:41.991 external_run_nvme_static 00:34:41.991 filesystem_btrfs 00:34:41.991 filesystem_ext4 00:34:41.991 filesystem_in_capsule_btrfs 00:34:41.991 filesystem_in_capsule_ext4 00:34:41.991 filesystem_in_capsule_xfs 00:34:41.991 filesystem_xfs 00:34:41.991 framework_get_governor 00:34:41.991 ftl 00:34:41.991 ftl_bdevperf 00:34:41.991 ftl_dirty_shutdown 00:34:41.991 ftl_fio_basic 00:34:41.991 ftl_restore 00:34:41.991 ftl_trim 00:34:41.991 ftl_upgrade_shutdown 00:34:41.991 go_rpc 00:34:41.991 guess_driver 00:34:41.991 hello_blob 00:34:41.991 hugepages 00:34:41.991 idle 00:34:41.991 interrupt_mode 00:34:41.991 ioat 00:34:41.991 ioat_perf 00:34:41.991 ioat_verify 00:34:41.991 is_zoned 00:34:41.991 iscsi_tgt 00:34:41.991 iscsi_tgt_bdev_io_wait 00:34:41.991 iscsi_tgt_calsoft 00:34:41.991 iscsi_tgt_digest 00:34:41.991 iscsi_tgt_digests 00:34:41.991 iscsi_tgt_filesystem 00:34:41.991 iscsi_tgt_filesystem_btrfs 00:34:41.991 iscsi_tgt_filesystem_ext4 00:34:41.991 iscsi_tgt_filesystem_xfs 00:34:41.991 iscsi_tgt_fio 00:34:41.991 iscsi_tgt_fuzz 00:34:41.991 iscsi_tgt_initiator 00:34:41.991 iscsi_tgt_ip_migration 00:34:41.991 iscsi_tgt_iscsi_lvol 00:34:41.991 iscsi_tgt_login_redirection 00:34:41.991 iscsi_tgt_multiconnection 00:34:41.991 iscsi_tgt_qos 00:34:41.991 iscsi_tgt_rbd 00:34:41.991 iscsi_tgt_reset 00:34:41.991 iscsi_tgt_resize 00:34:41.991 iscsi_tgt_rpc_config 00:34:41.991 iscsi_tgt_sock 00:34:41.991 iscsi_tgt_trace_record 00:34:41.991 json_config 00:34:41.991 json_config_extra_key 00:34:41.991 llvm_fuzz 00:34:41.991 locking_app_on_locked_coremask 00:34:41.991 locking_app_on_unlocked_coremask 00:34:41.991 locking_overlapped_coremask 00:34:41.991 locking_overlapped_coremask_via_rpc 00:34:41.991 lvol 00:34:41.991 lvol_basic 00:34:41.991 lvol_esnap 00:34:41.991 lvol_external_copy 00:34:41.991 lvol_external_snapshot 00:34:41.991 lvol_hotremove 00:34:41.991 lvol_provisioning 00:34:41.991 lvol_rename 00:34:41.991 lvol_resize 00:34:41.991 lvol_snapshot_clone 00:34:41.991 lvol_tasting 00:34:41.991 lvs_grow_clean 00:34:41.992 lvs_grow_dirty 00:34:41.992 make 00:34:41.992 no_shrink_alloc 00:34:41.992 non_locking_app_on_locked_coremask 00:34:41.992 nvme 00:34:41.992 nvme_arbitration 00:34:41.992 nvme_cli_cuse 00:34:41.992 nvme_cli_plugin 00:34:41.992 nvme_cmb 00:34:41.992 nvme_cuse 00:34:41.992 nvme_cuse_app 00:34:41.992 nvme_cuse_rpc 00:34:41.992 nvme_doorbell_aers 00:34:41.992 nvme_e2edp 00:34:41.992 nvme_err_injection 00:34:41.992 nvme_fdp 00:34:41.992 nvme_fio 00:34:41.992 nvme_flexible_data_placement 00:34:41.992 nvme_hello_world 00:34:41.992 nvme_identify 00:34:41.992 nvme_mount 00:34:41.992 nvme_multi_aen 00:34:41.992 nvme_multi_secondary 00:34:41.992 nvme_ns_manage_cuse 00:34:41.992 nvme_overhead 00:34:41.992 nvme_perf 00:34:41.992 nvme_pmr 00:34:41.992 nvme_pmr_persistence 00:34:41.992 nvme_reserve 00:34:41.992 nvme_reset 00:34:41.992 nvme_rpc 00:34:41.992 nvme_rpc_timeouts 00:34:41.992 nvme_scc 00:34:41.992 nvme_sgl 00:34:41.992 nvme_simple_copy 00:34:41.992 nvme_single_aen 00:34:41.992 nvme_smartctl_cuse 00:34:41.992 nvme_startup 00:34:41.992 nvme_xnvme 00:34:41.992 nvme_zns 00:34:41.992 nvmf_abort 00:34:41.992 nvmf_aer 00:34:41.992 nvmf_async_init 00:34:41.992 nvmf_auth_target 00:34:41.992 nvmf_bdev_io_wait 00:34:41.992 nvmf_bdevio 00:34:41.992 nvmf_bdevio_no_huge 00:34:41.992 nvmf_connect_disconnect 00:34:41.992 nvmf_connect_stress 00:34:41.992 nvmf_control_msg_list 00:34:41.992 nvmf_delete_subsystem 00:34:41.992 nvmf_example 00:34:41.992 nvmf_filesystem 00:34:41.992 nvmf_filesystem_in_capsule 00:34:41.992 nvmf_filesystem_no_in_capsule 00:34:41.992 nvmf_fio_host 00:34:41.992 nvmf_fio_target 00:34:41.992 nvmf_fips 00:34:41.992 nvmf_fused_ordering 00:34:41.992 nvmf_host_management 00:34:41.992 nvmf_identify 00:34:41.992 nvmf_invalid 00:34:41.992 nvmf_llvm_fuzz 00:34:41.992 nvmf_lvol 00:34:41.992 nvmf_lvs_grow 00:34:41.992 nvmf_multicontroller 00:34:41.992 nvmf_multitarget 00:34:41.992 nvmf_nmic 00:34:41.992 nvmf_ns_hotplug_stress 00:34:41.992 nvmf_ns_masking 00:34:41.992 nvmf_nvme_cli 00:34:41.992 nvmf_perf 00:34:41.992 nvmf_perf_adq 00:34:41.992 nvmf_queue_depth 00:34:41.992 nvmf_referrals 00:34:41.992 nvmf_rpc 00:34:41.992 nvmf_shutdown 00:34:41.992 nvmf_shutdown_tc1 00:34:41.992 nvmf_shutdown_tc2 00:34:41.992 nvmf_shutdown_tc3 00:34:41.992 nvmf_shutdown_tc4 00:34:41.992 nvmf_srq_overwhelm 00:34:41.992 nvmf_target_core 00:34:41.992 nvmf_target_discovery 00:34:41.992 nvmf_target_extra 00:34:41.992 nvmf_target_multipath 00:34:41.992 nvmf_tls 00:34:41.992 nvmf_vfio_user 00:34:41.992 nvmf_vfio_user_fuzz 00:34:41.992 nvmf_vfio_user_nvme_compliance 00:34:41.992 nvmf_vhost 00:34:41.992 nvmf_wait_for_buf 00:34:41.992 nvmf_zcopy 00:34:41.992 ocf 00:34:41.992 ocf_bdevperf_iotypes 00:34:41.992 ocf_configuration_change 00:34:41.992 ocf_create_destruct 00:34:41.992 ocf_fio_modes 00:34:41.992 ocf_flush 00:34:41.992 ocf_multicore 00:34:41.992 ocf_remove 00:34:41.992 ocf_stats 00:34:41.992 odd_alloc 00:34:41.992 packaging 00:34:41.992 raid0_resize_superblock_test 00:34:41.992 raid0_resize_test 00:34:41.992 raid1_resize_data_offset_test 00:34:41.992 raid1_resize_superblock_test 00:34:41.992 raid1_resize_test 00:34:41.992 raid5f_rebuild_test 00:34:41.992 raid5f_rebuild_test_sb 00:34:41.992 raid5f_state_function_test 00:34:41.992 raid5f_state_function_test_sb 00:34:41.992 raid5f_superblock_test 00:34:41.992 raid_function_test_concat 00:34:41.992 raid_function_test_raid0 00:34:41.992 raid_read_error_test 00:34:41.992 raid_rebuild_test 00:34:41.992 raid_rebuild_test_io 00:34:41.992 raid_rebuild_test_sb 00:34:41.992 raid_rebuild_test_sb_4k 00:34:41.992 raid_rebuild_test_sb_io 00:34:41.992 raid_rebuild_test_sb_md_interleaved 00:34:41.992 raid_rebuild_test_sb_md_separate 00:34:41.992 raid_state_function_test 00:34:41.992 raid_state_function_test_sb 00:34:41.992 raid_state_function_test_sb_4k 00:34:41.992 raid_state_function_test_sb_md_interleaved 00:34:41.992 raid_state_function_test_sb_md_separate 00:34:41.992 raid_superblock_test 00:34:41.992 raid_superblock_test_4k 00:34:41.992 raid_superblock_test_md_interleaved 00:34:41.992 raid_superblock_test_md_separate 00:34:41.992 raid_write_error_test 00:34:41.992 reactor_set_interrupt 00:34:41.992 reap_unregistered_poller 00:34:41.992 rocksdb 00:34:41.992 rocksdb_insert 00:34:41.992 rocksdb_overwrite 00:34:41.992 rocksdb_randread 00:34:41.992 rocksdb_readwrite 00:34:41.992 rocksdb_writesync 00:34:41.992 rpc 00:34:41.992 rpc_client 00:34:41.992 rpc_daemon_integrity 00:34:41.992 rpc_integrity 00:34:41.992 rpc_plugins 00:34:41.992 rpc_trace_cmd_test 00:34:41.992 rpm_packaging 00:34:41.992 scanbuild_make 00:34:41.992 scheduler 00:34:41.992 scheduler_create_thread 00:34:41.992 scheduler_opts 00:34:41.992 scheduler_rpc 00:34:41.992 setup.sh 00:34:41.992 single_node_setup 00:34:41.992 skip_rpc 00:34:41.992 skip_rpc_with_delay 00:34:41.992 skip_rpc_with_json 00:34:41.992 sma 00:34:41.992 sma_crypto 00:34:41.992 sma_discovery 00:34:41.992 sma_nvmf_tcp 00:34:41.992 sma_plugins 00:34:41.992 sma_qos 00:34:41.992 sma_vfiouser_qemu 00:34:41.992 sma_vhost 00:34:41.992 spdk_dd 00:34:41.992 spdk_dd_basic_rw 00:34:41.992 spdk_dd_bdev_to_bdev 00:34:41.992 spdk_dd_malloc 00:34:41.992 spdk_dd_negative 00:34:41.992 spdk_dd_posix 00:34:41.992 spdk_dd_sparse 00:34:41.992 spdk_dd_uring 00:34:41.992 spdkcli_iscsi 00:34:41.992 spdkcli_raid 00:34:41.992 spdkcli_rbd 00:34:41.992 spdkcli_tcp 00:34:41.992 spdkcli_vhost 00:34:41.992 spdkcli_virtio 00:34:41.992 static_as_default 00:34:41.992 sw_hotplug 00:34:41.992 test_bdev_lvol_delete_lvstore_with_clones 00:34:41.992 test_bdev_lvol_delete_ordering 00:34:41.992 test_clone_decouple_parent 00:34:41.992 test_clone_inflate 00:34:41.992 test_clone_snapshot_relations 00:34:41.992 test_construct_lvol 00:34:41.992 test_construct_lvol_alias_conflict 00:34:41.992 test_construct_lvol_fio_clear_method_none 00:34:41.992 test_construct_lvol_fio_clear_method_unmap 00:34:41.992 test_construct_lvol_full_lvs 00:34:41.992 test_construct_lvol_inexistent_lvs 00:34:41.992 test_construct_lvols_conflict_alias 00:34:41.992 test_construct_lvs 00:34:41.992 test_construct_lvs_clear_methods 00:34:41.992 test_construct_lvs_conflict_alias 00:34:41.992 test_construct_lvs_different_cluster_size 00:34:41.992 test_construct_lvs_nonexistent_bdev 00:34:41.992 test_construct_multi_lvols 00:34:41.992 test_construct_nested_lvol 00:34:41.992 test_construct_two_lvs_on_the_same_bdev 00:34:41.992 test_create_multi_ublk 00:34:41.992 test_create_snapshot_of_snapshot 00:34:41.992 test_create_snapshot_with_io 00:34:41.992 test_create_ublk 00:34:41.992 test_delete_lvol_store_persistent_positive 00:34:41.992 test_delete_snapshot_with_clone 00:34:41.992 test_delete_snapshot_with_snapshot 00:34:41.992 test_destroy_after_bdev_lvol_resize_positive 00:34:41.992 test_esnap_clones 00:34:41.992 test_esnap_late_arrival 00:34:41.992 test_esnap_reload 00:34:41.992 test_esnap_remove_degraded 00:34:41.992 test_hotremove_lvol_store 00:34:41.992 test_hotremove_lvol_store_base 00:34:41.992 test_hotremove_lvol_store_base_with_lvols 00:34:41.992 test_hotremove_lvol_store_multiple_lvols 00:34:41.992 test_lvol_bdev_readonly 00:34:41.992 test_lvol_list 00:34:41.992 test_lvol_rename_negative 00:34:41.992 test_lvol_set_parent_bdev_from_esnap 00:34:41.992 test_lvol_set_parent_bdev_from_none 00:34:41.992 test_lvol_set_parent_bdev_from_snapshot 00:34:41.992 test_lvol_set_parent_failed 00:34:41.992 test_lvol_set_parent_from_esnap 00:34:41.992 test_lvol_set_parent_from_none 00:34:41.992 test_lvol_set_parent_from_snapshot 00:34:41.992 test_rename_lvs_negative 00:34:41.992 test_rename_positive 00:34:41.992 test_resize_lvol 00:34:41.992 test_resize_lvol_negative 00:34:41.992 test_resize_lvol_with_io_traffic 00:34:41.992 test_save_ublk_config 00:34:41.992 test_shallow_copy_compare 00:34:41.992 test_sigterm 00:34:41.992 test_snapshot_compare_with_lvol_bdev 00:34:41.992 test_tasting 00:34:41.992 test_thin_lvol_check_integrity 00:34:41.992 test_thin_lvol_check_space 00:34:41.992 test_thin_lvol_check_zeroes 00:34:41.992 test_thin_lvol_resize 00:34:41.992 test_thin_overprovisioning 00:34:41.992 test_unregister_lvol_bdev 00:34:41.992 thread 00:34:41.992 thread_poller_perf 00:34:41.992 thread_spdk_lock 00:34:41.992 ublk 00:34:41.992 ublk_recovery 00:34:41.992 ubsan 00:34:41.992 unittest 00:34:41.992 unittest_accel 00:34:41.992 unittest_bdev 00:34:41.992 unittest_bdev_compress 00:34:41.992 unittest_bdev_crypto 00:34:41.992 unittest_bdev_raid5f 00:34:41.992 unittest_blob_blobfs 00:34:41.992 unittest_build 00:34:41.992 unittest_dma 00:34:41.992 unittest_dpdk_compressdev 00:34:41.992 unittest_event 00:34:41.992 unittest_fsdev 00:34:41.992 unittest_ftl 00:34:41.992 unittest_idxd_user 00:34:41.992 unittest_include 00:34:41.992 unittest_init 00:34:41.992 unittest_ioat 00:34:41.992 unittest_iobuf 00:34:41.992 unittest_iscsi 00:34:41.992 unittest_json 00:34:41.992 unittest_keyring 00:34:41.992 unittest_lib_reduce 00:34:41.992 unittest_log 00:34:41.992 unittest_lvol 00:34:41.992 unittest_notify 00:34:41.992 unittest_nvme 00:34:41.992 unittest_nvme_cuse 00:34:41.992 unittest_nvme_rdma 00:34:41.992 unittest_nvmf 00:34:41.992 unittest_nvmf_rdma 00:34:41.992 unittest_nvmf_transport 00:34:41.992 unittest_nvmf_vfio_user 00:34:41.992 unittest_pci_event 00:34:41.992 unittest_rdma 00:34:41.992 unittest_rpc 00:34:41.993 unittest_scsi 00:34:41.993 unittest_sock 00:34:41.993 unittest_thread 00:34:41.993 unittest_util 00:34:41.993 unittest_vhost 00:34:41.993 valgrind 00:34:41.993 version 00:34:41.993 vfio_llvm_fuzz 00:34:41.993 vfio_user_nvme_fio 00:34:41.993 vfio_user_nvme_restart_vm 00:34:41.993 vfio_user_qemu 00:34:41.993 vfio_user_virtio_bdevperf 00:34:41.993 vfio_user_virtio_blk_restart_vm 00:34:41.993 vfio_user_virtio_fs_fio 00:34:41.993 vfio_user_virtio_scsi_restart_vm 00:34:41.993 vhost 00:34:41.993 vhost_blk_lvol_integrity 00:34:41.993 vhost_blk_packed_ring_integrity 00:34:41.993 vhost_blockdev 00:34:41.993 vhost_boot 00:34:41.993 vhost_fuzz 00:34:41.993 vhost_negative 00:34:41.993 vhost_scsi_lvol_integrity 00:34:41.993 vhost_shared 00:34:41.993 xnvme_bdevperf 00:34:41.993 xnvme_to_malloc_dd_copy 00:34:41.993 zoned_fio 00:34:41.993 00:34:41.993 00:34:41.993 -----Tests Missing From Build------ 00:34:41.993 accel_mlx5_copy 00:34:41.993 accel_mlx5_crc32c 00:34:41.993 accel_mlx5_crypto 00:34:41.993 accel_scan_dsa_modules 00:34:41.993 accel_scan_iaa_modules 00:34:41.993 balanced 00:34:41.993 build_native_dpdk 00:34:41.993 build_shared_native_dpdk_rpm 00:34:41.993 build_shared_rpm_with_rpmed_dpdk 00:34:41.993 busy 00:34:41.993 core_load 00:34:41.993 fio_dif_1_default 00:34:41.993 fio_dif_1_multi_subsystems 00:34:41.993 fio_dif_digest 00:34:41.993 fio_dif_rand_params 00:34:41.993 ftl_restore_fast 00:34:41.993 iscsi_tgt_data_digest 00:34:41.993 iscsi_tgt_ext4test 00:34:41.993 kernel_target_abort 00:34:41.993 keyring_file 00:34:41.993 keyring_linux 00:34:41.993 nvme_boot_partition 00:34:41.993 nvme_bp 00:34:41.993 nvme_opal 00:34:41.993 nvme_opal_bdevio 00:34:41.993 nvme_opal_bdevperf 00:34:41.993 nvme_opal_spdk_tgt 00:34:41.993 nvmf_abort_qd_sizes 00:34:41.993 nvmf_auth_host 00:34:41.993 nvmf_bdevperf 00:34:41.993 nvmf_device_removal_pci_remove 00:34:41.993 nvmf_device_removal_pci_remove_no_srq 00:34:41.993 nvmf_dif 00:34:41.993 nvmf_digest 00:34:41.993 nvmf_digest_clean 00:34:41.993 nvmf_digest_dsa_initiator 00:34:41.993 nvmf_digest_dsa_target 00:34:41.993 nvmf_digest_error 00:34:41.993 nvmf_discovery_remove_ifc 00:34:41.993 nvmf_failover 00:34:41.993 nvmf_fc 00:34:41.993 nvmf_fuzz 00:34:41.993 nvmf_host 00:34:41.993 nvmf_host_discovery 00:34:41.993 nvmf_host_multipath 00:34:41.993 nvmf_host_multipath_status 00:34:41.993 nvmf_identify_kernel_target 00:34:41.993 nvmf_identify_passthru 00:34:41.993 nvmf_initiator_timeout 00:34:41.993 nvmf_interrupt 00:34:41.993 nvmf_mdns_discovery 00:34:41.993 nvmf_multiconnection 00:34:41.993 nvmf_rdma 00:34:41.993 nvmf_target_core_interrupt_mode 00:34:41.993 nvmf_target_disconnect 00:34:41.993 nvmf_target_disconnect_tc1 00:34:41.993 nvmf_target_disconnect_tc2 00:34:41.993 nvmf_target_disconnect_tc3 00:34:41.993 nvmf_tcp 00:34:41.993 nvmf_timeout 00:34:41.993 spdk_target_abort 00:34:41.993 spdkcli_nvmf_fc 00:34:41.993 spdkcli_nvmf_rdma 00:34:41.993 spdkcli_nvmf_tcp 00:34:41.993 unittest_nvmf_fc 00:34:41.993 unittest_nvmf_fc_ls 00:34:41.993 vhost_blk_cores_2ctrl 00:34:41.993 vhost_blk_fs_integrity 00:34:41.993 vhost_blk_hot_remove 00:34:41.993 vhost_blk_integrity 00:34:41.993 vhost_hotplug 00:34:41.993 vhost_migration 00:34:41.993 vhost_migration_tc1 00:34:41.993 vhost_migration_tc2 00:34:41.993 vhost_readonly 00:34:41.993 vhost_scsi_cores_2ctrl 00:34:41.993 vhost_scsi_fs_integrity 00:34:41.993 vhost_scsi_hot_remove 00:34:41.993 vhost_scsi_integrity 00:34:41.993 vmd 00:34:41.993 vmd_bdev_svc 00:34:41.993 vmd_fio 00:34:41.993 vmd_hello_world 00:34:41.993 vmd_identify 00:34:41.993 vmd_perf 00:34:41.993 00:34:41.993 00:34:41.993 -----Tests Missing ASAN------ 00:34:41.993 accel_cdev_comp 00:34:41.993 accel_cdev_decomp 00:34:41.993 accel_cdev_decomp_full 00:34:41.993 accel_cdev_decomp_full_mcore 00:34:41.993 accel_cdev_decomp_full_mthread 00:34:41.993 accel_cdev_decomp_mcore 00:34:41.993 accel_cdev_decomp_mthread 00:34:41.993 accel_mlx5_copy 00:34:41.993 accel_mlx5_crc32c 00:34:41.993 accel_mlx5_crypto 00:34:41.993 accel_scan_dsa_modules 00:34:41.993 accel_scan_iaa_modules 00:34:41.993 acl 00:34:41.993 allowed 00:34:41.993 autobuild 00:34:41.993 autobuild_build_doc 00:34:41.993 autobuild_check_dpdk_pci_api 00:34:41.993 autobuild_check_format 00:34:41.993 autobuild_check_so_deps 00:34:41.993 autobuild_external_code 00:34:41.993 autobuild_generated_files_check 00:34:41.993 autobuild_generated_files_check_post_clean 00:34:41.993 autobuild_header_dependency_check 00:34:41.993 autobuild_llvm_precompile 00:34:41.993 autobuild_make_install 00:34:41.993 autobuild_make_uninstall 00:34:41.993 autobuild_ocf_precompile 00:34:41.993 balanced 00:34:41.993 bdev_crypto_enomem 00:34:41.993 blockdev_crypto_aesni 00:34:41.993 blockdev_crypto_qat 00:34:41.993 blockdev_crypto_sw 00:34:41.993 blockdev_rbd 00:34:41.993 build_native_dpdk 00:34:41.993 build_rpm_from_gen_spec 00:34:41.993 build_shared_native_dpdk_rpm 00:34:41.993 build_shared_rpm 00:34:41.993 build_shared_rpm_with_rpmed_dpdk 00:34:41.993 busy 00:34:41.993 chaining 00:34:41.993 chap_during_discovery 00:34:41.993 chap_mutual_auth 00:34:41.993 check_header_filenames 00:34:41.993 cmb_copy 00:34:41.993 compress_compdev 00:34:41.993 compress_isal 00:34:41.993 confirm_abi_deps 00:34:41.993 confirm_makefile_deps 00:34:41.993 core_isolating 00:34:41.993 core_load 00:34:41.993 custom_alloc 00:34:41.993 dd_uring_copy 00:34:41.993 denied 00:34:41.993 devices 00:34:41.993 dm_mount 00:34:41.993 dma 00:34:41.993 dpdk_governor 00:34:41.993 driver 00:34:41.993 even_2G_alloc 00:34:41.993 external_make_accel_driver_shared 00:34:41.993 external_make_accel_driver_static 00:34:41.993 external_make_accel_module_shared 00:34:41.993 external_make_accel_module_static 00:34:41.993 external_make_hello_bdev_shared_combo 00:34:41.993 external_make_hello_bdev_shared_iso 00:34:41.993 external_make_hello_bdev_static 00:34:41.993 external_make_hello_no_bdev_shared_combo 00:34:41.993 external_make_hello_no_bdev_shared_iso 00:34:41.993 external_make_hello_no_bdev_static 00:34:41.993 external_make_nvme_shared 00:34:41.993 external_make_nvme_static 00:34:41.993 external_run_accel_driver_shared 00:34:41.993 external_run_accel_driver_static 00:34:41.993 external_run_accel_module_shared 00:34:41.993 external_run_accel_module_static 00:34:41.993 external_run_hello_bdev_shared_combo 00:34:41.993 external_run_hello_bdev_shared_iso 00:34:41.993 external_run_hello_bdev_static 00:34:41.993 external_run_hello_no_bdev_shared_combo 00:34:41.993 external_run_hello_no_bdev_shared_iso 00:34:41.993 external_run_hello_no_bdev_static 00:34:41.993 external_run_nvme_shared 00:34:41.993 external_run_nvme_static 00:34:41.993 filesystem_btrfs 00:34:41.993 filesystem_ext4 00:34:41.993 filesystem_in_capsule_btrfs 00:34:41.993 filesystem_in_capsule_ext4 00:34:41.993 filesystem_in_capsule_xfs 00:34:41.993 filesystem_xfs 00:34:41.993 fio_dif_1_default 00:34:41.993 fio_dif_1_multi_subsystems 00:34:41.993 fio_dif_digest 00:34:41.993 fio_dif_rand_params 00:34:41.993 framework_get_governor 00:34:41.993 ftl_restore_fast 00:34:41.993 go_rpc 00:34:41.993 guess_driver 00:34:41.993 hugepages 00:34:41.993 idle 00:34:41.993 interrupt_mode 00:34:41.993 ioat 00:34:41.993 ioat_perf 00:34:41.993 ioat_verify 00:34:41.993 is_zoned 00:34:41.993 iscsi_tgt 00:34:41.993 iscsi_tgt_bdev_io_wait 00:34:41.993 iscsi_tgt_calsoft 00:34:41.993 iscsi_tgt_data_digest 00:34:41.993 iscsi_tgt_digest 00:34:41.993 iscsi_tgt_digests 00:34:41.993 iscsi_tgt_ext4test 00:34:41.993 iscsi_tgt_filesystem 00:34:41.993 iscsi_tgt_filesystem_btrfs 00:34:41.993 iscsi_tgt_filesystem_ext4 00:34:41.993 iscsi_tgt_filesystem_xfs 00:34:41.993 iscsi_tgt_fio 00:34:41.993 iscsi_tgt_fuzz 00:34:41.993 iscsi_tgt_initiator 00:34:41.993 iscsi_tgt_ip_migration 00:34:41.993 iscsi_tgt_iscsi_lvol 00:34:41.993 iscsi_tgt_login_redirection 00:34:41.993 iscsi_tgt_multiconnection 00:34:41.993 iscsi_tgt_qos 00:34:41.993 iscsi_tgt_rbd 00:34:41.993 iscsi_tgt_reset 00:34:41.993 iscsi_tgt_resize 00:34:41.993 iscsi_tgt_rpc_config 00:34:41.993 iscsi_tgt_sock 00:34:41.993 iscsi_tgt_trace_record 00:34:41.993 kernel_target_abort 00:34:41.993 keyring_file 00:34:41.993 keyring_linux 00:34:41.993 llvm_fuzz 00:34:41.993 lvs_grow_clean 00:34:41.993 lvs_grow_dirty 00:34:41.993 no_shrink_alloc 00:34:41.993 nvme_boot_partition 00:34:41.993 nvme_bp 00:34:41.993 nvme_cli_cuse 00:34:41.993 nvme_cli_plugin 00:34:41.993 nvme_cmb 00:34:41.993 nvme_cuse 00:34:41.993 nvme_cuse_app 00:34:41.993 nvme_cuse_rpc 00:34:41.993 nvme_mount 00:34:41.993 nvme_ns_manage_cuse 00:34:41.993 nvme_opal 00:34:41.993 nvme_opal_bdevio 00:34:41.993 nvme_opal_bdevperf 00:34:41.993 nvme_opal_spdk_tgt 00:34:41.993 nvme_pmr 00:34:41.993 nvme_pmr_persistence 00:34:41.993 nvme_smartctl_cuse 00:34:41.993 nvme_zns 00:34:41.993 nvmf_abort 00:34:41.993 nvmf_abort_qd_sizes 00:34:41.993 nvmf_aer 00:34:41.993 nvmf_async_init 00:34:41.993 nvmf_auth_host 00:34:41.993 nvmf_auth_target 00:34:41.993 nvmf_bdev_io_wait 00:34:41.993 nvmf_bdevio 00:34:41.993 nvmf_bdevio_no_huge 00:34:41.993 nvmf_bdevperf 00:34:41.993 nvmf_connect_disconnect 00:34:41.993 nvmf_connect_stress 00:34:41.993 nvmf_control_msg_list 00:34:41.993 nvmf_delete_subsystem 00:34:41.993 nvmf_device_removal_pci_remove 00:34:41.993 nvmf_device_removal_pci_remove_no_srq 00:34:41.993 nvmf_dif 00:34:41.993 nvmf_digest 00:34:41.993 nvmf_digest_clean 00:34:41.993 nvmf_digest_dsa_initiator 00:34:41.993 nvmf_digest_dsa_target 00:34:41.993 nvmf_digest_error 00:34:41.993 nvmf_discovery_remove_ifc 00:34:41.993 nvmf_example 00:34:41.993 nvmf_failover 00:34:41.994 nvmf_fc 00:34:41.994 nvmf_filesystem 00:34:41.994 nvmf_filesystem_in_capsule 00:34:41.994 nvmf_filesystem_no_in_capsule 00:34:41.994 nvmf_fio_host 00:34:41.994 nvmf_fio_target 00:34:41.994 nvmf_fips 00:34:41.994 nvmf_fused_ordering 00:34:41.994 nvmf_fuzz 00:34:41.994 nvmf_host 00:34:41.994 nvmf_host_discovery 00:34:41.994 nvmf_host_management 00:34:41.994 nvmf_host_multipath 00:34:41.994 nvmf_host_multipath_status 00:34:41.994 nvmf_identify 00:34:41.994 nvmf_identify_kernel_target 00:34:41.994 nvmf_identify_passthru 00:34:41.994 nvmf_initiator_timeout 00:34:41.994 nvmf_interrupt 00:34:41.994 nvmf_invalid 00:34:41.994 nvmf_llvm_fuzz 00:34:41.994 nvmf_lvol 00:34:41.994 nvmf_lvs_grow 00:34:41.994 nvmf_mdns_discovery 00:34:41.994 nvmf_multiconnection 00:34:41.994 nvmf_multicontroller 00:34:41.994 nvmf_multitarget 00:34:41.994 nvmf_nmic 00:34:41.994 nvmf_ns_hotplug_stress 00:34:41.994 nvmf_ns_masking 00:34:41.994 nvmf_nvme_cli 00:34:41.994 nvmf_perf 00:34:41.994 nvmf_perf_adq 00:34:41.994 nvmf_queue_depth 00:34:41.994 nvmf_rdma 00:34:41.994 nvmf_referrals 00:34:41.994 nvmf_rpc 00:34:41.994 nvmf_shutdown 00:34:41.994 nvmf_shutdown_tc1 00:34:41.994 nvmf_shutdown_tc2 00:34:41.994 nvmf_shutdown_tc3 00:34:41.994 nvmf_shutdown_tc4 00:34:41.994 nvmf_srq_overwhelm 00:34:41.994 nvmf_target_core 00:34:41.994 nvmf_target_core_interrupt_mode 00:34:41.994 nvmf_target_disconnect 00:34:41.994 nvmf_target_disconnect_tc1 00:34:41.994 nvmf_target_disconnect_tc2 00:34:41.994 nvmf_target_disconnect_tc3 00:34:41.994 nvmf_target_discovery 00:34:41.994 nvmf_target_extra 00:34:41.994 nvmf_target_multipath 00:34:41.994 nvmf_tcp 00:34:41.994 nvmf_timeout 00:34:41.994 nvmf_tls 00:34:41.994 nvmf_vfio_user 00:34:41.994 nvmf_vfio_user_fuzz 00:34:41.994 nvmf_vfio_user_nvme_compliance 00:34:41.994 nvmf_wait_for_buf 00:34:41.994 nvmf_zcopy 00:34:41.994 ocf 00:34:41.994 ocf_bdevperf_iotypes 00:34:41.994 ocf_configuration_change 00:34:41.994 ocf_create_destruct 00:34:41.994 ocf_fio_modes 00:34:41.994 ocf_flush 00:34:41.994 ocf_multicore 00:34:41.994 ocf_remove 00:34:41.994 ocf_stats 00:34:41.994 odd_alloc 00:34:41.994 packaging 00:34:41.994 rpm_packaging 00:34:41.994 scanbuild_make 00:34:41.994 scheduler 00:34:41.994 scheduler_opts 00:34:41.994 scheduler_rpc 00:34:41.994 setup.sh 00:34:41.994 single_node_setup 00:34:41.994 spdk_dd_uring 00:34:41.994 spdk_target_abort 00:34:41.994 spdkcli_iscsi 00:34:41.994 spdkcli_nvmf_fc 00:34:41.994 spdkcli_nvmf_rdma 00:34:41.994 spdkcli_nvmf_tcp 00:34:41.994 spdkcli_rbd 00:34:41.994 spdkcli_virtio 00:34:41.994 static_as_default 00:34:41.994 unittest_bdev_compress 00:34:41.994 unittest_bdev_crypto 00:34:41.994 unittest_bdev_raid5f 00:34:41.994 unittest_dpdk_compressdev 00:34:41.994 unittest_lib_reduce 00:34:41.994 unittest_nvmf_fc 00:34:41.994 unittest_nvmf_fc_ls 00:34:41.994 unittest_nvmf_vfio_user 00:34:41.994 valgrind 00:34:41.994 vfio_llvm_fuzz 00:34:41.994 vhost_blk_cores_2ctrl 00:34:41.994 vhost_blk_fs_integrity 00:34:41.994 vhost_blk_hot_remove 00:34:41.994 vhost_blk_integrity 00:34:41.994 vhost_blockdev 00:34:41.994 vhost_fuzz 00:34:41.994 vhost_hotplug 00:34:41.994 vhost_migration 00:34:41.994 vhost_migration_tc1 00:34:41.994 vhost_migration_tc2 00:34:41.994 vhost_readonly 00:34:41.994 vhost_scsi_cores_2ctrl 00:34:41.994 vhost_scsi_fs_integrity 00:34:41.994 vhost_scsi_hot_remove 00:34:41.994 vhost_scsi_integrity 00:34:41.994 vhost_shared 00:34:41.994 vmd 00:34:41.994 vmd_bdev_svc 00:34:41.994 vmd_fio 00:34:41.994 vmd_hello_world 00:34:41.994 vmd_identify 00:34:41.994 vmd_perf 00:34:41.994 zoned_fio 00:34:41.994 00:34:41.994 00:34:41.994 -----Tests Missing UBSAN------ 00:34:41.994 accel_mlx5_copy 00:34:41.994 accel_mlx5_crc32c 00:34:41.994 accel_mlx5_crypto 00:34:41.994 accel_scan_dsa_modules 00:34:41.994 accel_scan_iaa_modules 00:34:41.994 autobuild 00:34:41.994 autobuild_build_doc 00:34:41.994 autobuild_check_dpdk_pci_api 00:34:41.994 autobuild_check_format 00:34:41.994 autobuild_check_so_deps 00:34:41.994 autobuild_external_code 00:34:41.994 autobuild_generated_files_check 00:34:41.994 autobuild_generated_files_check_post_clean 00:34:41.994 autobuild_header_dependency_check 00:34:41.994 autobuild_make_install 00:34:41.994 autobuild_make_uninstall 00:34:41.994 balanced 00:34:41.994 build_native_dpdk 00:34:41.994 build_release 00:34:41.994 build_rpm_from_gen_spec 00:34:41.994 build_shared_native_dpdk_rpm 00:34:41.994 build_shared_rpm 00:34:41.994 build_shared_rpm_with_rpmed_dpdk 00:34:41.994 busy 00:34:41.994 check_header_filenames 00:34:41.994 cmb_copy 00:34:41.994 confirm_abi_deps 00:34:41.994 confirm_makefile_deps 00:34:41.994 core_load 00:34:41.994 external_make_accel_driver_shared 00:34:41.994 external_make_accel_driver_static 00:34:41.994 external_make_accel_module_shared 00:34:41.994 external_make_accel_module_static 00:34:41.994 external_make_hello_bdev_shared_combo 00:34:41.994 external_make_hello_bdev_shared_iso 00:34:41.994 external_make_hello_bdev_static 00:34:41.994 external_make_hello_no_bdev_shared_combo 00:34:41.994 external_make_hello_no_bdev_shared_iso 00:34:41.994 external_make_hello_no_bdev_static 00:34:41.994 external_make_nvme_shared 00:34:41.994 external_make_nvme_static 00:34:41.994 external_run_accel_driver_shared 00:34:41.994 external_run_accel_driver_static 00:34:41.994 external_run_accel_module_shared 00:34:41.994 external_run_accel_module_static 00:34:41.994 external_run_hello_bdev_shared_combo 00:34:41.994 external_run_hello_bdev_shared_iso 00:34:41.994 external_run_hello_bdev_static 00:34:41.994 external_run_hello_no_bdev_shared_combo 00:34:41.994 external_run_hello_no_bdev_shared_iso 00:34:41.994 external_run_hello_no_bdev_static 00:34:41.994 external_run_nvme_shared 00:34:41.994 external_run_nvme_static 00:34:41.994 fio_dif_1_default 00:34:41.994 fio_dif_1_multi_subsystems 00:34:41.994 fio_dif_digest 00:34:41.994 fio_dif_rand_params 00:34:41.994 ftl_restore_fast 00:34:41.994 is_zoned 00:34:41.994 iscsi_tgt_data_digest 00:34:41.994 iscsi_tgt_ext4test 00:34:41.994 kernel_target_abort 00:34:41.994 keyring_file 00:34:41.994 keyring_linux 00:34:41.994 nvme_boot_partition 00:34:41.994 nvme_bp 00:34:41.994 nvme_cmb 00:34:41.994 nvme_opal 00:34:41.994 nvme_opal_bdevio 00:34:41.994 nvme_opal_bdevperf 00:34:41.994 nvme_opal_spdk_tgt 00:34:41.994 nvme_pmr 00:34:41.994 nvme_pmr_persistence 00:34:41.994 nvme_zns 00:34:41.994 nvmf_abort_qd_sizes 00:34:41.994 nvmf_auth_host 00:34:41.994 nvmf_bdevperf 00:34:41.994 nvmf_device_removal_pci_remove 00:34:41.994 nvmf_device_removal_pci_remove_no_srq 00:34:41.994 nvmf_dif 00:34:41.994 nvmf_digest 00:34:41.994 nvmf_digest_clean 00:34:41.994 nvmf_digest_dsa_initiator 00:34:41.994 nvmf_digest_dsa_target 00:34:41.994 nvmf_digest_error 00:34:41.994 nvmf_discovery_remove_ifc 00:34:41.994 nvmf_failover 00:34:41.994 nvmf_fc 00:34:41.994 nvmf_fuzz 00:34:41.994 nvmf_host 00:34:41.994 nvmf_host_discovery 00:34:41.994 nvmf_host_multipath 00:34:41.994 nvmf_host_multipath_status 00:34:41.994 nvmf_identify_kernel_target 00:34:41.994 nvmf_identify_passthru 00:34:41.994 nvmf_initiator_timeout 00:34:41.994 nvmf_interrupt 00:34:41.994 nvmf_mdns_discovery 00:34:41.994 nvmf_multiconnection 00:34:41.994 nvmf_rdma 00:34:41.994 nvmf_target_core_interrupt_mode 00:34:41.994 nvmf_target_disconnect 00:34:41.994 nvmf_target_disconnect_tc1 00:34:41.994 nvmf_target_disconnect_tc2 00:34:41.994 nvmf_target_disconnect_tc3 00:34:41.994 nvmf_tcp 00:34:41.994 nvmf_timeout 00:34:41.994 packaging 00:34:41.994 rpm_packaging 00:34:41.994 scanbuild_make 00:34:41.994 spdk_target_abort 00:34:41.994 spdkcli_nvmf_fc 00:34:41.994 spdkcli_nvmf_rdma 00:34:41.994 spdkcli_nvmf_tcp 00:34:41.994 unittest_bdev_compress 00:34:41.994 unittest_bdev_crypto 00:34:41.994 unittest_bdev_raid5f 00:34:41.994 unittest_dpdk_compressdev 00:34:41.994 unittest_lib_reduce 00:34:41.994 unittest_nvmf_fc 00:34:41.994 unittest_nvmf_fc_ls 00:34:41.994 unittest_nvmf_vfio_user 00:34:41.994 valgrind 00:34:41.994 vhost_blk_cores_2ctrl 00:34:41.994 vhost_blk_fs_integrity 00:34:41.994 vhost_blk_hot_remove 00:34:41.994 vhost_blk_integrity 00:34:41.994 vhost_hotplug 00:34:41.994 vhost_migration 00:34:41.994 vhost_migration_tc1 00:34:41.994 vhost_migration_tc2 00:34:41.994 vhost_readonly 00:34:41.994 vhost_scsi_cores_2ctrl 00:34:41.994 vhost_scsi_fs_integrity 00:34:41.994 vhost_scsi_hot_remove 00:34:41.994 vhost_scsi_integrity 00:34:41.994 vmd 00:34:41.994 vmd_bdev_svc 00:34:41.994 vmd_fio 00:34:41.994 vmd_hello_world 00:34:41.994 vmd_identify 00:34:41.994 vmd_perf 00:34:41.994 zoned_fio 00:34:41.994 Not all tests were run. Failing the build. 00:34:41.994 ['fio_dif_1_default', 'fio_dif_1_multi_subsystems', 'fio_dif_digest', 'fio_dif_rand_params', 'kernel_target_abort', 'keyring_file', 'nvmf_abort_qd_sizes', 'nvmf_auth_host', 'nvmf_bdevperf', 'nvmf_dif', 'nvmf_digest', 'nvmf_digest_clean', 'nvmf_digest_error', 'nvmf_discovery_remove_ifc', 'nvmf_failover', 'nvmf_host', 'nvmf_host_discovery', 'nvmf_host_multipath', 'nvmf_host_multipath_status', 'nvmf_identify_kernel_target', 'nvmf_identify_passthru', 'nvmf_interrupt', 'nvmf_mdns_discovery', 'nvmf_rdma', 'nvmf_target_core_interrupt_mode', 'nvmf_target_disconnect', 'nvmf_target_disconnect_tc1', 'nvmf_target_disconnect_tc2', 'nvmf_target_disconnect_tc3', 'nvmf_tcp', 'nvmf_timeout', 'spdk_target_abort', 'spdkcli_nvmf_rdma', 'spdkcli_nvmf_tcp'] 00:34:41.994 + compress_coverage_and_docs 00:34:41.994 + echo 'Start compress coverage and docs' 00:34:41.994 Start compress coverage and docs 00:34:41.994 + tar -C coverage -czf coverage_autotest-per-patch_130870.tar.gz ./ --remove-files 00:34:43.371 + tar -C ut_coverage -czf ut_coverage_autotest-per-patch_130870.tar.gz ./ --remove-files 00:34:43.934 + tar -C doc -czf doc_autotest-per-patch_130870.tar.gz ./ --remove-files 00:34:44.190 + [[ -d llvm ]] 00:34:44.190 + true 00:34:44.190 + echo 'End compress coverage and docs' 00:34:44.190 End compress coverage and docs 00:34:44.190 + remove_partial_coverage_files 00:34:44.190 + find . -name 'cov_*.info' -delete 00:34:44.447 + echo 'End stage post_gen_coverage.sh' 00:34:44.447 End stage post_gen_coverage.sh 00:34:44.460 [Pipeline] } 00:34:44.463 ERROR: script returned exit code 1 00:34:44.487 [Pipeline] // catchError 00:34:44.495 [Pipeline] catchError 00:34:44.497 [Pipeline] { 00:34:44.514 [Pipeline] dir 00:34:44.515 Running in /var/jenkins/workspace/autotest-per-patch_3/post_process 00:34:44.517 [Pipeline] {