00:00:00.000 Started by user sys_sgci 00:00:00.009 Checking out git https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool into /var/jenkins_home/workspace/autotest-per-patch_script/33b20b30f0a51e6b52980845e0f6aa336787973ad45e341fbbf98d1b65b265d4 to read jbp/jenkins/jjb-config/jobs/autotest-upstream/autotest.groovy 00:00:00.010 The recommended git tool is: git 00:00:00.010 using credential 00000000-0000-0000-0000-000000000002 00:00:00.012 > git rev-parse --resolve-git-dir /var/jenkins_home/workspace/autotest-per-patch_script/33b20b30f0a51e6b52980845e0f6aa336787973ad45e341fbbf98d1b65b265d4/jbp/.git # timeout=10 00:00:00.024 Fetching changes from the remote Git repository 00:00:00.026 > git config remote.origin.url https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool # timeout=10 00:00:00.037 Using shallow fetch with depth 1 00:00:00.037 Fetching upstream changes from https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool 00:00:00.037 > git --version # timeout=10 00:00:00.047 > git --version # 'git version 2.39.2' 00:00:00.047 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:00.047 Setting http proxy: proxy-dmz.intel.com:911 00:00:00.048 > git fetch --tags --force --progress --depth=1 -- https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool refs/heads/master # timeout=5 00:00:02.128 > git rev-parse origin/FETCH_HEAD^{commit} # timeout=10 00:00:02.138 > git rev-parse FETCH_HEAD^{commit} # timeout=10 00:00:02.149 Checking out Revision c7986954d8037b9c61764d44ed2af24625b251c6 (FETCH_HEAD) 00:00:02.149 > git config core.sparsecheckout # timeout=10 00:00:02.158 > git read-tree -mu HEAD # timeout=10 00:00:02.171 > git checkout -f c7986954d8037b9c61764d44ed2af24625b251c6 # timeout=5 00:00:02.189 Commit message: "inventory/dev: add missing long names" 00:00:02.189 > git rev-list --no-walk c7986954d8037b9c61764d44ed2af24625b251c6 # timeout=10 00:00:02.452 [Pipeline] Start of Pipeline 00:00:02.464 [Pipeline] library 00:00:02.465 Loading library shm_lib@master 00:00:02.466 Library shm_lib@master is cached. Copying from home. 00:00:02.489 [Pipeline] node 00:00:02.497 Running on ME2 in /var/jenkins/workspace/autotest-per-patch_2 00:00:02.499 [Pipeline] { 00:00:02.512 [Pipeline] cleanWs 00:00:02.519 [WS-CLEANUP] Deleting project workspace... 00:00:02.519 [WS-CLEANUP] Deferred wipeout is used... 00:00:02.525 [WS-CLEANUP] done 00:00:02.530 [Pipeline] stage 00:00:02.534 [Pipeline] { (Prologue) 00:00:02.617 [Pipeline] withCredentials 00:00:02.623 > git --version # timeout=10 00:00:02.637 > git --version # 'git version 2.39.2' 00:00:02.649 Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD or $GIT_ASKPASS 00:00:02.650 [Pipeline] { 00:00:02.658 [Pipeline] retry 00:00:02.659 [Pipeline] { 00:00:02.810 [Pipeline] sh 00:00:03.081 + git ls-remote https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool refs/heads/master 00:00:04.985 [Pipeline] } 00:00:05.009 [Pipeline] // retry 00:00:05.014 [Pipeline] } 00:00:05.035 [Pipeline] // withCredentials 00:00:05.043 [Pipeline] nodesByLabel 00:00:05.045 Found a total of 1 nodes with the 'sorcerer' label 00:00:05.054 [Pipeline] httpRequest 00:00:05.058 HttpMethod: GET 00:00:05.059 URL: http://10.211.164.101/packages/jbp_c7986954d8037b9c61764d44ed2af24625b251c6.tar.gz 00:00:05.059 Sending request to url: http://10.211.164.101/packages/jbp_c7986954d8037b9c61764d44ed2af24625b251c6.tar.gz 00:00:05.060 Response Code: HTTP/1.1 200 OK 00:00:05.060 Success: Status code 200 is in the accepted range: 200,404 00:00:05.061 Saving response body to /var/jenkins/workspace/autotest-per-patch_2/jbp_c7986954d8037b9c61764d44ed2af24625b251c6.tar.gz 00:00:05.200 [Pipeline] sh 00:00:05.474 + tar --no-same-owner -xf jbp_c7986954d8037b9c61764d44ed2af24625b251c6.tar.gz 00:00:05.496 [Pipeline] httpRequest 00:00:05.500 HttpMethod: GET 00:00:05.501 URL: http://10.211.164.101/packages/spdk_56756573693b99ea74c7bdacee713bdbf151966c.tar.gz 00:00:05.501 Sending request to url: http://10.211.164.101/packages/spdk_56756573693b99ea74c7bdacee713bdbf151966c.tar.gz 00:00:05.502 Response Code: HTTP/1.1 404 Not Found 00:00:05.502 Success: Status code 404 is in the accepted range: 200,404 00:00:05.502 Saving response body to /var/jenkins/workspace/autotest-per-patch_2/spdk_56756573693b99ea74c7bdacee713bdbf151966c.tar.gz 00:00:05.512 [Pipeline] sh 00:00:05.786 + rm -f spdk_56756573693b99ea74c7bdacee713bdbf151966c.tar.gz 00:00:05.801 [Pipeline] node 00:00:05.809 Running on GP10 in /var/jenkins/workspace/autotest-per-patch 00:00:05.817 [Pipeline] { 00:00:05.835 [Pipeline] cleanWs 00:00:05.846 [WS-CLEANUP] Deleting project workspace... 00:00:05.846 [WS-CLEANUP] Deferred wipeout is used... 00:00:05.853 [WS-CLEANUP] done 00:00:05.858 [Pipeline] sh 00:00:06.146 + find /storage/packages/ -type f -mtime +34 -print -delete 00:00:06.160 [Pipeline] retry 00:00:06.162 [Pipeline] { 00:00:06.186 [Pipeline] checkout 00:00:06.194 The recommended git tool is: NONE 00:00:06.208 using credential 00000000-0000-0000-0000-000000000002 00:00:06.214 Cloning the remote Git repository 00:00:06.218 Honoring refspec on initial clone 00:00:06.222 Cloning repository https://review.spdk.io/gerrit/a/spdk/spdk 00:00:06.222 > git init /var/jenkins/workspace/autotest-per-patch/spdk # timeout=10 00:00:06.230 Using reference repository: /var/ci_repos/spdk_multi 00:00:06.230 Fetching upstream changes from https://review.spdk.io/gerrit/a/spdk/spdk 00:00:06.230 > git --version # timeout=10 00:00:06.233 > git --version # 'git version 2.42.0' 00:00:06.234 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:06.234 Setting http proxy: proxy-dmz.intel.com:911 00:00:06.234 > git fetch --tags --force --progress -- https://review.spdk.io/gerrit/a/spdk/spdk refs/changes/89/21689/13 +refs/heads/master:refs/remotes/origin/master # timeout=10 00:00:37.834 Avoid second fetch 00:00:37.850 Checking out Revision 56756573693b99ea74c7bdacee713bdbf151966c (FETCH_HEAD) 00:00:38.161 Commit message: "blob: add blob set external parent" 00:00:38.169 First time build. Skipping changelog. 00:00:37.815 > git config remote.origin.url https://review.spdk.io/gerrit/a/spdk/spdk # timeout=10 00:00:37.819 > git config --add remote.origin.fetch refs/changes/89/21689/13 # timeout=10 00:00:37.823 > git config --add remote.origin.fetch +refs/heads/master:refs/remotes/origin/master # timeout=10 00:00:37.837 > git rev-parse origin/FETCH_HEAD^{commit} # timeout=10 00:00:37.845 > git rev-parse FETCH_HEAD^{commit} # timeout=10 00:00:37.853 > git config core.sparsecheckout # timeout=10 00:00:37.857 > git checkout -f 56756573693b99ea74c7bdacee713bdbf151966c # timeout=10 00:00:38.163 > git rev-list --no-walk 0c05a347642af0532b123998aee39a9003b3b3ab # timeout=10 00:00:38.175 > git remote # timeout=10 00:00:38.179 > git submodule init # timeout=10 00:00:38.227 > git submodule sync # timeout=10 00:00:38.273 > git config --get remote.origin.url # timeout=10 00:00:38.280 > git submodule init # timeout=10 00:00:38.324 > git config -f .gitmodules --get-regexp ^submodule\.(.+)\.url # timeout=10 00:00:38.328 > git config --get submodule.dpdk.url # timeout=10 00:00:38.331 > git remote # timeout=10 00:00:38.335 > git config --get remote.origin.url # timeout=10 00:00:38.338 > git config -f .gitmodules --get submodule.dpdk.path # timeout=10 00:00:38.342 > git config --get submodule.intel-ipsec-mb.url # timeout=10 00:00:38.345 > git remote # timeout=10 00:00:38.349 > git config --get remote.origin.url # timeout=10 00:00:38.353 > git config -f .gitmodules --get submodule.intel-ipsec-mb.path # timeout=10 00:00:38.356 > git config --get submodule.isa-l.url # timeout=10 00:00:38.359 > git remote # timeout=10 00:00:38.363 > git config --get remote.origin.url # timeout=10 00:00:38.367 > git config -f .gitmodules --get submodule.isa-l.path # timeout=10 00:00:38.370 > git config --get submodule.ocf.url # timeout=10 00:00:38.374 > git remote # timeout=10 00:00:38.377 > git config --get remote.origin.url # timeout=10 00:00:38.380 > git config -f .gitmodules --get submodule.ocf.path # timeout=10 00:00:38.384 > git config --get submodule.libvfio-user.url # timeout=10 00:00:38.388 > git remote # timeout=10 00:00:38.391 > git config --get remote.origin.url # timeout=10 00:00:38.395 > git config -f .gitmodules --get submodule.libvfio-user.path # timeout=10 00:00:38.398 > git config --get submodule.xnvme.url # timeout=10 00:00:38.402 > git remote # timeout=10 00:00:38.406 > git config --get remote.origin.url # timeout=10 00:00:38.409 > git config -f .gitmodules --get submodule.xnvme.path # timeout=10 00:00:38.413 > git config --get submodule.isa-l-crypto.url # timeout=10 00:00:38.416 > git remote # timeout=10 00:00:38.420 > git config --get remote.origin.url # timeout=10 00:00:38.424 > git config -f .gitmodules --get submodule.isa-l-crypto.path # timeout=10 00:00:38.427 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:38.427 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:38.428 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:38.428 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:38.428 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:38.428 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:38.428 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:38.428 Setting http proxy: proxy-dmz.intel.com:911 00:00:38.428 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi dpdk # timeout=10 00:00:38.428 Setting http proxy: proxy-dmz.intel.com:911 00:00:38.428 Setting http proxy: proxy-dmz.intel.com:911 00:00:38.428 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi isa-l # timeout=10 00:00:38.428 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi intel-ipsec-mb # timeout=10 00:00:38.429 Setting http proxy: proxy-dmz.intel.com:911 00:00:38.429 Setting http proxy: proxy-dmz.intel.com:911 00:00:38.429 Setting http proxy: proxy-dmz.intel.com:911 00:00:38.429 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi libvfio-user # timeout=10 00:00:38.429 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi ocf # timeout=10 00:00:38.429 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi xnvme # timeout=10 00:00:38.429 Setting http proxy: proxy-dmz.intel.com:911 00:00:38.429 > git submodule update --init --recursive --reference /var/ci_repos/spdk_multi isa-l-crypto # timeout=10 00:00:48.262 [Pipeline] dir 00:00:48.263 Running in /var/jenkins/workspace/autotest-per-patch/spdk 00:00:48.272 [Pipeline] { 00:00:48.294 [Pipeline] sh 00:00:48.583 ++ nproc 00:00:48.583 + threads=48 00:00:48.583 + git repack -a -d --threads=48 00:00:56.719 + git submodule foreach git repack -a -d --threads=48 00:00:56.719 Entering 'dpdk' 00:01:00.905 Entering 'intel-ipsec-mb' 00:01:00.905 Entering 'isa-l' 00:01:01.164 Entering 'isa-l-crypto' 00:01:01.164 Entering 'libvfio-user' 00:01:01.422 Entering 'ocf' 00:01:01.681 Entering 'xnvme' 00:01:01.941 + find .git -type f -name alternates -print -delete 00:01:01.941 .git/modules/libvfio-user/objects/info/alternates 00:01:01.941 .git/modules/dpdk/objects/info/alternates 00:01:01.941 .git/modules/ocf/objects/info/alternates 00:01:01.941 .git/modules/intel-ipsec-mb/objects/info/alternates 00:01:01.941 .git/modules/isa-l/objects/info/alternates 00:01:01.941 .git/modules/xnvme/objects/info/alternates 00:01:01.941 .git/modules/isa-l-crypto/objects/info/alternates 00:01:01.941 .git/objects/info/alternates 00:01:01.956 [Pipeline] } 00:01:01.983 [Pipeline] // dir 00:01:01.989 [Pipeline] } 00:01:02.014 [Pipeline] // retry 00:01:02.023 [Pipeline] sh 00:01:02.309 + tar -cf spdk_56756573693b99ea74c7bdacee713bdbf151966c.tar.gz -I pigz spdk 00:01:02.893 [Pipeline] httpRequest 00:01:02.900 HttpMethod: PUT 00:01:02.901 URL: http://10.211.164.101/cgi-bin/sorcerer.py?group=packages&filename=spdk_56756573693b99ea74c7bdacee713bdbf151966c.tar.gz 00:01:02.903 Sending request to url: http://10.211.164.101/cgi-bin/sorcerer.py?group=packages&filename=spdk_56756573693b99ea74c7bdacee713bdbf151966c.tar.gz 00:01:03.760 Response Code: HTTP/1.1 200 OK 00:01:03.765 Success: Status code 200 is in the accepted range: 200 00:01:03.769 [Pipeline] echo 00:01:03.770 00:01:03.770 Locking 00:01:03.770 Waited 0s for lock 00:01:03.770 Everything Fine. Saved: /storage/packages/spdk_56756573693b99ea74c7bdacee713bdbf151966c.tar.gz 00:01:03.770 00:01:03.776 [Pipeline] cleanWs 00:01:03.784 [WS-CLEANUP] Deleting project workspace... 00:01:03.784 [WS-CLEANUP] Deferred wipeout is used... 00:01:03.789 [WS-CLEANUP] done 00:01:03.793 [Pipeline] } 00:01:03.825 [Pipeline] // node 00:01:03.836 [Pipeline] retry 00:01:03.839 [Pipeline] { 00:01:03.860 [Pipeline] httpRequest 00:01:03.864 HttpMethod: GET 00:01:03.865 URL: http://10.211.164.101/packages/spdk_56756573693b99ea74c7bdacee713bdbf151966c.tar.gz 00:01:03.866 Sending request to url: http://10.211.164.101/packages/spdk_56756573693b99ea74c7bdacee713bdbf151966c.tar.gz 00:01:03.869 Response Code: HTTP/1.1 200 OK 00:01:03.869 Success: Status code 200 is in the accepted range: 200,404 00:01:03.870 Saving response body to /var/jenkins/workspace/autotest-per-patch_2/spdk_56756573693b99ea74c7bdacee713bdbf151966c.tar.gz 00:01:08.002 [Pipeline] } 00:01:08.024 [Pipeline] // retry 00:01:08.031 [Pipeline] sh 00:01:08.344 + tar --no-same-owner -xf spdk_56756573693b99ea74c7bdacee713bdbf151966c.tar.gz 00:01:10.886 [Pipeline] sh 00:01:11.169 + git -C spdk log --oneline -n5 00:01:11.169 567565736 blob: add blob set external parent 00:01:11.169 0e4f7fc9b blob: add blob set parent 00:01:11.169 4506c0c36 test/common: Enable inherit_errexit 00:01:11.169 b24df7cfa test: Drop superfluous calls to print_backtrace() 00:01:11.169 7b52e4c17 test/scheduler: Meassure utime of $spdk_pid threads as a fallback 00:01:11.188 [Pipeline] setCustomBuildProperty 00:01:11.197 [Pipeline] setCustomBuildProperty 00:01:11.203 [Pipeline] catchError 00:01:11.205 [Pipeline] { 00:01:11.220 [Pipeline] sh 00:01:11.501 + git -C spdk describe --tags --abbrev=0 origin/master 00:01:11.517 [Pipeline] sh 00:01:11.803 + git -C spdk describe --tags --abbrev=0 --exclude=LTS HEAD 00:01:11.820 [Pipeline] echo 00:01:11.821 Branch: master 00:01:11.825 [Pipeline] fileExists 00:01:11.841 [Pipeline] readJSON 00:01:11.855 [Pipeline] } 00:01:11.881 [Pipeline] // catchError 00:01:11.886 [Pipeline] } 00:01:11.911 [Pipeline] // stage 00:01:11.928 [Pipeline] catchError 00:01:11.929 [Pipeline] { 00:01:11.949 [Pipeline] stage 00:01:11.951 [Pipeline] { (Pre tests) 00:01:11.995 [Pipeline] parallel 00:01:12.007 [Pipeline] { (Branch: check-format-docker-autotest) 00:01:12.009 [Pipeline] { (Branch: check-so-deps-docker-autotest) 00:01:12.010 [Pipeline] { (Branch: doc-docker-autotest) 00:01:12.010 [Pipeline] { (Branch: build-files-docker-autotest) 00:01:12.032 [Pipeline] retry 00:01:12.033 [Pipeline] { 00:01:12.035 [Pipeline] retry 00:01:12.037 [Pipeline] { 00:01:12.040 [Pipeline] retry 00:01:12.041 [Pipeline] { 00:01:12.044 [Pipeline] retry 00:01:12.045 [Pipeline] { 00:01:12.063 [Pipeline] build 00:01:12.065 Scheduling project: check-format-docker-autotest 00:01:12.070 [Pipeline] build 00:01:12.071 Scheduling project: check-so-deps-docker-autotest 00:01:12.076 [Pipeline] build 00:01:12.078 Scheduling project: doc-docker-autotest 00:01:12.082 [Pipeline] build 00:01:12.084 Scheduling project: build-files-docker-autotest 00:01:20.122 Starting building: doc-docker-autotest #22336 00:01:20.125 Starting building: check-format-docker-autotest #22156 00:01:20.128 Starting building: build-files-docker-autotest #22145 00:01:20.131 Starting building: check-so-deps-docker-autotest #22169 00:01:54.768 Build doc-docker-autotest #22336 completed: SUCCESS 00:01:54.771 [Pipeline] } 00:01:54.807 [Pipeline] // retry 00:01:54.813 [Pipeline] } 00:02:07.820 Build check-format-docker-autotest #22156 completed: SUCCESS 00:02:07.824 [Pipeline] } 00:02:07.862 [Pipeline] // retry 00:02:07.871 [Pipeline] } 00:03:32.461 Build build-files-docker-autotest #22145 completed: SUCCESS 00:03:32.466 [Pipeline] } 00:03:32.502 [Pipeline] // retry 00:03:32.508 [Pipeline] } 00:04:07.365 Build check-so-deps-docker-autotest #22169 completed: SUCCESS 00:04:07.370 [Pipeline] } 00:04:07.399 [Pipeline] // retry 00:04:07.405 [Pipeline] } 00:04:07.459 [Pipeline] // parallel 00:04:07.465 [Pipeline] } 00:04:07.485 [Pipeline] // stage 00:04:07.491 [Pipeline] } 00:04:07.515 [Pipeline] // catchError 00:04:07.524 [Pipeline] catchError 00:04:07.526 [Pipeline] { 00:04:07.548 [Pipeline] stage 00:04:07.550 [Pipeline] { (Tests) 00:04:07.746 [Pipeline] parallel 00:04:07.848 [Pipeline] { (Branch: unittest-clang-docker-autotest) 00:04:07.850 [Pipeline] { (Branch: unittest-main-docker-autotest) 00:04:07.851 [Pipeline] { (Branch: autobuild-external-vg-autotest) 00:04:07.852 [Pipeline] { (Branch: scan-build-docker-autotest) 00:04:07.853 [Pipeline] { (Branch: unittest-valgrind-docker-autotest) 00:04:07.855 [Pipeline] { (Branch: lvol-vg-autotest) 00:04:07.856 [Pipeline] { (Branch: vhost-initiator-vg-autotest) 00:04:07.857 [Pipeline] { (Branch: nvmf-tcp-vg-autotest) 00:04:07.858 [Pipeline] { (Branch: nvmf-tcp-uring-vg-autotest) 00:04:07.860 [Pipeline] { (Branch: nvme-vg-autotest) 00:04:07.861 [Pipeline] { (Branch: centos7-vg-autotest) 00:04:07.862 [Pipeline] { (Branch: rocky9-vg-autotest) 00:04:07.863 [Pipeline] { (Branch: ubuntu20-vg-autotest) 00:04:07.864 [Pipeline] { (Branch: ubuntu22-vg-autotest) 00:04:07.865 [Pipeline] { (Branch: freebsd-vg-autotest) 00:04:07.866 [Pipeline] { (Branch: iscsi-vg-autotest) 00:04:07.867 [Pipeline] { (Branch: iscsi-uring-vg-autotest) 00:04:07.868 [Pipeline] { (Branch: zns-vg-autotest) 00:04:07.869 [Pipeline] { (Branch: nvme-cmb-pmr-vg-autotest) 00:04:07.871 [Pipeline] { (Branch: BlobFS-phy-autotest) 00:04:07.872 [Pipeline] { (Branch: crypto-phy-autotest) 00:04:07.873 [Pipeline] { (Branch: vhost-phy-autotest) 00:04:07.874 [Pipeline] { (Branch: nvmf-phy-autotest) 00:04:07.875 [Pipeline] { (Branch: nvmf-tcp-phy-autotest) 00:04:07.876 [Pipeline] { (Branch: nvme-phy-autotest) 00:04:07.877 [Pipeline] { (Branch: vfio-user-phy-autotest) 00:04:07.878 [Pipeline] { (Branch: short-fuzz-phy-autotest) 00:04:07.879 [Pipeline] { (Branch: dsa-phy-autotest) 00:04:07.880 [Pipeline] { (Branch: packaging-docker-autotest) 00:04:07.912 [Pipeline] retry 00:04:07.914 [Pipeline] { 00:04:07.920 [Pipeline] retry 00:04:07.921 [Pipeline] { 00:04:07.925 [Pipeline] retry 00:04:07.926 [Pipeline] { 00:04:07.930 [Pipeline] retry 00:04:07.931 [Pipeline] { 00:04:07.934 [Pipeline] retry 00:04:07.936 [Pipeline] { 00:04:07.939 [Pipeline] retry 00:04:07.940 [Pipeline] { 00:04:07.944 [Pipeline] retry 00:04:07.945 [Pipeline] { 00:04:07.949 [Pipeline] retry 00:04:07.950 [Pipeline] { 00:04:07.954 [Pipeline] retry 00:04:07.955 [Pipeline] { 00:04:07.958 [Pipeline] retry 00:04:07.959 [Pipeline] { 00:04:07.963 [Pipeline] retry 00:04:07.964 [Pipeline] { 00:04:07.968 [Pipeline] retry 00:04:07.969 [Pipeline] { 00:04:07.973 [Pipeline] retry 00:04:07.974 [Pipeline] { 00:04:07.978 [Pipeline] retry 00:04:07.979 [Pipeline] { 00:04:07.983 [Pipeline] retry 00:04:07.985 [Pipeline] { 00:04:07.990 [Pipeline] retry 00:04:07.994 [Pipeline] { 00:04:07.999 [Pipeline] retry 00:04:08.000 [Pipeline] { 00:04:08.004 [Pipeline] retry 00:04:08.005 [Pipeline] { 00:04:08.008 [Pipeline] retry 00:04:08.009 [Pipeline] { 00:04:08.012 [Pipeline] retry 00:04:08.013 [Pipeline] { 00:04:08.017 [Pipeline] retry 00:04:08.018 [Pipeline] { 00:04:08.021 [Pipeline] retry 00:04:08.022 [Pipeline] { 00:04:08.025 [Pipeline] retry 00:04:08.026 [Pipeline] { 00:04:08.029 [Pipeline] retry 00:04:08.030 [Pipeline] { 00:04:08.033 [Pipeline] retry 00:04:08.035 [Pipeline] { 00:04:08.038 [Pipeline] retry 00:04:08.039 [Pipeline] { 00:04:08.042 [Pipeline] retry 00:04:08.044 [Pipeline] { 00:04:08.047 [Pipeline] retry 00:04:08.049 [Pipeline] { 00:04:08.052 [Pipeline] retry 00:04:08.053 [Pipeline] { 00:04:08.078 [Pipeline] build 00:04:08.081 Scheduling project: unittest-clang-docker-autotest 00:04:08.089 [Pipeline] build 00:04:08.092 Scheduling project: unittest-main-docker-autotest 00:04:08.100 [Pipeline] build 00:04:08.103 Scheduling project: autobuild-external-vg-autotest 00:04:08.111 [Pipeline] build 00:04:08.114 Scheduling project: scan-build-docker-autotest 00:04:08.122 [Pipeline] build 00:04:08.125 Scheduling project: unittest-valgrind-docker-autotest 00:04:08.132 [Pipeline] build 00:04:08.135 Scheduling project: lvol-vg-autotest 00:04:08.143 [Pipeline] build 00:04:08.145 Scheduling project: vhost-initiator-vg-autotest 00:04:08.153 [Pipeline] build 00:04:08.155 Scheduling project: nvmf-tcp-vg-autotest 00:04:08.162 [Pipeline] build 00:04:08.164 Scheduling project: nvmf-tcp-uring-vg-autotest 00:04:08.172 [Pipeline] build 00:04:08.175 Scheduling project: nvme-vg-autotest 00:04:08.183 [Pipeline] build 00:04:08.185 Scheduling project: centos7-vg-autotest 00:04:08.192 [Pipeline] build 00:04:08.194 Scheduling project: rocky9-vg-autotest 00:04:08.200 [Pipeline] build 00:04:08.203 Scheduling project: ubuntu20-vg-autotest 00:04:08.210 [Pipeline] build 00:04:08.212 Scheduling project: ubuntu22-vg-autotest 00:04:08.218 [Pipeline] build 00:04:08.220 Scheduling project: freebsd-vg-autotest 00:04:08.227 [Pipeline] build 00:04:08.229 Scheduling project: iscsi-vg-autotest 00:04:08.235 [Pipeline] build 00:04:08.237 Scheduling project: iscsi-uring-vg-autotest 00:04:08.243 [Pipeline] build 00:04:08.245 Scheduling project: zns-vg-autotest 00:04:08.251 [Pipeline] build 00:04:08.254 Scheduling project: nvme-cmb-pmr-vg-autotest 00:04:08.261 [Pipeline] build 00:04:08.263 Scheduling project: BlobFS-phy-autotest 00:04:08.269 [Pipeline] build 00:04:08.271 Scheduling project: crypto-phy-autotest 00:04:08.276 [Pipeline] build 00:04:08.278 Scheduling project: vhost-phy-autotest 00:04:08.283 [Pipeline] build 00:04:08.285 Scheduling project: nvmf-phy-autotest 00:04:08.290 [Pipeline] build 00:04:08.292 Scheduling project: nvmf-tcp-phy-autotest 00:04:08.297 [Pipeline] build 00:04:08.299 Scheduling project: nvme-phy-autotest 00:04:08.304 [Pipeline] build 00:04:08.306 Scheduling project: vfio-user-phy-autotest 00:04:08.311 [Pipeline] build 00:04:08.313 Scheduling project: short-fuzz-phy-autotest 00:04:08.318 [Pipeline] build 00:04:08.320 Scheduling project: dsa-phy-autotest 00:04:08.325 [Pipeline] build 00:04:08.327 Scheduling project: packaging-docker-autotest 00:04:15.385 Starting building: vhost-phy-autotest #26237 00:04:15.388 Starting building: crypto-phy-autotest #26212 00:04:15.392 Starting building: short-fuzz-phy-autotest #26209 00:04:15.394 Starting building: vfio-user-phy-autotest #26220 00:04:15.397 Starting building: unittest-clang-docker-autotest #19873 00:04:15.400 Starting building: unittest-valgrind-docker-autotest #19877 00:04:15.402 Starting building: autobuild-external-vg-autotest #19836 00:04:15.405 Starting building: unittest-main-docker-autotest #19874 00:04:15.409 Starting building: rocky9-vg-autotest #16102 00:04:15.412 Starting building: dsa-phy-autotest #4912 00:04:15.416 Starting building: ubuntu20-vg-autotest #73811 00:04:15.420 Starting building: BlobFS-phy-autotest #26218 00:04:15.425 Starting building: iscsi-vg-autotest #77530 00:04:15.432 Starting building: nvme-cmb-pmr-vg-autotest #78615 00:04:15.440 Starting building: freebsd-vg-autotest #59136 00:04:15.444 Starting building: scan-build-docker-autotest #19876 00:04:15.447 Starting building: packaging-docker-autotest #19875 00:04:15.450 Starting building: lvol-vg-autotest #77441 00:04:15.461 Starting building: nvmf-tcp-vg-autotest #77483 00:04:15.467 Starting building: nvme-phy-autotest #72384 00:04:15.472 Starting building: centos7-vg-autotest #73671 00:04:15.475 Starting building: vhost-initiator-vg-autotest #77512 00:04:15.478 Starting building: nvmf-tcp-phy-autotest #72343 00:04:15.481 Starting building: nvme-vg-autotest #77492 00:04:15.484 Starting building: nvmf-phy-autotest #72456 00:04:15.486 Starting building: zns-vg-autotest #76598 00:04:15.490 Starting building: iscsi-uring-vg-autotest #77397 00:04:15.493 Starting building: nvmf-tcp-uring-vg-autotest #77417 00:04:15.495 Starting building: ubuntu22-vg-autotest #37825 00:07:24.325 Build unittest-clang-docker-autotest #19873 completed: SUCCESS 00:07:24.327 [Pipeline] } 00:07:24.391 [Pipeline] // retry 00:07:24.399 [Pipeline] } 00:08:02.532 Build packaging-docker-autotest #19875 completed: SUCCESS 00:08:02.536 [Pipeline] } 00:08:02.588 [Pipeline] // retry 00:08:02.593 [Pipeline] } 00:08:48.989 Build scan-build-docker-autotest #19876 completed: SUCCESS 00:08:48.993 [Pipeline] } 00:08:49.041 [Pipeline] // retry 00:08:49.047 [Pipeline] } 00:10:01.461 Build unittest-main-docker-autotest #19874 completed: SUCCESS 00:10:01.463 [Pipeline] } 00:10:01.493 [Pipeline] // retry 00:10:01.497 [Pipeline] } 00:13:33.759 Build unittest-valgrind-docker-autotest #19877 completed: SUCCESS 00:13:33.761 [Pipeline] } 00:13:33.800 [Pipeline] // retry 00:13:33.805 [Pipeline] } 00:13:49.313 Build short-fuzz-phy-autotest #26209 completed: SUCCESS 00:13:49.316 [Pipeline] } 00:13:49.350 [Pipeline] // retry 00:13:49.356 [Pipeline] } 00:16:15.303 Build autobuild-external-vg-autotest #19836 completed: SUCCESS 00:16:15.305 [Pipeline] } 00:16:15.335 [Pipeline] // retry 00:16:15.341 [Pipeline] } 00:18:35.411 Build vhost-phy-autotest #26237 completed: SUCCESS 00:18:35.414 [Pipeline] } 00:18:35.464 [Pipeline] // retry 00:18:35.470 [Pipeline] } 00:21:03.583 Build vhost-initiator-vg-autotest #77512 completed: SUCCESS 00:21:03.586 [Pipeline] } 00:21:03.634 [Pipeline] // retry 00:21:03.641 [Pipeline] } 00:21:53.502 Build zns-vg-autotest #76598 completed: SUCCESS 00:21:53.504 [Pipeline] } 00:21:53.547 [Pipeline] // retry 00:21:53.555 [Pipeline] } 00:22:35.370 Build nvme-cmb-pmr-vg-autotest #78615 completed: SUCCESS 00:22:35.372 [Pipeline] } 00:22:35.417 [Pipeline] // retry 00:22:35.423 [Pipeline] } 00:24:08.064 Build vfio-user-phy-autotest #26220 completed: SUCCESS 00:24:08.066 [Pipeline] } 00:24:08.095 [Pipeline] // retry 00:24:08.100 [Pipeline] } 00:24:58.233 Build nvme-vg-autotest #77492 completed: FAILURE 00:24:59.066 [Pipeline] echo 00:24:59.067 No retry patterns found. 00:24:59.068 [Pipeline] } 00:24:59.101 [Pipeline] // retry 00:24:59.110 [Pipeline] error 00:24:59.117 [Pipeline] } 00:24:59.123 Failed in branch nvme-vg-autotest 00:25:05.949 Build BlobFS-phy-autotest #26218 completed: SUCCESS 00:25:05.951 [Pipeline] } 00:25:05.994 [Pipeline] // retry 00:25:06.001 [Pipeline] } 00:25:06.521 Build nvme-phy-autotest #72384 completed: SUCCESS 00:25:06.523 [Pipeline] } 00:25:06.577 [Pipeline] // retry 00:25:06.582 [Pipeline] } 00:25:20.318 Build lvol-vg-autotest #77441 completed: SUCCESS 00:25:20.320 [Pipeline] } 00:25:20.376 [Pipeline] // retry 00:25:20.383 [Pipeline] } 00:26:14.184 Build iscsi-vg-autotest #77530 completed: SUCCESS 00:26:14.187 [Pipeline] } 00:26:14.221 [Pipeline] // retry 00:26:14.229 [Pipeline] } 00:27:01.501 Build freebsd-vg-autotest #59136 completed: SUCCESS 00:27:01.504 [Pipeline] } 00:27:01.556 [Pipeline] // retry 00:27:01.561 [Pipeline] } 00:28:54.338 Build iscsi-uring-vg-autotest #77397 completed: SUCCESS 00:28:54.341 [Pipeline] } 00:28:54.388 [Pipeline] // retry 00:28:54.394 [Pipeline] } 00:30:54.095 Build nvmf-phy-autotest #72456 completed: SUCCESS 00:30:54.097 [Pipeline] } 00:30:54.142 [Pipeline] // retry 00:30:54.150 [Pipeline] } 00:32:31.582 Build nvmf-tcp-phy-autotest #72343 completed: SUCCESS 00:32:31.585 [Pipeline] } 00:32:31.632 [Pipeline] // retry 00:32:31.639 [Pipeline] } 00:36:14.745 Build crypto-phy-autotest #26212 completed: SUCCESS 00:36:14.747 [Pipeline] } 00:36:14.790 [Pipeline] // retry 00:36:14.797 [Pipeline] } 00:36:29.588 Build nvmf-tcp-uring-vg-autotest #77417 completed: SUCCESS 00:36:29.591 [Pipeline] } 00:36:29.635 [Pipeline] // retry 00:36:29.641 [Pipeline] } 00:37:10.980 Build dsa-phy-autotest #4912 completed: SUCCESS 00:37:10.982 [Pipeline] } 00:37:11.028 [Pipeline] // retry 00:37:11.035 [Pipeline] } 00:37:14.926 Build nvmf-tcp-vg-autotest #77483 completed: SUCCESS 00:37:14.928 [Pipeline] } 00:37:14.986 [Pipeline] // retry 00:37:14.994 [Pipeline] } 00:40:08.086 Build centos7-vg-autotest #73671 completed: SUCCESS 00:40:08.089 [Pipeline] } 00:40:08.145 [Pipeline] // retry 00:40:08.149 [Pipeline] } 00:49:59.882 Build rocky9-vg-autotest #16102 completed: SUCCESS 00:49:59.885 [Pipeline] } 00:49:59.930 [Pipeline] // retry 00:49:59.937 [Pipeline] } 00:58:36.563 Build ubuntu20-vg-autotest #73811 completed: FAILURE 00:58:38.643 [Pipeline] echo 00:58:38.644 No retry patterns found. 00:58:38.645 [Pipeline] } 00:58:38.668 [Pipeline] // retry 00:58:38.673 [Pipeline] error 00:58:38.677 [Pipeline] } 00:58:38.679 Failed in branch ubuntu20-vg-autotest 01:02:42.776 Build ubuntu22-vg-autotest #37825 completed: SUCCESS 01:02:42.778 [Pipeline] } 01:02:42.806 [Pipeline] // retry 01:02:42.810 [Pipeline] } 01:02:43.053 [Pipeline] // parallel 01:02:43.061 [Pipeline] } 01:02:43.094 [Pipeline] // stage 01:02:43.103 [Pipeline] } 01:02:43.109 ERROR: Build nvme-vg-autotest #77492 failed 01:02:43.141 [Pipeline] // catchError 01:02:43.152 [Pipeline] stage 01:02:43.154 [Pipeline] { (Autorun Post and Coverage) 01:02:43.189 [Pipeline] setCustomBuildProperty 01:02:43.215 [Pipeline] dir 01:02:43.215 Running in /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_22336 01:02:43.217 [Pipeline] { 01:02:43.244 [Pipeline] copyArtifacts 01:02:43.520 Copied 6 artifacts from "doc-docker-autotest" build number 22336 01:02:43.524 [Pipeline] writeFile 01:02:43.543 [Pipeline] } 01:02:43.564 [Pipeline] // dir 01:02:43.577 [Pipeline] dir 01:02:43.578 Running in /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_22156 01:02:43.579 [Pipeline] { 01:02:43.598 [Pipeline] copyArtifacts 01:02:43.668 Copied 5 artifacts from "check-format-docker-autotest" build number 22156 01:02:43.672 [Pipeline] writeFile 01:02:43.693 [Pipeline] } 01:02:43.723 [Pipeline] // dir 01:02:43.813 [Pipeline] dir 01:02:43.814 Running in /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_22145 01:02:43.815 [Pipeline] { 01:02:43.837 [Pipeline] copyArtifacts 01:02:43.910 Copied 5 artifacts from "build-files-docker-autotest" build number 22145 01:02:43.914 [Pipeline] writeFile 01:02:43.964 [Pipeline] } 01:02:44.002 [Pipeline] // dir 01:02:44.065 [Pipeline] dir 01:02:44.065 Running in /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_22169 01:02:44.066 [Pipeline] { 01:02:44.087 [Pipeline] copyArtifacts 01:02:44.161 Copied 5 artifacts from "check-so-deps-docker-autotest" build number 22169 01:02:44.164 [Pipeline] writeFile 01:02:44.193 [Pipeline] } 01:02:44.227 [Pipeline] // dir 01:02:44.336 [Pipeline] dir 01:02:44.336 Running in /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_19873 01:02:44.338 [Pipeline] { 01:02:44.362 [Pipeline] copyArtifacts 01:02:44.442 Copied 5 artifacts from "unittest-clang-docker-autotest" build number 19873 01:02:44.446 [Pipeline] writeFile 01:02:44.542 [Pipeline] } 01:02:44.621 [Pipeline] // dir 01:02:44.961 [Pipeline] dir 01:02:44.961 Running in /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_19875 01:02:44.963 [Pipeline] { 01:02:45.016 [Pipeline] copyArtifacts 01:02:45.088 Copied 5 artifacts from "packaging-docker-autotest" build number 19875 01:02:45.093 [Pipeline] writeFile 01:02:45.245 [Pipeline] } 01:02:45.338 [Pipeline] // dir 01:02:45.364 [Pipeline] dir 01:02:45.364 Running in /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_19876 01:02:45.366 [Pipeline] { 01:02:45.393 [Pipeline] copyArtifacts 01:02:45.496 Copied 9 artifacts from "scan-build-docker-autotest" build number 19876 01:02:45.500 [Pipeline] writeFile 01:02:45.516 [Pipeline] } 01:02:45.540 [Pipeline] // dir 01:02:45.679 [Pipeline] dir 01:02:45.680 Running in /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_19874 01:02:45.681 [Pipeline] { 01:02:45.706 [Pipeline] copyArtifacts 01:02:46.038 Copied 6 artifacts from "unittest-main-docker-autotest" build number 19874 01:02:46.041 [Pipeline] writeFile 01:02:46.097 [Pipeline] } 01:02:46.149 [Pipeline] // dir 01:02:46.495 [Pipeline] dir 01:02:46.495 Running in /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_19877 01:02:46.496 [Pipeline] { 01:02:46.537 [Pipeline] copyArtifacts 01:02:46.820 Copied 6 artifacts from "unittest-valgrind-docker-autotest" build number 19877 01:02:46.824 [Pipeline] writeFile 01:02:46.952 [Pipeline] } 01:02:47.060 [Pipeline] // dir 01:02:47.402 [Pipeline] dir 01:02:47.403 Running in /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_26209 01:02:47.404 [Pipeline] { 01:02:47.444 [Pipeline] copyArtifacts 01:02:47.599 Copied 10 artifacts from "short-fuzz-phy-autotest" build number 26209 01:02:47.604 [Pipeline] writeFile 01:02:47.767 [Pipeline] } 01:02:47.866 [Pipeline] // dir 01:02:48.000 [Pipeline] dir 01:02:48.000 Running in /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_19836 01:02:48.001 [Pipeline] { 01:02:48.033 [Pipeline] copyArtifacts 01:02:48.122 Copied 8 artifacts from "autobuild-external-vg-autotest" build number 19836 01:02:48.125 [Pipeline] writeFile 01:02:48.179 [Pipeline] } 01:02:48.221 [Pipeline] // dir 01:02:48.665 [Pipeline] dir 01:02:48.665 Running in /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_26237 01:02:48.667 [Pipeline] { 01:02:48.713 [Pipeline] copyArtifacts 01:02:48.940 Copied 12 artifacts from "vhost-phy-autotest" build number 26237 01:02:48.944 [Pipeline] writeFile 01:02:49.100 [Pipeline] } 01:02:49.205 [Pipeline] // dir 01:02:49.491 [Pipeline] dir 01:02:49.491 Running in /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_77512 01:02:49.492 [Pipeline] { 01:02:49.529 [Pipeline] copyArtifacts 01:02:49.793 Copied 15 artifacts from "vhost-initiator-vg-autotest" build number 77512 01:02:49.797 [Pipeline] writeFile 01:02:49.941 [Pipeline] } 01:02:50.034 [Pipeline] // dir 01:02:50.348 [Pipeline] dir 01:02:50.349 Running in /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_76598 01:02:50.350 [Pipeline] { 01:02:50.388 [Pipeline] copyArtifacts 01:02:50.585 Copied 12 artifacts from "zns-vg-autotest" build number 76598 01:02:50.588 [Pipeline] writeFile 01:02:50.743 [Pipeline] } 01:02:50.836 [Pipeline] // dir 01:02:51.183 [Pipeline] dir 01:02:51.184 Running in /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_78615 01:02:51.185 [Pipeline] { 01:02:51.228 [Pipeline] copyArtifacts 01:02:51.448 Copied 13 artifacts from "nvme-cmb-pmr-vg-autotest" build number 78615 01:02:51.452 [Pipeline] writeFile 01:02:51.631 [Pipeline] } 01:02:51.732 [Pipeline] // dir 01:02:52.221 [Pipeline] dir 01:02:52.221 Running in /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_26220 01:02:52.222 [Pipeline] { 01:02:52.265 [Pipeline] copyArtifacts 01:02:52.490 Copied 12 artifacts from "vfio-user-phy-autotest" build number 26220 01:02:52.494 [Pipeline] writeFile 01:02:52.710 [Pipeline] } 01:02:52.812 [Pipeline] // dir 01:02:53.260 [Pipeline] dir 01:02:53.261 Running in /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_77492 01:02:53.262 [Pipeline] { 01:02:53.304 [Pipeline] copyArtifacts 01:02:53.556 Copied 11 artifacts from "nvme-vg-autotest" build number 77492 01:02:53.560 [Pipeline] writeFile 01:02:53.752 [Pipeline] } 01:02:53.872 [Pipeline] // dir 01:02:54.149 [Pipeline] dir 01:02:54.149 Running in /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_26218 01:02:54.150 [Pipeline] { 01:02:54.185 [Pipeline] copyArtifacts 01:02:54.542 Copied 28 artifacts from "BlobFS-phy-autotest" build number 26218 01:02:54.546 [Pipeline] writeFile 01:02:54.677 [Pipeline] } 01:02:54.765 [Pipeline] // dir 01:02:55.282 [Pipeline] dir 01:02:55.282 Running in /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_72384 01:02:55.283 [Pipeline] { 01:02:55.335 [Pipeline] copyArtifacts 01:02:55.547 Copied 12 artifacts from "nvme-phy-autotest" build number 72384 01:02:55.551 [Pipeline] writeFile 01:02:55.804 [Pipeline] } 01:02:55.930 [Pipeline] // dir 01:02:56.450 [Pipeline] dir 01:02:56.450 Running in /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_77441 01:02:56.451 [Pipeline] { 01:02:56.499 [Pipeline] copyArtifacts 01:02:56.648 Copied 12 artifacts from "lvol-vg-autotest" build number 77441 01:02:56.652 [Pipeline] writeFile 01:02:56.871 [Pipeline] } 01:02:57.010 [Pipeline] // dir 01:02:58.632 [Pipeline] dir 01:02:58.632 Running in /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_77530 01:02:58.633 [Pipeline] { 01:02:58.784 [Pipeline] copyArtifacts 01:03:01.203 Copied 16 artifacts from "iscsi-vg-autotest" build number 77530 01:03:01.207 [Pipeline] writeFile 01:03:02.062 [Pipeline] } 01:03:02.473 [Pipeline] // dir 01:03:03.216 [Pipeline] dir 01:03:03.217 Running in /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_59136 01:03:03.218 [Pipeline] { 01:03:03.281 [Pipeline] copyArtifacts 01:03:03.382 Copied 7 artifacts from "freebsd-vg-autotest" build number 59136 01:03:03.387 [Pipeline] writeFile 01:03:03.654 [Pipeline] } 01:03:03.796 [Pipeline] // dir 01:03:05.340 [Pipeline] dir 01:03:05.340 Running in /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_77397 01:03:05.341 [Pipeline] { 01:03:05.484 [Pipeline] copyArtifacts 01:03:08.447 Copied 16 artifacts from "iscsi-uring-vg-autotest" build number 77397 01:03:08.451 [Pipeline] writeFile 01:03:09.263 [Pipeline] } 01:03:09.629 [Pipeline] // dir 01:03:10.614 [Pipeline] dir 01:03:10.615 Running in /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_72456 01:03:10.616 [Pipeline] { 01:03:10.698 [Pipeline] copyArtifacts 01:03:10.955 Copied 13 artifacts from "nvmf-phy-autotest" build number 72456 01:03:10.959 [Pipeline] writeFile 01:03:11.434 [Pipeline] } 01:03:11.675 [Pipeline] // dir 01:03:12.948 [Pipeline] dir 01:03:12.948 Running in /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_72343 01:03:12.949 [Pipeline] { 01:03:13.058 [Pipeline] copyArtifacts 01:03:13.357 Copied 13 artifacts from "nvmf-tcp-phy-autotest" build number 72343 01:03:13.362 [Pipeline] writeFile 01:03:14.023 [Pipeline] } 01:03:14.371 [Pipeline] // dir 01:03:15.498 [Pipeline] dir 01:03:15.499 Running in /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_26212 01:03:15.500 [Pipeline] { 01:03:15.584 [Pipeline] copyArtifacts 01:03:15.807 Copied 12 artifacts from "crypto-phy-autotest" build number 26212 01:03:15.811 [Pipeline] writeFile 01:03:16.315 [Pipeline] } 01:03:16.519 [Pipeline] // dir 01:03:17.118 [Pipeline] dir 01:03:17.118 Running in /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_77417 01:03:17.119 [Pipeline] { 01:03:17.189 [Pipeline] copyArtifacts 01:03:17.440 Copied 14 artifacts from "nvmf-tcp-uring-vg-autotest" build number 77417 01:03:17.444 [Pipeline] writeFile 01:03:17.814 [Pipeline] } 01:03:17.979 [Pipeline] // dir 01:03:18.806 [Pipeline] dir 01:03:18.806 Running in /var/jenkins/workspace/autotest-per-patch_2/dsa-phy-autotest_4912 01:03:18.807 [Pipeline] { 01:03:18.875 [Pipeline] copyArtifacts 01:03:19.111 Copied 13 artifacts from "dsa-phy-autotest" build number 4912 01:03:19.115 [Pipeline] writeFile 01:03:19.457 [Pipeline] } 01:03:19.650 [Pipeline] // dir 01:03:20.481 [Pipeline] dir 01:03:20.482 Running in /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_77483 01:03:20.483 [Pipeline] { 01:03:20.553 [Pipeline] copyArtifacts 01:03:20.788 Copied 14 artifacts from "nvmf-tcp-vg-autotest" build number 77483 01:03:20.791 [Pipeline] writeFile 01:03:21.215 [Pipeline] } 01:03:21.435 [Pipeline] // dir 01:03:22.229 [Pipeline] dir 01:03:22.229 Running in /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_73671 01:03:22.230 [Pipeline] { 01:03:22.288 [Pipeline] copyArtifacts 01:03:22.655 Copied 13 artifacts from "centos7-vg-autotest" build number 73671 01:03:22.659 [Pipeline] writeFile 01:03:22.981 [Pipeline] } 01:03:23.142 [Pipeline] // dir 01:03:24.345 [Pipeline] dir 01:03:24.345 Running in /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_16102 01:03:24.347 [Pipeline] { 01:03:24.447 [Pipeline] copyArtifacts 01:03:24.820 Copied 13 artifacts from "rocky9-vg-autotest" build number 16102 01:03:24.826 [Pipeline] writeFile 01:03:25.339 [Pipeline] } 01:03:25.577 [Pipeline] // dir 01:03:26.871 [Pipeline] dir 01:03:26.871 Running in /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_73811 01:03:26.872 [Pipeline] { 01:03:26.955 [Pipeline] copyArtifacts 01:03:27.708 Copied 12 artifacts from "ubuntu20-vg-autotest" build number 73811 01:03:27.713 [Pipeline] writeFile 01:03:28.226 [Pipeline] } 01:03:28.467 [Pipeline] // dir 01:03:29.682 [Pipeline] dir 01:03:29.682 Running in /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_37825 01:03:29.683 [Pipeline] { 01:03:29.765 [Pipeline] copyArtifacts 01:03:30.129 Copied 13 artifacts from "ubuntu22-vg-autotest" build number 37825 01:03:30.133 [Pipeline] writeFile 01:03:30.648 [Pipeline] } 01:03:30.887 [Pipeline] // dir 01:03:30.898 [Pipeline] catchError 01:03:30.900 [Pipeline] { 01:03:30.929 [Pipeline] sh 01:03:31.207 + jbp/jenkins/jjb-config/jobs/scripts/post_gen_coverage.sh 01:03:31.207 + shopt -s globstar nullglob 01:03:31.207 + echo 'Start stage post_gen_coverage.sh' 01:03:31.207 Start stage post_gen_coverage.sh 01:03:31.207 + cd /var/jenkins/workspace/autotest-per-patch_2 01:03:31.207 + rm -rf /var/jenkins/workspace/autotest-per-patch_2/spdk/doc 01:03:31.207 + trap 'compress_coverage_and_docs; remove_partial_coverage_files && echo '\''End stage post_gen_coverage.sh'\''' EXIT 01:03:31.207 + move_artifacts 01:03:31.207 + local out_dirs 01:03:31.207 + out_dirs=(./**/output/) 01:03:31.207 + for dir in "${out_dirs[@]}" 01:03:31.207 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.207 + [[ -f ./autobuild-external-vg-autotest_19836/output//doc.tar.xz ]] 01:03:31.208 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.208 + [[ -f ./autobuild-external-vg-autotest_19836/output//ut_coverage.tar.xz ]] 01:03:31.208 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.208 + [[ -f ./autobuild-external-vg-autotest_19836/output//llvm.tar.xz ]] 01:03:31.208 + mv ./autobuild-external-vg-autotest_19836/output//build-pkg-manifest.txt ./autobuild-external-vg-autotest_19836/output//build-repo-manifest.txt ./autobuild-external-vg-autotest_19836/output//dmesg.init.log ./autobuild-external-vg-autotest_19836/output//dmesg.log ./autobuild-external-vg-autotest_19836/output//power.tar.xz ./autobuild-external-vg-autotest_19836/output//test_completions.txt ./autobuild-external-vg-autotest_19836/output//timing.svg ./autobuild-external-vg-autotest_19836/output//timing.txt ./autobuild-external-vg-autotest_19836/output//.. 01:03:31.208 + rmdir ./autobuild-external-vg-autotest_19836/output/ 01:03:31.208 + for dir in "${out_dirs[@]}" 01:03:31.208 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.208 + [[ -f ./BlobFS-phy-autotest_26218/output//doc.tar.xz ]] 01:03:31.208 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.208 + [[ -f ./BlobFS-phy-autotest_26218/output//ut_coverage.tar.xz ]] 01:03:31.208 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.208 + [[ -f ./BlobFS-phy-autotest_26218/output//llvm.tar.xz ]] 01:03:31.208 + mv ./BlobFS-phy-autotest_26218/output//all_tests.txt ./BlobFS-phy-autotest_26218/output//build-pkg-manifest.txt ./BlobFS-phy-autotest_26218/output//build-repo-manifest.txt ./BlobFS-phy-autotest_26218/output//cov_base.info.xz ./BlobFS-phy-autotest_26218/output//cov_test.info.xz ./BlobFS-phy-autotest_26218/output//cov_total.info.xz ./BlobFS-phy-autotest_26218/output//dmesg.init.log ./BlobFS-phy-autotest_26218/output//dmesg.log ./BlobFS-phy-autotest_26218/output//power.tar.xz ./BlobFS-phy-autotest_26218/output//rocksdb ./BlobFS-phy-autotest_26218/output//test_completions.txt ./BlobFS-phy-autotest_26218/output//timing.svg ./BlobFS-phy-autotest_26218/output//timing.txt ./BlobFS-phy-autotest_26218/output//.. 01:03:31.208 + rmdir ./BlobFS-phy-autotest_26218/output/ 01:03:31.208 + for dir in "${out_dirs[@]}" 01:03:31.208 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.208 + [[ -f ./build-files-docker-autotest_22145/output//doc.tar.xz ]] 01:03:31.208 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.208 + [[ -f ./build-files-docker-autotest_22145/output//ut_coverage.tar.xz ]] 01:03:31.208 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.208 + [[ -f ./build-files-docker-autotest_22145/output//llvm.tar.xz ]] 01:03:31.208 + mv ./build-files-docker-autotest_22145/output//build-pkg-manifest.txt ./build-files-docker-autotest_22145/output//build-repo-manifest.txt ./build-files-docker-autotest_22145/output//power.tar.xz ./build-files-docker-autotest_22145/output//test_completions.txt ./build-files-docker-autotest_22145/output//timing.txt ./build-files-docker-autotest_22145/output//.. 01:03:31.208 + rmdir ./build-files-docker-autotest_22145/output/ 01:03:31.208 + for dir in "${out_dirs[@]}" 01:03:31.208 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.208 + [[ -f ./centos7-vg-autotest_73671/output//doc.tar.xz ]] 01:03:31.208 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.208 + [[ -f ./centos7-vg-autotest_73671/output//ut_coverage.tar.xz ]] 01:03:31.208 + tar -C ./centos7-vg-autotest_73671/output/ -xf ./centos7-vg-autotest_73671/output//ut_coverage.tar.xz 01:03:31.465 + rm ./centos7-vg-autotest_73671/output//ut_coverage.tar.xz 01:03:31.465 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.465 + [[ -f ./centos7-vg-autotest_73671/output//llvm.tar.xz ]] 01:03:31.466 + mv ./centos7-vg-autotest_73671/output//all_tests.txt ./centos7-vg-autotest_73671/output//build-pkg-manifest.txt ./centos7-vg-autotest_73671/output//build-repo-manifest.txt ./centos7-vg-autotest_73671/output//cov_base.info.xz ./centos7-vg-autotest_73671/output//cov_test.info.xz ./centos7-vg-autotest_73671/output//cov_total.info.xz ./centos7-vg-autotest_73671/output//dmesg.init.log ./centos7-vg-autotest_73671/output//dmesg.log ./centos7-vg-autotest_73671/output//power.tar.xz ./centos7-vg-autotest_73671/output//test_completions.txt ./centos7-vg-autotest_73671/output//timing.svg ./centos7-vg-autotest_73671/output//timing.txt ./centos7-vg-autotest_73671/output//ut_coverage ./centos7-vg-autotest_73671/output//.. 01:03:31.466 + rmdir ./centos7-vg-autotest_73671/output/ 01:03:31.466 + for dir in "${out_dirs[@]}" 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./check-format-docker-autotest_22156/output//doc.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./check-format-docker-autotest_22156/output//ut_coverage.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./check-format-docker-autotest_22156/output//llvm.tar.xz ]] 01:03:31.466 + mv ./check-format-docker-autotest_22156/output//build-pkg-manifest.txt ./check-format-docker-autotest_22156/output//build-repo-manifest.txt ./check-format-docker-autotest_22156/output//power.tar.xz ./check-format-docker-autotest_22156/output//test_completions.txt ./check-format-docker-autotest_22156/output//timing.txt ./check-format-docker-autotest_22156/output//.. 01:03:31.466 + rmdir ./check-format-docker-autotest_22156/output/ 01:03:31.466 + for dir in "${out_dirs[@]}" 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./check-so-deps-docker-autotest_22169/output//doc.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./check-so-deps-docker-autotest_22169/output//ut_coverage.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./check-so-deps-docker-autotest_22169/output//llvm.tar.xz ]] 01:03:31.466 + mv ./check-so-deps-docker-autotest_22169/output//build-pkg-manifest.txt ./check-so-deps-docker-autotest_22169/output//build-repo-manifest.txt ./check-so-deps-docker-autotest_22169/output//power.tar.xz ./check-so-deps-docker-autotest_22169/output//test_completions.txt ./check-so-deps-docker-autotest_22169/output//timing.txt ./check-so-deps-docker-autotest_22169/output//.. 01:03:31.466 + rmdir ./check-so-deps-docker-autotest_22169/output/ 01:03:31.466 + for dir in "${out_dirs[@]}" 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./crypto-phy-autotest_26212/output//doc.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./crypto-phy-autotest_26212/output//ut_coverage.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./crypto-phy-autotest_26212/output//llvm.tar.xz ]] 01:03:31.466 + mv ./crypto-phy-autotest_26212/output//all_tests.txt ./crypto-phy-autotest_26212/output//build-pkg-manifest.txt ./crypto-phy-autotest_26212/output//build-repo-manifest.txt ./crypto-phy-autotest_26212/output//cov_base.info.xz ./crypto-phy-autotest_26212/output//cov_test.info.xz ./crypto-phy-autotest_26212/output//cov_total.info.xz ./crypto-phy-autotest_26212/output//dmesg.init.log ./crypto-phy-autotest_26212/output//dmesg.log ./crypto-phy-autotest_26212/output//power.tar.xz ./crypto-phy-autotest_26212/output//test_completions.txt ./crypto-phy-autotest_26212/output//timing.svg ./crypto-phy-autotest_26212/output//timing.txt ./crypto-phy-autotest_26212/output//.. 01:03:31.466 + rmdir ./crypto-phy-autotest_26212/output/ 01:03:31.466 + for dir in "${out_dirs[@]}" 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./doc-docker-autotest_22336/output//doc.tar.xz ]] 01:03:31.466 + tar -C ./doc-docker-autotest_22336/output/ -xf ./doc-docker-autotest_22336/output//doc.tar.xz 01:03:31.466 + rm ./doc-docker-autotest_22336/output//doc.tar.xz 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./doc-docker-autotest_22336/output//ut_coverage.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./doc-docker-autotest_22336/output//llvm.tar.xz ]] 01:03:31.466 + mv ./doc-docker-autotest_22336/output//build-pkg-manifest.txt ./doc-docker-autotest_22336/output//build-repo-manifest.txt ./doc-docker-autotest_22336/output//doc ./doc-docker-autotest_22336/output//power.tar.xz ./doc-docker-autotest_22336/output//test_completions.txt ./doc-docker-autotest_22336/output//timing.txt ./doc-docker-autotest_22336/output//.. 01:03:31.466 + rmdir ./doc-docker-autotest_22336/output/ 01:03:31.466 + for dir in "${out_dirs[@]}" 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./dsa-phy-autotest_4912/output//doc.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./dsa-phy-autotest_4912/output//ut_coverage.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./dsa-phy-autotest_4912/output//llvm.tar.xz ]] 01:03:31.466 + mv ./dsa-phy-autotest_4912/output//all_tests.txt ./dsa-phy-autotest_4912/output//build-pkg-manifest.txt ./dsa-phy-autotest_4912/output//build-repo-manifest.txt ./dsa-phy-autotest_4912/output//cov_base.info.xz ./dsa-phy-autotest_4912/output//cov_test.info.xz ./dsa-phy-autotest_4912/output//cov_total.info.xz ./dsa-phy-autotest_4912/output//dmesg.init.log ./dsa-phy-autotest_4912/output//dmesg.log ./dsa-phy-autotest_4912/output//nvmf_trace.0_shm.tar.gz ./dsa-phy-autotest_4912/output//power.tar.xz ./dsa-phy-autotest_4912/output//test_completions.txt ./dsa-phy-autotest_4912/output//timing.svg ./dsa-phy-autotest_4912/output//timing.txt ./dsa-phy-autotest_4912/output//.. 01:03:31.466 + rmdir ./dsa-phy-autotest_4912/output/ 01:03:31.466 + for dir in "${out_dirs[@]}" 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./freebsd-vg-autotest_59136/output//doc.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./freebsd-vg-autotest_59136/output//ut_coverage.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./freebsd-vg-autotest_59136/output//llvm.tar.xz ]] 01:03:31.466 + mv ./freebsd-vg-autotest_59136/output//all_tests.txt ./freebsd-vg-autotest_59136/output//build-pkg-manifest.txt ./freebsd-vg-autotest_59136/output//build-repo-manifest.txt ./freebsd-vg-autotest_59136/output//messages ./freebsd-vg-autotest_59136/output//power.tar.xz ./freebsd-vg-autotest_59136/output//test_completions.txt ./freebsd-vg-autotest_59136/output//timing.txt ./freebsd-vg-autotest_59136/output//.. 01:03:31.466 + rmdir ./freebsd-vg-autotest_59136/output/ 01:03:31.466 + for dir in "${out_dirs[@]}" 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./iscsi-uring-vg-autotest_77397/output//doc.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./iscsi-uring-vg-autotest_77397/output//ut_coverage.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./iscsi-uring-vg-autotest_77397/output//llvm.tar.xz ]] 01:03:31.466 + mv ./iscsi-uring-vg-autotest_77397/output//all_tests.txt ./iscsi-uring-vg-autotest_77397/output//build-pkg-manifest.txt ./iscsi-uring-vg-autotest_77397/output//build-repo-manifest.txt ./iscsi-uring-vg-autotest_77397/output//calsoft.json ./iscsi-uring-vg-autotest_77397/output//calsoft.tar.xz ./iscsi-uring-vg-autotest_77397/output//cov_base.info.xz ./iscsi-uring-vg-autotest_77397/output//cov_test.info.xz ./iscsi-uring-vg-autotest_77397/output//cov_total.info.xz ./iscsi-uring-vg-autotest_77397/output//dmesg.init.log ./iscsi-uring-vg-autotest_77397/output//dmesg.log ./iscsi-uring-vg-autotest_77397/output//iscsi_autofuzz_logs.txt ./iscsi-uring-vg-autotest_77397/output//iscsi_autofuzz_tgt_output.txt ./iscsi-uring-vg-autotest_77397/output//power.tar.xz ./iscsi-uring-vg-autotest_77397/output//test_completions.txt ./iscsi-uring-vg-autotest_77397/output//timing.svg ./iscsi-uring-vg-autotest_77397/output//timing.txt ./iscsi-uring-vg-autotest_77397/output//.. 01:03:31.466 + rmdir ./iscsi-uring-vg-autotest_77397/output/ 01:03:31.466 + for dir in "${out_dirs[@]}" 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./iscsi-vg-autotest_77530/output//doc.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./iscsi-vg-autotest_77530/output//ut_coverage.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./iscsi-vg-autotest_77530/output//llvm.tar.xz ]] 01:03:31.466 + mv ./iscsi-vg-autotest_77530/output//all_tests.txt ./iscsi-vg-autotest_77530/output//build-pkg-manifest.txt ./iscsi-vg-autotest_77530/output//build-repo-manifest.txt ./iscsi-vg-autotest_77530/output//calsoft.json ./iscsi-vg-autotest_77530/output//calsoft.tar.xz ./iscsi-vg-autotest_77530/output//cov_base.info.xz ./iscsi-vg-autotest_77530/output//cov_test.info.xz ./iscsi-vg-autotest_77530/output//cov_total.info.xz ./iscsi-vg-autotest_77530/output//dmesg.init.log ./iscsi-vg-autotest_77530/output//dmesg.log ./iscsi-vg-autotest_77530/output//iscsi_autofuzz_logs.txt ./iscsi-vg-autotest_77530/output//iscsi_autofuzz_tgt_output.txt ./iscsi-vg-autotest_77530/output//power.tar.xz ./iscsi-vg-autotest_77530/output//test_completions.txt ./iscsi-vg-autotest_77530/output//timing.svg ./iscsi-vg-autotest_77530/output//timing.txt ./iscsi-vg-autotest_77530/output//.. 01:03:31.466 + rmdir ./iscsi-vg-autotest_77530/output/ 01:03:31.466 + for dir in "${out_dirs[@]}" 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./lvol-vg-autotest_77441/output//doc.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./lvol-vg-autotest_77441/output//ut_coverage.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./lvol-vg-autotest_77441/output//llvm.tar.xz ]] 01:03:31.466 + mv ./lvol-vg-autotest_77441/output//all_tests.txt ./lvol-vg-autotest_77441/output//build-pkg-manifest.txt ./lvol-vg-autotest_77441/output//build-repo-manifest.txt ./lvol-vg-autotest_77441/output//cov_base.info.xz ./lvol-vg-autotest_77441/output//cov_test.info.xz ./lvol-vg-autotest_77441/output//cov_total.info.xz ./lvol-vg-autotest_77441/output//dmesg.init.log ./lvol-vg-autotest_77441/output//dmesg.log ./lvol-vg-autotest_77441/output//power.tar.xz ./lvol-vg-autotest_77441/output//test_completions.txt ./lvol-vg-autotest_77441/output//timing.svg ./lvol-vg-autotest_77441/output//timing.txt ./lvol-vg-autotest_77441/output//.. 01:03:31.466 + rmdir ./lvol-vg-autotest_77441/output/ 01:03:31.466 + for dir in "${out_dirs[@]}" 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./nvme-cmb-pmr-vg-autotest_78615/output//doc.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./nvme-cmb-pmr-vg-autotest_78615/output//ut_coverage.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./nvme-cmb-pmr-vg-autotest_78615/output//llvm.tar.xz ]] 01:03:31.466 + mv ./nvme-cmb-pmr-vg-autotest_78615/output//all_tests.txt ./nvme-cmb-pmr-vg-autotest_78615/output//build-kernel-manifest.txt ./nvme-cmb-pmr-vg-autotest_78615/output//build-pkg-manifest.txt ./nvme-cmb-pmr-vg-autotest_78615/output//build-repo-manifest.txt ./nvme-cmb-pmr-vg-autotest_78615/output//cov_base.info.xz ./nvme-cmb-pmr-vg-autotest_78615/output//cov_test.info.xz ./nvme-cmb-pmr-vg-autotest_78615/output//cov_total.info.xz ./nvme-cmb-pmr-vg-autotest_78615/output//dmesg.init.log ./nvme-cmb-pmr-vg-autotest_78615/output//dmesg.log ./nvme-cmb-pmr-vg-autotest_78615/output//power.tar.xz ./nvme-cmb-pmr-vg-autotest_78615/output//test_completions.txt ./nvme-cmb-pmr-vg-autotest_78615/output//timing.svg ./nvme-cmb-pmr-vg-autotest_78615/output//timing.txt ./nvme-cmb-pmr-vg-autotest_78615/output//.. 01:03:31.466 + rmdir ./nvme-cmb-pmr-vg-autotest_78615/output/ 01:03:31.466 + for dir in "${out_dirs[@]}" 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./nvme-phy-autotest_72384/output//doc.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./nvme-phy-autotest_72384/output//ut_coverage.tar.xz ]] 01:03:31.466 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.466 + [[ -f ./nvme-phy-autotest_72384/output//llvm.tar.xz ]] 01:03:31.466 + mv ./nvme-phy-autotest_72384/output//all_tests.txt ./nvme-phy-autotest_72384/output//build-pkg-manifest.txt ./nvme-phy-autotest_72384/output//build-repo-manifest.txt ./nvme-phy-autotest_72384/output//cov_base.info.xz ./nvme-phy-autotest_72384/output//cov_test.info.xz ./nvme-phy-autotest_72384/output//cov_total.info.xz ./nvme-phy-autotest_72384/output//dmesg.init.log ./nvme-phy-autotest_72384/output//dmesg.log ./nvme-phy-autotest_72384/output//power.tar.xz ./nvme-phy-autotest_72384/output//test_completions.txt ./nvme-phy-autotest_72384/output//timing.svg ./nvme-phy-autotest_72384/output//timing.txt ./nvme-phy-autotest_72384/output//.. 01:03:31.466 + rmdir ./nvme-phy-autotest_72384/output/ 01:03:31.467 + for dir in "${out_dirs[@]}" 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvme-vg-autotest_77492/output//doc.tar.xz ]] 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvme-vg-autotest_77492/output//ut_coverage.tar.xz ]] 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvme-vg-autotest_77492/output//llvm.tar.xz ]] 01:03:31.467 + mv ./nvme-vg-autotest_77492/output//all_tests.txt ./nvme-vg-autotest_77492/output//build-pkg-manifest.txt ./nvme-vg-autotest_77492/output//build-repo-manifest.txt ./nvme-vg-autotest_77492/output//cov_base.info.xz ./nvme-vg-autotest_77492/output//dmesg.init.log ./nvme-vg-autotest_77492/output//dmesg.log ./nvme-vg-autotest_77492/output//power.tar.xz ./nvme-vg-autotest_77492/output//proc_list.txt ./nvme-vg-autotest_77492/output//test_completions.txt ./nvme-vg-autotest_77492/output//timing.txt ./nvme-vg-autotest_77492/output//udev.log ./nvme-vg-autotest_77492/output//.. 01:03:31.467 + rmdir ./nvme-vg-autotest_77492/output/ 01:03:31.467 + for dir in "${out_dirs[@]}" 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvmf-phy-autotest_72456/output//doc.tar.xz ]] 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvmf-phy-autotest_72456/output//ut_coverage.tar.xz ]] 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvmf-phy-autotest_72456/output//llvm.tar.xz ]] 01:03:31.467 + mv ./nvmf-phy-autotest_72456/output//all_tests.txt ./nvmf-phy-autotest_72456/output//build-pkg-manifest.txt ./nvmf-phy-autotest_72456/output//build-repo-manifest.txt ./nvmf-phy-autotest_72456/output//cov_base.info.xz ./nvmf-phy-autotest_72456/output//cov_test.info.xz ./nvmf-phy-autotest_72456/output//cov_total.info.xz ./nvmf-phy-autotest_72456/output//dmesg.init.log ./nvmf-phy-autotest_72456/output//dmesg.log ./nvmf-phy-autotest_72456/output//nvmf_trace.0_shm.tar.gz ./nvmf-phy-autotest_72456/output//power.tar.xz ./nvmf-phy-autotest_72456/output//test_completions.txt ./nvmf-phy-autotest_72456/output//timing.svg ./nvmf-phy-autotest_72456/output//timing.txt ./nvmf-phy-autotest_72456/output//.. 01:03:31.467 + rmdir ./nvmf-phy-autotest_72456/output/ 01:03:31.467 + for dir in "${out_dirs[@]}" 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvmf-tcp-phy-autotest_72343/output//doc.tar.xz ]] 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvmf-tcp-phy-autotest_72343/output//ut_coverage.tar.xz ]] 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvmf-tcp-phy-autotest_72343/output//llvm.tar.xz ]] 01:03:31.467 + mv ./nvmf-tcp-phy-autotest_72343/output//all_tests.txt ./nvmf-tcp-phy-autotest_72343/output//build-pkg-manifest.txt ./nvmf-tcp-phy-autotest_72343/output//build-repo-manifest.txt ./nvmf-tcp-phy-autotest_72343/output//cov_base.info.xz ./nvmf-tcp-phy-autotest_72343/output//cov_test.info.xz ./nvmf-tcp-phy-autotest_72343/output//cov_total.info.xz ./nvmf-tcp-phy-autotest_72343/output//dmesg.init.log ./nvmf-tcp-phy-autotest_72343/output//dmesg.log ./nvmf-tcp-phy-autotest_72343/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-phy-autotest_72343/output//power.tar.xz ./nvmf-tcp-phy-autotest_72343/output//test_completions.txt ./nvmf-tcp-phy-autotest_72343/output//timing.svg ./nvmf-tcp-phy-autotest_72343/output//timing.txt ./nvmf-tcp-phy-autotest_72343/output//.. 01:03:31.467 + rmdir ./nvmf-tcp-phy-autotest_72343/output/ 01:03:31.467 + for dir in "${out_dirs[@]}" 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvmf-tcp-uring-vg-autotest_77417/output//doc.tar.xz ]] 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvmf-tcp-uring-vg-autotest_77417/output//ut_coverage.tar.xz ]] 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvmf-tcp-uring-vg-autotest_77417/output//llvm.tar.xz ]] 01:03:31.467 + mv ./nvmf-tcp-uring-vg-autotest_77417/output//all_tests.txt ./nvmf-tcp-uring-vg-autotest_77417/output//build-kernel-manifest.txt ./nvmf-tcp-uring-vg-autotest_77417/output//build-pkg-manifest.txt ./nvmf-tcp-uring-vg-autotest_77417/output//build-repo-manifest.txt ./nvmf-tcp-uring-vg-autotest_77417/output//cov_base.info.xz ./nvmf-tcp-uring-vg-autotest_77417/output//cov_test.info.xz ./nvmf-tcp-uring-vg-autotest_77417/output//cov_total.info.xz ./nvmf-tcp-uring-vg-autotest_77417/output//dmesg.init.log ./nvmf-tcp-uring-vg-autotest_77417/output//dmesg.log ./nvmf-tcp-uring-vg-autotest_77417/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-uring-vg-autotest_77417/output//power.tar.xz ./nvmf-tcp-uring-vg-autotest_77417/output//test_completions.txt ./nvmf-tcp-uring-vg-autotest_77417/output//timing.svg ./nvmf-tcp-uring-vg-autotest_77417/output//timing.txt ./nvmf-tcp-uring-vg-autotest_77417/output//.. 01:03:31.467 + rmdir ./nvmf-tcp-uring-vg-autotest_77417/output/ 01:03:31.467 + for dir in "${out_dirs[@]}" 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvmf-tcp-vg-autotest_77483/output//doc.tar.xz ]] 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvmf-tcp-vg-autotest_77483/output//ut_coverage.tar.xz ]] 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./nvmf-tcp-vg-autotest_77483/output//llvm.tar.xz ]] 01:03:31.467 + mv ./nvmf-tcp-vg-autotest_77483/output//all_tests.txt ./nvmf-tcp-vg-autotest_77483/output//build-kernel-manifest.txt ./nvmf-tcp-vg-autotest_77483/output//build-pkg-manifest.txt ./nvmf-tcp-vg-autotest_77483/output//build-repo-manifest.txt ./nvmf-tcp-vg-autotest_77483/output//cov_base.info.xz ./nvmf-tcp-vg-autotest_77483/output//cov_test.info.xz ./nvmf-tcp-vg-autotest_77483/output//cov_total.info.xz ./nvmf-tcp-vg-autotest_77483/output//dmesg.init.log ./nvmf-tcp-vg-autotest_77483/output//dmesg.log ./nvmf-tcp-vg-autotest_77483/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-vg-autotest_77483/output//power.tar.xz ./nvmf-tcp-vg-autotest_77483/output//test_completions.txt ./nvmf-tcp-vg-autotest_77483/output//timing.svg ./nvmf-tcp-vg-autotest_77483/output//timing.txt ./nvmf-tcp-vg-autotest_77483/output//.. 01:03:31.467 + rmdir ./nvmf-tcp-vg-autotest_77483/output/ 01:03:31.467 + for dir in "${out_dirs[@]}" 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./packaging-docker-autotest_19875/output//doc.tar.xz ]] 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./packaging-docker-autotest_19875/output//ut_coverage.tar.xz ]] 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./packaging-docker-autotest_19875/output//llvm.tar.xz ]] 01:03:31.467 + mv ./packaging-docker-autotest_19875/output//build-pkg-manifest.txt ./packaging-docker-autotest_19875/output//build-repo-manifest.txt ./packaging-docker-autotest_19875/output//power.tar.xz ./packaging-docker-autotest_19875/output//test_completions.txt ./packaging-docker-autotest_19875/output//timing.txt ./packaging-docker-autotest_19875/output//.. 01:03:31.467 + rmdir ./packaging-docker-autotest_19875/output/ 01:03:31.467 + for dir in "${out_dirs[@]}" 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./rocky9-vg-autotest_16102/output//doc.tar.xz ]] 01:03:31.467 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.467 + [[ -f ./rocky9-vg-autotest_16102/output//ut_coverage.tar.xz ]] 01:03:31.467 + tar -C ./rocky9-vg-autotest_16102/output/ -xf ./rocky9-vg-autotest_16102/output//ut_coverage.tar.xz 01:03:31.724 + rm ./rocky9-vg-autotest_16102/output//ut_coverage.tar.xz 01:03:31.724 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.724 + [[ -f ./rocky9-vg-autotest_16102/output//llvm.tar.xz ]] 01:03:31.724 + mv ./rocky9-vg-autotest_16102/output//all_tests.txt ./rocky9-vg-autotest_16102/output//build-pkg-manifest.txt ./rocky9-vg-autotest_16102/output//build-repo-manifest.txt ./rocky9-vg-autotest_16102/output//cov_base.info.xz ./rocky9-vg-autotest_16102/output//cov_test.info.xz ./rocky9-vg-autotest_16102/output//cov_total.info.xz ./rocky9-vg-autotest_16102/output//dmesg.init.log ./rocky9-vg-autotest_16102/output//dmesg.log ./rocky9-vg-autotest_16102/output//power.tar.xz ./rocky9-vg-autotest_16102/output//test_completions.txt ./rocky9-vg-autotest_16102/output//timing.svg ./rocky9-vg-autotest_16102/output//timing.txt ./rocky9-vg-autotest_16102/output//ut_coverage ./rocky9-vg-autotest_16102/output//.. 01:03:31.724 + rmdir ./rocky9-vg-autotest_16102/output/ 01:03:31.724 + for dir in "${out_dirs[@]}" 01:03:31.724 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.724 + [[ -f ./scan-build-docker-autotest_19876/output//doc.tar.xz ]] 01:03:31.724 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.724 + [[ -f ./scan-build-docker-autotest_19876/output//ut_coverage.tar.xz ]] 01:03:31.724 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.724 + [[ -f ./scan-build-docker-autotest_19876/output//llvm.tar.xz ]] 01:03:31.724 + mv ./scan-build-docker-autotest_19876/output//all_c_files.txt ./scan-build-docker-autotest_19876/output//build_output.txt ./scan-build-docker-autotest_19876/output//build-pkg-manifest.txt ./scan-build-docker-autotest_19876/output//build-repo-manifest.txt ./scan-build-docker-autotest_19876/output//built_c_files.txt ./scan-build-docker-autotest_19876/output//power.tar.xz ./scan-build-docker-autotest_19876/output//test_completions.txt ./scan-build-docker-autotest_19876/output//timing.txt ./scan-build-docker-autotest_19876/output//unbuilt_c_files.txt ./scan-build-docker-autotest_19876/output//.. 01:03:31.724 + rmdir ./scan-build-docker-autotest_19876/output/ 01:03:31.724 + for dir in "${out_dirs[@]}" 01:03:31.724 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.724 + [[ -f ./short-fuzz-phy-autotest_26209/output//doc.tar.xz ]] 01:03:31.724 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.724 + [[ -f ./short-fuzz-phy-autotest_26209/output//ut_coverage.tar.xz ]] 01:03:31.724 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.724 + [[ -f ./short-fuzz-phy-autotest_26209/output//llvm.tar.xz ]] 01:03:31.724 + tar -C ./short-fuzz-phy-autotest_26209/output/ -xf ./short-fuzz-phy-autotest_26209/output//llvm.tar.xz 01:03:31.724 + rm ./short-fuzz-phy-autotest_26209/output//llvm.tar.xz 01:03:31.724 + mv ./short-fuzz-phy-autotest_26209/output//all_tests.txt ./short-fuzz-phy-autotest_26209/output//build-pkg-manifest.txt ./short-fuzz-phy-autotest_26209/output//build-repo-manifest.txt ./short-fuzz-phy-autotest_26209/output//dmesg.init.log ./short-fuzz-phy-autotest_26209/output//dmesg.log ./short-fuzz-phy-autotest_26209/output//llvm ./short-fuzz-phy-autotest_26209/output//power.tar.xz ./short-fuzz-phy-autotest_26209/output//test_completions.txt ./short-fuzz-phy-autotest_26209/output//timing.svg ./short-fuzz-phy-autotest_26209/output//timing.txt ./short-fuzz-phy-autotest_26209/output//.. 01:03:31.724 + rmdir ./short-fuzz-phy-autotest_26209/output/ 01:03:31.724 + for dir in "${out_dirs[@]}" 01:03:31.724 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.724 + [[ -f ./ubuntu20-vg-autotest_73811/output//doc.tar.xz ]] 01:03:31.724 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.724 + [[ -f ./ubuntu20-vg-autotest_73811/output//ut_coverage.tar.xz ]] 01:03:31.724 + tar -C ./ubuntu20-vg-autotest_73811/output/ -xf ./ubuntu20-vg-autotest_73811/output//ut_coverage.tar.xz 01:03:31.982 + rm ./ubuntu20-vg-autotest_73811/output//ut_coverage.tar.xz 01:03:31.982 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.982 + [[ -f ./ubuntu20-vg-autotest_73811/output//llvm.tar.xz ]] 01:03:31.982 + mv ./ubuntu20-vg-autotest_73811/output//all_tests.txt ./ubuntu20-vg-autotest_73811/output//build-pkg-manifest.txt ./ubuntu20-vg-autotest_73811/output//build-repo-manifest.txt ./ubuntu20-vg-autotest_73811/output//cov_base.info.xz ./ubuntu20-vg-autotest_73811/output//dmesg.init.log ./ubuntu20-vg-autotest_73811/output//dmesg.log ./ubuntu20-vg-autotest_73811/output//power.tar.xz ./ubuntu20-vg-autotest_73811/output//proc_list.txt ./ubuntu20-vg-autotest_73811/output//test_completions.txt ./ubuntu20-vg-autotest_73811/output//timing.txt ./ubuntu20-vg-autotest_73811/output//udev.log ./ubuntu20-vg-autotest_73811/output//ut_coverage ./ubuntu20-vg-autotest_73811/output//.. 01:03:31.982 + rmdir ./ubuntu20-vg-autotest_73811/output/ 01:03:31.982 + for dir in "${out_dirs[@]}" 01:03:31.982 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.982 + [[ -f ./ubuntu22-vg-autotest_37825/output//doc.tar.xz ]] 01:03:31.982 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:31.982 + [[ -f ./ubuntu22-vg-autotest_37825/output//ut_coverage.tar.xz ]] 01:03:31.982 + tar -C ./ubuntu22-vg-autotest_37825/output/ -xf ./ubuntu22-vg-autotest_37825/output//ut_coverage.tar.xz 01:03:32.239 + rm ./ubuntu22-vg-autotest_37825/output//ut_coverage.tar.xz 01:03:32.239 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.239 + [[ -f ./ubuntu22-vg-autotest_37825/output//llvm.tar.xz ]] 01:03:32.239 + mv ./ubuntu22-vg-autotest_37825/output//all_tests.txt ./ubuntu22-vg-autotest_37825/output//build-pkg-manifest.txt ./ubuntu22-vg-autotest_37825/output//build-repo-manifest.txt ./ubuntu22-vg-autotest_37825/output//cov_base.info.xz ./ubuntu22-vg-autotest_37825/output//cov_test.info.xz ./ubuntu22-vg-autotest_37825/output//cov_total.info.xz ./ubuntu22-vg-autotest_37825/output//dmesg.init.log ./ubuntu22-vg-autotest_37825/output//dmesg.log ./ubuntu22-vg-autotest_37825/output//power.tar.xz ./ubuntu22-vg-autotest_37825/output//test_completions.txt ./ubuntu22-vg-autotest_37825/output//timing.svg ./ubuntu22-vg-autotest_37825/output//timing.txt ./ubuntu22-vg-autotest_37825/output//ut_coverage ./ubuntu22-vg-autotest_37825/output//.. 01:03:32.239 + rmdir ./ubuntu22-vg-autotest_37825/output/ 01:03:32.239 + for dir in "${out_dirs[@]}" 01:03:32.239 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.239 + [[ -f ./unittest-clang-docker-autotest_19873/output//doc.tar.xz ]] 01:03:32.239 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.239 + [[ -f ./unittest-clang-docker-autotest_19873/output//ut_coverage.tar.xz ]] 01:03:32.239 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.239 + [[ -f ./unittest-clang-docker-autotest_19873/output//llvm.tar.xz ]] 01:03:32.239 + mv ./unittest-clang-docker-autotest_19873/output//build-pkg-manifest.txt ./unittest-clang-docker-autotest_19873/output//build-repo-manifest.txt ./unittest-clang-docker-autotest_19873/output//power.tar.xz ./unittest-clang-docker-autotest_19873/output//test_completions.txt ./unittest-clang-docker-autotest_19873/output//timing.txt ./unittest-clang-docker-autotest_19873/output//.. 01:03:32.239 + rmdir ./unittest-clang-docker-autotest_19873/output/ 01:03:32.239 + for dir in "${out_dirs[@]}" 01:03:32.239 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.239 + [[ -f ./unittest-main-docker-autotest_19874/output//doc.tar.xz ]] 01:03:32.239 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.239 + [[ -f ./unittest-main-docker-autotest_19874/output//ut_coverage.tar.xz ]] 01:03:32.239 + tar -C ./unittest-main-docker-autotest_19874/output/ -xf ./unittest-main-docker-autotest_19874/output//ut_coverage.tar.xz 01:03:32.496 + rm ./unittest-main-docker-autotest_19874/output//ut_coverage.tar.xz 01:03:32.496 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.496 + [[ -f ./unittest-main-docker-autotest_19874/output//llvm.tar.xz ]] 01:03:32.496 + mv ./unittest-main-docker-autotest_19874/output//build-pkg-manifest.txt ./unittest-main-docker-autotest_19874/output//build-repo-manifest.txt ./unittest-main-docker-autotest_19874/output//power.tar.xz ./unittest-main-docker-autotest_19874/output//test_completions.txt ./unittest-main-docker-autotest_19874/output//timing.txt ./unittest-main-docker-autotest_19874/output//ut_coverage ./unittest-main-docker-autotest_19874/output//.. 01:03:32.496 + rmdir ./unittest-main-docker-autotest_19874/output/ 01:03:32.496 + for dir in "${out_dirs[@]}" 01:03:32.496 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.496 + [[ -f ./unittest-valgrind-docker-autotest_19877/output//doc.tar.xz ]] 01:03:32.496 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.496 + [[ -f ./unittest-valgrind-docker-autotest_19877/output//ut_coverage.tar.xz ]] 01:03:32.496 + tar -C ./unittest-valgrind-docker-autotest_19877/output/ -xf ./unittest-valgrind-docker-autotest_19877/output//ut_coverage.tar.xz 01:03:32.754 + rm ./unittest-valgrind-docker-autotest_19877/output//ut_coverage.tar.xz 01:03:32.754 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.754 + [[ -f ./unittest-valgrind-docker-autotest_19877/output//llvm.tar.xz ]] 01:03:32.754 + mv ./unittest-valgrind-docker-autotest_19877/output//build-pkg-manifest.txt ./unittest-valgrind-docker-autotest_19877/output//build-repo-manifest.txt ./unittest-valgrind-docker-autotest_19877/output//power.tar.xz ./unittest-valgrind-docker-autotest_19877/output//test_completions.txt ./unittest-valgrind-docker-autotest_19877/output//timing.txt ./unittest-valgrind-docker-autotest_19877/output//ut_coverage ./unittest-valgrind-docker-autotest_19877/output//.. 01:03:32.754 + rmdir ./unittest-valgrind-docker-autotest_19877/output/ 01:03:32.754 + for dir in "${out_dirs[@]}" 01:03:32.754 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.754 + [[ -f ./vfio-user-phy-autotest_26220/output//doc.tar.xz ]] 01:03:32.754 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.754 + [[ -f ./vfio-user-phy-autotest_26220/output//ut_coverage.tar.xz ]] 01:03:32.754 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.754 + [[ -f ./vfio-user-phy-autotest_26220/output//llvm.tar.xz ]] 01:03:32.754 + mv ./vfio-user-phy-autotest_26220/output//all_tests.txt ./vfio-user-phy-autotest_26220/output//build-pkg-manifest.txt ./vfio-user-phy-autotest_26220/output//build-repo-manifest.txt ./vfio-user-phy-autotest_26220/output//cov_base.info.xz ./vfio-user-phy-autotest_26220/output//cov_test.info.xz ./vfio-user-phy-autotest_26220/output//cov_total.info.xz ./vfio-user-phy-autotest_26220/output//dmesg.init.log ./vfio-user-phy-autotest_26220/output//dmesg.log ./vfio-user-phy-autotest_26220/output//power.tar.xz ./vfio-user-phy-autotest_26220/output//test_completions.txt ./vfio-user-phy-autotest_26220/output//timing.svg ./vfio-user-phy-autotest_26220/output//timing.txt ./vfio-user-phy-autotest_26220/output//.. 01:03:32.754 + rmdir ./vfio-user-phy-autotest_26220/output/ 01:03:32.754 + for dir in "${out_dirs[@]}" 01:03:32.754 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.754 + [[ -f ./vhost-initiator-vg-autotest_77512/output//doc.tar.xz ]] 01:03:32.754 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.754 + [[ -f ./vhost-initiator-vg-autotest_77512/output//ut_coverage.tar.xz ]] 01:03:32.754 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.754 + [[ -f ./vhost-initiator-vg-autotest_77512/output//llvm.tar.xz ]] 01:03:32.754 + mv ./vhost-initiator-vg-autotest_77512/output//all_tests.txt ./vhost-initiator-vg-autotest_77512/output//build-pkg-manifest.txt ./vhost-initiator-vg-autotest_77512/output//build-repo-manifest.txt ./vhost-initiator-vg-autotest_77512/output//cov_base.info.xz ./vhost-initiator-vg-autotest_77512/output//cov_test.info.xz ./vhost-initiator-vg-autotest_77512/output//cov_total.info.xz ./vhost-initiator-vg-autotest_77512/output//dmesg.init.log ./vhost-initiator-vg-autotest_77512/output//dmesg.log ./vhost-initiator-vg-autotest_77512/output//power.tar.xz ./vhost-initiator-vg-autotest_77512/output//test_completions.txt ./vhost-initiator-vg-autotest_77512/output//timing.svg ./vhost-initiator-vg-autotest_77512/output//timing.txt ./vhost-initiator-vg-autotest_77512/output//vhost_fuzz_output1.txt.xz ./vhost-initiator-vg-autotest_77512/output//vhost_fuzz_output2.txt.xz ./vhost-initiator-vg-autotest_77512/output//vhost_fuzz_tgt_output.txt.xz ./vhost-initiator-vg-autotest_77512/output//.. 01:03:32.754 + rmdir ./vhost-initiator-vg-autotest_77512/output/ 01:03:32.754 + for dir in "${out_dirs[@]}" 01:03:32.754 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.754 + [[ -f ./vhost-phy-autotest_26237/output//doc.tar.xz ]] 01:03:32.754 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.754 + [[ -f ./vhost-phy-autotest_26237/output//ut_coverage.tar.xz ]] 01:03:32.754 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.754 + [[ -f ./vhost-phy-autotest_26237/output//llvm.tar.xz ]] 01:03:32.754 + mv ./vhost-phy-autotest_26237/output//all_tests.txt ./vhost-phy-autotest_26237/output//build-pkg-manifest.txt ./vhost-phy-autotest_26237/output//build-repo-manifest.txt ./vhost-phy-autotest_26237/output//cov_base.info.xz ./vhost-phy-autotest_26237/output//cov_test.info.xz ./vhost-phy-autotest_26237/output//cov_total.info.xz ./vhost-phy-autotest_26237/output//dmesg.init.log ./vhost-phy-autotest_26237/output//dmesg.log ./vhost-phy-autotest_26237/output//power.tar.xz ./vhost-phy-autotest_26237/output//test_completions.txt ./vhost-phy-autotest_26237/output//timing.svg ./vhost-phy-autotest_26237/output//timing.txt ./vhost-phy-autotest_26237/output//.. 01:03:32.754 + rmdir ./vhost-phy-autotest_26237/output/ 01:03:32.754 + for dir in "${out_dirs[@]}" 01:03:32.754 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.754 + [[ -f ./zns-vg-autotest_76598/output//doc.tar.xz ]] 01:03:32.754 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.754 + [[ -f ./zns-vg-autotest_76598/output//ut_coverage.tar.xz ]] 01:03:32.754 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 01:03:32.754 + [[ -f ./zns-vg-autotest_76598/output//llvm.tar.xz ]] 01:03:32.754 + mv ./zns-vg-autotest_76598/output//all_tests.txt ./zns-vg-autotest_76598/output//build-pkg-manifest.txt ./zns-vg-autotest_76598/output//build-repo-manifest.txt ./zns-vg-autotest_76598/output//cov_base.info.xz ./zns-vg-autotest_76598/output//cov_test.info.xz ./zns-vg-autotest_76598/output//cov_total.info.xz ./zns-vg-autotest_76598/output//dmesg.init.log ./zns-vg-autotest_76598/output//dmesg.log ./zns-vg-autotest_76598/output//power.tar.xz ./zns-vg-autotest_76598/output//test_completions.txt ./zns-vg-autotest_76598/output//timing.svg ./zns-vg-autotest_76598/output//timing.txt ./zns-vg-autotest_76598/output//.. 01:03:32.754 + rmdir ./zns-vg-autotest_76598/output/ 01:03:32.754 + unpack_cov_files 01:03:32.754 + local info_files 01:03:32.754 + info_files=(*/cov_*.info.xz) 01:03:32.754 + xargs -P0 -r -n1 xz -d 01:03:32.755 + printf '%s\n' BlobFS-phy-autotest_26218/cov_base.info.xz BlobFS-phy-autotest_26218/cov_test.info.xz BlobFS-phy-autotest_26218/cov_total.info.xz centos7-vg-autotest_73671/cov_base.info.xz centos7-vg-autotest_73671/cov_test.info.xz centos7-vg-autotest_73671/cov_total.info.xz crypto-phy-autotest_26212/cov_base.info.xz crypto-phy-autotest_26212/cov_test.info.xz crypto-phy-autotest_26212/cov_total.info.xz dsa-phy-autotest_4912/cov_base.info.xz dsa-phy-autotest_4912/cov_test.info.xz dsa-phy-autotest_4912/cov_total.info.xz iscsi-uring-vg-autotest_77397/cov_base.info.xz iscsi-uring-vg-autotest_77397/cov_test.info.xz iscsi-uring-vg-autotest_77397/cov_total.info.xz iscsi-vg-autotest_77530/cov_base.info.xz iscsi-vg-autotest_77530/cov_test.info.xz iscsi-vg-autotest_77530/cov_total.info.xz lvol-vg-autotest_77441/cov_base.info.xz lvol-vg-autotest_77441/cov_test.info.xz lvol-vg-autotest_77441/cov_total.info.xz nvme-cmb-pmr-vg-autotest_78615/cov_base.info.xz nvme-cmb-pmr-vg-autotest_78615/cov_test.info.xz nvme-cmb-pmr-vg-autotest_78615/cov_total.info.xz nvme-phy-autotest_72384/cov_base.info.xz nvme-phy-autotest_72384/cov_test.info.xz nvme-phy-autotest_72384/cov_total.info.xz nvme-vg-autotest_77492/cov_base.info.xz nvmf-phy-autotest_72456/cov_base.info.xz nvmf-phy-autotest_72456/cov_test.info.xz nvmf-phy-autotest_72456/cov_total.info.xz nvmf-tcp-phy-autotest_72343/cov_base.info.xz nvmf-tcp-phy-autotest_72343/cov_test.info.xz nvmf-tcp-phy-autotest_72343/cov_total.info.xz nvmf-tcp-uring-vg-autotest_77417/cov_base.info.xz nvmf-tcp-uring-vg-autotest_77417/cov_test.info.xz nvmf-tcp-uring-vg-autotest_77417/cov_total.info.xz nvmf-tcp-vg-autotest_77483/cov_base.info.xz nvmf-tcp-vg-autotest_77483/cov_test.info.xz nvmf-tcp-vg-autotest_77483/cov_total.info.xz rocky9-vg-autotest_16102/cov_base.info.xz rocky9-vg-autotest_16102/cov_test.info.xz rocky9-vg-autotest_16102/cov_total.info.xz ubuntu20-vg-autotest_73811/cov_base.info.xz ubuntu22-vg-autotest_37825/cov_base.info.xz ubuntu22-vg-autotest_37825/cov_test.info.xz ubuntu22-vg-autotest_37825/cov_total.info.xz vfio-user-phy-autotest_26220/cov_base.info.xz vfio-user-phy-autotest_26220/cov_test.info.xz vfio-user-phy-autotest_26220/cov_total.info.xz vhost-initiator-vg-autotest_77512/cov_base.info.xz vhost-initiator-vg-autotest_77512/cov_test.info.xz vhost-initiator-vg-autotest_77512/cov_total.info.xz vhost-phy-autotest_26237/cov_base.info.xz vhost-phy-autotest_26237/cov_test.info.xz vhost-phy-autotest_26237/cov_total.info.xz zns-vg-autotest_76598/cov_base.info.xz zns-vg-autotest_76598/cov_test.info.xz zns-vg-autotest_76598/cov_total.info.xz 01:03:33.318 + fix_downstream_job_paths 01:03:33.318 + sed -i -e 's#^SF:/.\+/spdk/#SF:/var/jenkins/workspace/autotest-per-patch_2/spdk/#g' BlobFS-phy-autotest_26218/cov_total.info centos7-vg-autotest_73671/cov_total.info crypto-phy-autotest_26212/cov_total.info dsa-phy-autotest_4912/cov_total.info iscsi-uring-vg-autotest_77397/cov_total.info iscsi-vg-autotest_77530/cov_total.info lvol-vg-autotest_77441/cov_total.info nvme-cmb-pmr-vg-autotest_78615/cov_total.info nvme-phy-autotest_72384/cov_total.info nvmf-phy-autotest_72456/cov_total.info nvmf-tcp-phy-autotest_72343/cov_total.info nvmf-tcp-uring-vg-autotest_77417/cov_total.info nvmf-tcp-vg-autotest_77483/cov_total.info rocky9-vg-autotest_16102/cov_total.info ubuntu22-vg-autotest_37825/cov_total.info vfio-user-phy-autotest_26220/cov_total.info vhost-initiator-vg-autotest_77512/cov_total.info vhost-phy-autotest_26237/cov_total.info zns-vg-autotest_76598/cov_total.info 01:03:34.254 + ignore_ocf_headers 01:03:34.254 + sed -i 's#--show-details#--show-details --ignore-errors source#g' spdk/autorun_post.py 01:03:34.254 + create_errtrace_files_for_all_builds 01:03:34.254 + local build_log_list 01:03:34.254 + build_log_list=($(find "$WORKSPACE" -type f -name 'build.log')) 01:03:34.254 ++ find /var/jenkins/workspace/autotest-per-patch_2 -type f -name build.log 01:03:34.254 + for build in "${build_log_list[@]}" 01:03:34.254 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_77492/build.log 01:03:34.254 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.254 + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_77492/build.log 01:03:34.254 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_77492/build.log 01:03:34.254 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_77492 01:03:34.254 + job_name=nvme-vg-autotest_77492 01:03:34.254 + start_pattern='Backtrace start' 01:03:34.254 + end_pattern='Backtrace end' 01:03:34.254 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_77492/build.log ]] 01:03:34.254 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_77492/build.log 01:03:34.254 + backtraces='00:19:58.830 10:00:36 sw_hotplug -- common/autotest_common.sh@715 -- # time='\''========== Backtrace start: ========== 01:03:34.254 00:19:58.830 01:03:34.254 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.254 00:19:58.830 ... 01:03:34.254 00:19:58.830 710 01:03:34.254 00:19:58.830 711 # We redirect cmd'\''\'\'''\''s std{out,err} to a separate fd dup'\''\'\'''\''ed to stdin'\''\'\'''\''s device (or /dev/null) to 01:03:34.254 00:19:58.830 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:03:34.254 00:19:58.830 713 # but $() will return just the time'\''\'\'''\''s data, hence making it possible to just do: 01:03:34.254 00:19:58.830 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.254 00:19:58.830 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.254 00:19:58.830 716 01:03:34.254 00:19:58.830 717 echo "$time" 01:03:34.254 00:19:58.830 718 ) 01:03:34.254 00:19:58.830 719 01:03:34.254 00:19:58.830 720 function timing_enter() { 01:03:34.254 00:19:58.830 ... 01:03:34.254 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.254 00:19:58.830 ... 01:03:34.254 00:19:58.830 11 # Pci bus hotplug 01:03:34.254 00:19:58.830 12 # Helper function to remove/attach cotrollers 01:03:34.254 00:19:58.830 13 debug_remove_attach_helper() { 01:03:34.254 00:19:58.830 14 local helper_time=0 01:03:34.254 00:19:58.830 15 01:03:34.254 00:19:58.830 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.254 00:19:58.830 17 printf '\''\'\'''\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\''\'\'''\'' \ 01:03:34.254 00:19:58.830 18 "$helper_time" "$nvme_count" >&2 01:03:34.254 00:19:58.830 19 } 01:03:34.254 00:19:58.830 20 01:03:34.254 00:19:58.830 21 remove_attach_helper() { 01:03:34.254 00:19:58.830 ... 01:03:34.254 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.254 00:19:58.830 ... 01:03:34.254 00:19:58.830 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.254 00:19:58.830 106 done 01:03:34.254 00:19:58.830 107 01:03:34.254 00:19:58.830 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.254 00:19:58.830 109 01:03:34.254 00:19:58.830 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.254 00:19:58.830 111 # Verify reregistering hotplug poller 01:03:34.254 00:19:58.830 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.254 00:19:58.830 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.254 00:19:58.830 114 01:03:34.254 00:19:58.830 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.254 00:19:58.830 ... 01:03:34.254 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.254 00:19:58.830 ... 01:03:34.254 00:19:58.830 133 01:03:34.254 00:19:58.830 134 # Run pci bus hotplug test 01:03:34.254 00:19:58.830 135 run_hotplug 01:03:34.254 00:19:58.830 136 01:03:34.254 00:19:58.830 137 # Run SPDK target based hotplug 01:03:34.254 00:19:58.830 => 138 tgt_run_hotplug 01:03:34.254 00:19:58.830 ... 01:03:34.254 00:19:58.830 01:03:34.254 00:19:58.830 ========== Backtrace end =========='\'' 01:03:34.254 00:19:58.830 ========== Backtrace start: ========== 01:03:34.254 00:19:58.830 01:03:34.254 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.254 00:19:58.830 ... 01:03:34.254 00:19:58.830 710 01:03:34.254 00:19:58.830 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 01:03:34.254 00:19:58.830 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:03:34.254 00:19:58.830 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:03:34.254 00:19:58.830 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.254 00:19:58.830 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.254 00:19:58.830 716 01:03:34.254 00:19:58.830 717 echo "$time" 01:03:34.254 00:19:58.830 718 ) 01:03:34.254 00:19:58.830 719 01:03:34.254 00:19:58.830 720 function timing_enter() { 01:03:34.254 00:19:58.830 ... 01:03:34.254 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.254 00:19:58.830 ... 01:03:34.254 00:19:58.830 11 # Pci bus hotplug 01:03:34.254 00:19:58.830 12 # Helper function to remove/attach cotrollers 01:03:34.254 00:19:58.830 13 debug_remove_attach_helper() { 01:03:34.254 00:19:58.830 14 local helper_time=0 01:03:34.254 00:19:58.830 15 01:03:34.254 00:19:58.830 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.254 00:19:58.830 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.254 00:19:58.830 18 "$helper_time" "$nvme_count" >&2 01:03:34.254 00:19:58.830 19 } 01:03:34.254 00:19:58.830 20 01:03:34.254 00:19:58.830 21 remove_attach_helper() { 01:03:34.254 00:19:58.830 ... 01:03:34.254 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.254 00:19:58.830 ... 01:03:34.254 00:19:58.830 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.254 00:19:58.830 106 done 01:03:34.254 00:19:58.830 107 01:03:34.254 00:19:58.830 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.254 00:19:58.830 109 01:03:34.255 00:19:58.830 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.255 00:19:58.830 111 # Verify reregistering hotplug poller 01:03:34.255 00:19:58.830 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.255 00:19:58.830 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.255 00:19:58.830 114 01:03:34.255 00:19:58.830 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.255 00:19:58.830 ... 01:03:34.255 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.255 00:19:58.830 ... 01:03:34.255 00:19:58.830 133 01:03:34.255 00:19:58.830 134 # Run pci bus hotplug test 01:03:34.255 00:19:58.830 135 run_hotplug 01:03:34.255 00:19:58.830 136 01:03:34.255 00:19:58.830 137 # Run SPDK target based hotplug 01:03:34.255 00:19:58.830 => 138 tgt_run_hotplug 01:03:34.255 00:19:58.830 ... 01:03:34.255 00:19:58.830 01:03:34.255 00:19:58.830 ========== Backtrace end ========== 01:03:34.255 00:19:58.830 ========== Backtrace start: ========== 01:03:34.255 00:19:58.830 01:03:34.255 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.255 00:19:58.830 ... 01:03:34.255 00:19:58.830 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:03:34.255 00:19:58.830 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.255 00:19:58.830 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.255 00:19:58.830 716 01:03:34.255 00:19:58.830 717 echo "$time" 01:03:34.255 00:19:58.830 => 718 ) 01:03:34.255 00:19:58.830 719 01:03:34.255 00:19:58.830 720 function timing_enter() { 01:03:34.255 00:19:58.830 721 xtrace_disable 01:03:34.255 00:19:58.830 722 timing "enter" "$1" 01:03:34.255 00:19:58.830 723 xtrace_restore 01:03:34.255 00:19:58.830 ... 01:03:34.255 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.255 00:19:58.830 ... 01:03:34.255 00:19:58.830 11 # Pci bus hotplug 01:03:34.255 00:19:58.830 12 # Helper function to remove/attach cotrollers 01:03:34.255 00:19:58.830 13 debug_remove_attach_helper() { 01:03:34.255 00:19:58.830 14 local helper_time=0 01:03:34.255 00:19:58.830 15 01:03:34.255 00:19:58.830 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.255 00:19:58.830 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.255 00:19:58.830 18 "$helper_time" "$nvme_count" >&2 01:03:34.255 00:19:58.830 19 } 01:03:34.255 00:19:58.830 20 01:03:34.255 00:19:58.830 21 remove_attach_helper() { 01:03:34.255 00:19:58.830 ... 01:03:34.255 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.255 00:19:58.830 ... 01:03:34.255 00:19:58.830 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.255 00:19:58.831 106 done 01:03:34.255 00:19:58.831 107 01:03:34.255 00:19:58.831 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.255 00:19:58.831 109 01:03:34.255 00:19:58.831 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.255 00:19:58.831 111 # Verify reregistering hotplug poller 01:03:34.255 00:19:58.831 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.255 00:19:58.831 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.255 00:19:58.831 114 01:03:34.255 00:19:58.831 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.255 00:19:58.831 ... 01:03:34.255 00:19:58.831 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.255 00:19:58.831 ... 01:03:34.255 00:19:58.831 133 01:03:34.255 00:19:58.831 134 # Run pci bus hotplug test 01:03:34.255 00:19:58.831 135 run_hotplug 01:03:34.255 00:19:58.831 136 01:03:34.255 00:19:58.831 137 # Run SPDK target based hotplug 01:03:34.255 00:19:58.831 => 138 tgt_run_hotplug 01:03:34.255 00:19:58.831 ... 01:03:34.255 00:19:58.831 01:03:34.255 00:19:58.831 ========== Backtrace end ========== 01:03:34.255 00:19:58.831 ========== Backtrace start: ========== 01:03:34.255 00:19:58.831 01:03:34.255 00:19:58.831 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.255 00:19:58.831 ... 01:03:34.255 00:19:58.831 11 # Pci bus hotplug 01:03:34.255 00:19:58.831 12 # Helper function to remove/attach cotrollers 01:03:34.255 00:19:58.831 13 debug_remove_attach_helper() { 01:03:34.255 00:19:58.831 14 local helper_time=0 01:03:34.255 00:19:58.831 15 01:03:34.255 00:19:58.831 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.255 00:19:58.831 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.255 00:19:58.831 18 "$helper_time" "$nvme_count" >&2 01:03:34.255 00:19:58.831 19 } 01:03:34.255 00:19:58.831 20 01:03:34.255 00:19:58.831 21 remove_attach_helper() { 01:03:34.255 00:19:58.831 ... 01:03:34.255 00:19:58.831 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.255 00:19:58.831 ... 01:03:34.255 00:19:58.831 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.255 00:19:58.831 106 done 01:03:34.255 00:19:58.831 107 01:03:34.255 00:19:58.831 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.255 00:19:58.831 109 01:03:34.255 00:19:58.831 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.255 00:19:58.831 111 # Verify reregistering hotplug poller 01:03:34.255 00:19:58.831 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.255 00:19:58.831 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.255 00:19:58.831 114 01:03:34.255 00:19:58.831 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.255 00:19:58.831 ... 01:03:34.255 00:19:58.831 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.255 00:19:58.831 ... 01:03:34.255 00:19:58.831 133 01:03:34.255 00:19:58.831 134 # Run pci bus hotplug test 01:03:34.255 00:19:58.831 135 run_hotplug 01:03:34.255 00:19:58.831 136 01:03:34.255 00:19:58.831 137 # Run SPDK target based hotplug 01:03:34.255 00:19:58.831 => 138 tgt_run_hotplug 01:03:34.255 00:19:58.831 ... 01:03:34.255 00:19:58.831 01:03:34.255 00:19:58.831 ========== Backtrace end ========== 01:03:34.255 00:20:02.121 ========== Backtrace start: ========== 01:03:34.255 00:20:02.121 01:03:34.255 00:20:02.121 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 01:03:34.255 00:20:02.121 ... 01:03:34.255 00:20:02.121 1117 timing_enter $test_name 01:03:34.255 00:20:02.121 1118 echo "************************************" 01:03:34.255 00:20:02.121 1119 echo "START TEST $test_name" 01:03:34.255 00:20:02.121 1120 echo "************************************" 01:03:34.255 00:20:02.121 1121 xtrace_restore 01:03:34.255 00:20:02.122 1122 time "$@" 01:03:34.255 00:20:02.122 1123 xtrace_disable 01:03:34.255 00:20:02.122 1124 echo "************************************" 01:03:34.255 00:20:02.122 1125 echo "END TEST $test_name" 01:03:34.255 00:20:02.122 1126 echo "************************************" 01:03:34.255 00:20:02.122 1127 timing_exit $test_name 01:03:34.255 00:20:02.122 ... 01:03:34.255 00:20:02.122 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 01:03:34.255 00:20:02.122 ... 01:03:34.255 00:20:02.122 235 01:03:34.255 00:20:02.122 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 01:03:34.255 00:20:02.122 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 01:03:34.255 00:20:02.122 238 01:03:34.255 00:20:02.122 239 if [ $(uname -s) = Linux ]; then 01:03:34.255 00:20:02.122 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 01:03:34.255 00:20:02.122 241 fi 01:03:34.255 00:20:02.122 242 01:03:34.255 00:20:02.122 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 01:03:34.255 00:20:02.122 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 01:03:34.255 00:20:02.122 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 01:03:34.255 00:20:02.122 ... 01:03:34.255 00:20:02.122 01:03:34.255 00:20:02.122 ========== Backtrace end ==========' 01:03:34.255 + [[ -n 00:19:58.830 10:00:36 sw_hotplug -- common/autotest_common.sh@715 -- # time='========== Backtrace start: ========== 01:03:34.255 00:19:58.830 01:03:34.255 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.255 00:19:58.830 ... 01:03:34.255 00:19:58.830 710 01:03:34.255 00:19:58.830 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 01:03:34.255 00:19:58.830 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:03:34.255 00:19:58.830 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:03:34.255 00:19:58.830 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.255 00:19:58.830 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.255 00:19:58.830 716 01:03:34.255 00:19:58.830 717 echo "$time" 01:03:34.255 00:19:58.830 718 ) 01:03:34.255 00:19:58.830 719 01:03:34.255 00:19:58.830 720 function timing_enter() { 01:03:34.255 00:19:58.830 ... 01:03:34.255 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.255 00:19:58.830 ... 01:03:34.255 00:19:58.830 11 # Pci bus hotplug 01:03:34.255 00:19:58.830 12 # Helper function to remove/attach cotrollers 01:03:34.255 00:19:58.830 13 debug_remove_attach_helper() { 01:03:34.255 00:19:58.830 14 local helper_time=0 01:03:34.255 00:19:58.830 15 01:03:34.255 00:19:58.830 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.256 00:19:58.830 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.256 00:19:58.830 18 "$helper_time" "$nvme_count" >&2 01:03:34.256 00:19:58.830 19 } 01:03:34.256 00:19:58.830 20 01:03:34.256 00:19:58.830 21 remove_attach_helper() { 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.256 00:19:58.830 106 done 01:03:34.256 00:19:58.830 107 01:03:34.256 00:19:58.830 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.256 00:19:58.830 109 01:03:34.256 00:19:58.830 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.256 00:19:58.830 111 # Verify reregistering hotplug poller 01:03:34.256 00:19:58.830 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.256 00:19:58.830 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.256 00:19:58.830 114 01:03:34.256 00:19:58.830 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 133 01:03:34.256 00:19:58.830 134 # Run pci bus hotplug test 01:03:34.256 00:19:58.830 135 run_hotplug 01:03:34.256 00:19:58.830 136 01:03:34.256 00:19:58.830 137 # Run SPDK target based hotplug 01:03:34.256 00:19:58.830 => 138 tgt_run_hotplug 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 01:03:34.256 00:19:58.830 ========== Backtrace end ==========' 01:03:34.256 00:19:58.830 ========== Backtrace start: ========== 01:03:34.256 00:19:58.830 01:03:34.256 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 710 01:03:34.256 00:19:58.830 711 # We redirect cmd's std{out,err} to a separate fd dup'ed to stdin's device (or /dev/null) to 01:03:34.256 00:19:58.830 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:03:34.256 00:19:58.830 713 # but $() will return just the time's data, hence making it possible to just do: 01:03:34.256 00:19:58.830 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.256 00:19:58.830 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.256 00:19:58.830 716 01:03:34.256 00:19:58.830 717 echo "$time" 01:03:34.256 00:19:58.830 718 ) 01:03:34.256 00:19:58.830 719 01:03:34.256 00:19:58.830 720 function timing_enter() { 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 11 # Pci bus hotplug 01:03:34.256 00:19:58.830 12 # Helper function to remove/attach cotrollers 01:03:34.256 00:19:58.830 13 debug_remove_attach_helper() { 01:03:34.256 00:19:58.830 14 local helper_time=0 01:03:34.256 00:19:58.830 15 01:03:34.256 00:19:58.830 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.256 00:19:58.830 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 01:03:34.256 00:19:58.830 18 "$helper_time" "$nvme_count" >&2 01:03:34.256 00:19:58.830 19 } 01:03:34.256 00:19:58.830 20 01:03:34.256 00:19:58.830 21 remove_attach_helper() { 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.256 00:19:58.830 106 done 01:03:34.256 00:19:58.830 107 01:03:34.256 00:19:58.830 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.256 00:19:58.830 109 01:03:34.256 00:19:58.830 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.256 00:19:58.830 111 # Verify reregistering hotplug poller 01:03:34.256 00:19:58.830 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.256 00:19:58.830 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.256 00:19:58.830 114 01:03:34.256 00:19:58.830 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 133 01:03:34.256 00:19:58.830 134 # Run pci bus hotplug test 01:03:34.256 00:19:58.830 135 run_hotplug 01:03:34.256 00:19:58.830 136 01:03:34.256 00:19:58.830 137 # Run SPDK target based hotplug 01:03:34.256 00:19:58.830 => 138 tgt_run_hotplug 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 01:03:34.256 00:19:58.830 ========== Backtrace end ========== 01:03:34.256 00:19:58.830 ========== Backtrace start: ========== 01:03:34.256 00:19:58.830 01:03:34.256 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 713 # but $() will return just the time's data, hence making it possible to just do: 01:03:34.256 00:19:58.830 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.256 00:19:58.830 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.256 00:19:58.830 716 01:03:34.256 00:19:58.830 717 echo "$time" 01:03:34.256 00:19:58.830 => 718 ) 01:03:34.256 00:19:58.830 719 01:03:34.256 00:19:58.830 720 function timing_enter() { 01:03:34.256 00:19:58.830 721 xtrace_disable 01:03:34.256 00:19:58.830 722 timing "enter" "$1" 01:03:34.256 00:19:58.830 723 xtrace_restore 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 11 # Pci bus hotplug 01:03:34.256 00:19:58.830 12 # Helper function to remove/attach cotrollers 01:03:34.256 00:19:58.830 13 debug_remove_attach_helper() { 01:03:34.256 00:19:58.830 14 local helper_time=0 01:03:34.256 00:19:58.830 15 01:03:34.256 00:19:58.830 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.256 00:19:58.830 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 01:03:34.256 00:19:58.830 18 "$helper_time" "$nvme_count" >&2 01:03:34.256 00:19:58.830 19 } 01:03:34.256 00:19:58.830 20 01:03:34.256 00:19:58.830 21 remove_attach_helper() { 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.256 00:19:58.830 ... 01:03:34.256 00:19:58.830 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.256 00:19:58.831 106 done 01:03:34.256 00:19:58.831 107 01:03:34.256 00:19:58.831 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.256 00:19:58.831 109 01:03:34.256 00:19:58.831 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.256 00:19:58.831 111 # Verify reregistering hotplug poller 01:03:34.256 00:19:58.831 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.256 00:19:58.831 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.256 00:19:58.831 114 01:03:34.256 00:19:58.831 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.256 00:19:58.831 ... 01:03:34.257 00:19:58.831 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.257 00:19:58.831 ... 01:03:34.257 00:19:58.831 133 01:03:34.257 00:19:58.831 134 # Run pci bus hotplug test 01:03:34.257 00:19:58.831 135 run_hotplug 01:03:34.257 00:19:58.831 136 01:03:34.257 00:19:58.831 137 # Run SPDK target based hotplug 01:03:34.257 00:19:58.831 => 138 tgt_run_hotplug 01:03:34.257 00:19:58.831 ... 01:03:34.257 00:19:58.831 01:03:34.257 00:19:58.831 ========== Backtrace end ========== 01:03:34.257 00:19:58.831 ========== Backtrace start: ========== 01:03:34.257 00:19:58.831 01:03:34.257 00:19:58.831 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.257 00:19:58.831 ... 01:03:34.257 00:19:58.831 11 # Pci bus hotplug 01:03:34.257 00:19:58.831 12 # Helper function to remove/attach cotrollers 01:03:34.257 00:19:58.831 13 debug_remove_attach_helper() { 01:03:34.257 00:19:58.831 14 local helper_time=0 01:03:34.257 00:19:58.831 15 01:03:34.257 00:19:58.831 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.257 00:19:58.831 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 01:03:34.257 00:19:58.831 18 "$helper_time" "$nvme_count" >&2 01:03:34.257 00:19:58.831 19 } 01:03:34.257 00:19:58.831 20 01:03:34.257 00:19:58.831 21 remove_attach_helper() { 01:03:34.257 00:19:58.831 ... 01:03:34.257 00:19:58.831 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.257 00:19:58.831 ... 01:03:34.257 00:19:58.831 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.257 00:19:58.831 106 done 01:03:34.257 00:19:58.831 107 01:03:34.257 00:19:58.831 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.257 00:19:58.831 109 01:03:34.257 00:19:58.831 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.257 00:19:58.831 111 # Verify reregistering hotplug poller 01:03:34.257 00:19:58.831 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.257 00:19:58.831 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.257 00:19:58.831 114 01:03:34.257 00:19:58.831 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.257 00:19:58.831 ... 01:03:34.257 00:19:58.831 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.257 00:19:58.831 ... 01:03:34.257 00:19:58.831 133 01:03:34.257 00:19:58.831 134 # Run pci bus hotplug test 01:03:34.257 00:19:58.831 135 run_hotplug 01:03:34.257 00:19:58.831 136 01:03:34.257 00:19:58.831 137 # Run SPDK target based hotplug 01:03:34.257 00:19:58.831 => 138 tgt_run_hotplug 01:03:34.257 00:19:58.831 ... 01:03:34.257 00:19:58.831 01:03:34.257 00:19:58.831 ========== Backtrace end ========== 01:03:34.257 00:20:02.121 ========== Backtrace start: ========== 01:03:34.257 00:20:02.121 01:03:34.257 00:20:02.121 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 01:03:34.257 00:20:02.121 ... 01:03:34.257 00:20:02.121 1117 timing_enter $test_name 01:03:34.257 00:20:02.121 1118 echo "************************************" 01:03:34.257 00:20:02.121 1119 echo "START TEST $test_name" 01:03:34.257 00:20:02.121 1120 echo "************************************" 01:03:34.257 00:20:02.121 1121 xtrace_restore 01:03:34.257 00:20:02.122 1122 time "$@" 01:03:34.257 00:20:02.122 1123 xtrace_disable 01:03:34.257 00:20:02.122 1124 echo "************************************" 01:03:34.257 00:20:02.122 1125 echo "END TEST $test_name" 01:03:34.257 00:20:02.122 1126 echo "************************************" 01:03:34.257 00:20:02.122 1127 timing_exit $test_name 01:03:34.257 00:20:02.122 ... 01:03:34.257 00:20:02.122 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 01:03:34.257 00:20:02.122 ... 01:03:34.257 00:20:02.122 235 01:03:34.257 00:20:02.122 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 01:03:34.257 00:20:02.122 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 01:03:34.257 00:20:02.122 238 01:03:34.257 00:20:02.122 239 if [ $(uname -s) = Linux ]; then 01:03:34.257 00:20:02.122 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 01:03:34.257 00:20:02.122 241 fi 01:03:34.257 00:20:02.122 242 01:03:34.257 00:20:02.122 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 01:03:34.257 00:20:02.122 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 01:03:34.257 00:20:02.122 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 01:03:34.257 00:20:02.122 ... 01:03:34.257 00:20:02.122 01:03:34.257 00:20:02.122 ========== Backtrace end ========== ]] 01:03:34.257 + echo '00:19:58.830 10:00:36 sw_hotplug -- common/autotest_common.sh@715 -- # time='\''========== Backtrace start: ========== 01:03:34.257 00:19:58.830 01:03:34.257 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.257 00:19:58.830 ... 01:03:34.257 00:19:58.830 710 01:03:34.257 00:19:58.830 711 # We redirect cmd'\''\'\'''\''s std{out,err} to a separate fd dup'\''\'\'''\''ed to stdin'\''\'\'''\''s device (or /dev/null) to 01:03:34.257 00:19:58.830 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:03:34.257 00:19:58.830 713 # but $() will return just the time'\''\'\'''\''s data, hence making it possible to just do: 01:03:34.257 00:19:58.830 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.257 00:19:58.830 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.257 00:19:58.830 716 01:03:34.257 00:19:58.830 717 echo "$time" 01:03:34.257 00:19:58.830 718 ) 01:03:34.257 00:19:58.830 719 01:03:34.257 00:19:58.830 720 function timing_enter() { 01:03:34.257 00:19:58.830 ... 01:03:34.257 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.257 00:19:58.830 ... 01:03:34.257 00:19:58.830 11 # Pci bus hotplug 01:03:34.257 00:19:58.830 12 # Helper function to remove/attach cotrollers 01:03:34.257 00:19:58.830 13 debug_remove_attach_helper() { 01:03:34.257 00:19:58.830 14 local helper_time=0 01:03:34.257 00:19:58.830 15 01:03:34.257 00:19:58.830 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.257 00:19:58.830 17 printf '\''\'\'''\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\''\'\'''\'' \ 01:03:34.257 00:19:58.830 18 "$helper_time" "$nvme_count" >&2 01:03:34.257 00:19:58.830 19 } 01:03:34.257 00:19:58.830 20 01:03:34.257 00:19:58.830 21 remove_attach_helper() { 01:03:34.257 00:19:58.830 ... 01:03:34.257 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.257 00:19:58.830 ... 01:03:34.257 00:19:58.830 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.257 00:19:58.830 106 done 01:03:34.257 00:19:58.830 107 01:03:34.257 00:19:58.830 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.257 00:19:58.830 109 01:03:34.257 00:19:58.830 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.257 00:19:58.830 111 # Verify reregistering hotplug poller 01:03:34.257 00:19:58.830 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.257 00:19:58.830 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.257 00:19:58.830 114 01:03:34.257 00:19:58.830 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.257 00:19:58.830 ... 01:03:34.257 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.257 00:19:58.830 ... 01:03:34.257 00:19:58.830 133 01:03:34.258 00:19:58.830 134 # Run pci bus hotplug test 01:03:34.258 00:19:58.830 135 run_hotplug 01:03:34.258 00:19:58.830 136 01:03:34.258 00:19:58.830 137 # Run SPDK target based hotplug 01:03:34.258 00:19:58.830 => 138 tgt_run_hotplug 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 01:03:34.258 00:19:58.830 ========== Backtrace end =========='\'' 01:03:34.258 00:19:58.830 ========== Backtrace start: ========== 01:03:34.258 00:19:58.830 01:03:34.258 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 710 01:03:34.258 00:19:58.830 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 01:03:34.258 00:19:58.830 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:03:34.258 00:19:58.830 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:03:34.258 00:19:58.830 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.258 00:19:58.830 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.258 00:19:58.830 716 01:03:34.258 00:19:58.830 717 echo "$time" 01:03:34.258 00:19:58.830 718 ) 01:03:34.258 00:19:58.830 719 01:03:34.258 00:19:58.830 720 function timing_enter() { 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 11 # Pci bus hotplug 01:03:34.258 00:19:58.830 12 # Helper function to remove/attach cotrollers 01:03:34.258 00:19:58.830 13 debug_remove_attach_helper() { 01:03:34.258 00:19:58.830 14 local helper_time=0 01:03:34.258 00:19:58.830 15 01:03:34.258 00:19:58.830 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.258 00:19:58.830 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.258 00:19:58.830 18 "$helper_time" "$nvme_count" >&2 01:03:34.258 00:19:58.830 19 } 01:03:34.258 00:19:58.830 20 01:03:34.258 00:19:58.830 21 remove_attach_helper() { 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.258 00:19:58.830 106 done 01:03:34.258 00:19:58.830 107 01:03:34.258 00:19:58.830 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.258 00:19:58.830 109 01:03:34.258 00:19:58.830 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.258 00:19:58.830 111 # Verify reregistering hotplug poller 01:03:34.258 00:19:58.830 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.258 00:19:58.830 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.258 00:19:58.830 114 01:03:34.258 00:19:58.830 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 133 01:03:34.258 00:19:58.830 134 # Run pci bus hotplug test 01:03:34.258 00:19:58.830 135 run_hotplug 01:03:34.258 00:19:58.830 136 01:03:34.258 00:19:58.830 137 # Run SPDK target based hotplug 01:03:34.258 00:19:58.830 => 138 tgt_run_hotplug 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 01:03:34.258 00:19:58.830 ========== Backtrace end ========== 01:03:34.258 00:19:58.830 ========== Backtrace start: ========== 01:03:34.258 00:19:58.830 01:03:34.258 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:03:34.258 00:19:58.830 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.258 00:19:58.830 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.258 00:19:58.830 716 01:03:34.258 00:19:58.830 717 echo "$time" 01:03:34.258 00:19:58.830 => 718 ) 01:03:34.258 00:19:58.830 719 01:03:34.258 00:19:58.830 720 function timing_enter() { 01:03:34.258 00:19:58.830 721 xtrace_disable 01:03:34.258 00:19:58.830 722 timing "enter" "$1" 01:03:34.258 00:19:58.830 723 xtrace_restore 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 11 # Pci bus hotplug 01:03:34.258 00:19:58.830 12 # Helper function to remove/attach cotrollers 01:03:34.258 00:19:58.830 13 debug_remove_attach_helper() { 01:03:34.258 00:19:58.830 14 local helper_time=0 01:03:34.258 00:19:58.830 15 01:03:34.258 00:19:58.830 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.258 00:19:58.830 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.258 00:19:58.830 18 "$helper_time" "$nvme_count" >&2 01:03:34.258 00:19:58.830 19 } 01:03:34.258 00:19:58.830 20 01:03:34.258 00:19:58.830 21 remove_attach_helper() { 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.258 00:19:58.830 ... 01:03:34.258 00:19:58.830 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.258 00:19:58.831 106 done 01:03:34.258 00:19:58.831 107 01:03:34.258 00:19:58.831 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.258 00:19:58.831 109 01:03:34.258 00:19:58.831 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.258 00:19:58.831 111 # Verify reregistering hotplug poller 01:03:34.258 00:19:58.831 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.258 00:19:58.831 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.258 00:19:58.831 114 01:03:34.258 00:19:58.831 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.258 00:19:58.831 ... 01:03:34.258 00:19:58.831 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.258 00:19:58.831 ... 01:03:34.258 00:19:58.831 133 01:03:34.258 00:19:58.831 134 # Run pci bus hotplug test 01:03:34.258 00:19:58.831 135 run_hotplug 01:03:34.258 00:19:58.831 136 01:03:34.258 00:19:58.831 137 # Run SPDK target based hotplug 01:03:34.258 00:19:58.831 => 138 tgt_run_hotplug 01:03:34.258 00:19:58.831 ... 01:03:34.258 00:19:58.831 01:03:34.258 00:19:58.831 ========== Backtrace end ========== 01:03:34.258 00:19:58.831 ========== Backtrace start: ========== 01:03:34.258 00:19:58.831 01:03:34.258 00:19:58.831 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.258 00:19:58.831 ... 01:03:34.258 00:19:58.831 11 # Pci bus hotplug 01:03:34.258 00:19:58.831 12 # Helper function to remove/attach cotrollers 01:03:34.258 00:19:58.831 13 debug_remove_attach_helper() { 01:03:34.258 00:19:58.831 14 local helper_time=0 01:03:34.258 00:19:58.831 15 01:03:34.258 00:19:58.831 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.258 00:19:58.831 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.258 00:19:58.831 18 "$helper_time" "$nvme_count" >&2 01:03:34.258 00:19:58.831 19 } 01:03:34.258 00:19:58.831 20 01:03:34.258 00:19:58.831 21 remove_attach_helper() { 01:03:34.258 00:19:58.831 ... 01:03:34.258 00:19:58.831 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.258 00:19:58.831 ... 01:03:34.258 00:19:58.831 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.258 00:19:58.831 106 done 01:03:34.258 00:19:58.831 107 01:03:34.258 00:19:58.831 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.258 00:19:58.831 109 01:03:34.258 00:19:58.831 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.258 00:19:58.831 111 # Verify reregistering hotplug poller 01:03:34.258 00:19:58.831 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.258 00:19:58.831 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.258 00:19:58.831 114 01:03:34.258 00:19:58.831 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.258 00:19:58.831 ... 01:03:34.258 00:19:58.831 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.258 00:19:58.831 ... 01:03:34.258 00:19:58.831 133 01:03:34.259 00:19:58.831 134 # Run pci bus hotplug test 01:03:34.259 00:19:58.831 135 run_hotplug 01:03:34.259 00:19:58.831 136 01:03:34.259 00:19:58.831 137 # Run SPDK target based hotplug 01:03:34.259 00:19:58.831 => 138 tgt_run_hotplug 01:03:34.259 00:19:58.831 ... 01:03:34.259 00:19:58.831 01:03:34.259 00:19:58.831 ========== Backtrace end ========== 01:03:34.259 00:20:02.121 ========== Backtrace start: ========== 01:03:34.259 00:20:02.121 01:03:34.259 00:20:02.121 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 01:03:34.259 00:20:02.121 ... 01:03:34.259 00:20:02.121 1117 timing_enter $test_name 01:03:34.259 00:20:02.121 1118 echo "************************************" 01:03:34.259 00:20:02.121 1119 echo "START TEST $test_name" 01:03:34.259 00:20:02.121 1120 echo "************************************" 01:03:34.259 00:20:02.121 1121 xtrace_restore 01:03:34.259 00:20:02.122 1122 time "$@" 01:03:34.259 00:20:02.122 1123 xtrace_disable 01:03:34.259 00:20:02.122 1124 echo "************************************" 01:03:34.259 00:20:02.122 1125 echo "END TEST $test_name" 01:03:34.259 00:20:02.122 1126 echo "************************************" 01:03:34.259 00:20:02.122 1127 timing_exit $test_name 01:03:34.259 00:20:02.122 ... 01:03:34.259 00:20:02.122 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 01:03:34.259 00:20:02.122 ... 01:03:34.259 00:20:02.122 235 01:03:34.259 00:20:02.122 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 01:03:34.259 00:20:02.122 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 01:03:34.259 00:20:02.122 238 01:03:34.259 00:20:02.122 239 if [ $(uname -s) = Linux ]; then 01:03:34.259 00:20:02.122 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 01:03:34.259 00:20:02.122 241 fi 01:03:34.259 00:20:02.122 242 01:03:34.259 00:20:02.122 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 01:03:34.259 00:20:02.122 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 01:03:34.259 00:20:02.122 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 01:03:34.259 00:20:02.122 ... 01:03:34.259 00:20:02.122 01:03:34.259 00:20:02.122 ========== Backtrace end ==========' 01:03:34.259 + for build in "${build_log_list[@]}" 01:03:34.259 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_26220/build.log 01:03:34.259 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.259 + build_log=/var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_26220/build.log 01:03:34.259 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_26220/build.log 01:03:34.259 ++ basename /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_26220 01:03:34.259 + job_name=vfio-user-phy-autotest_26220 01:03:34.259 + start_pattern='Backtrace start' 01:03:34.259 + end_pattern='Backtrace end' 01:03:34.259 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_26220/build.log ]] 01:03:34.259 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_26220/build.log 01:03:34.259 + backtraces= 01:03:34.259 + [[ -n '' ]] 01:03:34.259 + for build in "${build_log_list[@]}" 01:03:34.259 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_22156/build.log 01:03:34.259 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.259 + build_log=/var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_22156/build.log 01:03:34.259 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_22156/build.log 01:03:34.259 ++ basename /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_22156 01:03:34.259 + job_name=check-format-docker-autotest_22156 01:03:34.259 + start_pattern='Backtrace start' 01:03:34.259 + end_pattern='Backtrace end' 01:03:34.259 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_22156/build.log ]] 01:03:34.259 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_22156/build.log 01:03:34.259 + backtraces= 01:03:34.259 + [[ -n '' ]] 01:03:34.259 + for build in "${build_log_list[@]}" 01:03:34.259 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_77483/build.log 01:03:34.259 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.259 + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_77483/build.log 01:03:34.259 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_77483/build.log 01:03:34.259 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_77483 01:03:34.259 + job_name=nvmf-tcp-vg-autotest_77483 01:03:34.259 + start_pattern='Backtrace start' 01:03:34.259 + end_pattern='Backtrace end' 01:03:34.259 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_77483/build.log ]] 01:03:34.259 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_77483/build.log 01:03:34.259 + backtraces= 01:03:34.259 + [[ -n '' ]] 01:03:34.259 + for build in "${build_log_list[@]}" 01:03:34.259 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_26209/build.log 01:03:34.259 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.259 + build_log=/var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_26209/build.log 01:03:34.259 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_26209/build.log 01:03:34.259 ++ basename /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_26209 01:03:34.259 + job_name=short-fuzz-phy-autotest_26209 01:03:34.259 + start_pattern='Backtrace start' 01:03:34.259 + end_pattern='Backtrace end' 01:03:34.259 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_26209/build.log ]] 01:03:34.259 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_26209/build.log 01:03:34.259 + backtraces= 01:03:34.259 + [[ -n '' ]] 01:03:34.259 + for build in "${build_log_list[@]}" 01:03:34.259 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_77397/build.log 01:03:34.259 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.259 + build_log=/var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_77397/build.log 01:03:34.259 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_77397/build.log 01:03:34.259 ++ basename /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_77397 01:03:34.259 + job_name=iscsi-uring-vg-autotest_77397 01:03:34.259 + start_pattern='Backtrace start' 01:03:34.259 + end_pattern='Backtrace end' 01:03:34.259 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_77397/build.log ]] 01:03:34.259 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_77397/build.log 01:03:34.259 + backtraces= 01:03:34.259 + [[ -n '' ]] 01:03:34.259 + for build in "${build_log_list[@]}" 01:03:34.259 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_19876/build.log 01:03:34.259 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.259 + build_log=/var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_19876/build.log 01:03:34.259 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_19876/build.log 01:03:34.259 ++ basename /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_19876 01:03:34.259 + job_name=scan-build-docker-autotest_19876 01:03:34.259 + start_pattern='Backtrace start' 01:03:34.259 + end_pattern='Backtrace end' 01:03:34.259 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_19876/build.log ]] 01:03:34.259 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_19876/build.log 01:03:34.259 + backtraces= 01:03:34.259 + [[ -n '' ]] 01:03:34.259 + for build in "${build_log_list[@]}" 01:03:34.259 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_72384/build.log 01:03:34.259 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.259 + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_72384/build.log 01:03:34.259 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_72384/build.log 01:03:34.259 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_72384 01:03:34.259 + job_name=nvme-phy-autotest_72384 01:03:34.259 + start_pattern='Backtrace start' 01:03:34.259 + end_pattern='Backtrace end' 01:03:34.259 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_72384/build.log ]] 01:03:34.259 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_72384/build.log 01:03:34.259 + backtraces= 01:03:34.259 + [[ -n '' ]] 01:03:34.259 + for build in "${build_log_list[@]}" 01:03:34.259 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_77441/build.log 01:03:34.259 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.259 + build_log=/var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_77441/build.log 01:03:34.260 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_77441/build.log 01:03:34.260 ++ basename /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_77441 01:03:34.260 + job_name=lvol-vg-autotest_77441 01:03:34.260 + start_pattern='Backtrace start' 01:03:34.260 + end_pattern='Backtrace end' 01:03:34.260 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_77441/build.log ]] 01:03:34.260 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_77441/build.log 01:03:34.260 + backtraces= 01:03:34.260 + [[ -n '' ]] 01:03:34.260 + for build in "${build_log_list[@]}" 01:03:34.260 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/dsa-phy-autotest_4912/build.log 01:03:34.260 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.260 + build_log=/var/jenkins/workspace/autotest-per-patch_2/dsa-phy-autotest_4912/build.log 01:03:34.260 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/dsa-phy-autotest_4912/build.log 01:03:34.260 ++ basename /var/jenkins/workspace/autotest-per-patch_2/dsa-phy-autotest_4912 01:03:34.260 + job_name=dsa-phy-autotest_4912 01:03:34.260 + start_pattern='Backtrace start' 01:03:34.260 + end_pattern='Backtrace end' 01:03:34.260 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/dsa-phy-autotest_4912/build.log ]] 01:03:34.260 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/dsa-phy-autotest_4912/build.log 01:03:34.260 + backtraces= 01:03:34.260 + [[ -n '' ]] 01:03:34.260 + for build in "${build_log_list[@]}" 01:03:34.260 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_72456/build.log 01:03:34.260 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.260 + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_72456/build.log 01:03:34.260 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_72456/build.log 01:03:34.260 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_72456 01:03:34.260 + job_name=nvmf-phy-autotest_72456 01:03:34.260 + start_pattern='Backtrace start' 01:03:34.260 + end_pattern='Backtrace end' 01:03:34.260 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_72456/build.log ]] 01:03:34.260 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_72456/build.log 01:03:34.260 + backtraces= 01:03:34.260 + [[ -n '' ]] 01:03:34.260 + for build in "${build_log_list[@]}" 01:03:34.260 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_73671/build.log 01:03:34.260 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.260 + build_log=/var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_73671/build.log 01:03:34.260 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_73671/build.log 01:03:34.260 ++ basename /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_73671 01:03:34.260 + job_name=centos7-vg-autotest_73671 01:03:34.260 + start_pattern='Backtrace start' 01:03:34.260 + end_pattern='Backtrace end' 01:03:34.260 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_73671/build.log ]] 01:03:34.260 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_73671/build.log 01:03:34.260 + backtraces= 01:03:34.260 + [[ -n '' ]] 01:03:34.260 + for build in "${build_log_list[@]}" 01:03:34.260 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_37825/build.log 01:03:34.260 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.260 + build_log=/var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_37825/build.log 01:03:34.260 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_37825/build.log 01:03:34.260 ++ basename /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_37825 01:03:34.260 + job_name=ubuntu22-vg-autotest_37825 01:03:34.260 + start_pattern='Backtrace start' 01:03:34.260 + end_pattern='Backtrace end' 01:03:34.260 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_37825/build.log ]] 01:03:34.260 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_37825/build.log 01:03:34.260 + backtraces= 01:03:34.260 + [[ -n '' ]] 01:03:34.260 + for build in "${build_log_list[@]}" 01:03:34.260 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_26218/build.log 01:03:34.260 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.260 + build_log=/var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_26218/build.log 01:03:34.260 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_26218/build.log 01:03:34.260 ++ basename /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_26218 01:03:34.260 + job_name=BlobFS-phy-autotest_26218 01:03:34.260 + start_pattern='Backtrace start' 01:03:34.260 + end_pattern='Backtrace end' 01:03:34.260 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_26218/build.log ]] 01:03:34.260 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_26218/build.log 01:03:34.260 + backtraces= 01:03:34.260 + [[ -n '' ]] 01:03:34.260 + for build in "${build_log_list[@]}" 01:03:34.260 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_19877/build.log 01:03:34.260 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.260 + build_log=/var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_19877/build.log 01:03:34.260 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_19877/build.log 01:03:34.260 ++ basename /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_19877 01:03:34.260 + job_name=unittest-valgrind-docker-autotest_19877 01:03:34.260 + start_pattern='Backtrace start' 01:03:34.260 + end_pattern='Backtrace end' 01:03:34.260 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_19877/build.log ]] 01:03:34.260 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_19877/build.log 01:03:34.260 + backtraces= 01:03:34.260 + [[ -n '' ]] 01:03:34.260 + for build in "${build_log_list[@]}" 01:03:34.261 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_76598/build.log 01:03:34.261 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.261 + build_log=/var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_76598/build.log 01:03:34.261 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_76598/build.log 01:03:34.261 ++ basename /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_76598 01:03:34.261 + job_name=zns-vg-autotest_76598 01:03:34.261 + start_pattern='Backtrace start' 01:03:34.261 + end_pattern='Backtrace end' 01:03:34.261 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_76598/build.log ]] 01:03:34.261 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_76598/build.log 01:03:34.261 + backtraces= 01:03:34.261 + [[ -n '' ]] 01:03:34.261 + for build in "${build_log_list[@]}" 01:03:34.261 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_77417/build.log 01:03:34.261 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.261 + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_77417/build.log 01:03:34.261 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_77417/build.log 01:03:34.261 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_77417 01:03:34.261 + job_name=nvmf-tcp-uring-vg-autotest_77417 01:03:34.261 + start_pattern='Backtrace start' 01:03:34.261 + end_pattern='Backtrace end' 01:03:34.261 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_77417/build.log ]] 01:03:34.261 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_77417/build.log 01:03:34.261 + backtraces= 01:03:34.261 + [[ -n '' ]] 01:03:34.261 + for build in "${build_log_list[@]}" 01:03:34.261 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_73811/build.log 01:03:34.261 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.261 + build_log=/var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_73811/build.log 01:03:34.261 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_73811/build.log 01:03:34.261 ++ basename /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_73811 01:03:34.261 + job_name=ubuntu20-vg-autotest_73811 01:03:34.261 + start_pattern='Backtrace start' 01:03:34.261 + end_pattern='Backtrace end' 01:03:34.261 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_73811/build.log ]] 01:03:34.261 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_73811/build.log 01:03:34.261 + backtraces='00:53:30.490 10:34:06 sw_hotplug -- common/autotest_common.sh@715 -- # time='\''========== Backtrace start: ========== 01:03:34.261 00:53:30.490 01:03:34.261 00:53:30.490 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.261 00:53:30.490 ... 01:03:34.261 00:53:30.490 710 01:03:34.261 00:53:30.490 711 # We redirect cmd'\''\'\'''\''s std{out,err} to a separate fd dup'\''\'\'''\''ed to stdin'\''\'\'''\''s device (or /dev/null) to 01:03:34.261 00:53:30.490 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:03:34.261 00:53:30.490 713 # but $() will return just the time'\''\'\'''\''s data, hence making it possible to just do: 01:03:34.261 00:53:30.490 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.261 00:53:30.490 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.261 00:53:30.490 716 01:03:34.261 00:53:30.490 717 echo "$time" 01:03:34.261 00:53:30.490 718 ) 01:03:34.261 00:53:30.490 719 01:03:34.261 00:53:30.490 720 function timing_enter() { 01:03:34.261 00:53:30.490 ... 01:03:34.261 00:53:30.490 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.261 00:53:30.490 ... 01:03:34.261 00:53:30.490 11 # Pci bus hotplug 01:03:34.261 00:53:30.490 12 # Helper function to remove/attach cotrollers 01:03:34.261 00:53:30.490 13 debug_remove_attach_helper() { 01:03:34.261 00:53:30.490 14 local helper_time=0 01:03:34.261 00:53:30.490 15 01:03:34.261 00:53:30.490 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.261 00:53:30.490 17 printf '\''\'\'''\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\''\'\'''\'' \ 01:03:34.261 00:53:30.490 18 "$helper_time" "$nvme_count" >&2 01:03:34.261 00:53:30.490 19 } 01:03:34.261 00:53:30.490 20 01:03:34.261 00:53:30.490 21 remove_attach_helper() { 01:03:34.261 00:53:30.490 ... 01:03:34.261 00:53:30.490 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.261 00:53:30.490 ... 01:03:34.261 00:53:30.490 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.261 00:53:30.490 106 done 01:03:34.261 00:53:30.490 107 01:03:34.261 00:53:30.490 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.261 00:53:30.490 109 01:03:34.261 00:53:30.490 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.261 00:53:30.490 111 # Verify reregistering hotplug poller 01:03:34.261 00:53:30.490 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.261 00:53:30.490 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.261 00:53:30.490 114 01:03:34.261 00:53:30.490 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.261 00:53:30.490 ... 01:03:34.261 00:53:30.490 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.261 00:53:30.490 ... 01:03:34.261 00:53:30.490 133 01:03:34.261 00:53:30.490 134 # Run pci bus hotplug test 01:03:34.261 00:53:30.490 135 run_hotplug 01:03:34.261 00:53:30.490 136 01:03:34.261 00:53:30.490 137 # Run SPDK target based hotplug 01:03:34.261 00:53:30.490 => 138 tgt_run_hotplug 01:03:34.261 00:53:30.490 ... 01:03:34.261 00:53:30.490 01:03:34.261 00:53:30.490 ========== Backtrace end ========== 01:03:34.261 00:53:30.490 ========== Backtrace start: ========== 01:03:34.261 00:53:30.490 01:03:34.261 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.261 00:53:30.491 ... 01:03:34.261 00:53:30.491 710 01:03:34.261 00:53:30.491 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 01:03:34.261 00:53:30.491 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:03:34.261 00:53:30.491 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:03:34.261 00:53:30.491 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.261 00:53:30.491 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.261 00:53:30.491 716 01:03:34.261 00:53:30.491 717 echo "$time" 01:03:34.261 00:53:30.491 718 ) 01:03:34.261 00:53:30.491 719 01:03:34.261 00:53:30.491 720 function timing_enter() { 01:03:34.261 00:53:30.491 ... 01:03:34.261 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.261 00:53:30.491 ... 01:03:34.261 00:53:30.491 11 # Pci bus hotplug 01:03:34.261 00:53:30.491 12 # Helper function to remove/attach cotrollers 01:03:34.261 00:53:30.491 13 debug_remove_attach_helper() { 01:03:34.261 00:53:30.491 14 local helper_time=0 01:03:34.261 00:53:30.491 15 01:03:34.261 00:53:30.491 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.261 00:53:30.491 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.261 00:53:30.491 18 "$helper_time" "$nvme_count" >&2 01:03:34.261 00:53:30.491 19 } 01:03:34.261 00:53:30.491 20 01:03:34.261 00:53:30.491 21 remove_attach_helper() { 01:03:34.261 00:53:30.491 ... 01:03:34.261 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.261 00:53:30.491 ... 01:03:34.261 00:53:30.491 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.262 00:53:30.491 106 done 01:03:34.262 00:53:30.491 107 01:03:34.262 00:53:30.491 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.262 00:53:30.491 109 01:03:34.262 00:53:30.491 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.262 00:53:30.491 111 # Verify reregistering hotplug poller 01:03:34.262 00:53:30.491 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.262 00:53:30.491 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.262 00:53:30.491 114 01:03:34.262 00:53:30.491 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 133 01:03:34.262 00:53:30.491 134 # Run pci bus hotplug test 01:03:34.262 00:53:30.491 135 run_hotplug 01:03:34.262 00:53:30.491 136 01:03:34.262 00:53:30.491 137 # Run SPDK target based hotplug 01:03:34.262 00:53:30.491 => 138 tgt_run_hotplug 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 01:03:34.262 00:53:30.491 ========== Backtrace end ========== 01:03:34.262 00:53:30.491 ========== Backtrace start: ========== 01:03:34.262 00:53:30.491 01:03:34.262 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:03:34.262 00:53:30.491 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.262 00:53:30.491 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.262 00:53:30.491 716 01:03:34.262 00:53:30.491 717 echo "$time" 01:03:34.262 00:53:30.491 => 718 ) 01:03:34.262 00:53:30.491 719 01:03:34.262 00:53:30.491 720 function timing_enter() { 01:03:34.262 00:53:30.491 721 xtrace_disable 01:03:34.262 00:53:30.491 722 timing "enter" "$1" 01:03:34.262 00:53:30.491 723 xtrace_restore 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 11 # Pci bus hotplug 01:03:34.262 00:53:30.491 12 # Helper function to remove/attach cotrollers 01:03:34.262 00:53:30.491 13 debug_remove_attach_helper() { 01:03:34.262 00:53:30.491 14 local helper_time=0 01:03:34.262 00:53:30.491 15 01:03:34.262 00:53:30.491 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.262 00:53:30.491 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.262 00:53:30.491 18 "$helper_time" "$nvme_count" >&2 01:03:34.262 00:53:30.491 19 } 01:03:34.262 00:53:30.491 20 01:03:34.262 00:53:30.491 21 remove_attach_helper() { 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.262 00:53:30.491 106 done 01:03:34.262 00:53:30.491 107 01:03:34.262 00:53:30.491 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.262 00:53:30.491 109 01:03:34.262 00:53:30.491 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.262 00:53:30.491 111 # Verify reregistering hotplug poller 01:03:34.262 00:53:30.491 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.262 00:53:30.491 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.262 00:53:30.491 114 01:03:34.262 00:53:30.491 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 133 01:03:34.262 00:53:30.491 134 # Run pci bus hotplug test 01:03:34.262 00:53:30.491 135 run_hotplug 01:03:34.262 00:53:30.491 136 01:03:34.262 00:53:30.491 137 # Run SPDK target based hotplug 01:03:34.262 00:53:30.491 => 138 tgt_run_hotplug 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 01:03:34.262 00:53:30.491 ========== Backtrace end ========== 01:03:34.262 00:53:30.491 ========== Backtrace start: ========== 01:03:34.262 00:53:30.491 01:03:34.262 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 11 # Pci bus hotplug 01:03:34.262 00:53:30.491 12 # Helper function to remove/attach cotrollers 01:03:34.262 00:53:30.491 13 debug_remove_attach_helper() { 01:03:34.262 00:53:30.491 14 local helper_time=0 01:03:34.262 00:53:30.491 15 01:03:34.262 00:53:30.491 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.262 00:53:30.491 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.262 00:53:30.491 18 "$helper_time" "$nvme_count" >&2 01:03:34.262 00:53:30.491 19 } 01:03:34.262 00:53:30.491 20 01:03:34.262 00:53:30.491 21 remove_attach_helper() { 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.262 00:53:30.491 106 done 01:03:34.262 00:53:30.491 107 01:03:34.262 00:53:30.491 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.262 00:53:30.491 109 01:03:34.262 00:53:30.491 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.262 00:53:30.491 111 # Verify reregistering hotplug poller 01:03:34.262 00:53:30.491 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.262 00:53:30.491 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.262 00:53:30.491 114 01:03:34.262 00:53:30.491 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 133 01:03:34.262 00:53:30.491 134 # Run pci bus hotplug test 01:03:34.262 00:53:30.491 135 run_hotplug 01:03:34.262 00:53:30.491 136 01:03:34.262 00:53:30.491 137 # Run SPDK target based hotplug 01:03:34.262 00:53:30.491 => 138 tgt_run_hotplug 01:03:34.262 00:53:30.491 ... 01:03:34.262 00:53:30.491 01:03:34.262 00:53:30.491 ========== Backtrace end ========== 01:03:34.262 00:53:32.647 ========== Backtrace start: ========== 01:03:34.262 00:53:32.647 01:03:34.262 00:53:32.647 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 01:03:34.262 00:53:32.647 ... 01:03:34.262 00:53:32.647 1117 timing_enter $test_name 01:03:34.262 00:53:32.647 1118 echo "************************************" 01:03:34.262 00:53:32.647 1119 echo "START TEST $test_name" 01:03:34.262 00:53:32.647 1120 echo "************************************" 01:03:34.262 00:53:32.647 1121 xtrace_restore 01:03:34.262 00:53:32.647 1122 time "$@" 01:03:34.262 00:53:32.647 1123 xtrace_disable 01:03:34.262 00:53:32.647 1124 echo "************************************" 01:03:34.262 00:53:32.647 1125 echo "END TEST $test_name" 01:03:34.262 00:53:32.647 1126 echo "************************************" 01:03:34.262 00:53:32.647 1127 timing_exit $test_name 01:03:34.262 00:53:32.647 ... 01:03:34.262 00:53:32.647 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 01:03:34.262 00:53:32.647 ... 01:03:34.262 00:53:32.647 235 01:03:34.262 00:53:32.647 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 01:03:34.262 00:53:32.647 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 01:03:34.262 00:53:32.647 238 01:03:34.262 00:53:32.647 239 if [ $(uname -s) = Linux ]; then 01:03:34.262 00:53:32.647 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 01:03:34.262 00:53:32.647 241 fi 01:03:34.262 00:53:32.647 242 01:03:34.262 00:53:32.647 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 01:03:34.262 00:53:32.647 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 01:03:34.262 00:53:32.647 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 01:03:34.262 00:53:32.647 ... 01:03:34.262 00:53:32.647 01:03:34.262 00:53:32.647 ========== Backtrace end ==========' 01:03:34.263 + [[ -n 00:53:30.490 10:34:06 sw_hotplug -- common/autotest_common.sh@715 -- # time='========== Backtrace start: ========== 01:03:34.263 00:53:30.490 01:03:34.263 00:53:30.490 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.263 00:53:30.490 ... 01:03:34.263 00:53:30.490 710 01:03:34.263 00:53:30.490 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 01:03:34.263 00:53:30.490 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:03:34.263 00:53:30.490 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:03:34.263 00:53:30.490 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.263 00:53:30.490 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.263 00:53:30.490 716 01:03:34.263 00:53:30.490 717 echo "$time" 01:03:34.263 00:53:30.490 718 ) 01:03:34.263 00:53:30.490 719 01:03:34.263 00:53:30.490 720 function timing_enter() { 01:03:34.263 00:53:30.490 ... 01:03:34.263 00:53:30.490 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.263 00:53:30.490 ... 01:03:34.263 00:53:30.490 11 # Pci bus hotplug 01:03:34.263 00:53:30.490 12 # Helper function to remove/attach cotrollers 01:03:34.263 00:53:30.490 13 debug_remove_attach_helper() { 01:03:34.263 00:53:30.490 14 local helper_time=0 01:03:34.263 00:53:30.490 15 01:03:34.263 00:53:30.490 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.263 00:53:30.490 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.263 00:53:30.490 18 "$helper_time" "$nvme_count" >&2 01:03:34.263 00:53:30.490 19 } 01:03:34.263 00:53:30.490 20 01:03:34.263 00:53:30.490 21 remove_attach_helper() { 01:03:34.263 00:53:30.490 ... 01:03:34.263 00:53:30.490 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.263 00:53:30.490 ... 01:03:34.263 00:53:30.490 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.263 00:53:30.490 106 done 01:03:34.263 00:53:30.490 107 01:03:34.263 00:53:30.490 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.263 00:53:30.490 109 01:03:34.263 00:53:30.490 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.263 00:53:30.490 111 # Verify reregistering hotplug poller 01:03:34.263 00:53:30.490 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.263 00:53:30.490 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.263 00:53:30.490 114 01:03:34.263 00:53:30.490 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.263 00:53:30.490 ... 01:03:34.263 00:53:30.490 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.263 00:53:30.490 ... 01:03:34.263 00:53:30.490 133 01:03:34.263 00:53:30.490 134 # Run pci bus hotplug test 01:03:34.263 00:53:30.490 135 run_hotplug 01:03:34.263 00:53:30.490 136 01:03:34.263 00:53:30.490 137 # Run SPDK target based hotplug 01:03:34.263 00:53:30.490 => 138 tgt_run_hotplug 01:03:34.263 00:53:30.490 ... 01:03:34.263 00:53:30.490 01:03:34.263 00:53:30.490 ========== Backtrace end ========== 01:03:34.263 00:53:30.490 ========== Backtrace start: ========== 01:03:34.263 00:53:30.490 01:03:34.263 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 710 01:03:34.263 00:53:30.491 711 # We redirect cmd's std{out,err} to a separate fd dup'ed to stdin's device (or /dev/null) to 01:03:34.263 00:53:30.491 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:03:34.263 00:53:30.491 713 # but $() will return just the time's data, hence making it possible to just do: 01:03:34.263 00:53:30.491 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.263 00:53:30.491 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.263 00:53:30.491 716 01:03:34.263 00:53:30.491 717 echo "$time" 01:03:34.263 00:53:30.491 718 ) 01:03:34.263 00:53:30.491 719 01:03:34.263 00:53:30.491 720 function timing_enter() { 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 11 # Pci bus hotplug 01:03:34.263 00:53:30.491 12 # Helper function to remove/attach cotrollers 01:03:34.263 00:53:30.491 13 debug_remove_attach_helper() { 01:03:34.263 00:53:30.491 14 local helper_time=0 01:03:34.263 00:53:30.491 15 01:03:34.263 00:53:30.491 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.263 00:53:30.491 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 01:03:34.263 00:53:30.491 18 "$helper_time" "$nvme_count" >&2 01:03:34.263 00:53:30.491 19 } 01:03:34.263 00:53:30.491 20 01:03:34.263 00:53:30.491 21 remove_attach_helper() { 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.263 00:53:30.491 106 done 01:03:34.263 00:53:30.491 107 01:03:34.263 00:53:30.491 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.263 00:53:30.491 109 01:03:34.263 00:53:30.491 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.263 00:53:30.491 111 # Verify reregistering hotplug poller 01:03:34.263 00:53:30.491 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.263 00:53:30.491 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.263 00:53:30.491 114 01:03:34.263 00:53:30.491 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 133 01:03:34.263 00:53:30.491 134 # Run pci bus hotplug test 01:03:34.263 00:53:30.491 135 run_hotplug 01:03:34.263 00:53:30.491 136 01:03:34.263 00:53:30.491 137 # Run SPDK target based hotplug 01:03:34.263 00:53:30.491 => 138 tgt_run_hotplug 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 01:03:34.263 00:53:30.491 ========== Backtrace end ========== 01:03:34.263 00:53:30.491 ========== Backtrace start: ========== 01:03:34.263 00:53:30.491 01:03:34.263 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 713 # but $() will return just the time's data, hence making it possible to just do: 01:03:34.263 00:53:30.491 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.263 00:53:30.491 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.263 00:53:30.491 716 01:03:34.263 00:53:30.491 717 echo "$time" 01:03:34.263 00:53:30.491 => 718 ) 01:03:34.263 00:53:30.491 719 01:03:34.263 00:53:30.491 720 function timing_enter() { 01:03:34.263 00:53:30.491 721 xtrace_disable 01:03:34.263 00:53:30.491 722 timing "enter" "$1" 01:03:34.263 00:53:30.491 723 xtrace_restore 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 11 # Pci bus hotplug 01:03:34.263 00:53:30.491 12 # Helper function to remove/attach cotrollers 01:03:34.263 00:53:30.491 13 debug_remove_attach_helper() { 01:03:34.263 00:53:30.491 14 local helper_time=0 01:03:34.263 00:53:30.491 15 01:03:34.263 00:53:30.491 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.263 00:53:30.491 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 01:03:34.263 00:53:30.491 18 "$helper_time" "$nvme_count" >&2 01:03:34.263 00:53:30.491 19 } 01:03:34.263 00:53:30.491 20 01:03:34.263 00:53:30.491 21 remove_attach_helper() { 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.263 00:53:30.491 106 done 01:03:34.263 00:53:30.491 107 01:03:34.263 00:53:30.491 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.263 00:53:30.491 109 01:03:34.263 00:53:30.491 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.263 00:53:30.491 111 # Verify reregistering hotplug poller 01:03:34.263 00:53:30.491 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.263 00:53:30.491 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.263 00:53:30.491 114 01:03:34.263 00:53:30.491 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 133 01:03:34.263 00:53:30.491 134 # Run pci bus hotplug test 01:03:34.263 00:53:30.491 135 run_hotplug 01:03:34.263 00:53:30.491 136 01:03:34.263 00:53:30.491 137 # Run SPDK target based hotplug 01:03:34.263 00:53:30.491 => 138 tgt_run_hotplug 01:03:34.263 00:53:30.491 ... 01:03:34.263 00:53:30.491 01:03:34.263 00:53:30.491 ========== Backtrace end ========== 01:03:34.263 00:53:30.491 ========== Backtrace start: ========== 01:03:34.263 00:53:30.491 01:03:34.264 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.264 00:53:30.491 ... 01:03:34.264 00:53:30.491 11 # Pci bus hotplug 01:03:34.264 00:53:30.491 12 # Helper function to remove/attach cotrollers 01:03:34.264 00:53:30.491 13 debug_remove_attach_helper() { 01:03:34.264 00:53:30.491 14 local helper_time=0 01:03:34.264 00:53:30.491 15 01:03:34.264 00:53:30.491 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.264 00:53:30.491 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 01:03:34.264 00:53:30.491 18 "$helper_time" "$nvme_count" >&2 01:03:34.264 00:53:30.491 19 } 01:03:34.264 00:53:30.491 20 01:03:34.264 00:53:30.491 21 remove_attach_helper() { 01:03:34.264 00:53:30.491 ... 01:03:34.264 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.264 00:53:30.491 ... 01:03:34.264 00:53:30.491 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.264 00:53:30.491 106 done 01:03:34.264 00:53:30.491 107 01:03:34.264 00:53:30.491 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.264 00:53:30.491 109 01:03:34.264 00:53:30.491 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.264 00:53:30.491 111 # Verify reregistering hotplug poller 01:03:34.264 00:53:30.491 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.264 00:53:30.491 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.264 00:53:30.491 114 01:03:34.264 00:53:30.491 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.264 00:53:30.491 ... 01:03:34.264 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.264 00:53:30.491 ... 01:03:34.264 00:53:30.491 133 01:03:34.264 00:53:30.491 134 # Run pci bus hotplug test 01:03:34.264 00:53:30.491 135 run_hotplug 01:03:34.264 00:53:30.491 136 01:03:34.264 00:53:30.491 137 # Run SPDK target based hotplug 01:03:34.264 00:53:30.491 => 138 tgt_run_hotplug 01:03:34.264 00:53:30.491 ... 01:03:34.264 00:53:30.491 01:03:34.264 00:53:30.491 ========== Backtrace end ========== 01:03:34.264 00:53:32.647 ========== Backtrace start: ========== 01:03:34.264 00:53:32.647 01:03:34.264 00:53:32.647 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 01:03:34.264 00:53:32.647 ... 01:03:34.264 00:53:32.647 1117 timing_enter $test_name 01:03:34.264 00:53:32.647 1118 echo "************************************" 01:03:34.264 00:53:32.647 1119 echo "START TEST $test_name" 01:03:34.264 00:53:32.647 1120 echo "************************************" 01:03:34.264 00:53:32.647 1121 xtrace_restore 01:03:34.264 00:53:32.647 1122 time "$@" 01:03:34.264 00:53:32.647 1123 xtrace_disable 01:03:34.264 00:53:32.647 1124 echo "************************************" 01:03:34.264 00:53:32.647 1125 echo "END TEST $test_name" 01:03:34.264 00:53:32.647 1126 echo "************************************" 01:03:34.264 00:53:32.647 1127 timing_exit $test_name 01:03:34.264 00:53:32.647 ... 01:03:34.264 00:53:32.647 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 01:03:34.264 00:53:32.647 ... 01:03:34.264 00:53:32.647 235 01:03:34.264 00:53:32.647 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 01:03:34.264 00:53:32.647 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 01:03:34.264 00:53:32.647 238 01:03:34.264 00:53:32.647 239 if [ $(uname -s) = Linux ]; then 01:03:34.264 00:53:32.647 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 01:03:34.264 00:53:32.647 241 fi 01:03:34.264 00:53:32.647 242 01:03:34.264 00:53:32.647 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 01:03:34.264 00:53:32.647 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 01:03:34.264 00:53:32.647 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 01:03:34.264 00:53:32.647 ... 01:03:34.264 00:53:32.647 01:03:34.264 00:53:32.647 ========== Backtrace end ========== ]] 01:03:34.264 + echo '00:53:30.490 10:34:06 sw_hotplug -- common/autotest_common.sh@715 -- # time='\''========== Backtrace start: ========== 01:03:34.264 00:53:30.490 01:03:34.264 00:53:30.490 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.264 00:53:30.490 ... 01:03:34.264 00:53:30.490 710 01:03:34.264 00:53:30.490 711 # We redirect cmd'\''\'\'''\''s std{out,err} to a separate fd dup'\''\'\'''\''ed to stdin'\''\'\'''\''s device (or /dev/null) to 01:03:34.264 00:53:30.490 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:03:34.264 00:53:30.490 713 # but $() will return just the time'\''\'\'''\''s data, hence making it possible to just do: 01:03:34.264 00:53:30.490 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.264 00:53:30.490 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.264 00:53:30.490 716 01:03:34.264 00:53:30.490 717 echo "$time" 01:03:34.264 00:53:30.490 718 ) 01:03:34.264 00:53:30.490 719 01:03:34.264 00:53:30.490 720 function timing_enter() { 01:03:34.264 00:53:30.490 ... 01:03:34.264 00:53:30.490 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.264 00:53:30.490 ... 01:03:34.264 00:53:30.490 11 # Pci bus hotplug 01:03:34.264 00:53:30.490 12 # Helper function to remove/attach cotrollers 01:03:34.264 00:53:30.490 13 debug_remove_attach_helper() { 01:03:34.264 00:53:30.490 14 local helper_time=0 01:03:34.264 00:53:30.490 15 01:03:34.264 00:53:30.490 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.264 00:53:30.490 17 printf '\''\'\'''\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\''\'\'''\'' \ 01:03:34.264 00:53:30.490 18 "$helper_time" "$nvme_count" >&2 01:03:34.264 00:53:30.490 19 } 01:03:34.264 00:53:30.490 20 01:03:34.264 00:53:30.490 21 remove_attach_helper() { 01:03:34.264 00:53:30.490 ... 01:03:34.264 00:53:30.490 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.264 00:53:30.490 ... 01:03:34.264 00:53:30.490 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.264 00:53:30.490 106 done 01:03:34.264 00:53:30.490 107 01:03:34.264 00:53:30.490 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.264 00:53:30.490 109 01:03:34.264 00:53:30.490 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.264 00:53:30.490 111 # Verify reregistering hotplug poller 01:03:34.264 00:53:30.490 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.264 00:53:30.490 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.264 00:53:30.490 114 01:03:34.264 00:53:30.490 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.264 00:53:30.490 ... 01:03:34.264 00:53:30.490 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.264 00:53:30.490 ... 01:03:34.264 00:53:30.490 133 01:03:34.264 00:53:30.490 134 # Run pci bus hotplug test 01:03:34.264 00:53:30.490 135 run_hotplug 01:03:34.264 00:53:30.490 136 01:03:34.264 00:53:30.490 137 # Run SPDK target based hotplug 01:03:34.264 00:53:30.490 => 138 tgt_run_hotplug 01:03:34.264 00:53:30.490 ... 01:03:34.264 00:53:30.490 01:03:34.264 00:53:30.490 ========== Backtrace end ========== 01:03:34.264 00:53:30.490 ========== Backtrace start: ========== 01:03:34.264 00:53:30.490 01:03:34.264 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.264 00:53:30.491 ... 01:03:34.264 00:53:30.491 710 01:03:34.264 00:53:30.491 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 01:03:34.264 00:53:30.491 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 01:03:34.264 00:53:30.491 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:03:34.264 00:53:30.491 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.264 00:53:30.491 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.264 00:53:30.491 716 01:03:34.264 00:53:30.491 717 echo "$time" 01:03:34.264 00:53:30.491 718 ) 01:03:34.264 00:53:30.491 719 01:03:34.264 00:53:30.491 720 function timing_enter() { 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 11 # Pci bus hotplug 01:03:34.265 00:53:30.491 12 # Helper function to remove/attach cotrollers 01:03:34.265 00:53:30.491 13 debug_remove_attach_helper() { 01:03:34.265 00:53:30.491 14 local helper_time=0 01:03:34.265 00:53:30.491 15 01:03:34.265 00:53:30.491 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.265 00:53:30.491 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.265 00:53:30.491 18 "$helper_time" "$nvme_count" >&2 01:03:34.265 00:53:30.491 19 } 01:03:34.265 00:53:30.491 20 01:03:34.265 00:53:30.491 21 remove_attach_helper() { 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.265 00:53:30.491 106 done 01:03:34.265 00:53:30.491 107 01:03:34.265 00:53:30.491 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.265 00:53:30.491 109 01:03:34.265 00:53:30.491 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.265 00:53:30.491 111 # Verify reregistering hotplug poller 01:03:34.265 00:53:30.491 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.265 00:53:30.491 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.265 00:53:30.491 114 01:03:34.265 00:53:30.491 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 133 01:03:34.265 00:53:30.491 134 # Run pci bus hotplug test 01:03:34.265 00:53:30.491 135 run_hotplug 01:03:34.265 00:53:30.491 136 01:03:34.265 00:53:30.491 137 # Run SPDK target based hotplug 01:03:34.265 00:53:30.491 => 138 tgt_run_hotplug 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 01:03:34.265 00:53:30.491 ========== Backtrace end ========== 01:03:34.265 00:53:30.491 ========== Backtrace start: ========== 01:03:34.265 00:53:30.491 01:03:34.265 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 713 # but $() will return just the time'\''s data, hence making it possible to just do: 01:03:34.265 00:53:30.491 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 01:03:34.265 00:53:30.491 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 01:03:34.265 00:53:30.491 716 01:03:34.265 00:53:30.491 717 echo "$time" 01:03:34.265 00:53:30.491 => 718 ) 01:03:34.265 00:53:30.491 719 01:03:34.265 00:53:30.491 720 function timing_enter() { 01:03:34.265 00:53:30.491 721 xtrace_disable 01:03:34.265 00:53:30.491 722 timing "enter" "$1" 01:03:34.265 00:53:30.491 723 xtrace_restore 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 11 # Pci bus hotplug 01:03:34.265 00:53:30.491 12 # Helper function to remove/attach cotrollers 01:03:34.265 00:53:30.491 13 debug_remove_attach_helper() { 01:03:34.265 00:53:30.491 14 local helper_time=0 01:03:34.265 00:53:30.491 15 01:03:34.265 00:53:30.491 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.265 00:53:30.491 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.265 00:53:30.491 18 "$helper_time" "$nvme_count" >&2 01:03:34.265 00:53:30.491 19 } 01:03:34.265 00:53:30.491 20 01:03:34.265 00:53:30.491 21 remove_attach_helper() { 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.265 00:53:30.491 106 done 01:03:34.265 00:53:30.491 107 01:03:34.265 00:53:30.491 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.265 00:53:30.491 109 01:03:34.265 00:53:30.491 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.265 00:53:30.491 111 # Verify reregistering hotplug poller 01:03:34.265 00:53:30.491 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.265 00:53:30.491 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.265 00:53:30.491 114 01:03:34.265 00:53:30.491 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 133 01:03:34.265 00:53:30.491 134 # Run pci bus hotplug test 01:03:34.265 00:53:30.491 135 run_hotplug 01:03:34.265 00:53:30.491 136 01:03:34.265 00:53:30.491 137 # Run SPDK target based hotplug 01:03:34.265 00:53:30.491 => 138 tgt_run_hotplug 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 01:03:34.265 00:53:30.491 ========== Backtrace end ========== 01:03:34.265 00:53:30.491 ========== Backtrace start: ========== 01:03:34.265 00:53:30.491 01:03:34.265 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 11 # Pci bus hotplug 01:03:34.265 00:53:30.491 12 # Helper function to remove/attach cotrollers 01:03:34.265 00:53:30.491 13 debug_remove_attach_helper() { 01:03:34.265 00:53:30.491 14 local helper_time=0 01:03:34.265 00:53:30.491 15 01:03:34.265 00:53:30.491 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 01:03:34.265 00:53:30.491 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 01:03:34.265 00:53:30.491 18 "$helper_time" "$nvme_count" >&2 01:03:34.265 00:53:30.491 19 } 01:03:34.265 00:53:30.491 20 01:03:34.265 00:53:30.491 21 remove_attach_helper() { 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 01:03:34.265 00:53:30.491 106 done 01:03:34.265 00:53:30.491 107 01:03:34.265 00:53:30.491 108 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.265 00:53:30.491 109 01:03:34.265 00:53:30.491 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.265 00:53:30.491 111 # Verify reregistering hotplug poller 01:03:34.265 00:53:30.491 112 rpc_cmd bdev_nvme_set_hotplug -d 01:03:34.265 00:53:30.491 113 rpc_cmd bdev_nvme_set_hotplug -e 01:03:34.265 00:53:30.491 114 01:03:34.265 00:53:30.491 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 133 01:03:34.265 00:53:30.491 134 # Run pci bus hotplug test 01:03:34.265 00:53:30.491 135 run_hotplug 01:03:34.265 00:53:30.491 136 01:03:34.265 00:53:30.491 137 # Run SPDK target based hotplug 01:03:34.265 00:53:30.491 => 138 tgt_run_hotplug 01:03:34.265 00:53:30.491 ... 01:03:34.265 00:53:30.491 01:03:34.265 00:53:30.491 ========== Backtrace end ========== 01:03:34.265 00:53:32.647 ========== Backtrace start: ========== 01:03:34.265 00:53:32.647 01:03:34.265 00:53:32.647 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 01:03:34.265 00:53:32.647 ... 01:03:34.265 00:53:32.647 1117 timing_enter $test_name 01:03:34.265 00:53:32.647 1118 echo "************************************" 01:03:34.265 00:53:32.647 1119 echo "START TEST $test_name" 01:03:34.265 00:53:32.647 1120 echo "************************************" 01:03:34.265 00:53:32.647 1121 xtrace_restore 01:03:34.265 00:53:32.647 1122 time "$@" 01:03:34.265 00:53:32.647 1123 xtrace_disable 01:03:34.265 00:53:32.647 1124 echo "************************************" 01:03:34.265 00:53:32.647 1125 echo "END TEST $test_name" 01:03:34.265 00:53:32.647 1126 echo "************************************" 01:03:34.265 00:53:32.647 1127 timing_exit $test_name 01:03:34.265 00:53:32.647 ... 01:03:34.265 00:53:32.647 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 01:03:34.266 00:53:32.647 ... 01:03:34.266 00:53:32.647 235 01:03:34.266 00:53:32.647 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 01:03:34.266 00:53:32.647 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 01:03:34.266 00:53:32.647 238 01:03:34.266 00:53:32.647 239 if [ $(uname -s) = Linux ]; then 01:03:34.266 00:53:32.647 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 01:03:34.266 00:53:32.647 241 fi 01:03:34.266 00:53:32.647 242 01:03:34.266 00:53:32.647 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 01:03:34.266 00:53:32.647 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 01:03:34.266 00:53:32.647 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 01:03:34.266 00:53:32.647 ... 01:03:34.266 00:53:32.647 01:03:34.266 00:53:32.647 ========== Backtrace end ==========' 01:03:34.266 + for build in "${build_log_list[@]}" 01:03:34.266 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_16102/build.log 01:03:34.266 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.266 + build_log=/var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_16102/build.log 01:03:34.266 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_16102/build.log 01:03:34.266 ++ basename /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_16102 01:03:34.266 + job_name=rocky9-vg-autotest_16102 01:03:34.266 + start_pattern='Backtrace start' 01:03:34.266 + end_pattern='Backtrace end' 01:03:34.266 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_16102/build.log ]] 01:03:34.266 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_16102/build.log 01:03:34.266 + backtraces= 01:03:34.266 + [[ -n '' ]] 01:03:34.266 + for build in "${build_log_list[@]}" 01:03:34.266 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_72343/build.log 01:03:34.266 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.266 + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_72343/build.log 01:03:34.266 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_72343/build.log 01:03:34.266 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_72343 01:03:34.266 + job_name=nvmf-tcp-phy-autotest_72343 01:03:34.266 + start_pattern='Backtrace start' 01:03:34.266 + end_pattern='Backtrace end' 01:03:34.266 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_72343/build.log ]] 01:03:34.266 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_72343/build.log 01:03:34.525 + backtraces= 01:03:34.525 + [[ -n '' ]] 01:03:34.525 + for build in "${build_log_list[@]}" 01:03:34.525 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_19873/build.log 01:03:34.525 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.525 + build_log=/var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_19873/build.log 01:03:34.525 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_19873/build.log 01:03:34.525 ++ basename /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_19873 01:03:34.525 + job_name=unittest-clang-docker-autotest_19873 01:03:34.525 + start_pattern='Backtrace start' 01:03:34.525 + end_pattern='Backtrace end' 01:03:34.525 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_19873/build.log ]] 01:03:34.525 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_19873/build.log 01:03:34.525 + backtraces= 01:03:34.525 + [[ -n '' ]] 01:03:34.525 + for build in "${build_log_list[@]}" 01:03:34.525 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_77530/build.log 01:03:34.525 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.525 + build_log=/var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_77530/build.log 01:03:34.525 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_77530/build.log 01:03:34.525 ++ basename /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_77530 01:03:34.525 + job_name=iscsi-vg-autotest_77530 01:03:34.525 + start_pattern='Backtrace start' 01:03:34.525 + end_pattern='Backtrace end' 01:03:34.525 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_77530/build.log ]] 01:03:34.525 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_77530/build.log 01:03:34.525 + backtraces= 01:03:34.525 + [[ -n '' ]] 01:03:34.525 + for build in "${build_log_list[@]}" 01:03:34.525 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_19836/build.log 01:03:34.525 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.525 + build_log=/var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_19836/build.log 01:03:34.525 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_19836/build.log 01:03:34.525 ++ basename /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_19836 01:03:34.525 + job_name=autobuild-external-vg-autotest_19836 01:03:34.525 + start_pattern='Backtrace start' 01:03:34.525 + end_pattern='Backtrace end' 01:03:34.525 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_19836/build.log ]] 01:03:34.525 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_19836/build.log 01:03:34.525 + backtraces= 01:03:34.525 + [[ -n '' ]] 01:03:34.525 + for build in "${build_log_list[@]}" 01:03:34.525 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_78615/build.log 01:03:34.525 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.525 + build_log=/var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_78615/build.log 01:03:34.525 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_78615/build.log 01:03:34.525 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_78615 01:03:34.525 + job_name=nvme-cmb-pmr-vg-autotest_78615 01:03:34.525 + start_pattern='Backtrace start' 01:03:34.525 + end_pattern='Backtrace end' 01:03:34.525 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_78615/build.log ]] 01:03:34.525 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_78615/build.log 01:03:34.525 + backtraces= 01:03:34.525 + [[ -n '' ]] 01:03:34.525 + for build in "${build_log_list[@]}" 01:03:34.525 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_22169/build.log 01:03:34.525 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.525 + build_log=/var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_22169/build.log 01:03:34.525 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_22169/build.log 01:03:34.525 ++ basename /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_22169 01:03:34.525 + job_name=check-so-deps-docker-autotest_22169 01:03:34.525 + start_pattern='Backtrace start' 01:03:34.525 + end_pattern='Backtrace end' 01:03:34.525 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_22169/build.log ]] 01:03:34.525 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_22169/build.log 01:03:34.525 + backtraces= 01:03:34.525 + [[ -n '' ]] 01:03:34.525 + for build in "${build_log_list[@]}" 01:03:34.525 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_77512/build.log 01:03:34.525 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.525 + build_log=/var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_77512/build.log 01:03:34.525 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_77512/build.log 01:03:34.525 ++ basename /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_77512 01:03:34.525 + job_name=vhost-initiator-vg-autotest_77512 01:03:34.525 + start_pattern='Backtrace start' 01:03:34.525 + end_pattern='Backtrace end' 01:03:34.525 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_77512/build.log ]] 01:03:34.525 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_77512/build.log 01:03:34.525 + backtraces= 01:03:34.525 + [[ -n '' ]] 01:03:34.525 + for build in "${build_log_list[@]}" 01:03:34.525 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_26212/build.log 01:03:34.526 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.526 + build_log=/var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_26212/build.log 01:03:34.526 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_26212/build.log 01:03:34.526 ++ basename /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_26212 01:03:34.526 + job_name=crypto-phy-autotest_26212 01:03:34.526 + start_pattern='Backtrace start' 01:03:34.526 + end_pattern='Backtrace end' 01:03:34.526 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_26212/build.log ]] 01:03:34.526 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_26212/build.log 01:03:34.526 + backtraces= 01:03:34.526 + [[ -n '' ]] 01:03:34.526 + for build in "${build_log_list[@]}" 01:03:34.526 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_22336/build.log 01:03:34.526 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.526 + build_log=/var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_22336/build.log 01:03:34.526 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_22336/build.log 01:03:34.526 ++ basename /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_22336 01:03:34.526 + job_name=doc-docker-autotest_22336 01:03:34.526 + start_pattern='Backtrace start' 01:03:34.526 + end_pattern='Backtrace end' 01:03:34.526 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_22336/build.log ]] 01:03:34.526 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_22336/build.log 01:03:34.526 + backtraces= 01:03:34.526 + [[ -n '' ]] 01:03:34.526 + for build in "${build_log_list[@]}" 01:03:34.526 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_26237/build.log 01:03:34.526 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.526 + build_log=/var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_26237/build.log 01:03:34.526 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_26237/build.log 01:03:34.526 ++ basename /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_26237 01:03:34.526 + job_name=vhost-phy-autotest_26237 01:03:34.526 + start_pattern='Backtrace start' 01:03:34.526 + end_pattern='Backtrace end' 01:03:34.526 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_26237/build.log ]] 01:03:34.526 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_26237/build.log 01:03:34.526 + backtraces= 01:03:34.526 + [[ -n '' ]] 01:03:34.526 + for build in "${build_log_list[@]}" 01:03:34.526 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_22145/build.log 01:03:34.526 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.526 + build_log=/var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_22145/build.log 01:03:34.526 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_22145/build.log 01:03:34.526 ++ basename /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_22145 01:03:34.526 + job_name=build-files-docker-autotest_22145 01:03:34.526 + start_pattern='Backtrace start' 01:03:34.526 + end_pattern='Backtrace end' 01:03:34.526 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_22145/build.log ]] 01:03:34.526 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_22145/build.log 01:03:34.526 + backtraces= 01:03:34.526 + [[ -n '' ]] 01:03:34.526 + for build in "${build_log_list[@]}" 01:03:34.526 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_19874/build.log 01:03:34.526 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.526 + build_log=/var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_19874/build.log 01:03:34.526 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_19874/build.log 01:03:34.526 ++ basename /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_19874 01:03:34.526 + job_name=unittest-main-docker-autotest_19874 01:03:34.526 + start_pattern='Backtrace start' 01:03:34.526 + end_pattern='Backtrace end' 01:03:34.526 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_19874/build.log ]] 01:03:34.526 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_19874/build.log 01:03:34.526 + backtraces= 01:03:34.526 + [[ -n '' ]] 01:03:34.526 + for build in "${build_log_list[@]}" 01:03:34.526 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_59136/build.log 01:03:34.526 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.526 + build_log=/var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_59136/build.log 01:03:34.526 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_59136/build.log 01:03:34.526 ++ basename /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_59136 01:03:34.526 + job_name=freebsd-vg-autotest_59136 01:03:34.526 + start_pattern='Backtrace start' 01:03:34.526 + end_pattern='Backtrace end' 01:03:34.526 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_59136/build.log ]] 01:03:34.526 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_59136/build.log 01:03:34.526 + backtraces= 01:03:34.526 + [[ -n '' ]] 01:03:34.526 + for build in "${build_log_list[@]}" 01:03:34.526 + create_errtrace_file /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_19875/build.log 01:03:34.526 + local build_log job_name start_pattern end_pattern backtraces 01:03:34.526 + build_log=/var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_19875/build.log 01:03:34.526 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_19875/build.log 01:03:34.526 ++ basename /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_19875 01:03:34.526 + job_name=packaging-docker-autotest_19875 01:03:34.526 + start_pattern='Backtrace start' 01:03:34.526 + end_pattern='Backtrace end' 01:03:34.526 + [[ -f /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_19875/build.log ]] 01:03:34.526 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_19875/build.log 01:03:34.526 + backtraces= 01:03:34.526 + [[ -n '' ]] 01:03:34.526 + run_flamegraph 01:03:34.526 + create_timing_file 01:03:34.526 + local array file 01:03:34.526 + array=($(find $WORKSPACE -type f -name 'timing.txt')) 01:03:34.526 ++ find /var/jenkins/workspace/autotest-per-patch_2 -type f -name timing.txt 01:03:34.526 + timing_dir=/var/jenkins/workspace/autotest-per-patch_2/post_process 01:03:34.526 + mkdir -p /var/jenkins/workspace/autotest-per-patch_2/post_process 01:03:34.526 + timing_file=/var/jenkins/workspace/autotest-per-patch_2/post_process/total_timing.txt 01:03:34.526 + for file in "${array[@]}" 01:03:34.526 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_77492/timing.txt 01:03:34.526 + local job_file job_name 01:03:34.526 + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_77492/timing.txt 01:03:34.526 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_77492/timing.txt 01:03:34.526 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_77492 01:03:34.526 + job_name=nvme-vg-autotest_77492 01:03:34.526 + sed 's/^/nvme-vg-autotest_77492;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvme-vg-autotest_77492/timing.txt 01:03:34.526 + for file in "${array[@]}" 01:03:34.526 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_26220/timing.txt 01:03:34.526 + local job_file job_name 01:03:34.526 + job_file=/var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_26220/timing.txt 01:03:34.526 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_26220/timing.txt 01:03:34.526 ++ basename /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_26220 01:03:34.526 + job_name=vfio-user-phy-autotest_26220 01:03:34.526 + sed 's/^/vfio-user-phy-autotest_26220;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_26220/timing.txt 01:03:34.526 + for file in "${array[@]}" 01:03:34.526 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_22156/timing.txt 01:03:34.526 + local job_file job_name 01:03:34.526 + job_file=/var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_22156/timing.txt 01:03:34.526 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_22156/timing.txt 01:03:34.526 ++ basename /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_22156 01:03:34.526 + job_name=check-format-docker-autotest_22156 01:03:34.526 + sed 's/^/check-format-docker-autotest_22156;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/check-format-docker-autotest_22156/timing.txt 01:03:34.526 + for file in "${array[@]}" 01:03:34.526 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_77483/timing.txt 01:03:34.526 + local job_file job_name 01:03:34.526 + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_77483/timing.txt 01:03:34.526 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_77483/timing.txt 01:03:34.526 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_77483 01:03:34.526 + job_name=nvmf-tcp-vg-autotest_77483 01:03:34.526 + sed 's/^/nvmf-tcp-vg-autotest_77483;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_77483/timing.txt 01:03:34.526 + for file in "${array[@]}" 01:03:34.526 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_26209/timing.txt 01:03:34.526 + local job_file job_name 01:03:34.526 + job_file=/var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_26209/timing.txt 01:03:34.526 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_26209/timing.txt 01:03:34.526 ++ basename /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_26209 01:03:34.526 + job_name=short-fuzz-phy-autotest_26209 01:03:34.527 + sed 's/^/short-fuzz-phy-autotest_26209;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/short-fuzz-phy-autotest_26209/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_77397/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_77397/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_77397/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_77397 01:03:34.527 + job_name=iscsi-uring-vg-autotest_77397 01:03:34.527 + sed 's/^/iscsi-uring-vg-autotest_77397;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_77397/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_19876/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_19876/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_19876/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_19876 01:03:34.527 + job_name=scan-build-docker-autotest_19876 01:03:34.527 + sed 's/^/scan-build-docker-autotest_19876;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/scan-build-docker-autotest_19876/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_72384/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_72384/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_72384/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_72384 01:03:34.527 + job_name=nvme-phy-autotest_72384 01:03:34.527 + sed 's/^/nvme-phy-autotest_72384;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_72384/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_77441/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_77441/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_77441/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_77441 01:03:34.527 + job_name=lvol-vg-autotest_77441 01:03:34.527 + sed 's/^/lvol-vg-autotest_77441;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_77441/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/dsa-phy-autotest_4912/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/dsa-phy-autotest_4912/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/dsa-phy-autotest_4912/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/dsa-phy-autotest_4912 01:03:34.527 + job_name=dsa-phy-autotest_4912 01:03:34.527 + sed 's/^/dsa-phy-autotest_4912;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/dsa-phy-autotest_4912/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_72456/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_72456/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_72456/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_72456 01:03:34.527 + job_name=nvmf-phy-autotest_72456 01:03:34.527 + sed 's/^/nvmf-phy-autotest_72456;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_72456/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_73671/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_73671/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_73671/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_73671 01:03:34.527 + job_name=centos7-vg-autotest_73671 01:03:34.527 + sed 's/^/centos7-vg-autotest_73671;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_73671/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_37825/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_37825/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_37825/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_37825 01:03:34.527 + job_name=ubuntu22-vg-autotest_37825 01:03:34.527 + sed 's/^/ubuntu22-vg-autotest_37825;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_37825/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_26218/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_26218/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_26218/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_26218 01:03:34.527 + job_name=BlobFS-phy-autotest_26218 01:03:34.527 + sed 's/^/BlobFS-phy-autotest_26218;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_26218/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_19877/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_19877/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_19877/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_19877 01:03:34.527 + job_name=unittest-valgrind-docker-autotest_19877 01:03:34.527 + sed 's/^/unittest-valgrind-docker-autotest_19877;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/unittest-valgrind-docker-autotest_19877/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_76598/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_76598/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_76598/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_76598 01:03:34.527 + job_name=zns-vg-autotest_76598 01:03:34.527 + sed 's/^/zns-vg-autotest_76598;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_76598/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_77417/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_77417/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_77417/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_77417 01:03:34.527 + job_name=nvmf-tcp-uring-vg-autotest_77417 01:03:34.527 + sed 's/^/nvmf-tcp-uring-vg-autotest_77417;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_77417/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_73811/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_73811/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_73811/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_73811 01:03:34.527 + job_name=ubuntu20-vg-autotest_73811 01:03:34.527 + sed 's/^/ubuntu20-vg-autotest_73811;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/ubuntu20-vg-autotest_73811/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_16102/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_16102/timing.txt 01:03:34.527 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_16102/timing.txt 01:03:34.527 ++ basename /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_16102 01:03:34.527 + job_name=rocky9-vg-autotest_16102 01:03:34.527 + sed 's/^/rocky9-vg-autotest_16102;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_16102/timing.txt 01:03:34.527 + for file in "${array[@]}" 01:03:34.527 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_72343/timing.txt 01:03:34.527 + local job_file job_name 01:03:34.527 + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_72343/timing.txt 01:03:34.528 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_72343/timing.txt 01:03:34.528 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_72343 01:03:34.528 + job_name=nvmf-tcp-phy-autotest_72343 01:03:34.528 + sed 's/^/nvmf-tcp-phy-autotest_72343;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_72343/timing.txt 01:03:34.528 + for file in "${array[@]}" 01:03:34.528 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_19873/timing.txt 01:03:34.528 + local job_file job_name 01:03:34.528 + job_file=/var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_19873/timing.txt 01:03:34.528 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_19873/timing.txt 01:03:34.528 ++ basename /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_19873 01:03:34.528 + job_name=unittest-clang-docker-autotest_19873 01:03:34.528 + sed 's/^/unittest-clang-docker-autotest_19873;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/unittest-clang-docker-autotest_19873/timing.txt 01:03:34.528 + for file in "${array[@]}" 01:03:34.528 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_77530/timing.txt 01:03:34.528 + local job_file job_name 01:03:34.528 + job_file=/var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_77530/timing.txt 01:03:34.528 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_77530/timing.txt 01:03:34.528 ++ basename /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_77530 01:03:34.528 + job_name=iscsi-vg-autotest_77530 01:03:34.528 + sed 's/^/iscsi-vg-autotest_77530;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_77530/timing.txt 01:03:34.528 + for file in "${array[@]}" 01:03:34.528 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_19836/timing.txt 01:03:34.528 + local job_file job_name 01:03:34.528 + job_file=/var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_19836/timing.txt 01:03:34.528 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_19836/timing.txt 01:03:34.528 ++ basename /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_19836 01:03:34.528 + job_name=autobuild-external-vg-autotest_19836 01:03:34.528 + sed 's/^/autobuild-external-vg-autotest_19836;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/autobuild-external-vg-autotest_19836/timing.txt 01:03:34.528 + for file in "${array[@]}" 01:03:34.528 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_78615/timing.txt 01:03:34.528 + local job_file job_name 01:03:34.528 + job_file=/var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_78615/timing.txt 01:03:34.528 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_78615/timing.txt 01:03:34.528 ++ basename /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_78615 01:03:34.528 + job_name=nvme-cmb-pmr-vg-autotest_78615 01:03:34.528 + sed 's/^/nvme-cmb-pmr-vg-autotest_78615;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_78615/timing.txt 01:03:34.528 + for file in "${array[@]}" 01:03:34.528 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_22169/timing.txt 01:03:34.528 + local job_file job_name 01:03:34.528 + job_file=/var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_22169/timing.txt 01:03:34.528 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_22169/timing.txt 01:03:34.528 ++ basename /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_22169 01:03:34.528 + job_name=check-so-deps-docker-autotest_22169 01:03:34.528 + sed 's/^/check-so-deps-docker-autotest_22169;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/check-so-deps-docker-autotest_22169/timing.txt 01:03:34.528 + for file in "${array[@]}" 01:03:34.528 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_77512/timing.txt 01:03:34.528 + local job_file job_name 01:03:34.528 + job_file=/var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_77512/timing.txt 01:03:34.528 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_77512/timing.txt 01:03:34.528 ++ basename /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_77512 01:03:34.528 + job_name=vhost-initiator-vg-autotest_77512 01:03:34.528 + sed 's/^/vhost-initiator-vg-autotest_77512;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_77512/timing.txt 01:03:34.528 + for file in "${array[@]}" 01:03:34.528 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_26212/timing.txt 01:03:34.528 + local job_file job_name 01:03:34.528 + job_file=/var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_26212/timing.txt 01:03:34.528 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_26212/timing.txt 01:03:34.528 ++ basename /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_26212 01:03:34.528 + job_name=crypto-phy-autotest_26212 01:03:34.528 + sed 's/^/crypto-phy-autotest_26212;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_26212/timing.txt 01:03:34.528 + for file in "${array[@]}" 01:03:34.528 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_22336/timing.txt 01:03:34.528 + local job_file job_name 01:03:34.528 + job_file=/var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_22336/timing.txt 01:03:34.528 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_22336/timing.txt 01:03:34.528 ++ basename /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_22336 01:03:34.528 + job_name=doc-docker-autotest_22336 01:03:34.528 + sed 's/^/doc-docker-autotest_22336;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/doc-docker-autotest_22336/timing.txt 01:03:34.528 + for file in "${array[@]}" 01:03:34.528 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_26237/timing.txt 01:03:34.528 + local job_file job_name 01:03:34.528 + job_file=/var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_26237/timing.txt 01:03:34.528 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_26237/timing.txt 01:03:34.528 ++ basename /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_26237 01:03:34.528 + job_name=vhost-phy-autotest_26237 01:03:34.528 + sed 's/^/vhost-phy-autotest_26237;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_26237/timing.txt 01:03:34.528 + for file in "${array[@]}" 01:03:34.528 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_22145/timing.txt 01:03:34.528 + local job_file job_name 01:03:34.528 + job_file=/var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_22145/timing.txt 01:03:34.528 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_22145/timing.txt 01:03:34.528 ++ basename /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_22145 01:03:34.528 + job_name=build-files-docker-autotest_22145 01:03:34.528 + sed 's/^/build-files-docker-autotest_22145;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/build-files-docker-autotest_22145/timing.txt 01:03:34.528 + for file in "${array[@]}" 01:03:34.528 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_19874/timing.txt 01:03:34.528 + local job_file job_name 01:03:34.528 + job_file=/var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_19874/timing.txt 01:03:34.528 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_19874/timing.txt 01:03:34.528 ++ basename /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_19874 01:03:34.528 + job_name=unittest-main-docker-autotest_19874 01:03:34.528 + sed 's/^/unittest-main-docker-autotest_19874;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/unittest-main-docker-autotest_19874/timing.txt 01:03:34.786 + for file in "${array[@]}" 01:03:34.786 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_59136/timing.txt 01:03:34.786 + local job_file job_name 01:03:34.786 + job_file=/var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_59136/timing.txt 01:03:34.786 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_59136/timing.txt 01:03:34.786 ++ basename /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_59136 01:03:34.786 + job_name=freebsd-vg-autotest_59136 01:03:34.786 + sed 's/^/freebsd-vg-autotest_59136;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/freebsd-vg-autotest_59136/timing.txt 01:03:34.786 + for file in "${array[@]}" 01:03:34.786 + add_sub_job_timing /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_19875/timing.txt 01:03:34.786 + local job_file job_name 01:03:34.786 + job_file=/var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_19875/timing.txt 01:03:34.786 +++ dirname /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_19875/timing.txt 01:03:34.786 ++ basename /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_19875 01:03:34.786 + job_name=packaging-docker-autotest_19875 01:03:34.786 + sed 's/^/packaging-docker-autotest_19875;autorun;/' /var/jenkins/workspace/autotest-per-patch_2/packaging-docker-autotest_19875/timing.txt 01:03:34.786 + /usr/local/FlameGraph/flamegraph.pl --title 'Build Timing' --nametype Step: --countname seconds /var/jenkins/workspace/autotest-per-patch_2/post_process/total_timing.txt 01:03:34.786 + run_autorun_post 01:03:34.786 + s_opt= 01:03:34.786 + [[ '' == true ]] 01:03:34.786 + spdk/autorun_post.py -d ./ -r ./spdk 01:03:38.962 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/module/bdev/nvme/bdev_mdns_client.c:613 01:03:38.962 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/module/bdev/nvme/bdev_mdns_client.c:561 01:03:38.962 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/module/bdev/nvme/bdev_mdns_client.c:582 01:03:38.962 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/module/bdev/nvme/bdev_mdns_client.c:456 01:03:53.814 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/util/xor.c:120 01:03:53.814 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/util/crc32c.c:21 01:03:53.814 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/util/crc16.c:634 01:03:53.814 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/util/crc16.c:628 01:03:54.072 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/util/crc64.c:165 01:04:00.627 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-per-patch_2/spdk/module/blobfs/bdev/blobfs_bdev.c:254 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/bdev_module.h! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/base64.h! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/histogram_data.h! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/trace.h! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/mmio.h! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/endian.h! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/nvme.h! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/nvmf_transport.h! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/thread.h! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/util.h! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/nvme_spec.h! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/histogram_data.h/histogram_ut.c! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/env_ocf/include/ocf/ocf_io.h! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/env_ocf/include/ocf/ocf_mngt.h! 01:04:56.842 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-per-patch_2/spdk/lib/env_ocf/include/ocf/ocf_core.h! 01:05:06.797 -----Begin Post Process Script------ 01:05:06.797 /var/jenkins/workspace/autotest-per-patch_2/vfio-user-phy-autotest_26220/cov_total.info 01:05:06.797 /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-vg-autotest_77483/cov_total.info 01:05:06.797 /var/jenkins/workspace/autotest-per-patch_2/iscsi-uring-vg-autotest_77397/cov_total.info 01:05:06.797 /var/jenkins/workspace/autotest-per-patch_2/nvme-phy-autotest_72384/cov_total.info 01:05:06.797 /var/jenkins/workspace/autotest-per-patch_2/lvol-vg-autotest_77441/cov_total.info 01:05:06.797 /var/jenkins/workspace/autotest-per-patch_2/dsa-phy-autotest_4912/cov_total.info 01:05:06.798 /var/jenkins/workspace/autotest-per-patch_2/nvmf-phy-autotest_72456/cov_total.info 01:05:06.798 /var/jenkins/workspace/autotest-per-patch_2/centos7-vg-autotest_73671/cov_total.info 01:05:06.798 /var/jenkins/workspace/autotest-per-patch_2/ubuntu22-vg-autotest_37825/cov_total.info 01:05:06.798 /var/jenkins/workspace/autotest-per-patch_2/BlobFS-phy-autotest_26218/cov_total.info 01:05:06.798 /var/jenkins/workspace/autotest-per-patch_2/zns-vg-autotest_76598/cov_total.info 01:05:06.798 /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-uring-vg-autotest_77417/cov_total.info 01:05:06.798 /var/jenkins/workspace/autotest-per-patch_2/rocky9-vg-autotest_16102/cov_total.info 01:05:06.798 /var/jenkins/workspace/autotest-per-patch_2/nvmf-tcp-phy-autotest_72343/cov_total.info 01:05:06.798 /var/jenkins/workspace/autotest-per-patch_2/iscsi-vg-autotest_77530/cov_total.info 01:05:06.798 /var/jenkins/workspace/autotest-per-patch_2/nvme-cmb-pmr-vg-autotest_78615/cov_total.info 01:05:06.798 /var/jenkins/workspace/autotest-per-patch_2/vhost-initiator-vg-autotest_77512/cov_total.info 01:05:06.798 /var/jenkins/workspace/autotest-per-patch_2/crypto-phy-autotest_26212/cov_total.info 01:05:06.798 /var/jenkins/workspace/autotest-per-patch_2/vhost-phy-autotest_26237/cov_total.info 01:05:06.798 01:05:06.798 01:05:06.798 -----Tests Executed in Build------ 01:05:06.798 accel 01:05:06.798 accel_assign_opcode 01:05:06.798 accel_cdev_comp 01:05:06.798 accel_cdev_decmop_full 01:05:06.798 accel_cdev_decomp 01:05:06.798 accel_cdev_decomp_full_mcore 01:05:06.798 accel_cdev_decomp_full_mthread 01:05:06.798 accel_cdev_decomp_mcore 01:05:06.798 accel_cdev_decomp_mthread 01:05:06.798 accel_comp 01:05:06.798 accel_compare 01:05:06.798 accel_compress_verify 01:05:06.798 accel_copy 01:05:06.798 accel_copy_crc32c 01:05:06.798 accel_copy_crc32c_C2 01:05:06.798 accel_crc32c 01:05:06.798 accel_crc32c_C2 01:05:06.798 accel_decmop_full 01:05:06.798 accel_decomp 01:05:06.798 accel_decomp_full_mcore 01:05:06.798 accel_decomp_full_mthread 01:05:06.798 accel_decomp_mcore 01:05:06.798 accel_decomp_mthread 01:05:06.798 accel_dif_functional_tests 01:05:06.798 accel_dif_generate 01:05:06.798 accel_dif_generate_copy 01:05:06.798 accel_dif_verify 01:05:06.798 accel_dualcast 01:05:06.798 accel_fill 01:05:06.798 accel_help 01:05:06.798 accel_missing_filename 01:05:06.798 accel_negative_buffers 01:05:06.798 accel_rpc 01:05:06.798 accel_scan_dsa_modules 01:05:06.798 accel_scan_iaa_modules 01:05:06.798 accel_wrong_workload 01:05:06.798 accel_xor 01:05:06.798 acl 01:05:06.798 alias_rpc 01:05:06.798 allowed 01:05:06.798 app_cmdline 01:05:06.798 app_repeat 01:05:06.798 asan 01:05:06.798 autobuild 01:05:06.798 autobuild_build_doc 01:05:06.798 autobuild_check_dpdk_pci_api 01:05:06.798 autobuild_check_format 01:05:06.798 autobuild_check_so_deps 01:05:06.798 autobuild_external_code 01:05:06.798 autobuild_generated_files_check 01:05:06.798 autobuild_generated_files_check_post_clean 01:05:06.798 autobuild_header_dependency_check 01:05:06.798 autobuild_llvm_precompile 01:05:06.798 autobuild_make_install 01:05:06.798 autobuild_make_uninstall 01:05:06.798 autobuild_ocf_precompile 01:05:06.798 bdev_bounds 01:05:06.798 bdev_crypto_enomem 01:05:06.798 bdev_error 01:05:06.798 bdev_fio 01:05:06.798 bdev_fio_rw_verify 01:05:06.798 bdev_fio_trim 01:05:06.798 bdev_gpt_uuid 01:05:06.798 bdev_hello_world 01:05:06.798 bdev_json_nonarray 01:05:06.798 bdev_json_nonenclosed 01:05:06.798 bdev_nbd 01:05:06.798 bdev_nvme_reset_stuck_adm_cmd 01:05:06.798 bdev_qd_sampling 01:05:06.798 bdev_qos 01:05:06.798 bdev_qos_bw 01:05:06.798 bdev_qos_iops 01:05:06.798 bdev_qos_ro_bw 01:05:06.798 bdev_raid 01:05:06.798 bdev_stat 01:05:06.798 bdev_verify 01:05:06.798 bdev_verify_big_io 01:05:06.798 bdev_write_zeroes 01:05:06.798 bdevperf_config 01:05:06.798 blob_io_wait 01:05:06.798 blobfs 01:05:06.798 blobfs_mkfs 01:05:06.798 blobstore 01:05:06.798 blobstore_grow 01:05:06.798 blockdev_crypto_aesni 01:05:06.798 blockdev_crypto_qat 01:05:06.798 blockdev_crypto_sw 01:05:06.798 blockdev_general 01:05:06.798 blockdev_nvme 01:05:06.798 blockdev_nvme_gpt 01:05:06.798 blockdev_raid5f 01:05:06.798 blockdev_rbd 01:05:06.798 build_rpm_from_gen_spec 01:05:06.798 build_shared_rpm 01:05:06.798 chaining 01:05:06.798 check_header_filenames 01:05:06.798 cmb_copy 01:05:06.798 compress_compdev 01:05:06.798 compress_isal 01:05:06.798 confirm_abi_deps 01:05:06.798 confirm_makefile_deps 01:05:06.798 cpu_locks 01:05:06.798 custom_alloc 01:05:06.798 dd_bs_lt_native_bs 01:05:06.798 dd_copy_to_out_bdev 01:05:06.798 dd_double_input 01:05:06.798 dd_double_output 01:05:06.798 dd_flag_append 01:05:06.798 dd_flag_append_forced_aio 01:05:06.798 dd_flag_directory 01:05:06.798 dd_flag_directory_forced_aio 01:05:06.798 dd_flag_noatime 01:05:06.798 dd_flag_noatime_forced_aio 01:05:06.798 dd_flag_nofollow 01:05:06.798 dd_flag_nofollow_forced_aio 01:05:06.798 dd_flags_misc 01:05:06.798 dd_flags_misc_forced_aio 01:05:06.798 dd_inflate_file 01:05:06.798 dd_invalid_arguments 01:05:06.798 dd_invalid_count 01:05:06.798 dd_invalid_iflag 01:05:06.798 dd_invalid_json 01:05:06.798 dd_invalid_oflag 01:05:06.798 dd_malloc_copy 01:05:06.798 dd_no_input 01:05:06.798 dd_no_output 01:05:06.798 dd_offset_magic 01:05:06.798 dd_rw 01:05:06.798 dd_rw_offset 01:05:06.798 dd_smaller_blocksize 01:05:06.798 dd_sparse_bdev_to_file 01:05:06.798 dd_sparse_file_to_bdev 01:05:06.798 dd_sparse_file_to_file 01:05:06.798 dd_unknown_flag 01:05:06.798 dd_uring_copy 01:05:06.798 dd_wrong_blocksize 01:05:06.798 default_locks 01:05:06.798 default_locks_via_rpc 01:05:06.798 default_setup 01:05:06.798 denied 01:05:06.798 devices 01:05:06.798 dm_mount 01:05:06.798 dma 01:05:06.798 dpdk_governor 01:05:06.798 dpdk_mem_utility 01:05:06.798 driver 01:05:06.798 env 01:05:06.798 env_dpdk_post_init 01:05:06.798 env_mem_callbacks 01:05:06.798 env_memory 01:05:06.798 env_pci 01:05:06.798 env_vtophys 01:05:06.798 even_2G_alloc 01:05:06.798 event 01:05:06.798 event_perf 01:05:06.798 event_reactor 01:05:06.798 event_reactor_perf 01:05:06.798 event_scheduler 01:05:06.798 exit_on_failed_rpc_init 01:05:06.798 external_make_accel_driver_shared 01:05:06.798 external_make_accel_driver_static 01:05:06.798 external_make_accel_module_shared 01:05:06.798 external_make_accel_module_static 01:05:06.798 external_make_hello_bdev_shared_combo 01:05:06.798 external_make_hello_bdev_shared_iso 01:05:06.798 external_make_hello_bdev_static 01:05:06.798 external_make_hello_no_bdev_shared_combo 01:05:06.798 external_make_hello_no_bdev_shared_iso 01:05:06.798 external_make_hello_no_bdev_static 01:05:06.798 external_make_nvme_shared 01:05:06.798 external_make_nvme_static 01:05:06.798 external_run_accel_driver_shared 01:05:06.798 external_run_accel_driver_static 01:05:06.798 external_run_accel_module_shared 01:05:06.798 external_run_accel_module_static 01:05:06.798 external_run_hello_bdev_shared_combo 01:05:06.798 external_run_hello_bdev_shared_iso 01:05:06.798 external_run_hello_bdev_static 01:05:06.798 external_run_hello_no_bdev_shared_combo 01:05:06.798 external_run_hello_no_bdev_shared_iso 01:05:06.798 external_run_hello_no_bdev_static 01:05:06.798 external_run_nvme_shared 01:05:06.798 external_run_nvme_static 01:05:06.798 filesystem_btrfs 01:05:06.798 filesystem_ext4 01:05:06.798 filesystem_in_capsule_btrfs 01:05:06.798 filesystem_in_capsule_ext4 01:05:06.798 filesystem_in_capsule_xfs 01:05:06.798 filesystem_xfs 01:05:06.798 fio_dif_1_default 01:05:06.798 fio_dif_1_multi_subsystems 01:05:06.798 fio_dif_digest 01:05:06.798 fio_dif_rand_params 01:05:06.798 go_rpc 01:05:06.798 guess_driver 01:05:06.798 hello_blob 01:05:06.798 hugepages 01:05:06.798 idle 01:05:06.798 interrupt_mode 01:05:06.798 ioat 01:05:06.798 ioat_perf 01:05:06.798 ioat_verify 01:05:06.798 is_zoned 01:05:06.798 iscsi_tgt 01:05:06.798 iscsi_tgt_bdev_io_wait 01:05:06.798 iscsi_tgt_calsoft 01:05:06.798 iscsi_tgt_digest 01:05:06.798 iscsi_tgt_digests 01:05:06.798 iscsi_tgt_filesystem 01:05:06.798 iscsi_tgt_filesystem_btrfs 01:05:06.798 iscsi_tgt_filesystem_ext4 01:05:06.798 iscsi_tgt_filesystem_xfs 01:05:06.798 iscsi_tgt_fio 01:05:06.799 iscsi_tgt_fuzz 01:05:06.799 iscsi_tgt_initiator 01:05:06.799 iscsi_tgt_ip_migration 01:05:06.799 iscsi_tgt_iscsi_lvol 01:05:06.799 iscsi_tgt_login_redirection 01:05:06.799 iscsi_tgt_multiconnection 01:05:06.799 iscsi_tgt_qos 01:05:06.799 iscsi_tgt_rbd 01:05:06.799 iscsi_tgt_reset 01:05:06.799 iscsi_tgt_resize 01:05:06.799 iscsi_tgt_rpc_config 01:05:06.799 iscsi_tgt_sock 01:05:06.799 iscsi_tgt_trace_record 01:05:06.799 json_config 01:05:06.799 json_config_extra_key 01:05:06.799 kernel_target_abort 01:05:06.799 keyring_file 01:05:06.799 llvm_fuzz 01:05:06.799 locking_app_on_locked_coremask 01:05:06.799 locking_app_on_unlocked_coremask 01:05:06.799 locking_overlapped_coremask 01:05:06.799 locking_overlapped_coremask_via_rpc 01:05:06.799 lvol 01:05:06.799 lvol_basic 01:05:06.799 lvol_esnap 01:05:06.799 lvol_external_copy 01:05:06.799 lvol_external_snapshot 01:05:06.799 lvol_hotremove 01:05:06.799 lvol_provisioning 01:05:06.799 lvol_rename 01:05:06.799 lvol_resize 01:05:06.799 lvol_snapshot_clone 01:05:06.799 lvol_tasting 01:05:06.799 lvs_grow_clean 01:05:06.799 lvs_grow_dirty 01:05:06.799 make 01:05:06.799 no_shrink_alloc 01:05:06.799 non_locking_app_on_locked_coremask 01:05:06.799 nvme 01:05:06.799 nvme_arbitration 01:05:06.799 nvme_cli_cuse 01:05:06.799 nvme_cli_plugin 01:05:06.799 nvme_cmb 01:05:06.799 nvme_cuse 01:05:06.799 nvme_cuse_app 01:05:06.799 nvme_cuse_rpc 01:05:06.799 nvme_doorbell_aers 01:05:06.799 nvme_e2edp 01:05:06.799 nvme_err_injection 01:05:06.799 nvme_fdp 01:05:06.799 nvme_fio 01:05:06.799 nvme_flexible_data_placement 01:05:06.799 nvme_hello_world 01:05:06.799 nvme_identify 01:05:06.799 nvme_mount 01:05:06.799 nvme_multi_aen 01:05:06.799 nvme_multi_secondary 01:05:06.799 nvme_ns_manage_cuse 01:05:06.799 nvme_overhead 01:05:06.799 nvme_perf 01:05:06.799 nvme_pmr 01:05:06.799 nvme_pmr_persistence 01:05:06.799 nvme_reserve 01:05:06.799 nvme_reset 01:05:06.799 nvme_rpc 01:05:06.799 nvme_rpc_timeouts 01:05:06.799 nvme_scc 01:05:06.799 nvme_sgl 01:05:06.799 nvme_simple_copy 01:05:06.799 nvme_single_aen 01:05:06.799 nvme_smartctl_cuse 01:05:06.799 nvme_startup 01:05:06.799 nvme_zns 01:05:06.799 nvmf_abort 01:05:06.799 nvmf_abort_qd_sizes 01:05:06.799 nvmf_aer 01:05:06.799 nvmf_async_init 01:05:06.799 nvmf_auth_host 01:05:06.799 nvmf_auth_target 01:05:06.799 nvmf_bdev_io_wait 01:05:06.799 nvmf_bdevio 01:05:06.799 nvmf_bdevio_no_huge 01:05:06.799 nvmf_bdevperf 01:05:06.799 nvmf_connect_disconnect 01:05:06.799 nvmf_connect_stress 01:05:06.799 nvmf_delete_subsystem 01:05:06.799 nvmf_device_removal 01:05:06.799 nvmf_device_removal_pci_remove 01:05:06.799 nvmf_device_removal_pci_remove_no_srq 01:05:06.799 nvmf_dif 01:05:06.799 nvmf_digest 01:05:06.799 nvmf_digest_clean 01:05:06.799 nvmf_digest_dsa_initiator 01:05:06.799 nvmf_digest_dsa_target 01:05:06.799 nvmf_digest_error 01:05:06.799 nvmf_discovery_remove_ifc 01:05:06.799 nvmf_example 01:05:06.799 nvmf_failover 01:05:06.799 nvmf_filesystem 01:05:06.799 nvmf_filesystem_in_capsule 01:05:06.799 nvmf_filesystem_no_in_capsule 01:05:06.799 nvmf_fio_host 01:05:06.799 nvmf_fio_target 01:05:06.799 nvmf_fips 01:05:06.799 nvmf_fused_ordering 01:05:06.799 nvmf_fuzz 01:05:06.799 nvmf_host_discovery 01:05:06.799 nvmf_host_management 01:05:06.799 nvmf_host_multipath 01:05:06.799 nvmf_host_multipath_status 01:05:06.799 nvmf_identify 01:05:06.799 nvmf_identify_kernel_target 01:05:06.799 nvmf_identify_passthru 01:05:06.799 nvmf_invalid 01:05:06.799 nvmf_lvol 01:05:06.799 nvmf_lvs_grow 01:05:06.799 nvmf_mdns_discovery 01:05:06.799 nvmf_multicontroller 01:05:06.799 nvmf_multitarget 01:05:06.799 nvmf_nmic 01:05:06.799 nvmf_ns_hotplug_stress 01:05:06.799 nvmf_ns_masking 01:05:06.799 nvmf_nvme_cli 01:05:06.799 nvmf_perf 01:05:06.799 nvmf_perf_adq 01:05:06.799 nvmf_queue_depth 01:05:06.799 nvmf_rdma 01:05:06.799 nvmf_referrals 01:05:06.799 nvmf_rpc 01:05:06.799 nvmf_shutdown 01:05:06.799 nvmf_shutdown_tc1 01:05:06.799 nvmf_shutdown_tc2 01:05:06.799 nvmf_shutdown_tc3 01:05:06.799 nvmf_srq_overwhelm 01:05:06.799 nvmf_target_disconnect 01:05:06.799 nvmf_target_disconnect_tc1 01:05:06.799 nvmf_target_disconnect_tc2 01:05:06.799 nvmf_target_disconnect_tc3 01:05:06.799 nvmf_target_discovery 01:05:06.799 nvmf_target_multipath 01:05:06.799 nvmf_tcp 01:05:06.799 nvmf_timeout 01:05:06.799 nvmf_tls 01:05:06.799 nvmf_vfio_user 01:05:06.799 nvmf_vfio_user_fuzz 01:05:06.799 nvmf_vfio_user_nvme_compliance 01:05:06.799 nvmf_vhost 01:05:06.799 nvmf_zcopy 01:05:06.799 ocf 01:05:06.799 ocf_bdevperf_iotypes 01:05:06.799 ocf_configuration_change 01:05:06.799 ocf_create_destruct 01:05:06.799 ocf_fio_modes 01:05:06.799 ocf_flush 01:05:06.799 ocf_multicore 01:05:06.799 ocf_remove 01:05:06.799 ocf_stats 01:05:06.799 odd_alloc 01:05:06.799 packaging 01:05:06.799 per_node_1G_alloc 01:05:06.799 raid0_resize_test 01:05:06.799 raid5f_rebuild_test 01:05:06.799 raid5f_rebuild_test_sb 01:05:06.799 raid5f_state_function_test 01:05:06.799 raid5f_state_function_test_sb 01:05:06.799 raid5f_superblock_test 01:05:06.799 raid_function_test_concat 01:05:06.799 raid_function_test_raid0 01:05:06.799 raid_rebuild_test 01:05:06.799 raid_rebuild_test_io 01:05:06.799 raid_rebuild_test_sb 01:05:06.799 raid_rebuild_test_sb_4k 01:05:06.799 raid_rebuild_test_sb_io 01:05:06.799 raid_rebuild_test_sb_md_interleaved 01:05:06.799 raid_rebuild_test_sb_md_separate 01:05:06.799 raid_state_function_test 01:05:06.799 raid_state_function_test_sb 01:05:06.799 raid_state_function_test_sb_4k 01:05:06.799 raid_state_function_test_sb_md_interleaved 01:05:06.799 raid_state_function_test_sb_md_separate 01:05:06.799 raid_superblock_test 01:05:06.799 raid_superblock_test_4k 01:05:06.799 raid_superblock_test_md_interleaved 01:05:06.799 raid_superblock_test_md_separate 01:05:06.799 reactor_set_interrupt 01:05:06.799 reap_unregistered_poller 01:05:06.799 rocksdb 01:05:06.799 rocksdb_insert 01:05:06.799 rocksdb_overwrite 01:05:06.799 rocksdb_randread 01:05:06.799 rocksdb_readwrite 01:05:06.799 rocksdb_writesync 01:05:06.799 rpc 01:05:06.799 rpc_client 01:05:06.799 rpc_daemon_integrity 01:05:06.799 rpc_integrity 01:05:06.799 rpc_plugins 01:05:06.799 rpc_trace_cmd_test 01:05:06.799 rpm_packaging 01:05:06.799 scanbuild_make 01:05:06.799 scheduler 01:05:06.799 scheduler_create_thread 01:05:06.799 setup.sh 01:05:06.799 skip_rpc 01:05:06.799 skip_rpc_with_delay 01:05:06.799 skip_rpc_with_json 01:05:06.799 sma 01:05:06.799 sma_crypto 01:05:06.799 sma_discovery 01:05:06.799 sma_nvmf_tcp 01:05:06.799 sma_plugins 01:05:06.799 sma_qos 01:05:06.799 sma_vfiouser_qemu 01:05:06.799 sma_vhost 01:05:06.799 spdk_dd 01:05:06.799 spdk_dd_basic_rw 01:05:06.799 spdk_dd_bdev_to_bdev 01:05:06.799 spdk_dd_malloc 01:05:06.799 spdk_dd_negative 01:05:06.799 spdk_dd_posix 01:05:06.799 spdk_dd_sparse 01:05:06.799 spdk_dd_uring 01:05:06.799 spdk_target_abort 01:05:06.799 spdkcli_iscsi 01:05:06.799 spdkcli_nvmf_rdma 01:05:06.799 spdkcli_nvmf_tcp 01:05:06.799 spdkcli_raid 01:05:06.799 spdkcli_rbd 01:05:06.799 spdkcli_tcp 01:05:06.799 spdkcli_vhost 01:05:06.799 spdkcli_virtio 01:05:06.799 sw_hotplug 01:05:06.800 test_bdev_lvol_delete_lvstore_with_clones 01:05:06.800 test_bdev_lvol_delete_ordering 01:05:06.800 test_clone_decouple_parent 01:05:06.800 test_clone_inflate 01:05:06.800 test_clone_snapshot_relations 01:05:06.800 test_construct_lvol 01:05:06.800 test_construct_lvol_alias_conflict 01:05:06.800 test_construct_lvol_fio_clear_method_none 01:05:06.800 test_construct_lvol_fio_clear_method_unmap 01:05:06.800 test_construct_lvol_full_lvs 01:05:06.800 test_construct_lvol_inexistent_lvs 01:05:06.800 test_construct_lvols_conflict_alias 01:05:06.800 test_construct_lvs 01:05:06.800 test_construct_lvs_clear_methods 01:05:06.800 test_construct_lvs_conflict_alias 01:05:06.800 test_construct_lvs_different_cluster_size 01:05:06.800 test_construct_lvs_nonexistent_bdev 01:05:06.800 test_construct_multi_lvols 01:05:06.800 test_construct_nested_lvol 01:05:06.800 test_construct_two_lvs_on_the_same_bdev 01:05:06.800 test_create_snapshot_of_snapshot 01:05:06.800 test_create_snapshot_with_io 01:05:06.800 test_delete_lvol_store_persistent_positive 01:05:06.800 test_delete_snapshot_with_clone 01:05:06.800 test_delete_snapshot_with_snapshot 01:05:06.800 test_destroy_after_bdev_lvol_resize_positive 01:05:06.800 test_esnap_clones 01:05:06.800 test_esnap_late_arrival 01:05:06.800 test_esnap_reload 01:05:06.800 test_esnap_remove_degraded 01:05:06.800 test_hotremove_lvol_store 01:05:06.800 test_hotremove_lvol_store_base 01:05:06.800 test_hotremove_lvol_store_base_with_lvols 01:05:06.800 test_hotremove_lvol_store_multiple_lvols 01:05:06.800 test_lvol_bdev_readonly 01:05:06.800 test_lvol_list 01:05:06.800 test_lvol_rename_negative 01:05:06.800 test_rename_lvs_negative 01:05:06.800 test_rename_positive 01:05:06.800 test_resize_lvol 01:05:06.800 test_resize_lvol_negative 01:05:06.800 test_resize_lvol_with_io_traffic 01:05:06.800 test_shallow_copy_compare 01:05:06.800 test_sigterm 01:05:06.800 test_snapshot_compare_with_lvol_bdev 01:05:06.800 test_tasting 01:05:06.800 test_thin_lvol_check_integrity 01:05:06.800 test_thin_lvol_check_space 01:05:06.800 test_thin_lvol_check_zeroes 01:05:06.800 test_thin_lvol_resize 01:05:06.800 test_thin_overprovisioning 01:05:06.800 test_unregister_lvol_bdev 01:05:06.800 thread 01:05:06.800 thread_poller_perf 01:05:06.800 thread_spdk_lock 01:05:06.800 ubsan 01:05:06.800 unittest 01:05:06.800 unittest_accel 01:05:06.800 unittest_bdev 01:05:06.800 unittest_bdev_compress 01:05:06.800 unittest_bdev_crypto 01:05:06.800 unittest_bdev_raid5f 01:05:06.800 unittest_blob_blobfs 01:05:06.800 unittest_build 01:05:06.800 unittest_dma 01:05:06.800 unittest_dpdk_compressdev 01:05:06.800 unittest_event 01:05:06.800 unittest_ftl 01:05:06.800 unittest_idxd_user 01:05:06.800 unittest_include 01:05:06.800 unittest_init 01:05:06.800 unittest_ioat 01:05:06.800 unittest_iobuf 01:05:06.800 unittest_iscsi 01:05:06.800 unittest_json 01:05:06.800 unittest_keyring 01:05:06.800 unittest_lib_reduce 01:05:06.800 unittest_log 01:05:06.800 unittest_lvol 01:05:06.800 unittest_notify 01:05:06.800 unittest_nvme 01:05:06.800 unittest_nvme_cuse 01:05:06.800 unittest_nvme_rdma 01:05:06.800 unittest_nvmf 01:05:06.800 unittest_nvmf_rdma 01:05:06.800 unittest_nvmf_transport 01:05:06.800 unittest_nvmf_vfio_user 01:05:06.800 unittest_pci_event 01:05:06.800 unittest_rdma 01:05:06.800 unittest_rpc 01:05:06.800 unittest_scsi 01:05:06.800 unittest_sock 01:05:06.800 unittest_thread 01:05:06.800 unittest_util 01:05:06.800 unittest_vhost 01:05:06.800 valgrind 01:05:06.800 version 01:05:06.800 vfio_fuzz 01:05:06.800 vfio_user_nvme_fio 01:05:06.800 vfio_user_nvme_restart_vm 01:05:06.800 vfio_user_qemu 01:05:06.800 vfio_user_virtio_bdevperf 01:05:06.800 vfio_user_virtio_blk_restart_vm 01:05:06.800 vfio_user_virtio_scsi_restart_vm 01:05:06.800 vhost 01:05:06.800 vhost_blk_lvol_integrity 01:05:06.800 vhost_blk_packed_ring_integrity 01:05:06.800 vhost_blockdev 01:05:06.800 vhost_boot 01:05:06.800 vhost_fuzz 01:05:06.800 vhost_negative 01:05:06.800 vhost_scsi_lvol_integrity 01:05:06.800 vhost_shared 01:05:06.800 zoned_fio 01:05:06.800 01:05:06.800 01:05:06.800 -----Tests Missing From Build------ 01:05:06.800 balanced 01:05:06.800 blockdev_xnvme 01:05:06.800 build_native_dpdk 01:05:06.800 build_shared_native_dpdk_rpm 01:05:06.800 build_shared_rpm_with_rpmed_dpdk 01:05:06.800 busy 01:05:06.800 core_load 01:05:06.800 ftl 01:05:06.800 ftl_bdevperf 01:05:06.800 ftl_dirty_shutdown 01:05:06.800 ftl_fio_basic 01:05:06.800 ftl_restore 01:05:06.800 ftl_restore_fast 01:05:06.800 ftl_trim 01:05:06.800 ftl_upgrade_shutdown 01:05:06.800 iscsi_tgt_data_digest 01:05:06.800 iscsi_tgt_ext4test 01:05:06.800 keyring_linux 01:05:06.800 nvme_boot_partition 01:05:06.800 nvme_bp 01:05:06.800 nvme_opal 01:05:06.800 nvme_opal_bdevio 01:05:06.800 nvme_opal_bdevperf 01:05:06.800 nvme_opal_spdk_tgt 01:05:06.800 nvme_xnvme 01:05:06.800 nvmf_fc 01:05:06.800 nvmf_initiator_timeout 01:05:06.800 nvmf_multiconnection 01:05:06.800 spdkcli_nvmf_fc 01:05:06.800 test_create_multi_ublk 01:05:06.800 test_create_ublk 01:05:06.800 test_save_ublk_config 01:05:06.800 ublk 01:05:06.800 ublk_recovery 01:05:06.800 unittest_nvmf_fc 01:05:06.800 unittest_nvmf_fc_ls 01:05:06.800 vhost_blk_cores_2ctrl 01:05:06.800 vhost_blk_fs_integrity 01:05:06.800 vhost_blk_hot_remove 01:05:06.800 vhost_blk_integrity 01:05:06.800 vhost_hotplug 01:05:06.800 vhost_migration 01:05:06.800 vhost_migration_tc1 01:05:06.800 vhost_migration_tc2 01:05:06.800 vhost_readonly 01:05:06.800 vhost_scsi_cores_2ctrl 01:05:06.800 vhost_scsi_fs_integrity 01:05:06.800 vhost_scsi_hot_remove 01:05:06.800 vhost_scsi_integrity 01:05:06.800 vmd 01:05:06.800 vmd_bdev_svc 01:05:06.800 vmd_fio 01:05:06.800 vmd_hello_world 01:05:06.800 vmd_identify 01:05:06.800 vmd_perf 01:05:06.800 xnvme_bdevperf 01:05:06.800 xnvme_to_malloc_dd_copy 01:05:06.800 01:05:06.800 01:05:06.800 -----Tests Missing ASAN------ 01:05:06.800 accel_cdev_comp 01:05:06.800 accel_cdev_decmop_full 01:05:06.800 accel_cdev_decomp 01:05:06.800 accel_cdev_decomp_full_mcore 01:05:06.800 accel_cdev_decomp_full_mthread 01:05:06.800 accel_cdev_decomp_mcore 01:05:06.800 accel_cdev_decomp_mthread 01:05:06.800 autobuild 01:05:06.800 autobuild_build_doc 01:05:06.800 autobuild_check_dpdk_pci_api 01:05:06.800 autobuild_check_format 01:05:06.800 autobuild_check_so_deps 01:05:06.800 autobuild_external_code 01:05:06.800 autobuild_generated_files_check 01:05:06.800 autobuild_generated_files_check_post_clean 01:05:06.800 autobuild_header_dependency_check 01:05:06.800 autobuild_llvm_precompile 01:05:06.800 autobuild_make_install 01:05:06.800 autobuild_make_uninstall 01:05:06.800 autobuild_ocf_precompile 01:05:06.800 balanced 01:05:06.800 bdev_crypto_enomem 01:05:06.800 blockdev_crypto_aesni 01:05:06.800 blockdev_crypto_qat 01:05:06.800 blockdev_crypto_sw 01:05:06.800 blockdev_rbd 01:05:06.800 blockdev_xnvme 01:05:06.800 build_native_dpdk 01:05:06.800 build_shared_native_dpdk_rpm 01:05:06.800 build_shared_rpm_with_rpmed_dpdk 01:05:06.800 busy 01:05:06.800 chaining 01:05:06.800 check_header_filenames 01:05:06.800 cmb_copy 01:05:06.800 compress_compdev 01:05:06.801 compress_isal 01:05:06.801 confirm_abi_deps 01:05:06.801 confirm_makefile_deps 01:05:06.801 core_load 01:05:06.801 dd_uring_copy 01:05:06.801 dpdk_governor 01:05:06.801 external_make_accel_driver_shared 01:05:06.801 external_make_accel_driver_static 01:05:06.801 external_make_accel_module_shared 01:05:06.801 external_make_accel_module_static 01:05:06.801 external_make_hello_bdev_shared_combo 01:05:06.801 external_make_hello_bdev_shared_iso 01:05:06.801 external_make_hello_bdev_static 01:05:06.801 external_make_hello_no_bdev_shared_combo 01:05:06.801 external_make_hello_no_bdev_shared_iso 01:05:06.801 external_make_hello_no_bdev_static 01:05:06.801 external_make_nvme_shared 01:05:06.801 external_make_nvme_static 01:05:06.801 external_run_accel_driver_shared 01:05:06.801 external_run_accel_driver_static 01:05:06.801 external_run_accel_module_shared 01:05:06.801 external_run_accel_module_static 01:05:06.801 external_run_hello_bdev_shared_combo 01:05:06.801 external_run_hello_bdev_shared_iso 01:05:06.801 external_run_hello_bdev_static 01:05:06.801 external_run_hello_no_bdev_shared_combo 01:05:06.801 external_run_hello_no_bdev_shared_iso 01:05:06.801 external_run_hello_no_bdev_static 01:05:06.801 external_run_nvme_shared 01:05:06.801 external_run_nvme_static 01:05:06.801 ftl 01:05:06.801 ftl_bdevperf 01:05:06.801 ftl_dirty_shutdown 01:05:06.801 ftl_fio_basic 01:05:06.801 ftl_restore 01:05:06.801 ftl_restore_fast 01:05:06.801 ftl_trim 01:05:06.801 ftl_upgrade_shutdown 01:05:06.801 go_rpc 01:05:06.801 idle 01:05:06.801 interrupt_mode 01:05:06.801 ioat 01:05:06.801 ioat_perf 01:05:06.801 ioat_verify 01:05:06.801 is_zoned 01:05:06.801 iscsi_tgt 01:05:06.801 iscsi_tgt_bdev_io_wait 01:05:06.801 iscsi_tgt_calsoft 01:05:06.801 iscsi_tgt_data_digest 01:05:06.801 iscsi_tgt_digest 01:05:06.801 iscsi_tgt_digests 01:05:06.801 iscsi_tgt_ext4test 01:05:06.801 iscsi_tgt_filesystem 01:05:06.801 iscsi_tgt_filesystem_btrfs 01:05:06.801 iscsi_tgt_filesystem_ext4 01:05:06.801 iscsi_tgt_filesystem_xfs 01:05:06.801 iscsi_tgt_fio 01:05:06.801 iscsi_tgt_fuzz 01:05:06.801 iscsi_tgt_initiator 01:05:06.801 iscsi_tgt_ip_migration 01:05:06.801 iscsi_tgt_iscsi_lvol 01:05:06.801 iscsi_tgt_login_redirection 01:05:06.801 iscsi_tgt_multiconnection 01:05:06.801 iscsi_tgt_qos 01:05:06.801 iscsi_tgt_rbd 01:05:06.801 iscsi_tgt_reset 01:05:06.801 iscsi_tgt_resize 01:05:06.801 iscsi_tgt_rpc_config 01:05:06.801 iscsi_tgt_sock 01:05:06.801 iscsi_tgt_trace_record 01:05:06.801 keyring_linux 01:05:06.801 llvm_fuzz 01:05:06.801 nvme_boot_partition 01:05:06.801 nvme_bp 01:05:06.801 nvme_cli_cuse 01:05:06.801 nvme_cli_plugin 01:05:06.801 nvme_cmb 01:05:06.801 nvme_cuse 01:05:06.801 nvme_cuse_app 01:05:06.801 nvme_cuse_rpc 01:05:06.801 nvme_ns_manage_cuse 01:05:06.801 nvme_opal 01:05:06.801 nvme_opal_bdevio 01:05:06.801 nvme_opal_bdevperf 01:05:06.801 nvme_opal_spdk_tgt 01:05:06.801 nvme_pmr 01:05:06.801 nvme_pmr_persistence 01:05:06.801 nvme_smartctl_cuse 01:05:06.801 nvme_xnvme 01:05:06.801 nvme_zns 01:05:06.801 nvmf_bdevperf 01:05:06.801 nvmf_device_removal 01:05:06.801 nvmf_device_removal_pci_remove 01:05:06.801 nvmf_device_removal_pci_remove_no_srq 01:05:06.801 nvmf_digest_clean 01:05:06.801 nvmf_fc 01:05:06.801 nvmf_fuzz 01:05:06.801 nvmf_host_multipath 01:05:06.801 nvmf_initiator_timeout 01:05:06.801 nvmf_mdns_discovery 01:05:06.801 nvmf_multiconnection 01:05:06.801 nvmf_nvme_cli 01:05:06.801 nvmf_perf_adq 01:05:06.801 nvmf_rdma 01:05:06.801 nvmf_shutdown 01:05:06.801 nvmf_shutdown_tc1 01:05:06.801 nvmf_shutdown_tc2 01:05:06.801 nvmf_shutdown_tc3 01:05:06.801 nvmf_srq_overwhelm 01:05:06.801 nvmf_target_disconnect 01:05:06.801 nvmf_target_disconnect_tc1 01:05:06.801 nvmf_target_disconnect_tc2 01:05:06.801 nvmf_target_disconnect_tc3 01:05:06.801 nvmf_timeout 01:05:06.801 nvmf_vfio_user 01:05:06.801 nvmf_vfio_user_fuzz 01:05:06.801 nvmf_vfio_user_nvme_compliance 01:05:06.801 ocf 01:05:06.801 ocf_bdevperf_iotypes 01:05:06.801 ocf_configuration_change 01:05:06.801 ocf_create_destruct 01:05:06.801 ocf_fio_modes 01:05:06.801 ocf_flush 01:05:06.801 ocf_multicore 01:05:06.801 ocf_remove 01:05:06.801 ocf_stats 01:05:06.801 scanbuild_make 01:05:06.801 scheduler 01:05:06.801 spdk_dd_uring 01:05:06.801 spdkcli_iscsi 01:05:06.801 spdkcli_nvmf_fc 01:05:06.801 spdkcli_nvmf_rdma 01:05:06.801 spdkcli_raid 01:05:06.801 spdkcli_rbd 01:05:06.801 spdkcli_virtio 01:05:06.801 test_create_multi_ublk 01:05:06.801 test_create_ublk 01:05:06.801 test_save_ublk_config 01:05:06.801 ublk 01:05:06.801 ublk_recovery 01:05:06.801 unittest_bdev_compress 01:05:06.801 unittest_bdev_crypto 01:05:06.801 unittest_dpdk_compressdev 01:05:06.801 unittest_lib_reduce 01:05:06.801 unittest_nvmf_fc 01:05:06.801 unittest_nvmf_fc_ls 01:05:06.801 unittest_nvmf_vfio_user 01:05:06.801 valgrind 01:05:06.801 vfio_fuzz 01:05:06.801 vhost_blk_cores_2ctrl 01:05:06.801 vhost_blk_fs_integrity 01:05:06.801 vhost_blk_hot_remove 01:05:06.801 vhost_blk_integrity 01:05:06.801 vhost_blockdev 01:05:06.801 vhost_fuzz 01:05:06.801 vhost_hotplug 01:05:06.801 vhost_migration 01:05:06.801 vhost_migration_tc1 01:05:06.801 vhost_migration_tc2 01:05:06.801 vhost_readonly 01:05:06.801 vhost_scsi_cores_2ctrl 01:05:06.801 vhost_scsi_fs_integrity 01:05:06.801 vhost_scsi_hot_remove 01:05:06.801 vhost_scsi_integrity 01:05:06.801 vhost_shared 01:05:06.801 vmd 01:05:06.801 vmd_bdev_svc 01:05:06.801 vmd_fio 01:05:06.801 vmd_hello_world 01:05:06.801 vmd_identify 01:05:06.801 vmd_perf 01:05:06.801 xnvme_bdevperf 01:05:06.801 xnvme_to_malloc_dd_copy 01:05:06.801 zoned_fio 01:05:06.801 01:05:06.801 01:05:06.801 -----Tests Missing UBSAN------ 01:05:06.801 autobuild 01:05:06.801 autobuild_build_doc 01:05:06.801 autobuild_check_dpdk_pci_api 01:05:06.801 autobuild_check_format 01:05:06.801 autobuild_check_so_deps 01:05:06.801 autobuild_external_code 01:05:06.801 autobuild_generated_files_check 01:05:06.801 autobuild_generated_files_check_post_clean 01:05:06.801 autobuild_header_dependency_check 01:05:06.801 autobuild_make_install 01:05:06.801 autobuild_make_uninstall 01:05:06.801 balanced 01:05:06.801 blockdev_xnvme 01:05:06.801 build_native_dpdk 01:05:06.801 build_rpm_from_gen_spec 01:05:06.801 build_shared_native_dpdk_rpm 01:05:06.801 build_shared_rpm 01:05:06.801 build_shared_rpm_with_rpmed_dpdk 01:05:06.801 busy 01:05:06.801 check_header_filenames 01:05:06.801 cmb_copy 01:05:06.801 confirm_abi_deps 01:05:06.801 confirm_makefile_deps 01:05:06.801 core_load 01:05:06.801 external_make_accel_driver_shared 01:05:06.801 external_make_accel_driver_static 01:05:06.801 external_make_accel_module_shared 01:05:06.801 external_make_accel_module_static 01:05:06.801 external_make_hello_bdev_shared_combo 01:05:06.801 external_make_hello_bdev_shared_iso 01:05:06.801 external_make_hello_bdev_static 01:05:06.801 external_make_hello_no_bdev_shared_combo 01:05:06.801 external_make_hello_no_bdev_shared_iso 01:05:06.802 external_make_hello_no_bdev_static 01:05:06.802 external_make_nvme_shared 01:05:06.802 external_make_nvme_static 01:05:06.802 external_run_accel_driver_shared 01:05:06.802 external_run_accel_driver_static 01:05:06.802 external_run_accel_module_shared 01:05:06.802 external_run_accel_module_static 01:05:06.802 external_run_hello_bdev_shared_combo 01:05:06.802 external_run_hello_bdev_shared_iso 01:05:06.802 external_run_hello_bdev_static 01:05:06.802 external_run_hello_no_bdev_shared_combo 01:05:06.802 external_run_hello_no_bdev_shared_iso 01:05:06.802 external_run_hello_no_bdev_static 01:05:06.802 external_run_nvme_shared 01:05:06.802 external_run_nvme_static 01:05:06.802 ftl 01:05:06.802 ftl_bdevperf 01:05:06.802 ftl_dirty_shutdown 01:05:06.802 ftl_fio_basic 01:05:06.802 ftl_restore 01:05:06.802 ftl_restore_fast 01:05:06.802 ftl_trim 01:05:06.802 ftl_upgrade_shutdown 01:05:06.802 is_zoned 01:05:06.802 iscsi_tgt_data_digest 01:05:06.802 iscsi_tgt_ext4test 01:05:06.802 keyring_linux 01:05:06.802 nvme_boot_partition 01:05:06.802 nvme_bp 01:05:06.802 nvme_cmb 01:05:06.802 nvme_opal 01:05:06.802 nvme_opal_bdevio 01:05:06.802 nvme_opal_bdevperf 01:05:06.802 nvme_opal_spdk_tgt 01:05:06.802 nvme_pmr 01:05:06.802 nvme_pmr_persistence 01:05:06.802 nvme_xnvme 01:05:06.802 nvme_zns 01:05:06.802 nvmf_fc 01:05:06.802 nvmf_initiator_timeout 01:05:06.802 nvmf_multiconnection 01:05:06.802 packaging 01:05:06.802 rpm_packaging 01:05:06.802 scanbuild_make 01:05:06.802 spdkcli_nvmf_fc 01:05:06.802 test_create_multi_ublk 01:05:06.802 test_create_ublk 01:05:06.802 test_save_ublk_config 01:05:06.802 ublk 01:05:06.802 ublk_recovery 01:05:06.802 unittest_bdev_compress 01:05:06.802 unittest_bdev_crypto 01:05:06.802 unittest_dpdk_compressdev 01:05:06.802 unittest_lib_reduce 01:05:06.802 unittest_nvmf_fc 01:05:06.802 unittest_nvmf_fc_ls 01:05:06.802 unittest_nvmf_vfio_user 01:05:06.802 valgrind 01:05:06.802 vhost_blk_cores_2ctrl 01:05:06.802 vhost_blk_fs_integrity 01:05:06.802 vhost_blk_hot_remove 01:05:06.802 vhost_blk_integrity 01:05:06.802 vhost_hotplug 01:05:06.802 vhost_migration 01:05:06.802 vhost_migration_tc1 01:05:06.802 vhost_migration_tc2 01:05:06.802 vhost_readonly 01:05:06.802 vhost_scsi_cores_2ctrl 01:05:06.802 vhost_scsi_fs_integrity 01:05:06.802 vhost_scsi_hot_remove 01:05:06.802 vhost_scsi_integrity 01:05:06.802 vmd 01:05:06.802 vmd_bdev_svc 01:05:06.802 vmd_fio 01:05:06.802 vmd_hello_world 01:05:06.802 vmd_identify 01:05:06.802 vmd_perf 01:05:06.802 xnvme_bdevperf 01:05:06.802 xnvme_to_malloc_dd_copy 01:05:06.802 zoned_fio 01:05:06.802 Not all tests were run. Failing the build. 01:05:06.802 ['blockdev_xnvme', 'ftl', 'ftl_bdevperf', 'ftl_dirty_shutdown', 'ftl_fio_basic', 'ftl_restore', 'ftl_trim', 'ftl_upgrade_shutdown', 'nvme_xnvme', 'test_create_multi_ublk', 'test_create_ublk', 'test_save_ublk_config', 'ublk', 'ublk_recovery', 'xnvme_bdevperf', 'xnvme_to_malloc_dd_copy'] 01:05:06.802 + compress_coverage_and_docs 01:05:06.802 + echo 'Start compress coverage and docs' 01:05:06.802 Start compress coverage and docs 01:05:06.802 + tar -C coverage -czf coverage_autotest-per-patch_122886.tar.gz ./ --remove-files 01:05:07.365 + tar -C ut_coverage -czf ut_coverage_autotest-per-patch_122886.tar.gz ./ --remove-files 01:05:08.294 + tar -C doc -czf doc_autotest-per-patch_122886.tar.gz ./ --remove-files 01:05:08.294 + [[ -d llvm ]] 01:05:08.294 + true 01:05:08.294 + echo 'End compress coverage and docs' 01:05:08.294 End compress coverage and docs 01:05:08.294 + remove_partial_coverage_files 01:05:08.294 + find . -name 'cov_*.info' -delete 01:05:08.554 + echo 'End stage post_gen_coverage.sh' 01:05:08.554 End stage post_gen_coverage.sh 01:05:08.570 [Pipeline] } 01:05:08.574 ERROR: script returned exit code 1 01:05:08.612 [Pipeline] // catchError 01:05:08.626 [Pipeline] catchError 01:05:08.628 [Pipeline] { 01:05:08.654 [Pipeline] dir 01:05:08.655 Running in /var/jenkins/workspace/autotest-per-patch_2/post_process 01:05:08.656 [Pipeline] {