00:00:00.001 Started by upstream project "nightly-trigger" build number 3087 00:00:00.001 originally caused by: 00:00:00.001 Started by timer 00:00:00.013 Checking out git https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool into /var/jenkins_home/workspace/autotest-spdk-master-vs-dpdk-main_script/33b20b30f0a51e6b52980845e0f6aa336787973ad45e341fbbf98d1b65b265d4 to read jbp/jenkins/jjb-config/jobs/autotest-upstream/autotest.groovy 00:00:00.013 The recommended git tool is: git 00:00:00.014 using credential 00000000-0000-0000-0000-000000000002 00:00:00.015 > git rev-parse --resolve-git-dir /var/jenkins_home/workspace/autotest-spdk-master-vs-dpdk-main_script/33b20b30f0a51e6b52980845e0f6aa336787973ad45e341fbbf98d1b65b265d4/jbp/.git # timeout=10 00:00:00.026 Fetching changes from the remote Git repository 00:00:00.028 > git config remote.origin.url https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool # timeout=10 00:00:00.037 Using shallow fetch with depth 1 00:00:00.037 Fetching upstream changes from https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool 00:00:00.037 > git --version # timeout=10 00:00:00.046 > git --version # 'git version 2.39.2' 00:00:00.046 using GIT_ASKPASS to set credentials SPDKCI HTTPS Credentials 00:00:00.046 Setting http proxy: proxy-dmz.intel.com:911 00:00:00.046 > git fetch --tags --force --progress --depth=1 -- https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool refs/heads/master # timeout=5 00:00:02.264 > git rev-parse origin/FETCH_HEAD^{commit} # timeout=10 00:00:02.277 > git rev-parse FETCH_HEAD^{commit} # timeout=10 00:00:02.289 Checking out Revision 10da8f6d99838e411e4e94523ded0bfebf3e7100 (FETCH_HEAD) 00:00:02.289 > git config core.sparsecheckout # timeout=10 00:00:02.299 > git read-tree -mu HEAD # timeout=10 00:00:02.315 > git checkout -f 10da8f6d99838e411e4e94523ded0bfebf3e7100 # timeout=5 00:00:02.336 Commit message: "scripts/create_git_mirror: Update path to xnvme submodule" 00:00:02.337 > git rev-list --no-walk 10da8f6d99838e411e4e94523ded0bfebf3e7100 # timeout=10 00:00:02.934 [Pipeline] Start of Pipeline 00:00:02.948 [Pipeline] library 00:00:02.949 Loading library shm_lib@master 00:00:02.949 Library shm_lib@master is cached. Copying from home. 00:00:02.967 [Pipeline] node 00:00:02.972 Running on ME3 in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main 00:00:02.977 [Pipeline] { 00:00:02.992 [Pipeline] cleanWs 00:00:03.000 [WS-CLEANUP] Deleting project workspace... 00:00:03.000 [WS-CLEANUP] Deferred wipeout is used... 00:00:03.006 [WS-CLEANUP] done 00:00:03.010 [Pipeline] stage 00:00:03.014 [Pipeline] { (Prologue) 00:00:03.099 [Pipeline] withCredentials 00:00:03.109 > git --version # timeout=10 00:00:03.119 > git --version # 'git version 2.39.2' 00:00:03.136 Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD or $GIT_ASKPASS 00:00:03.138 [Pipeline] { 00:00:03.146 [Pipeline] retry 00:00:03.148 [Pipeline] { 00:00:03.304 [Pipeline] sh 00:00:03.579 + git ls-remote https://review.spdk.io/gerrit/a/build_pool/jenkins_build_pool refs/heads/master 00:00:05.498 [Pipeline] } 00:00:05.525 [Pipeline] // retry 00:00:05.530 [Pipeline] } 00:00:05.555 [Pipeline] // withCredentials 00:00:05.564 [Pipeline] nodesByLabel 00:00:05.565 Found a total of 1 nodes with the 'sorcerer' label 00:00:05.576 [Pipeline] httpRequest 00:00:05.581 HttpMethod: GET 00:00:05.581 URL: http://10.211.164.101/packages/jbp_10da8f6d99838e411e4e94523ded0bfebf3e7100.tar.gz 00:00:05.582 Sending request to url: http://10.211.164.101/packages/jbp_10da8f6d99838e411e4e94523ded0bfebf3e7100.tar.gz 00:00:05.583 Response Code: HTTP/1.1 200 OK 00:00:05.584 Success: Status code 200 is in the accepted range: 200,404 00:00:05.584 Saving response body to /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/jbp_10da8f6d99838e411e4e94523ded0bfebf3e7100.tar.gz 00:00:05.723 [Pipeline] sh 00:00:06.004 + tar --no-same-owner -xf jbp_10da8f6d99838e411e4e94523ded0bfebf3e7100.tar.gz 00:00:06.022 [Pipeline] withCredentials 00:00:06.031 > git --version # timeout=10 00:00:06.043 > git --version # 'git version 2.39.2' 00:00:06.057 Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD or $GIT_ASKPASS 00:00:06.058 [Pipeline] { 00:00:06.065 [Pipeline] retry 00:00:06.066 [Pipeline] { 00:00:06.079 [Pipeline] sh 00:00:06.355 + git ls-remote https://review.spdk.io/gerrit/a/spdk/spdk refs/heads/master 00:00:09.654 [Pipeline] } 00:00:09.677 [Pipeline] // retry 00:00:09.682 [Pipeline] } 00:00:09.702 [Pipeline] // withCredentials 00:00:09.712 [Pipeline] httpRequest 00:00:09.717 HttpMethod: GET 00:00:09.717 URL: http://10.211.164.101/packages/spdk_4506c0c368f63ba9b9b013ecff216cef6ee8d0a4.tar.gz 00:00:09.717 Sending request to url: http://10.211.164.101/packages/spdk_4506c0c368f63ba9b9b013ecff216cef6ee8d0a4.tar.gz 00:00:09.718 Response Code: HTTP/1.1 200 OK 00:00:09.719 Success: Status code 200 is in the accepted range: 200,404 00:00:09.719 Saving response body to /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk_4506c0c368f63ba9b9b013ecff216cef6ee8d0a4.tar.gz 00:00:11.843 [Pipeline] sh 00:00:12.122 + tar --no-same-owner -xf spdk_4506c0c368f63ba9b9b013ecff216cef6ee8d0a4.tar.gz 00:00:14.030 [Pipeline] sh 00:00:14.309 + git -C spdk log --oneline -n5 00:00:14.309 4506c0c36 test/common: Enable inherit_errexit 00:00:14.309 b24df7cfa test: Drop superfluous calls to print_backtrace() 00:00:14.309 7b52e4c17 test/scheduler: Meassure utime of $spdk_pid threads as a fallback 00:00:14.309 1dc065205 test/scheduler: Calculate median of the cpu load samples 00:00:14.309 b22f1b34d test/scheduler: Enhance lookup of the $old_cgroup in move_proc() 00:00:14.324 [Pipeline] sh 00:00:14.602 + git -C spdk show -s --format=%s 4506c0c368f63ba9b9b013ecff216cef6ee8d0a4 00:00:14.618 [Pipeline] setCustomBuildProperty 00:00:14.626 [Pipeline] setCustomBuildProperty 00:00:14.643 [Pipeline] withCredentials 00:00:14.654 > git --version # timeout=10 00:00:14.664 > git --version # 'git version 2.39.2' 00:00:14.677 Masking supported pattern matches of $GIT_PASSWORD or $GIT_ASKPASS 00:00:14.679 [Pipeline] { 00:00:14.688 [Pipeline] retry 00:00:14.689 [Pipeline] { 00:00:14.705 [Pipeline] sh 00:00:14.983 + git ls-remote http://dpdk.org/git/dpdk main 00:00:15.253 [Pipeline] } 00:00:15.275 [Pipeline] // retry 00:00:15.279 [Pipeline] } 00:00:15.299 [Pipeline] // withCredentials 00:00:15.312 [Pipeline] httpRequest 00:00:15.316 HttpMethod: GET 00:00:15.317 URL: http://10.211.164.101/packages/dpdk_7e06c0de1952d3109a5b0c4779d7e7d8059c9d78.tar.gz 00:00:15.318 Sending request to url: http://10.211.164.101/packages/dpdk_7e06c0de1952d3109a5b0c4779d7e7d8059c9d78.tar.gz 00:00:15.319 Response Code: HTTP/1.1 200 OK 00:00:15.319 Success: Status code 200 is in the accepted range: 200,404 00:00:15.320 Saving response body to /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dpdk_7e06c0de1952d3109a5b0c4779d7e7d8059c9d78.tar.gz 00:00:16.397 [Pipeline] sh 00:00:16.679 + tar --no-same-owner -xf dpdk_7e06c0de1952d3109a5b0c4779d7e7d8059c9d78.tar.gz 00:00:17.627 [Pipeline] sh 00:00:17.907 + git -C dpdk log --oneline -n5 00:00:17.907 7e06c0de19 examples: move alignment attribute on types for MSVC 00:00:17.907 27595cd830 drivers: move alignment attribute on types for MSVC 00:00:17.907 0efea35a2b app: move alignment attribute on types for MSVC 00:00:17.907 e2e546ab5b version: 24.07-rc0 00:00:17.907 a9778aad62 version: 24.03.0 00:00:17.920 [Pipeline] catchError 00:00:17.922 [Pipeline] { 00:00:17.942 [Pipeline] readJSON 00:00:17.956 [Pipeline] } 00:00:17.981 [Pipeline] // catchError 00:00:17.986 [Pipeline] } 00:00:18.011 [Pipeline] // stage 00:00:18.030 [Pipeline] catchError 00:00:18.031 [Pipeline] { 00:00:18.052 [Pipeline] stage 00:00:18.054 [Pipeline] { (Tests) 00:00:18.265 [Pipeline] parallel 00:00:18.361 [Pipeline] { (Branch: unittest-vg-autotest) 00:00:18.363 [Pipeline] { (Branch: valgrind-vg-autotest) 00:00:18.364 [Pipeline] { (Branch: autobuild-vg-autotest) 00:00:18.365 [Pipeline] { (Branch: scanbuild-vg-autotest) 00:00:18.366 [Pipeline] { (Branch: lvol-vg-autotest) 00:00:18.368 [Pipeline] { (Branch: vhost-initiator-vg-autotest) 00:00:18.369 [Pipeline] { (Branch: nvmf-tcp-vg-autotest) 00:00:18.370 [Pipeline] { (Branch: nvmf-tcp-uring-vg-autotest) 00:00:18.371 [Pipeline] { (Branch: nvme-vg-autotest) 00:00:18.373 [Pipeline] { (Branch: centos7-vg-autotest) 00:00:18.374 [Pipeline] { (Branch: rocky9-vg-autotest) 00:00:18.375 [Pipeline] { (Branch: ubuntu20-vg-autotest) 00:00:18.376 [Pipeline] { (Branch: ubuntu22-vg-autotest) 00:00:18.377 [Pipeline] { (Branch: iscsi-vg-autotest) 00:00:18.379 [Pipeline] { (Branch: iscsi-uring-vg-autotest) 00:00:18.380 [Pipeline] { (Branch: zns-vg-autotest) 00:00:18.381 [Pipeline] { (Branch: clang-vg-autotest) 00:00:18.382 [Pipeline] { (Branch: nvme-cmb-pmr-vg-autotest) 00:00:18.383 [Pipeline] { (Branch: BlobFS-phy-autotest) 00:00:18.385 [Pipeline] { (Branch: crypto-phy-autotest) 00:00:18.386 [Pipeline] { (Branch: vhost-phy-autotest) 00:00:18.387 [Pipeline] { (Branch: nvmf-phy-autotest) 00:00:18.388 [Pipeline] { (Branch: nvmf-tcp-phy-autotest) 00:00:18.389 [Pipeline] { (Branch: nvme-phy-autotest) 00:00:18.390 [Pipeline] { (Branch: vfio-user-phy-autotest) 00:00:18.391 [Pipeline] { (Branch: short-fuzz-phy-autotest) 00:00:18.393 [Pipeline] { (Branch: dsa-phy-autotest) 00:00:18.394 [Pipeline] { (Branch: vmd-phy-autotest) 00:00:18.429 [Pipeline] retry 00:00:18.431 [Pipeline] { 00:00:18.436 [Pipeline] retry 00:00:18.438 [Pipeline] { 00:00:18.443 [Pipeline] retry 00:00:18.444 [Pipeline] { 00:00:18.449 [Pipeline] retry 00:00:18.451 [Pipeline] { 00:00:18.456 [Pipeline] retry 00:00:18.458 [Pipeline] { 00:00:18.462 [Pipeline] retry 00:00:18.464 [Pipeline] { 00:00:18.469 [Pipeline] retry 00:00:18.471 [Pipeline] { 00:00:18.475 [Pipeline] retry 00:00:18.477 [Pipeline] { 00:00:18.482 [Pipeline] retry 00:00:18.484 [Pipeline] { 00:00:18.488 [Pipeline] retry 00:00:18.490 [Pipeline] { 00:00:18.495 [Pipeline] retry 00:00:18.497 [Pipeline] { 00:00:18.502 [Pipeline] retry 00:00:18.503 [Pipeline] { 00:00:18.508 [Pipeline] retry 00:00:18.510 [Pipeline] { 00:00:18.515 [Pipeline] retry 00:00:18.517 [Pipeline] { 00:00:18.521 [Pipeline] retry 00:00:18.523 [Pipeline] { 00:00:18.527 [Pipeline] retry 00:00:18.529 [Pipeline] { 00:00:18.533 [Pipeline] retry 00:00:18.535 [Pipeline] { 00:00:18.540 [Pipeline] retry 00:00:18.541 [Pipeline] { 00:00:18.546 [Pipeline] retry 00:00:18.548 [Pipeline] { 00:00:18.553 [Pipeline] retry 00:00:18.554 [Pipeline] { 00:00:18.559 [Pipeline] retry 00:00:18.561 [Pipeline] { 00:00:18.565 [Pipeline] retry 00:00:18.567 [Pipeline] { 00:00:18.571 [Pipeline] retry 00:00:18.573 [Pipeline] { 00:00:18.578 [Pipeline] retry 00:00:18.580 [Pipeline] { 00:00:18.584 [Pipeline] retry 00:00:18.586 [Pipeline] { 00:00:18.590 [Pipeline] retry 00:00:18.592 [Pipeline] { 00:00:18.597 [Pipeline] retry 00:00:18.598 [Pipeline] { 00:00:18.603 [Pipeline] retry 00:00:18.605 [Pipeline] { 00:00:18.631 [Pipeline] build 00:00:18.633 Scheduling project: unittest-vg-autotest 00:00:18.640 [Pipeline] build 00:00:18.642 Scheduling project: valgrind-vg-autotest 00:00:18.649 [Pipeline] build 00:00:18.652 Scheduling project: autobuild-vg-autotest 00:00:18.659 [Pipeline] build 00:00:18.662 Scheduling project: scanbuild-vg-autotest 00:00:18.668 [Pipeline] build 00:00:18.670 Scheduling project: lvol-vg-autotest 00:00:18.676 [Pipeline] build 00:00:18.678 Scheduling project: vhost-initiator-vg-autotest 00:00:18.684 [Pipeline] build 00:00:18.687 Scheduling project: nvmf-tcp-vg-autotest 00:00:18.693 [Pipeline] build 00:00:18.695 Scheduling project: nvmf-tcp-uring-vg-autotest 00:00:18.702 [Pipeline] build 00:00:18.705 Scheduling project: nvme-vg-autotest 00:00:18.712 [Pipeline] build 00:00:18.714 Scheduling project: centos7-vg-autotest 00:00:18.721 [Pipeline] build 00:00:18.723 Scheduling project: rocky9-vg-autotest 00:00:18.729 [Pipeline] build 00:00:18.732 Scheduling project: ubuntu20-vg-autotest 00:00:18.737 [Pipeline] build 00:00:18.740 Scheduling project: ubuntu22-vg-autotest 00:00:18.745 [Pipeline] build 00:00:18.748 Scheduling project: iscsi-vg-autotest 00:00:18.753 [Pipeline] build 00:00:18.756 Scheduling project: iscsi-uring-vg-autotest 00:00:18.761 [Pipeline] build 00:00:18.763 Scheduling project: zns-vg-autotest 00:00:18.769 [Pipeline] build 00:00:18.771 Scheduling project: clang-vg-autotest 00:00:18.776 [Pipeline] build 00:00:18.778 Scheduling project: nvme-cmb-pmr-vg-autotest 00:00:18.784 [Pipeline] build 00:00:18.786 Scheduling project: BlobFS-phy-autotest 00:00:18.791 [Pipeline] build 00:00:18.793 Scheduling project: crypto-phy-autotest 00:00:18.798 [Pipeline] build 00:00:18.800 Scheduling project: vhost-phy-autotest 00:00:18.806 [Pipeline] build 00:00:18.808 Scheduling project: nvmf-phy-autotest 00:00:18.813 [Pipeline] build 00:00:18.815 Scheduling project: nvmf-tcp-phy-autotest 00:00:18.820 [Pipeline] build 00:00:18.822 Scheduling project: nvme-phy-autotest 00:00:18.827 [Pipeline] build 00:00:18.830 Scheduling project: vfio-user-phy-autotest 00:00:18.835 [Pipeline] build 00:00:18.837 Scheduling project: short-fuzz-phy-autotest 00:00:18.842 [Pipeline] build 00:00:18.844 Scheduling project: dsa-phy-autotest 00:00:18.850 [Pipeline] build 00:00:18.852 Scheduling project: vmd-phy-autotest 00:00:25.059 Starting building: BlobFS-phy-autotest #26186 00:00:25.062 Starting building: vfio-user-phy-autotest #26188 00:00:25.064 Starting building: vmd-phy-autotest #2618 00:00:25.066 Starting building: rocky9-vg-autotest #16070 00:00:25.068 Starting building: dsa-phy-autotest #4880 00:00:25.071 Starting building: vhost-phy-autotest #26205 00:00:25.074 Starting building: short-fuzz-phy-autotest #26177 00:00:25.076 Starting building: ubuntu22-vg-autotest #37793 00:00:25.079 Starting building: valgrind-vg-autotest #49190 00:00:25.081 Starting building: nvmf-tcp-vg-autotest #77451 00:00:25.083 Starting building: lvol-vg-autotest #77409 00:00:25.086 Starting building: unittest-vg-autotest #63373 00:00:25.090 Starting building: zns-vg-autotest #76566 00:00:25.092 Starting building: crypto-phy-autotest #26180 00:00:25.097 Starting building: iscsi-uring-vg-autotest #77365 00:00:25.100 Starting building: nvmf-tcp-phy-autotest #72311 00:00:25.102 Starting building: centos7-vg-autotest #73639 00:00:25.110 Starting building: iscsi-vg-autotest #77498 00:00:25.115 Starting building: scanbuild-vg-autotest #63375 00:00:25.121 Starting building: nvme-cmb-pmr-vg-autotest #78583 00:00:25.123 Starting building: ubuntu20-vg-autotest #73779 00:00:25.126 Starting building: vhost-initiator-vg-autotest #77480 00:00:25.133 Starting building: nvme-vg-autotest #77460 00:00:25.140 Starting building: nvmf-tcp-uring-vg-autotest #77385 00:00:25.143 Starting building: autobuild-vg-autotest #63416 00:00:25.146 Starting building: clang-vg-autotest #57624 00:00:25.149 Starting building: nvmf-phy-autotest #72424 00:00:25.150 Starting building: nvme-phy-autotest #72352 00:09:25.777 Build short-fuzz-phy-autotest #26177 completed: SUCCESS 00:09:25.780 [Pipeline] } 00:09:25.826 [Pipeline] // retry 00:09:25.831 [Pipeline] } 00:10:30.762 Build vmd-phy-autotest #2618 completed: SUCCESS 00:10:30.765 [Pipeline] } 00:10:30.812 [Pipeline] // retry 00:10:30.818 [Pipeline] } 00:10:45.675 Build vhost-initiator-vg-autotest #77480 completed: SUCCESS 00:10:45.679 [Pipeline] } 00:10:45.725 [Pipeline] // retry 00:10:45.731 [Pipeline] } 00:11:16.777 Build dsa-phy-autotest #4880 completed: SUCCESS 00:11:16.780 [Pipeline] } 00:11:16.825 [Pipeline] // retry 00:11:16.831 [Pipeline] } 00:12:02.256 Build zns-vg-autotest #76566 completed: SUCCESS 00:12:02.260 [Pipeline] } 00:12:02.311 [Pipeline] // retry 00:12:02.320 [Pipeline] } 00:12:19.395 Build lvol-vg-autotest #77409 completed: SUCCESS 00:12:19.398 [Pipeline] } 00:12:19.442 [Pipeline] // retry 00:12:19.448 [Pipeline] } 00:13:29.055 Build nvme-vg-autotest #77460 completed: FAILURE 00:13:29.818 [Pipeline] echo 00:13:29.819 No retry patterns found. 00:13:29.820 [Pipeline] } 00:13:29.853 [Pipeline] // retry 00:13:29.860 [Pipeline] error 00:13:29.864 [Pipeline] } 00:13:29.867 Failed in branch nvme-vg-autotest 00:13:33.596 Build scanbuild-vg-autotest #63375 completed: SUCCESS 00:13:33.600 [Pipeline] } 00:13:33.642 [Pipeline] // retry 00:13:33.648 [Pipeline] } 00:14:05.455 Build nvme-cmb-pmr-vg-autotest #78583 completed: SUCCESS 00:14:05.458 [Pipeline] } 00:14:05.495 [Pipeline] // retry 00:14:05.502 [Pipeline] } 00:21:41.879 Build vfio-user-phy-autotest #26188 completed: SUCCESS 00:21:41.882 [Pipeline] } 00:21:41.928 [Pipeline] // retry 00:21:41.935 [Pipeline] } 00:22:10.999 Build BlobFS-phy-autotest #26186 completed: SUCCESS 00:22:11.002 [Pipeline] } 00:22:11.047 [Pipeline] // retry 00:22:11.053 [Pipeline] } 00:23:44.244 Build nvmf-tcp-uring-vg-autotest #77385 completed: FAILURE 00:23:45.426 [Pipeline] echo 00:23:45.427 No retry patterns found. 00:23:45.427 [Pipeline] } 00:23:45.459 [Pipeline] // retry 00:23:45.464 [Pipeline] error 00:23:45.469 [Pipeline] } 00:23:45.473 Failed in branch nvmf-tcp-uring-vg-autotest 00:24:05.190 Build autobuild-vg-autotest #63416 completed: SUCCESS 00:24:05.193 [Pipeline] } 00:24:05.238 [Pipeline] // retry 00:24:05.244 [Pipeline] } 00:24:34.822 Build nvme-phy-autotest #72352 completed: SUCCESS 00:24:34.825 [Pipeline] } 00:24:34.871 [Pipeline] // retry 00:24:34.877 [Pipeline] } 00:25:38.131 Build unittest-vg-autotest #63373 completed: SUCCESS 00:25:38.134 [Pipeline] } 00:25:38.182 [Pipeline] // retry 00:25:38.188 [Pipeline] } 00:28:10.941 Build iscsi-uring-vg-autotest #77365 completed: SUCCESS 00:28:10.944 [Pipeline] } 00:28:10.992 [Pipeline] // retry 00:28:11.000 [Pipeline] } 00:28:27.949 Build centos7-vg-autotest #73639 completed: SUCCESS 00:28:27.953 [Pipeline] } 00:28:27.998 [Pipeline] // retry 00:28:28.004 [Pipeline] } 00:29:12.530 Build vhost-phy-autotest #26205 completed: SUCCESS 00:29:12.533 [Pipeline] } 00:29:12.576 [Pipeline] // retry 00:29:12.582 [Pipeline] } 00:31:48.709 Build clang-vg-autotest #57624 completed: SUCCESS 00:31:48.713 [Pipeline] } 00:31:48.755 [Pipeline] // retry 00:31:48.761 [Pipeline] } 00:31:53.754 Build nvmf-tcp-vg-autotest #77451 completed: SUCCESS 00:31:53.758 [Pipeline] } 00:31:53.801 [Pipeline] // retry 00:31:53.808 [Pipeline] } 00:32:02.827 Build iscsi-vg-autotest #77498 completed: SUCCESS 00:32:02.830 [Pipeline] } 00:32:02.869 [Pipeline] // retry 00:32:02.874 [Pipeline] } 00:32:14.729 Build nvmf-phy-autotest #72424 completed: FAILURE 00:32:15.959 [Pipeline] echo 00:32:15.960 No retry patterns found. 00:32:15.961 [Pipeline] } 00:32:15.993 [Pipeline] // retry 00:32:15.999 [Pipeline] error 00:32:16.005 [Pipeline] } 00:32:16.009 Failed in branch nvmf-phy-autotest 00:34:18.132 Build crypto-phy-autotest #26180 completed: SUCCESS 00:34:18.134 [Pipeline] } 00:34:18.180 [Pipeline] // retry 00:34:18.186 [Pipeline] } 00:39:00.600 Build nvmf-tcp-phy-autotest #72311 completed: SUCCESS 00:39:00.604 [Pipeline] } 00:39:00.646 [Pipeline] // retry 00:39:00.652 [Pipeline] } 00:41:37.506 Build valgrind-vg-autotest #49190 completed: SUCCESS 00:41:37.508 [Pipeline] } 00:41:37.542 [Pipeline] // retry 00:41:37.548 [Pipeline] } 00:41:41.635 Build ubuntu20-vg-autotest #73779 completed: FAILURE 00:41:43.749 [Pipeline] echo 00:41:43.750 No retry patterns found. 00:41:43.751 [Pipeline] } 00:41:43.772 [Pipeline] // retry 00:41:43.778 [Pipeline] error 00:41:43.784 [Pipeline] } 00:41:43.790 Failed in branch ubuntu20-vg-autotest 00:43:28.710 Build rocky9-vg-autotest #16070 completed: SUCCESS 00:43:28.714 [Pipeline] } 00:43:28.758 [Pipeline] // retry 00:43:28.763 [Pipeline] } 00:51:32.089 Build ubuntu22-vg-autotest #37793 completed: SUCCESS 00:51:32.091 [Pipeline] } 00:51:32.144 [Pipeline] // retry 00:51:32.150 [Pipeline] } 00:51:32.410 [Pipeline] // parallel 00:51:32.419 [Pipeline] } 00:51:32.456 [Pipeline] // stage 00:51:32.465 [Pipeline] } 00:51:32.475 ERROR: Build nvme-vg-autotest #77460 failed 00:51:32.504 [Pipeline] // catchError 00:51:32.513 [Pipeline] stage 00:51:32.516 [Pipeline] { (Autorun Post and Coverage) 00:51:32.547 [Pipeline] setCustomBuildProperty 00:51:33.005 [Pipeline] dir 00:51:33.005 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/short-fuzz-phy-autotest_26177 00:51:33.006 [Pipeline] { 00:51:33.056 [Pipeline] copyArtifacts 00:51:33.171 Copied 10 artifacts from "short-fuzz-phy-autotest" build number 26177 00:51:33.175 [Pipeline] writeFile 00:51:33.373 [Pipeline] } 00:51:33.509 [Pipeline] // dir 00:51:33.815 [Pipeline] dir 00:51:33.815 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vmd-phy-autotest_2618 00:51:33.816 [Pipeline] { 00:51:33.849 [Pipeline] copyArtifacts 00:51:34.207 Copied 12 artifacts from "vmd-phy-autotest" build number 2618 00:51:34.211 [Pipeline] writeFile 00:51:34.341 [Pipeline] } 00:51:34.414 [Pipeline] // dir 00:51:34.739 [Pipeline] dir 00:51:34.739 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-initiator-vg-autotest_77480 00:51:34.740 [Pipeline] { 00:51:34.780 [Pipeline] copyArtifacts 00:51:35.035 Copied 15 artifacts from "vhost-initiator-vg-autotest" build number 77480 00:51:35.039 [Pipeline] writeFile 00:51:35.165 [Pipeline] } 00:51:35.233 [Pipeline] // dir 00:51:35.532 [Pipeline] dir 00:51:35.532 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dsa-phy-autotest_4880 00:51:35.533 [Pipeline] { 00:51:35.571 [Pipeline] copyArtifacts 00:51:35.799 Copied 12 artifacts from "dsa-phy-autotest" build number 4880 00:51:35.804 [Pipeline] writeFile 00:51:35.932 [Pipeline] } 00:51:36.029 [Pipeline] // dir 00:51:36.422 [Pipeline] dir 00:51:36.423 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/zns-vg-autotest_76566 00:51:36.424 [Pipeline] { 00:51:36.463 [Pipeline] copyArtifacts 00:51:36.657 Copied 12 artifacts from "zns-vg-autotest" build number 76566 00:51:36.662 [Pipeline] writeFile 00:51:36.806 [Pipeline] } 00:51:36.922 [Pipeline] // dir 00:51:37.520 [Pipeline] dir 00:51:37.520 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/lvol-vg-autotest_77409 00:51:37.522 [Pipeline] { 00:51:37.585 [Pipeline] copyArtifacts 00:51:37.781 Copied 12 artifacts from "lvol-vg-autotest" build number 77409 00:51:37.786 [Pipeline] writeFile 00:51:38.005 [Pipeline] } 00:51:38.125 [Pipeline] // dir 00:51:38.620 [Pipeline] dir 00:51:38.620 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-vg-autotest_77460 00:51:38.621 [Pipeline] { 00:51:38.672 [Pipeline] copyArtifacts 00:51:38.858 Copied 11 artifacts from "nvme-vg-autotest" build number 77460 00:51:38.863 [Pipeline] writeFile 00:51:39.021 [Pipeline] } 00:51:39.136 [Pipeline] // dir 00:51:39.236 [Pipeline] dir 00:51:39.236 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/scanbuild-vg-autotest_63375 00:51:39.237 [Pipeline] { 00:51:39.264 [Pipeline] copyArtifacts 00:51:39.404 Copied 13 artifacts from "scanbuild-vg-autotest" build number 63375 00:51:39.409 [Pipeline] writeFile 00:51:39.467 [Pipeline] } 00:51:39.507 [Pipeline] // dir 00:51:39.893 [Pipeline] dir 00:51:39.894 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-cmb-pmr-vg-autotest_78583 00:51:39.895 [Pipeline] { 00:51:39.938 [Pipeline] copyArtifacts 00:51:40.135 Copied 12 artifacts from "nvme-cmb-pmr-vg-autotest" build number 78583 00:51:40.140 [Pipeline] writeFile 00:51:40.274 [Pipeline] } 00:51:40.365 [Pipeline] // dir 00:51:40.864 [Pipeline] dir 00:51:40.865 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vfio-user-phy-autotest_26188 00:51:40.866 [Pipeline] { 00:51:40.924 [Pipeline] copyArtifacts 00:51:41.133 Copied 12 artifacts from "vfio-user-phy-autotest" build number 26188 00:51:41.138 [Pipeline] writeFile 00:51:41.351 [Pipeline] } 00:51:41.499 [Pipeline] // dir 00:51:41.803 [Pipeline] dir 00:51:41.804 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/BlobFS-phy-autotest_26186 00:51:41.805 [Pipeline] { 00:51:41.845 [Pipeline] copyArtifacts 00:51:42.218 Copied 28 artifacts from "BlobFS-phy-autotest" build number 26186 00:51:42.222 [Pipeline] writeFile 00:51:42.361 [Pipeline] } 00:51:42.444 [Pipeline] // dir 00:51:43.355 [Pipeline] dir 00:51:43.355 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-uring-vg-autotest_77385 00:51:43.356 [Pipeline] { 00:51:43.429 [Pipeline] copyArtifacts 00:51:43.874 Copied 13 artifacts from "nvmf-tcp-uring-vg-autotest" build number 77385 00:51:43.878 [Pipeline] writeFile 00:51:44.302 [Pipeline] } 00:51:44.534 [Pipeline] // dir 00:51:44.749 [Pipeline] dir 00:51:44.750 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/autobuild-vg-autotest_63416 00:51:44.751 [Pipeline] { 00:51:44.780 [Pipeline] copyArtifacts 00:51:45.085 Copied 9 artifacts from "autobuild-vg-autotest" build number 63416 00:51:45.090 [Pipeline] writeFile 00:51:45.155 [Pipeline] } 00:51:45.234 [Pipeline] // dir 00:51:46.347 [Pipeline] dir 00:51:46.347 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-phy-autotest_72352 00:51:46.349 [Pipeline] { 00:51:46.401 [Pipeline] copyArtifacts 00:51:46.607 Copied 12 artifacts from "nvme-phy-autotest" build number 72352 00:51:46.611 [Pipeline] writeFile 00:51:46.902 [Pipeline] } 00:51:47.023 [Pipeline] // dir 00:51:47.397 [Pipeline] dir 00:51:47.397 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/unittest-vg-autotest_63373 00:51:47.398 [Pipeline] { 00:51:47.443 [Pipeline] copyArtifacts 00:51:47.827 Copied 14 artifacts from "unittest-vg-autotest" build number 63373 00:51:47.832 [Pipeline] writeFile 00:51:47.950 [Pipeline] } 00:51:48.049 [Pipeline] // dir 00:51:50.216 [Pipeline] dir 00:51:50.217 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-uring-vg-autotest_77365 00:51:50.218 [Pipeline] { 00:51:50.415 [Pipeline] copyArtifacts 00:51:53.485 Copied 17 artifacts from "iscsi-uring-vg-autotest" build number 77365 00:51:53.489 [Pipeline] writeFile 00:51:54.805 [Pipeline] } 00:51:55.330 [Pipeline] // dir 00:51:56.279 [Pipeline] dir 00:51:56.279 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/centos7-vg-autotest_73639 00:51:56.281 [Pipeline] { 00:51:56.359 [Pipeline] copyArtifacts 00:51:56.739 Copied 13 artifacts from "centos7-vg-autotest" build number 73639 00:51:56.744 [Pipeline] writeFile 00:51:57.079 [Pipeline] } 00:51:57.238 [Pipeline] // dir 00:51:57.908 [Pipeline] dir 00:51:57.909 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-phy-autotest_26205 00:51:57.910 [Pipeline] { 00:51:57.972 [Pipeline] copyArtifacts 00:51:58.210 Copied 13 artifacts from "vhost-phy-autotest" build number 26205 00:51:58.214 [Pipeline] writeFile 00:51:58.441 [Pipeline] } 00:51:58.607 [Pipeline] // dir 00:51:58.891 [Pipeline] dir 00:51:58.892 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/clang-vg-autotest_57624 00:51:58.893 [Pipeline] { 00:51:58.934 [Pipeline] copyArtifacts 00:51:59.040 Copied 9 artifacts from "clang-vg-autotest" build number 57624 00:51:59.045 [Pipeline] writeFile 00:51:59.208 [Pipeline] } 00:51:59.286 [Pipeline] // dir 00:52:00.141 [Pipeline] dir 00:52:00.141 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-vg-autotest_77451 00:52:00.142 [Pipeline] { 00:52:00.208 [Pipeline] copyArtifacts 00:52:00.447 Copied 14 artifacts from "nvmf-tcp-vg-autotest" build number 77451 00:52:00.452 [Pipeline] writeFile 00:52:00.836 [Pipeline] } 00:52:01.037 [Pipeline] // dir 00:52:02.931 [Pipeline] dir 00:52:02.931 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-vg-autotest_77498 00:52:02.932 [Pipeline] { 00:52:03.101 [Pipeline] copyArtifacts 00:52:05.813 Copied 16 artifacts from "iscsi-vg-autotest" build number 77498 00:52:05.818 [Pipeline] writeFile 00:52:06.730 [Pipeline] } 00:52:07.191 [Pipeline] // dir 00:52:07.992 [Pipeline] dir 00:52:07.992 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-phy-autotest_72424 00:52:07.993 [Pipeline] { 00:52:08.070 [Pipeline] copyArtifacts 00:52:09.405 Copied 18 artifacts from "nvmf-phy-autotest" build number 72424 00:52:09.410 [Pipeline] writeFile 00:52:09.728 [Pipeline] } 00:52:09.955 [Pipeline] // dir 00:52:11.620 [Pipeline] dir 00:52:11.620 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/crypto-phy-autotest_26180 00:52:11.622 [Pipeline] { 00:52:11.755 [Pipeline] copyArtifacts 00:52:11.962 Copied 12 artifacts from "crypto-phy-autotest" build number 26180 00:52:11.966 [Pipeline] writeFile 00:52:12.582 [Pipeline] } 00:52:12.868 [Pipeline] // dir 00:52:14.119 [Pipeline] dir 00:52:14.119 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-phy-autotest_72311 00:52:14.120 [Pipeline] { 00:52:14.216 [Pipeline] copyArtifacts 00:52:14.522 Copied 14 artifacts from "nvmf-tcp-phy-autotest" build number 72311 00:52:14.530 [Pipeline] writeFile 00:52:15.064 [Pipeline] } 00:52:15.309 [Pipeline] // dir 00:52:16.069 [Pipeline] dir 00:52:16.070 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/valgrind-vg-autotest_49190 00:52:16.071 [Pipeline] { 00:52:16.136 [Pipeline] copyArtifacts 00:52:16.532 Copied 13 artifacts from "valgrind-vg-autotest" build number 49190 00:52:16.534 [Pipeline] writeFile 00:52:16.975 [Pipeline] } 00:52:17.172 [Pipeline] // dir 00:52:18.419 [Pipeline] dir 00:52:18.420 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu20-vg-autotest_73779 00:52:18.421 [Pipeline] { 00:52:18.505 [Pipeline] copyArtifacts 00:52:19.134 Copied 12 artifacts from "ubuntu20-vg-autotest" build number 73779 00:52:19.138 [Pipeline] writeFile 00:52:19.616 [Pipeline] } 00:52:19.879 [Pipeline] // dir 00:52:21.259 [Pipeline] dir 00:52:21.260 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/rocky9-vg-autotest_16070 00:52:21.260 [Pipeline] { 00:52:21.351 [Pipeline] copyArtifacts 00:52:21.711 Copied 13 artifacts from "rocky9-vg-autotest" build number 16070 00:52:21.715 [Pipeline] writeFile 00:52:22.223 [Pipeline] } 00:52:22.470 [Pipeline] // dir 00:52:23.878 [Pipeline] dir 00:52:23.879 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu22-vg-autotest_37793 00:52:23.880 [Pipeline] { 00:52:23.967 [Pipeline] copyArtifacts 00:52:24.316 Copied 13 artifacts from "ubuntu22-vg-autotest" build number 37793 00:52:24.321 [Pipeline] writeFile 00:52:24.777 [Pipeline] } 00:52:25.036 [Pipeline] // dir 00:52:25.047 [Pipeline] catchError 00:52:25.049 [Pipeline] { 00:52:25.074 [Pipeline] sh 00:52:25.351 + jbp/jenkins/jjb-config/jobs/scripts/post_gen_coverage.sh 00:52:25.351 + shopt -s globstar nullglob 00:52:25.351 + echo 'Start stage post_gen_coverage.sh' 00:52:25.351 Start stage post_gen_coverage.sh 00:52:25.351 + cd /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main 00:52:25.351 + rm -rf /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/doc 00:52:25.351 + trap 'compress_coverage_and_docs; remove_partial_coverage_files && echo '\''End stage post_gen_coverage.sh'\''' EXIT 00:52:25.351 + move_artifacts 00:52:25.351 + local out_dirs 00:52:25.351 + out_dirs=(./**/output/) 00:52:25.351 + for dir in "${out_dirs[@]}" 00:52:25.351 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.351 + [[ -f ./autobuild-vg-autotest_63416/output//doc.tar.xz ]] 00:52:25.351 + tar -C ./autobuild-vg-autotest_63416/output/ -xf ./autobuild-vg-autotest_63416/output//doc.tar.xz 00:52:25.608 + rm ./autobuild-vg-autotest_63416/output//doc.tar.xz 00:52:25.608 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.608 + [[ -f ./autobuild-vg-autotest_63416/output//ut_coverage.tar.xz ]] 00:52:25.608 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.608 + [[ -f ./autobuild-vg-autotest_63416/output//llvm.tar.xz ]] 00:52:25.609 + mv ./autobuild-vg-autotest_63416/output//build-pkg-manifest.txt ./autobuild-vg-autotest_63416/output//build-repo-manifest.txt ./autobuild-vg-autotest_63416/output//dmesg.init.log ./autobuild-vg-autotest_63416/output//dmesg.log ./autobuild-vg-autotest_63416/output//doc ./autobuild-vg-autotest_63416/output//power.tar.xz ./autobuild-vg-autotest_63416/output//test_completions.txt ./autobuild-vg-autotest_63416/output//timing.svg ./autobuild-vg-autotest_63416/output//timing.txt ./autobuild-vg-autotest_63416/output//.. 00:52:25.609 + rmdir ./autobuild-vg-autotest_63416/output/ 00:52:25.609 + for dir in "${out_dirs[@]}" 00:52:25.609 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.609 + [[ -f ./BlobFS-phy-autotest_26186/output//doc.tar.xz ]] 00:52:25.609 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.609 + [[ -f ./BlobFS-phy-autotest_26186/output//ut_coverage.tar.xz ]] 00:52:25.609 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.609 + [[ -f ./BlobFS-phy-autotest_26186/output//llvm.tar.xz ]] 00:52:25.609 + mv ./BlobFS-phy-autotest_26186/output//all_tests.txt ./BlobFS-phy-autotest_26186/output//build-pkg-manifest.txt ./BlobFS-phy-autotest_26186/output//build-repo-manifest.txt ./BlobFS-phy-autotest_26186/output//cov_base.info.xz ./BlobFS-phy-autotest_26186/output//cov_test.info.xz ./BlobFS-phy-autotest_26186/output//cov_total.info.xz ./BlobFS-phy-autotest_26186/output//dmesg.init.log ./BlobFS-phy-autotest_26186/output//dmesg.log ./BlobFS-phy-autotest_26186/output//power.tar.xz ./BlobFS-phy-autotest_26186/output//rocksdb ./BlobFS-phy-autotest_26186/output//test_completions.txt ./BlobFS-phy-autotest_26186/output//timing.svg ./BlobFS-phy-autotest_26186/output//timing.txt ./BlobFS-phy-autotest_26186/output//.. 00:52:25.609 + rmdir ./BlobFS-phy-autotest_26186/output/ 00:52:25.609 + for dir in "${out_dirs[@]}" 00:52:25.609 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.609 + [[ -f ./centos7-vg-autotest_73639/output//doc.tar.xz ]] 00:52:25.609 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.609 + [[ -f ./centos7-vg-autotest_73639/output//ut_coverage.tar.xz ]] 00:52:25.609 + tar -C ./centos7-vg-autotest_73639/output/ -xf ./centos7-vg-autotest_73639/output//ut_coverage.tar.xz 00:52:25.868 + rm ./centos7-vg-autotest_73639/output//ut_coverage.tar.xz 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./centos7-vg-autotest_73639/output//llvm.tar.xz ]] 00:52:25.868 + mv ./centos7-vg-autotest_73639/output//all_tests.txt ./centos7-vg-autotest_73639/output//build-pkg-manifest.txt ./centos7-vg-autotest_73639/output//build-repo-manifest.txt ./centos7-vg-autotest_73639/output//cov_base.info.xz ./centos7-vg-autotest_73639/output//cov_test.info.xz ./centos7-vg-autotest_73639/output//cov_total.info.xz ./centos7-vg-autotest_73639/output//dmesg.init.log ./centos7-vg-autotest_73639/output//dmesg.log ./centos7-vg-autotest_73639/output//power.tar.xz ./centos7-vg-autotest_73639/output//test_completions.txt ./centos7-vg-autotest_73639/output//timing.svg ./centos7-vg-autotest_73639/output//timing.txt ./centos7-vg-autotest_73639/output//ut_coverage ./centos7-vg-autotest_73639/output//.. 00:52:25.868 + rmdir ./centos7-vg-autotest_73639/output/ 00:52:25.868 + for dir in "${out_dirs[@]}" 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./clang-vg-autotest_57624/output//doc.tar.xz ]] 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./clang-vg-autotest_57624/output//ut_coverage.tar.xz ]] 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./clang-vg-autotest_57624/output//llvm.tar.xz ]] 00:52:25.868 + mv ./clang-vg-autotest_57624/output//all_tests.txt ./clang-vg-autotest_57624/output//build-pkg-manifest.txt ./clang-vg-autotest_57624/output//build-repo-manifest.txt ./clang-vg-autotest_57624/output//dmesg.init.log ./clang-vg-autotest_57624/output//dmesg.log ./clang-vg-autotest_57624/output//power.tar.xz ./clang-vg-autotest_57624/output//test_completions.txt ./clang-vg-autotest_57624/output//timing.svg ./clang-vg-autotest_57624/output//timing.txt ./clang-vg-autotest_57624/output//.. 00:52:25.868 + rmdir ./clang-vg-autotest_57624/output/ 00:52:25.868 + for dir in "${out_dirs[@]}" 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./crypto-phy-autotest_26180/output//doc.tar.xz ]] 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./crypto-phy-autotest_26180/output//ut_coverage.tar.xz ]] 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./crypto-phy-autotest_26180/output//llvm.tar.xz ]] 00:52:25.868 + mv ./crypto-phy-autotest_26180/output//all_tests.txt ./crypto-phy-autotest_26180/output//build-pkg-manifest.txt ./crypto-phy-autotest_26180/output//build-repo-manifest.txt ./crypto-phy-autotest_26180/output//cov_base.info.xz ./crypto-phy-autotest_26180/output//cov_test.info.xz ./crypto-phy-autotest_26180/output//cov_total.info.xz ./crypto-phy-autotest_26180/output//dmesg.init.log ./crypto-phy-autotest_26180/output//dmesg.log ./crypto-phy-autotest_26180/output//power.tar.xz ./crypto-phy-autotest_26180/output//test_completions.txt ./crypto-phy-autotest_26180/output//timing.svg ./crypto-phy-autotest_26180/output//timing.txt ./crypto-phy-autotest_26180/output//.. 00:52:25.868 + rmdir ./crypto-phy-autotest_26180/output/ 00:52:25.868 + for dir in "${out_dirs[@]}" 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./dsa-phy-autotest_4880/output//doc.tar.xz ]] 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./dsa-phy-autotest_4880/output//ut_coverage.tar.xz ]] 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./dsa-phy-autotest_4880/output//llvm.tar.xz ]] 00:52:25.868 + mv ./dsa-phy-autotest_4880/output//all_tests.txt ./dsa-phy-autotest_4880/output//build-pkg-manifest.txt ./dsa-phy-autotest_4880/output//build-repo-manifest.txt ./dsa-phy-autotest_4880/output//cov_base.info.xz ./dsa-phy-autotest_4880/output//cov_test.info.xz ./dsa-phy-autotest_4880/output//cov_total.info.xz ./dsa-phy-autotest_4880/output//dmesg.init.log ./dsa-phy-autotest_4880/output//dmesg.log ./dsa-phy-autotest_4880/output//power.tar.xz ./dsa-phy-autotest_4880/output//test_completions.txt ./dsa-phy-autotest_4880/output//timing.svg ./dsa-phy-autotest_4880/output//timing.txt ./dsa-phy-autotest_4880/output//.. 00:52:25.868 + rmdir ./dsa-phy-autotest_4880/output/ 00:52:25.868 + for dir in "${out_dirs[@]}" 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./iscsi-uring-vg-autotest_77365/output//doc.tar.xz ]] 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./iscsi-uring-vg-autotest_77365/output//ut_coverage.tar.xz ]] 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./iscsi-uring-vg-autotest_77365/output//llvm.tar.xz ]] 00:52:25.868 + mv ./iscsi-uring-vg-autotest_77365/output//all_tests.txt ./iscsi-uring-vg-autotest_77365/output//build-kernel-manifest.txt ./iscsi-uring-vg-autotest_77365/output//build-pkg-manifest.txt ./iscsi-uring-vg-autotest_77365/output//build-repo-manifest.txt ./iscsi-uring-vg-autotest_77365/output//calsoft.json ./iscsi-uring-vg-autotest_77365/output//calsoft.tar.xz ./iscsi-uring-vg-autotest_77365/output//cov_base.info.xz ./iscsi-uring-vg-autotest_77365/output//cov_test.info.xz ./iscsi-uring-vg-autotest_77365/output//cov_total.info.xz ./iscsi-uring-vg-autotest_77365/output//dmesg.init.log ./iscsi-uring-vg-autotest_77365/output//dmesg.log ./iscsi-uring-vg-autotest_77365/output//iscsi_autofuzz_logs.txt ./iscsi-uring-vg-autotest_77365/output//iscsi_autofuzz_tgt_output.txt ./iscsi-uring-vg-autotest_77365/output//power.tar.xz ./iscsi-uring-vg-autotest_77365/output//test_completions.txt ./iscsi-uring-vg-autotest_77365/output//timing.svg ./iscsi-uring-vg-autotest_77365/output//timing.txt ./iscsi-uring-vg-autotest_77365/output//.. 00:52:25.868 + rmdir ./iscsi-uring-vg-autotest_77365/output/ 00:52:25.868 + for dir in "${out_dirs[@]}" 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./iscsi-vg-autotest_77498/output//doc.tar.xz ]] 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./iscsi-vg-autotest_77498/output//ut_coverage.tar.xz ]] 00:52:25.868 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.868 + [[ -f ./iscsi-vg-autotest_77498/output//llvm.tar.xz ]] 00:52:25.869 + mv ./iscsi-vg-autotest_77498/output//all_tests.txt ./iscsi-vg-autotest_77498/output//build-pkg-manifest.txt ./iscsi-vg-autotest_77498/output//build-repo-manifest.txt ./iscsi-vg-autotest_77498/output//calsoft.json ./iscsi-vg-autotest_77498/output//calsoft.tar.xz ./iscsi-vg-autotest_77498/output//cov_base.info.xz ./iscsi-vg-autotest_77498/output//cov_test.info.xz ./iscsi-vg-autotest_77498/output//cov_total.info.xz ./iscsi-vg-autotest_77498/output//dmesg.init.log ./iscsi-vg-autotest_77498/output//dmesg.log ./iscsi-vg-autotest_77498/output//iscsi_autofuzz_logs.txt ./iscsi-vg-autotest_77498/output//iscsi_autofuzz_tgt_output.txt ./iscsi-vg-autotest_77498/output//power.tar.xz ./iscsi-vg-autotest_77498/output//test_completions.txt ./iscsi-vg-autotest_77498/output//timing.svg ./iscsi-vg-autotest_77498/output//timing.txt ./iscsi-vg-autotest_77498/output//.. 00:52:25.869 + rmdir ./iscsi-vg-autotest_77498/output/ 00:52:25.869 + for dir in "${out_dirs[@]}" 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./lvol-vg-autotest_77409/output//doc.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./lvol-vg-autotest_77409/output//ut_coverage.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./lvol-vg-autotest_77409/output//llvm.tar.xz ]] 00:52:25.869 + mv ./lvol-vg-autotest_77409/output//all_tests.txt ./lvol-vg-autotest_77409/output//build-pkg-manifest.txt ./lvol-vg-autotest_77409/output//build-repo-manifest.txt ./lvol-vg-autotest_77409/output//cov_base.info.xz ./lvol-vg-autotest_77409/output//cov_test.info.xz ./lvol-vg-autotest_77409/output//cov_total.info.xz ./lvol-vg-autotest_77409/output//dmesg.init.log ./lvol-vg-autotest_77409/output//dmesg.log ./lvol-vg-autotest_77409/output//power.tar.xz ./lvol-vg-autotest_77409/output//test_completions.txt ./lvol-vg-autotest_77409/output//timing.svg ./lvol-vg-autotest_77409/output//timing.txt ./lvol-vg-autotest_77409/output//.. 00:52:25.869 + rmdir ./lvol-vg-autotest_77409/output/ 00:52:25.869 + for dir in "${out_dirs[@]}" 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvme-cmb-pmr-vg-autotest_78583/output//doc.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvme-cmb-pmr-vg-autotest_78583/output//ut_coverage.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvme-cmb-pmr-vg-autotest_78583/output//llvm.tar.xz ]] 00:52:25.869 + mv ./nvme-cmb-pmr-vg-autotest_78583/output//all_tests.txt ./nvme-cmb-pmr-vg-autotest_78583/output//build-pkg-manifest.txt ./nvme-cmb-pmr-vg-autotest_78583/output//build-repo-manifest.txt ./nvme-cmb-pmr-vg-autotest_78583/output//cov_base.info.xz ./nvme-cmb-pmr-vg-autotest_78583/output//cov_test.info.xz ./nvme-cmb-pmr-vg-autotest_78583/output//cov_total.info.xz ./nvme-cmb-pmr-vg-autotest_78583/output//dmesg.init.log ./nvme-cmb-pmr-vg-autotest_78583/output//dmesg.log ./nvme-cmb-pmr-vg-autotest_78583/output//power.tar.xz ./nvme-cmb-pmr-vg-autotest_78583/output//test_completions.txt ./nvme-cmb-pmr-vg-autotest_78583/output//timing.svg ./nvme-cmb-pmr-vg-autotest_78583/output//timing.txt ./nvme-cmb-pmr-vg-autotest_78583/output//.. 00:52:25.869 + rmdir ./nvme-cmb-pmr-vg-autotest_78583/output/ 00:52:25.869 + for dir in "${out_dirs[@]}" 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvme-phy-autotest_72352/output//doc.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvme-phy-autotest_72352/output//ut_coverage.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvme-phy-autotest_72352/output//llvm.tar.xz ]] 00:52:25.869 + mv ./nvme-phy-autotest_72352/output//all_tests.txt ./nvme-phy-autotest_72352/output//build-pkg-manifest.txt ./nvme-phy-autotest_72352/output//build-repo-manifest.txt ./nvme-phy-autotest_72352/output//cov_base.info.xz ./nvme-phy-autotest_72352/output//cov_test.info.xz ./nvme-phy-autotest_72352/output//cov_total.info.xz ./nvme-phy-autotest_72352/output//dmesg.init.log ./nvme-phy-autotest_72352/output//dmesg.log ./nvme-phy-autotest_72352/output//power.tar.xz ./nvme-phy-autotest_72352/output//test_completions.txt ./nvme-phy-autotest_72352/output//timing.svg ./nvme-phy-autotest_72352/output//timing.txt ./nvme-phy-autotest_72352/output//.. 00:52:25.869 + rmdir ./nvme-phy-autotest_72352/output/ 00:52:25.869 + for dir in "${out_dirs[@]}" 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvme-vg-autotest_77460/output//doc.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvme-vg-autotest_77460/output//ut_coverage.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvme-vg-autotest_77460/output//llvm.tar.xz ]] 00:52:25.869 + mv ./nvme-vg-autotest_77460/output//all_tests.txt ./nvme-vg-autotest_77460/output//build-pkg-manifest.txt ./nvme-vg-autotest_77460/output//build-repo-manifest.txt ./nvme-vg-autotest_77460/output//cov_base.info.xz ./nvme-vg-autotest_77460/output//dmesg.init.log ./nvme-vg-autotest_77460/output//dmesg.log ./nvme-vg-autotest_77460/output//power.tar.xz ./nvme-vg-autotest_77460/output//proc_list.txt ./nvme-vg-autotest_77460/output//test_completions.txt ./nvme-vg-autotest_77460/output//timing.txt ./nvme-vg-autotest_77460/output//udev.log ./nvme-vg-autotest_77460/output//.. 00:52:25.869 + rmdir ./nvme-vg-autotest_77460/output/ 00:52:25.869 + for dir in "${out_dirs[@]}" 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvmf-phy-autotest_72424/output//doc.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvmf-phy-autotest_72424/output//ut_coverage.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvmf-phy-autotest_72424/output//llvm.tar.xz ]] 00:52:25.869 + mv ./nvmf-phy-autotest_72424/output//all_tests.txt ./nvmf-phy-autotest_72424/output//build-pkg-manifest.txt ./nvmf-phy-autotest_72424/output//build-repo-manifest.txt ./nvmf-phy-autotest_72424/output//coredumps ./nvmf-phy-autotest_72424/output//cov_base.info.xz ./nvmf-phy-autotest_72424/output//dmesg.init.log ./nvmf-phy-autotest_72424/output//dmesg.log ./nvmf-phy-autotest_72424/output//nvmf_fuzz_tgt_output.txt ./nvmf-phy-autotest_72424/output//nvmf_trace.0_shm.tar.gz ./nvmf-phy-autotest_72424/output//power.tar.xz ./nvmf-phy-autotest_72424/output//proc_list.txt ./nvmf-phy-autotest_72424/output//test_completions.txt ./nvmf-phy-autotest_72424/output//timing.txt ./nvmf-phy-autotest_72424/output//udev.log ./nvmf-phy-autotest_72424/output//.. 00:52:25.869 + rmdir ./nvmf-phy-autotest_72424/output/ 00:52:25.869 + for dir in "${out_dirs[@]}" 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvmf-tcp-phy-autotest_72311/output//doc.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvmf-tcp-phy-autotest_72311/output//ut_coverage.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvmf-tcp-phy-autotest_72311/output//llvm.tar.xz ]] 00:52:25.869 + mv ./nvmf-tcp-phy-autotest_72311/output//all_tests.txt ./nvmf-tcp-phy-autotest_72311/output//build-pkg-manifest.txt ./nvmf-tcp-phy-autotest_72311/output//build-repo-manifest.txt ./nvmf-tcp-phy-autotest_72311/output//cov_base.info.xz ./nvmf-tcp-phy-autotest_72311/output//cov_test.info.xz ./nvmf-tcp-phy-autotest_72311/output//cov_total.info.xz ./nvmf-tcp-phy-autotest_72311/output//dmesg.init.log ./nvmf-tcp-phy-autotest_72311/output//dmesg.log ./nvmf-tcp-phy-autotest_72311/output//nvmf_fuzz_tgt_output.txt ./nvmf-tcp-phy-autotest_72311/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-phy-autotest_72311/output//power.tar.xz ./nvmf-tcp-phy-autotest_72311/output//test_completions.txt ./nvmf-tcp-phy-autotest_72311/output//timing.svg ./nvmf-tcp-phy-autotest_72311/output//timing.txt ./nvmf-tcp-phy-autotest_72311/output//.. 00:52:25.869 + rmdir ./nvmf-tcp-phy-autotest_72311/output/ 00:52:25.869 + for dir in "${out_dirs[@]}" 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvmf-tcp-uring-vg-autotest_77385/output//doc.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvmf-tcp-uring-vg-autotest_77385/output//ut_coverage.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvmf-tcp-uring-vg-autotest_77385/output//llvm.tar.xz ]] 00:52:25.869 + mv ./nvmf-tcp-uring-vg-autotest_77385/output//all_tests.txt ./nvmf-tcp-uring-vg-autotest_77385/output//build-pkg-manifest.txt ./nvmf-tcp-uring-vg-autotest_77385/output//build-repo-manifest.txt ./nvmf-tcp-uring-vg-autotest_77385/output//cov_base.info.xz ./nvmf-tcp-uring-vg-autotest_77385/output//dmesg.init.log ./nvmf-tcp-uring-vg-autotest_77385/output//dmesg.log ./nvmf-tcp-uring-vg-autotest_77385/output//nvmf_fuzz_tgt_output.txt ./nvmf-tcp-uring-vg-autotest_77385/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-uring-vg-autotest_77385/output//power.tar.xz ./nvmf-tcp-uring-vg-autotest_77385/output//proc_list.txt ./nvmf-tcp-uring-vg-autotest_77385/output//test_completions.txt ./nvmf-tcp-uring-vg-autotest_77385/output//timing.txt ./nvmf-tcp-uring-vg-autotest_77385/output//udev.log ./nvmf-tcp-uring-vg-autotest_77385/output//.. 00:52:25.869 + rmdir ./nvmf-tcp-uring-vg-autotest_77385/output/ 00:52:25.869 + for dir in "${out_dirs[@]}" 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvmf-tcp-vg-autotest_77451/output//doc.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvmf-tcp-vg-autotest_77451/output//ut_coverage.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./nvmf-tcp-vg-autotest_77451/output//llvm.tar.xz ]] 00:52:25.869 + mv ./nvmf-tcp-vg-autotest_77451/output//all_tests.txt ./nvmf-tcp-vg-autotest_77451/output//build-pkg-manifest.txt ./nvmf-tcp-vg-autotest_77451/output//build-repo-manifest.txt ./nvmf-tcp-vg-autotest_77451/output//cov_base.info.xz ./nvmf-tcp-vg-autotest_77451/output//cov_test.info.xz ./nvmf-tcp-vg-autotest_77451/output//cov_total.info.xz ./nvmf-tcp-vg-autotest_77451/output//dmesg.init.log ./nvmf-tcp-vg-autotest_77451/output//dmesg.log ./nvmf-tcp-vg-autotest_77451/output//nvmf_fuzz_tgt_output.txt ./nvmf-tcp-vg-autotest_77451/output//nvmf_trace.0_shm.tar.gz ./nvmf-tcp-vg-autotest_77451/output//power.tar.xz ./nvmf-tcp-vg-autotest_77451/output//test_completions.txt ./nvmf-tcp-vg-autotest_77451/output//timing.svg ./nvmf-tcp-vg-autotest_77451/output//timing.txt ./nvmf-tcp-vg-autotest_77451/output//.. 00:52:25.869 + rmdir ./nvmf-tcp-vg-autotest_77451/output/ 00:52:25.869 + for dir in "${out_dirs[@]}" 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./rocky9-vg-autotest_16070/output//doc.tar.xz ]] 00:52:25.869 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:25.869 + [[ -f ./rocky9-vg-autotest_16070/output//ut_coverage.tar.xz ]] 00:52:25.869 + tar -C ./rocky9-vg-autotest_16070/output/ -xf ./rocky9-vg-autotest_16070/output//ut_coverage.tar.xz 00:52:26.127 + rm ./rocky9-vg-autotest_16070/output//ut_coverage.tar.xz 00:52:26.127 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.127 + [[ -f ./rocky9-vg-autotest_16070/output//llvm.tar.xz ]] 00:52:26.127 + mv ./rocky9-vg-autotest_16070/output//all_tests.txt ./rocky9-vg-autotest_16070/output//build-pkg-manifest.txt ./rocky9-vg-autotest_16070/output//build-repo-manifest.txt ./rocky9-vg-autotest_16070/output//cov_base.info.xz ./rocky9-vg-autotest_16070/output//cov_test.info.xz ./rocky9-vg-autotest_16070/output//cov_total.info.xz ./rocky9-vg-autotest_16070/output//dmesg.init.log ./rocky9-vg-autotest_16070/output//dmesg.log ./rocky9-vg-autotest_16070/output//power.tar.xz ./rocky9-vg-autotest_16070/output//test_completions.txt ./rocky9-vg-autotest_16070/output//timing.svg ./rocky9-vg-autotest_16070/output//timing.txt ./rocky9-vg-autotest_16070/output//ut_coverage ./rocky9-vg-autotest_16070/output//.. 00:52:26.127 + rmdir ./rocky9-vg-autotest_16070/output/ 00:52:26.127 + for dir in "${out_dirs[@]}" 00:52:26.127 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.127 + [[ -f ./scanbuild-vg-autotest_63375/output//doc.tar.xz ]] 00:52:26.127 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.127 + [[ -f ./scanbuild-vg-autotest_63375/output//ut_coverage.tar.xz ]] 00:52:26.127 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.127 + [[ -f ./scanbuild-vg-autotest_63375/output//llvm.tar.xz ]] 00:52:26.127 + mv ./scanbuild-vg-autotest_63375/output//all_c_files.txt ./scanbuild-vg-autotest_63375/output//build-kernel-manifest.txt ./scanbuild-vg-autotest_63375/output//build_output.txt ./scanbuild-vg-autotest_63375/output//build-pkg-manifest.txt ./scanbuild-vg-autotest_63375/output//build-repo-manifest.txt ./scanbuild-vg-autotest_63375/output//built_c_files.txt ./scanbuild-vg-autotest_63375/output//dmesg.init.log ./scanbuild-vg-autotest_63375/output//dmesg.log ./scanbuild-vg-autotest_63375/output//power.tar.xz ./scanbuild-vg-autotest_63375/output//test_completions.txt ./scanbuild-vg-autotest_63375/output//timing.svg ./scanbuild-vg-autotest_63375/output//timing.txt ./scanbuild-vg-autotest_63375/output//unbuilt_c_files.txt ./scanbuild-vg-autotest_63375/output//.. 00:52:26.127 + rmdir ./scanbuild-vg-autotest_63375/output/ 00:52:26.127 + for dir in "${out_dirs[@]}" 00:52:26.127 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.127 + [[ -f ./short-fuzz-phy-autotest_26177/output//doc.tar.xz ]] 00:52:26.127 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.127 + [[ -f ./short-fuzz-phy-autotest_26177/output//ut_coverage.tar.xz ]] 00:52:26.127 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.127 + [[ -f ./short-fuzz-phy-autotest_26177/output//llvm.tar.xz ]] 00:52:26.127 + tar -C ./short-fuzz-phy-autotest_26177/output/ -xf ./short-fuzz-phy-autotest_26177/output//llvm.tar.xz 00:52:26.127 + rm ./short-fuzz-phy-autotest_26177/output//llvm.tar.xz 00:52:26.127 + mv ./short-fuzz-phy-autotest_26177/output//all_tests.txt ./short-fuzz-phy-autotest_26177/output//build-pkg-manifest.txt ./short-fuzz-phy-autotest_26177/output//build-repo-manifest.txt ./short-fuzz-phy-autotest_26177/output//dmesg.init.log ./short-fuzz-phy-autotest_26177/output//dmesg.log ./short-fuzz-phy-autotest_26177/output//llvm ./short-fuzz-phy-autotest_26177/output//power.tar.xz ./short-fuzz-phy-autotest_26177/output//test_completions.txt ./short-fuzz-phy-autotest_26177/output//timing.svg ./short-fuzz-phy-autotest_26177/output//timing.txt ./short-fuzz-phy-autotest_26177/output//.. 00:52:26.127 + rmdir ./short-fuzz-phy-autotest_26177/output/ 00:52:26.127 + for dir in "${out_dirs[@]}" 00:52:26.127 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.127 + [[ -f ./ubuntu20-vg-autotest_73779/output//doc.tar.xz ]] 00:52:26.127 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.127 + [[ -f ./ubuntu20-vg-autotest_73779/output//ut_coverage.tar.xz ]] 00:52:26.128 + tar -C ./ubuntu20-vg-autotest_73779/output/ -xf ./ubuntu20-vg-autotest_73779/output//ut_coverage.tar.xz 00:52:26.385 + rm ./ubuntu20-vg-autotest_73779/output//ut_coverage.tar.xz 00:52:26.385 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.385 + [[ -f ./ubuntu20-vg-autotest_73779/output//llvm.tar.xz ]] 00:52:26.385 + mv ./ubuntu20-vg-autotest_73779/output//all_tests.txt ./ubuntu20-vg-autotest_73779/output//build-pkg-manifest.txt ./ubuntu20-vg-autotest_73779/output//build-repo-manifest.txt ./ubuntu20-vg-autotest_73779/output//cov_base.info.xz ./ubuntu20-vg-autotest_73779/output//dmesg.init.log ./ubuntu20-vg-autotest_73779/output//dmesg.log ./ubuntu20-vg-autotest_73779/output//power.tar.xz ./ubuntu20-vg-autotest_73779/output//proc_list.txt ./ubuntu20-vg-autotest_73779/output//test_completions.txt ./ubuntu20-vg-autotest_73779/output//timing.txt ./ubuntu20-vg-autotest_73779/output//udev.log ./ubuntu20-vg-autotest_73779/output//ut_coverage ./ubuntu20-vg-autotest_73779/output//.. 00:52:26.385 + rmdir ./ubuntu20-vg-autotest_73779/output/ 00:52:26.385 + for dir in "${out_dirs[@]}" 00:52:26.385 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.385 + [[ -f ./ubuntu22-vg-autotest_37793/output//doc.tar.xz ]] 00:52:26.385 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.385 + [[ -f ./ubuntu22-vg-autotest_37793/output//ut_coverage.tar.xz ]] 00:52:26.385 + tar -C ./ubuntu22-vg-autotest_37793/output/ -xf ./ubuntu22-vg-autotest_37793/output//ut_coverage.tar.xz 00:52:26.643 + rm ./ubuntu22-vg-autotest_37793/output//ut_coverage.tar.xz 00:52:26.643 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.643 + [[ -f ./ubuntu22-vg-autotest_37793/output//llvm.tar.xz ]] 00:52:26.643 + mv ./ubuntu22-vg-autotest_37793/output//all_tests.txt ./ubuntu22-vg-autotest_37793/output//build-pkg-manifest.txt ./ubuntu22-vg-autotest_37793/output//build-repo-manifest.txt ./ubuntu22-vg-autotest_37793/output//cov_base.info.xz ./ubuntu22-vg-autotest_37793/output//cov_test.info.xz ./ubuntu22-vg-autotest_37793/output//cov_total.info.xz ./ubuntu22-vg-autotest_37793/output//dmesg.init.log ./ubuntu22-vg-autotest_37793/output//dmesg.log ./ubuntu22-vg-autotest_37793/output//power.tar.xz ./ubuntu22-vg-autotest_37793/output//test_completions.txt ./ubuntu22-vg-autotest_37793/output//timing.svg ./ubuntu22-vg-autotest_37793/output//timing.txt ./ubuntu22-vg-autotest_37793/output//ut_coverage ./ubuntu22-vg-autotest_37793/output//.. 00:52:26.643 + rmdir ./ubuntu22-vg-autotest_37793/output/ 00:52:26.643 + for dir in "${out_dirs[@]}" 00:52:26.643 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.643 + [[ -f ./unittest-vg-autotest_63373/output//doc.tar.xz ]] 00:52:26.643 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.643 + [[ -f ./unittest-vg-autotest_63373/output//ut_coverage.tar.xz ]] 00:52:26.643 + tar -C ./unittest-vg-autotest_63373/output/ -xf ./unittest-vg-autotest_63373/output//ut_coverage.tar.xz 00:52:26.901 + rm ./unittest-vg-autotest_63373/output//ut_coverage.tar.xz 00:52:26.901 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.901 + [[ -f ./unittest-vg-autotest_63373/output//llvm.tar.xz ]] 00:52:26.901 + mv ./unittest-vg-autotest_63373/output//all_tests.txt ./unittest-vg-autotest_63373/output//build-kernel-manifest.txt ./unittest-vg-autotest_63373/output//build-pkg-manifest.txt ./unittest-vg-autotest_63373/output//build-repo-manifest.txt ./unittest-vg-autotest_63373/output//cov_base.info.xz ./unittest-vg-autotest_63373/output//cov_test.info.xz ./unittest-vg-autotest_63373/output//cov_total.info.xz ./unittest-vg-autotest_63373/output//dmesg.init.log ./unittest-vg-autotest_63373/output//dmesg.log ./unittest-vg-autotest_63373/output//power.tar.xz ./unittest-vg-autotest_63373/output//test_completions.txt ./unittest-vg-autotest_63373/output//timing.svg ./unittest-vg-autotest_63373/output//timing.txt ./unittest-vg-autotest_63373/output//ut_coverage ./unittest-vg-autotest_63373/output//.. 00:52:26.901 + rmdir ./unittest-vg-autotest_63373/output/ 00:52:26.901 + for dir in "${out_dirs[@]}" 00:52:26.901 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.901 + [[ -f ./valgrind-vg-autotest_49190/output//doc.tar.xz ]] 00:52:26.901 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:26.901 + [[ -f ./valgrind-vg-autotest_49190/output//ut_coverage.tar.xz ]] 00:52:26.901 + tar -C ./valgrind-vg-autotest_49190/output/ -xf ./valgrind-vg-autotest_49190/output//ut_coverage.tar.xz 00:52:27.159 + rm ./valgrind-vg-autotest_49190/output//ut_coverage.tar.xz 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./valgrind-vg-autotest_49190/output//llvm.tar.xz ]] 00:52:27.159 + mv ./valgrind-vg-autotest_49190/output//all_tests.txt ./valgrind-vg-autotest_49190/output//build-pkg-manifest.txt ./valgrind-vg-autotest_49190/output//build-repo-manifest.txt ./valgrind-vg-autotest_49190/output//cov_base.info.xz ./valgrind-vg-autotest_49190/output//cov_test.info.xz ./valgrind-vg-autotest_49190/output//cov_total.info.xz ./valgrind-vg-autotest_49190/output//dmesg.init.log ./valgrind-vg-autotest_49190/output//dmesg.log ./valgrind-vg-autotest_49190/output//power.tar.xz ./valgrind-vg-autotest_49190/output//test_completions.txt ./valgrind-vg-autotest_49190/output//timing.svg ./valgrind-vg-autotest_49190/output//timing.txt ./valgrind-vg-autotest_49190/output//ut_coverage ./valgrind-vg-autotest_49190/output//.. 00:52:27.159 + rmdir ./valgrind-vg-autotest_49190/output/ 00:52:27.159 + for dir in "${out_dirs[@]}" 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./vfio-user-phy-autotest_26188/output//doc.tar.xz ]] 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./vfio-user-phy-autotest_26188/output//ut_coverage.tar.xz ]] 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./vfio-user-phy-autotest_26188/output//llvm.tar.xz ]] 00:52:27.159 + mv ./vfio-user-phy-autotest_26188/output//all_tests.txt ./vfio-user-phy-autotest_26188/output//build-pkg-manifest.txt ./vfio-user-phy-autotest_26188/output//build-repo-manifest.txt ./vfio-user-phy-autotest_26188/output//cov_base.info.xz ./vfio-user-phy-autotest_26188/output//cov_test.info.xz ./vfio-user-phy-autotest_26188/output//cov_total.info.xz ./vfio-user-phy-autotest_26188/output//dmesg.init.log ./vfio-user-phy-autotest_26188/output//dmesg.log ./vfio-user-phy-autotest_26188/output//power.tar.xz ./vfio-user-phy-autotest_26188/output//test_completions.txt ./vfio-user-phy-autotest_26188/output//timing.svg ./vfio-user-phy-autotest_26188/output//timing.txt ./vfio-user-phy-autotest_26188/output//.. 00:52:27.159 + rmdir ./vfio-user-phy-autotest_26188/output/ 00:52:27.159 + for dir in "${out_dirs[@]}" 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./vhost-initiator-vg-autotest_77480/output//doc.tar.xz ]] 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./vhost-initiator-vg-autotest_77480/output//ut_coverage.tar.xz ]] 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./vhost-initiator-vg-autotest_77480/output//llvm.tar.xz ]] 00:52:27.159 + mv ./vhost-initiator-vg-autotest_77480/output//all_tests.txt ./vhost-initiator-vg-autotest_77480/output//build-pkg-manifest.txt ./vhost-initiator-vg-autotest_77480/output//build-repo-manifest.txt ./vhost-initiator-vg-autotest_77480/output//cov_base.info.xz ./vhost-initiator-vg-autotest_77480/output//cov_test.info.xz ./vhost-initiator-vg-autotest_77480/output//cov_total.info.xz ./vhost-initiator-vg-autotest_77480/output//dmesg.init.log ./vhost-initiator-vg-autotest_77480/output//dmesg.log ./vhost-initiator-vg-autotest_77480/output//power.tar.xz ./vhost-initiator-vg-autotest_77480/output//test_completions.txt ./vhost-initiator-vg-autotest_77480/output//timing.svg ./vhost-initiator-vg-autotest_77480/output//timing.txt ./vhost-initiator-vg-autotest_77480/output//vhost_fuzz_output1.txt.xz ./vhost-initiator-vg-autotest_77480/output//vhost_fuzz_output2.txt.xz ./vhost-initiator-vg-autotest_77480/output//vhost_fuzz_tgt_output.txt.xz ./vhost-initiator-vg-autotest_77480/output//.. 00:52:27.159 + rmdir ./vhost-initiator-vg-autotest_77480/output/ 00:52:27.159 + for dir in "${out_dirs[@]}" 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./vhost-phy-autotest_26205/output//doc.tar.xz ]] 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./vhost-phy-autotest_26205/output//ut_coverage.tar.xz ]] 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./vhost-phy-autotest_26205/output//llvm.tar.xz ]] 00:52:27.159 + mv ./vhost-phy-autotest_26205/output//all_tests.txt ./vhost-phy-autotest_26205/output//build-pkg-manifest.txt ./vhost-phy-autotest_26205/output//build-repo-manifest.txt ./vhost-phy-autotest_26205/output//cov_base.info.xz ./vhost-phy-autotest_26205/output//cov_test.info.xz ./vhost-phy-autotest_26205/output//cov_total.info.xz ./vhost-phy-autotest_26205/output//dmesg.init.log ./vhost-phy-autotest_26205/output//dmesg.log ./vhost-phy-autotest_26205/output//nvmf_trace.0_shm.tar.gz ./vhost-phy-autotest_26205/output//power.tar.xz ./vhost-phy-autotest_26205/output//test_completions.txt ./vhost-phy-autotest_26205/output//timing.svg ./vhost-phy-autotest_26205/output//timing.txt ./vhost-phy-autotest_26205/output//.. 00:52:27.159 + rmdir ./vhost-phy-autotest_26205/output/ 00:52:27.159 + for dir in "${out_dirs[@]}" 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./vmd-phy-autotest_2618/output//doc.tar.xz ]] 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./vmd-phy-autotest_2618/output//ut_coverage.tar.xz ]] 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./vmd-phy-autotest_2618/output//llvm.tar.xz ]] 00:52:27.159 + mv ./vmd-phy-autotest_2618/output//all_tests.txt ./vmd-phy-autotest_2618/output//build-pkg-manifest.txt ./vmd-phy-autotest_2618/output//build-repo-manifest.txt ./vmd-phy-autotest_2618/output//cov_base.info.xz ./vmd-phy-autotest_2618/output//cov_test.info.xz ./vmd-phy-autotest_2618/output//cov_total.info.xz ./vmd-phy-autotest_2618/output//dmesg.init.log ./vmd-phy-autotest_2618/output//dmesg.log ./vmd-phy-autotest_2618/output//power.tar.xz ./vmd-phy-autotest_2618/output//test_completions.txt ./vmd-phy-autotest_2618/output//timing.svg ./vmd-phy-autotest_2618/output//timing.txt ./vmd-phy-autotest_2618/output//.. 00:52:27.159 + rmdir ./vmd-phy-autotest_2618/output/ 00:52:27.159 + for dir in "${out_dirs[@]}" 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./zns-vg-autotest_76566/output//doc.tar.xz ]] 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./zns-vg-autotest_76566/output//ut_coverage.tar.xz ]] 00:52:27.159 + for archive in "${dir}"/{doc,ut_coverage,llvm}.tar.xz 00:52:27.159 + [[ -f ./zns-vg-autotest_76566/output//llvm.tar.xz ]] 00:52:27.159 + mv ./zns-vg-autotest_76566/output//all_tests.txt ./zns-vg-autotest_76566/output//build-pkg-manifest.txt ./zns-vg-autotest_76566/output//build-repo-manifest.txt ./zns-vg-autotest_76566/output//cov_base.info.xz ./zns-vg-autotest_76566/output//cov_test.info.xz ./zns-vg-autotest_76566/output//cov_total.info.xz ./zns-vg-autotest_76566/output//dmesg.init.log ./zns-vg-autotest_76566/output//dmesg.log ./zns-vg-autotest_76566/output//power.tar.xz ./zns-vg-autotest_76566/output//test_completions.txt ./zns-vg-autotest_76566/output//timing.svg ./zns-vg-autotest_76566/output//timing.txt ./zns-vg-autotest_76566/output//.. 00:52:27.159 + rmdir ./zns-vg-autotest_76566/output/ 00:52:27.159 + unpack_cov_files 00:52:27.159 + local info_files 00:52:27.159 + info_files=(*/cov_*.info.xz) 00:52:27.159 + xargs -P0 -r -n1 xz -d 00:52:27.159 + printf '%s\n' BlobFS-phy-autotest_26186/cov_base.info.xz BlobFS-phy-autotest_26186/cov_test.info.xz BlobFS-phy-autotest_26186/cov_total.info.xz centos7-vg-autotest_73639/cov_base.info.xz centos7-vg-autotest_73639/cov_test.info.xz centos7-vg-autotest_73639/cov_total.info.xz crypto-phy-autotest_26180/cov_base.info.xz crypto-phy-autotest_26180/cov_test.info.xz crypto-phy-autotest_26180/cov_total.info.xz dsa-phy-autotest_4880/cov_base.info.xz dsa-phy-autotest_4880/cov_test.info.xz dsa-phy-autotest_4880/cov_total.info.xz iscsi-uring-vg-autotest_77365/cov_base.info.xz iscsi-uring-vg-autotest_77365/cov_test.info.xz iscsi-uring-vg-autotest_77365/cov_total.info.xz iscsi-vg-autotest_77498/cov_base.info.xz iscsi-vg-autotest_77498/cov_test.info.xz iscsi-vg-autotest_77498/cov_total.info.xz lvol-vg-autotest_77409/cov_base.info.xz lvol-vg-autotest_77409/cov_test.info.xz lvol-vg-autotest_77409/cov_total.info.xz nvme-cmb-pmr-vg-autotest_78583/cov_base.info.xz nvme-cmb-pmr-vg-autotest_78583/cov_test.info.xz nvme-cmb-pmr-vg-autotest_78583/cov_total.info.xz nvme-phy-autotest_72352/cov_base.info.xz nvme-phy-autotest_72352/cov_test.info.xz nvme-phy-autotest_72352/cov_total.info.xz nvme-vg-autotest_77460/cov_base.info.xz nvmf-phy-autotest_72424/cov_base.info.xz nvmf-tcp-phy-autotest_72311/cov_base.info.xz nvmf-tcp-phy-autotest_72311/cov_test.info.xz nvmf-tcp-phy-autotest_72311/cov_total.info.xz nvmf-tcp-uring-vg-autotest_77385/cov_base.info.xz nvmf-tcp-vg-autotest_77451/cov_base.info.xz nvmf-tcp-vg-autotest_77451/cov_test.info.xz nvmf-tcp-vg-autotest_77451/cov_total.info.xz rocky9-vg-autotest_16070/cov_base.info.xz rocky9-vg-autotest_16070/cov_test.info.xz rocky9-vg-autotest_16070/cov_total.info.xz ubuntu20-vg-autotest_73779/cov_base.info.xz ubuntu22-vg-autotest_37793/cov_base.info.xz ubuntu22-vg-autotest_37793/cov_test.info.xz ubuntu22-vg-autotest_37793/cov_total.info.xz unittest-vg-autotest_63373/cov_base.info.xz unittest-vg-autotest_63373/cov_test.info.xz unittest-vg-autotest_63373/cov_total.info.xz valgrind-vg-autotest_49190/cov_base.info.xz valgrind-vg-autotest_49190/cov_test.info.xz valgrind-vg-autotest_49190/cov_total.info.xz vfio-user-phy-autotest_26188/cov_base.info.xz vfio-user-phy-autotest_26188/cov_test.info.xz vfio-user-phy-autotest_26188/cov_total.info.xz vhost-initiator-vg-autotest_77480/cov_base.info.xz vhost-initiator-vg-autotest_77480/cov_test.info.xz vhost-initiator-vg-autotest_77480/cov_total.info.xz vhost-phy-autotest_26205/cov_base.info.xz vhost-phy-autotest_26205/cov_test.info.xz vhost-phy-autotest_26205/cov_total.info.xz vmd-phy-autotest_2618/cov_base.info.xz vmd-phy-autotest_2618/cov_test.info.xz vmd-phy-autotest_2618/cov_total.info.xz zns-vg-autotest_76566/cov_base.info.xz zns-vg-autotest_76566/cov_test.info.xz zns-vg-autotest_76566/cov_total.info.xz 00:52:27.724 + fix_downstream_job_paths 00:52:27.724 + sed -i -e 's#^SF:/.\+/spdk/#SF:/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/#g' BlobFS-phy-autotest_26186/cov_total.info centos7-vg-autotest_73639/cov_total.info crypto-phy-autotest_26180/cov_total.info dsa-phy-autotest_4880/cov_total.info iscsi-uring-vg-autotest_77365/cov_total.info iscsi-vg-autotest_77498/cov_total.info lvol-vg-autotest_77409/cov_total.info nvme-cmb-pmr-vg-autotest_78583/cov_total.info nvme-phy-autotest_72352/cov_total.info nvmf-tcp-phy-autotest_72311/cov_total.info nvmf-tcp-vg-autotest_77451/cov_total.info rocky9-vg-autotest_16070/cov_total.info ubuntu22-vg-autotest_37793/cov_total.info unittest-vg-autotest_63373/cov_total.info valgrind-vg-autotest_49190/cov_total.info vfio-user-phy-autotest_26188/cov_total.info vhost-initiator-vg-autotest_77480/cov_total.info vhost-phy-autotest_26205/cov_total.info vmd-phy-autotest_2618/cov_total.info zns-vg-autotest_76566/cov_total.info 00:52:28.663 + ignore_ocf_headers 00:52:28.663 + sed -i 's#--show-details#--show-details --ignore-errors source#g' spdk/autorun_post.py 00:52:28.663 + create_errtrace_files_for_all_builds 00:52:28.663 + local build_log_list 00:52:28.663 + build_log_list=($(find "$WORKSPACE" -type f -name 'build.log')) 00:52:28.663 ++ find /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main -type f -name build.log 00:52:28.663 + for build in "${build_log_list[@]}" 00:52:28.663 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu20-vg-autotest_73779/build.log 00:52:28.663 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.663 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu20-vg-autotest_73779/build.log 00:52:28.663 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu20-vg-autotest_73779/build.log 00:52:28.663 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu20-vg-autotest_73779 00:52:28.663 + job_name=ubuntu20-vg-autotest_73779 00:52:28.663 + start_pattern='Backtrace start' 00:52:28.663 + end_pattern='Backtrace end' 00:52:28.663 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu20-vg-autotest_73779/build.log ]] 00:52:28.663 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu20-vg-autotest_73779/build.log 00:52:28.663 + backtraces='00:40:32.547 06:11:27 sw_hotplug -- common/autotest_common.sh@715 -- # time='\''========== Backtrace start: ========== 00:52:28.663 00:40:32.547 00:52:28.663 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.663 00:40:32.547 ... 00:52:28.663 00:40:32.547 710 00:52:28.663 00:40:32.547 711 # We redirect cmd'\''\'\'''\''s std{out,err} to a separate fd dup'\''\'\'''\''ed to stdin'\''\'\'''\''s device (or /dev/null) to 00:52:28.663 00:40:32.547 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 00:52:28.663 00:40:32.547 713 # but $() will return just the time'\''\'\'''\''s data, hence making it possible to just do: 00:52:28.663 00:40:32.547 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.663 00:40:32.547 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.663 00:40:32.547 716 00:52:28.663 00:40:32.547 717 echo "$time" 00:52:28.663 00:40:32.547 718 ) 00:52:28.663 00:40:32.547 719 00:52:28.663 00:40:32.547 720 function timing_enter() { 00:52:28.663 00:40:32.547 ... 00:52:28.663 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.663 00:40:32.547 ... 00:52:28.663 00:40:32.547 11 # Pci bus hotplug 00:52:28.663 00:40:32.547 12 # Helper function to remove/attach cotrollers 00:52:28.663 00:40:32.547 13 debug_remove_attach_helper() { 00:52:28.663 00:40:32.547 14 local helper_time=0 00:52:28.663 00:40:32.547 15 00:52:28.663 00:40:32.547 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.663 00:40:32.547 17 printf '\''\'\'''\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\''\'\'''\'' \ 00:52:28.663 00:40:32.547 18 "$helper_time" "$nvme_count" >&2 00:52:28.663 00:40:32.547 19 } 00:52:28.663 00:40:32.547 20 00:52:28.663 00:40:32.547 21 remove_attach_helper() { 00:52:28.663 00:40:32.547 ... 00:52:28.663 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.663 00:40:32.547 ... 00:52:28.663 00:40:32.547 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.663 00:40:32.547 106 done 00:52:28.663 00:40:32.547 107 00:52:28.663 00:40:32.547 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.663 00:40:32.547 109 00:52:28.663 00:40:32.547 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.663 00:40:32.547 111 # Verify reregistering hotplug poller 00:52:28.663 00:40:32.547 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.663 00:40:32.547 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.663 00:40:32.547 114 00:52:28.663 00:40:32.547 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.663 00:40:32.547 ... 00:52:28.663 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.663 00:40:32.547 ... 00:52:28.663 00:40:32.547 133 00:52:28.663 00:40:32.547 134 # Run pci bus hotplug test 00:52:28.664 00:40:32.547 135 run_hotplug 00:52:28.664 00:40:32.547 136 00:52:28.664 00:40:32.547 137 # Run SPDK target based hotplug 00:52:28.664 00:40:32.547 => 138 tgt_run_hotplug 00:52:28.664 00:40:32.547 ... 00:52:28.664 00:40:32.547 00:52:28.664 00:40:32.547 ========== Backtrace end ========== 00:52:28.664 00:40:32.547 ========== Backtrace start: ========== 00:52:28.664 00:40:32.547 00:52:28.664 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.664 00:40:32.547 ... 00:52:28.664 00:40:32.547 710 00:52:28.664 00:40:32.547 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 00:52:28.664 00:40:32.547 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 00:52:28.664 00:40:32.547 713 # but $() will return just the time'\''s data, hence making it possible to just do: 00:52:28.664 00:40:32.547 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.664 00:40:32.547 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.664 00:40:32.547 716 00:52:28.664 00:40:32.547 717 echo "$time" 00:52:28.664 00:40:32.547 718 ) 00:52:28.664 00:40:32.547 719 00:52:28.664 00:40:32.547 720 function timing_enter() { 00:52:28.664 00:40:32.547 ... 00:52:28.664 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.664 00:40:32.547 ... 00:52:28.664 00:40:32.547 11 # Pci bus hotplug 00:52:28.664 00:40:32.547 12 # Helper function to remove/attach cotrollers 00:52:28.664 00:40:32.547 13 debug_remove_attach_helper() { 00:52:28.664 00:40:32.547 14 local helper_time=0 00:52:28.664 00:40:32.547 15 00:52:28.664 00:40:32.547 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.664 00:40:32.547 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.664 00:40:32.547 18 "$helper_time" "$nvme_count" >&2 00:52:28.664 00:40:32.547 19 } 00:52:28.664 00:40:32.547 20 00:52:28.664 00:40:32.547 21 remove_attach_helper() { 00:52:28.664 00:40:32.547 ... 00:52:28.664 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.664 00:40:32.547 ... 00:52:28.664 00:40:32.547 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.664 00:40:32.547 106 done 00:52:28.664 00:40:32.547 107 00:52:28.664 00:40:32.547 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.664 00:40:32.547 109 00:52:28.664 00:40:32.547 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.664 00:40:32.547 111 # Verify reregistering hotplug poller 00:52:28.664 00:40:32.547 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.664 00:40:32.547 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.664 00:40:32.547 114 00:52:28.664 00:40:32.547 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.664 00:40:32.547 ... 00:52:28.664 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.664 00:40:32.547 ... 00:52:28.664 00:40:32.547 133 00:52:28.664 00:40:32.547 134 # Run pci bus hotplug test 00:52:28.664 00:40:32.547 135 run_hotplug 00:52:28.664 00:40:32.547 136 00:52:28.664 00:40:32.547 137 # Run SPDK target based hotplug 00:52:28.664 00:40:32.547 => 138 tgt_run_hotplug 00:52:28.664 00:40:32.547 ... 00:52:28.664 00:40:32.547 00:52:28.664 00:40:32.547 ========== Backtrace end ========== 00:52:28.664 00:40:32.547 ========== Backtrace start: ========== 00:52:28.664 00:40:32.547 00:52:28.664 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.664 00:40:32.547 ... 00:52:28.664 00:40:32.547 713 # but $() will return just the time'\''s data, hence making it possible to just do: 00:52:28.664 00:40:32.547 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.664 00:40:32.547 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.664 00:40:32.547 716 00:52:28.664 00:40:32.547 717 echo "$time" 00:52:28.664 00:40:32.547 => 718 ) 00:52:28.664 00:40:32.547 719 00:52:28.664 00:40:32.547 720 function timing_enter() { 00:52:28.664 00:40:32.547 721 xtrace_disable 00:52:28.664 00:40:32.547 722 timing "enter" "$1" 00:52:28.664 00:40:32.547 723 xtrace_restore 00:52:28.664 00:40:32.547 ... 00:52:28.664 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.664 00:40:32.548 ... 00:52:28.664 00:40:32.548 11 # Pci bus hotplug 00:52:28.664 00:40:32.548 12 # Helper function to remove/attach cotrollers 00:52:28.664 00:40:32.548 13 debug_remove_attach_helper() { 00:52:28.664 00:40:32.548 14 local helper_time=0 00:52:28.664 00:40:32.548 15 00:52:28.664 00:40:32.548 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.664 00:40:32.548 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.664 00:40:32.548 18 "$helper_time" "$nvme_count" >&2 00:52:28.664 00:40:32.548 19 } 00:52:28.664 00:40:32.548 20 00:52:28.664 00:40:32.548 21 remove_attach_helper() { 00:52:28.664 00:40:32.548 ... 00:52:28.664 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.664 00:40:32.548 ... 00:52:28.664 00:40:32.548 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.664 00:40:32.548 106 done 00:52:28.664 00:40:32.548 107 00:52:28.664 00:40:32.548 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.664 00:40:32.548 109 00:52:28.664 00:40:32.548 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.664 00:40:32.548 111 # Verify reregistering hotplug poller 00:52:28.664 00:40:32.548 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.664 00:40:32.548 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.664 00:40:32.548 114 00:52:28.664 00:40:32.548 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.664 00:40:32.548 ... 00:52:28.664 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.664 00:40:32.548 ... 00:52:28.664 00:40:32.548 133 00:52:28.664 00:40:32.548 134 # Run pci bus hotplug test 00:52:28.664 00:40:32.548 135 run_hotplug 00:52:28.664 00:40:32.548 136 00:52:28.664 00:40:32.548 137 # Run SPDK target based hotplug 00:52:28.664 00:40:32.548 => 138 tgt_run_hotplug 00:52:28.664 00:40:32.548 ... 00:52:28.664 00:40:32.548 00:52:28.664 00:40:32.548 ========== Backtrace end ========== 00:52:28.664 00:40:32.548 ========== Backtrace start: ========== 00:52:28.664 00:40:32.548 00:52:28.664 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.664 00:40:32.548 ... 00:52:28.664 00:40:32.548 11 # Pci bus hotplug 00:52:28.664 00:40:32.548 12 # Helper function to remove/attach cotrollers 00:52:28.664 00:40:32.548 13 debug_remove_attach_helper() { 00:52:28.664 00:40:32.548 14 local helper_time=0 00:52:28.664 00:40:32.548 15 00:52:28.664 00:40:32.548 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.664 00:40:32.548 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.664 00:40:32.548 18 "$helper_time" "$nvme_count" >&2 00:52:28.664 00:40:32.548 19 } 00:52:28.664 00:40:32.548 20 00:52:28.664 00:40:32.548 21 remove_attach_helper() { 00:52:28.664 00:40:32.548 ... 00:52:28.664 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.664 00:40:32.548 ... 00:52:28.664 00:40:32.548 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.664 00:40:32.548 106 done 00:52:28.664 00:40:32.548 107 00:52:28.664 00:40:32.548 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.664 00:40:32.548 109 00:52:28.664 00:40:32.548 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.664 00:40:32.548 111 # Verify reregistering hotplug poller 00:52:28.664 00:40:32.548 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.664 00:40:32.548 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.664 00:40:32.548 114 00:52:28.664 00:40:32.548 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.664 00:40:32.548 ... 00:52:28.664 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.664 00:40:32.548 ... 00:52:28.664 00:40:32.548 133 00:52:28.664 00:40:32.548 134 # Run pci bus hotplug test 00:52:28.664 00:40:32.548 135 run_hotplug 00:52:28.664 00:40:32.548 136 00:52:28.664 00:40:32.548 137 # Run SPDK target based hotplug 00:52:28.664 00:40:32.548 => 138 tgt_run_hotplug 00:52:28.664 00:40:32.548 ... 00:52:28.664 00:40:32.548 00:52:28.664 00:40:32.548 ========== Backtrace end ========== 00:52:28.664 00:40:32.807 ========== Backtrace start: ========== 00:52:28.664 00:40:32.807 00:52:28.664 00:40:32.807 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 00:52:28.664 00:40:32.807 ... 00:52:28.664 00:40:32.807 1117 timing_enter $test_name 00:52:28.664 00:40:32.807 1118 echo "************************************" 00:52:28.664 00:40:32.807 1119 echo "START TEST $test_name" 00:52:28.664 00:40:32.807 1120 echo "************************************" 00:52:28.664 00:40:32.807 1121 xtrace_restore 00:52:28.664 00:40:32.807 1122 time "$@" 00:52:28.664 00:40:32.807 1123 xtrace_disable 00:52:28.664 00:40:32.807 1124 echo "************************************" 00:52:28.664 00:40:32.807 1125 echo "END TEST $test_name" 00:52:28.664 00:40:32.807 1126 echo "************************************" 00:52:28.664 00:40:32.807 1127 timing_exit $test_name 00:52:28.664 00:40:32.807 ... 00:52:28.664 00:40:32.807 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:52:28.664 00:40:32.807 ... 00:52:28.664 00:40:32.807 235 00:52:28.664 00:40:32.807 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 00:52:28.664 00:40:32.807 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 00:52:28.664 00:40:32.807 238 00:52:28.664 00:40:32.807 239 if [ $(uname -s) = Linux ]; then 00:52:28.664 00:40:32.807 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 00:52:28.664 00:40:32.807 241 fi 00:52:28.664 00:40:32.807 242 00:52:28.664 00:40:32.807 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 00:52:28.664 00:40:32.807 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 00:52:28.664 00:40:32.807 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 00:52:28.665 00:40:32.807 ... 00:52:28.665 00:40:32.807 00:52:28.665 00:40:32.807 ========== Backtrace end ==========' 00:52:28.665 + [[ -n 00:40:32.547 06:11:27 sw_hotplug -- common/autotest_common.sh@715 -- # time='========== Backtrace start: ========== 00:52:28.665 00:40:32.547 00:52:28.665 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 710 00:52:28.665 00:40:32.547 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 00:52:28.665 00:40:32.547 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 00:52:28.665 00:40:32.547 713 # but $() will return just the time'\''s data, hence making it possible to just do: 00:52:28.665 00:40:32.547 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.665 00:40:32.547 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.665 00:40:32.547 716 00:52:28.665 00:40:32.547 717 echo "$time" 00:52:28.665 00:40:32.547 718 ) 00:52:28.665 00:40:32.547 719 00:52:28.665 00:40:32.547 720 function timing_enter() { 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 11 # Pci bus hotplug 00:52:28.665 00:40:32.547 12 # Helper function to remove/attach cotrollers 00:52:28.665 00:40:32.547 13 debug_remove_attach_helper() { 00:52:28.665 00:40:32.547 14 local helper_time=0 00:52:28.665 00:40:32.547 15 00:52:28.665 00:40:32.547 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.665 00:40:32.547 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.665 00:40:32.547 18 "$helper_time" "$nvme_count" >&2 00:52:28.665 00:40:32.547 19 } 00:52:28.665 00:40:32.547 20 00:52:28.665 00:40:32.547 21 remove_attach_helper() { 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.665 00:40:32.547 106 done 00:52:28.665 00:40:32.547 107 00:52:28.665 00:40:32.547 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.665 00:40:32.547 109 00:52:28.665 00:40:32.547 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.665 00:40:32.547 111 # Verify reregistering hotplug poller 00:52:28.665 00:40:32.547 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.665 00:40:32.547 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.665 00:40:32.547 114 00:52:28.665 00:40:32.547 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 133 00:52:28.665 00:40:32.547 134 # Run pci bus hotplug test 00:52:28.665 00:40:32.547 135 run_hotplug 00:52:28.665 00:40:32.547 136 00:52:28.665 00:40:32.547 137 # Run SPDK target based hotplug 00:52:28.665 00:40:32.547 => 138 tgt_run_hotplug 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 00:52:28.665 00:40:32.547 ========== Backtrace end ========== 00:52:28.665 00:40:32.547 ========== Backtrace start: ========== 00:52:28.665 00:40:32.547 00:52:28.665 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 710 00:52:28.665 00:40:32.547 711 # We redirect cmd's std{out,err} to a separate fd dup'ed to stdin's device (or /dev/null) to 00:52:28.665 00:40:32.547 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 00:52:28.665 00:40:32.547 713 # but $() will return just the time's data, hence making it possible to just do: 00:52:28.665 00:40:32.547 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.665 00:40:32.547 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.665 00:40:32.547 716 00:52:28.665 00:40:32.547 717 echo "$time" 00:52:28.665 00:40:32.547 718 ) 00:52:28.665 00:40:32.547 719 00:52:28.665 00:40:32.547 720 function timing_enter() { 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 11 # Pci bus hotplug 00:52:28.665 00:40:32.547 12 # Helper function to remove/attach cotrollers 00:52:28.665 00:40:32.547 13 debug_remove_attach_helper() { 00:52:28.665 00:40:32.547 14 local helper_time=0 00:52:28.665 00:40:32.547 15 00:52:28.665 00:40:32.547 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.665 00:40:32.547 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 00:52:28.665 00:40:32.547 18 "$helper_time" "$nvme_count" >&2 00:52:28.665 00:40:32.547 19 } 00:52:28.665 00:40:32.547 20 00:52:28.665 00:40:32.547 21 remove_attach_helper() { 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.665 00:40:32.547 106 done 00:52:28.665 00:40:32.547 107 00:52:28.665 00:40:32.547 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.665 00:40:32.547 109 00:52:28.665 00:40:32.547 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.665 00:40:32.547 111 # Verify reregistering hotplug poller 00:52:28.665 00:40:32.547 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.665 00:40:32.547 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.665 00:40:32.547 114 00:52:28.665 00:40:32.547 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 133 00:52:28.665 00:40:32.547 134 # Run pci bus hotplug test 00:52:28.665 00:40:32.547 135 run_hotplug 00:52:28.665 00:40:32.547 136 00:52:28.665 00:40:32.547 137 # Run SPDK target based hotplug 00:52:28.665 00:40:32.547 => 138 tgt_run_hotplug 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 00:52:28.665 00:40:32.547 ========== Backtrace end ========== 00:52:28.665 00:40:32.547 ========== Backtrace start: ========== 00:52:28.665 00:40:32.547 00:52:28.665 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 713 # but $() will return just the time's data, hence making it possible to just do: 00:52:28.665 00:40:32.547 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.665 00:40:32.547 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.665 00:40:32.547 716 00:52:28.665 00:40:32.547 717 echo "$time" 00:52:28.665 00:40:32.547 => 718 ) 00:52:28.665 00:40:32.547 719 00:52:28.665 00:40:32.547 720 function timing_enter() { 00:52:28.665 00:40:32.547 721 xtrace_disable 00:52:28.665 00:40:32.547 722 timing "enter" "$1" 00:52:28.665 00:40:32.547 723 xtrace_restore 00:52:28.665 00:40:32.547 ... 00:52:28.665 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.665 00:40:32.548 ... 00:52:28.665 00:40:32.548 11 # Pci bus hotplug 00:52:28.665 00:40:32.548 12 # Helper function to remove/attach cotrollers 00:52:28.665 00:40:32.548 13 debug_remove_attach_helper() { 00:52:28.665 00:40:32.548 14 local helper_time=0 00:52:28.665 00:40:32.548 15 00:52:28.665 00:40:32.548 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.665 00:40:32.548 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 00:52:28.665 00:40:32.548 18 "$helper_time" "$nvme_count" >&2 00:52:28.665 00:40:32.548 19 } 00:52:28.665 00:40:32.548 20 00:52:28.665 00:40:32.548 21 remove_attach_helper() { 00:52:28.665 00:40:32.548 ... 00:52:28.665 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.665 00:40:32.548 ... 00:52:28.665 00:40:32.548 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.665 00:40:32.548 106 done 00:52:28.665 00:40:32.548 107 00:52:28.665 00:40:32.548 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.665 00:40:32.548 109 00:52:28.665 00:40:32.548 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.665 00:40:32.548 111 # Verify reregistering hotplug poller 00:52:28.665 00:40:32.548 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.665 00:40:32.548 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.665 00:40:32.548 114 00:52:28.665 00:40:32.548 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.665 00:40:32.548 ... 00:52:28.665 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.665 00:40:32.548 ... 00:52:28.665 00:40:32.548 133 00:52:28.665 00:40:32.548 134 # Run pci bus hotplug test 00:52:28.665 00:40:32.548 135 run_hotplug 00:52:28.665 00:40:32.548 136 00:52:28.665 00:40:32.548 137 # Run SPDK target based hotplug 00:52:28.665 00:40:32.548 => 138 tgt_run_hotplug 00:52:28.665 00:40:32.548 ... 00:52:28.665 00:40:32.548 00:52:28.665 00:40:32.548 ========== Backtrace end ========== 00:52:28.665 00:40:32.548 ========== Backtrace start: ========== 00:52:28.665 00:40:32.548 00:52:28.665 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.665 00:40:32.548 ... 00:52:28.665 00:40:32.548 11 # Pci bus hotplug 00:52:28.665 00:40:32.548 12 # Helper function to remove/attach cotrollers 00:52:28.665 00:40:32.548 13 debug_remove_attach_helper() { 00:52:28.665 00:40:32.548 14 local helper_time=0 00:52:28.665 00:40:32.548 15 00:52:28.665 00:40:32.548 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.665 00:40:32.548 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 00:52:28.665 00:40:32.548 18 "$helper_time" "$nvme_count" >&2 00:52:28.665 00:40:32.548 19 } 00:52:28.665 00:40:32.548 20 00:52:28.665 00:40:32.548 21 remove_attach_helper() { 00:52:28.665 00:40:32.548 ... 00:52:28.666 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.666 00:40:32.548 ... 00:52:28.666 00:40:32.548 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.666 00:40:32.548 106 done 00:52:28.666 00:40:32.548 107 00:52:28.666 00:40:32.548 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.666 00:40:32.548 109 00:52:28.666 00:40:32.548 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.666 00:40:32.548 111 # Verify reregistering hotplug poller 00:52:28.666 00:40:32.548 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.666 00:40:32.548 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.666 00:40:32.548 114 00:52:28.666 00:40:32.548 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.666 00:40:32.548 ... 00:52:28.666 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.666 00:40:32.548 ... 00:52:28.666 00:40:32.548 133 00:52:28.666 00:40:32.548 134 # Run pci bus hotplug test 00:52:28.666 00:40:32.548 135 run_hotplug 00:52:28.666 00:40:32.548 136 00:52:28.666 00:40:32.548 137 # Run SPDK target based hotplug 00:52:28.666 00:40:32.548 => 138 tgt_run_hotplug 00:52:28.666 00:40:32.548 ... 00:52:28.666 00:40:32.548 00:52:28.666 00:40:32.548 ========== Backtrace end ========== 00:52:28.666 00:40:32.807 ========== Backtrace start: ========== 00:52:28.666 00:40:32.807 00:52:28.666 00:40:32.807 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 00:52:28.666 00:40:32.807 ... 00:52:28.666 00:40:32.807 1117 timing_enter $test_name 00:52:28.666 00:40:32.807 1118 echo "************************************" 00:52:28.666 00:40:32.807 1119 echo "START TEST $test_name" 00:52:28.666 00:40:32.807 1120 echo "************************************" 00:52:28.666 00:40:32.807 1121 xtrace_restore 00:52:28.666 00:40:32.807 1122 time "$@" 00:52:28.666 00:40:32.807 1123 xtrace_disable 00:52:28.666 00:40:32.807 1124 echo "************************************" 00:52:28.666 00:40:32.807 1125 echo "END TEST $test_name" 00:52:28.666 00:40:32.807 1126 echo "************************************" 00:52:28.666 00:40:32.807 1127 timing_exit $test_name 00:52:28.666 00:40:32.807 ... 00:52:28.666 00:40:32.807 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:52:28.666 00:40:32.807 ... 00:52:28.666 00:40:32.807 235 00:52:28.666 00:40:32.807 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 00:52:28.666 00:40:32.807 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 00:52:28.666 00:40:32.807 238 00:52:28.666 00:40:32.807 239 if [ $(uname -s) = Linux ]; then 00:52:28.666 00:40:32.807 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 00:52:28.666 00:40:32.807 241 fi 00:52:28.666 00:40:32.807 242 00:52:28.666 00:40:32.807 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 00:52:28.666 00:40:32.807 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 00:52:28.666 00:40:32.807 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 00:52:28.666 00:40:32.807 ... 00:52:28.666 00:40:32.807 00:52:28.666 00:40:32.807 ========== Backtrace end ========== ]] 00:52:28.666 + echo '00:40:32.547 06:11:27 sw_hotplug -- common/autotest_common.sh@715 -- # time='\''========== Backtrace start: ========== 00:52:28.666 00:40:32.547 00:52:28.666 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 710 00:52:28.666 00:40:32.547 711 # We redirect cmd'\''\'\'''\''s std{out,err} to a separate fd dup'\''\'\'''\''ed to stdin'\''\'\'''\''s device (or /dev/null) to 00:52:28.666 00:40:32.547 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 00:52:28.666 00:40:32.547 713 # but $() will return just the time'\''\'\'''\''s data, hence making it possible to just do: 00:52:28.666 00:40:32.547 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.666 00:40:32.547 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.666 00:40:32.547 716 00:52:28.666 00:40:32.547 717 echo "$time" 00:52:28.666 00:40:32.547 718 ) 00:52:28.666 00:40:32.547 719 00:52:28.666 00:40:32.547 720 function timing_enter() { 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 11 # Pci bus hotplug 00:52:28.666 00:40:32.547 12 # Helper function to remove/attach cotrollers 00:52:28.666 00:40:32.547 13 debug_remove_attach_helper() { 00:52:28.666 00:40:32.547 14 local helper_time=0 00:52:28.666 00:40:32.547 15 00:52:28.666 00:40:32.547 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.666 00:40:32.547 17 printf '\''\'\'''\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\''\'\'''\'' \ 00:52:28.666 00:40:32.547 18 "$helper_time" "$nvme_count" >&2 00:52:28.666 00:40:32.547 19 } 00:52:28.666 00:40:32.547 20 00:52:28.666 00:40:32.547 21 remove_attach_helper() { 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.666 00:40:32.547 106 done 00:52:28.666 00:40:32.547 107 00:52:28.666 00:40:32.547 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.666 00:40:32.547 109 00:52:28.666 00:40:32.547 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.666 00:40:32.547 111 # Verify reregistering hotplug poller 00:52:28.666 00:40:32.547 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.666 00:40:32.547 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.666 00:40:32.547 114 00:52:28.666 00:40:32.547 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 133 00:52:28.666 00:40:32.547 134 # Run pci bus hotplug test 00:52:28.666 00:40:32.547 135 run_hotplug 00:52:28.666 00:40:32.547 136 00:52:28.666 00:40:32.547 137 # Run SPDK target based hotplug 00:52:28.666 00:40:32.547 => 138 tgt_run_hotplug 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 00:52:28.666 00:40:32.547 ========== Backtrace end ========== 00:52:28.666 00:40:32.547 ========== Backtrace start: ========== 00:52:28.666 00:40:32.547 00:52:28.666 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 710 00:52:28.666 00:40:32.547 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 00:52:28.666 00:40:32.547 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 00:52:28.666 00:40:32.547 713 # but $() will return just the time'\''s data, hence making it possible to just do: 00:52:28.666 00:40:32.547 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.666 00:40:32.547 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.666 00:40:32.547 716 00:52:28.666 00:40:32.547 717 echo "$time" 00:52:28.666 00:40:32.547 718 ) 00:52:28.666 00:40:32.547 719 00:52:28.666 00:40:32.547 720 function timing_enter() { 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 11 # Pci bus hotplug 00:52:28.666 00:40:32.547 12 # Helper function to remove/attach cotrollers 00:52:28.666 00:40:32.547 13 debug_remove_attach_helper() { 00:52:28.666 00:40:32.547 14 local helper_time=0 00:52:28.666 00:40:32.547 15 00:52:28.666 00:40:32.547 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.666 00:40:32.547 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.666 00:40:32.547 18 "$helper_time" "$nvme_count" >&2 00:52:28.666 00:40:32.547 19 } 00:52:28.666 00:40:32.547 20 00:52:28.666 00:40:32.547 21 remove_attach_helper() { 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.666 00:40:32.547 106 done 00:52:28.666 00:40:32.547 107 00:52:28.666 00:40:32.547 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.666 00:40:32.547 109 00:52:28.666 00:40:32.547 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.666 00:40:32.547 111 # Verify reregistering hotplug poller 00:52:28.666 00:40:32.547 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.666 00:40:32.547 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.666 00:40:32.547 114 00:52:28.666 00:40:32.547 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 133 00:52:28.666 00:40:32.547 134 # Run pci bus hotplug test 00:52:28.666 00:40:32.547 135 run_hotplug 00:52:28.666 00:40:32.547 136 00:52:28.666 00:40:32.547 137 # Run SPDK target based hotplug 00:52:28.666 00:40:32.547 => 138 tgt_run_hotplug 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 00:52:28.666 00:40:32.547 ========== Backtrace end ========== 00:52:28.666 00:40:32.547 ========== Backtrace start: ========== 00:52:28.666 00:40:32.547 00:52:28.666 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.666 00:40:32.547 ... 00:52:28.666 00:40:32.547 713 # but $() will return just the time'\''s data, hence making it possible to just do: 00:52:28.666 00:40:32.547 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.666 00:40:32.547 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.666 00:40:32.547 716 00:52:28.666 00:40:32.547 717 echo "$time" 00:52:28.666 00:40:32.547 => 718 ) 00:52:28.666 00:40:32.547 719 00:52:28.666 00:40:32.547 720 function timing_enter() { 00:52:28.666 00:40:32.547 721 xtrace_disable 00:52:28.666 00:40:32.547 722 timing "enter" "$1" 00:52:28.666 00:40:32.547 723 xtrace_restore 00:52:28.667 00:40:32.547 ... 00:52:28.667 00:40:32.547 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.667 00:40:32.548 ... 00:52:28.667 00:40:32.548 11 # Pci bus hotplug 00:52:28.667 00:40:32.548 12 # Helper function to remove/attach cotrollers 00:52:28.667 00:40:32.548 13 debug_remove_attach_helper() { 00:52:28.667 00:40:32.548 14 local helper_time=0 00:52:28.667 00:40:32.548 15 00:52:28.667 00:40:32.548 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.667 00:40:32.548 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.667 00:40:32.548 18 "$helper_time" "$nvme_count" >&2 00:52:28.667 00:40:32.548 19 } 00:52:28.667 00:40:32.548 20 00:52:28.667 00:40:32.548 21 remove_attach_helper() { 00:52:28.667 00:40:32.548 ... 00:52:28.667 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.667 00:40:32.548 ... 00:52:28.667 00:40:32.548 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.667 00:40:32.548 106 done 00:52:28.667 00:40:32.548 107 00:52:28.667 00:40:32.548 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.667 00:40:32.548 109 00:52:28.667 00:40:32.548 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.667 00:40:32.548 111 # Verify reregistering hotplug poller 00:52:28.667 00:40:32.548 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.667 00:40:32.548 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.667 00:40:32.548 114 00:52:28.667 00:40:32.548 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.667 00:40:32.548 ... 00:52:28.667 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.667 00:40:32.548 ... 00:52:28.667 00:40:32.548 133 00:52:28.667 00:40:32.548 134 # Run pci bus hotplug test 00:52:28.667 00:40:32.548 135 run_hotplug 00:52:28.667 00:40:32.548 136 00:52:28.667 00:40:32.548 137 # Run SPDK target based hotplug 00:52:28.667 00:40:32.548 => 138 tgt_run_hotplug 00:52:28.667 00:40:32.548 ... 00:52:28.667 00:40:32.548 00:52:28.667 00:40:32.548 ========== Backtrace end ========== 00:52:28.667 00:40:32.548 ========== Backtrace start: ========== 00:52:28.667 00:40:32.548 00:52:28.667 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.667 00:40:32.548 ... 00:52:28.667 00:40:32.548 11 # Pci bus hotplug 00:52:28.667 00:40:32.548 12 # Helper function to remove/attach cotrollers 00:52:28.667 00:40:32.548 13 debug_remove_attach_helper() { 00:52:28.667 00:40:32.548 14 local helper_time=0 00:52:28.667 00:40:32.548 15 00:52:28.667 00:40:32.548 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.667 00:40:32.548 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.667 00:40:32.548 18 "$helper_time" "$nvme_count" >&2 00:52:28.667 00:40:32.548 19 } 00:52:28.667 00:40:32.548 20 00:52:28.667 00:40:32.548 21 remove_attach_helper() { 00:52:28.667 00:40:32.548 ... 00:52:28.667 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.667 00:40:32.548 ... 00:52:28.667 00:40:32.548 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.667 00:40:32.548 106 done 00:52:28.667 00:40:32.548 107 00:52:28.667 00:40:32.548 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.667 00:40:32.548 109 00:52:28.667 00:40:32.548 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.667 00:40:32.548 111 # Verify reregistering hotplug poller 00:52:28.667 00:40:32.548 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.667 00:40:32.548 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.667 00:40:32.548 114 00:52:28.667 00:40:32.548 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.667 00:40:32.548 ... 00:52:28.667 00:40:32.548 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.667 00:40:32.548 ... 00:52:28.667 00:40:32.548 133 00:52:28.667 00:40:32.548 134 # Run pci bus hotplug test 00:52:28.667 00:40:32.548 135 run_hotplug 00:52:28.667 00:40:32.548 136 00:52:28.667 00:40:32.548 137 # Run SPDK target based hotplug 00:52:28.667 00:40:32.548 => 138 tgt_run_hotplug 00:52:28.667 00:40:32.548 ... 00:52:28.667 00:40:32.548 00:52:28.667 00:40:32.548 ========== Backtrace end ========== 00:52:28.667 00:40:32.807 ========== Backtrace start: ========== 00:52:28.667 00:40:32.807 00:52:28.667 00:40:32.807 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 00:52:28.667 00:40:32.807 ... 00:52:28.667 00:40:32.807 1117 timing_enter $test_name 00:52:28.667 00:40:32.807 1118 echo "************************************" 00:52:28.667 00:40:32.807 1119 echo "START TEST $test_name" 00:52:28.667 00:40:32.807 1120 echo "************************************" 00:52:28.667 00:40:32.807 1121 xtrace_restore 00:52:28.667 00:40:32.807 1122 time "$@" 00:52:28.667 00:40:32.807 1123 xtrace_disable 00:52:28.667 00:40:32.807 1124 echo "************************************" 00:52:28.667 00:40:32.807 1125 echo "END TEST $test_name" 00:52:28.667 00:40:32.807 1126 echo "************************************" 00:52:28.667 00:40:32.807 1127 timing_exit $test_name 00:52:28.667 00:40:32.807 ... 00:52:28.667 00:40:32.807 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:52:28.667 00:40:32.807 ... 00:52:28.667 00:40:32.807 235 00:52:28.667 00:40:32.807 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 00:52:28.667 00:40:32.807 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 00:52:28.667 00:40:32.807 238 00:52:28.667 00:40:32.807 239 if [ $(uname -s) = Linux ]; then 00:52:28.667 00:40:32.807 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 00:52:28.667 00:40:32.807 241 fi 00:52:28.667 00:40:32.807 242 00:52:28.667 00:40:32.807 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 00:52:28.667 00:40:32.807 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 00:52:28.667 00:40:32.807 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 00:52:28.667 00:40:32.807 ... 00:52:28.667 00:40:32.807 00:52:28.667 00:40:32.807 ========== Backtrace end ==========' 00:52:28.667 + for build in "${build_log_list[@]}" 00:52:28.667 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-uring-vg-autotest_77385/build.log 00:52:28.667 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.667 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-uring-vg-autotest_77385/build.log 00:52:28.667 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-uring-vg-autotest_77385/build.log 00:52:28.667 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-uring-vg-autotest_77385 00:52:28.667 + job_name=nvmf-tcp-uring-vg-autotest_77385 00:52:28.667 + start_pattern='Backtrace start' 00:52:28.667 + end_pattern='Backtrace end' 00:52:28.667 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-uring-vg-autotest_77385/build.log ]] 00:52:28.667 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-uring-vg-autotest_77385/build.log 00:52:28.667 + backtraces='00:22:40.266 ========== Backtrace start: ========== 00:52:28.667 00:22:40.266 00:52:28.667 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1349 -> fio_plugin(["/home/vagrant/spdk_repo/spdk/build/fio/spdk_bdev"],["--ioengine=spdk_bdev"],["--spdk_json_conf"],["/dev/fd/62"],["/dev/fd/61"]) 00:52:28.667 00:22:40.266 ... 00:52:28.667 00:22:40.266 1344 break 00:52:28.667 00:22:40.266 1345 fi 00:52:28.667 00:22:40.266 1346 done 00:52:28.667 00:22:40.266 1347 00:52:28.667 00:22:40.266 1348 # Preload the sanitizer library to fio if fio_plugin was compiled with it 00:52:28.667 00:22:40.266 1349 LD_PRELOAD="$asan_lib $plugin" "$fio_dir"/fio "$@" 00:52:28.667 00:22:40.266 1350 } 00:52:28.667 00:22:40.266 1351 00:52:28.667 00:22:40.266 1352 function fio_bdev() { 00:52:28.667 00:22:40.266 1353 fio_plugin "$rootdir/build/fio/spdk_bdev" "$@" 00:52:28.667 00:22:40.266 1354 } 00:52:28.667 00:22:40.266 ... 00:52:28.667 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1353 -> fio_bdev(["--ioengine=spdk_bdev"],["--spdk_json_conf"],["/dev/fd/62"],["/dev/fd/61"]) 00:52:28.667 00:22:40.266 ... 00:52:28.667 00:22:40.266 1348 # Preload the sanitizer library to fio if fio_plugin was compiled with it 00:52:28.667 00:22:40.266 1349 LD_PRELOAD="$asan_lib $plugin" "$fio_dir"/fio "$@" 00:52:28.667 00:22:40.266 1350 } 00:52:28.667 00:22:40.266 1351 00:52:28.667 00:22:40.266 1352 function fio_bdev() { 00:52:28.667 00:22:40.266 1353 fio_plugin "$rootdir/build/fio/spdk_bdev" "$@" 00:52:28.667 00:22:40.266 1354 } 00:52:28.667 00:22:40.266 1355 00:52:28.667 00:22:40.266 1356 function fio_nvme() { 00:52:28.667 00:22:40.266 1357 fio_plugin "$rootdir/build/fio/spdk_nvme" "$@" 00:52:28.667 00:22:40.266 1358 } 00:52:28.667 00:22:40.266 ... 00:52:28.667 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/nvmf/target/dif.sh:82 -> fio(["/dev/fd/62"]) 00:52:28.667 00:22:40.266 ... 00:52:28.667 00:22:40.266 77 FIO 00:52:28.667 00:22:40.266 78 done 00:52:28.667 00:22:40.266 79 } 00:52:28.667 00:22:40.266 80 00:52:28.667 00:22:40.266 81 fio() { 00:52:28.667 00:22:40.266 => 82 fio_bdev --ioengine=spdk_bdev --spdk_json_conf "$@" <(gen_fio_conf) 00:52:28.667 00:22:40.266 83 } 00:52:28.667 00:22:40.266 84 00:52:28.667 00:22:40.266 85 fio_dif_1() { 00:52:28.667 00:22:40.266 86 create_subsystems 0 00:52:28.667 00:22:40.266 87 fio <(create_json_sub_conf 0) 00:52:28.667 00:22:40.266 ... 00:52:28.667 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/nvmf/target/dif.sh:112 -> fio_dif_rand_params([]) 00:52:28.667 00:22:40.266 ... 00:52:28.667 00:22:40.266 107 destroy_subsystems 0 00:52:28.667 00:22:40.266 108 00:52:28.667 00:22:40.266 109 NULL_DIF=2 bs=4k numjobs=8 iodepth=16 runtime="" files=2 00:52:28.667 00:22:40.266 110 00:52:28.667 00:22:40.266 111 create_subsystems 0 1 2 00:52:28.667 00:22:40.266 => 112 fio <(create_json_sub_conf 0 1 2) 00:52:28.667 00:22:40.266 113 destroy_subsystems 0 1 2 00:52:28.667 00:22:40.266 114 00:52:28.668 00:22:40.266 115 NULL_DIF=1 bs=8k,16k,128k numjobs=2 iodepth=8 runtime=5 files=1 00:52:28.668 00:22:40.266 116 00:52:28.668 00:22:40.266 117 create_subsystems 0 1 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["fio_dif_rand_params"],["fio_dif_rand_params"]) 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 1117 timing_enter $test_name 00:52:28.668 00:22:40.266 1118 echo "************************************" 00:52:28.668 00:22:40.266 1119 echo "START TEST $test_name" 00:52:28.668 00:22:40.266 1120 echo "************************************" 00:52:28.668 00:22:40.266 1121 xtrace_restore 00:52:28.668 00:22:40.266 1122 time "$@" 00:52:28.668 00:22:40.266 1123 xtrace_disable 00:52:28.668 00:22:40.266 1124 echo "************************************" 00:52:28.668 00:22:40.266 1125 echo "END TEST $test_name" 00:52:28.668 00:22:40.266 1126 echo "************************************" 00:52:28.668 00:22:40.266 1127 timing_exit $test_name 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/nvmf/target/dif.sh:143 -> main(["--transport=tcp"],["--iso"]) 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 138 00:52:28.668 00:22:40.266 139 create_transport 00:52:28.668 00:22:40.266 140 00:52:28.668 00:22:40.266 141 run_test "fio_dif_1_default" fio_dif_1 00:52:28.668 00:22:40.266 142 run_test "fio_dif_1_multi_subsystems" fio_dif_1_multi_subsystems 00:52:28.668 00:22:40.266 => 143 run_test "fio_dif_rand_params" fio_dif_rand_params 00:52:28.668 00:22:40.266 144 run_test "fio_dif_digest" fio_dif_digest 00:52:28.668 00:22:40.266 145 00:52:28.668 00:22:40.266 146 trap - SIGINT SIGTERM EXIT 00:52:28.668 00:22:40.266 147 nvmftestfini 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 00:52:28.668 00:22:40.266 ========== Backtrace end ========== 00:52:28.668 00:22:41.301 ========== Backtrace start: ========== 00:52:28.668 00:22:41.301 00:52:28.668 00:22:41.301 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["nvmf_dif"],["/home/vagrant/spdk_repo/spdk/test/nvmf/target/dif.sh"]) 00:52:28.668 00:22:41.301 ... 00:52:28.668 00:22:41.301 1117 timing_enter $test_name 00:52:28.668 00:22:41.301 1118 echo "************************************" 00:52:28.668 00:22:41.301 1119 echo "START TEST $test_name" 00:52:28.668 00:22:41.301 1120 echo "************************************" 00:52:28.668 00:22:41.301 1121 xtrace_restore 00:52:28.668 00:22:41.301 1122 time "$@" 00:52:28.668 00:22:41.301 1123 xtrace_disable 00:52:28.668 00:22:41.301 1124 echo "************************************" 00:52:28.668 00:22:41.301 1125 echo "END TEST $test_name" 00:52:28.668 00:22:41.301 1126 echo "************************************" 00:52:28.668 00:22:41.301 1127 timing_exit $test_name 00:52:28.668 00:22:41.301 ... 00:52:28.668 00:22:41.301 in /home/vagrant/spdk_repo/spdk/autotest.sh:288 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:52:28.668 00:22:41.301 ... 00:52:28.668 00:22:41.301 283 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.668 00:22:41.301 284 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:52:28.668 00:22:41.301 285 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.668 00:22:41.301 286 run_test "nvmf_identify_passthru" $rootdir/test/nvmf/target/identify_passthru.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.668 00:22:41.301 287 fi 00:52:28.668 00:22:41.301 => 288 run_test "nvmf_dif" $rootdir/test/nvmf/target/dif.sh 00:52:28.668 00:22:41.301 289 run_test "nvmf_abort_qd_sizes" $rootdir/test/nvmf/target/abort_qd_sizes.sh 00:52:28.668 00:22:41.301 290 # The keyring tests utilize NVMe/TLS 00:52:28.668 00:22:41.301 291 run_test "keyring_file" "$rootdir/test/keyring/file.sh" 00:52:28.668 00:22:41.301 292 if [[ "$CONFIG_HAVE_KEYUTILS" == y ]]; then 00:52:28.668 00:22:41.301 293 run_test "keyring_linux" "$rootdir/test/keyring/linux.sh" 00:52:28.668 00:22:41.301 ... 00:52:28.668 00:22:41.301 00:52:28.668 00:22:41.301 ========== Backtrace end ==========' 00:52:28.668 + [[ -n 00:22:40.266 ========== Backtrace start: ========== 00:52:28.668 00:22:40.266 00:52:28.668 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1349 -> fio_plugin(["/home/vagrant/spdk_repo/spdk/build/fio/spdk_bdev"],["--ioengine=spdk_bdev"],["--spdk_json_conf"],["/dev/fd/62"],["/dev/fd/61"]) 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 1344 break 00:52:28.668 00:22:40.266 1345 fi 00:52:28.668 00:22:40.266 1346 done 00:52:28.668 00:22:40.266 1347 00:52:28.668 00:22:40.266 1348 # Preload the sanitizer library to fio if fio_plugin was compiled with it 00:52:28.668 00:22:40.266 1349 LD_PRELOAD="$asan_lib $plugin" "$fio_dir"/fio "$@" 00:52:28.668 00:22:40.266 1350 } 00:52:28.668 00:22:40.266 1351 00:52:28.668 00:22:40.266 1352 function fio_bdev() { 00:52:28.668 00:22:40.266 1353 fio_plugin "$rootdir/build/fio/spdk_bdev" "$@" 00:52:28.668 00:22:40.266 1354 } 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1353 -> fio_bdev(["--ioengine=spdk_bdev"],["--spdk_json_conf"],["/dev/fd/62"],["/dev/fd/61"]) 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 1348 # Preload the sanitizer library to fio if fio_plugin was compiled with it 00:52:28.668 00:22:40.266 1349 LD_PRELOAD="$asan_lib $plugin" "$fio_dir"/fio "$@" 00:52:28.668 00:22:40.266 1350 } 00:52:28.668 00:22:40.266 1351 00:52:28.668 00:22:40.266 1352 function fio_bdev() { 00:52:28.668 00:22:40.266 1353 fio_plugin "$rootdir/build/fio/spdk_bdev" "$@" 00:52:28.668 00:22:40.266 1354 } 00:52:28.668 00:22:40.266 1355 00:52:28.668 00:22:40.266 1356 function fio_nvme() { 00:52:28.668 00:22:40.266 1357 fio_plugin "$rootdir/build/fio/spdk_nvme" "$@" 00:52:28.668 00:22:40.266 1358 } 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/nvmf/target/dif.sh:82 -> fio(["/dev/fd/62"]) 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 77 FIO 00:52:28.668 00:22:40.266 78 done 00:52:28.668 00:22:40.266 79 } 00:52:28.668 00:22:40.266 80 00:52:28.668 00:22:40.266 81 fio() { 00:52:28.668 00:22:40.266 => 82 fio_bdev --ioengine=spdk_bdev --spdk_json_conf "$@" <(gen_fio_conf) 00:52:28.668 00:22:40.266 83 } 00:52:28.668 00:22:40.266 84 00:52:28.668 00:22:40.266 85 fio_dif_1() { 00:52:28.668 00:22:40.266 86 create_subsystems 0 00:52:28.668 00:22:40.266 87 fio <(create_json_sub_conf 0) 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/nvmf/target/dif.sh:112 -> fio_dif_rand_params([]) 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 107 destroy_subsystems 0 00:52:28.668 00:22:40.266 108 00:52:28.668 00:22:40.266 109 NULL_DIF=2 bs=4k numjobs=8 iodepth=16 runtime="" files=2 00:52:28.668 00:22:40.266 110 00:52:28.668 00:22:40.266 111 create_subsystems 0 1 2 00:52:28.668 00:22:40.266 => 112 fio <(create_json_sub_conf 0 1 2) 00:52:28.668 00:22:40.266 113 destroy_subsystems 0 1 2 00:52:28.668 00:22:40.266 114 00:52:28.668 00:22:40.266 115 NULL_DIF=1 bs=8k,16k,128k numjobs=2 iodepth=8 runtime=5 files=1 00:52:28.668 00:22:40.266 116 00:52:28.668 00:22:40.266 117 create_subsystems 0 1 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["fio_dif_rand_params"],["fio_dif_rand_params"]) 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 1117 timing_enter $test_name 00:52:28.668 00:22:40.266 1118 echo "************************************" 00:52:28.668 00:22:40.266 1119 echo "START TEST $test_name" 00:52:28.668 00:22:40.266 1120 echo "************************************" 00:52:28.668 00:22:40.266 1121 xtrace_restore 00:52:28.668 00:22:40.266 1122 time "$@" 00:52:28.668 00:22:40.266 1123 xtrace_disable 00:52:28.668 00:22:40.266 1124 echo "************************************" 00:52:28.668 00:22:40.266 1125 echo "END TEST $test_name" 00:52:28.668 00:22:40.266 1126 echo "************************************" 00:52:28.668 00:22:40.266 1127 timing_exit $test_name 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/nvmf/target/dif.sh:143 -> main(["--transport=tcp"],["--iso"]) 00:52:28.668 00:22:40.266 ... 00:52:28.668 00:22:40.266 138 00:52:28.669 00:22:40.266 139 create_transport 00:52:28.669 00:22:40.266 140 00:52:28.669 00:22:40.266 141 run_test "fio_dif_1_default" fio_dif_1 00:52:28.669 00:22:40.266 142 run_test "fio_dif_1_multi_subsystems" fio_dif_1_multi_subsystems 00:52:28.669 00:22:40.266 => 143 run_test "fio_dif_rand_params" fio_dif_rand_params 00:52:28.669 00:22:40.266 144 run_test "fio_dif_digest" fio_dif_digest 00:52:28.669 00:22:40.266 145 00:52:28.669 00:22:40.266 146 trap - SIGINT SIGTERM EXIT 00:52:28.669 00:22:40.266 147 nvmftestfini 00:52:28.669 00:22:40.266 ... 00:52:28.669 00:22:40.266 00:52:28.669 00:22:40.266 ========== Backtrace end ========== 00:52:28.669 00:22:41.301 ========== Backtrace start: ========== 00:52:28.669 00:22:41.301 00:52:28.669 00:22:41.301 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["nvmf_dif"],["/home/vagrant/spdk_repo/spdk/test/nvmf/target/dif.sh"]) 00:52:28.669 00:22:41.301 ... 00:52:28.669 00:22:41.301 1117 timing_enter $test_name 00:52:28.669 00:22:41.301 1118 echo "************************************" 00:52:28.669 00:22:41.301 1119 echo "START TEST $test_name" 00:52:28.669 00:22:41.301 1120 echo "************************************" 00:52:28.669 00:22:41.301 1121 xtrace_restore 00:52:28.669 00:22:41.301 1122 time "$@" 00:52:28.669 00:22:41.301 1123 xtrace_disable 00:52:28.669 00:22:41.301 1124 echo "************************************" 00:52:28.669 00:22:41.301 1125 echo "END TEST $test_name" 00:52:28.669 00:22:41.301 1126 echo "************************************" 00:52:28.669 00:22:41.301 1127 timing_exit $test_name 00:52:28.669 00:22:41.301 ... 00:52:28.669 00:22:41.301 in /home/vagrant/spdk_repo/spdk/autotest.sh:288 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:52:28.669 00:22:41.301 ... 00:52:28.669 00:22:41.301 283 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.669 00:22:41.301 284 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:52:28.669 00:22:41.301 285 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.669 00:22:41.301 286 run_test "nvmf_identify_passthru" $rootdir/test/nvmf/target/identify_passthru.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.669 00:22:41.301 287 fi 00:52:28.669 00:22:41.301 => 288 run_test "nvmf_dif" $rootdir/test/nvmf/target/dif.sh 00:52:28.669 00:22:41.301 289 run_test "nvmf_abort_qd_sizes" $rootdir/test/nvmf/target/abort_qd_sizes.sh 00:52:28.669 00:22:41.301 290 # The keyring tests utilize NVMe/TLS 00:52:28.669 00:22:41.301 291 run_test "keyring_file" "$rootdir/test/keyring/file.sh" 00:52:28.669 00:22:41.301 292 if [[ "$CONFIG_HAVE_KEYUTILS" == y ]]; then 00:52:28.669 00:22:41.301 293 run_test "keyring_linux" "$rootdir/test/keyring/linux.sh" 00:52:28.669 00:22:41.301 ... 00:52:28.669 00:22:41.301 00:52:28.669 00:22:41.301 ========== Backtrace end ========== ]] 00:52:28.669 + echo '00:22:40.266 ========== Backtrace start: ========== 00:52:28.669 00:22:40.266 00:52:28.669 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1349 -> fio_plugin(["/home/vagrant/spdk_repo/spdk/build/fio/spdk_bdev"],["--ioengine=spdk_bdev"],["--spdk_json_conf"],["/dev/fd/62"],["/dev/fd/61"]) 00:52:28.669 00:22:40.266 ... 00:52:28.669 00:22:40.266 1344 break 00:52:28.669 00:22:40.266 1345 fi 00:52:28.669 00:22:40.266 1346 done 00:52:28.669 00:22:40.266 1347 00:52:28.669 00:22:40.266 1348 # Preload the sanitizer library to fio if fio_plugin was compiled with it 00:52:28.669 00:22:40.266 1349 LD_PRELOAD="$asan_lib $plugin" "$fio_dir"/fio "$@" 00:52:28.669 00:22:40.266 1350 } 00:52:28.669 00:22:40.266 1351 00:52:28.669 00:22:40.266 1352 function fio_bdev() { 00:52:28.669 00:22:40.266 1353 fio_plugin "$rootdir/build/fio/spdk_bdev" "$@" 00:52:28.669 00:22:40.266 1354 } 00:52:28.669 00:22:40.266 ... 00:52:28.669 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1353 -> fio_bdev(["--ioengine=spdk_bdev"],["--spdk_json_conf"],["/dev/fd/62"],["/dev/fd/61"]) 00:52:28.669 00:22:40.266 ... 00:52:28.669 00:22:40.266 1348 # Preload the sanitizer library to fio if fio_plugin was compiled with it 00:52:28.669 00:22:40.266 1349 LD_PRELOAD="$asan_lib $plugin" "$fio_dir"/fio "$@" 00:52:28.669 00:22:40.266 1350 } 00:52:28.669 00:22:40.266 1351 00:52:28.669 00:22:40.266 1352 function fio_bdev() { 00:52:28.669 00:22:40.266 1353 fio_plugin "$rootdir/build/fio/spdk_bdev" "$@" 00:52:28.669 00:22:40.266 1354 } 00:52:28.669 00:22:40.266 1355 00:52:28.669 00:22:40.266 1356 function fio_nvme() { 00:52:28.669 00:22:40.266 1357 fio_plugin "$rootdir/build/fio/spdk_nvme" "$@" 00:52:28.669 00:22:40.266 1358 } 00:52:28.669 00:22:40.266 ... 00:52:28.669 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/nvmf/target/dif.sh:82 -> fio(["/dev/fd/62"]) 00:52:28.669 00:22:40.266 ... 00:52:28.669 00:22:40.266 77 FIO 00:52:28.669 00:22:40.266 78 done 00:52:28.669 00:22:40.266 79 } 00:52:28.669 00:22:40.266 80 00:52:28.669 00:22:40.266 81 fio() { 00:52:28.669 00:22:40.266 => 82 fio_bdev --ioengine=spdk_bdev --spdk_json_conf "$@" <(gen_fio_conf) 00:52:28.669 00:22:40.266 83 } 00:52:28.669 00:22:40.266 84 00:52:28.669 00:22:40.266 85 fio_dif_1() { 00:52:28.669 00:22:40.266 86 create_subsystems 0 00:52:28.669 00:22:40.266 87 fio <(create_json_sub_conf 0) 00:52:28.669 00:22:40.266 ... 00:52:28.669 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/nvmf/target/dif.sh:112 -> fio_dif_rand_params([]) 00:52:28.669 00:22:40.266 ... 00:52:28.669 00:22:40.266 107 destroy_subsystems 0 00:52:28.669 00:22:40.266 108 00:52:28.669 00:22:40.266 109 NULL_DIF=2 bs=4k numjobs=8 iodepth=16 runtime="" files=2 00:52:28.669 00:22:40.266 110 00:52:28.669 00:22:40.266 111 create_subsystems 0 1 2 00:52:28.669 00:22:40.266 => 112 fio <(create_json_sub_conf 0 1 2) 00:52:28.669 00:22:40.266 113 destroy_subsystems 0 1 2 00:52:28.669 00:22:40.266 114 00:52:28.669 00:22:40.266 115 NULL_DIF=1 bs=8k,16k,128k numjobs=2 iodepth=8 runtime=5 files=1 00:52:28.669 00:22:40.266 116 00:52:28.669 00:22:40.266 117 create_subsystems 0 1 00:52:28.669 00:22:40.266 ... 00:52:28.669 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["fio_dif_rand_params"],["fio_dif_rand_params"]) 00:52:28.669 00:22:40.266 ... 00:52:28.669 00:22:40.266 1117 timing_enter $test_name 00:52:28.669 00:22:40.266 1118 echo "************************************" 00:52:28.669 00:22:40.266 1119 echo "START TEST $test_name" 00:52:28.669 00:22:40.266 1120 echo "************************************" 00:52:28.669 00:22:40.266 1121 xtrace_restore 00:52:28.669 00:22:40.266 1122 time "$@" 00:52:28.669 00:22:40.266 1123 xtrace_disable 00:52:28.669 00:22:40.266 1124 echo "************************************" 00:52:28.669 00:22:40.266 1125 echo "END TEST $test_name" 00:52:28.669 00:22:40.266 1126 echo "************************************" 00:52:28.669 00:22:40.266 1127 timing_exit $test_name 00:52:28.669 00:22:40.266 ... 00:52:28.669 00:22:40.266 in /home/vagrant/spdk_repo/spdk/test/nvmf/target/dif.sh:143 -> main(["--transport=tcp"],["--iso"]) 00:52:28.669 00:22:40.266 ... 00:52:28.669 00:22:40.266 138 00:52:28.669 00:22:40.266 139 create_transport 00:52:28.669 00:22:40.266 140 00:52:28.669 00:22:40.266 141 run_test "fio_dif_1_default" fio_dif_1 00:52:28.669 00:22:40.266 142 run_test "fio_dif_1_multi_subsystems" fio_dif_1_multi_subsystems 00:52:28.669 00:22:40.266 => 143 run_test "fio_dif_rand_params" fio_dif_rand_params 00:52:28.669 00:22:40.266 144 run_test "fio_dif_digest" fio_dif_digest 00:52:28.669 00:22:40.266 145 00:52:28.669 00:22:40.266 146 trap - SIGINT SIGTERM EXIT 00:52:28.669 00:22:40.266 147 nvmftestfini 00:52:28.669 00:22:40.266 ... 00:52:28.669 00:22:40.266 00:52:28.669 00:22:40.266 ========== Backtrace end ========== 00:52:28.669 00:22:41.301 ========== Backtrace start: ========== 00:52:28.669 00:22:41.301 00:52:28.669 00:22:41.301 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["nvmf_dif"],["/home/vagrant/spdk_repo/spdk/test/nvmf/target/dif.sh"]) 00:52:28.669 00:22:41.301 ... 00:52:28.669 00:22:41.301 1117 timing_enter $test_name 00:52:28.669 00:22:41.301 1118 echo "************************************" 00:52:28.669 00:22:41.301 1119 echo "START TEST $test_name" 00:52:28.669 00:22:41.301 1120 echo "************************************" 00:52:28.669 00:22:41.301 1121 xtrace_restore 00:52:28.669 00:22:41.301 1122 time "$@" 00:52:28.669 00:22:41.301 1123 xtrace_disable 00:52:28.669 00:22:41.301 1124 echo "************************************" 00:52:28.669 00:22:41.301 1125 echo "END TEST $test_name" 00:52:28.669 00:22:41.301 1126 echo "************************************" 00:52:28.669 00:22:41.301 1127 timing_exit $test_name 00:52:28.669 00:22:41.301 ... 00:52:28.669 00:22:41.301 in /home/vagrant/spdk_repo/spdk/autotest.sh:288 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:52:28.669 00:22:41.301 ... 00:52:28.669 00:22:41.301 283 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.669 00:22:41.301 284 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:52:28.669 00:22:41.301 285 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.669 00:22:41.301 286 run_test "nvmf_identify_passthru" $rootdir/test/nvmf/target/identify_passthru.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.669 00:22:41.301 287 fi 00:52:28.669 00:22:41.301 => 288 run_test "nvmf_dif" $rootdir/test/nvmf/target/dif.sh 00:52:28.669 00:22:41.301 289 run_test "nvmf_abort_qd_sizes" $rootdir/test/nvmf/target/abort_qd_sizes.sh 00:52:28.669 00:22:41.301 290 # The keyring tests utilize NVMe/TLS 00:52:28.669 00:22:41.301 291 run_test "keyring_file" "$rootdir/test/keyring/file.sh" 00:52:28.669 00:22:41.301 292 if [[ "$CONFIG_HAVE_KEYUTILS" == y ]]; then 00:52:28.669 00:22:41.301 293 run_test "keyring_linux" "$rootdir/test/keyring/linux.sh" 00:52:28.669 00:22:41.301 ... 00:52:28.669 00:22:41.301 00:52:28.669 00:22:41.301 ========== Backtrace end ==========' 00:52:28.669 + for build in "${build_log_list[@]}" 00:52:28.669 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/rocky9-vg-autotest_16070/build.log 00:52:28.669 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.669 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/rocky9-vg-autotest_16070/build.log 00:52:28.669 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/rocky9-vg-autotest_16070/build.log 00:52:28.669 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/rocky9-vg-autotest_16070 00:52:28.669 + job_name=rocky9-vg-autotest_16070 00:52:28.669 + start_pattern='Backtrace start' 00:52:28.669 + end_pattern='Backtrace end' 00:52:28.669 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/rocky9-vg-autotest_16070/build.log ]] 00:52:28.669 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/rocky9-vg-autotest_16070/build.log 00:52:28.670 + backtraces= 00:52:28.670 + [[ -n '' ]] 00:52:28.670 + for build in "${build_log_list[@]}" 00:52:28.670 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-cmb-pmr-vg-autotest_78583/build.log 00:52:28.670 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.670 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-cmb-pmr-vg-autotest_78583/build.log 00:52:28.670 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-cmb-pmr-vg-autotest_78583/build.log 00:52:28.670 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-cmb-pmr-vg-autotest_78583 00:52:28.670 + job_name=nvme-cmb-pmr-vg-autotest_78583 00:52:28.670 + start_pattern='Backtrace start' 00:52:28.670 + end_pattern='Backtrace end' 00:52:28.670 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-cmb-pmr-vg-autotest_78583/build.log ]] 00:52:28.670 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-cmb-pmr-vg-autotest_78583/build.log 00:52:28.670 + backtraces= 00:52:28.670 + [[ -n '' ]] 00:52:28.670 + for build in "${build_log_list[@]}" 00:52:28.670 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/lvol-vg-autotest_77409/build.log 00:52:28.670 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.670 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/lvol-vg-autotest_77409/build.log 00:52:28.670 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/lvol-vg-autotest_77409/build.log 00:52:28.670 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/lvol-vg-autotest_77409 00:52:28.670 + job_name=lvol-vg-autotest_77409 00:52:28.670 + start_pattern='Backtrace start' 00:52:28.670 + end_pattern='Backtrace end' 00:52:28.670 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/lvol-vg-autotest_77409/build.log ]] 00:52:28.670 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/lvol-vg-autotest_77409/build.log 00:52:28.670 + backtraces= 00:52:28.670 + [[ -n '' ]] 00:52:28.670 + for build in "${build_log_list[@]}" 00:52:28.670 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/crypto-phy-autotest_26180/build.log 00:52:28.670 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.670 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/crypto-phy-autotest_26180/build.log 00:52:28.670 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/crypto-phy-autotest_26180/build.log 00:52:28.670 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/crypto-phy-autotest_26180 00:52:28.670 + job_name=crypto-phy-autotest_26180 00:52:28.670 + start_pattern='Backtrace start' 00:52:28.670 + end_pattern='Backtrace end' 00:52:28.670 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/crypto-phy-autotest_26180/build.log ]] 00:52:28.670 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/crypto-phy-autotest_26180/build.log 00:52:28.670 + backtraces= 00:52:28.670 + [[ -n '' ]] 00:52:28.670 + for build in "${build_log_list[@]}" 00:52:28.670 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/centos7-vg-autotest_73639/build.log 00:52:28.670 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.670 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/centos7-vg-autotest_73639/build.log 00:52:28.670 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/centos7-vg-autotest_73639/build.log 00:52:28.670 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/centos7-vg-autotest_73639 00:52:28.670 + job_name=centos7-vg-autotest_73639 00:52:28.670 + start_pattern='Backtrace start' 00:52:28.670 + end_pattern='Backtrace end' 00:52:28.670 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/centos7-vg-autotest_73639/build.log ]] 00:52:28.670 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/centos7-vg-autotest_73639/build.log 00:52:28.670 + backtraces= 00:52:28.670 + [[ -n '' ]] 00:52:28.670 + for build in "${build_log_list[@]}" 00:52:28.670 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu22-vg-autotest_37793/build.log 00:52:28.670 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.670 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu22-vg-autotest_37793/build.log 00:52:28.670 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu22-vg-autotest_37793/build.log 00:52:28.670 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu22-vg-autotest_37793 00:52:28.670 + job_name=ubuntu22-vg-autotest_37793 00:52:28.670 + start_pattern='Backtrace start' 00:52:28.670 + end_pattern='Backtrace end' 00:52:28.670 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu22-vg-autotest_37793/build.log ]] 00:52:28.670 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu22-vg-autotest_37793/build.log 00:52:28.670 + backtraces= 00:52:28.670 + [[ -n '' ]] 00:52:28.670 + for build in "${build_log_list[@]}" 00:52:28.670 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-phy-autotest_72311/build.log 00:52:28.670 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.670 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-phy-autotest_72311/build.log 00:52:28.670 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-phy-autotest_72311/build.log 00:52:28.670 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-phy-autotest_72311 00:52:28.670 + job_name=nvmf-tcp-phy-autotest_72311 00:52:28.670 + start_pattern='Backtrace start' 00:52:28.670 + end_pattern='Backtrace end' 00:52:28.670 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-phy-autotest_72311/build.log ]] 00:52:28.670 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-phy-autotest_72311/build.log 00:52:28.932 + backtraces= 00:52:28.932 + [[ -n '' ]] 00:52:28.932 + for build in "${build_log_list[@]}" 00:52:28.932 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/autobuild-vg-autotest_63416/build.log 00:52:28.932 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.932 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/autobuild-vg-autotest_63416/build.log 00:52:28.932 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/autobuild-vg-autotest_63416/build.log 00:52:28.932 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/autobuild-vg-autotest_63416 00:52:28.932 + job_name=autobuild-vg-autotest_63416 00:52:28.932 + start_pattern='Backtrace start' 00:52:28.932 + end_pattern='Backtrace end' 00:52:28.932 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/autobuild-vg-autotest_63416/build.log ]] 00:52:28.932 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/autobuild-vg-autotest_63416/build.log 00:52:28.932 + backtraces= 00:52:28.932 + [[ -n '' ]] 00:52:28.932 + for build in "${build_log_list[@]}" 00:52:28.932 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-vg-autotest_77451/build.log 00:52:28.932 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.932 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-vg-autotest_77451/build.log 00:52:28.932 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-vg-autotest_77451/build.log 00:52:28.932 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-vg-autotest_77451 00:52:28.932 + job_name=nvmf-tcp-vg-autotest_77451 00:52:28.932 + start_pattern='Backtrace start' 00:52:28.932 + end_pattern='Backtrace end' 00:52:28.932 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-vg-autotest_77451/build.log ]] 00:52:28.932 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-vg-autotest_77451/build.log 00:52:28.932 + backtraces= 00:52:28.932 + [[ -n '' ]] 00:52:28.932 + for build in "${build_log_list[@]}" 00:52:28.932 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-vg-autotest_77460/build.log 00:52:28.932 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.932 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-vg-autotest_77460/build.log 00:52:28.932 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-vg-autotest_77460/build.log 00:52:28.932 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-vg-autotest_77460 00:52:28.932 + job_name=nvme-vg-autotest_77460 00:52:28.932 + start_pattern='Backtrace start' 00:52:28.932 + end_pattern='Backtrace end' 00:52:28.932 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-vg-autotest_77460/build.log ]] 00:52:28.932 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-vg-autotest_77460/build.log 00:52:28.932 + backtraces='00:12:24.162 05:43:18 sw_hotplug -- common/autotest_common.sh@715 -- # time='\''========== Backtrace start: ========== 00:52:28.932 00:12:24.162 00:52:28.932 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.932 00:12:24.162 ... 00:52:28.932 00:12:24.162 710 00:52:28.932 00:12:24.162 711 # We redirect cmd'\''\'\'''\''s std{out,err} to a separate fd dup'\''\'\'''\''ed to stdin'\''\'\'''\''s device (or /dev/null) to 00:52:28.932 00:12:24.162 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 00:52:28.932 00:12:24.162 713 # but $() will return just the time'\''\'\'''\''s data, hence making it possible to just do: 00:52:28.932 00:12:24.162 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.932 00:12:24.162 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.932 00:12:24.162 716 00:52:28.932 00:12:24.162 717 echo "$time" 00:52:28.932 00:12:24.162 718 ) 00:52:28.932 00:12:24.162 719 00:52:28.932 00:12:24.162 720 function timing_enter() { 00:52:28.932 00:12:24.162 ... 00:52:28.932 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.932 00:12:24.162 ... 00:52:28.932 00:12:24.162 11 # Pci bus hotplug 00:52:28.932 00:12:24.162 12 # Helper function to remove/attach cotrollers 00:52:28.932 00:12:24.162 13 debug_remove_attach_helper() { 00:52:28.932 00:12:24.162 14 local helper_time=0 00:52:28.932 00:12:24.162 15 00:52:28.932 00:12:24.162 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.932 00:12:24.162 17 printf '\''\'\'''\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\''\'\'''\'' \ 00:52:28.932 00:12:24.162 18 "$helper_time" "$nvme_count" >&2 00:52:28.932 00:12:24.162 19 } 00:52:28.932 00:12:24.162 20 00:52:28.932 00:12:24.162 21 remove_attach_helper() { 00:52:28.932 00:12:24.162 ... 00:52:28.932 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.932 00:12:24.162 ... 00:52:28.932 00:12:24.162 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.932 00:12:24.162 106 done 00:52:28.932 00:12:24.162 107 00:52:28.932 00:12:24.162 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.932 00:12:24.162 109 00:52:28.932 00:12:24.162 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.932 00:12:24.162 111 # Verify reregistering hotplug poller 00:52:28.932 00:12:24.162 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.932 00:12:24.162 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.932 00:12:24.162 114 00:52:28.932 00:12:24.162 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.932 00:12:24.162 ... 00:52:28.932 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.932 00:12:24.162 ... 00:52:28.932 00:12:24.162 133 00:52:28.932 00:12:24.162 134 # Run pci bus hotplug test 00:52:28.932 00:12:24.162 135 run_hotplug 00:52:28.932 00:12:24.162 136 00:52:28.932 00:12:24.162 137 # Run SPDK target based hotplug 00:52:28.932 00:12:24.162 => 138 tgt_run_hotplug 00:52:28.932 00:12:24.162 ... 00:52:28.932 00:12:24.162 00:52:28.932 00:12:24.162 ========== Backtrace end =========='\'' 00:52:28.932 00:12:24.162 ========== Backtrace start: ========== 00:52:28.932 00:12:24.162 00:52:28.932 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.932 00:12:24.162 ... 00:52:28.932 00:12:24.162 710 00:52:28.932 00:12:24.162 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 00:52:28.932 00:12:24.162 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 00:52:28.932 00:12:24.162 713 # but $() will return just the time'\''s data, hence making it possible to just do: 00:52:28.932 00:12:24.162 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.932 00:12:24.162 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.932 00:12:24.162 716 00:52:28.932 00:12:24.162 717 echo "$time" 00:52:28.932 00:12:24.162 718 ) 00:52:28.932 00:12:24.162 719 00:52:28.932 00:12:24.162 720 function timing_enter() { 00:52:28.932 00:12:24.162 ... 00:52:28.932 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.932 00:12:24.162 ... 00:52:28.932 00:12:24.162 11 # Pci bus hotplug 00:52:28.932 00:12:24.162 12 # Helper function to remove/attach cotrollers 00:52:28.932 00:12:24.162 13 debug_remove_attach_helper() { 00:52:28.932 00:12:24.162 14 local helper_time=0 00:52:28.932 00:12:24.162 15 00:52:28.932 00:12:24.162 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.932 00:12:24.162 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.932 00:12:24.162 18 "$helper_time" "$nvme_count" >&2 00:52:28.932 00:12:24.162 19 } 00:52:28.932 00:12:24.162 20 00:52:28.932 00:12:24.162 21 remove_attach_helper() { 00:52:28.932 00:12:24.162 ... 00:52:28.932 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.932 00:12:24.162 ... 00:52:28.932 00:12:24.162 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.932 00:12:24.162 106 done 00:52:28.932 00:12:24.162 107 00:52:28.932 00:12:24.163 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.932 00:12:24.163 109 00:52:28.932 00:12:24.163 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.932 00:12:24.163 111 # Verify reregistering hotplug poller 00:52:28.932 00:12:24.163 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.932 00:12:24.163 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.932 00:12:24.163 114 00:52:28.932 00:12:24.163 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.932 00:12:24.163 ... 00:52:28.932 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.932 00:12:24.163 ... 00:52:28.932 00:12:24.163 133 00:52:28.932 00:12:24.163 134 # Run pci bus hotplug test 00:52:28.932 00:12:24.163 135 run_hotplug 00:52:28.932 00:12:24.163 136 00:52:28.932 00:12:24.163 137 # Run SPDK target based hotplug 00:52:28.932 00:12:24.163 => 138 tgt_run_hotplug 00:52:28.932 00:12:24.163 ... 00:52:28.932 00:12:24.163 00:52:28.932 00:12:24.163 ========== Backtrace end ========== 00:52:28.932 00:12:24.163 ========== Backtrace start: ========== 00:52:28.932 00:12:24.163 00:52:28.933 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 713 # but $() will return just the time'\''s data, hence making it possible to just do: 00:52:28.933 00:12:24.163 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.933 00:12:24.163 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.933 00:12:24.163 716 00:52:28.933 00:12:24.163 717 echo "$time" 00:52:28.933 00:12:24.163 => 718 ) 00:52:28.933 00:12:24.163 719 00:52:28.933 00:12:24.163 720 function timing_enter() { 00:52:28.933 00:12:24.163 721 xtrace_disable 00:52:28.933 00:12:24.163 722 timing "enter" "$1" 00:52:28.933 00:12:24.163 723 xtrace_restore 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 11 # Pci bus hotplug 00:52:28.933 00:12:24.163 12 # Helper function to remove/attach cotrollers 00:52:28.933 00:12:24.163 13 debug_remove_attach_helper() { 00:52:28.933 00:12:24.163 14 local helper_time=0 00:52:28.933 00:12:24.163 15 00:52:28.933 00:12:24.163 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.933 00:12:24.163 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.933 00:12:24.163 18 "$helper_time" "$nvme_count" >&2 00:52:28.933 00:12:24.163 19 } 00:52:28.933 00:12:24.163 20 00:52:28.933 00:12:24.163 21 remove_attach_helper() { 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.933 00:12:24.163 106 done 00:52:28.933 00:12:24.163 107 00:52:28.933 00:12:24.163 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.933 00:12:24.163 109 00:52:28.933 00:12:24.163 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.933 00:12:24.163 111 # Verify reregistering hotplug poller 00:52:28.933 00:12:24.163 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.933 00:12:24.163 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.933 00:12:24.163 114 00:52:28.933 00:12:24.163 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 133 00:52:28.933 00:12:24.163 134 # Run pci bus hotplug test 00:52:28.933 00:12:24.163 135 run_hotplug 00:52:28.933 00:12:24.163 136 00:52:28.933 00:12:24.163 137 # Run SPDK target based hotplug 00:52:28.933 00:12:24.163 => 138 tgt_run_hotplug 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 00:52:28.933 00:12:24.163 ========== Backtrace end ========== 00:52:28.933 00:12:24.163 ========== Backtrace start: ========== 00:52:28.933 00:12:24.163 00:52:28.933 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 11 # Pci bus hotplug 00:52:28.933 00:12:24.163 12 # Helper function to remove/attach cotrollers 00:52:28.933 00:12:24.163 13 debug_remove_attach_helper() { 00:52:28.933 00:12:24.163 14 local helper_time=0 00:52:28.933 00:12:24.163 15 00:52:28.933 00:12:24.163 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.933 00:12:24.163 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.933 00:12:24.163 18 "$helper_time" "$nvme_count" >&2 00:52:28.933 00:12:24.163 19 } 00:52:28.933 00:12:24.163 20 00:52:28.933 00:12:24.163 21 remove_attach_helper() { 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.933 00:12:24.163 106 done 00:52:28.933 00:12:24.163 107 00:52:28.933 00:12:24.163 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.933 00:12:24.163 109 00:52:28.933 00:12:24.163 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.933 00:12:24.163 111 # Verify reregistering hotplug poller 00:52:28.933 00:12:24.163 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.933 00:12:24.163 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.933 00:12:24.163 114 00:52:28.933 00:12:24.163 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 133 00:52:28.933 00:12:24.163 134 # Run pci bus hotplug test 00:52:28.933 00:12:24.163 135 run_hotplug 00:52:28.933 00:12:24.163 136 00:52:28.933 00:12:24.163 137 # Run SPDK target based hotplug 00:52:28.933 00:12:24.163 => 138 tgt_run_hotplug 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 00:52:28.933 00:12:24.163 ========== Backtrace end ========== 00:52:28.933 00:12:24.163 ========== Backtrace start: ========== 00:52:28.933 00:12:24.163 00:52:28.933 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 1117 timing_enter $test_name 00:52:28.933 00:12:24.163 1118 echo "************************************" 00:52:28.933 00:12:24.163 1119 echo "START TEST $test_name" 00:52:28.933 00:12:24.163 1120 echo "************************************" 00:52:28.933 00:12:24.163 1121 xtrace_restore 00:52:28.933 00:12:24.163 1122 time "$@" 00:52:28.933 00:12:24.163 1123 xtrace_disable 00:52:28.933 00:12:24.163 1124 echo "************************************" 00:52:28.933 00:12:24.163 1125 echo "END TEST $test_name" 00:52:28.933 00:12:24.163 1126 echo "************************************" 00:52:28.933 00:12:24.163 1127 timing_exit $test_name 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 235 00:52:28.933 00:12:24.163 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 00:52:28.933 00:12:24.163 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 00:52:28.933 00:12:24.163 238 00:52:28.933 00:12:24.163 239 if [ $(uname -s) = Linux ]; then 00:52:28.933 00:12:24.163 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 00:52:28.933 00:12:24.163 241 fi 00:52:28.933 00:12:24.163 242 00:52:28.933 00:12:24.163 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 00:52:28.933 00:12:24.163 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 00:52:28.933 00:12:24.163 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 00:52:28.933 00:12:24.163 ... 00:52:28.933 00:12:24.163 00:52:28.933 00:12:24.163 ========== Backtrace end ==========' 00:52:28.933 + [[ -n 00:12:24.162 05:43:18 sw_hotplug -- common/autotest_common.sh@715 -- # time='========== Backtrace start: ========== 00:52:28.933 00:12:24.162 00:52:28.933 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.933 00:12:24.162 ... 00:52:28.933 00:12:24.162 710 00:52:28.933 00:12:24.162 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 00:52:28.933 00:12:24.162 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 00:52:28.933 00:12:24.162 713 # but $() will return just the time'\''s data, hence making it possible to just do: 00:52:28.933 00:12:24.162 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.933 00:12:24.162 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.933 00:12:24.162 716 00:52:28.933 00:12:24.162 717 echo "$time" 00:52:28.933 00:12:24.162 718 ) 00:52:28.933 00:12:24.162 719 00:52:28.933 00:12:24.162 720 function timing_enter() { 00:52:28.933 00:12:24.162 ... 00:52:28.933 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.933 00:12:24.162 ... 00:52:28.933 00:12:24.162 11 # Pci bus hotplug 00:52:28.933 00:12:24.162 12 # Helper function to remove/attach cotrollers 00:52:28.933 00:12:24.162 13 debug_remove_attach_helper() { 00:52:28.933 00:12:24.162 14 local helper_time=0 00:52:28.933 00:12:24.162 15 00:52:28.933 00:12:24.162 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.933 00:12:24.162 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.933 00:12:24.162 18 "$helper_time" "$nvme_count" >&2 00:52:28.933 00:12:24.162 19 } 00:52:28.933 00:12:24.162 20 00:52:28.933 00:12:24.162 21 remove_attach_helper() { 00:52:28.933 00:12:24.162 ... 00:52:28.933 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.933 00:12:24.162 ... 00:52:28.933 00:12:24.162 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.933 00:12:24.162 106 done 00:52:28.933 00:12:24.162 107 00:52:28.933 00:12:24.162 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.933 00:12:24.162 109 00:52:28.933 00:12:24.162 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.933 00:12:24.162 111 # Verify reregistering hotplug poller 00:52:28.933 00:12:24.162 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.933 00:12:24.162 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.933 00:12:24.162 114 00:52:28.933 00:12:24.162 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.933 00:12:24.162 ... 00:52:28.933 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.933 00:12:24.162 ... 00:52:28.933 00:12:24.162 133 00:52:28.933 00:12:24.162 134 # Run pci bus hotplug test 00:52:28.933 00:12:24.162 135 run_hotplug 00:52:28.933 00:12:24.162 136 00:52:28.933 00:12:24.162 137 # Run SPDK target based hotplug 00:52:28.933 00:12:24.162 => 138 tgt_run_hotplug 00:52:28.933 00:12:24.162 ... 00:52:28.933 00:12:24.162 00:52:28.933 00:12:24.162 ========== Backtrace end ==========' 00:52:28.933 00:12:24.162 ========== Backtrace start: ========== 00:52:28.933 00:12:24.162 00:52:28.933 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.933 00:12:24.162 ... 00:52:28.933 00:12:24.162 710 00:52:28.933 00:12:24.162 711 # We redirect cmd's std{out,err} to a separate fd dup'ed to stdin's device (or /dev/null) to 00:52:28.934 00:12:24.162 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 00:52:28.934 00:12:24.162 713 # but $() will return just the time's data, hence making it possible to just do: 00:52:28.934 00:12:24.162 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.934 00:12:24.162 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.934 00:12:24.162 716 00:52:28.934 00:12:24.162 717 echo "$time" 00:52:28.934 00:12:24.162 718 ) 00:52:28.934 00:12:24.162 719 00:52:28.934 00:12:24.162 720 function timing_enter() { 00:52:28.934 00:12:24.162 ... 00:52:28.934 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.934 00:12:24.162 ... 00:52:28.934 00:12:24.162 11 # Pci bus hotplug 00:52:28.934 00:12:24.162 12 # Helper function to remove/attach cotrollers 00:52:28.934 00:12:24.162 13 debug_remove_attach_helper() { 00:52:28.934 00:12:24.162 14 local helper_time=0 00:52:28.934 00:12:24.162 15 00:52:28.934 00:12:24.162 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.934 00:12:24.162 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 00:52:28.934 00:12:24.162 18 "$helper_time" "$nvme_count" >&2 00:52:28.934 00:12:24.162 19 } 00:52:28.934 00:12:24.162 20 00:52:28.934 00:12:24.162 21 remove_attach_helper() { 00:52:28.934 00:12:24.162 ... 00:52:28.934 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.934 00:12:24.162 ... 00:52:28.934 00:12:24.162 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.934 00:12:24.162 106 done 00:52:28.934 00:12:24.162 107 00:52:28.934 00:12:24.163 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.934 00:12:24.163 109 00:52:28.934 00:12:24.163 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.934 00:12:24.163 111 # Verify reregistering hotplug poller 00:52:28.934 00:12:24.163 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.934 00:12:24.163 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.934 00:12:24.163 114 00:52:28.934 00:12:24.163 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 133 00:52:28.934 00:12:24.163 134 # Run pci bus hotplug test 00:52:28.934 00:12:24.163 135 run_hotplug 00:52:28.934 00:12:24.163 136 00:52:28.934 00:12:24.163 137 # Run SPDK target based hotplug 00:52:28.934 00:12:24.163 => 138 tgt_run_hotplug 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 00:52:28.934 00:12:24.163 ========== Backtrace end ========== 00:52:28.934 00:12:24.163 ========== Backtrace start: ========== 00:52:28.934 00:12:24.163 00:52:28.934 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 713 # but $() will return just the time's data, hence making it possible to just do: 00:52:28.934 00:12:24.163 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.934 00:12:24.163 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.934 00:12:24.163 716 00:52:28.934 00:12:24.163 717 echo "$time" 00:52:28.934 00:12:24.163 => 718 ) 00:52:28.934 00:12:24.163 719 00:52:28.934 00:12:24.163 720 function timing_enter() { 00:52:28.934 00:12:24.163 721 xtrace_disable 00:52:28.934 00:12:24.163 722 timing "enter" "$1" 00:52:28.934 00:12:24.163 723 xtrace_restore 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 11 # Pci bus hotplug 00:52:28.934 00:12:24.163 12 # Helper function to remove/attach cotrollers 00:52:28.934 00:12:24.163 13 debug_remove_attach_helper() { 00:52:28.934 00:12:24.163 14 local helper_time=0 00:52:28.934 00:12:24.163 15 00:52:28.934 00:12:24.163 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.934 00:12:24.163 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 00:52:28.934 00:12:24.163 18 "$helper_time" "$nvme_count" >&2 00:52:28.934 00:12:24.163 19 } 00:52:28.934 00:12:24.163 20 00:52:28.934 00:12:24.163 21 remove_attach_helper() { 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.934 00:12:24.163 106 done 00:52:28.934 00:12:24.163 107 00:52:28.934 00:12:24.163 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.934 00:12:24.163 109 00:52:28.934 00:12:24.163 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.934 00:12:24.163 111 # Verify reregistering hotplug poller 00:52:28.934 00:12:24.163 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.934 00:12:24.163 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.934 00:12:24.163 114 00:52:28.934 00:12:24.163 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 133 00:52:28.934 00:12:24.163 134 # Run pci bus hotplug test 00:52:28.934 00:12:24.163 135 run_hotplug 00:52:28.934 00:12:24.163 136 00:52:28.934 00:12:24.163 137 # Run SPDK target based hotplug 00:52:28.934 00:12:24.163 => 138 tgt_run_hotplug 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 00:52:28.934 00:12:24.163 ========== Backtrace end ========== 00:52:28.934 00:12:24.163 ========== Backtrace start: ========== 00:52:28.934 00:12:24.163 00:52:28.934 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 11 # Pci bus hotplug 00:52:28.934 00:12:24.163 12 # Helper function to remove/attach cotrollers 00:52:28.934 00:12:24.163 13 debug_remove_attach_helper() { 00:52:28.934 00:12:24.163 14 local helper_time=0 00:52:28.934 00:12:24.163 15 00:52:28.934 00:12:24.163 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.934 00:12:24.163 17 printf 'remove_attach_helper took %ss to complete (handling %u nvme drive(s))' \ 00:52:28.934 00:12:24.163 18 "$helper_time" "$nvme_count" >&2 00:52:28.934 00:12:24.163 19 } 00:52:28.934 00:12:24.163 20 00:52:28.934 00:12:24.163 21 remove_attach_helper() { 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.934 00:12:24.163 106 done 00:52:28.934 00:12:24.163 107 00:52:28.934 00:12:24.163 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.934 00:12:24.163 109 00:52:28.934 00:12:24.163 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.934 00:12:24.163 111 # Verify reregistering hotplug poller 00:52:28.934 00:12:24.163 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.934 00:12:24.163 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.934 00:12:24.163 114 00:52:28.934 00:12:24.163 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 133 00:52:28.934 00:12:24.163 134 # Run pci bus hotplug test 00:52:28.934 00:12:24.163 135 run_hotplug 00:52:28.934 00:12:24.163 136 00:52:28.934 00:12:24.163 137 # Run SPDK target based hotplug 00:52:28.934 00:12:24.163 => 138 tgt_run_hotplug 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 00:52:28.934 00:12:24.163 ========== Backtrace end ========== 00:52:28.934 00:12:24.163 ========== Backtrace start: ========== 00:52:28.934 00:12:24.163 00:52:28.934 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 1117 timing_enter $test_name 00:52:28.934 00:12:24.163 1118 echo "************************************" 00:52:28.934 00:12:24.163 1119 echo "START TEST $test_name" 00:52:28.934 00:12:24.163 1120 echo "************************************" 00:52:28.934 00:12:24.163 1121 xtrace_restore 00:52:28.934 00:12:24.163 1122 time "$@" 00:52:28.934 00:12:24.163 1123 xtrace_disable 00:52:28.934 00:12:24.163 1124 echo "************************************" 00:52:28.934 00:12:24.163 1125 echo "END TEST $test_name" 00:52:28.934 00:12:24.163 1126 echo "************************************" 00:52:28.934 00:12:24.163 1127 timing_exit $test_name 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 235 00:52:28.934 00:12:24.163 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 00:52:28.934 00:12:24.163 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 00:52:28.934 00:12:24.163 238 00:52:28.934 00:12:24.163 239 if [ $(uname -s) = Linux ]; then 00:52:28.934 00:12:24.163 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 00:52:28.934 00:12:24.163 241 fi 00:52:28.934 00:12:24.163 242 00:52:28.934 00:12:24.163 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 00:52:28.934 00:12:24.163 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 00:52:28.934 00:12:24.163 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 00:52:28.934 00:12:24.163 ... 00:52:28.934 00:12:24.163 00:52:28.934 00:12:24.163 ========== Backtrace end ========== ]] 00:52:28.934 + echo '00:12:24.162 05:43:18 sw_hotplug -- common/autotest_common.sh@715 -- # time='\''========== Backtrace start: ========== 00:52:28.934 00:12:24.162 00:52:28.934 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.934 00:12:24.162 ... 00:52:28.934 00:12:24.162 710 00:52:28.934 00:12:24.162 711 # We redirect cmd'\''\'\'''\''s std{out,err} to a separate fd dup'\''\'\'''\''ed to stdin'\''\'\'''\''s device (or /dev/null) to 00:52:28.935 00:12:24.162 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 00:52:28.935 00:12:24.162 713 # but $() will return just the time'\''\'\'''\''s data, hence making it possible to just do: 00:52:28.935 00:12:24.162 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.935 00:12:24.162 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.935 00:12:24.162 716 00:52:28.935 00:12:24.162 717 echo "$time" 00:52:28.935 00:12:24.162 718 ) 00:52:28.935 00:12:24.162 719 00:52:28.935 00:12:24.162 720 function timing_enter() { 00:52:28.935 00:12:24.162 ... 00:52:28.935 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.935 00:12:24.162 ... 00:52:28.935 00:12:24.162 11 # Pci bus hotplug 00:52:28.935 00:12:24.162 12 # Helper function to remove/attach cotrollers 00:52:28.935 00:12:24.162 13 debug_remove_attach_helper() { 00:52:28.935 00:12:24.162 14 local helper_time=0 00:52:28.935 00:12:24.162 15 00:52:28.935 00:12:24.162 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.935 00:12:24.162 17 printf '\''\'\'''\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\''\'\'''\'' \ 00:52:28.935 00:12:24.162 18 "$helper_time" "$nvme_count" >&2 00:52:28.935 00:12:24.162 19 } 00:52:28.935 00:12:24.162 20 00:52:28.935 00:12:24.162 21 remove_attach_helper() { 00:52:28.935 00:12:24.162 ... 00:52:28.935 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.935 00:12:24.162 ... 00:52:28.935 00:12:24.162 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.935 00:12:24.162 106 done 00:52:28.935 00:12:24.162 107 00:52:28.935 00:12:24.162 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.935 00:12:24.162 109 00:52:28.935 00:12:24.162 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.935 00:12:24.162 111 # Verify reregistering hotplug poller 00:52:28.935 00:12:24.162 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.935 00:12:24.162 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.935 00:12:24.162 114 00:52:28.935 00:12:24.162 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.935 00:12:24.162 ... 00:52:28.935 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.935 00:12:24.162 ... 00:52:28.935 00:12:24.162 133 00:52:28.935 00:12:24.162 134 # Run pci bus hotplug test 00:52:28.935 00:12:24.162 135 run_hotplug 00:52:28.935 00:12:24.162 136 00:52:28.935 00:12:24.162 137 # Run SPDK target based hotplug 00:52:28.935 00:12:24.162 => 138 tgt_run_hotplug 00:52:28.935 00:12:24.162 ... 00:52:28.935 00:12:24.162 00:52:28.935 00:12:24.162 ========== Backtrace end =========='\'' 00:52:28.935 00:12:24.162 ========== Backtrace start: ========== 00:52:28.935 00:12:24.162 00:52:28.935 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:715 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.935 00:12:24.162 ... 00:52:28.935 00:12:24.162 710 00:52:28.935 00:12:24.162 711 # We redirect cmd'\''s std{out,err} to a separate fd dup'\''ed to stdin'\''s device (or /dev/null) to 00:52:28.935 00:12:24.162 712 # catch only output from the time builtin - output from the actual cmd would be still visible, 00:52:28.935 00:12:24.162 713 # but $() will return just the time'\''s data, hence making it possible to just do: 00:52:28.935 00:12:24.162 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.935 00:12:24.162 => 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.935 00:12:24.162 716 00:52:28.935 00:12:24.162 717 echo "$time" 00:52:28.935 00:12:24.162 718 ) 00:52:28.935 00:12:24.162 719 00:52:28.935 00:12:24.162 720 function timing_enter() { 00:52:28.935 00:12:24.162 ... 00:52:28.935 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.935 00:12:24.162 ... 00:52:28.935 00:12:24.162 11 # Pci bus hotplug 00:52:28.935 00:12:24.162 12 # Helper function to remove/attach cotrollers 00:52:28.935 00:12:24.162 13 debug_remove_attach_helper() { 00:52:28.935 00:12:24.162 14 local helper_time=0 00:52:28.935 00:12:24.162 15 00:52:28.935 00:12:24.162 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.935 00:12:24.162 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.935 00:12:24.162 18 "$helper_time" "$nvme_count" >&2 00:52:28.935 00:12:24.162 19 } 00:52:28.935 00:12:24.162 20 00:52:28.935 00:12:24.162 21 remove_attach_helper() { 00:52:28.935 00:12:24.162 ... 00:52:28.935 00:12:24.162 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.935 00:12:24.162 ... 00:52:28.935 00:12:24.162 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.935 00:12:24.162 106 done 00:52:28.935 00:12:24.162 107 00:52:28.935 00:12:24.163 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.935 00:12:24.163 109 00:52:28.935 00:12:24.163 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.935 00:12:24.163 111 # Verify reregistering hotplug poller 00:52:28.935 00:12:24.163 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.935 00:12:24.163 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.935 00:12:24.163 114 00:52:28.935 00:12:24.163 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 133 00:52:28.935 00:12:24.163 134 # Run pci bus hotplug test 00:52:28.935 00:12:24.163 135 run_hotplug 00:52:28.935 00:12:24.163 136 00:52:28.935 00:12:24.163 137 # Run SPDK target based hotplug 00:52:28.935 00:12:24.163 => 138 tgt_run_hotplug 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 00:52:28.935 00:12:24.163 ========== Backtrace end ========== 00:52:28.935 00:12:24.163 ========== Backtrace start: ========== 00:52:28.935 00:12:24.163 00:52:28.935 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:718 -> timing_cmd(["remove_attach_helper"],["3"],["6"],["true"]) 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 713 # but $() will return just the time'\''s data, hence making it possible to just do: 00:52:28.935 00:12:24.163 714 # time_of_super_verbose_cmd=$(timing_cmd super_verbose_cmd) 00:52:28.935 00:12:24.163 715 time=$({ time "$@" >&"$cmd_out" 2>&1; } 2>&1) 00:52:28.935 00:12:24.163 716 00:52:28.935 00:12:24.163 717 echo "$time" 00:52:28.935 00:12:24.163 => 718 ) 00:52:28.935 00:12:24.163 719 00:52:28.935 00:12:24.163 720 function timing_enter() { 00:52:28.935 00:12:24.163 721 xtrace_disable 00:52:28.935 00:12:24.163 722 timing "enter" "$1" 00:52:28.935 00:12:24.163 723 xtrace_restore 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 11 # Pci bus hotplug 00:52:28.935 00:12:24.163 12 # Helper function to remove/attach cotrollers 00:52:28.935 00:12:24.163 13 debug_remove_attach_helper() { 00:52:28.935 00:12:24.163 14 local helper_time=0 00:52:28.935 00:12:24.163 15 00:52:28.935 00:12:24.163 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.935 00:12:24.163 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.935 00:12:24.163 18 "$helper_time" "$nvme_count" >&2 00:52:28.935 00:12:24.163 19 } 00:52:28.935 00:12:24.163 20 00:52:28.935 00:12:24.163 21 remove_attach_helper() { 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.935 00:12:24.163 106 done 00:52:28.935 00:12:24.163 107 00:52:28.935 00:12:24.163 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.935 00:12:24.163 109 00:52:28.935 00:12:24.163 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.935 00:12:24.163 111 # Verify reregistering hotplug poller 00:52:28.935 00:12:24.163 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.935 00:12:24.163 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.935 00:12:24.163 114 00:52:28.935 00:12:24.163 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 133 00:52:28.935 00:12:24.163 134 # Run pci bus hotplug test 00:52:28.935 00:12:24.163 135 run_hotplug 00:52:28.935 00:12:24.163 136 00:52:28.935 00:12:24.163 137 # Run SPDK target based hotplug 00:52:28.935 00:12:24.163 => 138 tgt_run_hotplug 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 00:52:28.935 00:12:24.163 ========== Backtrace end ========== 00:52:28.935 00:12:24.163 ========== Backtrace start: ========== 00:52:28.935 00:12:24.163 00:52:28.935 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:16 -> debug_remove_attach_helper(["3"],["6"],["true"]) 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 11 # Pci bus hotplug 00:52:28.935 00:12:24.163 12 # Helper function to remove/attach cotrollers 00:52:28.935 00:12:24.163 13 debug_remove_attach_helper() { 00:52:28.935 00:12:24.163 14 local helper_time=0 00:52:28.935 00:12:24.163 15 00:52:28.935 00:12:24.163 => 16 helper_time=$(timing_cmd remove_attach_helper "$@") 00:52:28.935 00:12:24.163 17 printf '\''remove_attach_helper took %ss to complete (handling %u nvme drive(s))'\'' \ 00:52:28.935 00:12:24.163 18 "$helper_time" "$nvme_count" >&2 00:52:28.935 00:12:24.163 19 } 00:52:28.935 00:12:24.163 20 00:52:28.935 00:12:24.163 21 remove_attach_helper() { 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:110 -> tgt_run_hotplug([]) 00:52:28.935 00:12:24.163 ... 00:52:28.935 00:12:24.163 105 waitforbdev "Nvme0${dev}n1" "$hotplug_wait" 00:52:28.935 00:12:24.163 106 done 00:52:28.935 00:12:24.163 107 00:52:28.935 00:12:24.163 108 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.935 00:12:24.163 109 00:52:28.935 00:12:24.163 => 110 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.935 00:12:24.163 111 # Verify reregistering hotplug poller 00:52:28.935 00:12:24.163 112 rpc_cmd bdev_nvme_set_hotplug -d 00:52:28.935 00:12:24.163 113 rpc_cmd bdev_nvme_set_hotplug -e 00:52:28.935 00:12:24.163 114 00:52:28.935 00:12:24.163 115 debug_remove_attach_helper "$hotplug_events" "$hotplug_wait" true 00:52:28.936 00:12:24.163 ... 00:52:28.936 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh:138 -> main([]) 00:52:28.936 00:12:24.163 ... 00:52:28.936 00:12:24.163 133 00:52:28.936 00:12:24.163 134 # Run pci bus hotplug test 00:52:28.936 00:12:24.163 135 run_hotplug 00:52:28.936 00:12:24.163 136 00:52:28.936 00:12:24.163 137 # Run SPDK target based hotplug 00:52:28.936 00:12:24.163 => 138 tgt_run_hotplug 00:52:28.936 00:12:24.163 ... 00:52:28.936 00:12:24.163 00:52:28.936 00:12:24.163 ========== Backtrace end ========== 00:52:28.936 00:12:24.163 ========== Backtrace start: ========== 00:52:28.936 00:12:24.163 00:52:28.936 00:12:24.163 in /home/vagrant/spdk_repo/spdk/test/common/autotest_common.sh:1122 -> run_test(["sw_hotplug"],["/home/vagrant/spdk_repo/spdk/test/nvme/sw_hotplug.sh"]) 00:52:28.936 00:12:24.163 ... 00:52:28.936 00:12:24.163 1117 timing_enter $test_name 00:52:28.936 00:12:24.163 1118 echo "************************************" 00:52:28.936 00:12:24.163 1119 echo "START TEST $test_name" 00:52:28.936 00:12:24.163 1120 echo "************************************" 00:52:28.936 00:12:24.163 1121 xtrace_restore 00:52:28.936 00:12:24.163 1122 time "$@" 00:52:28.936 00:12:24.163 1123 xtrace_disable 00:52:28.936 00:12:24.163 1124 echo "************************************" 00:52:28.936 00:12:24.163 1125 echo "END TEST $test_name" 00:52:28.936 00:12:24.163 1126 echo "************************************" 00:52:28.936 00:12:24.163 1127 timing_exit $test_name 00:52:28.936 00:12:24.163 ... 00:52:28.936 00:12:24.163 in /home/vagrant/spdk_repo/spdk/autotest.sh:240 -> main(["/home/vagrant/spdk_repo/autorun-spdk.conf"]) 00:52:28.936 00:12:24.163 ... 00:52:28.936 00:12:24.163 235 00:52:28.936 00:12:24.163 236 run_test "nvme_rpc" $rootdir/test/nvme/nvme_rpc.sh 00:52:28.936 00:12:24.163 237 run_test "nvme_rpc_timeouts" $rootdir/test/nvme/nvme_rpc_timeouts.sh 00:52:28.936 00:12:24.163 238 00:52:28.936 00:12:24.163 239 if [ $(uname -s) = Linux ]; then 00:52:28.936 00:12:24.163 => 240 run_test "sw_hotplug" $rootdir/test/nvme/sw_hotplug.sh 00:52:28.936 00:12:24.163 241 fi 00:52:28.936 00:12:24.163 242 00:52:28.936 00:12:24.163 243 if [[ $SPDK_TEST_XNVME -eq 1 ]]; then 00:52:28.936 00:12:24.163 244 run_test "nvme_xnvme" $rootdir/test/nvme/xnvme/xnvme.sh 00:52:28.936 00:12:24.163 245 run_test "blockdev_xnvme" $rootdir/test/bdev/blockdev.sh "xnvme" 00:52:28.936 00:12:24.163 ... 00:52:28.936 00:12:24.163 00:52:28.936 00:12:24.163 ========== Backtrace end ==========' 00:52:28.936 + for build in "${build_log_list[@]}" 00:52:28.936 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vfio-user-phy-autotest_26188/build.log 00:52:28.936 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.936 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vfio-user-phy-autotest_26188/build.log 00:52:28.936 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vfio-user-phy-autotest_26188/build.log 00:52:28.936 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vfio-user-phy-autotest_26188 00:52:28.936 + job_name=vfio-user-phy-autotest_26188 00:52:28.936 + start_pattern='Backtrace start' 00:52:28.936 + end_pattern='Backtrace end' 00:52:28.936 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vfio-user-phy-autotest_26188/build.log ]] 00:52:28.936 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vfio-user-phy-autotest_26188/build.log 00:52:28.936 + backtraces= 00:52:28.936 + [[ -n '' ]] 00:52:28.936 + for build in "${build_log_list[@]}" 00:52:28.936 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/scanbuild-vg-autotest_63375/build.log 00:52:28.936 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.936 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/scanbuild-vg-autotest_63375/build.log 00:52:28.936 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/scanbuild-vg-autotest_63375/build.log 00:52:28.936 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/scanbuild-vg-autotest_63375 00:52:28.936 + job_name=scanbuild-vg-autotest_63375 00:52:28.936 + start_pattern='Backtrace start' 00:52:28.936 + end_pattern='Backtrace end' 00:52:28.936 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/scanbuild-vg-autotest_63375/build.log ]] 00:52:28.936 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/scanbuild-vg-autotest_63375/build.log 00:52:28.936 + backtraces= 00:52:28.936 + [[ -n '' ]] 00:52:28.936 + for build in "${build_log_list[@]}" 00:52:28.936 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/valgrind-vg-autotest_49190/build.log 00:52:28.936 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.936 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/valgrind-vg-autotest_49190/build.log 00:52:28.936 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/valgrind-vg-autotest_49190/build.log 00:52:28.936 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/valgrind-vg-autotest_49190 00:52:28.936 + job_name=valgrind-vg-autotest_49190 00:52:28.936 + start_pattern='Backtrace start' 00:52:28.936 + end_pattern='Backtrace end' 00:52:28.936 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/valgrind-vg-autotest_49190/build.log ]] 00:52:28.936 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/valgrind-vg-autotest_49190/build.log 00:52:28.936 + backtraces= 00:52:28.936 + [[ -n '' ]] 00:52:28.936 + for build in "${build_log_list[@]}" 00:52:28.936 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/BlobFS-phy-autotest_26186/build.log 00:52:28.936 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.936 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/BlobFS-phy-autotest_26186/build.log 00:52:28.936 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/BlobFS-phy-autotest_26186/build.log 00:52:28.936 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/BlobFS-phy-autotest_26186 00:52:28.936 + job_name=BlobFS-phy-autotest_26186 00:52:28.936 + start_pattern='Backtrace start' 00:52:28.936 + end_pattern='Backtrace end' 00:52:28.936 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/BlobFS-phy-autotest_26186/build.log ]] 00:52:28.936 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/BlobFS-phy-autotest_26186/build.log 00:52:28.936 + backtraces= 00:52:28.936 + [[ -n '' ]] 00:52:28.936 + for build in "${build_log_list[@]}" 00:52:28.936 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-uring-vg-autotest_77365/build.log 00:52:28.936 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.936 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-uring-vg-autotest_77365/build.log 00:52:28.936 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-uring-vg-autotest_77365/build.log 00:52:28.936 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-uring-vg-autotest_77365 00:52:28.936 + job_name=iscsi-uring-vg-autotest_77365 00:52:28.936 + start_pattern='Backtrace start' 00:52:28.936 + end_pattern='Backtrace end' 00:52:28.936 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-uring-vg-autotest_77365/build.log ]] 00:52:28.936 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-uring-vg-autotest_77365/build.log 00:52:28.936 + backtraces= 00:52:28.936 + [[ -n '' ]] 00:52:28.936 + for build in "${build_log_list[@]}" 00:52:28.936 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-phy-autotest_26205/build.log 00:52:28.936 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.936 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-phy-autotest_26205/build.log 00:52:28.936 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-phy-autotest_26205/build.log 00:52:28.936 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-phy-autotest_26205 00:52:28.936 + job_name=vhost-phy-autotest_26205 00:52:28.936 + start_pattern='Backtrace start' 00:52:28.936 + end_pattern='Backtrace end' 00:52:28.936 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-phy-autotest_26205/build.log ]] 00:52:28.936 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-phy-autotest_26205/build.log 00:52:28.936 + backtraces= 00:52:28.936 + [[ -n '' ]] 00:52:28.936 + for build in "${build_log_list[@]}" 00:52:28.936 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-phy-autotest_72424/build.log 00:52:28.936 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.936 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-phy-autotest_72424/build.log 00:52:28.936 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-phy-autotest_72424/build.log 00:52:28.936 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-phy-autotest_72424 00:52:28.937 + job_name=nvmf-phy-autotest_72424 00:52:28.937 + start_pattern='Backtrace start' 00:52:28.937 + end_pattern='Backtrace end' 00:52:28.937 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-phy-autotest_72424/build.log ]] 00:52:28.937 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-phy-autotest_72424/build.log 00:52:28.937 + backtraces='00:28:19.985 ========== Backtrace start: ========== 00:52:28.937 00:28:19.985 00:52:28.937 00:28:19.985 in test/nvmf/target/device_removal.sh:116 -> stop_bdevperf([]) 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 111 00:52:28.937 00:28:19.985 112 sleep 5 00:52:28.937 00:28:19.985 113 } 00:52:28.937 00:28:19.985 114 00:52:28.937 00:28:19.985 115 function stop_bdevperf() { 00:52:28.937 00:28:19.985 => 116 wait $bdevperf_rpc_pid 00:52:28.937 00:28:19.985 117 00:52:28.937 00:28:19.985 118 killprocess $bdevperf_pid 00:52:28.937 00:28:19.985 119 bdevperf_pid= 00:52:28.937 00:28:19.985 120 00:52:28.937 00:28:19.985 121 cat $testdir/try.txt 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 in test/nvmf/target/device_removal.sh:200 -> test_remove_and_rescan([]) 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 195 fi 00:52:28.937 00:28:19.985 196 sleep 2 00:52:28.937 00:28:19.985 197 done 00:52:28.937 00:28:19.985 198 done 00:52:28.937 00:28:19.985 199 00:52:28.937 00:28:19.985 => 200 stop_bdevperf 00:52:28.937 00:28:19.985 201 00:52:28.937 00:28:19.985 202 killprocess $nvmfpid 00:52:28.937 00:28:19.985 203 nvmfpid= 00:52:28.937 00:28:19.985 204 00:52:28.937 00:28:19.985 205 return 0 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1122 -> run_test(["nvmf_device_removal_pci_remove"],["test_remove_and_rescan"]) 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 1117 timing_enter $test_name 00:52:28.937 00:28:19.985 1118 echo "************************************" 00:52:28.937 00:28:19.985 1119 echo "START TEST $test_name" 00:52:28.937 00:28:19.985 1120 echo "************************************" 00:52:28.937 00:28:19.985 1121 xtrace_restore 00:52:28.937 00:28:19.985 1122 time "$@" 00:52:28.937 00:28:19.985 1123 xtrace_disable 00:52:28.937 00:28:19.985 1124 echo "************************************" 00:52:28.937 00:28:19.985 1125 echo "END TEST $test_name" 00:52:28.937 00:28:19.985 1126 echo "************************************" 00:52:28.937 00:28:19.985 1127 timing_exit $test_name 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 in test/nvmf/target/device_removal.sh:312 -> main(["--transport=rdma"]) 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 307 fi 00:52:28.937 00:28:19.985 308 test_bonding_slaves_on_nics "${target_nics[@]}" 00:52:28.937 00:28:19.985 309 } 00:52:28.937 00:28:19.985 310 00:52:28.937 00:28:19.986 311 run_test "nvmf_device_removal_pci_remove_no_srq" test_remove_and_rescan --no-srq 00:52:28.937 00:28:19.986 => 312 run_test "nvmf_device_removal_pci_remove" test_remove_and_rescan 00:52:28.937 00:28:19.986 313 # bond slaves case needs lag_master & vport_manager are enabled by mlxconfig 00:52:28.937 00:28:19.986 314 # and not work on CI machine currently. 00:52:28.937 00:28:19.986 315 # run_test "nvmf_device_removal_bond_slaves" test_bond_slaves 00:52:28.937 00:28:19.986 316 00:52:28.937 00:28:19.986 317 nvmftestfini 00:52:28.937 00:28:19.986 ... 00:52:28.937 00:28:19.986 00:52:28.937 00:28:19.986 ========== Backtrace end ========== 00:52:28.937 00:28:19.995 ========== Backtrace start: ========== 00:52:28.937 00:28:19.995 00:52:28.937 00:28:19.995 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1122 -> run_test(["nvmf_device_removal"],["test/nvmf/target/device_removal.sh"],["--transport=rdma"]) 00:52:28.937 00:28:19.995 ... 00:52:28.937 00:28:19.995 1117 timing_enter $test_name 00:52:28.937 00:28:19.995 1118 echo "************************************" 00:52:28.937 00:28:19.995 1119 echo "START TEST $test_name" 00:52:28.937 00:28:19.995 1120 echo "************************************" 00:52:28.937 00:28:19.995 1121 xtrace_restore 00:52:28.937 00:28:19.995 1122 time "$@" 00:52:28.937 00:28:19.995 1123 xtrace_disable 00:52:28.937 00:28:19.995 1124 echo "************************************" 00:52:28.937 00:28:19.995 1125 echo "END TEST $test_name" 00:52:28.937 00:28:19.995 1126 echo "************************************" 00:52:28.937 00:28:19.995 1127 timing_exit $test_name 00:52:28.937 00:28:19.995 ... 00:52:28.937 00:28:19.995 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf.sh:79 -> main(["--transport=rdma"]) 00:52:28.937 00:28:19.995 ... 00:52:28.937 00:28:19.995 74 TCP_INTERFACE_LIST=("${net_devs[@]}") 00:52:28.937 00:28:19.995 75 if ((${#TCP_INTERFACE_LIST[@]} > 0)); then 00:52:28.937 00:28:19.995 76 run_test "nvmf_perf_adq" $rootdir/test/nvmf/target/perf_adq.sh "${TEST_ARGS[@]}" 00:52:28.937 00:28:19.995 77 fi 00:52:28.937 00:28:19.995 78 elif [[ $SPDK_TEST_NVMF_TRANSPORT == "rdma" ]]; then 00:52:28.937 00:28:19.995 => 79 run_test "nvmf_device_removal" test/nvmf/target/device_removal.sh "${TEST_ARGS[@]}" 00:52:28.937 00:28:19.995 80 run_test "nvmf_srq_overwhelm" "$rootdir/test/nvmf/target/srq_overwhelm.sh" "${TEST_ARGS[@]}" 00:52:28.937 00:28:19.995 81 fi 00:52:28.937 00:28:19.995 82 run_test "nvmf_shutdown" $rootdir/test/nvmf/target/shutdown.sh "${TEST_ARGS[@]}" 00:52:28.937 00:28:19.995 83 fi 00:52:28.937 00:28:19.995 84 00:52:28.937 00:28:19.995 ... 00:52:28.937 00:28:19.995 00:52:28.937 00:28:19.995 ========== Backtrace end ========== 00:52:28.937 00:28:19.996 ========== Backtrace start: ========== 00:52:28.937 00:28:19.996 00:52:28.937 00:28:19.996 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1122 -> run_test(["nvmf_rdma"],["/var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf.sh"],["--transport=rdma"]) 00:52:28.937 00:28:19.996 ... 00:52:28.937 00:28:19.996 1117 timing_enter $test_name 00:52:28.937 00:28:19.996 1118 echo "************************************" 00:52:28.937 00:28:19.996 1119 echo "START TEST $test_name" 00:52:28.937 00:28:19.996 1120 echo "************************************" 00:52:28.937 00:28:19.996 1121 xtrace_restore 00:52:28.937 00:28:19.996 1122 time "$@" 00:52:28.937 00:28:19.996 1123 xtrace_disable 00:52:28.937 00:28:19.996 1124 echo "************************************" 00:52:28.937 00:28:19.996 1125 echo "END TEST $test_name" 00:52:28.937 00:28:19.996 1126 echo "************************************" 00:52:28.937 00:28:19.996 1127 timing_exit $test_name 00:52:28.937 00:28:19.996 ... 00:52:28.937 00:28:19.996 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/autotest.sh:280 -> main(["/var/jenkins/workspace/nvmf-phy-autotest/autorun-spdk.conf"]) 00:52:28.937 00:28:19.996 ... 00:52:28.937 00:28:19.996 275 if [ $SPDK_TEST_NVMF -eq 1 ]; then 00:52:28.937 00:28:19.996 276 export NET_TYPE 00:52:28.937 00:28:19.996 277 # The NVMe-oF run test cases are split out like this so that the parser that compiles the 00:52:28.937 00:28:19.996 278 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:52:28.937 00:28:19.996 279 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:52:28.937 00:28:19.996 => 280 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.937 00:28:19.996 281 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.937 00:28:19.996 282 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:52:28.937 00:28:19.996 283 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.937 00:28:19.996 284 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:52:28.937 00:28:19.996 285 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.937 00:28:19.996 ... 00:52:28.937 00:28:19.996 00:52:28.937 00:28:19.996 ========== Backtrace end ==========' 00:52:28.937 + [[ -n 00:28:19.985 ========== Backtrace start: ========== 00:52:28.937 00:28:19.985 00:52:28.937 00:28:19.985 in test/nvmf/target/device_removal.sh:116 -> stop_bdevperf([]) 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 111 00:52:28.937 00:28:19.985 112 sleep 5 00:52:28.937 00:28:19.985 113 } 00:52:28.937 00:28:19.985 114 00:52:28.937 00:28:19.985 115 function stop_bdevperf() { 00:52:28.937 00:28:19.985 => 116 wait $bdevperf_rpc_pid 00:52:28.937 00:28:19.985 117 00:52:28.937 00:28:19.985 118 killprocess $bdevperf_pid 00:52:28.937 00:28:19.985 119 bdevperf_pid= 00:52:28.937 00:28:19.985 120 00:52:28.937 00:28:19.985 121 cat $testdir/try.txt 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 in test/nvmf/target/device_removal.sh:200 -> test_remove_and_rescan([]) 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 195 fi 00:52:28.937 00:28:19.985 196 sleep 2 00:52:28.937 00:28:19.985 197 done 00:52:28.937 00:28:19.985 198 done 00:52:28.937 00:28:19.985 199 00:52:28.937 00:28:19.985 => 200 stop_bdevperf 00:52:28.937 00:28:19.985 201 00:52:28.937 00:28:19.985 202 killprocess $nvmfpid 00:52:28.937 00:28:19.985 203 nvmfpid= 00:52:28.937 00:28:19.985 204 00:52:28.937 00:28:19.985 205 return 0 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1122 -> run_test(["nvmf_device_removal_pci_remove"],["test_remove_and_rescan"]) 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 1117 timing_enter $test_name 00:52:28.937 00:28:19.985 1118 echo "************************************" 00:52:28.937 00:28:19.985 1119 echo "START TEST $test_name" 00:52:28.937 00:28:19.985 1120 echo "************************************" 00:52:28.937 00:28:19.985 1121 xtrace_restore 00:52:28.937 00:28:19.985 1122 time "$@" 00:52:28.937 00:28:19.985 1123 xtrace_disable 00:52:28.937 00:28:19.985 1124 echo "************************************" 00:52:28.937 00:28:19.985 1125 echo "END TEST $test_name" 00:52:28.937 00:28:19.985 1126 echo "************************************" 00:52:28.937 00:28:19.985 1127 timing_exit $test_name 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 in test/nvmf/target/device_removal.sh:312 -> main(["--transport=rdma"]) 00:52:28.937 00:28:19.985 ... 00:52:28.937 00:28:19.985 307 fi 00:52:28.937 00:28:19.985 308 test_bonding_slaves_on_nics "${target_nics[@]}" 00:52:28.937 00:28:19.985 309 } 00:52:28.937 00:28:19.985 310 00:52:28.937 00:28:19.986 311 run_test "nvmf_device_removal_pci_remove_no_srq" test_remove_and_rescan --no-srq 00:52:28.937 00:28:19.986 => 312 run_test "nvmf_device_removal_pci_remove" test_remove_and_rescan 00:52:28.937 00:28:19.986 313 # bond slaves case needs lag_master & vport_manager are enabled by mlxconfig 00:52:28.938 00:28:19.986 314 # and not work on CI machine currently. 00:52:28.938 00:28:19.986 315 # run_test "nvmf_device_removal_bond_slaves" test_bond_slaves 00:52:28.938 00:28:19.986 316 00:52:28.938 00:28:19.986 317 nvmftestfini 00:52:28.938 00:28:19.986 ... 00:52:28.938 00:28:19.986 00:52:28.938 00:28:19.986 ========== Backtrace end ========== 00:52:28.938 00:28:19.995 ========== Backtrace start: ========== 00:52:28.938 00:28:19.995 00:52:28.938 00:28:19.995 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1122 -> run_test(["nvmf_device_removal"],["test/nvmf/target/device_removal.sh"],["--transport=rdma"]) 00:52:28.938 00:28:19.995 ... 00:52:28.938 00:28:19.995 1117 timing_enter $test_name 00:52:28.938 00:28:19.995 1118 echo "************************************" 00:52:28.938 00:28:19.995 1119 echo "START TEST $test_name" 00:52:28.938 00:28:19.995 1120 echo "************************************" 00:52:28.938 00:28:19.995 1121 xtrace_restore 00:52:28.938 00:28:19.995 1122 time "$@" 00:52:28.938 00:28:19.995 1123 xtrace_disable 00:52:28.938 00:28:19.995 1124 echo "************************************" 00:52:28.938 00:28:19.995 1125 echo "END TEST $test_name" 00:52:28.938 00:28:19.995 1126 echo "************************************" 00:52:28.938 00:28:19.995 1127 timing_exit $test_name 00:52:28.938 00:28:19.995 ... 00:52:28.938 00:28:19.995 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf.sh:79 -> main(["--transport=rdma"]) 00:52:28.938 00:28:19.995 ... 00:52:28.938 00:28:19.995 74 TCP_INTERFACE_LIST=("${net_devs[@]}") 00:52:28.938 00:28:19.995 75 if ((${#TCP_INTERFACE_LIST[@]} > 0)); then 00:52:28.938 00:28:19.995 76 run_test "nvmf_perf_adq" $rootdir/test/nvmf/target/perf_adq.sh "${TEST_ARGS[@]}" 00:52:28.938 00:28:19.995 77 fi 00:52:28.938 00:28:19.995 78 elif [[ $SPDK_TEST_NVMF_TRANSPORT == "rdma" ]]; then 00:52:28.938 00:28:19.995 => 79 run_test "nvmf_device_removal" test/nvmf/target/device_removal.sh "${TEST_ARGS[@]}" 00:52:28.938 00:28:19.995 80 run_test "nvmf_srq_overwhelm" "$rootdir/test/nvmf/target/srq_overwhelm.sh" "${TEST_ARGS[@]}" 00:52:28.938 00:28:19.995 81 fi 00:52:28.938 00:28:19.995 82 run_test "nvmf_shutdown" $rootdir/test/nvmf/target/shutdown.sh "${TEST_ARGS[@]}" 00:52:28.938 00:28:19.995 83 fi 00:52:28.938 00:28:19.995 84 00:52:28.938 00:28:19.995 ... 00:52:28.938 00:28:19.995 00:52:28.938 00:28:19.995 ========== Backtrace end ========== 00:52:28.938 00:28:19.996 ========== Backtrace start: ========== 00:52:28.938 00:28:19.996 00:52:28.938 00:28:19.996 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1122 -> run_test(["nvmf_rdma"],["/var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf.sh"],["--transport=rdma"]) 00:52:28.938 00:28:19.996 ... 00:52:28.938 00:28:19.996 1117 timing_enter $test_name 00:52:28.938 00:28:19.996 1118 echo "************************************" 00:52:28.938 00:28:19.996 1119 echo "START TEST $test_name" 00:52:28.938 00:28:19.996 1120 echo "************************************" 00:52:28.938 00:28:19.996 1121 xtrace_restore 00:52:28.938 00:28:19.996 1122 time "$@" 00:52:28.938 00:28:19.996 1123 xtrace_disable 00:52:28.938 00:28:19.996 1124 echo "************************************" 00:52:28.938 00:28:19.996 1125 echo "END TEST $test_name" 00:52:28.938 00:28:19.996 1126 echo "************************************" 00:52:28.938 00:28:19.996 1127 timing_exit $test_name 00:52:28.938 00:28:19.996 ... 00:52:28.938 00:28:19.996 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/autotest.sh:280 -> main(["/var/jenkins/workspace/nvmf-phy-autotest/autorun-spdk.conf"]) 00:52:28.938 00:28:19.996 ... 00:52:28.938 00:28:19.996 275 if [ $SPDK_TEST_NVMF -eq 1 ]; then 00:52:28.938 00:28:19.996 276 export NET_TYPE 00:52:28.938 00:28:19.996 277 # The NVMe-oF run test cases are split out like this so that the parser that compiles the 00:52:28.938 00:28:19.996 278 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:52:28.938 00:28:19.996 279 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:52:28.938 00:28:19.996 => 280 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.938 00:28:19.996 281 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.938 00:28:19.996 282 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:52:28.938 00:28:19.996 283 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.938 00:28:19.996 284 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:52:28.938 00:28:19.996 285 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.938 00:28:19.996 ... 00:52:28.938 00:28:19.996 00:52:28.938 00:28:19.996 ========== Backtrace end ========== ]] 00:52:28.938 + echo '00:28:19.985 ========== Backtrace start: ========== 00:52:28.938 00:28:19.985 00:52:28.938 00:28:19.985 in test/nvmf/target/device_removal.sh:116 -> stop_bdevperf([]) 00:52:28.938 00:28:19.985 ... 00:52:28.938 00:28:19.985 111 00:52:28.938 00:28:19.985 112 sleep 5 00:52:28.938 00:28:19.985 113 } 00:52:28.938 00:28:19.985 114 00:52:28.938 00:28:19.985 115 function stop_bdevperf() { 00:52:28.938 00:28:19.985 => 116 wait $bdevperf_rpc_pid 00:52:28.938 00:28:19.985 117 00:52:28.938 00:28:19.985 118 killprocess $bdevperf_pid 00:52:28.938 00:28:19.985 119 bdevperf_pid= 00:52:28.938 00:28:19.985 120 00:52:28.938 00:28:19.985 121 cat $testdir/try.txt 00:52:28.938 00:28:19.985 ... 00:52:28.938 00:28:19.985 in test/nvmf/target/device_removal.sh:200 -> test_remove_and_rescan([]) 00:52:28.938 00:28:19.985 ... 00:52:28.938 00:28:19.985 195 fi 00:52:28.938 00:28:19.985 196 sleep 2 00:52:28.938 00:28:19.985 197 done 00:52:28.938 00:28:19.985 198 done 00:52:28.938 00:28:19.985 199 00:52:28.938 00:28:19.985 => 200 stop_bdevperf 00:52:28.938 00:28:19.985 201 00:52:28.938 00:28:19.985 202 killprocess $nvmfpid 00:52:28.938 00:28:19.985 203 nvmfpid= 00:52:28.938 00:28:19.985 204 00:52:28.938 00:28:19.985 205 return 0 00:52:28.938 00:28:19.985 ... 00:52:28.938 00:28:19.985 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1122 -> run_test(["nvmf_device_removal_pci_remove"],["test_remove_and_rescan"]) 00:52:28.938 00:28:19.985 ... 00:52:28.938 00:28:19.985 1117 timing_enter $test_name 00:52:28.938 00:28:19.985 1118 echo "************************************" 00:52:28.938 00:28:19.985 1119 echo "START TEST $test_name" 00:52:28.938 00:28:19.985 1120 echo "************************************" 00:52:28.938 00:28:19.985 1121 xtrace_restore 00:52:28.938 00:28:19.985 1122 time "$@" 00:52:28.938 00:28:19.985 1123 xtrace_disable 00:52:28.938 00:28:19.985 1124 echo "************************************" 00:52:28.938 00:28:19.985 1125 echo "END TEST $test_name" 00:52:28.938 00:28:19.985 1126 echo "************************************" 00:52:28.938 00:28:19.985 1127 timing_exit $test_name 00:52:28.938 00:28:19.985 ... 00:52:28.938 00:28:19.985 in test/nvmf/target/device_removal.sh:312 -> main(["--transport=rdma"]) 00:52:28.938 00:28:19.985 ... 00:52:28.938 00:28:19.985 307 fi 00:52:28.938 00:28:19.985 308 test_bonding_slaves_on_nics "${target_nics[@]}" 00:52:28.938 00:28:19.985 309 } 00:52:28.938 00:28:19.985 310 00:52:28.938 00:28:19.986 311 run_test "nvmf_device_removal_pci_remove_no_srq" test_remove_and_rescan --no-srq 00:52:28.938 00:28:19.986 => 312 run_test "nvmf_device_removal_pci_remove" test_remove_and_rescan 00:52:28.938 00:28:19.986 313 # bond slaves case needs lag_master & vport_manager are enabled by mlxconfig 00:52:28.938 00:28:19.986 314 # and not work on CI machine currently. 00:52:28.938 00:28:19.986 315 # run_test "nvmf_device_removal_bond_slaves" test_bond_slaves 00:52:28.938 00:28:19.986 316 00:52:28.938 00:28:19.986 317 nvmftestfini 00:52:28.938 00:28:19.986 ... 00:52:28.938 00:28:19.986 00:52:28.938 00:28:19.986 ========== Backtrace end ========== 00:52:28.938 00:28:19.995 ========== Backtrace start: ========== 00:52:28.938 00:28:19.995 00:52:28.938 00:28:19.995 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1122 -> run_test(["nvmf_device_removal"],["test/nvmf/target/device_removal.sh"],["--transport=rdma"]) 00:52:28.938 00:28:19.995 ... 00:52:28.938 00:28:19.995 1117 timing_enter $test_name 00:52:28.938 00:28:19.995 1118 echo "************************************" 00:52:28.938 00:28:19.995 1119 echo "START TEST $test_name" 00:52:28.938 00:28:19.995 1120 echo "************************************" 00:52:28.938 00:28:19.995 1121 xtrace_restore 00:52:28.938 00:28:19.995 1122 time "$@" 00:52:28.938 00:28:19.995 1123 xtrace_disable 00:52:28.938 00:28:19.995 1124 echo "************************************" 00:52:28.938 00:28:19.995 1125 echo "END TEST $test_name" 00:52:28.938 00:28:19.995 1126 echo "************************************" 00:52:28.938 00:28:19.995 1127 timing_exit $test_name 00:52:28.938 00:28:19.995 ... 00:52:28.938 00:28:19.995 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf.sh:79 -> main(["--transport=rdma"]) 00:52:28.938 00:28:19.995 ... 00:52:28.938 00:28:19.995 74 TCP_INTERFACE_LIST=("${net_devs[@]}") 00:52:28.938 00:28:19.995 75 if ((${#TCP_INTERFACE_LIST[@]} > 0)); then 00:52:28.938 00:28:19.995 76 run_test "nvmf_perf_adq" $rootdir/test/nvmf/target/perf_adq.sh "${TEST_ARGS[@]}" 00:52:28.938 00:28:19.995 77 fi 00:52:28.938 00:28:19.995 78 elif [[ $SPDK_TEST_NVMF_TRANSPORT == "rdma" ]]; then 00:52:28.938 00:28:19.995 => 79 run_test "nvmf_device_removal" test/nvmf/target/device_removal.sh "${TEST_ARGS[@]}" 00:52:28.938 00:28:19.995 80 run_test "nvmf_srq_overwhelm" "$rootdir/test/nvmf/target/srq_overwhelm.sh" "${TEST_ARGS[@]}" 00:52:28.938 00:28:19.995 81 fi 00:52:28.938 00:28:19.995 82 run_test "nvmf_shutdown" $rootdir/test/nvmf/target/shutdown.sh "${TEST_ARGS[@]}" 00:52:28.938 00:28:19.995 83 fi 00:52:28.938 00:28:19.995 84 00:52:28.938 00:28:19.995 ... 00:52:28.938 00:28:19.995 00:52:28.938 00:28:19.995 ========== Backtrace end ========== 00:52:28.938 00:28:19.996 ========== Backtrace start: ========== 00:52:28.938 00:28:19.996 00:52:28.938 00:28:19.996 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/test/common/autotest_common.sh:1122 -> run_test(["nvmf_rdma"],["/var/jenkins/workspace/nvmf-phy-autotest/spdk/test/nvmf/nvmf.sh"],["--transport=rdma"]) 00:52:28.938 00:28:19.996 ... 00:52:28.938 00:28:19.996 1117 timing_enter $test_name 00:52:28.938 00:28:19.996 1118 echo "************************************" 00:52:28.938 00:28:19.996 1119 echo "START TEST $test_name" 00:52:28.938 00:28:19.996 1120 echo "************************************" 00:52:28.938 00:28:19.996 1121 xtrace_restore 00:52:28.938 00:28:19.996 1122 time "$@" 00:52:28.938 00:28:19.996 1123 xtrace_disable 00:52:28.938 00:28:19.996 1124 echo "************************************" 00:52:28.938 00:28:19.996 1125 echo "END TEST $test_name" 00:52:28.938 00:28:19.996 1126 echo "************************************" 00:52:28.938 00:28:19.996 1127 timing_exit $test_name 00:52:28.939 00:28:19.996 ... 00:52:28.939 00:28:19.996 in /var/jenkins/workspace/nvmf-phy-autotest/spdk/autotest.sh:280 -> main(["/var/jenkins/workspace/nvmf-phy-autotest/autorun-spdk.conf"]) 00:52:28.939 00:28:19.996 ... 00:52:28.939 00:28:19.996 275 if [ $SPDK_TEST_NVMF -eq 1 ]; then 00:52:28.939 00:28:19.996 276 export NET_TYPE 00:52:28.939 00:28:19.996 277 # The NVMe-oF run test cases are split out like this so that the parser that compiles the 00:52:28.939 00:28:19.996 278 # list of all tests can properly differentiate them. Please do not merge them into one line. 00:52:28.939 00:28:19.996 279 if [ "$SPDK_TEST_NVMF_TRANSPORT" = "rdma" ]; then 00:52:28.939 00:28:19.996 => 280 run_test "nvmf_rdma" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.939 00:28:19.996 281 run_test "spdkcli_nvmf_rdma" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.939 00:28:19.996 282 elif [ "$SPDK_TEST_NVMF_TRANSPORT" = "tcp" ]; then 00:52:28.939 00:28:19.996 283 run_test "nvmf_tcp" $rootdir/test/nvmf/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.939 00:28:19.996 284 if [[ $SPDK_TEST_URING -eq 0 ]]; then 00:52:28.939 00:28:19.996 285 run_test "spdkcli_nvmf_tcp" $rootdir/test/spdkcli/nvmf.sh --transport=$SPDK_TEST_NVMF_TRANSPORT 00:52:28.939 00:28:19.996 ... 00:52:28.939 00:28:19.996 00:52:28.939 00:28:19.996 ========== Backtrace end ==========' 00:52:28.939 + for build in "${build_log_list[@]}" 00:52:28.939 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/short-fuzz-phy-autotest_26177/build.log 00:52:28.939 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.939 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/short-fuzz-phy-autotest_26177/build.log 00:52:28.939 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/short-fuzz-phy-autotest_26177/build.log 00:52:28.939 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/short-fuzz-phy-autotest_26177 00:52:28.939 + job_name=short-fuzz-phy-autotest_26177 00:52:28.939 + start_pattern='Backtrace start' 00:52:28.939 + end_pattern='Backtrace end' 00:52:28.939 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/short-fuzz-phy-autotest_26177/build.log ]] 00:52:28.939 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/short-fuzz-phy-autotest_26177/build.log 00:52:28.939 + backtraces= 00:52:28.939 + [[ -n '' ]] 00:52:28.939 + for build in "${build_log_list[@]}" 00:52:28.939 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/zns-vg-autotest_76566/build.log 00:52:28.939 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.939 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/zns-vg-autotest_76566/build.log 00:52:28.939 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/zns-vg-autotest_76566/build.log 00:52:28.939 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/zns-vg-autotest_76566 00:52:28.939 + job_name=zns-vg-autotest_76566 00:52:28.939 + start_pattern='Backtrace start' 00:52:28.939 + end_pattern='Backtrace end' 00:52:28.939 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/zns-vg-autotest_76566/build.log ]] 00:52:28.939 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/zns-vg-autotest_76566/build.log 00:52:28.939 + backtraces= 00:52:28.939 + [[ -n '' ]] 00:52:28.939 + for build in "${build_log_list[@]}" 00:52:28.939 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-initiator-vg-autotest_77480/build.log 00:52:28.939 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.939 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-initiator-vg-autotest_77480/build.log 00:52:28.939 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-initiator-vg-autotest_77480/build.log 00:52:28.939 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-initiator-vg-autotest_77480 00:52:28.939 + job_name=vhost-initiator-vg-autotest_77480 00:52:28.939 + start_pattern='Backtrace start' 00:52:28.939 + end_pattern='Backtrace end' 00:52:28.939 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-initiator-vg-autotest_77480/build.log ]] 00:52:28.939 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-initiator-vg-autotest_77480/build.log 00:52:28.939 + backtraces= 00:52:28.939 + [[ -n '' ]] 00:52:28.939 + for build in "${build_log_list[@]}" 00:52:28.939 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/clang-vg-autotest_57624/build.log 00:52:28.939 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.939 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/clang-vg-autotest_57624/build.log 00:52:28.939 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/clang-vg-autotest_57624/build.log 00:52:28.939 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/clang-vg-autotest_57624 00:52:28.939 + job_name=clang-vg-autotest_57624 00:52:28.939 + start_pattern='Backtrace start' 00:52:28.939 + end_pattern='Backtrace end' 00:52:28.939 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/clang-vg-autotest_57624/build.log ]] 00:52:28.939 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/clang-vg-autotest_57624/build.log 00:52:28.939 + backtraces= 00:52:28.939 + [[ -n '' ]] 00:52:28.939 + for build in "${build_log_list[@]}" 00:52:28.939 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-phy-autotest_72352/build.log 00:52:28.939 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.939 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-phy-autotest_72352/build.log 00:52:28.939 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-phy-autotest_72352/build.log 00:52:28.939 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-phy-autotest_72352 00:52:28.939 + job_name=nvme-phy-autotest_72352 00:52:28.939 + start_pattern='Backtrace start' 00:52:28.939 + end_pattern='Backtrace end' 00:52:28.939 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-phy-autotest_72352/build.log ]] 00:52:28.939 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-phy-autotest_72352/build.log 00:52:28.939 + backtraces= 00:52:28.939 + [[ -n '' ]] 00:52:28.939 + for build in "${build_log_list[@]}" 00:52:28.939 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vmd-phy-autotest_2618/build.log 00:52:28.939 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.939 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vmd-phy-autotest_2618/build.log 00:52:28.939 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vmd-phy-autotest_2618/build.log 00:52:28.939 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vmd-phy-autotest_2618 00:52:28.939 + job_name=vmd-phy-autotest_2618 00:52:28.939 + start_pattern='Backtrace start' 00:52:28.939 + end_pattern='Backtrace end' 00:52:28.939 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vmd-phy-autotest_2618/build.log ]] 00:52:28.939 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vmd-phy-autotest_2618/build.log 00:52:28.939 + backtraces= 00:52:28.939 + [[ -n '' ]] 00:52:28.939 + for build in "${build_log_list[@]}" 00:52:28.939 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/unittest-vg-autotest_63373/build.log 00:52:28.939 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.939 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/unittest-vg-autotest_63373/build.log 00:52:28.939 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/unittest-vg-autotest_63373/build.log 00:52:28.939 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/unittest-vg-autotest_63373 00:52:28.939 + job_name=unittest-vg-autotest_63373 00:52:28.939 + start_pattern='Backtrace start' 00:52:28.939 + end_pattern='Backtrace end' 00:52:28.939 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/unittest-vg-autotest_63373/build.log ]] 00:52:28.939 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/unittest-vg-autotest_63373/build.log 00:52:28.939 + backtraces= 00:52:28.939 + [[ -n '' ]] 00:52:28.939 + for build in "${build_log_list[@]}" 00:52:28.939 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-vg-autotest_77498/build.log 00:52:28.939 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.939 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-vg-autotest_77498/build.log 00:52:28.939 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-vg-autotest_77498/build.log 00:52:28.939 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-vg-autotest_77498 00:52:28.939 + job_name=iscsi-vg-autotest_77498 00:52:28.939 + start_pattern='Backtrace start' 00:52:28.939 + end_pattern='Backtrace end' 00:52:28.939 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-vg-autotest_77498/build.log ]] 00:52:28.939 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-vg-autotest_77498/build.log 00:52:28.939 + backtraces= 00:52:28.939 + [[ -n '' ]] 00:52:28.939 + for build in "${build_log_list[@]}" 00:52:28.939 + create_errtrace_file /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dsa-phy-autotest_4880/build.log 00:52:28.939 + local build_log job_name start_pattern end_pattern backtraces 00:52:28.939 + build_log=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dsa-phy-autotest_4880/build.log 00:52:28.939 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dsa-phy-autotest_4880/build.log 00:52:28.939 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dsa-phy-autotest_4880 00:52:28.939 + job_name=dsa-phy-autotest_4880 00:52:28.939 + start_pattern='Backtrace start' 00:52:28.939 + end_pattern='Backtrace end' 00:52:28.939 + [[ -f /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dsa-phy-autotest_4880/build.log ]] 00:52:28.939 ++ sed -n '/Backtrace start/,/Backtrace end/p' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dsa-phy-autotest_4880/build.log 00:52:28.939 + backtraces= 00:52:28.939 + [[ -n '' ]] 00:52:28.939 + run_flamegraph 00:52:28.939 + create_timing_file 00:52:28.939 + local array file 00:52:28.939 + array=($(find $WORKSPACE -type f -name 'timing.txt')) 00:52:28.939 ++ find /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main -type f -name timing.txt 00:52:29.199 + timing_dir=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/post_process 00:52:29.199 + mkdir -p /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/post_process 00:52:29.199 + timing_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/post_process/total_timing.txt 00:52:29.199 + for file in "${array[@]}" 00:52:29.199 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu20-vg-autotest_73779/timing.txt 00:52:29.199 + local job_file job_name 00:52:29.199 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu20-vg-autotest_73779/timing.txt 00:52:29.199 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu20-vg-autotest_73779/timing.txt 00:52:29.199 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu20-vg-autotest_73779 00:52:29.199 + job_name=ubuntu20-vg-autotest_73779 00:52:29.199 + sed 's/^/ubuntu20-vg-autotest_73779;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu20-vg-autotest_73779/timing.txt 00:52:29.199 + for file in "${array[@]}" 00:52:29.199 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-uring-vg-autotest_77385/timing.txt 00:52:29.199 + local job_file job_name 00:52:29.199 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-uring-vg-autotest_77385/timing.txt 00:52:29.199 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-uring-vg-autotest_77385/timing.txt 00:52:29.199 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-uring-vg-autotest_77385 00:52:29.199 + job_name=nvmf-tcp-uring-vg-autotest_77385 00:52:29.199 + sed 's/^/nvmf-tcp-uring-vg-autotest_77385;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-uring-vg-autotest_77385/timing.txt 00:52:29.199 + for file in "${array[@]}" 00:52:29.199 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/rocky9-vg-autotest_16070/timing.txt 00:52:29.199 + local job_file job_name 00:52:29.199 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/rocky9-vg-autotest_16070/timing.txt 00:52:29.199 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/rocky9-vg-autotest_16070/timing.txt 00:52:29.199 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/rocky9-vg-autotest_16070 00:52:29.199 + job_name=rocky9-vg-autotest_16070 00:52:29.199 + sed 's/^/rocky9-vg-autotest_16070;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/rocky9-vg-autotest_16070/timing.txt 00:52:29.199 + for file in "${array[@]}" 00:52:29.199 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-cmb-pmr-vg-autotest_78583/timing.txt 00:52:29.199 + local job_file job_name 00:52:29.199 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-cmb-pmr-vg-autotest_78583/timing.txt 00:52:29.199 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-cmb-pmr-vg-autotest_78583/timing.txt 00:52:29.199 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-cmb-pmr-vg-autotest_78583 00:52:29.199 + job_name=nvme-cmb-pmr-vg-autotest_78583 00:52:29.199 + sed 's/^/nvme-cmb-pmr-vg-autotest_78583;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-cmb-pmr-vg-autotest_78583/timing.txt 00:52:29.199 + for file in "${array[@]}" 00:52:29.199 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/lvol-vg-autotest_77409/timing.txt 00:52:29.199 + local job_file job_name 00:52:29.199 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/lvol-vg-autotest_77409/timing.txt 00:52:29.199 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/lvol-vg-autotest_77409/timing.txt 00:52:29.199 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/lvol-vg-autotest_77409 00:52:29.199 + job_name=lvol-vg-autotest_77409 00:52:29.199 + sed 's/^/lvol-vg-autotest_77409;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/lvol-vg-autotest_77409/timing.txt 00:52:29.199 + for file in "${array[@]}" 00:52:29.199 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/crypto-phy-autotest_26180/timing.txt 00:52:29.199 + local job_file job_name 00:52:29.199 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/crypto-phy-autotest_26180/timing.txt 00:52:29.199 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/crypto-phy-autotest_26180/timing.txt 00:52:29.199 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/crypto-phy-autotest_26180 00:52:29.199 + job_name=crypto-phy-autotest_26180 00:52:29.199 + sed 's/^/crypto-phy-autotest_26180;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/crypto-phy-autotest_26180/timing.txt 00:52:29.199 + for file in "${array[@]}" 00:52:29.199 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/centos7-vg-autotest_73639/timing.txt 00:52:29.199 + local job_file job_name 00:52:29.199 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/centos7-vg-autotest_73639/timing.txt 00:52:29.199 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/centos7-vg-autotest_73639/timing.txt 00:52:29.199 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/centos7-vg-autotest_73639 00:52:29.199 + job_name=centos7-vg-autotest_73639 00:52:29.199 + sed 's/^/centos7-vg-autotest_73639;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/centos7-vg-autotest_73639/timing.txt 00:52:29.199 + for file in "${array[@]}" 00:52:29.199 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu22-vg-autotest_37793/timing.txt 00:52:29.199 + local job_file job_name 00:52:29.199 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu22-vg-autotest_37793/timing.txt 00:52:29.199 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu22-vg-autotest_37793/timing.txt 00:52:29.199 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu22-vg-autotest_37793 00:52:29.199 + job_name=ubuntu22-vg-autotest_37793 00:52:29.199 + sed 's/^/ubuntu22-vg-autotest_37793;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu22-vg-autotest_37793/timing.txt 00:52:29.199 + for file in "${array[@]}" 00:52:29.199 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-phy-autotest_72311/timing.txt 00:52:29.199 + local job_file job_name 00:52:29.199 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-phy-autotest_72311/timing.txt 00:52:29.199 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-phy-autotest_72311/timing.txt 00:52:29.199 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-phy-autotest_72311 00:52:29.199 + job_name=nvmf-tcp-phy-autotest_72311 00:52:29.199 + sed 's/^/nvmf-tcp-phy-autotest_72311;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-phy-autotest_72311/timing.txt 00:52:29.199 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/autobuild-vg-autotest_63416/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/autobuild-vg-autotest_63416/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/autobuild-vg-autotest_63416/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/autobuild-vg-autotest_63416 00:52:29.200 + job_name=autobuild-vg-autotest_63416 00:52:29.200 + sed 's/^/autobuild-vg-autotest_63416;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/autobuild-vg-autotest_63416/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-vg-autotest_77451/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-vg-autotest_77451/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-vg-autotest_77451/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-vg-autotest_77451 00:52:29.200 + job_name=nvmf-tcp-vg-autotest_77451 00:52:29.200 + sed 's/^/nvmf-tcp-vg-autotest_77451;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-vg-autotest_77451/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-vg-autotest_77460/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-vg-autotest_77460/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-vg-autotest_77460/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-vg-autotest_77460 00:52:29.200 + job_name=nvme-vg-autotest_77460 00:52:29.200 + sed 's/^/nvme-vg-autotest_77460;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-vg-autotest_77460/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vfio-user-phy-autotest_26188/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vfio-user-phy-autotest_26188/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vfio-user-phy-autotest_26188/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vfio-user-phy-autotest_26188 00:52:29.200 + job_name=vfio-user-phy-autotest_26188 00:52:29.200 + sed 's/^/vfio-user-phy-autotest_26188;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vfio-user-phy-autotest_26188/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/scanbuild-vg-autotest_63375/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/scanbuild-vg-autotest_63375/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/scanbuild-vg-autotest_63375/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/scanbuild-vg-autotest_63375 00:52:29.200 + job_name=scanbuild-vg-autotest_63375 00:52:29.200 + sed 's/^/scanbuild-vg-autotest_63375;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/scanbuild-vg-autotest_63375/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/valgrind-vg-autotest_49190/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/valgrind-vg-autotest_49190/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/valgrind-vg-autotest_49190/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/valgrind-vg-autotest_49190 00:52:29.200 + job_name=valgrind-vg-autotest_49190 00:52:29.200 + sed 's/^/valgrind-vg-autotest_49190;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/valgrind-vg-autotest_49190/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/BlobFS-phy-autotest_26186/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/BlobFS-phy-autotest_26186/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/BlobFS-phy-autotest_26186/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/BlobFS-phy-autotest_26186 00:52:29.200 + job_name=BlobFS-phy-autotest_26186 00:52:29.200 + sed 's/^/BlobFS-phy-autotest_26186;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/BlobFS-phy-autotest_26186/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-uring-vg-autotest_77365/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-uring-vg-autotest_77365/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-uring-vg-autotest_77365/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-uring-vg-autotest_77365 00:52:29.200 + job_name=iscsi-uring-vg-autotest_77365 00:52:29.200 + sed 's/^/iscsi-uring-vg-autotest_77365;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-uring-vg-autotest_77365/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-phy-autotest_26205/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-phy-autotest_26205/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-phy-autotest_26205/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-phy-autotest_26205 00:52:29.200 + job_name=vhost-phy-autotest_26205 00:52:29.200 + sed 's/^/vhost-phy-autotest_26205;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-phy-autotest_26205/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-phy-autotest_72424/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-phy-autotest_72424/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-phy-autotest_72424/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-phy-autotest_72424 00:52:29.200 + job_name=nvmf-phy-autotest_72424 00:52:29.200 + sed 's/^/nvmf-phy-autotest_72424;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-phy-autotest_72424/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/short-fuzz-phy-autotest_26177/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/short-fuzz-phy-autotest_26177/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/short-fuzz-phy-autotest_26177/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/short-fuzz-phy-autotest_26177 00:52:29.200 + job_name=short-fuzz-phy-autotest_26177 00:52:29.200 + sed 's/^/short-fuzz-phy-autotest_26177;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/short-fuzz-phy-autotest_26177/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/zns-vg-autotest_76566/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/zns-vg-autotest_76566/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/zns-vg-autotest_76566/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/zns-vg-autotest_76566 00:52:29.200 + job_name=zns-vg-autotest_76566 00:52:29.200 + sed 's/^/zns-vg-autotest_76566;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/zns-vg-autotest_76566/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-initiator-vg-autotest_77480/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-initiator-vg-autotest_77480/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-initiator-vg-autotest_77480/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-initiator-vg-autotest_77480 00:52:29.200 + job_name=vhost-initiator-vg-autotest_77480 00:52:29.200 + sed 's/^/vhost-initiator-vg-autotest_77480;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-initiator-vg-autotest_77480/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/clang-vg-autotest_57624/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/clang-vg-autotest_57624/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/clang-vg-autotest_57624/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/clang-vg-autotest_57624 00:52:29.200 + job_name=clang-vg-autotest_57624 00:52:29.200 + sed 's/^/clang-vg-autotest_57624;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/clang-vg-autotest_57624/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-phy-autotest_72352/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-phy-autotest_72352/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-phy-autotest_72352/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-phy-autotest_72352 00:52:29.200 + job_name=nvme-phy-autotest_72352 00:52:29.200 + sed 's/^/nvme-phy-autotest_72352;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-phy-autotest_72352/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vmd-phy-autotest_2618/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vmd-phy-autotest_2618/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vmd-phy-autotest_2618/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vmd-phy-autotest_2618 00:52:29.200 + job_name=vmd-phy-autotest_2618 00:52:29.200 + sed 's/^/vmd-phy-autotest_2618;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vmd-phy-autotest_2618/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/unittest-vg-autotest_63373/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/unittest-vg-autotest_63373/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/unittest-vg-autotest_63373/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/unittest-vg-autotest_63373 00:52:29.200 + job_name=unittest-vg-autotest_63373 00:52:29.200 + sed 's/^/unittest-vg-autotest_63373;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/unittest-vg-autotest_63373/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-vg-autotest_77498/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-vg-autotest_77498/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-vg-autotest_77498/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-vg-autotest_77498 00:52:29.200 + job_name=iscsi-vg-autotest_77498 00:52:29.200 + sed 's/^/iscsi-vg-autotest_77498;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-vg-autotest_77498/timing.txt 00:52:29.200 + for file in "${array[@]}" 00:52:29.200 + add_sub_job_timing /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dsa-phy-autotest_4880/timing.txt 00:52:29.200 + local job_file job_name 00:52:29.200 + job_file=/var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dsa-phy-autotest_4880/timing.txt 00:52:29.200 +++ dirname /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dsa-phy-autotest_4880/timing.txt 00:52:29.200 ++ basename /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dsa-phy-autotest_4880 00:52:29.200 + job_name=dsa-phy-autotest_4880 00:52:29.200 + sed 's/^/dsa-phy-autotest_4880;autorun;/' /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dsa-phy-autotest_4880/timing.txt 00:52:29.201 + /usr/local/FlameGraph/flamegraph.pl --title 'Build Timing' --nametype Step: --countname seconds /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/post_process/total_timing.txt 00:52:29.201 + run_autorun_post 00:52:29.201 + s_opt= 00:52:29.201 + [[ '' == true ]] 00:52:29.201 + spdk/autorun_post.py -d ./ -r ./spdk 00:52:39.163 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/lib/util/crc64.c:165 00:52:39.163 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/lib/util/crc32c.c:21 00:52:40.095 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/lib/util/crc16.c:628 00:52:40.095 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/lib/util/crc16.c:634 00:52:40.353 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/lib/util/xor.c:120 00:52:48.456 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/module/bdev/nvme/bdev_mdns_client.c:613 00:52:48.456 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/module/bdev/nvme/bdev_mdns_client.c:456 00:52:48.457 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/module/bdev/nvme/bdev_mdns_client.c:582 00:52:48.457 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/module/bdev/nvme/bdev_mdns_client.c:561 00:52:55.004 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/module/blobfs/bdev/blobfs_bdev.c:254 00:52:59.182 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/module/blobfs/bdev/blobfs_bdev.c:254 00:53:25.708 lcov: WARNING: function data mismatch at /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/module/blobfs/bdev/blobfs_bdev.c:254 00:53:52.228 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/nvmf_transport.h! 00:53:52.228 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/trace.h! 00:53:52.228 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/nvme_spec.h! 00:53:52.228 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/bdev_module.h! 00:53:52.228 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/nvme.h! 00:53:52.228 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/thread.h! 00:53:52.228 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/endian.h! 00:53:52.228 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/util.h! 00:53:52.228 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/base64.h! 00:53:52.228 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/histogram_data.h! 00:53:52.228 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/mmio.h! 00:53:52.228 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/histogram_data.h/histogram_ut.c! 00:53:52.792 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/lib/env_ocf/include/ocf/ocf_mngt.h! 00:53:52.792 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/lib/env_ocf/include/ocf/ocf_io.h! 00:53:52.792 genhtml: WARNING: cannot read /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/spdk/lib/env_ocf/include/ocf/ocf_core.h! 00:54:10.848 -----Begin Post Process Script------ 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/rocky9-vg-autotest_16070/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-cmb-pmr-vg-autotest_78583/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/lvol-vg-autotest_77409/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/crypto-phy-autotest_26180/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/centos7-vg-autotest_73639/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/ubuntu22-vg-autotest_37793/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-phy-autotest_72311/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvmf-tcp-vg-autotest_77451/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vfio-user-phy-autotest_26188/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/valgrind-vg-autotest_49190/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/BlobFS-phy-autotest_26186/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-uring-vg-autotest_77365/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-phy-autotest_26205/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/zns-vg-autotest_76566/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vhost-initiator-vg-autotest_77480/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/nvme-phy-autotest_72352/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/vmd-phy-autotest_2618/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/unittest-vg-autotest_63373/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/iscsi-vg-autotest_77498/cov_total.info 00:54:10.848 /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/dsa-phy-autotest_4880/cov_total.info 00:54:10.848 00:54:10.848 00:54:10.848 -----Tests Executed in Build------ 00:54:10.848 accel 00:54:10.848 accel_assign_opcode 00:54:10.848 accel_cdev_comp 00:54:10.848 accel_cdev_decmop_full 00:54:10.848 accel_cdev_decomp 00:54:10.848 accel_cdev_decomp_full_mcore 00:54:10.848 accel_cdev_decomp_full_mthread 00:54:10.848 accel_cdev_decomp_mcore 00:54:10.848 accel_cdev_decomp_mthread 00:54:10.848 accel_comp 00:54:10.848 accel_compare 00:54:10.848 accel_compress_verify 00:54:10.848 accel_copy 00:54:10.848 accel_copy_crc32c 00:54:10.848 accel_copy_crc32c_C2 00:54:10.848 accel_crc32c 00:54:10.848 accel_crc32c_C2 00:54:10.848 accel_decmop_full 00:54:10.848 accel_decomp 00:54:10.848 accel_decomp_full_mcore 00:54:10.848 accel_decomp_full_mthread 00:54:10.848 accel_decomp_mcore 00:54:10.848 accel_decomp_mthread 00:54:10.848 accel_dif_functional_tests 00:54:10.848 accel_dif_generate 00:54:10.848 accel_dif_generate_copy 00:54:10.848 accel_dif_verify 00:54:10.848 accel_dualcast 00:54:10.848 accel_fill 00:54:10.848 accel_help 00:54:10.848 accel_missing_filename 00:54:10.848 accel_negative_buffers 00:54:10.848 accel_rpc 00:54:10.848 accel_scan_dsa_modules 00:54:10.848 accel_scan_iaa_modules 00:54:10.848 accel_wrong_workload 00:54:10.848 accel_xor 00:54:10.848 acl 00:54:10.848 alias_rpc 00:54:10.848 allowed 00:54:10.848 app_cmdline 00:54:10.848 app_repeat 00:54:10.848 asan 00:54:10.848 autobuild 00:54:10.848 autobuild_build_doc 00:54:10.848 autobuild_check_dpdk_pci_api 00:54:10.848 autobuild_check_format 00:54:10.849 autobuild_check_so_deps 00:54:10.849 autobuild_external_code 00:54:10.849 autobuild_generated_files_check 00:54:10.849 autobuild_generated_files_check_post_clean 00:54:10.849 autobuild_header_dependency_check 00:54:10.849 autobuild_llvm_precompile 00:54:10.849 autobuild_make_install 00:54:10.849 autobuild_make_uninstall 00:54:10.849 autobuild_ocf_precompile 00:54:10.849 bdev_bounds 00:54:10.849 bdev_crypto_enomem 00:54:10.849 bdev_error 00:54:10.849 bdev_fio 00:54:10.849 bdev_fio_rw_verify 00:54:10.849 bdev_fio_trim 00:54:10.849 bdev_gpt_uuid 00:54:10.849 bdev_hello_world 00:54:10.849 bdev_json_nonarray 00:54:10.849 bdev_json_nonenclosed 00:54:10.849 bdev_nbd 00:54:10.849 bdev_nvme_reset_stuck_adm_cmd 00:54:10.849 bdev_qd_sampling 00:54:10.849 bdev_qos 00:54:10.849 bdev_qos_bw 00:54:10.849 bdev_qos_iops 00:54:10.849 bdev_qos_ro_bw 00:54:10.849 bdev_raid 00:54:10.849 bdev_stat 00:54:10.849 bdev_verify 00:54:10.849 bdev_verify_big_io 00:54:10.849 bdev_write_zeroes 00:54:10.849 bdevperf_config 00:54:10.849 blob_io_wait 00:54:10.849 blobfs 00:54:10.849 blobfs_mkfs 00:54:10.849 blobstore 00:54:10.849 blobstore_grow 00:54:10.849 blockdev_crypto_aesni 00:54:10.849 blockdev_crypto_qat 00:54:10.849 blockdev_crypto_sw 00:54:10.849 blockdev_general 00:54:10.849 blockdev_nvme 00:54:10.849 blockdev_nvme_gpt 00:54:10.849 blockdev_raid5f 00:54:10.849 blockdev_rbd 00:54:10.849 build_native_dpdk 00:54:10.849 build_rpm_from_gen_spec 00:54:10.849 build_shared_native_dpdk_rpm 00:54:10.849 build_shared_rpm 00:54:10.849 build_shared_rpm_with_rpmed_dpdk 00:54:10.849 chaining 00:54:10.849 check_header_filenames 00:54:10.849 cmb_copy 00:54:10.849 compress_compdev 00:54:10.849 compress_isal 00:54:10.849 confirm_abi_deps 00:54:10.849 confirm_makefile_deps 00:54:10.849 cpu_locks 00:54:10.849 custom_alloc 00:54:10.849 dd_bs_lt_native_bs 00:54:10.849 dd_copy_to_out_bdev 00:54:10.849 dd_double_input 00:54:10.849 dd_double_output 00:54:10.849 dd_flag_append 00:54:10.849 dd_flag_append_forced_aio 00:54:10.849 dd_flag_directory 00:54:10.849 dd_flag_directory_forced_aio 00:54:10.849 dd_flag_noatime 00:54:10.849 dd_flag_noatime_forced_aio 00:54:10.849 dd_flag_nofollow 00:54:10.849 dd_flag_nofollow_forced_aio 00:54:10.849 dd_flags_misc 00:54:10.849 dd_flags_misc_forced_aio 00:54:10.849 dd_inflate_file 00:54:10.849 dd_invalid_arguments 00:54:10.849 dd_invalid_count 00:54:10.849 dd_invalid_iflag 00:54:10.849 dd_invalid_json 00:54:10.849 dd_invalid_oflag 00:54:10.849 dd_malloc_copy 00:54:10.849 dd_no_input 00:54:10.849 dd_no_output 00:54:10.849 dd_offset_magic 00:54:10.849 dd_rw 00:54:10.849 dd_rw_offset 00:54:10.849 dd_smaller_blocksize 00:54:10.849 dd_sparse_bdev_to_file 00:54:10.849 dd_sparse_file_to_bdev 00:54:10.849 dd_sparse_file_to_file 00:54:10.849 dd_unknown_flag 00:54:10.849 dd_uring_copy 00:54:10.849 dd_wrong_blocksize 00:54:10.849 default_locks 00:54:10.849 default_locks_via_rpc 00:54:10.849 default_setup 00:54:10.849 denied 00:54:10.849 devices 00:54:10.849 dm_mount 00:54:10.849 dma 00:54:10.849 dpdk_governor 00:54:10.849 dpdk_mem_utility 00:54:10.849 driver 00:54:10.849 env 00:54:10.849 env_dpdk_post_init 00:54:10.849 env_mem_callbacks 00:54:10.849 env_memory 00:54:10.849 env_pci 00:54:10.849 env_vtophys 00:54:10.849 even_2G_alloc 00:54:10.849 event 00:54:10.849 event_perf 00:54:10.849 event_reactor 00:54:10.849 event_reactor_perf 00:54:10.849 event_scheduler 00:54:10.849 exit_on_failed_rpc_init 00:54:10.849 external_make_accel_driver_shared 00:54:10.849 external_make_accel_driver_static 00:54:10.849 external_make_accel_module_shared 00:54:10.849 external_make_accel_module_static 00:54:10.849 external_make_hello_bdev_shared_combo 00:54:10.849 external_make_hello_bdev_shared_iso 00:54:10.849 external_make_hello_bdev_static 00:54:10.849 external_make_hello_no_bdev_shared_combo 00:54:10.849 external_make_hello_no_bdev_shared_iso 00:54:10.849 external_make_hello_no_bdev_static 00:54:10.849 external_make_nvme_shared 00:54:10.849 external_make_nvme_static 00:54:10.849 external_run_accel_driver_shared 00:54:10.849 external_run_accel_driver_static 00:54:10.849 external_run_accel_module_shared 00:54:10.849 external_run_accel_module_static 00:54:10.849 external_run_hello_bdev_shared_combo 00:54:10.849 external_run_hello_bdev_shared_iso 00:54:10.849 external_run_hello_bdev_static 00:54:10.849 external_run_hello_no_bdev_shared_combo 00:54:10.849 external_run_hello_no_bdev_shared_iso 00:54:10.849 external_run_hello_no_bdev_static 00:54:10.849 external_run_nvme_shared 00:54:10.849 external_run_nvme_static 00:54:10.849 filesystem_btrfs 00:54:10.849 filesystem_ext4 00:54:10.849 filesystem_in_capsule_btrfs 00:54:10.849 filesystem_in_capsule_ext4 00:54:10.849 filesystem_in_capsule_xfs 00:54:10.849 filesystem_xfs 00:54:10.849 fio_dif_1_default 00:54:10.849 fio_dif_1_multi_subsystems 00:54:10.849 fio_dif_digest 00:54:10.849 fio_dif_rand_params 00:54:10.849 go_rpc 00:54:10.849 guess_driver 00:54:10.849 hello_blob 00:54:10.849 hugepages 00:54:10.849 idle 00:54:10.849 interrupt_mode 00:54:10.849 ioat 00:54:10.849 ioat_perf 00:54:10.849 ioat_verify 00:54:10.849 is_zoned 00:54:10.849 iscsi_tgt 00:54:10.849 iscsi_tgt_bdev_io_wait 00:54:10.849 iscsi_tgt_calsoft 00:54:10.849 iscsi_tgt_digest 00:54:10.849 iscsi_tgt_digests 00:54:10.849 iscsi_tgt_ext4test 00:54:10.849 iscsi_tgt_filesystem 00:54:10.849 iscsi_tgt_filesystem_btrfs 00:54:10.849 iscsi_tgt_filesystem_ext4 00:54:10.849 iscsi_tgt_filesystem_xfs 00:54:10.849 iscsi_tgt_fio 00:54:10.849 iscsi_tgt_fuzz 00:54:10.849 iscsi_tgt_initiator 00:54:10.849 iscsi_tgt_ip_migration 00:54:10.849 iscsi_tgt_iscsi_lvol 00:54:10.849 iscsi_tgt_login_redirection 00:54:10.849 iscsi_tgt_multiconnection 00:54:10.849 iscsi_tgt_qos 00:54:10.849 iscsi_tgt_rbd 00:54:10.849 iscsi_tgt_reset 00:54:10.849 iscsi_tgt_resize 00:54:10.849 iscsi_tgt_rpc_config 00:54:10.849 iscsi_tgt_sock 00:54:10.849 iscsi_tgt_trace_record 00:54:10.849 json_config 00:54:10.849 json_config_extra_key 00:54:10.849 kernel_target_abort 00:54:10.849 keyring_file 00:54:10.849 llvm_fuzz 00:54:10.849 locking_app_on_locked_coremask 00:54:10.849 locking_app_on_unlocked_coremask 00:54:10.849 locking_overlapped_coremask 00:54:10.849 locking_overlapped_coremask_via_rpc 00:54:10.849 lvol 00:54:10.849 lvol_basic 00:54:10.849 lvol_esnap 00:54:10.849 lvol_external_copy 00:54:10.849 lvol_external_snapshot 00:54:10.849 lvol_hotremove 00:54:10.849 lvol_provisioning 00:54:10.849 lvol_rename 00:54:10.849 lvol_resize 00:54:10.849 lvol_snapshot_clone 00:54:10.849 lvol_tasting 00:54:10.849 lvs_grow_clean 00:54:10.849 lvs_grow_dirty 00:54:10.849 make 00:54:10.849 no_shrink_alloc 00:54:10.849 non_locking_app_on_locked_coremask 00:54:10.849 nvme 00:54:10.849 nvme_arbitration 00:54:10.849 nvme_cli_cuse 00:54:10.849 nvme_cli_plugin 00:54:10.849 nvme_cmb 00:54:10.849 nvme_cuse 00:54:10.849 nvme_cuse_app 00:54:10.849 nvme_cuse_rpc 00:54:10.849 nvme_doorbell_aers 00:54:10.849 nvme_e2edp 00:54:10.849 nvme_err_injection 00:54:10.849 nvme_fdp 00:54:10.849 nvme_fio 00:54:10.849 nvme_flexible_data_placement 00:54:10.849 nvme_hello_world 00:54:10.849 nvme_identify 00:54:10.849 nvme_mount 00:54:10.849 nvme_multi_aen 00:54:10.849 nvme_multi_secondary 00:54:10.849 nvme_ns_manage_cuse 00:54:10.849 nvme_overhead 00:54:10.849 nvme_perf 00:54:10.849 nvme_pmr 00:54:10.849 nvme_pmr_persistence 00:54:10.849 nvme_reserve 00:54:10.849 nvme_reset 00:54:10.849 nvme_rpc 00:54:10.849 nvme_rpc_timeouts 00:54:10.849 nvme_scc 00:54:10.849 nvme_sgl 00:54:10.849 nvme_simple_copy 00:54:10.849 nvme_single_aen 00:54:10.849 nvme_smartctl_cuse 00:54:10.849 nvme_startup 00:54:10.849 nvme_zns 00:54:10.849 nvmf_abort 00:54:10.849 nvmf_abort_qd_sizes 00:54:10.849 nvmf_aer 00:54:10.849 nvmf_async_init 00:54:10.849 nvmf_auth_host 00:54:10.849 nvmf_auth_target 00:54:10.849 nvmf_bdev_io_wait 00:54:10.849 nvmf_bdevio 00:54:10.849 nvmf_bdevio_no_huge 00:54:10.849 nvmf_bdevperf 00:54:10.849 nvmf_connect_disconnect 00:54:10.849 nvmf_connect_stress 00:54:10.849 nvmf_delete_subsystem 00:54:10.849 nvmf_device_removal_pci_remove_no_srq 00:54:10.849 nvmf_dif 00:54:10.849 nvmf_digest 00:54:10.849 nvmf_digest_clean 00:54:10.849 nvmf_digest_error 00:54:10.849 nvmf_discovery_remove_ifc 00:54:10.849 nvmf_example 00:54:10.849 nvmf_failover 00:54:10.849 nvmf_filesystem 00:54:10.849 nvmf_filesystem_in_capsule 00:54:10.849 nvmf_filesystem_no_in_capsule 00:54:10.849 nvmf_fio_host 00:54:10.849 nvmf_fio_target 00:54:10.849 nvmf_fips 00:54:10.849 nvmf_fused_ordering 00:54:10.849 nvmf_fuzz 00:54:10.849 nvmf_host_discovery 00:54:10.849 nvmf_host_management 00:54:10.849 nvmf_host_multipath 00:54:10.849 nvmf_host_multipath_status 00:54:10.849 nvmf_identify 00:54:10.849 nvmf_identify_kernel_target 00:54:10.849 nvmf_identify_passthru 00:54:10.849 nvmf_initiator_timeout 00:54:10.849 nvmf_invalid 00:54:10.849 nvmf_lvol 00:54:10.849 nvmf_lvs_grow 00:54:10.850 nvmf_mdns_discovery 00:54:10.850 nvmf_multiconnection 00:54:10.850 nvmf_multicontroller 00:54:10.850 nvmf_multitarget 00:54:10.850 nvmf_nmic 00:54:10.850 nvmf_ns_hotplug_stress 00:54:10.850 nvmf_ns_masking 00:54:10.850 nvmf_nvme_cli 00:54:10.850 nvmf_perf 00:54:10.850 nvmf_perf_adq 00:54:10.850 nvmf_queue_depth 00:54:10.850 nvmf_referrals 00:54:10.850 nvmf_rpc 00:54:10.850 nvmf_shutdown 00:54:10.850 nvmf_shutdown_tc1 00:54:10.850 nvmf_shutdown_tc2 00:54:10.850 nvmf_shutdown_tc3 00:54:10.850 nvmf_target_disconnect 00:54:10.850 nvmf_target_disconnect_tc1 00:54:10.850 nvmf_target_disconnect_tc2 00:54:10.850 nvmf_target_discovery 00:54:10.850 nvmf_target_multipath 00:54:10.850 nvmf_tcp 00:54:10.850 nvmf_timeout 00:54:10.850 nvmf_tls 00:54:10.850 nvmf_vfio_user 00:54:10.850 nvmf_vfio_user_fuzz 00:54:10.850 nvmf_vfio_user_nvme_compliance 00:54:10.850 nvmf_vhost 00:54:10.850 nvmf_zcopy 00:54:10.850 ocf 00:54:10.850 ocf_bdevperf_iotypes 00:54:10.850 ocf_configuration_change 00:54:10.850 ocf_create_destruct 00:54:10.850 ocf_fio_modes 00:54:10.850 ocf_flush 00:54:10.850 ocf_multicore 00:54:10.850 ocf_remove 00:54:10.850 ocf_stats 00:54:10.850 odd_alloc 00:54:10.850 packaging 00:54:10.850 per_node_1G_alloc 00:54:10.850 raid0_resize_test 00:54:10.850 raid5f_rebuild_test 00:54:10.850 raid5f_rebuild_test_sb 00:54:10.850 raid5f_state_function_test 00:54:10.850 raid5f_state_function_test_sb 00:54:10.850 raid5f_superblock_test 00:54:10.850 raid_function_test_concat 00:54:10.850 raid_function_test_raid0 00:54:10.850 raid_rebuild_test 00:54:10.850 raid_rebuild_test_io 00:54:10.850 raid_rebuild_test_sb 00:54:10.850 raid_rebuild_test_sb_4k 00:54:10.850 raid_rebuild_test_sb_io 00:54:10.850 raid_rebuild_test_sb_md_interleaved 00:54:10.850 raid_rebuild_test_sb_md_separate 00:54:10.850 raid_state_function_test 00:54:10.850 raid_state_function_test_sb 00:54:10.850 raid_state_function_test_sb_4k 00:54:10.850 raid_state_function_test_sb_md_interleaved 00:54:10.850 raid_state_function_test_sb_md_separate 00:54:10.850 raid_superblock_test 00:54:10.850 raid_superblock_test_4k 00:54:10.850 raid_superblock_test_md_interleaved 00:54:10.850 raid_superblock_test_md_separate 00:54:10.850 reactor_set_interrupt 00:54:10.850 reap_unregistered_poller 00:54:10.850 rocksdb 00:54:10.850 rocksdb_insert 00:54:10.850 rocksdb_overwrite 00:54:10.850 rocksdb_randread 00:54:10.850 rocksdb_readwrite 00:54:10.850 rocksdb_writesync 00:54:10.850 rpc 00:54:10.850 rpc_client 00:54:10.850 rpc_daemon_integrity 00:54:10.850 rpc_integrity 00:54:10.850 rpc_plugins 00:54:10.850 rpc_trace_cmd_test 00:54:10.850 rpm_packaging 00:54:10.850 scanbuild_make 00:54:10.850 scheduler 00:54:10.850 scheduler_create_thread 00:54:10.850 setup.sh 00:54:10.850 skip_rpc 00:54:10.850 skip_rpc_with_delay 00:54:10.850 skip_rpc_with_json 00:54:10.850 sma 00:54:10.850 sma_crypto 00:54:10.850 sma_discovery 00:54:10.850 sma_nvmf_tcp 00:54:10.850 sma_plugins 00:54:10.850 sma_qos 00:54:10.850 sma_vfiouser_qemu 00:54:10.850 sma_vhost 00:54:10.850 spdk_dd 00:54:10.850 spdk_dd_basic_rw 00:54:10.850 spdk_dd_bdev_to_bdev 00:54:10.850 spdk_dd_malloc 00:54:10.850 spdk_dd_negative 00:54:10.850 spdk_dd_posix 00:54:10.850 spdk_dd_sparse 00:54:10.850 spdk_dd_uring 00:54:10.850 spdk_target_abort 00:54:10.850 spdkcli_iscsi 00:54:10.850 spdkcli_nvmf_tcp 00:54:10.850 spdkcli_raid 00:54:10.850 spdkcli_rbd 00:54:10.850 spdkcli_tcp 00:54:10.850 spdkcli_vhost 00:54:10.850 spdkcli_virtio 00:54:10.850 sw_hotplug 00:54:10.850 test_bdev_lvol_delete_lvstore_with_clones 00:54:10.850 test_bdev_lvol_delete_ordering 00:54:10.850 test_clone_decouple_parent 00:54:10.850 test_clone_inflate 00:54:10.850 test_clone_snapshot_relations 00:54:10.850 test_construct_lvol 00:54:10.850 test_construct_lvol_alias_conflict 00:54:10.850 test_construct_lvol_fio_clear_method_none 00:54:10.850 test_construct_lvol_fio_clear_method_unmap 00:54:10.850 test_construct_lvol_full_lvs 00:54:10.850 test_construct_lvol_inexistent_lvs 00:54:10.850 test_construct_lvols_conflict_alias 00:54:10.850 test_construct_lvs 00:54:10.850 test_construct_lvs_clear_methods 00:54:10.850 test_construct_lvs_conflict_alias 00:54:10.850 test_construct_lvs_different_cluster_size 00:54:10.850 test_construct_lvs_nonexistent_bdev 00:54:10.850 test_construct_multi_lvols 00:54:10.850 test_construct_nested_lvol 00:54:10.850 test_construct_two_lvs_on_the_same_bdev 00:54:10.850 test_create_snapshot_of_snapshot 00:54:10.850 test_create_snapshot_with_io 00:54:10.850 test_delete_lvol_store_persistent_positive 00:54:10.850 test_delete_snapshot_with_clone 00:54:10.850 test_delete_snapshot_with_snapshot 00:54:10.850 test_destroy_after_bdev_lvol_resize_positive 00:54:10.850 test_esnap_clones 00:54:10.850 test_esnap_late_arrival 00:54:10.850 test_esnap_reload 00:54:10.850 test_esnap_remove_degraded 00:54:10.850 test_hotremove_lvol_store 00:54:10.850 test_hotremove_lvol_store_base 00:54:10.850 test_hotremove_lvol_store_base_with_lvols 00:54:10.850 test_hotremove_lvol_store_multiple_lvols 00:54:10.850 test_lvol_bdev_readonly 00:54:10.850 test_lvol_list 00:54:10.850 test_lvol_rename_negative 00:54:10.850 test_rename_lvs_negative 00:54:10.850 test_rename_positive 00:54:10.850 test_resize_lvol 00:54:10.850 test_resize_lvol_negative 00:54:10.850 test_resize_lvol_with_io_traffic 00:54:10.850 test_shallow_copy_compare 00:54:10.850 test_sigterm 00:54:10.850 test_snapshot_compare_with_lvol_bdev 00:54:10.850 test_tasting 00:54:10.850 test_thin_lvol_check_integrity 00:54:10.850 test_thin_lvol_check_space 00:54:10.850 test_thin_lvol_check_zeroes 00:54:10.850 test_thin_lvol_resize 00:54:10.850 test_thin_overprovisioning 00:54:10.850 test_unregister_lvol_bdev 00:54:10.850 thread 00:54:10.850 thread_poller_perf 00:54:10.850 thread_spdk_lock 00:54:10.850 ubsan 00:54:10.850 unittest 00:54:10.850 unittest_accel 00:54:10.850 unittest_bdev 00:54:10.850 unittest_bdev_compress 00:54:10.850 unittest_bdev_crypto 00:54:10.850 unittest_bdev_raid5f 00:54:10.850 unittest_blob_blobfs 00:54:10.851 unittest_build 00:54:10.851 unittest_dma 00:54:10.851 unittest_dpdk_compressdev 00:54:10.851 unittest_event 00:54:10.851 unittest_ftl 00:54:10.851 unittest_idxd_user 00:54:10.851 unittest_include 00:54:10.851 unittest_init 00:54:10.851 unittest_ioat 00:54:10.851 unittest_iobuf 00:54:10.851 unittest_iscsi 00:54:10.851 unittest_json 00:54:10.851 unittest_keyring 00:54:10.851 unittest_lib_reduce 00:54:10.851 unittest_log 00:54:10.851 unittest_lvol 00:54:10.851 unittest_notify 00:54:10.851 unittest_nvme 00:54:10.851 unittest_nvme_cuse 00:54:10.851 unittest_nvme_rdma 00:54:10.851 unittest_nvmf 00:54:10.851 unittest_nvmf_rdma 00:54:10.851 unittest_nvmf_transport 00:54:10.851 unittest_nvmf_vfio_user 00:54:10.851 unittest_pci_event 00:54:10.851 unittest_rdma 00:54:10.851 unittest_rpc 00:54:10.851 unittest_scsi 00:54:10.851 unittest_sock 00:54:10.851 unittest_thread 00:54:10.851 unittest_util 00:54:10.851 unittest_vhost 00:54:10.851 valgrind 00:54:10.851 version 00:54:10.851 vfio_fuzz 00:54:10.851 vfio_user_nvme_fio 00:54:10.851 vfio_user_nvme_restart_vm 00:54:10.851 vfio_user_qemu 00:54:10.851 vfio_user_virtio_bdevperf 00:54:10.851 vfio_user_virtio_blk_restart_vm 00:54:10.851 vfio_user_virtio_scsi_restart_vm 00:54:10.851 vhost 00:54:10.851 vhost_blk_cores_2ctrl 00:54:10.851 vhost_blk_fs_integrity 00:54:10.851 vhost_blk_integrity 00:54:10.851 vhost_blk_lvol_integrity 00:54:10.851 vhost_blk_packed_ring_integrity 00:54:10.851 vhost_blockdev 00:54:10.851 vhost_boot 00:54:10.851 vhost_fuzz 00:54:10.851 vhost_migration 00:54:10.851 vhost_migration_tc1 00:54:10.851 vhost_migration_tc2 00:54:10.851 vhost_negative 00:54:10.851 vhost_readonly 00:54:10.851 vhost_scsi_cores_2ctrl 00:54:10.851 vhost_scsi_fs_integrity 00:54:10.851 vhost_scsi_integrity 00:54:10.851 vhost_scsi_lvol_integrity 00:54:10.851 vhost_shared 00:54:10.851 vmd 00:54:10.851 vmd_bdev_svc 00:54:10.851 vmd_fio 00:54:10.851 vmd_hello_world 00:54:10.851 vmd_identify 00:54:10.851 vmd_perf 00:54:10.851 zoned_fio 00:54:10.851 00:54:10.851 00:54:10.851 -----Tests Missing From Build------ 00:54:10.851 balanced 00:54:10.851 blockdev_xnvme 00:54:10.851 busy 00:54:10.851 core_load 00:54:10.851 ftl 00:54:10.851 ftl_bdevperf 00:54:10.851 ftl_dirty_shutdown 00:54:10.851 ftl_fio_basic 00:54:10.851 ftl_restore 00:54:10.851 ftl_restore_fast 00:54:10.851 ftl_trim 00:54:10.851 ftl_upgrade_shutdown 00:54:10.851 iscsi_tgt_data_digest 00:54:10.851 keyring_linux 00:54:10.851 nvme_boot_partition 00:54:10.851 nvme_bp 00:54:10.851 nvme_opal 00:54:10.851 nvme_opal_bdevio 00:54:10.851 nvme_opal_bdevperf 00:54:10.851 nvme_opal_spdk_tgt 00:54:10.851 nvme_xnvme 00:54:10.851 nvmf_device_removal 00:54:10.851 nvmf_device_removal_pci_remove 00:54:10.851 nvmf_digest_dsa_initiator 00:54:10.851 nvmf_digest_dsa_target 00:54:10.851 nvmf_fc 00:54:10.851 nvmf_rdma 00:54:10.851 nvmf_srq_overwhelm 00:54:10.851 nvmf_target_disconnect_tc3 00:54:10.851 spdkcli_nvmf_fc 00:54:10.851 spdkcli_nvmf_rdma 00:54:10.851 test_create_multi_ublk 00:54:10.851 test_create_ublk 00:54:10.851 test_save_ublk_config 00:54:10.851 ublk 00:54:10.851 ublk_recovery 00:54:10.851 unittest_nvmf_fc 00:54:10.851 unittest_nvmf_fc_ls 00:54:10.851 vhost_blk_hot_remove 00:54:10.851 vhost_hotplug 00:54:10.851 vhost_scsi_hot_remove 00:54:10.851 xnvme_bdevperf 00:54:10.851 xnvme_to_malloc_dd_copy 00:54:10.851 00:54:10.851 00:54:10.851 -----Tests Missing ASAN------ 00:54:10.851 accel_cdev_comp 00:54:10.851 accel_cdev_decmop_full 00:54:10.851 accel_cdev_decomp 00:54:10.851 accel_cdev_decomp_full_mcore 00:54:10.851 accel_cdev_decomp_full_mthread 00:54:10.851 accel_cdev_decomp_mcore 00:54:10.851 accel_cdev_decomp_mthread 00:54:10.851 autobuild_llvm_precompile 00:54:10.851 balanced 00:54:10.851 bdev_crypto_enomem 00:54:10.851 blockdev_crypto_aesni 00:54:10.851 blockdev_crypto_qat 00:54:10.851 blockdev_crypto_sw 00:54:10.851 blockdev_rbd 00:54:10.851 blockdev_xnvme 00:54:10.851 busy 00:54:10.851 chaining 00:54:10.851 cmb_copy 00:54:10.851 compress_compdev 00:54:10.851 compress_isal 00:54:10.851 core_load 00:54:10.851 dd_uring_copy 00:54:10.851 dma 00:54:10.851 dpdk_governor 00:54:10.851 filesystem_btrfs 00:54:10.851 filesystem_ext4 00:54:10.851 filesystem_in_capsule_btrfs 00:54:10.851 filesystem_in_capsule_ext4 00:54:10.851 filesystem_in_capsule_xfs 00:54:10.851 filesystem_xfs 00:54:10.851 fio_dif_1_default 00:54:10.851 fio_dif_1_multi_subsystems 00:54:10.851 fio_dif_digest 00:54:10.851 fio_dif_rand_params 00:54:10.851 ftl 00:54:10.851 ftl_bdevperf 00:54:10.851 ftl_dirty_shutdown 00:54:10.851 ftl_fio_basic 00:54:10.851 ftl_restore 00:54:10.851 ftl_restore_fast 00:54:10.851 ftl_trim 00:54:10.851 ftl_upgrade_shutdown 00:54:10.851 go_rpc 00:54:10.851 idle 00:54:10.851 interrupt_mode 00:54:10.851 ioat 00:54:10.851 ioat_perf 00:54:10.851 ioat_verify 00:54:10.851 is_zoned 00:54:10.851 iscsi_tgt 00:54:10.851 iscsi_tgt_bdev_io_wait 00:54:10.851 iscsi_tgt_calsoft 00:54:10.851 iscsi_tgt_data_digest 00:54:10.851 iscsi_tgt_digest 00:54:10.851 iscsi_tgt_digests 00:54:10.851 iscsi_tgt_ext4test 00:54:10.851 iscsi_tgt_filesystem 00:54:10.851 iscsi_tgt_filesystem_btrfs 00:54:10.851 iscsi_tgt_filesystem_ext4 00:54:10.851 iscsi_tgt_filesystem_xfs 00:54:10.851 iscsi_tgt_fio 00:54:10.851 iscsi_tgt_fuzz 00:54:10.851 iscsi_tgt_initiator 00:54:10.851 iscsi_tgt_ip_migration 00:54:10.851 iscsi_tgt_iscsi_lvol 00:54:10.851 iscsi_tgt_login_redirection 00:54:10.851 iscsi_tgt_multiconnection 00:54:10.851 iscsi_tgt_qos 00:54:10.851 iscsi_tgt_rbd 00:54:10.851 iscsi_tgt_reset 00:54:10.851 iscsi_tgt_resize 00:54:10.851 iscsi_tgt_rpc_config 00:54:10.851 iscsi_tgt_sock 00:54:10.851 iscsi_tgt_trace_record 00:54:10.851 kernel_target_abort 00:54:10.851 keyring_file 00:54:10.851 keyring_linux 00:54:10.851 llvm_fuzz 00:54:10.851 lvs_grow_clean 00:54:10.851 lvs_grow_dirty 00:54:10.851 nvme_boot_partition 00:54:10.851 nvme_bp 00:54:10.851 nvme_cli_cuse 00:54:10.851 nvme_cli_plugin 00:54:10.851 nvme_cmb 00:54:10.851 nvme_cuse 00:54:10.851 nvme_cuse_app 00:54:10.851 nvme_cuse_rpc 00:54:10.851 nvme_ns_manage_cuse 00:54:10.851 nvme_opal 00:54:10.851 nvme_opal_bdevio 00:54:10.851 nvme_opal_bdevperf 00:54:10.851 nvme_opal_spdk_tgt 00:54:10.851 nvme_pmr 00:54:10.851 nvme_pmr_persistence 00:54:10.851 nvme_smartctl_cuse 00:54:10.851 nvme_xnvme 00:54:10.851 nvme_zns 00:54:10.851 nvmf_abort 00:54:10.851 nvmf_abort_qd_sizes 00:54:10.851 nvmf_aer 00:54:10.851 nvmf_async_init 00:54:10.851 nvmf_auth_host 00:54:10.851 nvmf_auth_target 00:54:10.851 nvmf_bdev_io_wait 00:54:10.851 nvmf_bdevio 00:54:10.851 nvmf_bdevio_no_huge 00:54:10.851 nvmf_bdevperf 00:54:10.851 nvmf_connect_disconnect 00:54:10.851 nvmf_connect_stress 00:54:10.851 nvmf_delete_subsystem 00:54:10.851 nvmf_device_removal 00:54:10.851 nvmf_device_removal_pci_remove 00:54:10.851 nvmf_device_removal_pci_remove_no_srq 00:54:10.851 nvmf_dif 00:54:10.851 nvmf_digest 00:54:10.851 nvmf_digest_clean 00:54:10.851 nvmf_digest_dsa_initiator 00:54:10.851 nvmf_digest_dsa_target 00:54:10.851 nvmf_digest_error 00:54:10.851 nvmf_discovery_remove_ifc 00:54:10.851 nvmf_example 00:54:10.851 nvmf_failover 00:54:10.851 nvmf_fc 00:54:10.851 nvmf_filesystem 00:54:10.851 nvmf_filesystem_in_capsule 00:54:10.851 nvmf_filesystem_no_in_capsule 00:54:10.851 nvmf_fio_host 00:54:10.851 nvmf_fio_target 00:54:10.851 nvmf_fips 00:54:10.851 nvmf_fused_ordering 00:54:10.851 nvmf_fuzz 00:54:10.851 nvmf_host_discovery 00:54:10.851 nvmf_host_management 00:54:10.851 nvmf_host_multipath 00:54:10.851 nvmf_host_multipath_status 00:54:10.851 nvmf_identify 00:54:10.851 nvmf_identify_kernel_target 00:54:10.851 nvmf_identify_passthru 00:54:10.851 nvmf_initiator_timeout 00:54:10.851 nvmf_invalid 00:54:10.851 nvmf_lvol 00:54:10.851 nvmf_lvs_grow 00:54:10.851 nvmf_mdns_discovery 00:54:10.851 nvmf_multiconnection 00:54:10.851 nvmf_multicontroller 00:54:10.851 nvmf_multitarget 00:54:10.851 nvmf_nmic 00:54:10.851 nvmf_ns_hotplug_stress 00:54:10.851 nvmf_ns_masking 00:54:10.851 nvmf_nvme_cli 00:54:10.851 nvmf_perf 00:54:10.851 nvmf_perf_adq 00:54:10.851 nvmf_queue_depth 00:54:10.851 nvmf_rdma 00:54:10.851 nvmf_referrals 00:54:10.851 nvmf_rpc 00:54:10.851 nvmf_shutdown 00:54:10.851 nvmf_shutdown_tc1 00:54:10.851 nvmf_shutdown_tc2 00:54:10.851 nvmf_shutdown_tc3 00:54:10.851 nvmf_srq_overwhelm 00:54:10.851 nvmf_target_disconnect 00:54:10.851 nvmf_target_disconnect_tc1 00:54:10.851 nvmf_target_disconnect_tc2 00:54:10.851 nvmf_target_disconnect_tc3 00:54:10.851 nvmf_target_discovery 00:54:10.851 nvmf_target_multipath 00:54:10.851 nvmf_tcp 00:54:10.851 nvmf_timeout 00:54:10.851 nvmf_tls 00:54:10.851 nvmf_vfio_user 00:54:10.851 nvmf_vfio_user_fuzz 00:54:10.851 nvmf_vfio_user_nvme_compliance 00:54:10.851 nvmf_zcopy 00:54:10.852 ocf 00:54:10.852 ocf_bdevperf_iotypes 00:54:10.852 ocf_configuration_change 00:54:10.852 ocf_create_destruct 00:54:10.852 ocf_fio_modes 00:54:10.852 ocf_flush 00:54:10.852 ocf_multicore 00:54:10.852 ocf_remove 00:54:10.852 ocf_stats 00:54:10.852 scheduler 00:54:10.852 spdk_dd_uring 00:54:10.852 spdk_target_abort 00:54:10.852 spdkcli_iscsi 00:54:10.852 spdkcli_nvmf_fc 00:54:10.852 spdkcli_nvmf_rdma 00:54:10.852 spdkcli_nvmf_tcp 00:54:10.852 spdkcli_raid 00:54:10.852 spdkcli_rbd 00:54:10.852 spdkcli_virtio 00:54:10.852 test_create_multi_ublk 00:54:10.852 test_create_ublk 00:54:10.852 test_save_ublk_config 00:54:10.852 ublk 00:54:10.852 ublk_recovery 00:54:10.852 unittest_nvmf_fc 00:54:10.852 unittest_nvmf_fc_ls 00:54:10.852 valgrind 00:54:10.852 vfio_fuzz 00:54:10.852 vhost_blk_hot_remove 00:54:10.852 vhost_blockdev 00:54:10.852 vhost_fuzz 00:54:10.852 vhost_hotplug 00:54:10.852 vhost_scsi_hot_remove 00:54:10.852 vhost_shared 00:54:10.852 xnvme_bdevperf 00:54:10.852 xnvme_to_malloc_dd_copy 00:54:10.852 zoned_fio 00:54:10.852 00:54:10.852 00:54:10.852 -----Tests Missing UBSAN------ 00:54:10.852 balanced 00:54:10.852 blockdev_xnvme 00:54:10.852 busy 00:54:10.852 cmb_copy 00:54:10.852 core_load 00:54:10.852 ftl 00:54:10.852 ftl_bdevperf 00:54:10.852 ftl_dirty_shutdown 00:54:10.852 ftl_fio_basic 00:54:10.852 ftl_restore 00:54:10.852 ftl_restore_fast 00:54:10.852 ftl_trim 00:54:10.852 ftl_upgrade_shutdown 00:54:10.852 is_zoned 00:54:10.852 iscsi_tgt_data_digest 00:54:10.852 keyring_linux 00:54:10.852 nvme_boot_partition 00:54:10.852 nvme_bp 00:54:10.852 nvme_cmb 00:54:10.852 nvme_opal 00:54:10.852 nvme_opal_bdevio 00:54:10.852 nvme_opal_bdevperf 00:54:10.852 nvme_opal_spdk_tgt 00:54:10.852 nvme_pmr 00:54:10.852 nvme_pmr_persistence 00:54:10.852 nvme_xnvme 00:54:10.852 nvme_zns 00:54:10.852 nvmf_device_removal 00:54:10.852 nvmf_device_removal_pci_remove 00:54:10.852 nvmf_digest_dsa_initiator 00:54:10.852 nvmf_digest_dsa_target 00:54:10.852 nvmf_fc 00:54:10.852 nvmf_rdma 00:54:10.852 nvmf_srq_overwhelm 00:54:10.852 nvmf_target_disconnect_tc3 00:54:10.852 spdkcli_nvmf_fc 00:54:10.852 spdkcli_nvmf_rdma 00:54:10.852 test_create_multi_ublk 00:54:10.852 test_create_ublk 00:54:10.852 test_save_ublk_config 00:54:10.852 ublk 00:54:10.852 ublk_recovery 00:54:10.852 unittest_nvmf_fc 00:54:10.852 unittest_nvmf_fc_ls 00:54:10.852 vhost_blk_hot_remove 00:54:10.852 vhost_hotplug 00:54:10.852 vhost_scsi_hot_remove 00:54:10.852 xnvme_bdevperf 00:54:10.852 xnvme_to_malloc_dd_copy 00:54:10.852 zoned_fio 00:54:10.852 Not all tests were run. Failing the build. 00:54:10.852 ['blockdev_xnvme', 'ftl', 'ftl_bdevperf', 'ftl_dirty_shutdown', 'ftl_fio_basic', 'ftl_restore', 'ftl_trim', 'ftl_upgrade_shutdown', 'nvme_xnvme', 'nvmf_device_removal', 'nvmf_device_removal_pci_remove', 'nvmf_rdma', 'nvmf_srq_overwhelm', 'nvmf_target_disconnect_tc3', 'spdkcli_nvmf_rdma', 'test_create_multi_ublk', 'test_create_ublk', 'test_save_ublk_config', 'ublk', 'ublk_recovery', 'xnvme_bdevperf', 'xnvme_to_malloc_dd_copy'] 00:54:10.852 + compress_coverage_and_docs 00:54:10.852 + echo 'Start compress coverage and docs' 00:54:10.852 Start compress coverage and docs 00:54:10.852 + tar -C coverage -czf coverage_autotest-spdk-master-vs-dpdk-main_3476.tar.gz ./ --remove-files 00:54:10.852 + tar -C ut_coverage -czf ut_coverage_autotest-spdk-master-vs-dpdk-main_3476.tar.gz ./ --remove-files 00:54:11.782 + tar -C doc -czf doc_autotest-spdk-master-vs-dpdk-main_3476.tar.gz ./ --remove-files 00:54:11.782 + [[ -d llvm ]] 00:54:11.782 + true 00:54:11.782 + echo 'End compress coverage and docs' 00:54:11.782 End compress coverage and docs 00:54:11.782 + remove_partial_coverage_files 00:54:11.782 + find . -name 'cov_*.info' -delete 00:54:12.040 + echo 'End stage post_gen_coverage.sh' 00:54:12.040 End stage post_gen_coverage.sh 00:54:12.056 [Pipeline] } 00:54:12.060 ERROR: script returned exit code 1 00:54:12.092 [Pipeline] // catchError 00:54:12.102 [Pipeline] catchError 00:54:12.103 [Pipeline] { 00:54:12.126 [Pipeline] dir 00:54:12.126 Running in /var/jenkins/workspace/autotest-spdk-master-vs-dpdk-main/post_process 00:54:12.128 [Pipeline] {