[Thu Dec 5 12:27:27 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh reset (start) [Thu Dec 5 12:27:31 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh reset (done) [Thu Dec 5 12:27:34 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh status (start) [Thu Dec 5 12:27:37 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh status (done) [Thu Dec 5 12:27:37 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh (start) [Thu Dec 5 12:27:39 2024] ioatdma 0000:00:04.7: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:00:04.6: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:00:04.5: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:00:04.4: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:00:04.3: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:00:04.2: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:00:04.1: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:00:04.0: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:80:04.7: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:80:04.6: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:80:04.5: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:80:04.4: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:80:04.3: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:80:04.2: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:80:04.1: Removing dma and dca services [Thu Dec 5 12:27:39 2024] ioatdma 0000:80:04.0: Removing dma and dca services [Thu Dec 5 12:27:42 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh (done) [Thu Dec 5 12:27:43 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh reset (start) [Thu Dec 5 12:27:46 2024] nvme nvme0: pci function 0000:5e:00.0 [Thu Dec 5 12:27:46 2024] nvme nvme0: 72/0/0 default/read/poll queues [Thu Dec 5 12:27:48 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh reset (done) [Thu Dec 5 12:27:48 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh (start) [Thu Dec 5 12:27:51 2024] ioatdma 0000:00:04.7: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:00:04.6: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:00:04.5: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:00:04.4: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:00:04.3: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:00:04.2: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:00:04.1: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:00:04.0: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:80:04.7: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:80:04.6: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:80:04.5: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:80:04.4: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:80:04.3: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:80:04.2: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:80:04.1: Removing dma and dca services [Thu Dec 5 12:27:51 2024] ioatdma 0000:80:04.0: Removing dma and dca services [Thu Dec 5 12:27:54 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh (done) [Thu Dec 5 12:28:09 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh reset (start) [Thu Dec 5 12:28:12 2024] nvme nvme0: pci function 0000:5e:00.0 [Thu Dec 5 12:28:12 2024] nvme nvme0: 72/0/0 default/read/poll queues [Thu Dec 5 12:28:14 2024] sysrq: HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(v) show-blocked-tasks(w) dump-ftrace-buffer(z) [Thu Dec 5 12:28:14 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh reset (done) [Thu Dec 5 12:28:14 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh cleanup (start) [Thu Dec 5 12:28:15 2024] sysrq: HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(v) show-blocked-tasks(w) dump-ftrace-buffer(z) [Thu Dec 5 12:28:16 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh cleanup (done) [Thu Dec 5 12:28:17 2024] ######## Autotest Cleanup Dump ######## [Thu Dec 5 12:28:17 2024] sysrq: Show backtrace of all active CPUs [Thu Dec 5 12:28:17 2024] NMI backtrace for cpu 20 [Thu Dec 5 12:28:17 2024] CPU: 20 PID: 1466021 Comm: bash Tainted: G OE 6.8.9-200.fc39.x86_64 #1 [Thu Dec 5 12:28:17 2024] Hardware name: Intel Corporation S2600WFQ/S2600WFQ, BIOS SE5C620.86B.02.01.0008.031920191559 03/19/2019 [Thu Dec 5 12:28:17 2024] Call Trace: [Thu Dec 5 12:28:17 2024] [Thu Dec 5 12:28:17 2024] dump_stack_lvl+0x64/0x80 [Thu Dec 5 12:28:17 2024] nmi_cpu_backtrace+0xd6/0x110 [Thu Dec 5 12:28:17 2024] ? __pfx_nmi_raise_cpu_backtrace+0x10/0x10 [Thu Dec 5 12:28:17 2024] nmi_trigger_cpumask_backtrace+0xf3/0x140 [Thu Dec 5 12:28:17 2024] __handle_sysrq+0xe1/0x260 [Thu Dec 5 12:28:17 2024] write_sysrq_trigger+0x59/0x80 [Thu Dec 5 12:28:17 2024] proc_reg_write+0x5a/0xa0 [Thu Dec 5 12:28:17 2024] vfs_write+0xed/0x470 [Thu Dec 5 12:28:17 2024] ? do_syscall_64+0x90/0x170 [Thu Dec 5 12:28:18 2024] ksys_write+0x6f/0xf0 [Thu Dec 5 12:28:18 2024] do_syscall_64+0x83/0x170 [Thu Dec 5 12:28:18 2024] ? syscall_exit_to_user_mode+0x83/0x230 [Thu Dec 5 12:28:18 2024] ? do_syscall_64+0x90/0x170 [Thu Dec 5 12:28:18 2024] ? exc_page_fault+0x7f/0x180 [Thu Dec 5 12:28:18 2024] entry_SYSCALL_64_after_hwframe+0x78/0x80 [Thu Dec 5 12:28:18 2024] RIP: 0033:0x7fa10ce20ee4 [Thu Dec 5 12:28:18 2024] Code: c7 00 16 00 00 00 b8 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 80 3d 85 74 0d 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 55 48 89 e5 48 83 ec 20 48 89 [Thu Dec 5 12:28:18 2024] RSP: 002b:00007ffc43c3ba08 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [Thu Dec 5 12:28:18 2024] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007fa10ce20ee4 [Thu Dec 5 12:28:18 2024] RDX: 0000000000000002 RSI: 00005608909a3990 RDI: 0000000000000001 [Thu Dec 5 12:28:18 2024] RBP: 00007ffc43c3ba30 R08: 0000000000000073 R09: 0000000000000001 [Thu Dec 5 12:28:18 2024] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000002 [Thu Dec 5 12:28:19 2024] R13: 00005608909a3990 R14: 00007fa10cef15c0 R15: 00007fa10ceeef20 [Thu Dec 5 12:28:19 2024] [Thu Dec 5 12:28:19 2024] Sending NMI from CPU 20 to CPUs 0-19,21-71: [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 2 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 46 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 67 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 39 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 56 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 15 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 51 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 50 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 5 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 19 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 7 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 59 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 3 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 18 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 17 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 41 [Thu Dec 5 12:28:19 2024] CPU: 41 PID: 1484428 Comm: bash Tainted: G OE 6.8.9-200.fc39.x86_64 #1 [Thu Dec 5 12:28:19 2024] Hardware name: Intel Corporation S2600WFQ/S2600WFQ, BIOS SE5C620.86B.02.01.0008.031920191559 03/19/2019 [Thu Dec 5 12:28:19 2024] RIP: 0010:kernfs_seq_next+0x0/0x90 [Thu Dec 5 12:28:19 2024] Code: 4f 01 00 00 5b 89 e8 5d 41 5c 41 5d c3 cc cc cc cc 66 0f 1f 84 00 00 00 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0f 1e fa 0f 1f 44 00 00 48 8b 47 70 48 8b 00 48 8b 40 40 48 8b [Thu Dec 5 12:28:19 2024] RSP: 0018:ffffa55d23e67df0 EFLAGS: 00000283 [Thu Dec 5 12:28:19 2024] RAX: ffffffffb7599a10 RBX: ffff8c5b070df438 RCX: 0000000000000001 [Thu Dec 5 12:28:19 2024] RDX: ffff8c5b070df460 RSI: 0000000000000001 RDI: ffff8c5b070df438 [Thu Dec 5 12:28:19 2024] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000007 [Thu Dec 5 12:28:19 2024] R10: 0000000000000000 R11: ffff8c6610ba1780 R12: ffffa55d23e67e78 [Thu Dec 5 12:28:19 2024] R13: ffffa55d23e67e50 R14: 0000000000000001 R15: 0000000000000000 [Thu Dec 5 12:28:19 2024] FS: 00007f21ec4f5740(0000) GS:ffff8c6610b80000(0000) knlGS:0000000000000000 [Thu Dec 5 12:28:19 2024] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [Thu Dec 5 12:28:19 2024] CR2: 00007f5b5320b3e8 CR3: 0000000e1c49c004 CR4: 00000000007706f0 [Thu Dec 5 12:28:19 2024] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [Thu Dec 5 12:28:19 2024] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [Thu Dec 5 12:28:19 2024] PKRU: 55555554 [Thu Dec 5 12:28:19 2024] Call Trace: [Thu Dec 5 12:28:19 2024] [Thu Dec 5 12:28:19 2024] ? nmi_cpu_backtrace+0x99/0x110 [Thu Dec 5 12:28:19 2024] ? nmi_cpu_backtrace_handler+0x11/0x20 [Thu Dec 5 12:28:19 2024] ? nmi_handle+0x65/0x170 [Thu Dec 5 12:28:19 2024] ? default_do_nmi+0x40/0x100 [Thu Dec 5 12:28:19 2024] ? exc_nmi+0x125/0x1a0 [Thu Dec 5 12:28:19 2024] ? end_repeat_nmi+0xf/0x60 [Thu Dec 5 12:28:19 2024] ? __pfx_kernfs_seq_next+0x10/0x10 [Thu Dec 5 12:28:19 2024] ? __pfx_kernfs_seq_next+0x10/0x10 [Thu Dec 5 12:28:19 2024] ? __pfx_kernfs_seq_next+0x10/0x10 [Thu Dec 5 12:28:19 2024] ? __pfx_kernfs_seq_next+0x10/0x10 [Thu Dec 5 12:28:19 2024] [Thu Dec 5 12:28:19 2024] [Thu Dec 5 12:28:19 2024] seq_read_iter+0x2e1/0x480 [Thu Dec 5 12:28:19 2024] vfs_read+0x24c/0x380 [Thu Dec 5 12:28:19 2024] ksys_read+0x6f/0xf0 [Thu Dec 5 12:28:19 2024] do_syscall_64+0x83/0x170 [Thu Dec 5 12:28:19 2024] ? __irq_exit_rcu+0x4b/0xc0 [Thu Dec 5 12:28:19 2024] entry_SYSCALL_64_after_hwframe+0x78/0x80 [Thu Dec 5 12:28:19 2024] RIP: 0033:0x7f21ec6003d1 [Thu Dec 5 12:28:19 2024] Code: 00 48 8b 15 61 fa 0c 00 f7 d8 64 89 02 b8 ff ff ff ff eb bd e8 30 cf 01 00 f3 0f 1e fa 80 3d 95 7f 0d 00 00 74 13 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 4f c3 66 0f 1f 44 00 00 55 48 89 e5 48 83 ec [Thu Dec 5 12:28:19 2024] RSP: 002b:00007fff215c76f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 [Thu Dec 5 12:28:19 2024] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f21ec6003d1 [Thu Dec 5 12:28:19 2024] RDX: 0000000000001000 RSI: 00007fff215c77a0 RDI: 0000000000000003 [Thu Dec 5 12:28:19 2024] RBP: 00007fff215c7730 R08: 0000000000000000 R09: 0000000000000007 [Thu Dec 5 12:28:19 2024] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000003 [Thu Dec 5 12:28:19 2024] R13: 00007fff215c77a0 R14: 0000000000001000 R15: 00007fff215c77a0 [Thu Dec 5 12:28:19 2024] [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 10 [Thu Dec 5 12:28:19 2024] CPU: 10 PID: 1466193 Comm: bash Tainted: G OE 6.8.9-200.fc39.x86_64 #1 [Thu Dec 5 12:28:19 2024] Hardware name: Intel Corporation S2600WFQ/S2600WFQ, BIOS SE5C620.86B.02.01.0008.031920191559 03/19/2019 [Thu Dec 5 12:28:19 2024] RIP: 0033:0x7fefa34d25e8 [Thu Dec 5 12:28:19 2024] Code: 83 bd 04 00 00 0f b7 04 58 48 3b 05 2a ac 13 00 4c 8d 5b 10 72 58 e9 a7 04 00 00 0f 1f 80 00 00 00 00 4c 8d 52 10 48 8b 7a 10 <4c> 89 d0 48 c1 e8 0c 48 31 c7 49 89 78 10 48 8b 3d 7b 1b 14 00 48 [Thu Dec 5 12:28:19 2024] RSP: 002b:00007fff7440f0c0 EFLAGS: 00000202 [Thu Dec 5 12:28:19 2024] RAX: 0000000000000006 RBX: 0000000000000001 RCX: 000055e27b9d1760 [Thu Dec 5 12:28:19 2024] RDX: 000055e27ba9cb80 RSI: 00007fefa360dad8 RDI: 000055e7258e04dc [Thu Dec 5 12:28:19 2024] RBP: 00007fff7440f110 R08: 00007fefa360dac8 R09: 0000000000000000 [Thu Dec 5 12:28:19 2024] R10: 000055e27ba9cb90 R11: 0000000000000011 R12: 00007fefa360dac0 [Thu Dec 5 12:28:19 2024] R13: 0000000000000020 R14: 0000000000000003 R15: ffffffffffffffc0 [Thu Dec 5 12:28:19 2024] FS: 00007fefa3432740 GS: 0000000000000000 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 31 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 29 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 8 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 71 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 28 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 37 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 12 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 30 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 40 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 68 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 45 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 43 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 65 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 14 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 55 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 4 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 63 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 38 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 35 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 6 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 42 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 21 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 54 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 0 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 24 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 53 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 69 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 1 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 22 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 66 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 44 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 58 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 47 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 60 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 9 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 11 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 33 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 13 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 70 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 26 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 48 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 32 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 49 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 23 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.009 msecs [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 16 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 52 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.027 msecs [Thu Dec 5 12:28:19 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.041 msecs [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 34 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 36 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.064 msecs [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 62 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.073 msecs [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 64 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.084 msecs [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 61 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.102 msecs [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 25 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.115 msecs [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 57 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.128 msecs [Thu Dec 5 12:28:19 2024] NMI backtrace for cpu 27 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:19 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.137 msecs [Thu Dec 5 12:28:19 2024] sysrq: Show Memory [Thu Dec 5 12:28:30 2024] Mem-Info: [Thu Dec 5 12:28:30 2024] active_anon:1944741 inactive_anon:0 isolated_anon:0 active_file:98377 inactive_file:922898 isolated_file:0 unevictable:768 dirty:0 writeback:0 slab_reclaimable:51535 slab_unreclaimable:115651 mapped:38655 shmem:1828147 pagetables:2628 sec_pagetables:0 bounce:0 kernel_misc_reclaimable:0 free:19739019 free_pcp:50026 free_cma:0 [Thu Dec 5 12:28:31 2024] Node 0 active_anon:2763252kB inactive_anon:0kB active_file:179660kB inactive_file:109636kB unevictable:3072kB isolated(anon):0kB isolated(file):0kB mapped:60404kB dirty:0kB writeback:0kB shmem:2478580kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:9240kB pagetables:4316kB sec_pagetables:0kB all_unreclaimable? no [Thu Dec 5 12:28:31 2024] Node 1 active_anon:5012120kB inactive_anon:0kB active_file:213848kB inactive_file:3581956kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:94220kB dirty:0kB writeback:0kB shmem:4834020kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:6824kB pagetables:2724kB sec_pagetables:0kB all_unreclaimable? no [Thu Dec 5 12:28:32 2024] Node 0 DMA free:11264kB boost:0kB min:12kB low:24kB high:36kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15980kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [Thu Dec 5 12:28:32 2024] lowmem_reserve[]: 0 1663 46971 46971 46971 [Thu Dec 5 12:28:32 2024] Node 0 DMA32 free:1699236kB boost:0kB min:1664kB low:3364kB high:5064kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:1768868kB managed:1703332kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [Thu Dec 5 12:28:32 2024] lowmem_reserve[]: 0 0 45307 45307 45307 [Thu Dec 5 12:28:32 2024] Node 0 Normal free:42541152kB boost:0kB min:45340kB low:91732kB high:138124kB reserved_highatomic:0KB active_anon:2765284kB inactive_anon:0kB active_file:179660kB inactive_file:109636kB unevictable:3072kB writepending:0kB present:47173632kB managed:46395220kB mlocked:0kB bounce:0kB free_pcp:106940kB local_pcp:200kB free_cma:0kB [Thu Dec 5 12:28:33 2024] lowmem_reserve[]: 0 0 0 0 0 [Thu Dec 5 12:28:33 2024] Node 1 Normal free:34701544kB boost:0kB min:43088kB low:87176kB high:131264kB reserved_highatomic:0KB active_anon:5012000kB inactive_anon:0kB active_file:213848kB inactive_file:3581956kB unevictable:0kB writepending:0kB present:45088768kB managed:44171508kB mlocked:0kB bounce:0kB free_pcp:89808kB local_pcp:588kB free_cma:0kB [Thu Dec 5 12:28:33 2024] lowmem_reserve[]: 0 0 0 0 0 [Thu Dec 5 12:28:33 2024] Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 2*4096kB (M) = 11264kB [Thu Dec 5 12:28:33 2024] Node 0 DMA32: 9*4kB (UM) 12*8kB (UM) 12*16kB (UM) 9*32kB (UM) 13*64kB (UM) 10*128kB (UM) 9*256kB (UM) 7*512kB (UM) 7*1024kB (UM) 12*2048kB (M) 405*4096kB (M) = 1699236kB [Thu Dec 5 12:28:33 2024] Node 0 Normal: 50702*4kB (UME) 43408*8kB (UM) 24948*16kB (UME) 9666*32kB (M) 2922*64kB (UME) 2226*128kB (UME) 2083*256kB (UME) 752*512kB (UME) 172*1024kB (UME) 39*2048kB (UM) 9678*4096kB (M) = 42545848kB [Thu Dec 5 12:28:34 2024] Node 1 Normal: 43889*4kB (UME) 36928*8kB (UME) 25368*16kB (UME) 13788*32kB (UME) 6973*64kB (UME) 3125*128kB (UME) 883*256kB (UME) 339*512kB (UME) 94*1024kB (UME) 24*2048kB (UME) 7811*4096kB (ME) = 34703236kB [Thu Dec 5 12:28:34 2024] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB [Thu Dec 5 12:28:34 2024] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [Thu Dec 5 12:28:34 2024] Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB [Thu Dec 5 12:28:34 2024] Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [Thu Dec 5 12:28:34 2024] 2849434 total pagecache pages [Thu Dec 5 12:28:34 2024] 0 pages in swap cache [Thu Dec 5 12:28:34 2024] Free swap = 8388604kB [Thu Dec 5 12:28:34 2024] Total swap = 8388604kB [Thu Dec 5 12:28:35 2024] 23511812 pages RAM [Thu Dec 5 12:28:35 2024] 0 pages HighMem/MovableOnly [Thu Dec 5 12:28:35 2024] 440457 pages reserved [Thu Dec 5 12:28:35 2024] 0 pages cma reserved [Thu Dec 5 12:28:35 2024] 0 pages hwpoisoned [Thu Dec 5 12:28:35 2024] sysrq: Show Blocked State [Thu Dec 5 12:28:35 2024] task:ipmitool state:D stack:0 pid:1484427 tgid:1484427 ppid:1466239 flags:0x00004002 [Thu Dec 5 12:28:35 2024] Call Trace: [Thu Dec 5 12:28:35 2024] [Thu Dec 5 12:28:35 2024] __schedule+0x413/0x1720 [Thu Dec 5 12:28:35 2024] ? __schedule+0x41b/0x1720 [Thu Dec 5 12:28:35 2024] schedule+0x32/0xd0 [Thu Dec 5 12:28:35 2024] schedule_timeout+0x151/0x160 [Thu Dec 5 12:28:35 2024] wait_for_completion+0x8a/0x160 [Thu Dec 5 12:28:35 2024] __wait_rcu_gp+0x15b/0x170 [Thu Dec 5 12:28:35 2024] synchronize_rcu+0x133/0x150 [Thu Dec 5 12:28:35 2024] ? __pfx_call_rcu_hurry+0x10/0x10 [Thu Dec 5 12:28:35 2024] ? __pfx_wakeme_after_rcu+0x10/0x10 [Thu Dec 5 12:28:35 2024] _ipmi_destroy_user+0x2aa/0x320 [ipmi_msghandler] [Thu Dec 5 12:28:35 2024] ipmi_destroy_user+0x12/0x60 [ipmi_msghandler] [Thu Dec 5 12:28:35 2024] ipmi_release+0x1f/0x60 [ipmi_devintf] [Thu Dec 5 12:28:36 2024] __fput+0x97/0x2c0 [Thu Dec 5 12:28:36 2024] __x64_sys_close+0x3d/0x80 [Thu Dec 5 12:28:36 2024] do_syscall_64+0x83/0x170 [Thu Dec 5 12:28:36 2024] ? __count_memcg_events+0x69/0x100 [Thu Dec 5 12:28:36 2024] ? count_memcg_events.constprop.0+0x1a/0x30 [Thu Dec 5 12:28:36 2024] ? handle_mm_fault+0xa2/0x360 [Thu Dec 5 12:28:36 2024] ? do_user_addr_fault+0x304/0x690 [Thu Dec 5 12:28:36 2024] ? exc_page_fault+0x7f/0x180 [Thu Dec 5 12:28:36 2024] entry_SYSCALL_64_after_hwframe+0x78/0x80 [Thu Dec 5 12:28:36 2024] RIP: 0033:0x7f58a7924804 [Thu Dec 5 12:28:36 2024] RSP: 002b:00007ffcbf307b18 EFLAGS: 00000202 ORIG_RAX: 0000000000000003 [Thu Dec 5 12:28:36 2024] RAX: ffffffffffffffda RBX: 00005594d00fb9e0 RCX: 00007f58a7924804 [Thu Dec 5 12:28:36 2024] RDX: 0000000000000003 RSI: 00005594d12699b0 RDI: 0000000000000003 [Thu Dec 5 12:28:36 2024] RBP: 00007ffcbf307b30 R08: 00005591886b8b39 R09: 0000000000000007 [Thu Dec 5 12:28:36 2024] R10: 00005594d12699c0 R11: 0000000000000202 R12: 0000000000000000 [Thu Dec 5 12:28:36 2024] R13: 0000000000000000 R14: 00005594d00fb9e0 R15: 0000000000000007 [Thu Dec 5 12:28:37 2024]