[Thu Dec 5 12:27:03 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh reset (start) [Thu Dec 5 12:27:07 2024] pcieport 0000:d7:00.0: bridge window [io 0x1000-0x0fff] to [bus d8] add_size 1000 [Thu Dec 5 12:27:07 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: can't assign; no space [Thu Dec 5 12:27:07 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: failed to assign [Thu Dec 5 12:27:07 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: can't assign; no space [Thu Dec 5 12:27:07 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: failed to assign [Thu Dec 5 12:27:07 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh reset (done) [Thu Dec 5 12:27:14 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh status (start) [Thu Dec 5 12:27:17 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh status (done) [Thu Dec 5 12:27:17 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh (start) [Thu Dec 5 12:27:21 2024] ioatdma 0000:00:04.7: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:00:04.6: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:00:04.5: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:00:04.4: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:00:04.3: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:00:04.2: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:00:04.1: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:00:04.0: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:80:04.7: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:80:04.6: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:80:04.5: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:80:04.4: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:80:04.3: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:80:04.2: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:80:04.1: Removing dma and dca services [Thu Dec 5 12:27:21 2024] ioatdma 0000:80:04.0: Removing dma and dca services [Thu Dec 5 12:27:24 2024] pcieport 0000:d7:00.0: bridge window [io 0x1000-0x0fff] to [bus d8] add_size 1000 [Thu Dec 5 12:27:24 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: can't assign; no space [Thu Dec 5 12:27:24 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: failed to assign [Thu Dec 5 12:27:24 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: can't assign; no space [Thu Dec 5 12:27:24 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: failed to assign [Thu Dec 5 12:27:24 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh (done) [Thu Dec 5 12:27:25 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh reset (start) [Thu Dec 5 12:27:29 2024] nvme nvme0: pci function 0000:5e:00.0 [Thu Dec 5 12:27:29 2024] nvme nvme0: Shutdown timeout set to 15 seconds [Thu Dec 5 12:27:29 2024] nvme nvme0: 72/0/0 default/read/poll queues [Thu Dec 5 12:27:29 2024] nvme nvme0: Ignoring bogus Namespace Identifiers [Thu Dec 5 12:27:30 2024] pcieport 0000:d7:00.0: bridge window [io 0x1000-0x0fff] to [bus d8] add_size 1000 [Thu Dec 5 12:27:30 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: can't assign; no space [Thu Dec 5 12:27:30 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: failed to assign [Thu Dec 5 12:27:30 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: can't assign; no space [Thu Dec 5 12:27:30 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: failed to assign [Thu Dec 5 12:27:30 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh reset (done) [Thu Dec 5 12:27:30 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh (start) [Thu Dec 5 12:27:34 2024] ioatdma 0000:00:04.7: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:00:04.6: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:00:04.5: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:00:04.4: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:00:04.3: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:00:04.2: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:00:04.1: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:00:04.0: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:80:04.7: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:80:04.6: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:80:04.5: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:80:04.4: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:80:04.3: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:80:04.2: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:80:04.1: Removing dma and dca services [Thu Dec 5 12:27:34 2024] ioatdma 0000:80:04.0: Removing dma and dca services [Thu Dec 5 12:27:37 2024] pcieport 0000:d7:00.0: bridge window [io 0x1000-0x0fff] to [bus d8] add_size 1000 [Thu Dec 5 12:27:37 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: can't assign; no space [Thu Dec 5 12:27:37 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: failed to assign [Thu Dec 5 12:27:37 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: can't assign; no space [Thu Dec 5 12:27:37 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: failed to assign [Thu Dec 5 12:27:37 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh (done) [Thu Dec 5 12:27:53 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh reset (start) [Thu Dec 5 12:27:57 2024] nvme nvme0: pci function 0000:5e:00.0 [Thu Dec 5 12:27:57 2024] nvme nvme0: Shutdown timeout set to 15 seconds [Thu Dec 5 12:27:57 2024] nvme nvme0: 72/0/0 default/read/poll queues [Thu Dec 5 12:27:57 2024] nvme nvme0: Ignoring bogus Namespace Identifiers [Thu Dec 5 12:27:58 2024] pcieport 0000:d7:00.0: bridge window [io 0x1000-0x0fff] to [bus d8] add_size 1000 [Thu Dec 5 12:27:58 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: can't assign; no space [Thu Dec 5 12:27:58 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: failed to assign [Thu Dec 5 12:27:58 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: can't assign; no space [Thu Dec 5 12:27:58 2024] pcieport 0000:d7:00.0: bridge window [io size 0x1000]: failed to assign [Thu Dec 5 12:27:58 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh reset (done) [Thu Dec 5 12:27:58 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh cleanup (start) [Thu Dec 5 12:28:02 2024] spdk: /var/jenkins/workspace/vhost-phy-autotest/spdk/scripts/setup.sh cleanup (done) [Thu Dec 5 12:28:02 2024] ######## Autotest Cleanup Dump ######## [Thu Dec 5 12:28:02 2024] sysrq: Show backtrace of all active CPUs [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 10 [Thu Dec 5 12:28:02 2024] CPU: 10 PID: 2193592 Comm: bash Tainted: G OE 6.8.9-200.fc39.x86_64 #1 [Thu Dec 5 12:28:02 2024] Hardware name: Intel Corporation S2600WFT/S2600WFT, BIOS SE5C620.86B.02.01.0015.032120220358 03/21/2022 [Thu Dec 5 12:28:02 2024] Call Trace: [Thu Dec 5 12:28:02 2024] [Thu Dec 5 12:28:02 2024] dump_stack_lvl+0x64/0x80 [Thu Dec 5 12:28:02 2024] nmi_cpu_backtrace+0xd6/0x110 [Thu Dec 5 12:28:02 2024] ? __pfx_nmi_raise_cpu_backtrace+0x10/0x10 [Thu Dec 5 12:28:02 2024] nmi_trigger_cpumask_backtrace+0xf3/0x140 [Thu Dec 5 12:28:02 2024] __handle_sysrq+0xe1/0x260 [Thu Dec 5 12:28:02 2024] write_sysrq_trigger+0x59/0x80 [Thu Dec 5 12:28:02 2024] proc_reg_write+0x5a/0xa0 [Thu Dec 5 12:28:02 2024] vfs_write+0xed/0x470 [Thu Dec 5 12:28:02 2024] ? filp_flush+0x52/0x70 [Thu Dec 5 12:28:02 2024] ksys_write+0x6f/0xf0 [Thu Dec 5 12:28:02 2024] do_syscall_64+0x83/0x170 [Thu Dec 5 12:28:02 2024] ? count_memcg_events.constprop.0+0x1a/0x30 [Thu Dec 5 12:28:02 2024] ? handle_mm_fault+0xa2/0x360 [Thu Dec 5 12:28:02 2024] ? do_user_addr_fault+0x304/0x690 [Thu Dec 5 12:28:02 2024] ? exc_page_fault+0x7f/0x180 [Thu Dec 5 12:28:02 2024] entry_SYSCALL_64_after_hwframe+0x78/0x80 [Thu Dec 5 12:28:02 2024] RIP: 0033:0x7f2b3db21ee4 [Thu Dec 5 12:28:02 2024] Code: c7 00 16 00 00 00 b8 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 80 3d 85 74 0d 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 55 48 89 e5 48 83 ec 20 48 89 [Thu Dec 5 12:28:02 2024] RSP: 002b:00007fffee93c2f8 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [Thu Dec 5 12:28:02 2024] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f2b3db21ee4 [Thu Dec 5 12:28:02 2024] RDX: 0000000000000002 RSI: 000055a1cd631c20 RDI: 0000000000000001 [Thu Dec 5 12:28:02 2024] RBP: 00007fffee93c320 R08: 0000000000000073 R09: 0000000000000001 [Thu Dec 5 12:28:02 2024] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000002 [Thu Dec 5 12:28:02 2024] R13: 000055a1cd631c20 R14: 00007f2b3dbf25c0 R15: 00007f2b3dbeff20 [Thu Dec 5 12:28:02 2024] [Thu Dec 5 12:28:02 2024] Sending NMI from CPU 10 to CPUs 0-9,11-71: [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 46 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 71 [Thu Dec 5 12:28:02 2024] CPU: 71 PID: 0 Comm: swapper/71 Tainted: G OE 6.8.9-200.fc39.x86_64 #1 [Thu Dec 5 12:28:02 2024] Hardware name: Intel Corporation S2600WFT/S2600WFT, BIOS SE5C620.86B.02.01.0015.032120220358 03/21/2022 [Thu Dec 5 12:28:02 2024] RIP: 0010:psi_group_change+0x1c4/0x500 [Thu Dec 5 12:28:02 2024] Code: f9 02 0f 84 87 00 00 00 45 8b 63 04 45 85 e4 0f 95 c0 31 c9 84 c0 74 50 89 d0 d3 e0 83 c1 01 41 09 c2 44 89 d0 25 00 01 00 00 <83> f9 03 75 c9 45 8b 43 08 45 85 c0 0f 85 c1 01 00 00 b9 04 00 00 [Thu Dec 5 12:28:02 2024] RSP: 0018:ffff9ec4049bfd48 EFLAGS: 00000046 [Thu Dec 5 12:28:02 2024] RAX: 0000000000000000 RBX: 00000000fffffffb RCX: 0000000000000005 [Thu Dec 5 12:28:02 2024] RDX: 0000000000000001 RSI: 0000000000000047 RDI: ffff8c6d395c3800 [Thu Dec 5 12:28:02 2024] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 [Thu Dec 5 12:28:02 2024] R10: 0000000000000010 R11: ffff8c6d3c5cbec0 R12: 0000000000000000 [Thu Dec 5 12:28:02 2024] R13: 0000000000000000 R14: ffff8c6d395c3800 R15: 0000000000000000 [Thu Dec 5 12:28:02 2024] FS: 0000000000000000(0000) GS:ffff8c6d3c580000(0000) knlGS:0000000000000000 [Thu Dec 5 12:28:02 2024] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [Thu Dec 5 12:28:02 2024] CR2: 000056217b778880 CR3: 0000000cd705e001 CR4: 00000000007706f0 [Thu Dec 5 12:28:02 2024] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [Thu Dec 5 12:28:02 2024] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [Thu Dec 5 12:28:02 2024] PKRU: 55555554 [Thu Dec 5 12:28:02 2024] Call Trace: [Thu Dec 5 12:28:02 2024] [Thu Dec 5 12:28:02 2024] ? nmi_cpu_backtrace+0x99/0x110 [Thu Dec 5 12:28:02 2024] ? nmi_cpu_backtrace_handler+0x11/0x20 [Thu Dec 5 12:28:02 2024] ? nmi_handle+0x65/0x170 [Thu Dec 5 12:28:02 2024] ? default_do_nmi+0x40/0x100 [Thu Dec 5 12:28:02 2024] ? exc_nmi+0x125/0x1a0 [Thu Dec 5 12:28:02 2024] ? end_repeat_nmi+0xf/0x60 [Thu Dec 5 12:28:02 2024] ? psi_group_change+0x1c4/0x500 [Thu Dec 5 12:28:02 2024] ? psi_group_change+0x1c4/0x500 [Thu Dec 5 12:28:02 2024] ? psi_group_change+0x1c4/0x500 [Thu Dec 5 12:28:02 2024] [Thu Dec 5 12:28:02 2024] [Thu Dec 5 12:28:02 2024] ? update_load_avg+0x7e/0x800 [Thu Dec 5 12:28:02 2024] psi_task_change+0x89/0xb0 [Thu Dec 5 12:28:02 2024] enqueue_task+0xec/0x160 [Thu Dec 5 12:28:02 2024] ? sched_clock+0x10/0x30 [Thu Dec 5 12:28:02 2024] ttwu_do_activate+0x53/0x230 [Thu Dec 5 12:28:02 2024] sched_ttwu_pending+0xe3/0x1a0 [Thu Dec 5 12:28:02 2024] __flush_smp_call_function_queue+0x146/0x410 [Thu Dec 5 12:28:02 2024] flush_smp_call_function_queue+0x3a/0x70 [Thu Dec 5 12:28:02 2024] do_idle+0x169/0x270 [Thu Dec 5 12:28:02 2024] cpu_startup_entry+0x2a/0x30 [Thu Dec 5 12:28:02 2024] start_secondary+0x12d/0x160 [Thu Dec 5 12:28:02 2024] secondary_startup_64_no_verify+0x184/0x18b [Thu Dec 5 12:28:02 2024] [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 53 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 59 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 34 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 70 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 31 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 47 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 60 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 12 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 61 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 25 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 3 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 63 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 27 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 6 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 42 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 32 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 51 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 24 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 14 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 50 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 15 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 23 [Thu Dec 5 12:28:02 2024] CPU: 23 PID: 2193705 Comm: bash Tainted: G OE 6.8.9-200.fc39.x86_64 #1 [Thu Dec 5 12:28:02 2024] Hardware name: Intel Corporation S2600WFT/S2600WFT, BIOS SE5C620.86B.02.01.0015.032120220358 03/21/2022 [Thu Dec 5 12:28:02 2024] RIP: 0033:0x7fe5bc567a62 [Thu Dec 5 12:28:02 2024] Code: bc c9 f3 a4 c3 0f 1f 00 f3 0f 1e fa 89 f8 62 a1 fd 00 ef c0 25 ff 0f 00 00 3d e0 0f 00 00 0f 87 24 01 00 00 62 f1 7d 20 74 07 fb 93 c0 85 c0 74 16 0f bc c0 c3 66 90 0f bc c0 29 cf 8d 84 07 [Thu Dec 5 12:28:02 2024] RSP: 002b:00007ffef5530c28 EFLAGS: 00000287 [Thu Dec 5 12:28:02 2024] RAX: 0000000000000b30 RBX: 00005635434b2c40 RCX: 0000000000000030 [Thu Dec 5 12:28:02 2024] RDX: 0000563542b2ca20 RSI: 0000000000000030 RDI: 0000563542d61b30 [Thu Dec 5 12:28:02 2024] RBP: 00007ffef5530c50 R08: 00007fe5bc5ceac0 R09: 0000563543509b49 [Thu Dec 5 12:28:02 2024] R10: 00005635434ec760 R11: 0000000000000010 R12: 0000563542d61b30 [Thu Dec 5 12:28:02 2024] R13: 0000563541731370 R14: 0000563542d7a7b0 R15: 00005635434d8460 [Thu Dec 5 12:28:02 2024] FS: 00007fe5bc3f3740 GS: 0000000000000000 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 45 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 58 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 22 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 30 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 7 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 20 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 56 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 17 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 55 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 9 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 48 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 16 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 67 skipped: idling at intel_idle+0x71/0xe0 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 43 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 19 skipped: idling at intel_idle+0x71/0xe0 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 35 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 13 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.010 msecs [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 65 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 29 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 40 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 62 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 39 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.024 msecs [Thu Dec 5 12:28:02 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.036 msecs [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 57 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.055 msecs [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 21 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 49 [Thu Dec 5 12:28:02 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.069 msecs [Thu Dec 5 12:28:02 2024] CPU: 49 PID: 2530 Comm: telegraf Tainted: G OE 6.8.9-200.fc39.x86_64 #1 [Thu Dec 5 12:28:02 2024] Hardware name: Intel Corporation S2600WFT/S2600WFT, BIOS SE5C620.86B.02.01.0015.032120220358 03/21/2022 [Thu Dec 5 12:28:02 2024] RIP: 0033:0x442e74 [Thu Dec 5 12:28:02 2024] Code: 10 01 00 00 48 8b 44 24 48 48 8b 4c 24 40 31 f6 31 ff 0f 1f 40 00 e9 a9 02 00 00 41 89 d0 31 d2 41 f7 f0 89 94 24 b0 00 00 00 <8b> 94 24 ac 00 00 00 0f 1f 44 00 00 39 94 24 a8 00 00 00 0f 84 78 [Thu Dec 5 12:28:02 2024] RSP: 002b:00007f71ae9ffc78 EFLAGS: 00000246 [Thu Dec 5 12:28:02 2024] RAX: 0000000000000001 RBX: 000000c0000b5800 RCX: 000059ea261ca3d2 [Thu Dec 5 12:28:02 2024] RDX: 000000000000000b RSI: 0000000000000000 RDI: 0000000000000000 [Thu Dec 5 12:28:02 2024] RBP: 00007f71ae9ffd90 R08: 0000000000000048 R09: 000000c000052050 [Thu Dec 5 12:28:02 2024] R10: 000000000000002a R11: 000000c000114000 R12: 000059e9ea81f1ab [Thu Dec 5 12:28:02 2024] R13: 000000c000702180 R14: 00000000041ac211 R15: 0000000000000000 [Thu Dec 5 12:28:02 2024] FS: 00007f71aea006c0 GS: 0000000000000000 [Thu Dec 5 12:28:02 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.199 msecs [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 0 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.217 msecs [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 38 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.232 msecs [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 68 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 37 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 26 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 66 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 1 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.245 msecs [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 64 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.259 msecs [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 44 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 69 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 33 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 4 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 54 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 2 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 28 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 52 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 18 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 36 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 5 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 8 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 11 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] NMI backtrace for cpu 41 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Thu Dec 5 12:28:02 2024] sysrq: Show Memory [Thu Dec 5 12:28:03 2024] Mem-Info: [Thu Dec 5 12:28:03 2024] active_anon:3007022 inactive_anon:0 isolated_anon:0 active_file:102146 inactive_file:946798 isolated_file:0 unevictable:768 dirty:0 writeback:0 slab_reclaimable:54541 slab_unreclaimable:133157 mapped:39981 shmem:2889162 pagetables:1814 sec_pagetables:0 bounce:0 kernel_misc_reclaimable:0 free:10355914 free_pcp:73411 free_cma:0 [Thu Dec 5 12:28:03 2024] Node 0 active_anon:1806240kB inactive_anon:0kB active_file:152660kB inactive_file:111128kB unevictable:3072kB isolated(anon):0kB isolated(file):0kB mapped:36068kB dirty:0kB writeback:0kB shmem:1687104kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:7432kB pagetables:2732kB sec_pagetables:0kB all_unreclaimable? no [Thu Dec 5 12:28:03 2024] Node 1 active_anon:10221932kB inactive_anon:0kB active_file:255924kB inactive_file:3676064kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:124264kB dirty:0kB writeback:0kB shmem:9869544kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:9512kB pagetables:4548kB sec_pagetables:0kB all_unreclaimable? no [Thu Dec 5 12:28:03 2024] Node 0 DMA free:11264kB boost:0kB min:20kB low:32kB high:44kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15980kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [Thu Dec 5 12:28:03 2024] lowmem_reserve[]: 0 1666 31798 31798 31798 [Thu Dec 5 12:28:03 2024] Node 0 DMA32 free:1702868kB boost:0kB min:2552kB low:4256kB high:5960kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:1772500kB managed:1706964kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [Thu Dec 5 12:28:03 2024] lowmem_reserve[]: 0 0 30131 30131 30131 [Thu Dec 5 12:28:03 2024] Node 0 Normal free:27982272kB boost:0kB min:46180kB low:77032kB high:107884kB reserved_highatomic:0KB active_anon:1806240kB inactive_anon:0kB active_file:152660kB inactive_file:111128kB unevictable:3072kB writepending:0kB present:31444992kB managed:30863284kB mlocked:0kB bounce:0kB free_pcp:182656kB local_pcp:1984kB free_cma:0kB [Thu Dec 5 12:28:03 2024] lowmem_reserve[]: 0 0 0 0 0 [Thu Dec 5 12:28:03 2024] Node 1 Normal free:11731276kB boost:0kB min:41352kB low:68980kB high:96608kB reserved_highatomic:0KB active_anon:10222424kB inactive_anon:0kB active_file:255924kB inactive_file:3676064kB unevictable:0kB writepending:0kB present:28311552kB managed:27705540kB mlocked:0kB bounce:0kB free_pcp:105680kB local_pcp:0kB free_cma:0kB [Thu Dec 5 12:28:03 2024] lowmem_reserve[]: 0 0 0 0 0 [Thu Dec 5 12:28:03 2024] Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 2*4096kB (M) = 11264kB [Thu Dec 5 12:28:03 2024] Node 0 DMA32: 9*4kB (UM) 4*8kB (UM) 11*16kB (UM) 9*32kB (UM) 13*64kB (UM) 7*128kB (UM) 13*256kB (UM) 11*512kB (UM) 6*1024kB (UM) 5*2048kB (M) 409*4096kB (M) = 1702868kB [Thu Dec 5 12:28:03 2024] Node 0 Normal: 26590*4kB (UME) 41470*8kB (UME) 29896*16kB (UME) 16293*32kB (UME) 8298*64kB (UME) 3063*128kB (UME) 1261*256kB (UME) 834*512kB (UME) 536*1024kB (UME) 254*2048kB (UME) 5812*4096kB (UME) = 27985800kB [Thu Dec 5 12:28:03 2024] Node 1 Normal: 47269*4kB (UME) 35830*8kB (UME) 20234*16kB (UME) 9809*32kB (UME) 4758*64kB (UME) 1689*128kB (UME) 583*256kB (UME) 447*512kB (UME) 468*1024kB (UME) 254*2048kB (UME) 2130*4096kB (UME) = 11736068kB [Thu Dec 5 12:28:03 2024] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB [Thu Dec 5 12:28:03 2024] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [Thu Dec 5 12:28:03 2024] Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB [Thu Dec 5 12:28:03 2024] Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [Thu Dec 5 12:28:03 2024] 3938106 total pagecache pages [Thu Dec 5 12:28:03 2024] 0 pages in swap cache [Thu Dec 5 12:28:03 2024] Free swap = 8388604kB [Thu Dec 5 12:28:03 2024] Total swap = 8388604kB [Thu Dec 5 12:28:03 2024] 15386256 pages RAM [Thu Dec 5 12:28:03 2024] 0 pages HighMem/MovableOnly [Thu Dec 5 12:28:03 2024] 313469 pages reserved [Thu Dec 5 12:28:03 2024] 0 pages cma reserved [Thu Dec 5 12:28:03 2024] 0 pages hwpoisoned [Thu Dec 5 12:28:03 2024] sysrq: Show Blocked State [Thu Dec 5 12:28:03 2024] task:ipmitool state:D stack:0 pid:2213698 tgid:2213698 ppid:2193749 flags:0x00004002 [Thu Dec 5 12:28:03 2024] Call Trace: [Thu Dec 5 12:28:03 2024] [Thu Dec 5 12:28:03 2024] __schedule+0x413/0x1720 [Thu Dec 5 12:28:03 2024] ? ttwu_do_activate+0x68/0x230 [Thu Dec 5 12:28:04 2024] schedule+0x32/0xd0 [Thu Dec 5 12:28:04 2024] schedule_timeout+0x151/0x160 [Thu Dec 5 12:28:04 2024] wait_for_completion+0x8a/0x160 [Thu Dec 5 12:28:04 2024] __wait_rcu_gp+0x15b/0x170 [Thu Dec 5 12:28:04 2024] synchronize_rcu+0x133/0x150 [Thu Dec 5 12:28:04 2024] ? __pfx_call_rcu_hurry+0x10/0x10 [Thu Dec 5 12:28:04 2024] ? __pfx_wakeme_after_rcu+0x10/0x10 [Thu Dec 5 12:28:04 2024] _ipmi_destroy_user+0x2aa/0x320 [ipmi_msghandler] [Thu Dec 5 12:28:04 2024] ipmi_destroy_user+0x12/0x60 [ipmi_msghandler] [Thu Dec 5 12:28:04 2024] ipmi_release+0x1f/0x60 [ipmi_devintf] [Thu Dec 5 12:28:04 2024] __fput+0x97/0x2c0 [Thu Dec 5 12:28:04 2024] __x64_sys_close+0x3d/0x80 [Thu Dec 5 12:28:04 2024] do_syscall_64+0x83/0x170 [Thu Dec 5 12:28:04 2024] ? handle_mm_fault+0xa2/0x360 [Thu Dec 5 12:28:04 2024] ? do_user_addr_fault+0x304/0x690 [Thu Dec 5 12:28:04 2024] ? exc_page_fault+0x7f/0x180 [Thu Dec 5 12:28:04 2024] entry_SYSCALL_64_after_hwframe+0x78/0x80 [Thu Dec 5 12:28:04 2024] RIP: 0033:0x7faeec149804 [Thu Dec 5 12:28:04 2024] RSP: 002b:00007ffee42d62a8 EFLAGS: 00000202 ORIG_RAX: 0000000000000003 [Thu Dec 5 12:28:04 2024] RAX: ffffffffffffffda RBX: 000055b34f05c9e0 RCX: 00007faeec149804 [Thu Dec 5 12:28:04 2024] RDX: 0000000000000003 RSI: 000055b350a2bd90 RDI: 0000000000000003 [Thu Dec 5 12:28:04 2024] RBP: 00007ffee42d62c0 R08: 000055b60b97b71b R09: 0000000000000007 [Thu Dec 5 12:28:04 2024] R10: 000055b350a2bda0 R11: 0000000000000202 R12: 0000000000000000 [Thu Dec 5 12:28:04 2024] R13: 0000000000000000 R14: 000055b34f05c9e0 R15: 0000000000000007 [Thu Dec 5 12:28:04 2024]