[Sun Dec 15 18:23:20 2024] sysrq: HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(v) show-blocked-tasks(w) dump-ftrace-buffer(z) [Sun Dec 15 18:23:21 2024] sysrq: HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(v) show-blocked-tasks(w) dump-ftrace-buffer(z) [Sun Dec 15 18:23:38 2024] sysrq: HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(v) show-blocked-tasks(w) dump-ftrace-buffer(z) [Sun Dec 15 18:24:03 2024] sysrq: HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(v) show-blocked-tasks(w) dump-ftrace-buffer(z) [Sun Dec 15 18:25:47 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh reset (start) [Sun Dec 15 18:25:51 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh reset (done) [Sun Dec 15 18:25:56 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh status (start) [Sun Dec 15 18:25:59 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh status (done) [Sun Dec 15 18:25:59 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh (start) [Sun Dec 15 18:26:02 2024] ioatdma 0000:00:04.7: Removing dma and dca services [Sun Dec 15 18:26:02 2024] ioatdma 0000:00:04.6: Removing dma and dca services [Sun Dec 15 18:26:02 2024] ioatdma 0000:00:04.5: Removing dma and dca services [Sun Dec 15 18:26:02 2024] ioatdma 0000:00:04.4: Removing dma and dca services [Sun Dec 15 18:26:02 2024] ioatdma 0000:00:04.3: Removing dma and dca services [Sun Dec 15 18:26:02 2024] ioatdma 0000:00:04.2: Removing dma and dca services [Sun Dec 15 18:26:02 2024] ioatdma 0000:00:04.1: Removing dma and dca services [Sun Dec 15 18:26:02 2024] ioatdma 0000:00:04.0: Removing dma and dca services [Sun Dec 15 18:26:02 2024] ioatdma 0000:80:04.7: Removing dma and dca services [Sun Dec 15 18:26:02 2024] ioatdma 0000:80:04.6: Removing dma and dca services [Sun Dec 15 18:26:02 2024] ioatdma 0000:80:04.5: Removing dma and dca services [Sun Dec 15 18:26:03 2024] ioatdma 0000:80:04.4: Removing dma and dca services [Sun Dec 15 18:26:03 2024] ioatdma 0000:80:04.3: Removing dma and dca services [Sun Dec 15 18:26:03 2024] ioatdma 0000:80:04.2: Removing dma and dca services [Sun Dec 15 18:26:03 2024] ioatdma 0000:80:04.1: Removing dma and dca services [Sun Dec 15 18:26:03 2024] ioatdma 0000:80:04.0: Removing dma and dca services [Sun Dec 15 18:26:05 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh (done) [Sun Dec 15 18:26:06 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh reset (start) [Sun Dec 15 18:26:09 2024] nvme nvme0: pci function 0000:5e:00.0 [Sun Dec 15 18:26:09 2024] nvme nvme0: 72/0/0 default/read/poll queues [Sun Dec 15 18:26:11 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh reset (done) [Sun Dec 15 18:26:11 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh (start) [Sun Dec 15 18:26:14 2024] ioatdma 0000:00:04.7: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:00:04.6: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:00:04.5: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:00:04.4: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:00:04.3: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:00:04.2: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:00:04.1: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:00:04.0: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:80:04.7: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:80:04.6: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:80:04.5: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:80:04.4: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:80:04.3: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:80:04.2: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:80:04.1: Removing dma and dca services [Sun Dec 15 18:26:14 2024] ioatdma 0000:80:04.0: Removing dma and dca services [Sun Dec 15 18:26:17 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh (done) [Sun Dec 15 18:26:32 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh reset (start) [Sun Dec 15 18:26:35 2024] nvme nvme0: pci function 0000:5e:00.0 [Sun Dec 15 18:26:35 2024] nvme nvme0: 72/0/0 default/read/poll queues [Sun Dec 15 18:26:36 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh reset (done) [Sun Dec 15 18:26:36 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh cleanup (start) [Sun Dec 15 18:26:39 2024] spdk: /var/jenkins/workspace/vmd-phy-autotest/spdk/scripts/setup.sh cleanup (done) [Sun Dec 15 18:26:39 2024] ######## Autotest Cleanup Dump ######## [Sun Dec 15 18:26:39 2024] sysrq: Show backtrace of all active CPUs [Sun Dec 15 18:26:39 2024] NMI backtrace for cpu 0 [Sun Dec 15 18:26:39 2024] CPU: 0 PID: 980663 Comm: bash Tainted: G OE 6.8.9-200.fc39.x86_64 #1 [Sun Dec 15 18:26:39 2024] Hardware name: Intel Corporation S2600WFQ/S2600WFQ, BIOS SE5C620.86B.02.01.0008.031920191559 03/19/2019 [Sun Dec 15 18:26:39 2024] Call Trace: [Sun Dec 15 18:26:39 2024] [Sun Dec 15 18:26:39 2024] dump_stack_lvl+0x64/0x80 [Sun Dec 15 18:26:39 2024] nmi_cpu_backtrace+0xd6/0x110 [Sun Dec 15 18:26:39 2024] ? __pfx_nmi_raise_cpu_backtrace+0x10/0x10 [Sun Dec 15 18:26:39 2024] nmi_trigger_cpumask_backtrace+0xf3/0x140 [Sun Dec 15 18:26:39 2024] __handle_sysrq+0xe1/0x260 [Sun Dec 15 18:26:39 2024] write_sysrq_trigger+0x59/0x80 [Sun Dec 15 18:26:39 2024] proc_reg_write+0x5a/0xa0 [Sun Dec 15 18:26:39 2024] vfs_write+0xed/0x470 [Sun Dec 15 18:26:39 2024] ? __count_memcg_events+0x69/0x100 [Sun Dec 15 18:26:39 2024] ksys_write+0x6f/0xf0 [Sun Dec 15 18:26:39 2024] do_syscall_64+0x83/0x170 [Sun Dec 15 18:26:39 2024] ? exc_page_fault+0x7f/0x180 [Sun Dec 15 18:26:39 2024] entry_SYSCALL_64_after_hwframe+0x78/0x80 [Sun Dec 15 18:26:39 2024] RIP: 0033:0x7faf92966ee4 [Sun Dec 15 18:26:39 2024] Code: c7 00 16 00 00 00 b8 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 80 3d 85 74 0d 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 55 48 89 e5 48 83 ec 20 48 89 [Sun Dec 15 18:26:39 2024] RSP: 002b:00007fffab0c5a48 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [Sun Dec 15 18:26:39 2024] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007faf92966ee4 [Sun Dec 15 18:26:39 2024] RDX: 0000000000000002 RSI: 00005581afd5e990 RDI: 0000000000000001 [Sun Dec 15 18:26:39 2024] RBP: 00007fffab0c5a70 R08: 0000000000000073 R09: 0000000000000001 [Sun Dec 15 18:26:39 2024] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000002 [Sun Dec 15 18:26:39 2024] R13: 00005581afd5e990 R14: 00007faf92a375c0 R15: 00007faf92a34f20 [Sun Dec 15 18:26:39 2024] [Sun Dec 15 18:26:39 2024] NMI backtrace for cpu 2 [Sun Dec 15 18:26:39 2024] CPU: 2 PID: 285393 Comm: kworker/2:2 Tainted: G OE 6.8.9-200.fc39.x86_64 #1 [Sun Dec 15 18:26:39 2024] Hardware name: Intel Corporation S2600WFQ/S2600WFQ, BIOS SE5C620.86B.02.01.0008.031920191559 03/19/2019 [Sun Dec 15 18:26:39 2024] Workqueue: events drm_fb_helper_damage_work [Sun Dec 15 18:26:39 2024] RIP: 0010:memcpy_toio+0x71/0xb0 [Sun Dec 15 18:26:39 2024] Code: 5c c3 cc cc cc cc 48 85 db 74 f2 40 f6 c5 01 75 45 48 83 fb 01 76 06 40 f6 c5 02 75 25 48 89 d9 48 89 ef 4c 89 e6 48 c1 e9 02 a5 f6 c3 02 74 02 66 a5 f6 c3 01 74 01 a4 5b 5d 41 5c c3 cc cc [Sun Dec 15 18:26:39 2024] RSP: 0018:ffffa793f0df7ac0 EFLAGS: 00010206 [Sun Dec 15 18:26:39 2024] RAX: 0000000000000000 RBX: 0000000000001000 RCX: 00000000000003f0 [Sun Dec 15 18:26:39 2024] RDX: 0000000000001000 RSI: ffffa793cddb8040 RDI: ffffa793e10b5040 [Sun Dec 15 18:26:39 2024] RBP: ffffa793e10b5000 R08: ffffa793f0df7b90 R09: 0000000000000001 [Sun Dec 15 18:26:39 2024] Sending NMI from CPU 0 to CPUs 1-71: [Sun Dec 15 18:26:39 2024] R10: 0000000000000004 R11: ffff97dac9519308 R12: ffffa793cddb8000 [Sun Dec 15 18:26:39 2024] R13: ffffa793cddb8000 R14: ffff97dada217d40 R15: 0000000000000014 [Sun Dec 15 18:26:39 2024] FS: 0000000000000000(0000) GS:ffff97e5d0100000(0000) knlGS:0000000000000000 [Sun Dec 15 18:26:39 2024] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [Sun Dec 15 18:26:39 2024] CR2: 00005581afd5e990 CR3: 00000001c4c52003 CR4: 00000000007706f0 [Sun Dec 15 18:26:39 2024] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [Sun Dec 15 18:26:39 2024] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [Sun Dec 15 18:26:39 2024] PKRU: 55555554 [Sun Dec 15 18:26:39 2024] Call Trace: [Sun Dec 15 18:26:39 2024] [Sun Dec 15 18:26:39 2024] ? nmi_cpu_backtrace+0x99/0x110 [Sun Dec 15 18:26:39 2024] ? nmi_cpu_backtrace_handler+0x11/0x20 [Sun Dec 15 18:26:39 2024] ? nmi_handle+0x65/0x170 [Sun Dec 15 18:26:39 2024] ? default_do_nmi+0x40/0x100 [Sun Dec 15 18:26:39 2024] ? exc_nmi+0x125/0x1a0 [Sun Dec 15 18:26:39 2024] ? end_repeat_nmi+0xf/0x60 [Sun Dec 15 18:26:39 2024] ? memcpy_toio+0x71/0xb0 [Sun Dec 15 18:26:39 2024] ? memcpy_toio+0x71/0xb0 [Sun Dec 15 18:26:39 2024] ? memcpy_toio+0x71/0xb0 [Sun Dec 15 18:26:39 2024] [Sun Dec 15 18:26:39 2024] [Sun Dec 15 18:26:39 2024] drm_fb_memcpy+0x164/0x220 [Sun Dec 15 18:26:39 2024] ? drm_plane_get_damage_clips+0x27/0x80 [Sun Dec 15 18:26:39 2024] ast_primary_plane_helper_atomic_update+0x156/0x200 [ast] [Sun Dec 15 18:26:39 2024] drm_atomic_helper_commit_planes+0x10d/0x310 [Sun Dec 15 18:26:39 2024] drm_atomic_helper_commit_tail_rpm+0x56/0x90 [Sun Dec 15 18:26:39 2024] ast_mode_config_helper_atomic_commit_tail+0x29/0x40 [ast] [Sun Dec 15 18:26:39 2024] commit_tail+0xac/0x160 [Sun Dec 15 18:26:39 2024] drm_atomic_helper_commit+0x11a/0x140 [Sun Dec 15 18:26:39 2024] drm_atomic_commit+0x97/0xd0 [Sun Dec 15 18:26:39 2024] ? __pfx___drm_printfn_info+0x10/0x10 [Sun Dec 15 18:26:39 2024] drm_atomic_helper_dirtyfb+0x1ad/0x280 [Sun Dec 15 18:26:39 2024] drm_fbdev_generic_helper_fb_dirty+0x240/0x2f0 [Sun Dec 15 18:26:39 2024] drm_fb_helper_damage_work+0x93/0x170 [Sun Dec 15 18:26:39 2024] process_one_work+0x173/0x340 [Sun Dec 15 18:26:39 2024] worker_thread+0x27b/0x3a0 [Sun Dec 15 18:26:39 2024] ? __pfx_worker_thread+0x10/0x10 [Sun Dec 15 18:26:39 2024] kthread+0xe5/0x120 [Sun Dec 15 18:26:39 2024] ? __pfx_kthread+0x10/0x10 [Sun Dec 15 18:26:39 2024] ret_from_fork+0x31/0x50 [Sun Dec 15 18:26:39 2024] ? __pfx_kthread+0x10/0x10 [Sun Dec 15 18:26:39 2024] ret_from_fork_asm+0x1b/0x30 [Sun Dec 15 18:26:39 2024] [Sun Dec 15 18:26:39 2024] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 96.121 msecs [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 36 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 18 [Sun Dec 15 18:26:40 2024] CPU: 18 PID: 999543 Comm: bash Tainted: G OE 6.8.9-200.fc39.x86_64 #1 [Sun Dec 15 18:26:40 2024] Hardware name: Intel Corporation S2600WFQ/S2600WFQ, BIOS SE5C620.86B.02.01.0008.031920191559 03/19/2019 [Sun Dec 15 18:26:40 2024] RIP: 0010:count_memcg_events.constprop.0+0x25/0x30 [Sun Dec 15 18:26:40 2024] Code: 90 90 90 90 90 53 9c 58 0f 1f 40 00 48 89 c3 fa 0f 1f 44 00 00 ba 01 00 00 00 e8 f6 d3 09 00 80 e7 02 74 06 fb 0f 1f 44 00 00 <5b> c3 cc cc cc cc 0f 1f 44 00 00 90 90 90 90 90 90 90 90 90 90 90 [Sun Dec 15 18:26:40 2024] RSP: 0000:ffffa793ef927ea8 EFLAGS: 00000202 [Sun Dec 15 18:26:40 2024] RAX: 0000000000000000 RBX: 0000000000000286 RCX: 0000000000000021 [Sun Dec 15 18:26:40 2024] RDX: 0000000000000001 RSI: 0000000000000012 RDI: ffff97e60a2b7000 [Sun Dec 15 18:26:40 2024] RBP: 000055b8d10d4748 R08: 0000000000000001 R09: 000000000000001e [Sun Dec 15 18:26:40 2024] R10: 000000000000000a R11: 00000000000065e7 R12: ffffa793ef927f58 [Sun Dec 15 18:26:40 2024] R13: ffff97e7e8e88b00 R14: 0000000000000000 R15: 0000000000000000 [Sun Dec 15 18:26:40 2024] FS: 00007f961cf3b740(0000) GS:ffff97f1fb400000(0000) knlGS:0000000000000000 [Sun Dec 15 18:26:40 2024] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [Sun Dec 15 18:26:40 2024] CR2: 000055b8d10d4748 CR3: 0000000c58542001 CR4: 00000000007706f0 [Sun Dec 15 18:26:40 2024] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [Sun Dec 15 18:26:40 2024] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [Sun Dec 15 18:26:40 2024] PKRU: 55555554 [Sun Dec 15 18:26:40 2024] Call Trace: [Sun Dec 15 18:26:40 2024] [Sun Dec 15 18:26:40 2024] ? nmi_cpu_backtrace+0x99/0x110 [Sun Dec 15 18:26:40 2024] ? nmi_cpu_backtrace_handler+0x11/0x20 [Sun Dec 15 18:26:40 2024] ? nmi_handle+0x65/0x170 [Sun Dec 15 18:26:40 2024] ? default_do_nmi+0x40/0x100 [Sun Dec 15 18:26:40 2024] ? exc_nmi+0x125/0x1a0 [Sun Dec 15 18:26:40 2024] ? end_repeat_nmi+0xf/0x60 [Sun Dec 15 18:26:40 2024] ? count_memcg_events.constprop.0+0x25/0x30 [Sun Dec 15 18:26:40 2024] ? count_memcg_events.constprop.0+0x25/0x30 [Sun Dec 15 18:26:40 2024] ? count_memcg_events.constprop.0+0x25/0x30 [Sun Dec 15 18:26:40 2024] [Sun Dec 15 18:26:40 2024] [Sun Dec 15 18:26:40 2024] handle_mm_fault+0x9d/0x360 [Sun Dec 15 18:26:40 2024] do_user_addr_fault+0x15b/0x690 [Sun Dec 15 18:26:40 2024] exc_page_fault+0x7f/0x180 [Sun Dec 15 18:26:40 2024] asm_exc_page_fault+0x26/0x30 [Sun Dec 15 18:26:40 2024] RIP: 0033:0x7f961cfdcdfd [Sun Dec 15 18:26:40 2024] Code: ff ff ff 49 8d 7c 24 10 48 8b 04 fa a8 0f 0f 85 89 01 00 00 48 89 c6 83 e9 01 48 c1 ee 0c 48 33 30 48 89 34 fa 66 42 89 0c 62 <48> c7 40 08 00 00 00 00 48 83 c4 18 5b 41 5c 41 5d 5d c3 48 89 de [Sun Dec 15 18:26:40 2024] RSP: 002b:00007ffe3e719740 EFLAGS: 00010206 [Sun Dec 15 18:26:40 2024] RAX: 000055b8d10d4740 RBX: 0000000000000020 RCX: 0000000000000002 [Sun Dec 15 18:26:40 2024] RDX: 000055b8d1015010 RSI: 000055b8d1034d30 RDI: 0000000000000011 [Sun Dec 15 18:26:40 2024] RBP: 00007ffe3e719770 R08: 000055b8d1015010 R09: 0000000000000007 [Sun Dec 15 18:26:40 2024] R10: 000055b8d10d3350 R11: 596e9e3668f839c2 R12: 0000000000000001 [Sun Dec 15 18:26:40 2024] R13: ffffffffffffffc0 R14: 000055b8d10df3b0 R15: 000055b8d1034820 [Sun Dec 15 18:26:40 2024] [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 63 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 50 skipped: idling at intel_idle+0x71/0xe0 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 27 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 46 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 67 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 9 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 34 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 70 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 11 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 47 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 19 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 13 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 1 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 57 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 38 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 20 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 5 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 39 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 29 skipped: idling at intel_idle+0x71/0xe0 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 65 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 51 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 30 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 66 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 16 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 55 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 3 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 62 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 26 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 14 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 58 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 37 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 7 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 33 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 10 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 52 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 44 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 28 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 64 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 4 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 59 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 23 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 48 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 54 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 8 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 56 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 17 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 31 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 43 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 24 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 40 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 60 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 53 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 22 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 41 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 35 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 71 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 49 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 69 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 45 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 32 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 12 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 61 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 25 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 15 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 68 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 21 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 42 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] NMI backtrace for cpu 6 skipped: idling at intel_idle_ibrs+0xa1/0x140 [Sun Dec 15 18:26:40 2024] sysrq: Show Memory [Sun Dec 15 18:26:41 2024] Mem-Info: [Sun Dec 15 18:26:41 2024] active_anon:1916621 inactive_anon:0 isolated_anon:0 active_file:97376 inactive_file:922665 isolated_file:0 unevictable:768 dirty:0 writeback:0 slab_reclaimable:49204 slab_unreclaimable:101935 mapped:37456 shmem:1792394 pagetables:2956 sec_pagetables:0 bounce:0 kernel_misc_reclaimable:0 free:19757851 free_pcp:80792 free_cma:0 [Sun Dec 15 18:26:41 2024] Node 0 active_anon:2987520kB inactive_anon:0kB active_file:255724kB inactive_file:106720kB unevictable:3072kB isolated(anon):0kB isolated(file):0kB mapped:101264kB dirty:0kB writeback:0kB shmem:2710320kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:9576kB pagetables:4024kB sec_pagetables:0kB all_unreclaimable? no [Sun Dec 15 18:26:41 2024] Node 1 active_anon:4675272kB inactive_anon:0kB active_file:133780kB inactive_file:3583940kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:48580kB dirty:0kB writeback:0kB shmem:4459256kB shmem_thp:0kB shmem_pmdmapped:0kB anon_thp:0kB writeback_tmp:0kB kernel_stack:6712kB pagetables:4592kB sec_pagetables:0kB all_unreclaimable? no [Sun Dec 15 18:26:41 2024] Node 0 DMA free:11264kB boost:0kB min:12kB low:24kB high:36kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15980kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [Sun Dec 15 18:26:41 2024] lowmem_reserve[]: 0 1663 46915 46915 46915 [Sun Dec 15 18:26:41 2024] Node 0 DMA32 free:1699236kB boost:0kB min:1664kB low:3364kB high:5064kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:1768868kB managed:1703332kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [Sun Dec 15 18:26:41 2024] lowmem_reserve[]: 0 0 45251 45251 45251 [Sun Dec 15 18:26:41 2024] Node 0 Normal free:42189300kB boost:0kB min:45284kB low:91620kB high:137956kB reserved_highatomic:0KB active_anon:2987152kB inactive_anon:0kB active_file:255724kB inactive_file:106720kB unevictable:3072kB writepending:0kB present:47173632kB managed:46346152kB mlocked:0kB bounce:0kB free_pcp:133184kB local_pcp:15176kB free_cma:0kB [Sun Dec 15 18:26:41 2024] lowmem_reserve[]: 0 0 0 0 0 [Sun Dec 15 18:26:41 2024] Node 1 Normal free:35144120kB boost:0kB min:43144kB low:87292kB high:131440kB reserved_highatomic:0KB active_anon:4673968kB inactive_anon:0kB active_file:133780kB inactive_file:3583940kB unevictable:0kB writepending:0kB present:45088768kB managed:44220548kB mlocked:0kB bounce:0kB free_pcp:179464kB local_pcp:668kB free_cma:0kB [Sun Dec 15 18:26:41 2024] lowmem_reserve[]: 0 0 0 0 0 [Sun Dec 15 18:26:41 2024] Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 2*4096kB (M) = 11264kB [Sun Dec 15 18:26:41 2024] Node 0 DMA32: 9*4kB (UM) 12*8kB (UM) 14*16kB (UM) 10*32kB (UM) 12*64kB (UM) 10*128kB (UM) 9*256kB (UM) 7*512kB (UM) 7*1024kB (UM) 12*2048kB (M) 405*4096kB (M) = 1699236kB [Sun Dec 15 18:26:41 2024] Node 0 Normal: 53083*4kB (UME) 42282*8kB (UME) 18589*16kB (UME) 13543*32kB (UME) 4822*64kB (UME) 1715*128kB (UME) 617*256kB (UME) 185*512kB (UM) 103*1024kB (UM) 56*2048kB (UME) 9743*4096kB (UM) = 42189676kB [Sun Dec 15 18:26:41 2024] Node 1 Normal: 34572*4kB (UME) 44713*8kB (UM) 31513*16kB (UME) 14582*32kB (UME) 5537*64kB (UME) 2661*128kB (UME) 760*256kB (UME) 256*512kB (UME) 90*1024kB (UME) 84*2048kB (UME) 7909*4096kB (UM) = 35146888kB [Sun Dec 15 18:26:41 2024] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB [Sun Dec 15 18:26:41 2024] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [Sun Dec 15 18:26:41 2024] Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB [Sun Dec 15 18:26:41 2024] Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB [Sun Dec 15 18:26:41 2024] 2812436 total pagecache pages [Sun Dec 15 18:26:41 2024] 0 pages in swap cache [Sun Dec 15 18:26:41 2024] Free swap = 8388604kB [Sun Dec 15 18:26:41 2024] Total swap = 8388604kB [Sun Dec 15 18:26:41 2024] 23511812 pages RAM [Sun Dec 15 18:26:41 2024] 0 pages HighMem/MovableOnly [Sun Dec 15 18:26:41 2024] 440464 pages reserved [Sun Dec 15 18:26:41 2024] 0 pages cma reserved [Sun Dec 15 18:26:41 2024] 0 pages hwpoisoned [Sun Dec 15 18:26:41 2024] sysrq: Show Blocked State [Sun Dec 15 18:26:41 2024] task:ipmitool state:D stack:0 pid:999593 tgid:999593 ppid:980955 flags:0x00000002 [Sun Dec 15 18:26:41 2024] Call Trace: [Sun Dec 15 18:26:41 2024] [Sun Dec 15 18:26:41 2024] __schedule+0x413/0x1720 [Sun Dec 15 18:26:41 2024] ? __schedule+0x41b/0x1720 [Sun Dec 15 18:26:41 2024] schedule+0x32/0xd0 [Sun Dec 15 18:26:41 2024] schedule_timeout+0x151/0x160 [Sun Dec 15 18:26:41 2024] wait_for_completion+0x8a/0x160 [Sun Dec 15 18:26:41 2024] __wait_rcu_gp+0x15b/0x170 [Sun Dec 15 18:26:41 2024] synchronize_rcu+0x133/0x150 [Sun Dec 15 18:26:41 2024] ? __pfx_call_rcu_hurry+0x10/0x10 [Sun Dec 15 18:26:41 2024] ? __pfx_wakeme_after_rcu+0x10/0x10 [Sun Dec 15 18:26:41 2024] _ipmi_destroy_user+0x2aa/0x320 [ipmi_msghandler] [Sun Dec 15 18:26:41 2024] ipmi_destroy_user+0x12/0x60 [ipmi_msghandler] [Sun Dec 15 18:26:41 2024] ipmi_release+0x1f/0x60 [ipmi_devintf] [Sun Dec 15 18:26:41 2024] __fput+0x97/0x2c0 [Sun Dec 15 18:26:41 2024] __x64_sys_close+0x3d/0x80 [Sun Dec 15 18:26:41 2024] do_syscall_64+0x83/0x170 [Sun Dec 15 18:26:41 2024] ? do_syscall_64+0x90/0x170 [Sun Dec 15 18:26:41 2024] ? __check_object_size+0x53/0x2e0 [Sun Dec 15 18:26:41 2024] ? __mod_memcg_lruvec_state+0xc5/0x170 [Sun Dec 15 18:26:41 2024] ? xas_find+0x75/0x1d0 [Sun Dec 15 18:26:41 2024] ? next_uptodate_folio+0xa5/0x2e0 [Sun Dec 15 18:26:41 2024] ? filemap_map_pages+0x474/0x550 [Sun Dec 15 18:26:41 2024] ? do_fault+0x246/0x490 [Sun Dec 15 18:26:41 2024] ? __handle_mm_fault+0x827/0xe40 [Sun Dec 15 18:26:41 2024] ? __count_memcg_events+0x69/0x100 [Sun Dec 15 18:26:41 2024] ? count_memcg_events.constprop.0+0x1a/0x30 [Sun Dec 15 18:26:41 2024] ? handle_mm_fault+0xa2/0x360 [Sun Dec 15 18:26:41 2024] ? do_user_addr_fault+0x304/0x690 [Sun Dec 15 18:26:41 2024] ? exc_page_fault+0x7f/0x180 [Sun Dec 15 18:26:41 2024] entry_SYSCALL_64_after_hwframe+0x78/0x80 [Sun Dec 15 18:26:41 2024] RIP: 0033:0x7f3cce540804 [Sun Dec 15 18:26:41 2024] RSP: 002b:00007fff4a108e88 EFLAGS: 00000202 ORIG_RAX: 0000000000000003 [Sun Dec 15 18:26:41 2024] RAX: ffffffffffffffda RBX: 00005573762609e0 RCX: 00007f3cce540804 [Sun Dec 15 18:26:41 2024] RDX: 0000000000000003 RSI: 0000557376e7b9b0 RDI: 0000000000000003 [Sun Dec 15 18:26:41 2024] RBP: 00007fff4a108ea0 R08: 0000557621d0d72b R09: 0000000000000007 [Sun Dec 15 18:26:41 2024] R10: 0000557376e7b9c0 R11: 0000000000000202 R12: 0000000000000000 [Sun Dec 15 18:26:41 2024] R13: 0000000000000000 R14: 00005573762609e0 R15: 0000000000000007 [Sun Dec 15 18:26:41 2024]