[2024-11-28 11:38:33.267049] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 23.11.0 initialization... [2024-11-28 11:38:33.267307] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1268022 ] Set seed to 1732790313260410 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-28 11:38:33.385897] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-28 11:38:33.589172] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-28 11:38:33.589223] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1268022' to capture a snapshot of events at runtime. [2024-11-28 11:38:33.589233] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1268022 for offline analysis/debug. [2024-11-28 11:38:33.589281] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Thu Nov 28 11:38:42 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/28-11:38:45 ... thread 0: (1000000,1000000) ops and (355298.1,355298.1) ops/second in (2.814538,2.814538) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/9 630.01 MB 6.3 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 352.7 2.32 2.08 12 0.193 0 0 0.0 0.0 L2 3/2 237.12 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.0 1.8 252.7 252.6 1.38 1.33 2 0.690 354K 177 0.0 0.0 L3 11/0 944.90 MB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.60 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1040/0 85.57 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1180/11 96.94 GB 0.0 0.3 0.2 0.2 1.1 1.0 0.0 1.4 94.2 315.4 3.70 3.41 14 0.264 354K 177 0.0 0.0 Int 0/0 0.00 KB 0.0 0.3 0.2 0.2 1.1 1.0 0.0 1.4 94.2 315.4 3.70 3.41 14 0.264 354K 177 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 0.0 0.0 252.7 252.6 1.38 1.33 2 0.690 354K 177 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 352.7 2.32 2.08 12 0.193 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.0 total, 2.8 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.14 GB write, 385.80 MB/s write, 0.34 GB read, 115.21 MB/s read, 3.7 seconds Interval compaction: 1.14 GB write, 414.50 MB/s write, 0.34 GB read, 123.78 MB/s read, 3.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 1, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 1, interval: 1 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 123138 Average: 2.0457 StdDev: 15.28 Min: 1 Median: 1.3580 Max: 3613 Percentiles: P50: 1.36 P75: 1.77 P99: 5.46 P99.9: 9.90 P99.99: 26.12 ------------------------------------------------------ [ 0, 1 ] 34602 28.100% 28.100% ###### ( 1, 2 ] 75336 61.180% 89.280% ############ ( 2, 3 ] 7313 5.939% 95.219% # ( 3, 4 ] 3032 2.462% 97.681% ( 4, 6 ] 2223 1.805% 99.487% ( 6, 10 ] 524 0.426% 99.912% ( 10, 15 ] 80 0.065% 99.977% ( 15, 22 ] 17 0.014% 99.991% ( 22, 34 ] 2 0.002% 99.993% ( 51, 76 ] 1 0.001% 99.994% ( 110, 170 ] 5 0.004% 99.998% ( 170, 250 ] 7 0.006% 100.003% ( 250, 380 ] 4 0.003% 100.006% ( 380, 580 ] 3 0.002% 100.009% ( 580, 870 ] 2 0.002% 100.011% ( 870, 1300 ] 1 0.001% 100.011% ( 1900, 2900 ] 2 0.002% 100.013% ( 2900, 4400 ] 1 0.001% 100.014% ** Level 2 read latency histogram (micros): Count: 70017 Average: 2.2147 StdDev: 22.40 Min: 1 Median: 1.3392 Max: 4485 Percentiles: P50: 1.34 P75: 1.75 P99: 5.93 P99.9: 58.97 P99.99: 539.93 ------------------------------------------------------ [ 0, 1 ] 20625 29.457% 29.457% ###### ( 1, 2 ] 42407 60.567% 90.024% ############ ( 2, 3 ] 3437 4.909% 94.933% # ( 3, 4 ] 1789 2.555% 97.488% # ( 4, 6 ] 1100 1.571% 99.059% ( 6, 10 ] 455 0.650% 99.709% ( 10, 15 ] 103 0.147% 99.856% ( 15, 22 ] 16 0.023% 99.879% ( 34, 51 ] 1 0.001% 99.880% ( 51, 76 ] 47 0.067% 99.947% ( 76, 110 ] 11 0.016% 99.963% ( 110, 170 ] 5 0.007% 99.970% ( 170, 250 ] 3 0.004% 99.974% ( 250, 380 ] 8 0.011% 99.986% ( 380, 580 ] 5 0.007% 99.993% ( 580, 870 ] 2 0.003% 99.996% ( 870, 1300 ] 1 0.001% 99.997% ( 1300, 1900 ] 1 0.001% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ( 4400, 6600 ] 1 0.001% 100.001% ** Level 3 read latency histogram (micros): Count: 55 Average: 3174.0000 StdDev: 2415.79 Min: 99 Median: 2400.0000 Max: 9447 Percentiles: P50: 2400.00 P75: 5018.75 P99: 9447.00 P99.9: 9447.00 P99.99: 9447.00 ------------------------------------------------------ ( 76, 110 ] 1 1.818% 1.818% ( 110, 170 ] 1 1.818% 3.636% ( 170, 250 ] 2 3.636% 7.273% # ( 250, 380 ] 2 3.636% 10.909% # ( 380, 580 ] 1 1.818% 12.727% ( 580, 870 ] 2 3.636% 16.364% # ( 870, 1300 ] 5 9.091% 25.455% ## ( 1300, 1900 ] 10 18.182% 43.636% #### ( 1900, 2900 ] 7 12.727% 56.364% ### ( 2900, 4400 ] 8 14.545% 70.909% ### ( 4400, 6600 ] 8 14.545% 85.455% ### ( 6600, 9900 ] 8 14.545% 100.000% ### ** Level 4 read latency histogram (micros): Count: 20 Average: 3940.3500 StdDev: 2074.98 Min: 966 Median: 3275.0000 Max: 7780 Percentiles: P50: 3275.00 P75: 5866.67 P99: 7780.00 P99.9: 7780.00 P99.99: 7780.00 ------------------------------------------------------ ( 870, 1300 ] 1 5.000% 5.000% # ( 1300, 1900 ] 3 15.000% 20.000% ### ( 1900, 2900 ] 5 25.000% 45.000% ##### ( 2900, 4400 ] 4 20.000% 65.000% #### ( 4400, 6600 ] 3 15.000% 80.000% ### ( 6600, 9900 ] 4 20.000% 100.000% #### ** DB Stats ** Uptime(secs): 3.0 total, 2.8 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 325.22 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.204 H:M:S, 6.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 349.37 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.204 H:M:S, 7.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/28-11:38:48 ... thread 0: (1000000,2000000) ops and (325622.9,339813.9) ops/second in (3.071037,5.885575) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/4 1.29 GB 100.7 0.8 0.0 0.8 2.6 1.8 0.0 1.4 98.6 318.4 8.31 7.69 29 0.286 831K 894 0.0 0.0 L1 1/0 3.15 MB 0.3 0.3 0.3 0.0 0.3 0.3 0.2 1.0 238.6 238.3 1.32 1.28 1 1.320 319K 421 0.0 0.0 L2 2/1 146.50 MB 1.0 0.3 0.2 0.2 0.3 0.2 0.3 1.8 252.7 252.6 1.38 1.33 2 0.690 354K 177 0.0 0.0 L3 20/11 1.15 GB 0.2 0.2 0.2 0.0 0.2 0.2 0.2 1.0 218.6 218.6 0.85 0.79 1 0.855 189K 0 0.0 0.0 L4 173/0 9.76 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.0 1.0 125.2 125.2 1.34 0.90 1 1.341 170K 0 0.0 0.0 L5 1040/0 85.57 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1246/16 97.92 GB 0.0 1.8 0.8 1.0 3.6 2.6 0.7 2.0 139.2 277.4 13.20 11.98 34 0.388 1865K 1492 0.0 0.0 Int 0/0 0.00 KB 0.0 1.8 0.8 1.0 3.6 2.6 0.7 2.0 139.2 277.4 13.20 11.98 34 0.388 1865K 1492 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.8 0.8 1.0 1.8 0.8 0.0 0.0 229.0 228.8 8.02 7.37 8 1.003 1865K 1492 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 352.8 5.18 4.61 26 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.1 total, 5.9 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.58 GB write, 600.72 MB/s write, 1.79 GB read, 301.34 MB/s read, 13.2 seconds Interval compaction: 3.58 GB write, 622.11 MB/s write, 1.79 GB read, 312.06 MB/s read, 13.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 6, interval: 6 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 378930 Average: 2.2334 StdDev: 12.60 Min: 1 Median: 1.4964 Max: 3613 Percentiles: P50: 1.50 P75: 1.88 P99: 5.72 P99.9: 10.30 P99.99: 16.69 ------------------------------------------------------ [ 0, 1 ] 66958 17.670% 17.670% #### ( 1, 2 ] 246795 65.129% 82.800% ############# ( 2, 3 ] 38372 10.126% 92.926% ## ( 3, 4 ] 14186 3.744% 96.670% # ( 4, 6 ] 10262 2.708% 99.378% # ( 6, 10 ] 1957 0.516% 99.894% ( 10, 15 ] 351 0.093% 99.987% ( 15, 22 ] 46 0.012% 99.999% ( 22, 34 ] 5 0.001% 100.001% ( 34, 51 ] 3 0.001% 100.001% ( 51, 76 ] 2 0.001% 100.002% ( 110, 170 ] 6 0.002% 100.003% ( 170, 250 ] 16 0.004% 100.008% ( 250, 380 ] 8 0.002% 100.010% ( 380, 580 ] 8 0.002% 100.012% ( 580, 870 ] 7 0.002% 100.014% ( 870, 1300 ] 4 0.001% 100.015% ( 1300, 1900 ] 4 0.001% 100.016% ( 1900, 2900 ] 5 0.001% 100.017% ( 2900, 4400 ] 1 0.000% 100.017% ** Level 1 read latency histogram (micros): Count: 20 Average: 203.4500 StdDev: 416.20 Min: 1 Median: 1.5000 Max: 1579 Percentiles: P50: 1.50 P75: 250.00 P99: 1579.00 P99.9: 1579.00 P99.99: 1579.00 ------------------------------------------------------ [ 0, 1 ] 8 40.000% 40.000% ######## ( 1, 2 ] 4 20.000% 60.000% #### ( 2, 3 ] 1 5.000% 65.000% # ( 4, 6 ] 1 5.000% 70.000% # ( 170, 250 ] 1 5.000% 75.000% # ( 250, 380 ] 2 10.000% 85.000% ## ( 380, 580 ] 1 5.000% 90.000% # ( 870, 1300 ] 1 5.000% 95.000% # ( 1300, 1900 ] 1 5.000% 100.000% # ** Level 2 read latency histogram (micros): Count: 88334 Average: 2.2413 StdDev: 19.95 Min: 1 Median: 1.3994 Max: 4485 Percentiles: P50: 1.40 P75: 1.80 P99: 5.93 P99.9: 24.66 P99.99: 466.66 ------------------------------------------------------ [ 0, 1 ] 22406 25.365% 25.365% ##### ( 1, 2 ] 54482 61.677% 87.042% ############ ( 2, 3 ] 5936 6.720% 93.762% # ( 3, 4 ] 2851 3.228% 96.990% # ( 4, 6 ] 1843 2.086% 99.076% ( 6, 10 ] 580 0.657% 99.733% ( 10, 15 ] 128 0.145% 99.878% ( 15, 22 ] 19 0.022% 99.899% ( 22, 34 ] 3 0.003% 99.903% ( 34, 51 ] 1 0.001% 99.904% ( 51, 76 ] 47 0.053% 99.957% ( 76, 110 ] 11 0.012% 99.969% ( 110, 170 ] 5 0.006% 99.975% ( 170, 250 ] 3 0.003% 99.978% ( 250, 380 ] 8 0.009% 99.988% ( 380, 580 ] 5 0.006% 99.993% ( 580, 870 ] 2 0.002% 99.995% ( 870, 1300 ] 1 0.001% 99.997% ( 1300, 1900 ] 1 0.001% 99.998% ( 1900, 2900 ] 1 0.001% 99.999% ( 4400, 6600 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 204787 Average: 3.8306 StdDev: 67.38 Min: 1 Median: 1.5967 Max: 9447 Percentiles: P50: 1.60 P75: 1.99 P99: 9.05 P99.9: 171.08 P99.99: 776.06 ------------------------------------------------------ [ 0, 1 ] 25138 12.275% 12.275% ## ( 1, 2 ] 129482 63.228% 75.503% ############# ( 2, 3 ] 28702 14.016% 89.518% ### ( 3, 4 ] 9358 4.570% 94.088% # ( 4, 6 ] 7453 3.639% 97.727% # ( 6, 10 ] 3421 1.671% 99.398% ( 10, 15 ] 581 0.284% 99.682% ( 15, 22 ] 71 0.035% 99.716% ( 22, 34 ] 7 0.003% 99.720% ( 34, 51 ] 1 0.000% 99.720% ( 76, 110 ] 117 0.057% 99.777% ( 110, 170 ] 250 0.122% 99.899% ( 170, 250 ] 90 0.044% 99.943% ( 250, 380 ] 32 0.016% 99.959% ( 380, 580 ] 50 0.024% 99.983% ( 580, 870 ] 20 0.010% 99.993% ( 870, 1300 ] 15 0.007% 100.000% ( 1300, 1900 ] 13 0.006% 100.007% ( 1900, 2900 ] 8 0.004% 100.011% ( 2900, 4400 ] 8 0.004% 100.015% ( 4400, 6600 ] 8 0.004% 100.019% ( 6600, 9900 ] 8 0.004% 100.022% ** Level 4 read latency histogram (micros): Count: 305 Average: 264.4426 StdDev: 1109.75 Min: 1 Median: 1.6808 Max: 7780 Percentiles: P50: 1.68 P75: 5.05 P99: 7383.75 P99.9: 7780.00 P99.99: 7780.00 ------------------------------------------------------ [ 0, 1 ] 64 20.984% 20.984% #### ( 1, 2 ] 130 42.623% 63.607% ######### ( 2, 3 ] 26 8.525% 72.131% ## ( 3, 4 ] 3 0.984% 73.115% ( 4, 6 ] 11 3.607% 76.721% # ( 6, 10 ] 41 13.443% 90.164% ### ( 10, 15 ] 7 2.295% 92.459% ( 250, 380 ] 2 0.656% 93.115% ( 380, 580 ] 1 0.328% 93.443% ( 870, 1300 ] 1 0.328% 93.770% ( 1300, 1900 ] 3 0.984% 94.754% ( 1900, 2900 ] 5 1.639% 96.393% ( 2900, 4400 ] 4 1.311% 97.705% ( 4400, 6600 ] 3 0.984% 98.689% ( 6600, 9900 ] 4 1.311% 100.000% ** DB Stats ** Uptime(secs): 6.1 total, 3.1 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 322.83 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.588 H:M:S, 9.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 320.48 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.383 H:M:S, 12.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/28-11:38:51 ... thread 0: (1000000,3000000) ops and (332152.4,337221.1) ops/second in (3.010666,8.896241) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/4 2.27 GB 201.4 1.9 0.0 1.9 4.7 2.8 0.0 1.7 126.3 309.4 15.47 14.44 47 0.329 1982K 2096 0.0 0.0 L1 1/0 3.15 MB 0.3 0.3 0.3 0.0 0.3 0.3 0.2 1.0 238.6 238.3 1.32 1.28 1 1.320 319K 421 0.0 0.0 L2 1/0 96.26 MB 1.0 0.3 0.2 0.2 0.3 0.2 0.3 1.8 252.7 252.6 1.38 1.33 2 0.690 354K 177 0.0 0.0 L3 21/0 969.92 MB 1.0 1.2 0.2 0.9 1.2 0.2 0.2 5.0 238.5 238.1 4.96 4.57 2 2.478 1200K 1928 0.0 0.0 L4 173/0 9.70 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.3 1.0 125.2 125.2 1.34 0.90 1 1.341 170K 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1252/4 98.90 GB 0.0 3.9 0.9 3.0 6.6 3.7 1.3 2.4 162.1 277.8 24.46 22.52 53 0.462 4027K 4622 0.0 0.0 Int 0/0 0.00 KB 0.0 3.9 0.9 3.0 6.6 3.7 1.3 2.4 162.1 277.8 24.46 22.52 53 0.462 4027K 4622 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.9 0.9 3.0 3.9 0.9 0.0 0.0 240.5 240.3 16.49 15.43 13 1.268 4027K 4622 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 355.5 7.97 7.09 40 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.1 total, 8.9 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.64 GB write, 746.23 MB/s write, 3.87 GB read, 435.49 MB/s read, 24.5 seconds Interval compaction: 6.64 GB write, 763.80 MB/s write, 3.87 GB read, 445.75 MB/s read, 24.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 10, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 10, interval: 10 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 642285 Average: 2.2037 StdDev: 12.86 Min: 1 Median: 1.4821 Max: 3613 Percentiles: P50: 1.48 P75: 1.86 P99: 5.65 P99.9: 9.95 P99.99: 14.94 ------------------------------------------------------ [ 0, 1 ] 117681 18.322% 18.322% #### ( 1, 2 ] 421990 65.701% 84.024% ############# ( 2, 3 ] 61065 9.507% 93.531% ## ( 3, 4 ] 22463 3.497% 97.028% # ( 4, 6 ] 15394 2.397% 99.425% ( 6, 10 ] 3092 0.481% 99.907% ( 10, 15 ] 542 0.084% 99.991% ( 15, 22 ] 66 0.010% 100.001% ( 22, 34 ] 10 0.002% 100.003% ( 34, 51 ] 6 0.001% 100.004% ( 51, 76 ] 2 0.000% 100.004% ( 76, 110 ] 2 0.000% 100.004% ( 110, 170 ] 10 0.002% 100.006% ( 170, 250 ] 22 0.003% 100.009% ( 250, 380 ] 10 0.002% 100.011% ( 380, 580 ] 16 0.002% 100.013% ( 580, 870 ] 12 0.002% 100.015% ( 870, 1300 ] 10 0.002% 100.017% ( 1300, 1900 ] 5 0.001% 100.018% ( 1900, 2900 ] 7 0.001% 100.019% ( 2900, 4400 ] 3 0.000% 100.019% ** Level 1 read latency histogram (micros): Count: 20 Average: 203.4500 StdDev: 416.20 Min: 1 Median: 1.5000 Max: 1579 Percentiles: P50: 1.50 P75: 250.00 P99: 1579.00 P99.9: 1579.00 P99.99: 1579.00 ------------------------------------------------------ [ 0, 1 ] 8 40.000% 40.000% ######## ( 1, 2 ] 4 20.000% 60.000% #### ( 2, 3 ] 1 5.000% 65.000% # ( 4, 6 ] 1 5.000% 70.000% # ( 170, 250 ] 1 5.000% 75.000% # ( 250, 380 ] 2 10.000% 85.000% ## ( 380, 580 ] 1 5.000% 90.000% # ( 870, 1300 ] 1 5.000% 95.000% # ( 1300, 1900 ] 1 5.000% 100.000% # ** Level 2 read latency histogram (micros): Count: 100768 Average: 2.2931 StdDev: 19.38 Min: 1 Median: 1.4128 Max: 4485 Percentiles: P50: 1.41 P75: 1.81 P99: 6.16 P99.9: 58.57 P99.99: 492.09 ------------------------------------------------------ [ 0, 1 ] 24405 24.219% 24.219% ##### ( 1, 2 ] 62938 62.458% 86.677% ############ ( 2, 3 ] 7093 7.039% 93.716% # ( 3, 4 ] 3211 3.187% 96.903% # ( 4, 6 ] 2082 2.066% 98.969% ( 6, 10 ] 760 0.754% 99.723% ( 10, 15 ] 140 0.139% 99.862% ( 15, 22 ] 20 0.020% 99.882% ( 22, 34 ] 3 0.003% 99.885% ( 34, 51 ] 1 0.001% 99.886% ( 51, 76 ] 47 0.047% 99.933% ( 76, 110 ] 11 0.011% 99.943% ( 110, 170 ] 26 0.026% 99.969% ( 170, 250 ] 8 0.008% 99.977% ( 250, 380 ] 9 0.009% 99.986% ( 380, 580 ] 7 0.007% 99.993% ( 580, 870 ] 2 0.002% 99.995% ( 870, 1300 ] 2 0.002% 99.997% ( 1300, 1900 ] 1 0.001% 99.998% ( 1900, 2900 ] 1 0.001% 99.999% ( 4400, 6600 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 282619 Average: 3.6189 StdDev: 58.08 Min: 1 Median: 1.5829 Max: 9447 Percentiles: P50: 1.58 P75: 1.97 P99: 9.03 P99.9: 192.59 P99.99: 786.26 ------------------------------------------------------ [ 0, 1 ] 35345 12.506% 12.506% ### ( 1, 2 ] 181777 64.319% 76.825% ############# ( 2, 3 ] 37299 13.198% 90.023% ### ( 3, 4 ] 12424 4.396% 94.419% # ( 4, 6 ] 9333 3.302% 97.721% # ( 6, 10 ] 4779 1.691% 99.412% ( 10, 15 ] 721 0.255% 99.667% ( 15, 22 ] 82 0.029% 99.696% ( 22, 34 ] 8 0.003% 99.699% ( 34, 51 ] 1 0.000% 99.699% ( 76, 110 ] 179 0.063% 99.763% ( 110, 170 ] 348 0.123% 99.886% ( 170, 250 ] 143 0.051% 99.936% ( 250, 380 ] 58 0.021% 99.957% ( 380, 580 ] 66 0.023% 99.980% ( 580, 870 ] 39 0.014% 99.994% ( 870, 1300 ] 17 0.006% 100.000% ( 1300, 1900 ] 14 0.005% 100.005% ( 1900, 2900 ] 8 0.003% 100.008% ( 2900, 4400 ] 8 0.003% 100.011% ( 4400, 6600 ] 8 0.003% 100.013% ( 6600, 9900 ] 8 0.003% 100.016% ** Level 4 read latency histogram (micros): Count: 305 Average: 264.4426 StdDev: 1109.75 Min: 1 Median: 1.6808 Max: 7780 Percentiles: P50: 1.68 P75: 5.05 P99: 7383.75 P99.9: 7780.00 P99.99: 7780.00 ------------------------------------------------------ [ 0, 1 ] 64 20.984% 20.984% #### ( 1, 2 ] 130 42.623% 63.607% ######### ( 2, 3 ] 26 8.525% 72.131% ## ( 3, 4 ] 3 0.984% 73.115% ( 4, 6 ] 11 3.607% 76.721% # ( 6, 10 ] 41 13.443% 90.164% ### ( 10, 15 ] 7 2.295% 92.459% ( 250, 380 ] 2 0.656% 93.115% ( 380, 580 ] 1 0.328% 93.443% ( 870, 1300 ] 1 0.328% 93.770% ( 1300, 1900 ] 3 0.984% 94.754% ( 1900, 2900 ] 5 1.639% 96.393% ( 2900, 4400 ] 4 1.311% 97.705% ( 4400, 6600 ] 3 0.984% 98.689% ( 6600, 9900 ] 4 1.311% 100.000% ** Level 5 read latency histogram (micros): Count: 20 Average: 1044.8000 StdDev: 1176.40 Min: 164 Median: 380.0000 Max: 3769 Percentiles: P50: 380.00 P75: 1600.00 P99: 3769.00 P99.9: 3769.00 P99.99: 3769.00 ------------------------------------------------------ ( 110, 170 ] 2 10.000% 10.000% ## ( 170, 250 ] 3 15.000% 25.000% ### ( 250, 380 ] 5 25.000% 50.000% ##### ( 380, 580 ] 2 10.000% 60.000% ## ( 580, 870 ] 1 5.000% 65.000% # ( 870, 1300 ] 1 5.000% 70.000% # ( 1300, 1900 ] 2 10.000% 80.000% ## ( 1900, 2900 ] 1 5.000% 85.000% # ( 2900, 4400 ] 3 15.000% 100.000% ### ** DB Stats ** Uptime(secs): 9.1 total, 3.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 324.18 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.952 H:M:S, 10.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 326.90 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.365 H:M:S, 12.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/28-11:38:54 ... thread 0: (1000000,4000000) ops and (334871.5,336630.6) ops/second in (2.986220,11.882461) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/4 3.26 GB 308.3 2.8 0.0 2.8 6.5 3.7 0.0 1.7 131.3 309.0 21.60 20.20 64 0.337 2878K 3041 0.0 0.0 L1 1/0 3.15 MB 0.3 0.3 0.3 0.0 0.3 0.3 0.2 1.0 238.6 238.3 1.32 1.28 1 1.320 319K 421 0.0 0.0 L2 1/0 96.26 MB 1.0 0.3 0.2 0.2 0.3 0.2 0.3 1.8 252.7 252.6 1.38 1.33 2 0.690 354K 177 0.0 0.0 L3 21/0 969.92 MB 1.0 1.2 0.2 0.9 1.2 0.2 0.2 5.0 238.5 238.1 4.96 4.57 2 2.478 1200K 1928 0.0 0.0 L4 173/0 9.70 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.3 1.0 125.2 125.2 1.34 0.90 1 1.341 170K 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1257/4 99.88 GB 0.0 4.7 0.9 3.8 8.5 4.6 1.3 2.3 158.5 283.9 30.59 28.28 70 0.437 4923K 5567 0.0 0.0 Int 0/0 0.00 KB 0.0 4.7 0.9 3.8 8.5 4.6 1.3 2.3 158.5 283.9 30.59 28.28 70 0.437 4923K 5567 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.7 0.9 3.8 4.7 0.9 0.0 0.0 244.1 243.8 19.86 18.73 16 1.241 4923K 5567 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 358.0 10.73 9.55 54 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.1 total, 11.9 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.48 GB write, 718.15 MB/s write, 4.73 GB read, 400.88 MB/s read, 30.6 seconds Interval compaction: 8.48 GB write, 730.81 MB/s write, 4.73 GB read, 407.95 MB/s read, 30.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 13, interval: 13 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 901782 Average: 2.1519 StdDev: 13.84 Min: 1 Median: 1.4506 Max: 3999 Percentiles: P50: 1.45 P75: 1.83 P99: 5.51 P99.9: 9.87 P99.99: 14.98 ------------------------------------------------------ [ 0, 1 ] 184845 20.498% 20.498% #### ( 1, 2 ] 590441 65.475% 85.973% ############# ( 2, 3 ] 75883 8.415% 94.387% ## ( 3, 4 ] 27824 3.085% 97.473% # ( 4, 6 ] 18224 2.021% 99.494% ( 6, 10 ] 3792 0.421% 99.914% ( 10, 15 ] 687 0.076% 99.990% ( 15, 22 ] 93 0.010% 100.001% ( 22, 34 ] 12 0.001% 100.002% ( 34, 51 ] 8 0.001% 100.003% ( 51, 76 ] 3 0.000% 100.003% ( 76, 110 ] 2 0.000% 100.004% ( 110, 170 ] 15 0.002% 100.005% ( 170, 250 ] 25 0.003% 100.008% ( 250, 380 ] 14 0.002% 100.010% ( 380, 580 ] 20 0.002% 100.012% ( 580, 870 ] 16 0.002% 100.014% ( 870, 1300 ] 17 0.002% 100.015% ( 1300, 1900 ] 8 0.001% 100.016% ( 1900, 2900 ] 8 0.001% 100.017% ( 2900, 4400 ] 6 0.001% 100.018% ** Level 1 read latency histogram (micros): Count: 20 Average: 203.4500 StdDev: 416.20 Min: 1 Median: 1.5000 Max: 1579 Percentiles: P50: 1.50 P75: 250.00 P99: 1579.00 P99.9: 1579.00 P99.99: 1579.00 ------------------------------------------------------ [ 0, 1 ] 8 40.000% 40.000% ######## ( 1, 2 ] 4 20.000% 60.000% #### ( 2, 3 ] 1 5.000% 65.000% # ( 4, 6 ] 1 5.000% 70.000% # ( 170, 250 ] 1 5.000% 75.000% # ( 250, 380 ] 2 10.000% 85.000% ## ( 380, 580 ] 1 5.000% 90.000% # ( 870, 1300 ] 1 5.000% 95.000% # ( 1300, 1900 ] 1 5.000% 100.000% # ** Level 2 read latency histogram (micros): Count: 100768 Average: 2.2931 StdDev: 19.38 Min: 1 Median: 1.4128 Max: 4485 Percentiles: P50: 1.41 P75: 1.81 P99: 6.16 P99.9: 58.57 P99.99: 492.09 ------------------------------------------------------ [ 0, 1 ] 24405 24.219% 24.219% ##### ( 1, 2 ] 62938 62.458% 86.677% ############ ( 2, 3 ] 7093 7.039% 93.716% # ( 3, 4 ] 3211 3.187% 96.903% # ( 4, 6 ] 2082 2.066% 98.969% ( 6, 10 ] 760 0.754% 99.723% ( 10, 15 ] 140 0.139% 99.862% ( 15, 22 ] 20 0.020% 99.882% ( 22, 34 ] 3 0.003% 99.885% ( 34, 51 ] 1 0.001% 99.886% ( 51, 76 ] 47 0.047% 99.933% ( 76, 110 ] 11 0.011% 99.943% ( 110, 170 ] 26 0.026% 99.969% ( 170, 250 ] 8 0.008% 99.977% ( 250, 380 ] 9 0.009% 99.986% ( 380, 580 ] 7 0.007% 99.993% ( 580, 870 ] 2 0.002% 99.995% ( 870, 1300 ] 2 0.002% 99.997% ( 1300, 1900 ] 1 0.001% 99.998% ( 1900, 2900 ] 1 0.001% 99.999% ( 4400, 6600 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 282619 Average: 3.6189 StdDev: 58.08 Min: 1 Median: 1.5829 Max: 9447 Percentiles: P50: 1.58 P75: 1.97 P99: 9.03 P99.9: 192.59 P99.99: 786.26 ------------------------------------------------------ [ 0, 1 ] 35345 12.506% 12.506% ### ( 1, 2 ] 181777 64.319% 76.825% ############# ( 2, 3 ] 37299 13.198% 90.023% ### ( 3, 4 ] 12424 4.396% 94.419% # ( 4, 6 ] 9333 3.302% 97.721% # ( 6, 10 ] 4779 1.691% 99.412% ( 10, 15 ] 721 0.255% 99.667% ( 15, 22 ] 82 0.029% 99.696% ( 22, 34 ] 8 0.003% 99.699% ( 34, 51 ] 1 0.000% 99.699% ( 76, 110 ] 179 0.063% 99.763% ( 110, 170 ] 348 0.123% 99.886% ( 170, 250 ] 143 0.051% 99.936% ( 250, 380 ] 58 0.021% 99.957% ( 380, 580 ] 66 0.023% 99.980% ( 580, 870 ] 39 0.014% 99.994% ( 870, 1300 ] 17 0.006% 100.000% ( 1300, 1900 ] 14 0.005% 100.005% ( 1900, 2900 ] 8 0.003% 100.008% ( 2900, 4400 ] 8 0.003% 100.011% ( 4400, 6600 ] 8 0.003% 100.013% ( 6600, 9900 ] 8 0.003% 100.016% ** Level 4 read latency histogram (micros): Count: 305 Average: 264.4426 StdDev: 1109.75 Min: 1 Median: 1.6808 Max: 7780 Percentiles: P50: 1.68 P75: 5.05 P99: 7383.75 P99.9: 7780.00 P99.99: 7780.00 ------------------------------------------------------ [ 0, 1 ] 64 20.984% 20.984% #### ( 1, 2 ] 130 42.623% 63.607% ######### ( 2, 3 ] 26 8.525% 72.131% ## ( 3, 4 ] 3 0.984% 73.115% ( 4, 6 ] 11 3.607% 76.721% # ( 6, 10 ] 41 13.443% 90.164% ### ( 10, 15 ] 7 2.295% 92.459% ( 250, 380 ] 2 0.656% 93.115% ( 380, 580 ] 1 0.328% 93.443% ( 870, 1300 ] 1 0.328% 93.770% ( 1300, 1900 ] 3 0.984% 94.754% ( 1900, 2900 ] 5 1.639% 96.393% ( 2900, 4400 ] 4 1.311% 97.705% ( 4400, 6600 ] 3 0.984% 98.689% ( 6600, 9900 ] 4 1.311% 100.000% ** Level 5 read latency histogram (micros): Count: 20 Average: 1044.8000 StdDev: 1176.40 Min: 164 Median: 380.0000 Max: 3769 Percentiles: P50: 380.00 P75: 1600.00 P99: 3769.00 P99.9: 3769.00 P99.99: 3769.00 ------------------------------------------------------ ( 110, 170 ] 2 10.000% 10.000% ## ( 170, 250 ] 3 15.000% 25.000% ### ( 250, 380 ] 5 25.000% 50.000% ##### ( 380, 580 ] 2 10.000% 60.000% ## ( 580, 870 ] 1 5.000% 65.000% # ( 870, 1300 ] 1 5.000% 70.000% # ( 1300, 1900 ] 2 10.000% 80.000% ## ( 1900, 2900 ] 1 5.000% 85.000% # ( 2900, 4400 ] 3 15.000% 100.000% ### ** DB Stats ** Uptime(secs): 12.1 total, 3.0 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 325.51 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.308 H:M:S, 10.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 329.57 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.356 H:M:S, 11.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/28-11:38:57 ... thread 0: (1000000,5000000) ops and (276270.0,322536.8) ops/second in (3.619647,15.502108) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/18 4.24 GB 0.0 4.1 0.0 4.1 8.8 4.7 0.0 1.9 141.6 306.7 29.36 27.56 85 0.345 4221K 4352 0.0 0.0 L1 1/1 3.15 MB 0.0 0.3 0.3 0.0 0.3 0.3 0.2 1.0 238.6 238.3 1.32 1.28 1 1.320 319K 421 0.0 0.0 L2 1/0 96.26 MB 1.0 0.3 0.2 0.2 0.3 0.2 0.3 1.8 252.7 252.6 1.38 1.33 2 0.690 354K 177 0.0 0.0 L3 21/0 969.92 MB 1.0 1.2 0.2 0.9 1.2 0.2 0.2 5.0 238.5 238.1 4.96 4.57 2 2.478 1200K 1928 0.0 0.0 L4 173/0 9.70 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.3 1.0 125.2 125.2 1.34 0.90 1 1.341 170K 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1258/19 100.87 GB 0.0 6.0 0.9 5.1 10.8 5.6 1.3 2.3 160.9 287.2 38.36 35.64 91 0.422 6266K 6878 0.0 0.0 Int 0/0 0.00 KB 0.0 6.0 0.9 5.1 10.8 5.6 1.3 2.3 160.9 287.2 38.36 35.64 91 0.422 6266K 6878 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.0 0.9 5.1 6.0 0.9 0.0 0.0 248.9 248.6 24.79 23.58 21 1.181 6266K 6878 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 357.6 13.56 12.06 70 0.194 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.7 total, 15.5 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.76 GB write, 700.97 MB/s write, 6.03 GB read, 392.73 MB/s read, 38.4 seconds Interval compaction: 10.76 GB write, 710.45 MB/s write, 6.03 GB read, 398.04 MB/s read, 38.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 7, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 7, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 7, total-stops: 13, interval: 20 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1187839 Average: 2.1323 StdDev: 13.40 Min: 1 Median: 1.4212 Max: 3999 Percentiles: P50: 1.42 P75: 1.81 P99: 5.44 P99.9: 10.87 P99.99: 106.14 ------------------------------------------------------ [ 0, 1 ] 269110 22.655% 22.655% ##### ( 1, 2 ] 771214 64.926% 87.581% ############# ( 2, 3 ] 87992 7.408% 94.989% # ( 3, 4 ] 32781 2.760% 97.749% # ( 4, 6 ] 20706 1.743% 99.492% ( 6, 10 ] 4693 0.395% 99.887% ( 10, 15 ] 896 0.075% 99.962% ( 15, 22 ] 127 0.011% 99.973% ( 22, 34 ] 17 0.001% 99.974% ( 34, 51 ] 10 0.001% 99.975% ( 51, 76 ] 4 0.000% 99.976% ( 76, 110 ] 192 0.016% 99.992% ( 110, 170 ] 134 0.011% 100.003% ( 170, 250 ] 44 0.004% 100.007% ( 250, 380 ] 19 0.002% 100.008% ( 380, 580 ] 30 0.003% 100.011% ( 580, 870 ] 28 0.002% 100.013% ( 870, 1300 ] 19 0.002% 100.015% ( 1300, 1900 ] 12 0.001% 100.016% ( 1900, 2900 ] 9 0.001% 100.017% ( 2900, 4400 ] 7 0.001% 100.017% ** Level 1 read latency histogram (micros): Count: 21 Average: 201.9048 StdDev: 406.23 Min: 1 Median: 1.6250 Max: 1579 Percentiles: P50: 1.62 P75: 240.00 P99: 1579.00 P99.9: 1579.00 P99.99: 1579.00 ------------------------------------------------------ [ 0, 1 ] 8 38.095% 38.095% ######## ( 1, 2 ] 4 19.048% 57.143% #### ( 2, 3 ] 1 4.762% 61.905% # ( 4, 6 ] 1 4.762% 66.667% # ( 170, 250 ] 2 9.524% 76.190% ## ( 250, 380 ] 2 9.524% 85.714% ## ( 380, 580 ] 1 4.762% 90.476% # ( 870, 1300 ] 1 4.762% 95.238% # ( 1300, 1900 ] 1 4.762% 100.000% # ** Level 2 read latency histogram (micros): Count: 100768 Average: 2.2931 StdDev: 19.38 Min: 1 Median: 1.4128 Max: 4485 Percentiles: P50: 1.41 P75: 1.81 P99: 6.16 P99.9: 58.57 P99.99: 492.09 ------------------------------------------------------ [ 0, 1 ] 24405 24.219% 24.219% ##### ( 1, 2 ] 62938 62.458% 86.677% ############ ( 2, 3 ] 7093 7.039% 93.716% # ( 3, 4 ] 3211 3.187% 96.903% # ( 4, 6 ] 2082 2.066% 98.969% ( 6, 10 ] 760 0.754% 99.723% ( 10, 15 ] 140 0.139% 99.862% ( 15, 22 ] 20 0.020% 99.882% ( 22, 34 ] 3 0.003% 99.885% ( 34, 51 ] 1 0.001% 99.886% ( 51, 76 ] 47 0.047% 99.933% ( 76, 110 ] 11 0.011% 99.943% ( 110, 170 ] 26 0.026% 99.969% ( 170, 250 ] 8 0.008% 99.977% ( 250, 380 ] 9 0.009% 99.986% ( 380, 580 ] 7 0.007% 99.993% ( 580, 870 ] 2 0.002% 99.995% ( 870, 1300 ] 2 0.002% 99.997% ( 1300, 1900 ] 1 0.001% 99.998% ( 1900, 2900 ] 1 0.001% 99.999% ( 4400, 6600 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 282619 Average: 3.6189 StdDev: 58.08 Min: 1 Median: 1.5829 Max: 9447 Percentiles: P50: 1.58 P75: 1.97 P99: 9.03 P99.9: 192.59 P99.99: 786.26 ------------------------------------------------------ [ 0, 1 ] 35345 12.506% 12.506% ### ( 1, 2 ] 181777 64.319% 76.825% ############# ( 2, 3 ] 37299 13.198% 90.023% ### ( 3, 4 ] 12424 4.396% 94.419% # ( 4, 6 ] 9333 3.302% 97.721% # ( 6, 10 ] 4779 1.691% 99.412% ( 10, 15 ] 721 0.255% 99.667% ( 15, 22 ] 82 0.029% 99.696% ( 22, 34 ] 8 0.003% 99.699% ( 34, 51 ] 1 0.000% 99.699% ( 76, 110 ] 179 0.063% 99.763% ( 110, 170 ] 348 0.123% 99.886% ( 170, 250 ] 143 0.051% 99.936% ( 250, 380 ] 58 0.021% 99.957% ( 380, 580 ] 66 0.023% 99.980% ( 580, 870 ] 39 0.014% 99.994% ( 870, 1300 ] 17 0.006% 100.000% ( 1300, 1900 ] 14 0.005% 100.005% ( 1900, 2900 ] 8 0.003% 100.008% ( 2900, 4400 ] 8 0.003% 100.011% ( 4400, 6600 ] 8 0.003% 100.013% ( 6600, 9900 ] 8 0.003% 100.016% ** Level 4 read latency histogram (micros): Count: 305 Average: 264.4426 StdDev: 1109.75 Min: 1 Median: 1.6808 Max: 7780 Percentiles: P50: 1.68 P75: 5.05 P99: 7383.75 P99.9: 7780.00 P99.99: 7780.00 ------------------------------------------------------ [ 0, 1 ] 64 20.984% 20.984% #### ( 1, 2 ] 130 42.623% 63.607% ######### ( 2, 3 ] 26 8.525% 72.131% ## ( 3, 4 ] 3 0.984% 73.115% ( 4, 6 ] 11 3.607% 76.721% # ( 6, 10 ] 41 13.443% 90.164% ### ( 10, 15 ] 7 2.295% 92.459% ( 250, 380 ] 2 0.656% 93.115% ( 380, 580 ] 1 0.328% 93.443% ( 870, 1300 ] 1 0.328% 93.770% ( 1300, 1900 ] 3 0.984% 94.754% ( 1900, 2900 ] 5 1.639% 96.393% ( 2900, 4400 ] 4 1.311% 97.705% ( 4400, 6600 ] 3 0.984% 98.689% ( 6600, 9900 ] 4 1.311% 100.000% ** Level 5 read latency histogram (micros): Count: 20 Average: 1044.8000 StdDev: 1176.40 Min: 164 Median: 380.0000 Max: 3769 Percentiles: P50: 380.00 P75: 1600.00 P99: 3769.00 P99.9: 3769.00 P99.99: 3769.00 ------------------------------------------------------ ( 110, 170 ] 2 10.000% 10.000% ## ( 170, 250 ] 3 15.000% 25.000% ### ( 250, 380 ] 5 25.000% 50.000% ##### ( 380, 580 ] 2 10.000% 60.000% ## ( 580, 870 ] 1 5.000% 65.000% # ( 870, 1300 ] 1 5.000% 70.000% # ( 1300, 1900 ] 2 10.000% 80.000% ## ( 1900, 2900 ] 1 5.000% 85.000% # ( 2900, 4400 ] 3 15.000% 100.000% ### ** DB Stats ** Uptime(secs): 15.7 total, 3.6 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 313.16 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.304 H:M:S, 14.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 271.90 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.996 H:M:S, 27.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/28-11:39:19 ... thread 0: (1000000,6000000) ops and (45160.3,159381.7) ops/second in (22.143362,37.645470) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 7/4 944.27 MB 62.9 4.6 0.0 4.6 10.3 5.7 0.0 1.8 137.2 305.3 34.44 32.20 101 0.341 4797K 5046 0.0 0.0 L1 11/11 1.32 GB 0.0 4.6 4.5 0.0 4.5 4.5 0.2 1.0 216.2 212.0 21.56 19.57 2 10.780 4732K 90K 0.0 0.0 L2 24/13 2.93 GB 13.6 0.3 0.2 0.2 0.3 0.2 3.1 1.8 252.7 252.6 1.38 1.33 2 0.690 354K 177 0.0 0.0 L3 21/2 969.92 MB 0.8 1.2 0.2 0.9 1.2 0.2 0.2 5.0 238.5 238.1 4.96 4.57 2 2.478 1200K 1928 0.0 0.0 L4 173/0 9.70 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.3 1.0 125.2 125.2 1.34 0.90 1 1.341 170K 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1280/30 101.70 GB 0.0 10.8 5.1 5.7 16.4 10.7 4.1 2.9 174.1 263.5 63.68 58.57 108 0.590 11M 97K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.8 5.1 5.7 16.4 10.7 4.1 2.9 174.1 263.5 63.68 58.57 108 0.590 11M 97K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.8 5.1 5.7 10.7 5.0 0.0 0.0 234.4 232.3 47.29 44.02 24 1.970 11M 97K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 353.6 16.39 14.54 84 0.195 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 37.9 total, 37.6 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.39 GB write, 443.28 MB/s write, 10.82 GB read, 292.77 MB/s read, 63.7 seconds Interval compaction: 16.39 GB write, 445.75 MB/s write, 10.82 GB read, 294.40 MB/s read, 63.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 15, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 15, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 15, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 15, total-stops: 15, interval: 30 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2448311 Average: 2.7604 StdDev: 16.97 Min: 0 Median: 1.3858 Max: 3999 Percentiles: P50: 1.39 P75: 1.78 P99: 7.72 P99.9: 160.96 P99.99: 454.54 ------------------------------------------------------ [ 0, 1 ] 618313 25.255% 25.255% ##### ( 1, 2 ] 1570455 64.144% 89.399% ############# ( 2, 3 ] 132285 5.403% 94.802% # ( 3, 4 ] 59220 2.419% 97.221% ( 4, 6 ] 36501 1.491% 98.712% ( 6, 10 ] 16381 0.669% 99.381% ( 10, 15 ] 2313 0.094% 99.475% ( 15, 22 ] 654 0.027% 99.502% ( 22, 34 ] 33 0.001% 99.503% ( 34, 51 ] 11 0.000% 99.504% ( 51, 76 ] 12 0.000% 99.504% ( 76, 110 ] 4004 0.164% 99.668% ( 110, 170 ] 6690 0.273% 99.941% ( 170, 250 ] 616 0.025% 99.966% ( 250, 380 ] 423 0.017% 99.984% ( 380, 580 ] 419 0.017% 100.001% ( 580, 870 ] 170 0.007% 100.008% ( 870, 1300 ] 36 0.001% 100.009% ( 1300, 1900 ] 17 0.001% 100.010% ( 1900, 2900 ] 19 0.001% 100.011% ( 2900, 4400 ] 10 0.000% 100.011% ** Level 1 read latency histogram (micros): Count: 313581 Average: 3.4196 StdDev: 24.64 Min: 1 Median: 1.7896 Max: 4376 Percentiles: P50: 1.79 P75: 2.57 P99: 8.45 P99.9: 15.74 P99.99: 146.77 ------------------------------------------------------ [ 0, 1 ] 8704 2.776% 2.776% # ( 1, 2 ] 187548 59.808% 62.584% ############ ( 2, 3 ] 68169 21.739% 84.323% #### ( 3, 4 ] 21122 6.736% 91.059% # ( 4, 6 ] 22064 7.036% 98.095% # ( 6, 10 ] 4644 1.481% 99.576% ( 10, 15 ] 1006 0.321% 99.897% ( 15, 22 ] 108 0.034% 99.931% ( 22, 34 ] 3 0.001% 99.932% ( 34, 51 ] 3 0.001% 99.933% ( 51, 76 ] 4 0.001% 99.934% ( 76, 110 ] 88 0.028% 99.962% ( 110, 170 ] 143 0.046% 100.008% ( 170, 250 ] 100 0.032% 100.040% ( 250, 380 ] 85 0.027% 100.067% ( 380, 580 ] 77 0.025% 100.092% ( 580, 870 ] 52 0.017% 100.108% ( 870, 1300 ] 44 0.014% 100.122% ( 1300, 1900 ] 12 0.004% 100.126% ( 1900, 2900 ] 7 0.002% 100.128% ( 2900, 4400 ] 1 0.000% 100.129% ** Level 2 read latency histogram (micros): Count: 100768 Average: 2.2931 StdDev: 19.38 Min: 1 Median: 1.4128 Max: 4485 Percentiles: P50: 1.41 P75: 1.81 P99: 6.16 P99.9: 58.57 P99.99: 492.09 ------------------------------------------------------ [ 0, 1 ] 24405 24.219% 24.219% ##### ( 1, 2 ] 62938 62.458% 86.677% ############ ( 2, 3 ] 7093 7.039% 93.716% # ( 3, 4 ] 3211 3.187% 96.903% # ( 4, 6 ] 2082 2.066% 98.969% ( 6, 10 ] 760 0.754% 99.723% ( 10, 15 ] 140 0.139% 99.862% ( 15, 22 ] 20 0.020% 99.882% ( 22, 34 ] 3 0.003% 99.885% ( 34, 51 ] 1 0.001% 99.886% ( 51, 76 ] 47 0.047% 99.933% ( 76, 110 ] 11 0.011% 99.943% ( 110, 170 ] 26 0.026% 99.969% ( 170, 250 ] 8 0.008% 99.977% ( 250, 380 ] 9 0.009% 99.986% ( 380, 580 ] 7 0.007% 99.993% ( 580, 870 ] 2 0.002% 99.995% ( 870, 1300 ] 2 0.002% 99.997% ( 1300, 1900 ] 1 0.001% 99.998% ( 1900, 2900 ] 1 0.001% 99.999% ( 4400, 6600 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 282619 Average: 3.6189 StdDev: 58.08 Min: 1 Median: 1.5829 Max: 9447 Percentiles: P50: 1.58 P75: 1.97 P99: 9.03 P99.9: 192.59 P99.99: 786.26 ------------------------------------------------------ [ 0, 1 ] 35345 12.506% 12.506% ### ( 1, 2 ] 181777 64.319% 76.825% ############# ( 2, 3 ] 37299 13.198% 90.023% ### ( 3, 4 ] 12424 4.396% 94.419% # ( 4, 6 ] 9333 3.302% 97.721% # ( 6, 10 ] 4779 1.691% 99.412% ( 10, 15 ] 721 0.255% 99.667% ( 15, 22 ] 82 0.029% 99.696% ( 22, 34 ] 8 0.003% 99.699% ( 34, 51 ] 1 0.000% 99.699% ( 76, 110 ] 179 0.063% 99.763% ( 110, 170 ] 348 0.123% 99.886% ( 170, 250 ] 143 0.051% 99.936% ( 250, 380 ] 58 0.021% 99.957% ( 380, 580 ] 66 0.023% 99.980% ( 580, 870 ] 39 0.014% 99.994% ( 870, 1300 ] 17 0.006% 100.000% ( 1300, 1900 ] 14 0.005% 100.005% ( 1900, 2900 ] 8 0.003% 100.008% ( 2900, 4400 ] 8 0.003% 100.011% ( 4400, 6600 ] 8 0.003% 100.013% ( 6600, 9900 ] 8 0.003% 100.016% ** Level 4 read latency histogram (micros): Count: 305 Average: 264.4426 StdDev: 1109.75 Min: 1 Median: 1.6808 Max: 7780 Percentiles: P50: 1.68 P75: 5.05 P99: 7383.75 P99.9: 7780.00 P99.99: 7780.00 ------------------------------------------------------ [ 0, 1 ] 64 20.984% 20.984% #### ( 1, 2 ] 130 42.623% 63.607% ######### ( 2, 3 ] 26 8.525% 72.131% ## ( 3, 4 ] 3 0.984% 73.115% ( 4, 6 ] 11 3.607% 76.721% # ( 6, 10 ] 41 13.443% 90.164% ### ( 10, 15 ] 7 2.295% 92.459% ( 250, 380 ] 2 0.656% 93.115% ( 380, 580 ] 1 0.328% 93.443% ( 870, 1300 ] 1 0.328% 93.770% ( 1300, 1900 ] 3 0.984% 94.754% ( 1900, 2900 ] 5 1.639% 96.393% ( 2900, 4400 ] 4 1.311% 97.705% ( 4400, 6600 ] 3 0.984% 98.689% ( 6600, 9900 ] 4 1.311% 100.000% ** Level 5 read latency histogram (micros): Count: 20 Average: 1044.8000 StdDev: 1176.40 Min: 164 Median: 380.0000 Max: 3769 Percentiles: P50: 380.00 P75: 1600.00 P99: 3769.00 P99.9: 3769.00 P99.99: 3769.00 ------------------------------------------------------ ( 110, 170 ] 2 10.000% 10.000% ## ( 170, 250 ] 3 15.000% 25.000% ### ( 250, 380 ] 5 25.000% 50.000% ##### ( 380, 580 ] 2 10.000% 60.000% ## ( 580, 870 ] 1 5.000% 65.000% # ( 870, 1300 ] 1 5.000% 70.000% # ( 1300, 1900 ] 2 10.000% 80.000% ## ( 1900, 2900 ] 1 5.000% 85.000% # ( 2900, 4400 ] 3 15.000% 100.000% ### ** DB Stats ** Uptime(secs): 37.9 total, 22.1 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 155.98 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:21.216 H:M:S, 56.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 44.45 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:18.912 H:M:S, 85.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 2024/11/28-11:39:23 ... thread 0: (1000000,7000000) ops and (275196.1,169576.8) ops/second in (3.633773,41.279243) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/4 1.91 GB 169.9 5.5 0.0 5.5 12.1 6.6 0.0 1.8 135.8 300.5 41.28 38.39 117 0.353 5692K 6057 0.0 0.0 L1 11/11 1.32 GB 0.0 4.6 4.5 0.0 4.5 4.5 0.2 1.0 216.2 212.0 21.56 19.57 2 10.780 4732K 90K 0.0 0.0 L2 10/8 1.17 GB 2.6 0.3 0.2 0.2 0.3 0.2 3.1 1.8 252.7 252.6 1.38 1.33 2 0.690 354K 177 0.0 0.0 L3 37/9 1.98 GB 1.5 3.2 2.0 1.2 3.1 2.0 0.2 1.6 218.5 217.9 14.79 13.63 5 2.958 3281K 7753 0.0 0.0 L4 177/5 9.75 GB 1.0 0.2 0.2 0.0 0.2 0.2 1.0 1.0 125.2 125.2 1.34 0.90 1 1.341 170K 0 0.0 0.0 L5 1052/0 86.55 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1298/37 102.68 GB 0.0 13.7 6.9 6.8 20.2 13.4 5.5 3.0 174.4 257.8 80.34 73.82 127 0.633 14M 104K 0.0 0.0 Int 0/0 0.00 KB 0.0 13.7 6.9 6.8 20.2 13.4 5.5 3.0 174.4 257.8 80.34 73.82 127 0.633 14M 104K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.7 6.9 6.8 13.6 6.8 0.0 0.0 230.9 229.2 60.70 56.59 30 2.023 14M 104K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 346.3 19.65 17.23 97 0.203 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 41.5 total, 41.3 interval Flush(GB): cumulative 6.644, interval 6.644 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 20.23 GB write, 499.22 MB/s write, 13.69 GB read, 337.76 MB/s read, 80.3 seconds Interval compaction: 20.23 GB write, 501.75 MB/s write, 13.69 GB read, 339.48 MB/s read, 80.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 15, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 15, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 15, total-stops: 27, interval: 42 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2686464 Average: 2.7571 StdDev: 16.60 Min: 0 Median: 1.4156 Max: 3999 Percentiles: P50: 1.42 P75: 1.81 P99: 7.60 P99.9: 158.90 P99.99: 441.78 ------------------------------------------------------ [ 0, 1 ] 631734 23.515% 23.515% ##### ( 1, 2 ] 1711869 63.722% 87.237% ############# ( 2, 3 ] 175092 6.518% 93.755% # ( 3, 4 ] 78039 2.905% 96.660% # ( 4, 6 ] 55349 2.060% 98.720% ( 6, 10 ] 18757 0.698% 99.418% ( 10, 15 ] 2728 0.102% 99.520% ( 15, 22 ] 690 0.026% 99.546% ( 22, 34 ] 38 0.001% 99.547% ( 34, 51 ] 11 0.000% 99.547% ( 51, 76 ] 13 0.000% 99.548% ( 76, 110 ] 4004 0.149% 99.697% ( 110, 170 ] 6691 0.249% 99.946% ( 170, 250 ] 624 0.023% 99.969% ( 250, 380 ] 426 0.016% 99.985% ( 380, 580 ] 422 0.016% 100.001% ( 580, 870 ] 177 0.007% 100.007% ( 870, 1300 ] 40 0.001% 100.009% ( 1300, 1900 ] 22 0.001% 100.010% ( 1900, 2900 ] 20 0.001% 100.010% ( 2900, 4400 ] 11 0.000% 100.011% ** Level 1 read latency histogram (micros): Count: 917801 Average: 3.8013 StdDev: 28.39 Min: 1 Median: 1.8535 Max: 4376 Percentiles: P50: 1.85 P75: 2.72 P99: 9.12 P99.9: 18.06 P99.99: 173.90 ------------------------------------------------------ [ 0, 1 ] 19101 2.081% 2.081% ( 1, 2 ] 515275 56.142% 58.224% ########### ( 2, 3 ] 215270 23.455% 81.678% ##### ( 3, 4 ] 69508 7.573% 89.252% ## ( 4, 6 ] 74888 8.160% 97.411% ## ( 6, 10 ] 18697 2.037% 99.448% ( 10, 15 ] 3985 0.434% 99.883% ( 15, 22 ] 369 0.040% 99.923% ( 22, 34 ] 16 0.002% 99.925% ( 34, 51 ] 5 0.001% 99.925% ( 51, 76 ] 5 0.001% 99.926% ( 76, 110 ] 158 0.017% 99.943% ( 110, 170 ] 416 0.045% 99.988% ( 170, 250 ] 374 0.041% 100.029% ( 250, 380 ] 397 0.043% 100.072% ( 380, 580 ] 394 0.043% 100.115% ( 580, 870 ] 256 0.028% 100.143% ( 870, 1300 ] 129 0.014% 100.157% ( 1300, 1900 ] 46 0.005% 100.162% ( 1900, 2900 ] 22 0.002% 100.165% ( 2900, 4400 ] 8 0.001% 100.165% ** Level 2 read latency histogram (micros): Count: 100768 Average: 2.2931 StdDev: 19.38 Min: 1 Median: 1.4128 Max: 4485 Percentiles: P50: 1.41 P75: 1.81 P99: 6.16 P99.9: 58.57 P99.99: 492.09 ------------------------------------------------------ [ 0, 1 ] 24405 24.219% 24.219% ##### ( 1, 2 ] 62938 62.458% 86.677% ############ ( 2, 3 ] 7093 7.039% 93.716% # ( 3, 4 ] 3211 3.187% 96.903% # ( 4, 6 ] 2082 2.066% 98.969% ( 6, 10 ] 760 0.754% 99.723% ( 10, 15 ] 140 0.139% 99.862% ( 15, 22 ] 20 0.020% 99.882% ( 22, 34 ] 3 0.003% 99.885% ( 34, 51 ] 1 0.001% 99.886% ( 51, 76 ] 47 0.047% 99.933% ( 76, 110 ] 11 0.011% 99.943% ( 110, 170 ] 26 0.026% 99.969% ( 170, 250 ] 8 0.008% 99.977% ( 250, 380 ] 9 0.009% 99.986% ( 380, 580 ] 7 0.007% 99.993% ( 580, 870 ] 2 0.002% 99.995% ( 870, 1300 ] 2 0.002% 99.997% ( 1300, 1900 ] 1 0.001% 99.998% ( 1900, 2900 ] 1 0.001% 99.999% ( 4400, 6600 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 317716 Average: 4.1945 StdDev: 60.95 Min: 1 Median: 1.6217 Max: 9447 Percentiles: P50: 1.62 P75: 2.10 P99: 9.30 P99.9: 326.35 P99.99: 1035.36 ------------------------------------------------------ [ 0, 1 ] 35838 11.280% 11.280% ## ( 1, 2 ] 197876 62.281% 73.561% ############ ( 2, 3 ] 46644 14.681% 88.242% ### ( 3, 4 ] 15994 5.034% 93.276% # ( 4, 6 ] 13463 4.237% 97.513% # ( 6, 10 ] 5729 1.803% 99.316% ( 10, 15 ] 878 0.276% 99.593% ( 15, 22 ] 98 0.031% 99.624% ( 22, 34 ] 11 0.003% 99.627% ( 34, 51 ] 10 0.003% 99.630% ( 76, 110 ] 184 0.058% 99.688% ( 110, 170 ] 386 0.121% 99.810% ( 170, 250 ] 208 0.065% 99.875% ( 250, 380 ] 135 0.042% 99.918% ( 380, 580 ] 128 0.040% 99.958% ( 580, 870 ] 83 0.026% 99.984% ( 870, 1300 ] 50 0.016% 100.000% ( 1300, 1900 ] 32 0.010% 100.010% ( 1900, 2900 ] 18 0.006% 100.015% ( 2900, 4400 ] 13 0.004% 100.020% ( 4400, 6600 ] 8 0.003% 100.022% ( 6600, 9900 ] 8 0.003% 100.025% ** Level 4 read latency histogram (micros): Count: 597 Average: 513.9598 StdDev: 985.95 Min: 1 Median: 6.6977 Max: 7780 Percentiles: P50: 6.70 P75: 689.52 P99: 5155.33 P99.9: 7780.00 P99.99: 7780.00 ------------------------------------------------------ [ 0, 1 ] 66 11.055% 11.055% ## ( 1, 2 ] 163 27.303% 38.358% ##### ( 2, 3 ] 42 7.035% 45.394% # ( 3, 4 ] 8 1.340% 46.734% ( 4, 6 ] 12 2.010% 48.744% ( 6, 10 ] 43 7.203% 55.946% # ( 10, 15 ] 7 1.173% 57.119% ( 22, 34 ] 1 0.168% 57.286% ( 110, 170 ] 4 0.670% 57.956% ( 170, 250 ] 13 2.178% 60.134% ( 250, 380 ] 23 3.853% 63.987% # ( 380, 580 ] 48 8.040% 72.027% ## ( 580, 870 ] 47 7.873% 79.899% ## ( 870, 1300 ] 48 8.040% 87.940% ## ( 1300, 1900 ] 33 5.528% 93.467% # ( 1900, 2900 ] 20 3.350% 96.817% # ( 2900, 4400 ] 12 2.010% 98.827% ( 4400, 6600 ] 3 0.503% 99.330% ( 6600, 9900 ] 4 0.670% 100.000% ** Level 5 read latency histogram (micros): Count: 60 Average: 1090.6000 StdDev: 1175.93 Min: 97 Median: 480.0000 Max: 4855 Percentiles: P50: 480.00 P75: 1700.00 P99: 4855.00 P99.9: 4855.00 P99.99: 4855.00 ------------------------------------------------------ ( 76, 110 ] 1 1.667% 1.667% ( 110, 170 ] 7 11.667% 13.333% ## ( 170, 250 ] 8 13.333% 26.667% ### ( 250, 380 ] 9 15.000% 41.667% ### ( 380, 580 ] 10 16.667% 58.333% ### ( 580, 870 ] 2 3.333% 61.667% # ( 870, 1300 ] 4 6.667% 68.333% # ( 1300, 1900 ] 6 10.000% 78.333% ## ( 1900, 2900 ] 7 11.667% 90.000% ## ( 2900, 4400 ] 5 8.333% 98.333% ## ( 4400, 6600 ] 1 1.667% 100.000% ** DB Stats ** Uptime(secs): 41.5 total, 3.6 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 166.04 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:22.066 H:M:S, 53.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 270.84 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.850 H:M:S, 23.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/11/28-11:39:29 ... thread 0: (1000000,8000000) ops and (174001.6,170117.5) ops/second in (5.747075,47.026318) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/15 2.83 GB 12.6 6.1 0.0 6.1 13.7 7.6 0.0 1.8 125.8 282.0 49.57 43.50 130 0.381 6332K 6863 0.0 0.0 L1 0/0 0.00 KB 0.0 4.6 4.5 0.0 4.5 4.5 0.2 1.0 216.2 212.0 21.56 19.57 2 10.780 4732K 90K 0.0 0.0 L2 1/0 72.37 MB 0.7 1.8 1.5 0.2 1.7 1.5 3.1 1.2 193.3 192.9 9.27 7.81 3 3.091 451K 177 0.0 0.0 L3 15/0 980.38 MB 1.0 5.0 3.4 1.6 5.0 3.4 1.2 1.5 183.2 182.7 27.95 22.49 15 1.864 5200K 11K 0.0 0.0 L4 182/0 9.55 GB 1.0 3.7 3.5 0.1 3.7 3.5 1.0 1.0 113.0 112.8 33.27 18.45 48 0.693 3729K 2960 0.0 0.0 L5 1096/0 90.18 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1310/15 103.59 GB 0.0 21.1 13.0 8.1 28.5 20.4 10.2 3.8 152.3 206.1 141.63 111.81 198 0.715 20M 112K 0.0 0.0 Int 0/0 0.00 KB 0.0 21.1 13.0 8.1 28.5 20.4 10.2 3.8 152.3 206.1 141.63 111.81 198 0.715 20M 112K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 21.1 13.0 8.1 20.9 12.9 0.0 0.0 184.7 183.7 116.75 92.01 90 1.297 20M 112K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 311.5 24.88 19.80 108 0.230 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 47.2 total, 47.0 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 28.51 GB write, 618.04 MB/s write, 21.06 GB read, 456.57 MB/s read, 141.6 seconds Interval compaction: 28.51 GB write, 620.79 MB/s write, 21.06 GB read, 458.60 MB/s read, 141.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 15, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 15, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 65, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 15, total-stops: 65, interval: 80 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2926674 Average: 3.0352 StdDev: 35.36 Min: 0 Median: 1.4453 Max: 23033 Percentiles: P50: 1.45 P75: 1.84 P99: 7.90 P99.9: 161.99 P99.99: 642.25 ------------------------------------------------------ [ 0, 1 ] 642699 21.960% 21.960% #### ( 1, 2 ] 1842973 62.972% 84.932% ############# ( 2, 3 ] 223806 7.647% 92.579% ## ( 3, 4 ] 97565 3.334% 95.912% # ( 4, 6 ] 79463 2.715% 98.628% # ( 6, 10 ] 22948 0.784% 99.412% ( 10, 15 ] 3619 0.124% 99.535% ( 15, 22 ] 780 0.027% 99.562% ( 22, 34 ] 43 0.001% 99.563% ( 34, 51 ] 18 0.001% 99.564% ( 51, 76 ] 14 0.000% 99.564% ( 76, 110 ] 4005 0.137% 99.701% ( 110, 170 ] 6710 0.229% 99.931% ( 170, 250 ] 656 0.022% 99.953% ( 250, 380 ] 490 0.017% 99.970% ( 380, 580 ] 526 0.018% 99.988% ( 580, 870 ] 309 0.011% 99.998% ( 870, 1300 ] 167 0.006% 100.004% ( 1300, 1900 ] 77 0.003% 100.007% ( 1900, 2900 ] 60 0.002% 100.009% ( 2900, 4400 ] 62 0.002% 100.011% ( 4400, 6600 ] 8 0.000% 100.011% ( 6600, 9900 ] 4 0.000% 100.011% ( 14000, 22000 ] 2 0.000% 100.011% ( 22000, 33000 ] 1 0.000% 100.011% ** Level 1 read latency histogram (micros): Count: 1156384 Average: 5.4308 StdDev: 79.82 Min: 1 Median: 1.8882 Max: 8859 Percentiles: P50: 1.89 P75: 2.79 P99: 9.50 P99.9: 154.25 P99.99: 393.85 ------------------------------------------------------ [ 0, 1 ] 22054 1.907% 1.907% ( 1, 2 ] 626116 54.144% 56.051% ########### ( 2, 3 ] 277783 24.022% 80.073% ##### ( 3, 4 ] 93146 8.055% 88.128% ## ( 4, 6 ] 103624 8.961% 97.089% ## ( 6, 10 ] 25274 2.186% 99.275% ( 10, 15 ] 6092 0.527% 99.802% ( 15, 22 ] 612 0.053% 99.854% ( 22, 34 ] 32 0.003% 99.857% ( 34, 51 ] 5 0.000% 99.858% ( 51, 76 ] 9 0.001% 99.858% ( 76, 110 ] 165 0.014% 99.873% ( 110, 170 ] 428 0.037% 99.910% ( 170, 250 ] 418 0.036% 99.946% ( 250, 380 ] 474 0.041% 99.987% ( 380, 580 ] 525 0.045% 100.032% ( 580, 870 ] 436 0.038% 100.070% ( 870, 1300 ] 256 0.022% 100.092% ( 1300, 1900 ] 153 0.013% 100.105% ( 1900, 2900 ] 163 0.014% 100.119% ( 2900, 4400 ] 213 0.018% 100.138% ( 4400, 6600 ] 65 0.006% 100.143% ( 6600, 9900 ] 15 0.001% 100.145% ** Level 2 read latency histogram (micros): Count: 280950 Average: 6.5248 StdDev: 109.81 Min: 1 Median: 1.7607 Max: 11469 Percentiles: P50: 1.76 P75: 2.64 P99: 9.37 P99.9: 746.56 P99.99: 2240.57 ------------------------------------------------------ [ 0, 1 ] 26476 9.424% 9.424% ## ( 1, 2 ] 149858 53.340% 62.763% ########### ( 2, 3 ] 53876 19.176% 81.940% #### ( 3, 4 ] 20538 7.310% 89.250% # ( 4, 6 ] 23257 8.278% 97.528% ## ( 6, 10 ] 4914 1.749% 99.277% ( 10, 15 ] 1277 0.455% 99.732% ( 15, 22 ] 122 0.043% 99.775% ( 22, 34 ] 13 0.005% 99.780% ( 34, 51 ] 6 0.002% 99.782% ( 51, 76 ] 49 0.017% 99.799% ( 76, 110 ] 13 0.005% 99.804% ( 110, 170 ] 32 0.011% 99.815% ( 170, 250 ] 24 0.009% 99.824% ( 250, 380 ] 66 0.023% 99.847% ( 380, 580 ] 82 0.029% 99.876% ( 580, 870 ] 115 0.041% 99.917% ( 870, 1300 ] 105 0.037% 99.955% ( 1300, 1900 ] 72 0.026% 99.980% ( 1900, 2900 ] 79 0.028% 100.009% ( 2900, 4400 ] 73 0.026% 100.035% ( 4400, 6600 ] 28 0.010% 100.044% ( 6600, 9900 ] 13 0.005% 100.049% ( 9900, 14000 ] 1 0.000% 100.049% ** Level 3 read latency histogram (micros): Count: 1178927 Average: 8.5597 StdDev: 131.95 Min: 1 Median: 1.9196 Max: 21362 Percentiles: P50: 1.92 P75: 2.90 P99: 9.96 P99.9: 431.19 P99.99: 802.59 ------------------------------------------------------ [ 0, 1 ] 45835 3.888% 3.888% # ( 1, 2 ] 591176 50.145% 54.033% ########## ( 2, 3 ] 275349 23.356% 77.389% ##### ( 3, 4 ] 106723 9.053% 86.442% ## ( 4, 6 ] 121255 10.285% 96.727% ## ( 6, 10 ] 27087 2.298% 99.024% ( 10, 15 ] 7530 0.639% 99.663% ( 15, 22 ] 978 0.083% 99.746% ( 22, 34 ] 63 0.005% 99.751% ( 34, 51 ] 26 0.002% 99.754% ( 51, 76 ] 12 0.001% 99.755% ( 76, 110 ] 194 0.016% 99.771% ( 110, 170 ] 460 0.039% 99.810% ( 170, 250 ] 415 0.035% 99.845% ( 250, 380 ] 481 0.041% 99.886% ( 380, 580 ] 641 0.054% 99.940% ( 580, 870 ] 761 0.065% 100.005% ( 870, 1300 ] 589 0.050% 100.055% ( 1300, 1900 ] 361 0.031% 100.086% ( 1900, 2900 ] 435 0.037% 100.122% ( 2900, 4400 ] 545 0.046% 100.169% ( 4400, 6600 ] 147 0.012% 100.181% ( 6600, 9900 ] 71 0.006% 100.187% ( 9900, 14000 ] 4 0.000% 100.188% ( 14000, 22000 ] 1 0.000% 100.188% ** Level 4 read latency histogram (micros): Count: 37397 Average: 112.6843 StdDev: 485.59 Min: 1 Median: 2.1404 Max: 12517 Percentiles: P50: 2.14 P75: 3.53 P99: 2763.26 P99.9: 5575.85 P99.99: 8769.87 ------------------------------------------------------ [ 0, 1 ] 911 2.436% 2.436% ( 1, 2 ] 16584 44.346% 46.782% ######### ( 2, 3 ] 8574 22.927% 69.709% ##### ( 3, 4 ] 3766 10.070% 79.779% ## ( 4, 6 ] 3108 8.311% 88.090% ## ( 6, 10 ] 841 2.249% 90.339% ( 10, 15 ] 247 0.660% 90.999% ( 15, 22 ] 60 0.160% 91.160% ( 22, 34 ] 18 0.048% 91.208% ( 34, 51 ] 6 0.016% 91.224% ( 51, 76 ] 1 0.003% 91.227% ( 76, 110 ] 5 0.013% 91.240% ( 110, 170 ] 54 0.144% 91.384% ( 170, 250 ] 190 0.508% 91.892% ( 250, 380 ] 255 0.682% 92.574% ( 380, 580 ] 450 1.203% 93.778% ( 580, 870 ] 672 1.797% 95.575% ( 870, 1300 ] 647 1.730% 97.305% ( 1300, 1900 ] 388 1.038% 98.342% ( 1900, 2900 ] 285 0.762% 99.104% ( 2900, 4400 ] 265 0.709% 99.813% ( 4400, 6600 ] 61 0.163% 99.976% ( 6600, 9900 ] 8 0.021% 99.997% ( 9900, 14000 ] 1 0.003% 100.000% ** Level 5 read latency histogram (micros): Count: 280 Average: 1654.2250 StdDev: 1592.03 Min: 97 Median: 985.7692 Max: 8178 Percentiles: P50: 985.77 P75: 2629.73 P99: 6820.00 P99.9: 8178.00 P99.99: 8178.00 ------------------------------------------------------ ( 76, 110 ] 2 0.714% 0.714% ( 110, 170 ] 14 5.000% 5.714% # ( 170, 250 ] 20 7.143% 12.857% # ( 250, 380 ] 36 12.857% 25.714% ### ( 380, 580 ] 29 10.357% 36.071% ## ( 580, 870 ] 32 11.429% 47.500% ## ( 870, 1300 ] 26 9.286% 56.786% ## ( 1300, 1900 ] 24 8.571% 65.357% ## ( 1900, 2900 ] 37 13.214% 78.571% ### ( 2900, 4400 ] 40 14.286% 92.857% ### ( 4400, 6600 ] 17 6.071% 98.929% # ( 6600, 9900 ] 3 1.071% 100.000% ** DB Stats ** Uptime(secs): 47.2 total, 5.7 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 166.68 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:24.764 H:M:S, 52.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 171.25 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.698 H:M:S, 46.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/28-11:39:33 ... thread 0: (1000000,9000000) ops and (224124.0,174797.6) ops/second in (4.461815,51.488133) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/11 3.81 GB 144.8 7.2 0.0 7.2 15.7 8.5 0.0 1.8 129.9 284.0 56.75 50.27 149 0.381 7483K 8145 0.0 0.0 L1 0/0 0.00 KB 0.0 4.6 4.5 0.0 4.5 4.5 0.2 1.0 216.2 212.0 21.56 19.57 2 10.780 4732K 90K 0.0 0.0 L2 1/0 72.37 MB 0.7 1.8 1.5 0.2 1.7 1.5 3.1 1.2 193.3 192.9 9.27 7.81 3 3.091 451K 177 0.0 0.0 L3 15/0 980.38 MB 1.0 5.0 3.4 1.6 5.0 3.4 1.2 1.5 183.2 182.7 27.95 22.49 15 1.864 5200K 11K 0.0 0.0 L4 182/0 9.55 GB 1.0 3.7 3.5 0.1 3.7 3.5 1.0 1.0 113.0 112.8 33.27 18.45 48 0.693 3729K 2960 0.0 0.0 L5 1096/0 90.18 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1313/11 104.57 GB 0.0 22.2 13.0 9.2 30.6 21.4 10.2 3.6 152.6 210.6 148.80 118.58 217 0.686 21M 113K 0.0 0.0 Int 0/0 0.00 KB 0.0 22.2 13.0 9.2 30.6 21.4 10.2 3.6 152.6 210.6 148.80 118.58 217 0.686 21M 113K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 22.2 13.0 9.2 22.0 12.9 0.0 0.0 187.6 186.6 121.03 96.22 94 1.288 21M 113K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.6 8.6 0.0 0.0 0.0 315.3 27.77 22.37 123 0.226 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 51.7 total, 51.5 interval Flush(GB): cumulative 8.551, interval 8.551 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 30.60 GB write, 606.11 MB/s write, 22.17 GB read, 439.10 MB/s read, 148.8 seconds Interval compaction: 30.60 GB write, 608.57 MB/s write, 22.17 GB read, 440.88 MB/s read, 148.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 21, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 21, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 66, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 21, total-stops: 66, interval: 87 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3427232 Average: 2.9201 StdDev: 32.84 Min: 0 Median: 1.4526 Max: 23033 Percentiles: P50: 1.45 P75: 1.84 P99: 7.65 P99.9: 157.46 P99.99: 549.02 ------------------------------------------------------ [ 0, 1 ] 722393 21.078% 21.078% #### ( 1, 2 ] 2189985 63.900% 84.978% ############# ( 2, 3 ] 270298 7.887% 92.864% ## ( 3, 4 ] 111074 3.241% 96.105% # ( 4, 6 ] 87799 2.562% 98.667% # ( 6, 10 ] 27591 0.805% 99.472% ( 10, 15 ] 4199 0.123% 99.595% ( 15, 22 ] 889 0.026% 99.621% ( 22, 34 ] 51 0.001% 99.622% ( 34, 51 ] 18 0.001% 99.623% ( 51, 76 ] 19 0.001% 99.623% ( 76, 110 ] 4096 0.120% 99.743% ( 110, 170 ] 6819 0.199% 99.942% ( 170, 250 ] 687 0.020% 99.962% ( 250, 380 ] 510 0.015% 99.977% ( 380, 580 ] 547 0.016% 99.993% ( 580, 870 ] 321 0.009% 100.002% ( 870, 1300 ] 176 0.005% 100.007% ( 1300, 1900 ] 79 0.002% 100.009% ( 1900, 2900 ] 61 0.002% 100.011% ( 2900, 4400 ] 62 0.002% 100.013% ( 4400, 6600 ] 8 0.000% 100.013% ( 6600, 9900 ] 4 0.000% 100.013% ( 14000, 22000 ] 2 0.000% 100.013% ( 22000, 33000 ] 1 0.000% 100.013% ** Level 1 read latency histogram (micros): Count: 1156384 Average: 5.4308 StdDev: 79.82 Min: 1 Median: 1.8882 Max: 8859 Percentiles: P50: 1.89 P75: 2.79 P99: 9.50 P99.9: 154.25 P99.99: 393.85 ------------------------------------------------------ [ 0, 1 ] 22054 1.907% 1.907% ( 1, 2 ] 626116 54.144% 56.051% ########### ( 2, 3 ] 277783 24.022% 80.073% ##### ( 3, 4 ] 93146 8.055% 88.128% ## ( 4, 6 ] 103624 8.961% 97.089% ## ( 6, 10 ] 25274 2.186% 99.275% ( 10, 15 ] 6092 0.527% 99.802% ( 15, 22 ] 612 0.053% 99.854% ( 22, 34 ] 32 0.003% 99.857% ( 34, 51 ] 5 0.000% 99.858% ( 51, 76 ] 9 0.001% 99.858% ( 76, 110 ] 165 0.014% 99.873% ( 110, 170 ] 428 0.037% 99.910% ( 170, 250 ] 418 0.036% 99.946% ( 250, 380 ] 474 0.041% 99.987% ( 380, 580 ] 525 0.045% 100.032% ( 580, 870 ] 436 0.038% 100.070% ( 870, 1300 ] 256 0.022% 100.092% ( 1300, 1900 ] 153 0.013% 100.105% ( 1900, 2900 ] 163 0.014% 100.119% ( 2900, 4400 ] 213 0.018% 100.138% ( 4400, 6600 ] 65 0.006% 100.143% ( 6600, 9900 ] 15 0.001% 100.145% ** Level 2 read latency histogram (micros): Count: 280950 Average: 6.5248 StdDev: 109.81 Min: 1 Median: 1.7607 Max: 11469 Percentiles: P50: 1.76 P75: 2.64 P99: 9.37 P99.9: 746.56 P99.99: 2240.57 ------------------------------------------------------ [ 0, 1 ] 26476 9.424% 9.424% ## ( 1, 2 ] 149858 53.340% 62.763% ########### ( 2, 3 ] 53876 19.176% 81.940% #### ( 3, 4 ] 20538 7.310% 89.250% # ( 4, 6 ] 23257 8.278% 97.528% ## ( 6, 10 ] 4914 1.749% 99.277% ( 10, 15 ] 1277 0.455% 99.732% ( 15, 22 ] 122 0.043% 99.775% ( 22, 34 ] 13 0.005% 99.780% ( 34, 51 ] 6 0.002% 99.782% ( 51, 76 ] 49 0.017% 99.799% ( 76, 110 ] 13 0.005% 99.804% ( 110, 170 ] 32 0.011% 99.815% ( 170, 250 ] 24 0.009% 99.824% ( 250, 380 ] 66 0.023% 99.847% ( 380, 580 ] 82 0.029% 99.876% ( 580, 870 ] 115 0.041% 99.917% ( 870, 1300 ] 105 0.037% 99.955% ( 1300, 1900 ] 72 0.026% 99.980% ( 1900, 2900 ] 79 0.028% 100.009% ( 2900, 4400 ] 73 0.026% 100.035% ( 4400, 6600 ] 28 0.010% 100.044% ( 6600, 9900 ] 13 0.005% 100.049% ( 9900, 14000 ] 1 0.000% 100.049% ** Level 3 read latency histogram (micros): Count: 1178927 Average: 8.5597 StdDev: 131.95 Min: 1 Median: 1.9196 Max: 21362 Percentiles: P50: 1.92 P75: 2.90 P99: 9.96 P99.9: 431.19 P99.99: 802.59 ------------------------------------------------------ [ 0, 1 ] 45835 3.888% 3.888% # ( 1, 2 ] 591176 50.145% 54.033% ########## ( 2, 3 ] 275349 23.356% 77.389% ##### ( 3, 4 ] 106723 9.053% 86.442% ## ( 4, 6 ] 121255 10.285% 96.727% ## ( 6, 10 ] 27087 2.298% 99.024% ( 10, 15 ] 7530 0.639% 99.663% ( 15, 22 ] 978 0.083% 99.746% ( 22, 34 ] 63 0.005% 99.751% ( 34, 51 ] 26 0.002% 99.754% ( 51, 76 ] 12 0.001% 99.755% ( 76, 110 ] 194 0.016% 99.771% ( 110, 170 ] 460 0.039% 99.810% ( 170, 250 ] 415 0.035% 99.845% ( 250, 380 ] 481 0.041% 99.886% ( 380, 580 ] 641 0.054% 99.940% ( 580, 870 ] 761 0.065% 100.005% ( 870, 1300 ] 589 0.050% 100.055% ( 1300, 1900 ] 361 0.031% 100.086% ( 1900, 2900 ] 435 0.037% 100.122% ( 2900, 4400 ] 545 0.046% 100.169% ( 4400, 6600 ] 147 0.012% 100.181% ( 6600, 9900 ] 71 0.006% 100.187% ( 9900, 14000 ] 4 0.000% 100.188% ( 14000, 22000 ] 1 0.000% 100.188% ** Level 4 read latency histogram (micros): Count: 37397 Average: 112.6843 StdDev: 485.59 Min: 1 Median: 2.1404 Max: 12517 Percentiles: P50: 2.14 P75: 3.53 P99: 2763.26 P99.9: 5575.85 P99.99: 8769.87 ------------------------------------------------------ [ 0, 1 ] 911 2.436% 2.436% ( 1, 2 ] 16584 44.346% 46.782% ######### ( 2, 3 ] 8574 22.927% 69.709% ##### ( 3, 4 ] 3766 10.070% 79.779% ## ( 4, 6 ] 3108 8.311% 88.090% ## ( 6, 10 ] 841 2.249% 90.339% ( 10, 15 ] 247 0.660% 90.999% ( 15, 22 ] 60 0.160% 91.160% ( 22, 34 ] 18 0.048% 91.208% ( 34, 51 ] 6 0.016% 91.224% ( 51, 76 ] 1 0.003% 91.227% ( 76, 110 ] 5 0.013% 91.240% ( 110, 170 ] 54 0.144% 91.384% ( 170, 250 ] 190 0.508% 91.892% ( 250, 380 ] 255 0.682% 92.574% ( 380, 580 ] 450 1.203% 93.778% ( 580, 870 ] 672 1.797% 95.575% ( 870, 1300 ] 647 1.730% 97.305% ( 1300, 1900 ] 388 1.038% 98.342% ( 1900, 2900 ] 285 0.762% 99.104% ( 2900, 4400 ] 265 0.709% 99.813% ( 4400, 6600 ] 61 0.163% 99.976% ( 6600, 9900 ] 8 0.021% 99.997% ( 9900, 14000 ] 1 0.003% 100.000% ** Level 5 read latency histogram (micros): Count: 280 Average: 1654.2250 StdDev: 1592.03 Min: 97 Median: 985.7692 Max: 8178 Percentiles: P50: 985.77 P75: 2629.73 P99: 6820.00 P99.9: 8178.00 P99.99: 8178.00 ------------------------------------------------------ ( 76, 110 ] 2 0.714% 0.714% ( 110, 170 ] 14 5.000% 5.714% # ( 170, 250 ] 20 7.143% 12.857% # ( 250, 380 ] 36 12.857% 25.714% ### ( 380, 580 ] 29 10.357% 36.071% ## ( 580, 870 ] 32 11.429% 47.500% ## ( 870, 1300 ] 26 9.286% 56.786% ## ( 1300, 1900 ] 24 8.571% 65.357% ## ( 1900, 2900 ] 37 13.214% 78.571% ### ( 2900, 4400 ] 40 14.286% 92.857% ### ( 4400, 6600 ] 17 6.071% 98.929% # ( 6600, 9900 ] 3 1.071% 100.000% ** DB Stats ** Uptime(secs): 51.7 total, 4.5 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 171.33 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:26.541 H:M:S, 51.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 220.58 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.777 H:M:S, 39.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 6.120 micros/op 163402 ops/sec 60.311 seconds 9854999 operations; 158.3 MB/s Microseconds per write: Count: 9854999 Average: 6.1199 StdDev: 624.33 Min: 1 Median: 1.7731 Max: 957896 Percentiles: P50: 1.77 P75: 2.79 P99: 8.27 P99.9: 1176.56 P99.99: 2765.36 ------------------------------------------------------ [ 0, 1 ] 1108996 11.253% 11.253% ## ( 1, 2 ] 4938961 50.116% 61.369% ########## ( 2, 3 ] 1693469 17.184% 78.553% ### ( 3, 4 ] 1402138 14.228% 92.781% ### ( 4, 6 ] 559909 5.681% 98.462% # ( 6, 10 ] 93338 0.947% 99.410% ( 10, 15 ] 23126 0.235% 99.644% ( 15, 22 ] 6375 0.065% 99.709% ( 22, 34 ] 1143 0.012% 99.721% ( 34, 51 ] 174 0.002% 99.722% ( 51, 76 ] 53 0.001% 99.723% ( 76, 110 ] 6921 0.070% 99.793% ( 110, 170 ] 2726 0.028% 99.821% ( 170, 250 ] 253 0.003% 99.823% ( 250, 380 ] 99 0.001% 99.824% ( 380, 580 ] 37 0.000% 99.825% ( 580, 870 ] 43 0.000% 99.825% ( 870, 1300 ] 10356 0.105% 99.930% ( 1300, 1900 ] 25 0.000% 99.930% ( 1900, 2900 ] 6785 0.069% 99.999% ( 2900, 4400 ] 38 0.000% 100.000% ( 4400, 6600 ] 2 0.000% 100.000% ( 6600, 9900 ] 2 0.000% 100.000% ( 9900, 14000 ] 3 0.000% 100.000% ( 14000, 22000 ] 1 0.000% 100.000% ( 22000, 33000 ] 1 0.000% 100.000% ( 50000, 75000 ] 1 0.000% 100.000% ( 110000, 170000 ] 4 0.000% 100.000% ( 170000, 250000 ] 12 0.000% 100.000% ( 250000, 380000 ] 3 0.000% 100.000% ( 380000, 570000 ] 1 0.000% 100.000% ( 570000, 860000 ] 2 0.000% 100.000% ( 860000, 1200000 ] 2 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 195176 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 9854999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 10170358968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 587 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 32896160 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 31958709378 rocksdb.compact.write.bytes COUNT : 31755515712 rocksdb.flush.write.bytes COUNT : 10106694877 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 64 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 311 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 10025292800 rocksdb.memtable.garbage.bytes.at.flush COUNT : 3976192 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 32060164249 rocksdb.non.last.level.read.count COUNT : 7746100 rocksdb.block.checksum.compute.count COUNT : 7744990 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.413693 P95 : 3.883347 P99 : 6.335363 P100 : 957892.000000 COUNT : 9854999 SUM : 55510084 rocksdb.compaction.times.micros P50 : 860000.000000 P95 : 3936000.000000 P99 : 13748000.000000 P100 : 20240017.000000 COUNT : 106 SUM : 153089251 rocksdb.compaction.times.cpu_micros P50 : 570000.000000 P95 : 3390000.000000 P99 : 13748000.000000 P100 : 18290263.000000 COUNT : 106 SUM : 119332401 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 761.250000 P95 : 10412.500000 P99 : 47025.000000 P100 : 184419.000000 COUNT : 135 SUM : 479754 rocksdb.compaction.outfile.sync.micros P50 : 1297.378049 P95 : 87250.000000 P99 : 202480.000000 P100 : 325101.000000 COUNT : 397 SUM : 5750172 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 138.750000 P95 : 2640.000000 P99 : 4649.333333 P100 : 6202.000000 COUNT : 266 SUM : 130438 rocksdb.table.open.io.micros P50 : 1661.956522 P95 : 10383.214286 P99 : 19565.000000 P100 : 22284.000000 COUNT : 587 SUM : 1650814 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.714626 P95 : 8.509111 P99 : 13.301643 P100 : 29107.000000 COUNT : 7743446 SUM : 63594080 rocksdb.read.block.get.micros P50 : 563.505155 P95 : 4241.780822 P99 : 7621.058824 P100 : 9494.000000 COUNT : 1174 SUM : 1300578 rocksdb.write.raw.block.micros P50 : 0.516611 P95 : 0.981561 P99 : 3.004975 P100 : 32910.000000 COUNT : 10143497 SUM : 24426470 rocksdb.numfiles.in.singlecompaction P50 : 3.030769 P95 : 6.800000 P99 : 14.000000 P100 : 19.000000 COUNT : 160 SUM : 555 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.565412 P95 : 1273.802137 P99 : 2684.626888 P100 : 957717.000000 COUNT : 148363 SUM : 32908115 rocksdb.sst.read.micros P50 : 1.657587 P95 : 5.196984 P99 : 9.599650 P100 : 29103.000000 COUNT : 7746562 SUM : 47885001 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 9854999 SUM : 10170358968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 215393.258427 P95 : 549642.857143 P99 : 1047000.000000 P100 : 1117727.000000 COUNT : 135 SUM : 32531194 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 374 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0