[2024-10-12 09:13:11.074367] Starting SPDK v25.01-pre git sha1 bbce7a874 / DPDK 24.03.0 initialization... [2024-10-12 09:13:11.074519] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3611969 ] Set seed to 1728717191070824 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-10-12 09:13:11.181819] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-10-12 09:13:11.288769] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-10-12 09:13:11.288815] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3611969' to capture a snapshot of events at runtime. [2024-10-12 09:13:11.288826] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-10-12 09:13:11.288836] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-10-12 09:13:11.288844] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3611969 for offline analysis/debug. [2024-10-12 09:13:11.290363] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sat Oct 12 09:13:19 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/10/12-09:13:22 ... thread 0: (1000000,1000000) ops and (339992.4,339992.4) ops/second in (2.941242,2.941242) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 8/6 566.99 MB 18.9 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 352.0 2.51 2.24 12 0.209 0 0 0.0 0.0 L1 1/0 2.09 MB 0.2 0.2 0.2 0.1 0.2 0.2 0.1 1.3 245.2 245.0 1.03 1.00 1 1.028 255K 189 0.0 0.0 L2 3/2 298.62 MB 1.2 0.1 0.1 0.0 0.1 0.1 0.2 1.8 229.6 229.5 0.49 0.46 1 0.486 113K 22 0.0 0.0 L3 17/10 818.43 MB 0.0 0.1 0.1 0.0 0.1 0.1 0.0 1.0 193.6 193.7 0.32 0.26 1 0.323 63K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1189/18 97.00 GB 0.0 0.4 0.3 0.1 1.3 1.2 0.4 1.5 98.1 301.2 4.34 3.96 15 0.289 432K 211 0.0 0.0 Int 0/0 0.00 KB 0.0 0.4 0.3 0.1 1.3 1.2 0.4 1.5 98.1 301.2 4.34 3.96 15 0.289 432K 211 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.4 0.3 0.1 0.4 0.3 0.0 0.0 232.0 231.9 1.84 1.72 3 0.612 432K 211 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 352.0 2.51 2.24 12 0.209 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.2 total, 2.9 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.28 GB write, 408.29 MB/s write, 0.42 GB read, 132.99 MB/s read, 4.3 seconds Interval compaction: 1.28 GB write, 444.33 MB/s write, 0.42 GB read, 144.73 MB/s read, 4.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 6, interval: 6 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 125457 Average: 2.0968 StdDev: 21.83 Min: 0 Median: 1.3564 Max: 4529 Percentiles: P50: 1.36 P75: 1.76 P99: 5.16 P99.9: 9.96 P99.99: 239.08 ------------------------------------------------------ [ 0, 1 ] 35095 27.974% 27.974% ###### ( 1, 2 ] 77536 61.803% 89.777% ############ ( 2, 3 ] 7669 6.113% 95.889% # ( 3, 4 ] 2989 2.382% 98.272% ( 4, 6 ] 1585 1.263% 99.535% ( 6, 10 ] 465 0.371% 99.906% ( 10, 15 ] 86 0.069% 99.974% ( 15, 22 ] 9 0.007% 99.982% ( 22, 34 ] 5 0.004% 99.986% ( 110, 170 ] 5 0.004% 99.990% ( 170, 250 ] 4 0.003% 99.993% ( 250, 380 ] 2 0.002% 99.994% ( 380, 580 ] 3 0.002% 99.997% ( 580, 870 ] 1 0.001% 99.998% ( 870, 1300 ] 2 0.002% 99.999% ( 1300, 1900 ] 1 0.001% 100.000% ( 1900, 2900 ] 3 0.002% 100.002% ( 2900, 4400 ] 2 0.002% 100.004% ( 4400, 6600 ] 1 0.001% 100.005% ** Level 1 read latency histogram (micros): Count: 15408 Average: 2.8621 StdDev: 11.21 Min: 0 Median: 1.8267 Max: 919 Percentiles: P50: 1.83 P75: 2.60 P99: 8.26 P99.9: 14.18 P99.99: 713.14 ------------------------------------------------------ [ 0, 1 ] 546 3.544% 3.544% # ( 1, 2 ] 8659 56.198% 59.742% ########### ( 2, 3 ] 3895 25.279% 85.021% ##### ( 3, 4 ] 885 5.744% 90.765% # ( 4, 6 ] 1132 7.347% 98.111% # ( 6, 10 ] 244 1.584% 99.695% ( 10, 15 ] 39 0.253% 99.948% ( 15, 22 ] 4 0.026% 99.974% ( 22, 34 ] 1 0.006% 99.981% ( 250, 380 ] 1 0.006% 99.987% ( 380, 580 ] 1 0.006% 99.994% ( 580, 870 ] 1 0.006% 100.000% ( 870, 1300 ] 1 0.006% 100.006% ** Level 2 read latency histogram (micros): Count: 28210 Average: 2.5510 StdDev: 42.48 Min: 1 Median: 1.2526 Max: 4910 Percentiles: P50: 1.25 P75: 1.70 P99: 5.90 P99.9: 187.84 P99.99: 631.91 ------------------------------------------------------ [ 0, 1 ] 10098 35.796% 35.796% ####### ( 1, 2 ] 15861 56.225% 92.021% ########### ( 2, 3 ] 1377 4.881% 96.902% # ( 3, 4 ] 424 1.503% 98.405% ( 4, 6 ] 177 0.627% 99.032% ( 6, 10 ] 151 0.535% 99.568% ( 10, 15 ] 44 0.156% 99.724% ( 15, 22 ] 3 0.011% 99.734% ( 22, 34 ] 1 0.004% 99.738% ( 34, 51 ] 1 0.004% 99.741% ( 51, 76 ] 29 0.103% 99.844% ( 76, 110 ] 6 0.021% 99.865% ( 110, 170 ] 6 0.021% 99.887% ( 170, 250 ] 17 0.060% 99.947% ( 250, 380 ] 10 0.035% 99.982% ( 380, 580 ] 2 0.007% 99.989% ( 580, 870 ] 1 0.004% 99.993% ( 4400, 6600 ] 2 0.007% 100.000% ** Level 3 read latency histogram (micros): Count: 31825 Average: 7.4134 StdDev: 126.41 Min: 1 Median: 1.6766 Max: 8251 Percentiles: P50: 1.68 P75: 2.32 P99: 10.73 P99.9: 882.54 P99.99: 5174.81 ------------------------------------------------------ [ 0, 1 ] 2625 8.248% 8.248% ## ( 1, 2 ] 19639 61.709% 69.958% ############ ( 2, 3 ] 5057 15.890% 85.848% ### ( 3, 4 ] 1796 5.643% 91.491% # ( 4, 6 ] 1696 5.329% 96.820% # ( 6, 10 ] 676 2.124% 98.944% ( 10, 15 ] 128 0.402% 99.346% ( 15, 22 ] 24 0.075% 99.422% ( 22, 34 ] 6 0.019% 99.441% ( 76, 110 ] 11 0.035% 99.475% ( 110, 170 ] 45 0.141% 99.617% ( 170, 250 ] 46 0.145% 99.761% ( 250, 380 ] 22 0.069% 99.830% ( 380, 580 ] 12 0.038% 99.868% ( 580, 870 ] 11 0.035% 99.903% ( 870, 1300 ] 6 0.019% 99.921% ( 1300, 1900 ] 3 0.009% 99.931% ( 1900, 2900 ] 5 0.016% 99.947% ( 2900, 4400 ] 12 0.038% 99.984% ( 4400, 6600 ] 8 0.025% 100.009% ( 6600, 9900 ] 1 0.003% 100.013% ** Level 4 read latency histogram (micros): Count: 30 Average: 3911.4333 StdDev: 1776.97 Min: 1370 Median: 3650.0000 Max: 7491 Percentiles: P50: 3650.00 P75: 5500.00 P99: 7491.00 P99.9: 7491.00 P99.99: 7491.00 ------------------------------------------------------ ( 1300, 1900 ] 3 10.000% 10.000% ## ( 1900, 2900 ] 10 33.333% 43.333% ####### ( 2900, 4400 ] 4 13.333% 56.667% ### ( 4400, 6600 ] 11 36.667% 93.333% ####### ( 6600, 9900 ] 2 6.667% 100.000% # ** DB Stats ** Uptime(secs): 3.2 total, 2.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 307.27 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.426 H:M:S, 13.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 334.31 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.426 H:M:S, 14.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/10/12-09:13:25 ... thread 0: (1000000,2000000) ops and (336473.1,338223.6) ops/second in (2.972006,5.913248) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/8 1.54 GB 25.2 0.9 0.0 0.9 2.7 1.8 0.0 1.5 101.0 317.4 8.73 8.08 29 0.301 895K 1134 0.0 0.0 L1 1/1 2.09 MB 0.0 0.2 0.2 0.1 0.2 0.2 0.1 1.3 245.2 245.0 1.03 1.00 1 1.028 255K 189 0.0 0.0 L2 1/1 120.90 MB 0.0 0.1 0.1 0.0 0.1 0.1 0.2 1.8 229.6 229.5 0.49 0.46 1 0.486 113K 22 0.0 0.0 L3 78/33 995.63 MB 0.9 1.0 0.2 0.8 1.0 0.2 0.0 4.4 209.9 209.8 5.01 4.31 3 1.671 1068K 556 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1251/43 97.98 GB 0.0 2.2 0.5 1.8 4.1 2.3 0.4 2.2 150.6 274.4 15.26 13.85 34 0.449 2333K 1901 0.0 0.0 Int 0/0 0.00 KB 0.0 2.2 0.5 1.8 4.1 2.3 0.4 2.2 150.6 274.4 15.26 13.85 34 0.449 2333K 1901 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.2 0.5 1.8 2.2 0.5 0.0 0.0 231.0 230.8 9.95 9.13 8 1.243 2333K 1901 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 356.0 5.31 4.72 26 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.2 total, 5.9 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 4.09 GB write, 677.92 MB/s write, 2.24 GB read, 372.16 MB/s read, 15.3 seconds Interval compaction: 4.09 GB write, 707.69 MB/s write, 2.24 GB read, 388.50 MB/s read, 15.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 10, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 10, interval: 10 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 346841 Average: 2.2884 StdDev: 15.10 Min: 0 Median: 1.5294 Max: 4529 Percentiles: P50: 1.53 P75: 1.92 P99: 5.68 P99.9: 10.51 P99.99: 188.95 ------------------------------------------------------ [ 0, 1 ] 54714 15.775% 15.775% ### ( 1, 2 ] 224221 64.647% 80.422% ############# ( 2, 3 ] 45501 13.119% 93.540% ### ( 3, 4 ] 11777 3.396% 96.936% # ( 4, 6 ] 8511 2.454% 99.390% ( 6, 10 ] 1741 0.502% 99.892% ( 10, 15 ] 298 0.086% 99.978% ( 15, 22 ] 21 0.006% 99.984% ( 22, 34 ] 9 0.003% 99.986% ( 51, 76 ] 4 0.001% 99.987% ( 110, 170 ] 7 0.002% 99.989% ( 170, 250 ] 14 0.004% 99.993% ( 250, 380 ] 4 0.001% 99.995% ( 380, 580 ] 10 0.003% 99.997% ( 580, 870 ] 7 0.002% 99.999% ( 870, 1300 ] 6 0.002% 100.001% ( 1300, 1900 ] 3 0.001% 100.002% ( 1900, 2900 ] 4 0.001% 100.003% ( 2900, 4400 ] 2 0.001% 100.004% ( 4400, 6600 ] 1 0.000% 100.004% ** Level 1 read latency histogram (micros): Count: 38787 Average: 2.6691 StdDev: 7.12 Min: 0 Median: 1.7912 Max: 919 Percentiles: P50: 1.79 P75: 2.51 P99: 7.19 P99.9: 13.48 P99.99: 265.77 ------------------------------------------------------ [ 0, 1 ] 1533 3.952% 3.952% # ( 1, 2 ] 22575 58.202% 62.155% ############ ( 2, 3 ] 9822 25.323% 87.478% ##### ( 3, 4 ] 1977 5.097% 92.575% # ( 4, 6 ] 2367 6.103% 98.677% # ( 6, 10 ] 420 1.083% 99.760% ( 10, 15 ] 78 0.201% 99.961% ( 15, 22 ] 10 0.026% 99.987% ( 22, 34 ] 1 0.003% 99.990% ( 250, 380 ] 1 0.003% 99.992% ( 380, 580 ] 1 0.003% 99.995% ( 580, 870 ] 1 0.003% 99.997% ( 870, 1300 ] 1 0.003% 100.000% ** Level 2 read latency histogram (micros): Count: 40626 Average: 2.3959 StdDev: 35.40 Min: 1 Median: 1.3334 Max: 4910 Percentiles: P50: 1.33 P75: 1.75 P99: 5.38 P99.9: 95.12 P99.99: 473.74 ------------------------------------------------------ [ 0, 1 ] 12138 29.877% 29.877% ###### ( 1, 2 ] 24519 60.353% 90.230% ############ ( 2, 3 ] 2553 6.284% 96.515% # ( 3, 4 ] 763 1.878% 98.393% ( 4, 6 ] 357 0.879% 99.271% ( 6, 10 ] 171 0.421% 99.692% ( 10, 15 ] 46 0.113% 99.806% ( 15, 22 ] 3 0.007% 99.813% ( 22, 34 ] 2 0.005% 99.818% ( 34, 51 ] 1 0.002% 99.820% ( 51, 76 ] 29 0.071% 99.892% ( 76, 110 ] 6 0.015% 99.906% ( 110, 170 ] 6 0.015% 99.921% ( 170, 250 ] 17 0.042% 99.963% ( 250, 380 ] 10 0.025% 99.988% ( 380, 580 ] 2 0.005% 99.993% ( 580, 870 ] 1 0.002% 99.995% ( 4400, 6600 ] 2 0.005% 100.000% ** Level 3 read latency histogram (micros): Count: 208197 Average: 3.7303 StdDev: 51.63 Min: 1 Median: 1.6302 Max: 8251 Percentiles: P50: 1.63 P75: 2.08 P99: 9.03 P99.9: 233.57 P99.99: 791.56 ------------------------------------------------------ [ 0, 1 ] 19996 9.604% 9.604% ## ( 1, 2 ] 133449 64.097% 73.702% ############# ( 2, 3 ] 31962 15.352% 89.054% ### ( 3, 4 ] 9904 4.757% 93.811% # ( 4, 6 ] 7703 3.700% 97.511% # ( 6, 10 ] 4091 1.965% 99.475% ( 10, 15 ] 358 0.172% 99.647% ( 15, 22 ] 49 0.024% 99.671% ( 22, 34 ] 17 0.008% 99.679% ( 34, 51 ] 5 0.002% 99.682% ( 51, 76 ] 1 0.000% 99.682% ( 76, 110 ] 74 0.036% 99.718% ( 110, 170 ] 263 0.126% 99.844% ( 170, 250 ] 147 0.071% 99.915% ( 250, 380 ] 77 0.037% 99.951% ( 380, 580 ] 51 0.024% 99.976% ( 580, 870 ] 40 0.019% 99.995% ( 870, 1300 ] 13 0.006% 100.001% ( 1300, 1900 ] 4 0.002% 100.003% ( 1900, 2900 ] 5 0.002% 100.006% ( 2900, 4400 ] 12 0.006% 100.012% ( 4400, 6600 ] 8 0.004% 100.015% ( 6600, 9900 ] 1 0.000% 100.016% ** Level 4 read latency histogram (micros): Count: 30 Average: 3911.4333 StdDev: 1776.97 Min: 1370 Median: 3650.0000 Max: 7491 Percentiles: P50: 3650.00 P75: 5500.00 P99: 7491.00 P99.9: 7491.00 P99.99: 7491.00 ------------------------------------------------------ ( 1300, 1900 ] 3 10.000% 10.000% ## ( 1900, 2900 ] 10 33.333% 43.333% ####### ( 2900, 4400 ] 4 13.333% 56.667% ### ( 4400, 6600 ] 11 36.667% 93.333% ####### ( 6600, 9900 ] 2 6.667% 100.000% # ** DB Stats ** Uptime(secs): 6.2 total, 3.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 318.77 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.795 H:M:S, 12.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 331.16 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.369 H:M:S, 12.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/12-09:13:28 ... thread 0: (1000000,3000000) ops and (333123.6,336506.3) ops/second in (3.001889,8.915137) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/16 2.46 GB 56.6 1.4 0.0 1.4 4.2 2.8 0.0 1.5 106.8 315.6 13.57 12.60 44 0.308 1471K 1753 0.0 0.0 L1 1/1 2.09 MB 0.0 0.2 0.2 0.1 0.2 0.2 0.1 1.3 245.2 245.0 1.03 1.00 1 1.028 255K 189 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.2 1.8 229.6 229.5 0.49 0.46 1 0.486 113K 22 0.0 0.0 L3 29/0 999.64 MB 1.0 1.2 0.4 0.9 1.2 0.4 0.0 3.5 211.7 211.6 5.99 5.23 4 1.498 1263K 556 0.0 0.0 L4 140/0 9.65 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1232/17 98.90 GB 0.0 3.0 0.6 2.4 5.8 3.4 0.5 2.1 146.2 280.6 21.07 19.28 50 0.421 3103K 2520 0.0 0.0 Int 0/0 0.00 KB 0.0 3.0 0.6 2.4 5.8 3.4 0.5 2.1 146.2 280.6 21.07 19.28 50 0.421 3103K 2520 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.0 0.6 2.4 3.0 0.6 0.0 0.0 234.8 234.6 13.12 12.20 11 1.193 3103K 2520 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 356.4 7.95 7.08 39 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.2 total, 8.9 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.77 GB write, 644.34 MB/s write, 3.01 GB read, 335.70 MB/s read, 21.1 seconds Interval compaction: 5.77 GB write, 663.11 MB/s write, 3.01 GB read, 345.48 MB/s read, 21.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 15, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 15, interval: 15 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 747888 Average: 2.3281 StdDev: 11.50 Min: 0 Median: 1.5811 Max: 4529 Percentiles: P50: 1.58 P75: 1.96 P99: 5.71 P99.9: 9.80 P99.99: 14.53 ------------------------------------------------------ [ 0, 1 ] 88399 11.820% 11.820% ## ( 1, 2 ] 491410 65.706% 77.526% ############# ( 2, 3 ] 115074 15.387% 92.913% ### ( 3, 4 ] 28028 3.748% 96.660% # ( 4, 6 ] 20508 2.742% 99.402% # ( 6, 10 ] 3920 0.524% 99.927% ( 10, 15 ] 527 0.070% 99.997% ( 15, 22 ] 50 0.007% 100.004% ( 22, 34 ] 16 0.002% 100.006% ( 34, 51 ] 1 0.000% 100.006% ( 51, 76 ] 7 0.001% 100.007% ( 76, 110 ] 13 0.002% 100.009% ( 110, 170 ] 19 0.003% 100.011% ( 170, 250 ] 27 0.004% 100.015% ( 250, 380 ] 10 0.001% 100.016% ( 380, 580 ] 17 0.002% 100.018% ( 580, 870 ] 14 0.002% 100.020% ( 870, 1300 ] 10 0.001% 100.022% ( 1300, 1900 ] 4 0.001% 100.022% ( 1900, 2900 ] 5 0.001% 100.023% ( 2900, 4400 ] 2 0.000% 100.023% ( 4400, 6600 ] 1 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 63396 Average: 2.5617 StdDev: 5.61 Min: 0 Median: 1.7452 Max: 919 Percentiles: P50: 1.75 P75: 2.41 P99: 6.01 P99.9: 12.39 P99.99: 29.92 ------------------------------------------------------ [ 0, 1 ] 2942 4.641% 4.641% # ( 1, 2 ] 38588 60.868% 65.509% ############ ( 2, 3 ] 14855 23.432% 88.941% ##### ( 3, 4 ] 3066 4.836% 93.777% # ( 4, 6 ] 3310 5.221% 98.998% # ( 6, 10 ] 530 0.836% 99.834% ( 10, 15 ] 87 0.137% 99.972% ( 15, 22 ] 11 0.017% 99.989% ( 22, 34 ] 1 0.002% 99.991% ( 51, 76 ] 2 0.003% 99.994% ( 250, 380 ] 1 0.002% 99.995% ( 380, 580 ] 1 0.002% 99.997% ( 580, 870 ] 1 0.002% 99.998% ( 870, 1300 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 40626 Average: 2.3959 StdDev: 35.40 Min: 1 Median: 1.3334 Max: 4910 Percentiles: P50: 1.33 P75: 1.75 P99: 5.38 P99.9: 95.12 P99.99: 473.74 ------------------------------------------------------ [ 0, 1 ] 12138 29.877% 29.877% ###### ( 1, 2 ] 24519 60.353% 90.230% ############ ( 2, 3 ] 2553 6.284% 96.515% # ( 3, 4 ] 763 1.878% 98.393% ( 4, 6 ] 357 0.879% 99.271% ( 6, 10 ] 171 0.421% 99.692% ( 10, 15 ] 46 0.113% 99.806% ( 15, 22 ] 3 0.007% 99.813% ( 22, 34 ] 2 0.005% 99.818% ( 34, 51 ] 1 0.002% 99.820% ( 51, 76 ] 29 0.071% 99.892% ( 76, 110 ] 6 0.015% 99.906% ( 110, 170 ] 6 0.015% 99.921% ( 170, 250 ] 17 0.042% 99.963% ( 250, 380 ] 10 0.025% 99.988% ( 380, 580 ] 2 0.005% 99.993% ( 580, 870 ] 1 0.002% 99.995% ( 4400, 6600 ] 2 0.005% 100.000% ** Level 3 read latency histogram (micros): Count: 230760 Average: 3.6106 StdDev: 49.10 Min: 1 Median: 1.6359 Max: 8251 Percentiles: P50: 1.64 P75: 2.11 P99: 8.87 P99.9: 223.28 P99.99: 782.45 ------------------------------------------------------ [ 0, 1 ] 21374 9.262% 9.262% ## ( 1, 2 ] 147836 64.065% 73.327% ############# ( 2, 3 ] 36648 15.881% 89.209% ### ( 3, 4 ] 11017 4.774% 93.983% # ( 4, 6 ] 8555 3.707% 97.690% # ( 6, 10 ] 4206 1.823% 99.513% ( 10, 15 ] 381 0.165% 99.678% ( 15, 22 ] 52 0.023% 99.701% ( 22, 34 ] 18 0.008% 99.708% ( 34, 51 ] 5 0.002% 99.711% ( 51, 76 ] 1 0.000% 99.711% ( 76, 110 ] 74 0.032% 99.743% ( 110, 170 ] 263 0.114% 99.857% ( 170, 250 ] 149 0.065% 99.922% ( 250, 380 ] 78 0.034% 99.955% ( 380, 580 ] 52 0.023% 99.978% ( 580, 870 ] 40 0.017% 99.995% ( 870, 1300 ] 14 0.006% 100.001% ( 1300, 1900 ] 4 0.002% 100.003% ( 1900, 2900 ] 5 0.002% 100.005% ( 2900, 4400 ] 12 0.005% 100.010% ( 4400, 6600 ] 8 0.003% 100.014% ( 6600, 9900 ] 1 0.000% 100.014% ** Level 4 read latency histogram (micros): Count: 30 Average: 3911.4333 StdDev: 1776.97 Min: 1370 Median: 3650.0000 Max: 7491 Percentiles: P50: 3650.00 P75: 5500.00 P99: 7491.00 P99.9: 7491.00 P99.99: 7491.00 ------------------------------------------------------ ( 1300, 1900 ] 3 10.000% 10.000% ## ( 1900, 2900 ] 10 33.333% 43.333% ####### ( 2900, 4400 ] 4 13.333% 56.667% ### ( 4400, 6600 ] 11 36.667% 93.333% ####### ( 6600, 9900 ] 2 6.667% 100.000% # ** DB Stats ** Uptime(secs): 9.2 total, 3.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 321.74 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.204 H:M:S, 13.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 327.86 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.409 H:M:S, 13.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/10/12-09:13:31 ... thread 0: (1000000,4000000) ops and (331295.8,335188.4) ops/second in (3.018451,11.933588) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/4 2.09 GB 188.8 2.5 0.0 2.5 6.2 3.7 0.0 1.7 124.4 306.4 20.76 19.31 63 0.329 2622K 2943 0.0 0.0 L1 11/0 1.29 GB 131.6 1.5 1.5 0.1 1.5 1.5 0.1 1.0 228.9 227.8 6.88 6.67 2 3.442 1600K 7593 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.2 1.8 229.6 229.5 0.49 0.46 1 0.486 113K 22 0.0 0.0 L3 29/0 999.64 MB 1.0 1.2 0.4 0.9 1.2 0.4 0.0 3.5 211.7 211.6 5.99 5.23 4 1.498 1263K 556 0.0 0.0 L4 140/0 9.65 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1237/4 99.82 GB 0.0 5.4 1.9 3.5 9.1 5.6 0.5 2.5 162.3 272.8 34.12 31.67 70 0.487 5599K 11K 0.0 0.0 Int 0/0 0.00 KB 0.0 5.4 1.9 3.5 9.1 5.6 0.5 2.5 162.3 272.8 34.12 31.67 70 0.487 5599K 11K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.4 1.9 3.5 5.4 1.9 0.0 0.0 236.2 235.7 23.45 22.22 16 1.466 5599K 11K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 354.4 10.67 9.44 54 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.2 total, 11.9 interval Flush(GB): cumulative 3.691, interval 3.691 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 9.09 GB write, 763.16 MB/s write, 5.41 GB read, 454.16 MB/s read, 34.1 seconds Interval compaction: 9.09 GB write, 779.78 MB/s write, 5.41 GB read, 464.04 MB/s read, 34.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 3, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 3, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 16, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 3, total-stops: 16, interval: 19 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1118287 Average: 2.3515 StdDev: 12.77 Min: 0 Median: 1.5771 Max: 4529 Percentiles: P50: 1.58 P75: 1.96 P99: 5.72 P99.9: 9.83 P99.99: 17.23 ------------------------------------------------------ [ 0, 1 ] 133392 11.928% 11.928% ## ( 1, 2 ] 737792 65.975% 77.903% ############# ( 2, 3 ] 170289 15.228% 93.131% ### ( 3, 4 ] 40357 3.609% 96.740% # ( 4, 6 ] 29350 2.625% 99.364% # ( 6, 10 ] 6248 0.559% 99.923% ( 10, 15 ] 724 0.065% 99.988% ( 15, 22 ] 76 0.007% 99.995% ( 22, 34 ] 24 0.002% 99.997% ( 34, 51 ] 6 0.001% 99.997% ( 51, 76 ] 11 0.001% 99.998% ( 76, 110 ] 53 0.005% 100.003% ( 110, 170 ] 69 0.006% 100.009% ( 170, 250 ] 66 0.006% 100.015% ( 250, 380 ] 34 0.003% 100.018% ( 380, 580 ] 29 0.003% 100.021% ( 580, 870 ] 30 0.003% 100.024% ( 870, 1300 ] 18 0.002% 100.025% ( 1300, 1900 ] 5 0.000% 100.026% ( 1900, 2900 ] 7 0.001% 100.026% ( 2900, 4400 ] 5 0.000% 100.027% ( 4400, 6600 ] 1 0.000% 100.027% ** Level 1 read latency histogram (micros): Count: 63981 Average: 3.0466 StdDev: 20.06 Min: 0 Median: 1.7450 Max: 1958 Percentiles: P50: 1.75 P75: 2.41 P99: 6.66 P99.9: 144.33 P99.99: 1360.19 ------------------------------------------------------ [ 0, 1 ] 2995 4.681% 4.681% # ( 1, 2 ] 38919 60.829% 65.510% ############ ( 2, 3 ] 14937 23.346% 88.856% ##### ( 3, 4 ] 3086 4.823% 93.679% # ( 4, 6 ] 3315 5.181% 98.861% # ( 6, 10 ] 539 0.842% 99.703% ( 10, 15 ] 87 0.136% 99.839% ( 15, 22 ] 11 0.017% 99.856% ( 22, 34 ] 1 0.002% 99.858% ( 51, 76 ] 2 0.003% 99.861% ( 76, 110 ] 9 0.014% 99.875% ( 110, 170 ] 28 0.044% 99.919% ( 170, 250 ] 9 0.014% 99.933% ( 250, 380 ] 14 0.022% 99.955% ( 380, 580 ] 15 0.023% 99.978% ( 580, 870 ] 3 0.005% 99.983% ( 870, 1300 ] 4 0.006% 99.989% ( 1300, 1900 ] 6 0.009% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 40626 Average: 2.3959 StdDev: 35.40 Min: 1 Median: 1.3334 Max: 4910 Percentiles: P50: 1.33 P75: 1.75 P99: 5.38 P99.9: 95.12 P99.99: 473.74 ------------------------------------------------------ [ 0, 1 ] 12138 29.877% 29.877% ###### ( 1, 2 ] 24519 60.353% 90.230% ############ ( 2, 3 ] 2553 6.284% 96.515% # ( 3, 4 ] 763 1.878% 98.393% ( 4, 6 ] 357 0.879% 99.271% ( 6, 10 ] 171 0.421% 99.692% ( 10, 15 ] 46 0.113% 99.806% ( 15, 22 ] 3 0.007% 99.813% ( 22, 34 ] 2 0.005% 99.818% ( 34, 51 ] 1 0.002% 99.820% ( 51, 76 ] 29 0.071% 99.892% ( 76, 110 ] 6 0.015% 99.906% ( 110, 170 ] 6 0.015% 99.921% ( 170, 250 ] 17 0.042% 99.963% ( 250, 380 ] 10 0.025% 99.988% ( 380, 580 ] 2 0.005% 99.993% ( 580, 870 ] 1 0.002% 99.995% ( 4400, 6600 ] 2 0.005% 100.000% ** Level 3 read latency histogram (micros): Count: 230760 Average: 3.6106 StdDev: 49.10 Min: 1 Median: 1.6359 Max: 8251 Percentiles: P50: 1.64 P75: 2.11 P99: 8.87 P99.9: 223.28 P99.99: 782.45 ------------------------------------------------------ [ 0, 1 ] 21374 9.262% 9.262% ## ( 1, 2 ] 147836 64.065% 73.327% ############# ( 2, 3 ] 36648 15.881% 89.209% ### ( 3, 4 ] 11017 4.774% 93.983% # ( 4, 6 ] 8555 3.707% 97.690% # ( 6, 10 ] 4206 1.823% 99.513% ( 10, 15 ] 381 0.165% 99.678% ( 15, 22 ] 52 0.023% 99.701% ( 22, 34 ] 18 0.008% 99.708% ( 34, 51 ] 5 0.002% 99.711% ( 51, 76 ] 1 0.000% 99.711% ( 76, 110 ] 74 0.032% 99.743% ( 110, 170 ] 263 0.114% 99.857% ( 170, 250 ] 149 0.065% 99.922% ( 250, 380 ] 78 0.034% 99.955% ( 380, 580 ] 52 0.023% 99.978% ( 580, 870 ] 40 0.017% 99.995% ( 870, 1300 ] 14 0.006% 100.001% ( 1300, 1900 ] 4 0.002% 100.003% ( 1900, 2900 ] 5 0.002% 100.005% ( 2900, 4400 ] 12 0.005% 100.010% ( 4400, 6600 ] 8 0.003% 100.014% ( 6600, 9900 ] 1 0.000% 100.014% ** Level 4 read latency histogram (micros): Count: 30 Average: 3911.4333 StdDev: 1776.97 Min: 1370 Median: 3650.0000 Max: 7491 Percentiles: P50: 3650.00 P75: 5500.00 P99: 7491.00 P99.9: 7491.00 P99.99: 7491.00 ------------------------------------------------------ ( 1300, 1900 ] 3 10.000% 10.000% ## ( 1900, 2900 ] 10 33.333% 43.333% ####### ( 2900, 4400 ] 4 13.333% 56.667% ### ( 4400, 6600 ] 11 36.667% 93.333% ####### ( 6600, 9900 ] 2 6.667% 100.000% # ** DB Stats ** Uptime(secs): 12.2 total, 3.0 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 322.81 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.608 H:M:S, 13.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 326.06 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.404 H:M:S, 13.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/10/12-09:13:35 ... thread 0: (1000000,5000000) ops and (288637.4,324714.5) ops/second in (3.464555,15.398143) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/14 3.13 GB 81.8 3.3 0.0 3.3 8.1 4.7 0.0 1.7 126.8 307.4 26.84 25.00 80 0.335 3454K 3797 0.0 0.0 L1 11/11 1.29 GB 0.0 1.5 1.5 0.1 1.5 1.5 0.1 1.0 228.9 227.8 6.88 6.67 2 3.442 1600K 7593 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.2 1.8 229.6 229.5 0.49 0.46 1 0.486 113K 22 0.0 0.0 L3 29/0 999.64 MB 1.0 1.2 0.4 0.9 1.2 0.4 0.0 3.5 211.7 211.6 5.99 5.23 4 1.498 1263K 556 0.0 0.0 L4 140/0 9.65 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1242/25 100.86 GB 0.0 6.2 1.9 4.3 10.9 6.6 0.5 2.3 158.2 278.5 40.20 37.35 87 0.462 6430K 11K 0.0 0.0 Int 0/0 0.00 KB 0.0 6.2 1.9 4.3 10.9 6.6 0.5 2.3 158.2 278.5 40.20 37.35 87 0.462 6430K 11K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.2 1.9 4.3 6.2 1.9 0.0 0.0 239.4 238.9 26.56 25.27 19 1.398 6430K 11K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 355.6 13.64 12.08 68 0.201 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.7 total, 15.4 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.93 GB write, 714.95 MB/s write, 6.21 GB read, 405.98 MB/s read, 40.2 seconds Interval compaction: 10.93 GB write, 727.01 MB/s write, 6.21 GB read, 412.83 MB/s read, 40.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 5, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 5, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 21, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 5, total-stops: 21, interval: 26 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1435403 Average: 2.3492 StdDev: 12.89 Min: 0 Median: 1.5734 Max: 4529 Percentiles: P50: 1.57 P75: 1.95 P99: 5.69 P99.9: 9.84 P99.99: 25.90 ------------------------------------------------------ [ 0, 1 ] 172302 12.004% 12.004% ## ( 1, 2 ] 951243 66.270% 78.274% ############# ( 2, 3 ] 216001 15.048% 93.322% ### ( 3, 4 ] 50660 3.529% 96.851% # ( 4, 6 ] 36504 2.543% 99.394% # ( 6, 10 ] 7564 0.527% 99.921% ( 10, 15 ] 877 0.061% 99.982% ( 15, 22 ] 101 0.007% 99.989% ( 22, 34 ] 26 0.002% 99.991% ( 34, 51 ] 8 0.001% 99.992% ( 51, 76 ] 12 0.001% 99.993% ( 76, 110 ] 96 0.007% 99.999% ( 110, 170 ] 120 0.008% 100.008% ( 170, 250 ] 83 0.006% 100.014% ( 250, 380 ] 44 0.003% 100.017% ( 380, 580 ] 39 0.003% 100.019% ( 580, 870 ] 40 0.003% 100.022% ( 870, 1300 ] 23 0.002% 100.024% ( 1300, 1900 ] 7 0.000% 100.024% ( 1900, 2900 ] 9 0.001% 100.025% ( 2900, 4400 ] 7 0.000% 100.025% ( 4400, 6600 ] 1 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 133523 Average: 2.8418 StdDev: 15.66 Min: 0 Median: 1.6448 Max: 1958 Percentiles: P50: 1.64 P75: 2.13 P99: 7.85 P99.9: 138.72 P99.99: 772.55 ------------------------------------------------------ [ 0, 1 ] 11776 8.819% 8.819% ## ( 1, 2 ] 85275 63.865% 72.685% ############# ( 2, 3 ] 24626 18.443% 91.128% #### ( 3, 4 ] 4967 3.720% 94.848% # ( 4, 6 ] 4800 3.595% 98.443% # ( 6, 10 ] 1612 1.207% 99.650% ( 10, 15 ] 160 0.120% 99.770% ( 15, 22 ] 21 0.016% 99.786% ( 22, 34 ] 2 0.001% 99.787% ( 34, 51 ] 1 0.001% 99.788% ( 51, 76 ] 2 0.001% 99.790% ( 76, 110 ] 102 0.076% 99.866% ( 110, 170 ] 95 0.071% 99.937% ( 170, 250 ] 23 0.017% 99.954% ( 250, 380 ] 20 0.015% 99.969% ( 380, 580 ] 23 0.017% 99.987% ( 580, 870 ] 7 0.005% 99.992% ( 870, 1300 ] 4 0.003% 99.995% ( 1300, 1900 ] 6 0.004% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 40626 Average: 2.3959 StdDev: 35.40 Min: 1 Median: 1.3334 Max: 4910 Percentiles: P50: 1.33 P75: 1.75 P99: 5.38 P99.9: 95.12 P99.99: 473.74 ------------------------------------------------------ [ 0, 1 ] 12138 29.877% 29.877% ###### ( 1, 2 ] 24519 60.353% 90.230% ############ ( 2, 3 ] 2553 6.284% 96.515% # ( 3, 4 ] 763 1.878% 98.393% ( 4, 6 ] 357 0.879% 99.271% ( 6, 10 ] 171 0.421% 99.692% ( 10, 15 ] 46 0.113% 99.806% ( 15, 22 ] 3 0.007% 99.813% ( 22, 34 ] 2 0.005% 99.818% ( 34, 51 ] 1 0.002% 99.820% ( 51, 76 ] 29 0.071% 99.892% ( 76, 110 ] 6 0.015% 99.906% ( 110, 170 ] 6 0.015% 99.921% ( 170, 250 ] 17 0.042% 99.963% ( 250, 380 ] 10 0.025% 99.988% ( 380, 580 ] 2 0.005% 99.993% ( 580, 870 ] 1 0.002% 99.995% ( 4400, 6600 ] 2 0.005% 100.000% ** Level 3 read latency histogram (micros): Count: 230760 Average: 3.6106 StdDev: 49.10 Min: 1 Median: 1.6359 Max: 8251 Percentiles: P50: 1.64 P75: 2.11 P99: 8.87 P99.9: 223.28 P99.99: 782.45 ------------------------------------------------------ [ 0, 1 ] 21374 9.262% 9.262% ## ( 1, 2 ] 147836 64.065% 73.327% ############# ( 2, 3 ] 36648 15.881% 89.209% ### ( 3, 4 ] 11017 4.774% 93.983% # ( 4, 6 ] 8555 3.707% 97.690% # ( 6, 10 ] 4206 1.823% 99.513% ( 10, 15 ] 381 0.165% 99.678% ( 15, 22 ] 52 0.023% 99.701% ( 22, 34 ] 18 0.008% 99.708% ( 34, 51 ] 5 0.002% 99.711% ( 51, 76 ] 1 0.000% 99.711% ( 76, 110 ] 74 0.032% 99.743% ( 110, 170 ] 263 0.114% 99.857% ( 170, 250 ] 149 0.065% 99.922% ( 250, 380 ] 78 0.034% 99.955% ( 380, 580 ] 52 0.023% 99.978% ( 580, 870 ] 40 0.017% 99.995% ( 870, 1300 ] 14 0.006% 100.001% ( 1300, 1900 ] 4 0.002% 100.003% ( 1900, 2900 ] 5 0.002% 100.005% ( 2900, 4400 ] 12 0.005% 100.010% ( 4400, 6600 ] 8 0.003% 100.014% ( 6600, 9900 ] 1 0.000% 100.014% ** Level 4 read latency histogram (micros): Count: 30 Average: 3911.4333 StdDev: 1776.97 Min: 1370 Median: 3650.0000 Max: 7491 Percentiles: P50: 3650.00 P75: 5500.00 P99: 7491.00 P99.9: 7491.00 P99.99: 7491.00 ------------------------------------------------------ ( 1300, 1900 ] 3 10.000% 10.000% ## ( 1900, 2900 ] 10 33.333% 43.333% ####### ( 2900, 4400 ] 4 13.333% 56.667% ### ( 4400, 6600 ] 11 36.667% 93.333% ####### ( 6600, 9900 ] 2 6.667% 100.000% # ** DB Stats ** Uptime(secs): 15.7 total, 3.5 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 314.24 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.444 H:M:S, 15.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 284.07 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.836 H:M:S, 24.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/12-09:13:42 ... thread 0: (1000000,6000000) ops and (140686.1,266593.6) ops/second in (7.108021,22.506164) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/10 4.05 GB 201.3 5.0 0.0 5.0 10.6 5.7 0.0 1.9 141.9 302.9 35.96 33.74 100 0.360 5180K 6509 0.0 0.0 L1 11/11 1.29 GB 0.0 1.5 1.5 0.1 1.5 1.5 0.1 1.0 228.9 227.8 6.88 6.67 2 3.442 1600K 7593 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.2 1.8 229.6 229.5 0.49 0.46 1 0.486 113K 22 0.0 0.0 L3 29/0 999.64 MB 1.0 1.2 0.4 0.9 1.2 0.4 0.0 3.5 211.7 211.6 5.99 5.23 4 1.498 1263K 556 0.0 0.0 L4 140/0 9.65 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1242/21 101.78 GB 0.0 7.9 1.9 6.0 13.5 7.5 0.5 2.4 163.4 280.6 49.32 46.09 107 0.461 8157K 14K 0.0 0.0 Int 0/0 0.00 KB 0.0 7.9 1.9 6.0 13.5 7.5 0.5 2.4 163.4 280.6 49.32 46.09 107 0.461 8157K 14K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 7.9 1.9 6.0 7.9 1.9 0.0 0.0 244.3 243.8 32.99 31.61 24 1.374 8157K 14K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 354.9 16.33 14.48 83 0.197 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 22.8 total, 22.5 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.51 GB write, 607.82 MB/s write, 7.87 GB read, 353.92 MB/s read, 49.3 seconds Interval compaction: 13.51 GB write, 614.83 MB/s write, 7.87 GB read, 358.01 MB/s read, 49.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 21, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 21, interval: 34 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2071782 Average: 2.3469 StdDev: 12.14 Min: 0 Median: 1.5619 Max: 4529 Percentiles: P50: 1.56 P75: 1.94 P99: 5.69 P99.9: 9.94 P99.99: 131.76 ------------------------------------------------------ [ 0, 1 ] 256145 12.364% 12.364% ## ( 1, 2 ] 1387722 66.982% 79.346% ############# ( 2, 3 ] 299646 14.463% 93.809% ### ( 3, 4 ] 67158 3.242% 97.050% # ( 4, 6 ] 47885 2.311% 99.362% ( 6, 10 ] 11317 0.546% 99.908% ( 10, 15 ] 1153 0.056% 99.964% ( 15, 22 ] 156 0.008% 99.971% ( 22, 34 ] 39 0.002% 99.973% ( 34, 51 ] 12 0.001% 99.974% ( 51, 76 ] 14 0.001% 99.974% ( 76, 110 ] 219 0.011% 99.985% ( 110, 170 ] 300 0.014% 99.999% ( 170, 250 ] 173 0.008% 100.008% ( 250, 380 ] 82 0.004% 100.012% ( 380, 580 ] 85 0.004% 100.016% ( 580, 870 ] 62 0.003% 100.019% ( 870, 1300 ] 31 0.001% 100.020% ( 1300, 1900 ] 10 0.000% 100.021% ( 1900, 2900 ] 13 0.001% 100.021% ( 2900, 4400 ] 7 0.000% 100.022% ( 4400, 6600 ] 1 0.000% 100.022% ** Level 1 read latency histogram (micros): Count: 286061 Average: 2.6405 StdDev: 12.01 Min: 0 Median: 1.5956 Max: 1958 Percentiles: P50: 1.60 P75: 1.97 P99: 7.83 P99.9: 120.14 P99.99: 532.46 ------------------------------------------------------ [ 0, 1 ] 30507 10.665% 10.665% ## ( 1, 2 ] 188912 66.039% 76.704% ############# ( 2, 3 ] 45523 15.914% 92.617% ### ( 3, 4 ] 8718 3.048% 95.665% # ( 4, 6 ] 7841 2.741% 98.406% # ( 6, 10 ] 3710 1.297% 99.703% ( 10, 15 ] 270 0.094% 99.797% ( 15, 22 ] 36 0.013% 99.810% ( 22, 34 ] 4 0.001% 99.811% ( 34, 51 ] 1 0.000% 99.812% ( 51, 76 ] 2 0.001% 99.812% ( 76, 110 ] 220 0.077% 99.889% ( 110, 170 ] 183 0.064% 99.953% ( 170, 250 ] 48 0.017% 99.970% ( 250, 380 ] 33 0.012% 99.981% ( 380, 580 ] 32 0.011% 99.993% ( 580, 870 ] 10 0.003% 99.996% ( 870, 1300 ] 4 0.001% 99.998% ( 1300, 1900 ] 6 0.002% 100.000% ( 1900, 2900 ] 1 0.000% 100.000% ** Level 2 read latency histogram (micros): Count: 40626 Average: 2.3959 StdDev: 35.40 Min: 1 Median: 1.3334 Max: 4910 Percentiles: P50: 1.33 P75: 1.75 P99: 5.38 P99.9: 95.12 P99.99: 473.74 ------------------------------------------------------ [ 0, 1 ] 12138 29.877% 29.877% ###### ( 1, 2 ] 24519 60.353% 90.230% ############ ( 2, 3 ] 2553 6.284% 96.515% # ( 3, 4 ] 763 1.878% 98.393% ( 4, 6 ] 357 0.879% 99.271% ( 6, 10 ] 171 0.421% 99.692% ( 10, 15 ] 46 0.113% 99.806% ( 15, 22 ] 3 0.007% 99.813% ( 22, 34 ] 2 0.005% 99.818% ( 34, 51 ] 1 0.002% 99.820% ( 51, 76 ] 29 0.071% 99.892% ( 76, 110 ] 6 0.015% 99.906% ( 110, 170 ] 6 0.015% 99.921% ( 170, 250 ] 17 0.042% 99.963% ( 250, 380 ] 10 0.025% 99.988% ( 380, 580 ] 2 0.005% 99.993% ( 580, 870 ] 1 0.002% 99.995% ( 4400, 6600 ] 2 0.005% 100.000% ** Level 3 read latency histogram (micros): Count: 230760 Average: 3.6106 StdDev: 49.10 Min: 1 Median: 1.6359 Max: 8251 Percentiles: P50: 1.64 P75: 2.11 P99: 8.87 P99.9: 223.28 P99.99: 782.45 ------------------------------------------------------ [ 0, 1 ] 21374 9.262% 9.262% ## ( 1, 2 ] 147836 64.065% 73.327% ############# ( 2, 3 ] 36648 15.881% 89.209% ### ( 3, 4 ] 11017 4.774% 93.983% # ( 4, 6 ] 8555 3.707% 97.690% # ( 6, 10 ] 4206 1.823% 99.513% ( 10, 15 ] 381 0.165% 99.678% ( 15, 22 ] 52 0.023% 99.701% ( 22, 34 ] 18 0.008% 99.708% ( 34, 51 ] 5 0.002% 99.711% ( 51, 76 ] 1 0.000% 99.711% ( 76, 110 ] 74 0.032% 99.743% ( 110, 170 ] 263 0.114% 99.857% ( 170, 250 ] 149 0.065% 99.922% ( 250, 380 ] 78 0.034% 99.955% ( 380, 580 ] 52 0.023% 99.978% ( 580, 870 ] 40 0.017% 99.995% ( 870, 1300 ] 14 0.006% 100.001% ( 1300, 1900 ] 4 0.002% 100.003% ( 1900, 2900 ] 5 0.002% 100.005% ( 2900, 4400 ] 12 0.005% 100.010% ( 4400, 6600 ] 8 0.003% 100.014% ( 6600, 9900 ] 1 0.000% 100.014% ** Level 4 read latency histogram (micros): Count: 30 Average: 3911.4333 StdDev: 1776.97 Min: 1370 Median: 3650.0000 Max: 7491 Percentiles: P50: 3650.00 P75: 5500.00 P99: 7491.00 P99.9: 7491.00 P99.99: 7491.00 ------------------------------------------------------ ( 1300, 1900 ] 3 10.000% 10.000% ## ( 1900, 2900 ] 10 33.333% 43.333% ####### ( 2900, 4400 ] 4 13.333% 56.667% ### ( 4400, 6600 ] 11 36.667% 93.333% ####### ( 6600, 9900 ] 2 6.667% 100.000% # ** DB Stats ** Uptime(secs): 22.8 total, 7.1 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 259.36 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.714 H:M:S, 29.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 138.46 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.270 H:M:S, 60.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/10/12-09:13:49 ... thread 0: (1000000,7000000) ops and (128547.7,231134.7) ops/second in (7.779212,30.285376) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/13 2.89 GB 0.0 6.3 0.0 6.3 12.9 6.6 0.0 1.9 144.8 298.0 44.36 41.39 118 0.376 6523K 9372 0.0 0.0 L1 16/16 1.50 GB 0.0 4.9 3.6 1.3 4.9 3.5 0.1 1.4 220.1 217.6 22.87 21.61 3 7.622 5109K 57K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 2.1 1.8 229.6 229.5 0.49 0.46 1 0.486 113K 22 0.0 0.0 L3 14/0 996.78 MB 1.0 1.7 0.8 0.9 1.7 0.8 1.4 2.2 186.5 186.2 9.27 7.44 12 0.773 1691K 556 0.0 0.0 L4 170/0 9.51 GB 1.0 0.7 0.7 0.1 0.7 0.6 1.3 1.1 162.1 161.7 4.46 3.42 6 0.744 735K 1740 0.0 0.0 L5 1068/0 87.78 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1281/29 102.65 GB 0.0 13.7 5.0 8.7 20.3 11.6 6.9 3.1 172.1 254.8 81.45 74.31 140 0.582 14M 68K 0.0 0.0 Int 0/0 0.00 KB 0.0 13.7 5.0 8.7 20.3 11.6 6.9 3.1 172.1 254.8 81.45 74.31 140 0.582 14M 68K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.7 5.0 8.7 13.6 5.0 0.0 0.0 225.7 224.5 62.13 57.27 42 1.479 14M 68K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 352.2 19.32 17.04 98 0.197 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 30.5 total, 30.3 interval Flush(GB): cumulative 6.644, interval 6.644 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 20.27 GB write, 679.41 MB/s write, 13.69 GB read, 458.99 MB/s read, 81.4 seconds Interval compaction: 20.27 GB write, 685.24 MB/s write, 13.69 GB read, 462.93 MB/s read, 81.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 19, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 19, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 22, pending-compaction-bytes-delays: 14, pending-compaction-bytes-stops: 0, total-delays: 33, total-stops: 22, interval: 55 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2648993 Average: 2.6733 StdDev: 17.34 Min: 0 Median: 1.5655 Max: 5614 Percentiles: P50: 1.57 P75: 1.94 P99: 5.96 P99.9: 130.41 P99.99: 341.18 ------------------------------------------------------ [ 0, 1 ] 328669 12.407% 12.407% ## ( 1, 2 ] 1761080 66.481% 78.888% ############# ( 2, 3 ] 381884 14.416% 93.305% ### ( 3, 4 ] 87013 3.285% 96.589% # ( 4, 6 ] 65119 2.458% 99.048% ( 6, 10 ] 18297 0.691% 99.738% ( 10, 15 ] 1702 0.064% 99.803% ( 15, 22 ] 248 0.009% 99.812% ( 22, 34 ] 48 0.002% 99.814% ( 34, 51 ] 17 0.001% 99.814% ( 51, 76 ] 56 0.002% 99.817% ( 76, 110 ] 1471 0.056% 99.872% ( 110, 170 ] 2175 0.082% 99.954% ( 170, 250 ] 651 0.025% 99.979% ( 250, 380 ] 425 0.016% 99.995% ( 380, 580 ] 335 0.013% 100.007% ( 580, 870 ] 186 0.007% 100.014% ( 870, 1300 ] 82 0.003% 100.018% ( 1300, 1900 ] 26 0.001% 100.019% ( 1900, 2900 ] 22 0.001% 100.019% ( 2900, 4400 ] 15 0.001% 100.020% ( 4400, 6600 ] 2 0.000% 100.020% ** Level 1 read latency histogram (micros): Count: 642685 Average: 3.2893 StdDev: 30.87 Min: 0 Median: 1.7314 Max: 7018 Percentiles: P50: 1.73 P75: 2.46 P99: 8.41 P99.9: 92.98 P99.99: 160.09 ------------------------------------------------------ [ 0, 1 ] 49265 7.665% 7.665% ## ( 1, 2 ] 371995 57.881% 65.547% ############ ( 2, 3 ] 132865 20.673% 86.220% #### ( 3, 4 ] 36892 5.740% 91.961% # ( 4, 6 ] 38481 5.988% 97.948% # ( 6, 10 ] 11216 1.745% 99.693% ( 10, 15 ] 990 0.154% 99.847% ( 15, 22 ] 138 0.021% 99.869% ( 22, 34 ] 11 0.002% 99.871% ( 34, 51 ] 6 0.001% 99.871% ( 51, 76 ] 11 0.002% 99.873% ( 76, 110 ] 345 0.054% 99.927% ( 110, 170 ] 486 0.076% 100.002% ( 170, 250 ] 162 0.025% 100.028% ( 250, 380 ] 130 0.020% 100.048% ( 380, 580 ] 139 0.022% 100.070% ( 580, 870 ] 61 0.009% 100.079% ( 870, 1300 ] 33 0.005% 100.084% ( 1300, 1900 ] 26 0.004% 100.088% ( 1900, 2900 ] 16 0.002% 100.091% ( 2900, 4400 ] 16 0.002% 100.093% ( 4400, 6600 ] 2 0.000% 100.094% ( 6600, 9900 ] 1 0.000% 100.094% ** Level 2 read latency histogram (micros): Count: 40626 Average: 2.3959 StdDev: 35.40 Min: 1 Median: 1.3334 Max: 4910 Percentiles: P50: 1.33 P75: 1.75 P99: 5.38 P99.9: 95.12 P99.99: 473.74 ------------------------------------------------------ [ 0, 1 ] 12138 29.877% 29.877% ###### ( 1, 2 ] 24519 60.353% 90.230% ############ ( 2, 3 ] 2553 6.284% 96.515% # ( 3, 4 ] 763 1.878% 98.393% ( 4, 6 ] 357 0.879% 99.271% ( 6, 10 ] 171 0.421% 99.692% ( 10, 15 ] 46 0.113% 99.806% ( 15, 22 ] 3 0.007% 99.813% ( 22, 34 ] 2 0.005% 99.818% ( 34, 51 ] 1 0.002% 99.820% ( 51, 76 ] 29 0.071% 99.892% ( 76, 110 ] 6 0.015% 99.906% ( 110, 170 ] 6 0.015% 99.921% ( 170, 250 ] 17 0.042% 99.963% ( 250, 380 ] 10 0.025% 99.988% ( 380, 580 ] 2 0.005% 99.993% ( 580, 870 ] 1 0.002% 99.995% ( 4400, 6600 ] 2 0.005% 100.000% ** Level 3 read latency histogram (micros): Count: 285900 Average: 6.0108 StdDev: 75.00 Min: 1 Median: 1.6989 Max: 8691 Percentiles: P50: 1.70 P75: 2.37 P99: 9.91 P99.9: 695.87 P99.99: 1711.15 ------------------------------------------------------ [ 0, 1 ] 22553 7.888% 7.888% ## ( 1, 2 ] 172272 60.256% 68.144% ############ ( 2, 3 ] 52627 18.407% 86.552% #### ( 3, 4 ] 16513 5.776% 92.328% # ( 4, 6 ] 13537 4.735% 97.063% # ( 6, 10 ] 5660 1.980% 99.042% ( 10, 15 ] 589 0.206% 99.248% ( 15, 22 ] 82 0.029% 99.277% ( 22, 34 ] 18 0.006% 99.283% ( 34, 51 ] 5 0.002% 99.285% ( 51, 76 ] 2 0.001% 99.286% ( 76, 110 ] 142 0.050% 99.335% ( 110, 170 ] 494 0.173% 99.508% ( 170, 250 ] 364 0.127% 99.636% ( 250, 380 ] 322 0.113% 99.748% ( 380, 580 ] 343 0.120% 99.868% ( 580, 870 ] 228 0.080% 99.948% ( 870, 1300 ] 93 0.033% 99.980% ( 1300, 1900 ] 40 0.014% 99.994% ( 1900, 2900 ] 18 0.006% 100.001% ( 2900, 4400 ] 26 0.009% 100.010% ( 4400, 6600 ] 20 0.007% 100.017% ( 6600, 9900 ] 4 0.001% 100.018% ** Level 4 read latency histogram (micros): Count: 225 Average: 588.4178 StdDev: 1480.22 Min: 0 Median: 1.9308 Max: 7491 Percentiles: P50: 1.93 P75: 127.50 P99: 6550.00 P99.9: 7491.00 P99.99: 7491.00 ------------------------------------------------------ [ 0, 1 ] 52 23.111% 23.111% ##### ( 1, 2 ] 65 28.889% 52.000% ###### ( 2, 3 ] 11 4.889% 56.889% # ( 3, 4 ] 8 3.556% 60.444% # ( 4, 6 ] 3 1.333% 61.778% ( 6, 10 ] 7 3.111% 64.889% # ( 10, 15 ] 17 7.556% 72.444% ## ( 15, 22 ] 4 1.778% 74.222% ( 110, 170 ] 6 2.667% 76.889% # ( 170, 250 ] 5 2.222% 79.111% ( 250, 380 ] 5 2.222% 81.333% ( 380, 580 ] 6 2.667% 84.000% # ( 580, 870 ] 1 0.444% 84.444% ( 870, 1300 ] 2 0.889% 85.333% ( 1300, 1900 ] 5 2.222% 87.556% ( 1900, 2900 ] 10 4.444% 92.000% # ( 2900, 4400 ] 5 2.222% 94.222% ( 4400, 6600 ] 11 4.889% 99.111% # ( 6600, 9900 ] 2 0.889% 100.000% ** Level 5 read latency histogram (micros): Count: 120 Average: 1191.4583 StdDev: 1354.84 Min: 89 Median: 530.0000 Max: 5854 Percentiles: P50: 530.00 P75: 1900.00 P99: 5720.00 P99.9: 5854.00 P99.99: 5854.00 ------------------------------------------------------ ( 76, 110 ] 4 3.333% 3.333% # ( 110, 170 ] 16 13.333% 16.667% ### ( 170, 250 ] 17 14.167% 30.833% ### ( 250, 380 ] 17 14.167% 45.000% ### ( 380, 580 ] 8 6.667% 51.667% # ( 580, 870 ] 13 10.833% 62.500% ## ( 870, 1300 ] 6 5.000% 67.500% # ( 1300, 1900 ] 9 7.500% 75.000% ## ( 1900, 2900 ] 12 10.000% 85.000% ## ( 2900, 4400 ] 15 12.500% 97.500% ### ( 4400, 6600 ] 3 2.500% 100.000% # ** DB Stats ** Uptime(secs): 30.5 total, 7.8 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 225.53 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:11.628 H:M:S, 38.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 126.52 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.914 H:M:S, 63.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/12-09:13:52 ... thread 0: (1000000,8000000) ops and (325978.6,239858.0) ops/second in (3.067686,33.353062) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/13 3.87 GB 100.7 7.1 0.0 7.1 14.7 7.6 0.0 1.9 144.7 297.9 50.50 47.14 134 0.377 7419K 10K 0.0 0.0 L1 16/16 1.50 GB 0.0 4.9 3.6 1.3 4.9 3.5 0.1 1.4 220.1 217.6 22.87 21.61 3 7.622 5109K 57K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 2.1 1.8 229.6 229.5 0.49 0.46 1 0.486 113K 22 0.0 0.0 L3 14/0 996.78 MB 1.0 1.7 0.8 0.9 1.7 0.8 1.4 2.2 186.5 186.2 9.27 7.44 12 0.773 1691K 556 0.0 0.0 L4 170/0 9.51 GB 1.0 0.7 0.7 0.1 0.7 0.6 1.3 1.1 162.1 161.7 4.46 3.42 6 0.744 735K 1740 0.0 0.0 L5 1068/0 87.78 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1286/29 103.63 GB 0.0 14.6 5.0 9.5 22.1 12.5 6.9 2.9 170.1 257.8 87.59 80.06 156 0.561 15M 69K 0.0 0.0 Int 0/0 0.00 KB 0.0 14.6 5.0 9.5 22.1 12.5 6.9 2.9 170.1 257.8 87.59 80.06 156 0.561 15M 69K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.6 5.0 9.5 14.5 5.0 0.0 0.0 227.4 226.3 65.54 60.61 45 1.456 15M 69K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 351.3 22.06 19.46 111 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 33.6 total, 33.4 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 22.05 GB write, 671.72 MB/s write, 14.55 GB read, 443.34 MB/s read, 87.6 seconds Interval compaction: 22.05 GB write, 676.96 MB/s write, 14.55 GB read, 446.79 MB/s read, 87.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 20, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 20, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 14, pending-compaction-bytes-stops: 0, total-delays: 34, total-stops: 26, interval: 60 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3065115 Average: 2.6324 StdDev: 16.54 Min: 0 Median: 1.5729 Max: 5614 Percentiles: P50: 1.57 P75: 1.95 P99: 5.95 P99.9: 115.00 P99.99: 278.94 ------------------------------------------------------ [ 0, 1 ] 365483 11.924% 11.924% ## ( 1, 2 ] 2037208 66.464% 78.388% ############# ( 2, 3 ] 454923 14.842% 93.230% ### ( 3, 4 ] 101595 3.315% 96.545% # ( 4, 6 ] 77120 2.516% 99.061% # ( 6, 10 ] 21673 0.707% 99.768% ( 10, 15 ] 1972 0.064% 99.832% ( 15, 22 ] 293 0.010% 99.842% ( 22, 34 ] 53 0.002% 99.844% ( 34, 51 ] 20 0.001% 99.844% ( 51, 76 ] 59 0.002% 99.846% ( 76, 110 ] 1471 0.048% 99.894% ( 110, 170 ] 2183 0.071% 99.965% ( 170, 250 ] 662 0.022% 99.987% ( 250, 380 ] 429 0.014% 100.001% ( 380, 580 ] 346 0.011% 100.012% ( 580, 870 ] 189 0.006% 100.018% ( 870, 1300 ] 86 0.003% 100.021% ( 1300, 1900 ] 27 0.001% 100.022% ( 1900, 2900 ] 25 0.001% 100.023% ( 2900, 4400 ] 16 0.001% 100.023% ( 4400, 6600 ] 2 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 642685 Average: 3.2893 StdDev: 30.87 Min: 0 Median: 1.7314 Max: 7018 Percentiles: P50: 1.73 P75: 2.46 P99: 8.41 P99.9: 92.98 P99.99: 160.09 ------------------------------------------------------ [ 0, 1 ] 49265 7.665% 7.665% ## ( 1, 2 ] 371995 57.881% 65.547% ############ ( 2, 3 ] 132865 20.673% 86.220% #### ( 3, 4 ] 36892 5.740% 91.961% # ( 4, 6 ] 38481 5.988% 97.948% # ( 6, 10 ] 11216 1.745% 99.693% ( 10, 15 ] 990 0.154% 99.847% ( 15, 22 ] 138 0.021% 99.869% ( 22, 34 ] 11 0.002% 99.871% ( 34, 51 ] 6 0.001% 99.871% ( 51, 76 ] 11 0.002% 99.873% ( 76, 110 ] 345 0.054% 99.927% ( 110, 170 ] 486 0.076% 100.002% ( 170, 250 ] 162 0.025% 100.028% ( 250, 380 ] 130 0.020% 100.048% ( 380, 580 ] 139 0.022% 100.070% ( 580, 870 ] 61 0.009% 100.079% ( 870, 1300 ] 33 0.005% 100.084% ( 1300, 1900 ] 26 0.004% 100.088% ( 1900, 2900 ] 16 0.002% 100.091% ( 2900, 4400 ] 16 0.002% 100.093% ( 4400, 6600 ] 2 0.000% 100.094% ( 6600, 9900 ] 1 0.000% 100.094% ** Level 2 read latency histogram (micros): Count: 40626 Average: 2.3959 StdDev: 35.40 Min: 1 Median: 1.3334 Max: 4910 Percentiles: P50: 1.33 P75: 1.75 P99: 5.38 P99.9: 95.12 P99.99: 473.74 ------------------------------------------------------ [ 0, 1 ] 12138 29.877% 29.877% ###### ( 1, 2 ] 24519 60.353% 90.230% ############ ( 2, 3 ] 2553 6.284% 96.515% # ( 3, 4 ] 763 1.878% 98.393% ( 4, 6 ] 357 0.879% 99.271% ( 6, 10 ] 171 0.421% 99.692% ( 10, 15 ] 46 0.113% 99.806% ( 15, 22 ] 3 0.007% 99.813% ( 22, 34 ] 2 0.005% 99.818% ( 34, 51 ] 1 0.002% 99.820% ( 51, 76 ] 29 0.071% 99.892% ( 76, 110 ] 6 0.015% 99.906% ( 110, 170 ] 6 0.015% 99.921% ( 170, 250 ] 17 0.042% 99.963% ( 250, 380 ] 10 0.025% 99.988% ( 380, 580 ] 2 0.005% 99.993% ( 580, 870 ] 1 0.002% 99.995% ( 4400, 6600 ] 2 0.005% 100.000% ** Level 3 read latency histogram (micros): Count: 285900 Average: 6.0108 StdDev: 75.00 Min: 1 Median: 1.6989 Max: 8691 Percentiles: P50: 1.70 P75: 2.37 P99: 9.91 P99.9: 695.87 P99.99: 1711.15 ------------------------------------------------------ [ 0, 1 ] 22553 7.888% 7.888% ## ( 1, 2 ] 172272 60.256% 68.144% ############ ( 2, 3 ] 52627 18.407% 86.552% #### ( 3, 4 ] 16513 5.776% 92.328% # ( 4, 6 ] 13537 4.735% 97.063% # ( 6, 10 ] 5660 1.980% 99.042% ( 10, 15 ] 589 0.206% 99.248% ( 15, 22 ] 82 0.029% 99.277% ( 22, 34 ] 18 0.006% 99.283% ( 34, 51 ] 5 0.002% 99.285% ( 51, 76 ] 2 0.001% 99.286% ( 76, 110 ] 142 0.050% 99.335% ( 110, 170 ] 494 0.173% 99.508% ( 170, 250 ] 364 0.127% 99.636% ( 250, 380 ] 322 0.113% 99.748% ( 380, 580 ] 343 0.120% 99.868% ( 580, 870 ] 228 0.080% 99.948% ( 870, 1300 ] 93 0.033% 99.980% ( 1300, 1900 ] 40 0.014% 99.994% ( 1900, 2900 ] 18 0.006% 100.001% ( 2900, 4400 ] 26 0.009% 100.010% ( 4400, 6600 ] 20 0.007% 100.017% ( 6600, 9900 ] 4 0.001% 100.018% ** Level 4 read latency histogram (micros): Count: 225 Average: 588.4178 StdDev: 1480.22 Min: 0 Median: 1.9308 Max: 7491 Percentiles: P50: 1.93 P75: 127.50 P99: 6550.00 P99.9: 7491.00 P99.99: 7491.00 ------------------------------------------------------ [ 0, 1 ] 52 23.111% 23.111% ##### ( 1, 2 ] 65 28.889% 52.000% ###### ( 2, 3 ] 11 4.889% 56.889% # ( 3, 4 ] 8 3.556% 60.444% # ( 4, 6 ] 3 1.333% 61.778% ( 6, 10 ] 7 3.111% 64.889% # ( 10, 15 ] 17 7.556% 72.444% ## ( 15, 22 ] 4 1.778% 74.222% ( 110, 170 ] 6 2.667% 76.889% # ( 170, 250 ] 5 2.222% 79.111% ( 250, 380 ] 5 2.222% 81.333% ( 380, 580 ] 6 2.667% 84.000% # ( 580, 870 ] 1 0.444% 84.444% ( 870, 1300 ] 2 0.889% 85.333% ( 1300, 1900 ] 5 2.222% 87.556% ( 1900, 2900 ] 10 4.444% 92.000% # ( 2900, 4400 ] 5 2.222% 94.222% ( 4400, 6600 ] 11 4.889% 99.111% # ( 6600, 9900 ] 2 0.889% 100.000% ** Level 5 read latency histogram (micros): Count: 120 Average: 1191.4583 StdDev: 1354.84 Min: 89 Median: 530.0000 Max: 5854 Percentiles: P50: 530.00 P75: 1900.00 P99: 5720.00 P99.9: 5854.00 P99.99: 5854.00 ------------------------------------------------------ ( 76, 110 ] 4 3.333% 3.333% # ( 110, 170 ] 16 13.333% 16.667% ### ( 170, 250 ] 17 14.167% 30.833% ### ( 250, 380 ] 17 14.167% 45.000% ### ( 380, 580 ] 8 6.667% 51.667% # ( 580, 870 ] 13 10.833% 62.500% ## ( 870, 1300 ] 6 5.000% 67.500% # ( 1300, 1900 ] 9 7.500% 75.000% ## ( 1900, 2900 ] 12 10.000% 85.000% ## ( 2900, 4400 ] 15 12.500% 97.500% ### ( 4400, 6600 ] 3 2.500% 100.000% # ** DB Stats ** Uptime(secs): 33.6 total, 3.1 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 234.23 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:12.080 H:M:S, 35.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 320.82 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.452 H:M:S, 14.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/12-09:14:12 ... thread 0: (1000000,9000000) ops and (52378.0,171608.2) ops/second in (19.091971,52.445033) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/14 2.27 GB 0.0 7.9 0.0 7.9 16.5 8.5 0.0 1.9 141.5 293.8 57.42 52.85 151 0.380 8250K 11K 0.0 0.0 L1 13/13 1.60 GB 0.0 9.0 6.1 2.9 8.9 6.0 0.1 1.4 220.7 217.7 41.73 39.22 4 10.433 7790K 57K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 4.5 1.8 229.6 229.5 0.49 0.46 1 0.486 113K 22 0.0 0.0 L3 21/2 1.18 GB 0.9 3.4 1.6 1.8 3.4 1.6 3.0 2.1 160.2 159.3 21.86 15.75 18 1.214 3493K 20K 0.0 0.0 L4 171/6 9.71 GB 1.0 3.3 2.8 0.4 3.3 2.8 1.3 1.2 141.1 140.6 23.77 15.74 31 0.767 3407K 12K 0.0 0.0 L5 1092/0 89.75 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1311/35 104.52 GB 0.0 23.7 10.7 13.1 32.1 19.0 12.8 3.8 167.3 226.4 145.26 124.01 205 0.709 23M 100K 0.0 0.0 Int 0/0 0.00 KB 0.0 23.7 10.7 13.1 32.1 19.0 12.8 3.8 167.3 226.4 145.26 124.01 205 0.709 23M 100K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 23.7 10.7 13.1 23.6 10.5 0.0 0.0 203.4 202.0 119.48 101.98 80 1.493 23M 100K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.6 8.6 0.0 0.0 0.0 339.7 25.78 22.03 125 0.206 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 52.7 total, 52.4 interval Flush(GB): cumulative 8.551, interval 8.551 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 32.12 GB write, 623.99 MB/s write, 23.73 GB read, 461.10 MB/s read, 145.3 seconds Interval compaction: 32.12 GB write, 627.08 MB/s write, 23.73 GB read, 463.39 MB/s read, 145.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 25, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 25, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 31, pending-compaction-bytes-delays: 40, pending-compaction-bytes-stops: 0, total-delays: 65, total-stops: 31, interval: 96 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3841062 Average: 2.9145 StdDev: 26.22 Min: 0 Median: 1.5703 Max: 5808 Percentiles: P50: 1.57 P75: 1.95 P99: 6.53 P99.9: 145.81 P99.99: 477.54 ------------------------------------------------------ [ 0, 1 ] 486363 12.662% 12.662% ### ( 1, 2 ] 2514866 65.473% 78.135% ############# ( 2, 3 ] 555854 14.471% 92.607% ### ( 3, 4 ] 132189 3.441% 96.048% # ( 4, 6 ] 109375 2.848% 98.896% # ( 6, 10 ] 30322 0.789% 99.685% ( 10, 15 ] 2928 0.076% 99.761% ( 15, 22 ] 450 0.012% 99.773% ( 22, 34 ] 58 0.002% 99.775% ( 34, 51 ] 22 0.001% 99.775% ( 51, 76 ] 67 0.002% 99.777% ( 76, 110 ] 2628 0.068% 99.845% ( 110, 170 ] 3518 0.092% 99.937% ( 170, 250 ] 972 0.025% 99.962% ( 250, 380 ] 726 0.019% 99.981% ( 380, 580 ] 699 0.018% 99.999% ( 580, 870 ] 415 0.011% 100.010% ( 870, 1300 ] 236 0.006% 100.016% ( 1300, 1900 ] 77 0.002% 100.018% ( 1900, 2900 ] 65 0.002% 100.020% ( 2900, 4400 ] 76 0.002% 100.022% ( 4400, 6600 ] 11 0.000% 100.022% ** Level 1 read latency histogram (micros): Count: 1726896 Average: 4.8473 StdDev: 72.82 Min: 0 Median: 1.7671 Max: 8516 Percentiles: P50: 1.77 P75: 2.64 P99: 8.96 P99.9: 96.83 P99.99: 157.81 ------------------------------------------------------ [ 0, 1 ] 176939 10.246% 10.246% ## ( 1, 2 ] 894896 51.821% 62.067% ########## ( 2, 3 ] 346646 20.073% 82.140% #### ( 3, 4 ] 124883 7.232% 89.372% # ( 4, 6 ] 140316 8.125% 97.497% ## ( 6, 10 ] 35086 2.032% 99.529% ( 10, 15 ] 4675 0.271% 99.800% ( 15, 22 ] 662 0.038% 99.838% ( 22, 34 ] 55 0.003% 99.841% ( 34, 51 ] 26 0.002% 99.843% ( 51, 76 ] 63 0.004% 99.847% ( 76, 110 ] 1505 0.087% 99.934% ( 110, 170 ] 1219 0.071% 100.004% ( 170, 250 ] 383 0.022% 100.027% ( 250, 380 ] 447 0.026% 100.052% ( 380, 580 ] 546 0.032% 100.084% ( 580, 870 ] 527 0.031% 100.115% ( 870, 1300 ] 382 0.022% 100.137% ( 1300, 1900 ] 209 0.012% 100.149% ( 1900, 2900 ] 198 0.011% 100.160% ( 2900, 4400 ] 254 0.015% 100.175% ( 4400, 6600 ] 68 0.004% 100.179% ( 6600, 9900 ] 26 0.002% 100.180% ** Level 2 read latency histogram (micros): Count: 40626 Average: 2.3959 StdDev: 35.40 Min: 1 Median: 1.3334 Max: 4910 Percentiles: P50: 1.33 P75: 1.75 P99: 5.38 P99.9: 95.12 P99.99: 473.74 ------------------------------------------------------ [ 0, 1 ] 12138 29.877% 29.877% ###### ( 1, 2 ] 24519 60.353% 90.230% ############ ( 2, 3 ] 2553 6.284% 96.515% # ( 3, 4 ] 763 1.878% 98.393% ( 4, 6 ] 357 0.879% 99.271% ( 6, 10 ] 171 0.421% 99.692% ( 10, 15 ] 46 0.113% 99.806% ( 15, 22 ] 3 0.007% 99.813% ( 22, 34 ] 2 0.005% 99.818% ( 34, 51 ] 1 0.002% 99.820% ( 51, 76 ] 29 0.071% 99.892% ( 76, 110 ] 6 0.015% 99.906% ( 110, 170 ] 6 0.015% 99.921% ( 170, 250 ] 17 0.042% 99.963% ( 250, 380 ] 10 0.025% 99.988% ( 380, 580 ] 2 0.005% 99.993% ( 580, 870 ] 1 0.002% 99.995% ( 4400, 6600 ] 2 0.005% 100.000% ** Level 3 read latency histogram (micros): Count: 671173 Average: 5.8420 StdDev: 74.18 Min: 1 Median: 1.9612 Max: 8691 Percentiles: P50: 1.96 P75: 2.89 P99: 9.28 P99.9: 379.95 P99.99: 605.80 ------------------------------------------------------ [ 0, 1 ] 23736 3.536% 3.536% # ( 1, 2 ] 324430 48.338% 51.874% ########## ( 2, 3 ] 174693 26.028% 77.902% ##### ( 3, 4 ] 63768 9.501% 87.403% ## ( 4, 6 ] 67365 10.037% 97.440% ## ( 6, 10 ] 12751 1.900% 99.340% ( 10, 15 ] 1887 0.281% 99.621% ( 15, 22 ] 206 0.031% 99.652% ( 22, 34 ] 27 0.004% 99.656% ( 34, 51 ] 8 0.001% 99.657% ( 51, 76 ] 16 0.002% 99.659% ( 76, 110 ] 149 0.022% 99.682% ( 110, 170 ] 558 0.083% 99.765% ( 170, 250 ] 437 0.065% 99.830% ( 250, 380 ] 471 0.070% 99.900% ( 380, 580 ] 565 0.084% 99.984% ( 580, 870 ] 437 0.065% 100.049% ( 870, 1300 ] 234 0.035% 100.084% ( 1300, 1900 ] 88 0.013% 100.097% ( 1900, 2900 ] 54 0.008% 100.105% ( 2900, 4400 ] 105 0.016% 100.121% ( 4400, 6600 ] 30 0.004% 100.125% ( 6600, 9900 ] 5 0.001% 100.126% ** Level 4 read latency histogram (micros): Count: 100517 Average: 17.6214 StdDev: 187.39 Min: 0 Median: 2.2714 Max: 10738 Percentiles: P50: 2.27 P75: 3.23 P99: 352.24 P99.9: 2628.93 P99.99: 4137.47 ------------------------------------------------------ [ 0, 1 ] 703 0.699% 0.699% ( 1, 2 ] 41192 40.980% 41.680% ######## ( 2, 3 ] 30821 30.662% 72.342% ###### ( 3, 4 ] 11424 11.365% 83.707% ## ( 4, 6 ] 11994 11.932% 95.640% ## ( 6, 10 ] 2514 2.501% 98.141% # ( 10, 15 ] 459 0.457% 98.597% ( 15, 22 ] 51 0.051% 98.648% ( 22, 34 ] 11 0.011% 98.659% ( 34, 51 ] 4 0.004% 98.663% ( 51, 76 ] 1 0.001% 98.664% ( 76, 110 ] 8 0.008% 98.672% ( 110, 170 ] 67 0.067% 98.739% ( 170, 250 ] 137 0.136% 98.875% ( 250, 380 ] 160 0.159% 99.034% ( 380, 580 ] 215 0.214% 99.248% ( 580, 870 ] 306 0.304% 99.552% ( 870, 1300 ] 207 0.206% 99.758% ( 1300, 1900 ] 90 0.090% 99.848% ( 1900, 2900 ] 72 0.072% 99.919% ( 2900, 4400 ] 86 0.086% 100.005% ( 4400, 6600 ] 30 0.030% 100.035% ( 6600, 9900 ] 6 0.006% 100.041% ( 9900, 14000 ] 2 0.002% 100.043% ** Level 5 read latency histogram (micros): Count: 240 Average: 1349.5958 StdDev: 1514.84 Min: 82 Median: 676.6667 Max: 8693 Percentiles: P50: 676.67 P75: 2066.67 P99: 6320.00 P99.9: 8693.00 P99.99: 8693.00 ------------------------------------------------------ ( 76, 110 ] 9 3.750% 3.750% # ( 110, 170 ] 30 12.500% 16.250% ### ( 170, 250 ] 26 10.833% 27.083% ## ( 250, 380 ] 28 11.667% 38.750% ## ( 380, 580 ] 19 7.917% 46.667% ## ( 580, 870 ] 24 10.000% 56.667% ## ( 870, 1300 ] 18 7.500% 64.167% ## ( 1300, 1900 ] 22 9.167% 73.333% ## ( 1900, 2900 ] 24 10.000% 83.333% ## ( 2900, 4400 ] 28 11.667% 95.000% ## ( 4400, 6600 ] 11 4.583% 99.583% # ( 6600, 9900 ] 1 0.417% 100.000% ** DB Stats ** Uptime(secs): 52.7 total, 19.1 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 168.06 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:28.118 H:M:S, 53.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 51.55 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:16.039 H:M:S, 84.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 2024/10/12-09:14:15 ... thread 0: (1000000,10000000) ops and (307976.4,179558.9) ops/second in (3.247002,55.692035) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/14 3.20 GB 100.7 8.8 0.0 8.8 18.3 9.5 0.0 1.9 141.7 294.1 63.57 58.60 168 0.378 9146K 12K 0.0 0.0 L1 13/13 1.60 GB 0.0 9.0 6.1 2.9 8.9 6.0 0.1 1.4 220.7 217.7 41.73 39.22 4 10.433 7790K 57K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 4.5 1.8 229.6 229.5 0.49 0.46 1 0.486 113K 22 0.0 0.0 L3 19/0 947.15 MB 0.9 3.4 1.6 1.8 3.4 1.6 3.0 2.1 160.2 159.3 21.86 15.75 18 1.214 3493K 20K 0.0 0.0 L4 169/0 9.66 GB 1.0 3.7 3.1 0.6 3.7 3.1 1.3 1.2 145.1 144.6 26.15 17.76 33 0.792 3853K 12K 0.0 0.0 L5 1096/0 90.06 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1316/27 105.44 GB 0.0 25.0 10.9 14.1 34.3 20.2 13.1 3.6 166.6 228.6 153.79 131.78 224 0.687 24M 102K 0.0 0.0 Int 0/0 0.00 KB 0.0 25.0 10.9 14.1 34.3 20.2 13.1 3.6 166.6 228.6 153.79 131.78 224 0.687 24M 102K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 25.0 10.9 14.1 24.9 10.7 0.0 0.0 204.4 203.1 125.34 107.40 85 1.475 24M 102K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 9.5 9.5 0.0 0.0 0.0 341.0 28.45 24.38 139 0.205 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 56.0 total, 55.7 interval Flush(GB): cumulative 9.474, interval 9.474 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 34.33 GB write, 628.24 MB/s write, 25.02 GB read, 457.95 MB/s read, 153.8 seconds Interval compaction: 34.33 GB write, 631.17 MB/s write, 25.02 GB read, 460.09 MB/s read, 153.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 28, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 28, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 33, pending-compaction-bytes-delays: 40, pending-compaction-bytes-stops: 0, total-delays: 68, total-stops: 33, interval: 101 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 4281321 Average: 2.8629 StdDev: 24.93 Min: 0 Median: 1.5787 Max: 5808 Percentiles: P50: 1.58 P75: 1.96 P99: 6.49 P99.9: 135.55 P99.99: 413.77 ------------------------------------------------------ [ 0, 1 ] 520141 12.149% 12.149% ## ( 1, 2 ] 2800050 65.402% 77.551% ############# ( 2, 3 ] 639573 14.939% 92.489% ### ( 3, 4 ] 150457 3.514% 96.004% # ( 4, 6 ] 124053 2.898% 98.901% # ( 6, 10 ] 34715 0.811% 99.712% ( 10, 15 ] 3284 0.077% 99.789% ( 15, 22 ] 484 0.011% 99.800% ( 22, 34 ] 61 0.001% 99.801% ( 34, 51 ] 26 0.001% 99.802% ( 51, 76 ] 69 0.002% 99.804% ( 76, 110 ] 2629 0.061% 99.865% ( 110, 170 ] 3520 0.082% 99.947% ( 170, 250 ] 983 0.023% 99.970% ( 250, 380 ] 730 0.017% 99.987% ( 380, 580 ] 704 0.016% 100.004% ( 580, 870 ] 420 0.010% 100.014% ( 870, 1300 ] 241 0.006% 100.019% ( 1300, 1900 ] 78 0.002% 100.021% ( 1900, 2900 ] 66 0.002% 100.022% ( 2900, 4400 ] 76 0.002% 100.024% ( 4400, 6600 ] 11 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 1739695 Average: 4.8328 StdDev: 72.55 Min: 0 Median: 1.7685 Max: 8516 Percentiles: P50: 1.77 P75: 2.65 P99: 8.96 P99.9: 96.50 P99.99: 157.65 ------------------------------------------------------ [ 0, 1 ] 177054 10.177% 10.177% ## ( 1, 2 ] 901482 51.818% 61.996% ########## ( 2, 3 ] 350549 20.150% 82.146% #### ( 3, 4 ] 125799 7.231% 89.377% # ( 4, 6 ] 141260 8.120% 97.497% ## ( 6, 10 ] 35383 2.034% 99.530% ( 10, 15 ] 4706 0.271% 99.801% ( 15, 22 ] 669 0.038% 99.839% ( 22, 34 ] 56 0.003% 99.843% ( 34, 51 ] 27 0.002% 99.844% ( 51, 76 ] 63 0.004% 99.848% ( 76, 110 ] 1505 0.087% 99.934% ( 110, 170 ] 1219 0.070% 100.004% ( 170, 250 ] 384 0.022% 100.026% ( 250, 380 ] 447 0.026% 100.052% ( 380, 580 ] 546 0.031% 100.084% ( 580, 870 ] 527 0.030% 100.114% ( 870, 1300 ] 382 0.022% 100.136% ( 1300, 1900 ] 209 0.012% 100.148% ( 1900, 2900 ] 198 0.011% 100.159% ( 2900, 4400 ] 254 0.015% 100.174% ( 4400, 6600 ] 68 0.004% 100.178% ( 6600, 9900 ] 26 0.001% 100.179% ** Level 2 read latency histogram (micros): Count: 40626 Average: 2.3959 StdDev: 35.40 Min: 1 Median: 1.3334 Max: 4910 Percentiles: P50: 1.33 P75: 1.75 P99: 5.38 P99.9: 95.12 P99.99: 473.74 ------------------------------------------------------ [ 0, 1 ] 12138 29.877% 29.877% ###### ( 1, 2 ] 24519 60.353% 90.230% ############ ( 2, 3 ] 2553 6.284% 96.515% # ( 3, 4 ] 763 1.878% 98.393% ( 4, 6 ] 357 0.879% 99.271% ( 6, 10 ] 171 0.421% 99.692% ( 10, 15 ] 46 0.113% 99.806% ( 15, 22 ] 3 0.007% 99.813% ( 22, 34 ] 2 0.005% 99.818% ( 34, 51 ] 1 0.002% 99.820% ( 51, 76 ] 29 0.071% 99.892% ( 76, 110 ] 6 0.015% 99.906% ( 110, 170 ] 6 0.015% 99.921% ( 170, 250 ] 17 0.042% 99.963% ( 250, 380 ] 10 0.025% 99.988% ( 380, 580 ] 2 0.005% 99.993% ( 580, 870 ] 1 0.002% 99.995% ( 4400, 6600 ] 2 0.005% 100.000% ** Level 3 read latency histogram (micros): Count: 671173 Average: 5.8420 StdDev: 74.18 Min: 1 Median: 1.9612 Max: 8691 Percentiles: P50: 1.96 P75: 2.89 P99: 9.28 P99.9: 379.95 P99.99: 605.80 ------------------------------------------------------ [ 0, 1 ] 23736 3.536% 3.536% # ( 1, 2 ] 324430 48.338% 51.874% ########## ( 2, 3 ] 174693 26.028% 77.902% ##### ( 3, 4 ] 63768 9.501% 87.403% ## ( 4, 6 ] 67365 10.037% 97.440% ## ( 6, 10 ] 12751 1.900% 99.340% ( 10, 15 ] 1887 0.281% 99.621% ( 15, 22 ] 206 0.031% 99.652% ( 22, 34 ] 27 0.004% 99.656% ( 34, 51 ] 8 0.001% 99.657% ( 51, 76 ] 16 0.002% 99.659% ( 76, 110 ] 149 0.022% 99.682% ( 110, 170 ] 558 0.083% 99.765% ( 170, 250 ] 437 0.065% 99.830% ( 250, 380 ] 471 0.070% 99.900% ( 380, 580 ] 565 0.084% 99.984% ( 580, 870 ] 437 0.065% 100.049% ( 870, 1300 ] 234 0.035% 100.084% ( 1300, 1900 ] 88 0.013% 100.097% ( 1900, 2900 ] 54 0.008% 100.105% ( 2900, 4400 ] 105 0.016% 100.121% ( 4400, 6600 ] 30 0.004% 100.125% ( 6600, 9900 ] 5 0.001% 100.126% ** Level 4 read latency histogram (micros): Count: 131329 Average: 14.4293 StdDev: 164.43 Min: 0 Median: 2.1800 Max: 10738 Percentiles: P50: 2.18 P75: 3.02 P99: 180.99 P99.9: 2062.10 P99.99: 3909.31 ------------------------------------------------------ [ 0, 1 ] 907 0.691% 0.691% ( 1, 2 ] 57604 43.862% 44.553% ######### ( 2, 3 ] 39737 30.258% 74.811% ###### ( 3, 4 ] 13663 10.404% 85.214% ## ( 4, 6 ] 14178 10.796% 96.010% ## ( 6, 10 ] 3176 2.418% 98.428% ( 10, 15 ] 540 0.411% 98.840% ( 15, 22 ] 57 0.043% 98.883% ( 22, 34 ] 12 0.009% 98.892% ( 34, 51 ] 5 0.004% 98.896% ( 51, 76 ] 2 0.002% 98.897% ( 76, 110 ] 17 0.013% 98.910% ( 110, 170 ] 96 0.073% 98.983% ( 170, 250 ] 158 0.120% 99.104% ( 250, 380 ] 181 0.138% 99.242% ( 380, 580 ] 234 0.178% 99.420% ( 580, 870 ] 319 0.243% 99.663% ( 870, 1300 ] 210 0.160% 99.823% ( 1300, 1900 ] 90 0.069% 99.891% ( 1900, 2900 ] 72 0.055% 99.946% ( 2900, 4400 ] 86 0.065% 100.011% ( 4400, 6600 ] 30 0.023% 100.034% ( 6600, 9900 ] 6 0.005% 100.039% ( 9900, 14000 ] 2 0.002% 100.040% ** Level 5 read latency histogram (micros): Count: 260 Average: 1309.1115 StdDev: 1490.27 Min: 75 Median: 649.6000 Max: 8693 Percentiles: P50: 649.60 P75: 2011.11 P99: 6280.00 P99.9: 8693.00 P99.99: 8693.00 ------------------------------------------------------ ( 51, 76 ] 1 0.385% 0.385% ( 76, 110 ] 11 4.231% 4.615% # ( 110, 170 ] 35 13.462% 18.077% ### ( 170, 250 ] 27 10.385% 28.462% ## ( 250, 380 ] 31 11.923% 40.385% ## ( 380, 580 ] 19 7.308% 47.692% # ( 580, 870 ] 25 9.615% 57.308% ## ( 870, 1300 ] 21 8.077% 65.385% ## ( 1300, 1900 ] 22 8.462% 73.846% ## ( 1900, 2900 ] 27 10.385% 84.231% ## ( 2900, 4400 ] 29 11.154% 95.385% ## ( 4400, 6600 ] 11 4.231% 99.615% # ( 6600, 9900 ] 1 0.385% 100.000% ** DB Stats ** Uptime(secs): 56.0 total, 3.2 interval Cumulative writes: 0 writes, 10M keys, 0 commit groups, 0.0 writes per commit group, ingest: 9.61 GB, 175.89 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:28.687 H:M:S, 51.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 303.11 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.568 H:M:S, 17.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 5.601 micros/op 178526 ops/sec 60.002 seconds 10711999 operations; 173.0 MB/s Microseconds per write: Count: 10711999 Average: 5.6014 StdDev: 241.99 Min: 0 Median: 1.7192 Max: 263825 Percentiles: P50: 1.72 P75: 2.72 P99: 8.75 P99.9: 1162.17 P99.99: 2761.23 ------------------------------------------------------ [ 0, 1 ] 1556118 14.527% 14.527% ### ( 1, 2 ] 5283448 49.323% 63.850% ########## ( 2, 3 ] 1663040 15.525% 79.375% ### ( 3, 4 ] 1470265 13.725% 93.100% ### ( 4, 6 ] 556195 5.192% 98.292% # ( 6, 10 ] 110436 1.031% 99.323% ( 10, 15 ] 31995 0.299% 99.622% ( 15, 22 ] 7763 0.072% 99.694% ( 22, 34 ] 1531 0.014% 99.709% ( 34, 51 ] 204 0.002% 99.711% ( 51, 76 ] 25 0.000% 99.711% ( 76, 110 ] 7757 0.072% 99.783% ( 110, 170 ] 2715 0.025% 99.809% ( 170, 250 ] 290 0.003% 99.811% ( 250, 380 ] 122 0.001% 99.812% ( 380, 580 ] 11 0.000% 99.813% ( 580, 870 ] 17 0.000% 99.813% ( 870, 1300 ] 13768 0.129% 99.941% ( 1300, 1900 ] 94 0.001% 99.942% ( 1900, 2900 ] 5961 0.056% 99.998% ( 2900, 4400 ] 173 0.002% 99.999% ( 4400, 6600 ] 43 0.000% 100.000% ( 6600, 9900 ] 8 0.000% 100.000% ( 9900, 14000 ] 1 0.000% 100.000% ( 14000, 22000 ] 4 0.000% 100.000% ( 33000, 50000 ] 1 0.000% 100.000% ( 110000, 170000 ] 2 0.000% 100.000% ( 170000, 250000 ] 11 0.000% 100.000% ( 250000, 380000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 202397 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 10711999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 11054782968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 570 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 31014226 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 30387795897 rocksdb.compact.write.bytes COUNT : 30186721872 rocksdb.flush.write.bytes COUNT : 10965803866 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 62 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 249 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 10877184000 rocksdb.memtable.garbage.bytes.at.flush COUNT : 4044800 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 30488773882 rocksdb.non.last.level.read.count COUNT : 7364106 rocksdb.block.checksum.compute.count COUNT : 7363028 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.359427 P95 : 3.878151 P99 : 7.099302 P100 : 263823.000000 COUNT : 10711999 SUM : 55130121 rocksdb.compaction.times.micros P50 : 1001666.666667 P95 : 3285000.000000 P99 : 18440000.000000 P100 : 18866131.000000 COUNT : 89 SUM : 129268607 rocksdb.compaction.times.cpu_micros P50 : 909354.838710 P95 : 3285000.000000 P99 : 17605971.000000 P100 : 17605971.000000 COUNT : 89 SUM : 111252157 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 709.050000 P95 : 1184.911765 P99 : 17920.000000 P100 : 29665.000000 COUNT : 151 SUM : 149044 rocksdb.compaction.outfile.sync.micros P50 : 1039.077670 P95 : 54791.666667 P99 : 163400.000000 P100 : 279620.000000 COUNT : 377 SUM : 3576228 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 102.688172 P95 : 1042.000000 P99 : 3060.000000 P100 : 3994.000000 COUNT : 268 SUM : 67759 rocksdb.table.open.io.micros P50 : 1343.478261 P95 : 9546.428571 P99 : 17854.545455 P100 : 21659.000000 COUNT : 570 SUM : 1370073 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.708453 P95 : 7.714670 P99 : 10.092534 P100 : 13960.000000 COUNT : 7361703 SUM : 43094016 rocksdb.read.block.get.micros P50 : 467.500000 P95 : 3967.796610 P99 : 6366.250000 P100 : 8818.000000 COUNT : 1140 SUM : 1094639 rocksdb.write.raw.block.micros P50 : 0.513652 P95 : 0.975938 P99 : 2.916598 P100 : 13746.000000 COUNT : 9969260 SUM : 19719179 rocksdb.numfiles.in.singlecompaction P50 : 3.140000 P95 : 4.750000 P99 : 28.200000 P100 : 34.000000 COUNT : 145 SUM : 558 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.558965 P95 : 1197.125318 P99 : 2614.171985 P100 : 263641.000000 COUNT : 190682 SUM : 31030155 rocksdb.sst.read.micros P50 : 1.650650 P95 : 4.650318 P99 : 8.236792 P100 : 10738.000000 COUNT : 7364817 SUM : 28258389 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 10711999 SUM : 11054782968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 212333.333333 P95 : 343031.250000 P99 : 474366.666667 P100 : 554115.000000 COUNT : 151 SUM : 30675651 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 360 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0