[2024-10-17 18:20:53.534831] Starting SPDK v24.09.1-pre git sha1 b18e1bd62 / DPDK 22.11.4 initialization... [2024-10-17 18:20:53.535139] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3190398 ] Set seed to 1729182053528086 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-10-17 18:20:53.631016] app.c: 917:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-10-17 18:20:53.672991] app.c: 610:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-10-17 18:20:53.673041] app.c: 611:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3190398' to capture a snapshot of events at runtime. [2024-10-17 18:20:53.673052] app.c: 616:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-10-17 18:20:53.673062] app.c: 617:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-10-17 18:20:53.673070] app.c: 618:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3190398 for offline analysis/debug. [2024-10-17 18:20:53.673155] reactor.c: 990:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Thu Oct 17 18:20:57 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/10/17-18:21:00 ... thread 0: (1000000,1000000) ops and (336867.9,336867.9) ops/second in (2.968523,2.968523) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 8/5 566.97 MB 25.2 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 352.3 2.50 2.25 12 0.209 0 0 0.0 0.0 L1 3/3 251.75 MB 0.0 0.2 0.2 0.1 0.2 0.2 0.1 1.3 256.0 255.8 0.98 0.96 1 0.984 255K 187 0.0 0.0 L2 1/0 48.91 MB 0.5 0.1 0.1 0.0 0.1 0.1 0.0 1.8 247.7 247.7 0.45 0.42 1 0.450 113K 40 0.0 0.0 L3 18/0 944.37 MB 0.9 0.1 0.1 0.0 0.1 0.1 0.0 1.0 197.8 197.9 0.32 0.26 1 0.316 63K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1040/0 85.69 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1186/8 97.00 GB 0.0 0.4 0.3 0.1 1.3 1.2 0.1 1.5 100.1 307.4 4.25 3.89 15 0.284 432K 227 0.0 0.0 Int 0/0 0.00 KB 0.0 0.4 0.3 0.1 1.3 1.2 0.1 1.5 100.1 307.4 4.25 3.89 15 0.284 432K 227 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.4 0.3 0.1 0.4 0.3 0.0 0.0 243.4 243.2 1.75 1.65 3 0.583 432K 227 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 352.3 2.50 2.25 12 0.209 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.2 total, 3.0 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.28 GB write, 412.16 MB/s write, 0.42 GB read, 134.23 MB/s read, 4.3 seconds Interval compaction: 1.28 GB write, 440.26 MB/s write, 0.42 GB read, 143.38 MB/s read, 4.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 5, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 5, interval: 5 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 104871 Average: 1.8135 StdDev: 18.33 Min: 0 Median: 1.1638 Max: 4601 Percentiles: P50: 1.16 P75: 1.61 P99: 3.38 P99.9: 8.73 P99.99: 364.17 ------------------------------------------------------ [ 0, 1 ] 42824 40.835% 40.835% ######## ( 1, 2 ] 58697 55.971% 96.806% ########### ( 2, 3 ] 1997 1.904% 98.710% ( 3, 4 ] 804 0.767% 99.476% ( 4, 6 ] 333 0.318% 99.794% ( 6, 10 ] 164 0.156% 99.950% ( 10, 15 ] 23 0.022% 99.972% ( 15, 22 ] 5 0.005% 99.977% ( 22, 34 ] 1 0.001% 99.978% ( 110, 170 ] 9 0.009% 99.987% ( 170, 250 ] 1 0.001% 99.988% ( 250, 380 ] 4 0.004% 99.991% ( 380, 580 ] 1 0.001% 99.992% ( 580, 870 ] 5 0.005% 99.997% ( 1300, 1900 ] 3 0.003% 100.000% ( 4400, 6600 ] 1 0.001% 100.001% ** Level 1 read latency histogram (micros): Count: 19956 Average: 1.7632 StdDev: 9.17 Min: 1 Median: 1.1688 Max: 1008 Percentiles: P50: 1.17 P75: 1.61 P99: 3.25 P99.9: 8.71 P99.99: 581.25 ------------------------------------------------------ [ 0, 1 ] 8062 40.399% 40.399% ######## ( 1, 2 ] 11351 56.880% 97.279% ########### ( 2, 3 ] 302 1.513% 98.792% ( 3, 4 ] 170 0.852% 99.644% ( 4, 6 ] 29 0.145% 99.790% ( 6, 10 ] 34 0.170% 99.960% ( 10, 15 ] 3 0.015% 99.975% ( 15, 22 ] 2 0.010% 99.985% ( 250, 380 ] 2 0.010% 99.995% ( 580, 870 ] 1 0.005% 100.000% ( 870, 1300 ] 1 0.005% 100.005% ** Level 2 read latency histogram (micros): Count: 28161 Average: 2.3389 StdDev: 21.48 Min: 1 Median: 1.2224 Max: 2356 Percentiles: P50: 1.22 P75: 1.68 P99: 6.46 P99.9: 88.51 P99.99: 949.08 ------------------------------------------------------ [ 0, 1 ] 10663 37.864% 37.864% ######## ( 1, 2 ] 15369 54.575% 92.440% ########### ( 2, 3 ] 1193 4.236% 96.676% # ( 3, 4 ] 338 1.200% 97.876% ( 4, 6 ] 295 1.048% 98.924% ( 6, 10 ] 188 0.668% 99.592% ( 10, 15 ] 30 0.107% 99.698% ( 15, 22 ] 5 0.018% 99.716% ( 34, 51 ] 40 0.142% 99.858% ( 51, 76 ] 10 0.036% 99.893% ( 76, 110 ] 5 0.018% 99.911% ( 110, 170 ] 4 0.014% 99.925% ( 170, 250 ] 4 0.014% 99.940% ( 250, 380 ] 5 0.018% 99.957% ( 380, 580 ] 5 0.018% 99.975% ( 580, 870 ] 4 0.014% 99.989% ( 870, 1300 ] 1 0.004% 99.993% ( 1300, 1900 ] 1 0.004% 99.996% ( 1900, 2900 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 90 Average: 932.2889 StdDev: 1379.92 Min: 1 Median: 250.0000 Max: 5439 Percentiles: P50: 250.00 P75: 1120.83 P99: 5439.00 P99.9: 5439.00 P99.99: 5439.00 ------------------------------------------------------ [ 0, 1 ] 21 23.333% 23.333% ##### ( 1, 2 ] 14 15.556% 38.889% ### ( 3, 4 ] 1 1.111% 40.000% ( 6, 10 ] 1 1.111% 41.111% ( 51, 76 ] 3 3.333% 44.444% # ( 110, 170 ] 1 1.111% 45.556% ( 170, 250 ] 4 4.444% 50.000% # ( 250, 380 ] 1 1.111% 51.111% ( 380, 580 ] 5 5.556% 56.667% # ( 580, 870 ] 13 14.444% 71.111% ### ( 870, 1300 ] 6 6.667% 77.778% # ( 1300, 1900 ] 3 3.333% 81.111% # ( 1900, 2900 ] 6 6.667% 87.778% # ( 2900, 4400 ] 8 8.889% 96.667% ## ( 4400, 6600 ] 3 3.333% 100.000% # ** Level 4 read latency histogram (micros): Count: 25 Average: 1499.5600 StdDev: 1410.55 Min: 150 Median: 923.7500 Max: 5123 Percentiles: P50: 923.75 P75: 2150.00 P99: 5123.00 P99.9: 5123.00 P99.99: 5123.00 ------------------------------------------------------ ( 110, 170 ] 2 8.000% 8.000% ## ( 250, 380 ] 1 4.000% 12.000% # ( 380, 580 ] 5 20.000% 32.000% #### ( 580, 870 ] 4 16.000% 48.000% ### ( 870, 1300 ] 4 16.000% 64.000% ### ( 1300, 1900 ] 2 8.000% 72.000% ## ( 1900, 2900 ] 3 12.000% 84.000% ## ( 2900, 4400 ] 3 12.000% 96.000% ## ( 4400, 6600 ] 1 4.000% 100.000% # ** DB Stats ** Uptime(secs): 3.2 total, 3.0 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 310.21 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.349 H:M:S, 11.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 331.28 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.349 H:M:S, 11.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/10/17-18:21:03 ... thread 0: (1000000,2000000) ops and (347426.9,342065.9) ops/second in (2.878303,5.846826) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/8 1.17 GB 62.9 0.6 0.0 0.6 2.3 1.8 0.0 1.3 77.6 327.6 7.30 6.69 28 0.261 575K 589 0.0 0.0 L1 2/0 50.36 MB 5.0 0.8 0.5 0.3 0.8 0.5 0.1 1.6 242.5 242.0 3.38 3.28 2 1.688 831K 1449 0.0 0.0 L2 5/4 563.97 MB 1.3 0.1 0.1 0.0 0.1 0.1 0.5 1.8 247.7 247.7 0.45 0.42 1 0.450 113K 40 0.0 0.0 L3 18/11 944.37 MB 0.0 0.1 0.1 0.0 0.1 0.1 0.0 1.0 197.8 197.9 0.32 0.26 1 0.316 63K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1040/0 85.69 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1192/23 97.92 GB 0.0 1.5 0.6 0.9 3.3 2.4 0.6 1.9 136.2 295.7 11.45 10.66 32 0.358 1583K 2078 0.0 0.0 Int 0/0 0.00 KB 0.0 1.5 0.6 0.9 3.3 2.4 0.6 1.9 136.2 295.7 11.45 10.66 32 0.358 1583K 2078 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.5 0.6 0.9 1.5 0.6 0.0 0.0 249.1 248.7 6.26 6.05 6 1.044 1583K 2078 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 352.3 5.19 4.62 26 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.1 total, 5.8 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.30 GB write, 559.32 MB/s write, 1.52 GB read, 257.75 MB/s read, 11.4 seconds Interval compaction: 3.30 GB write, 578.69 MB/s write, 1.52 GB read, 266.67 MB/s read, 11.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 6, interval: 6 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 386644 Average: 2.1599 StdDev: 11.96 Min: 0 Median: 1.4648 Max: 4601 Percentiles: P50: 1.46 P75: 1.86 P99: 5.64 P99.9: 9.96 P99.99: 14.97 ------------------------------------------------------ [ 0, 1 ] 80285 20.765% 20.765% #### ( 1, 2 ] 243193 62.898% 83.663% ############# ( 2, 3 ] 38881 10.056% 93.719% ## ( 3, 4 ] 12862 3.327% 97.046% # ( 4, 6 ] 9233 2.388% 99.434% ( 6, 10 ] 1825 0.472% 99.906% ( 10, 15 ] 330 0.085% 99.991% ( 15, 22 ] 41 0.011% 100.002% ( 22, 34 ] 11 0.003% 100.004% ( 34, 51 ] 2 0.001% 100.005% ( 51, 76 ] 2 0.001% 100.005% ( 76, 110 ] 1 0.000% 100.006% ( 110, 170 ] 14 0.004% 100.009% ( 170, 250 ] 12 0.003% 100.012% ( 250, 380 ] 8 0.002% 100.014% ( 380, 580 ] 4 0.001% 100.016% ( 580, 870 ] 11 0.003% 100.018% ( 870, 1300 ] 1 0.000% 100.019% ( 1300, 1900 ] 4 0.001% 100.020% ( 2900, 4400 ] 1 0.000% 100.020% ( 4400, 6600 ] 1 0.000% 100.020% ** Level 1 read latency histogram (micros): Count: 108155 Average: 2.3418 StdDev: 8.81 Min: 1 Median: 1.5790 Max: 1665 Percentiles: P50: 1.58 P75: 1.98 P99: 5.86 P99.9: 11.85 P99.99: 280.80 ------------------------------------------------------ [ 0, 1 ] 14954 13.826% 13.826% ### ( 1, 2 ] 67572 62.477% 76.303% ############ ( 2, 3 ] 16550 15.302% 91.606% ### ( 3, 4 ] 4481 4.143% 95.749% # ( 4, 6 ] 3788 3.502% 99.251% # ( 6, 10 ] 658 0.608% 99.859% ( 10, 15 ] 121 0.112% 99.971% ( 15, 22 ] 16 0.015% 99.986% ( 51, 76 ] 1 0.001% 99.987% ( 110, 170 ] 2 0.002% 99.989% ( 170, 250 ] 1 0.001% 99.990% ( 250, 380 ] 5 0.005% 99.994% ( 380, 580 ] 1 0.001% 99.995% ( 580, 870 ] 3 0.003% 99.998% ( 870, 1300 ] 1 0.001% 99.999% ( 1300, 1900 ] 2 0.002% 100.001% ** Level 2 read latency histogram (micros): Count: 28201 Average: 2.3418 StdDev: 21.47 Min: 1 Median: 1.2235 Max: 2356 Percentiles: P50: 1.22 P75: 1.68 P99: 6.47 P99.9: 88.23 P99.99: 947.36 ------------------------------------------------------ [ 0, 1 ] 10663 37.811% 37.811% ######## ( 1, 2 ] 15382 54.544% 92.355% ########### ( 2, 3 ] 1216 4.312% 96.667% # ( 3, 4 ] 341 1.209% 97.876% ( 4, 6 ] 295 1.046% 98.922% ( 6, 10 ] 188 0.667% 99.589% ( 10, 15 ] 30 0.106% 99.695% ( 15, 22 ] 5 0.018% 99.713% ( 34, 51 ] 40 0.142% 99.855% ( 51, 76 ] 11 0.039% 99.894% ( 76, 110 ] 5 0.018% 99.911% ( 110, 170 ] 4 0.014% 99.926% ( 170, 250 ] 4 0.014% 99.940% ( 250, 380 ] 5 0.018% 99.957% ( 380, 580 ] 5 0.018% 99.975% ( 580, 870 ] 4 0.014% 99.989% ( 870, 1300 ] 1 0.004% 99.993% ( 1300, 1900 ] 1 0.004% 99.996% ( 1900, 2900 ] 1 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 66408 Average: 4.7038 StdDev: 64.89 Min: 1 Median: 1.7756 Max: 5439 Percentiles: P50: 1.78 P75: 2.55 P99: 9.74 P99.9: 423.73 P99.99: 2959.87 ------------------------------------------------------ [ 0, 1 ] 3782 5.695% 5.695% # ( 1, 2 ] 37933 57.121% 62.816% ########### ( 2, 3 ] 14613 22.005% 84.821% #### ( 3, 4 ] 4012 6.041% 90.863% # ( 4, 6 ] 4039 6.082% 96.945% # ( 6, 10 ] 1459 2.197% 99.142% ( 10, 15 ] 234 0.352% 99.494% ( 15, 22 ] 58 0.087% 99.581% ( 22, 34 ] 19 0.029% 99.610% ( 34, 51 ] 2 0.003% 99.613% ( 51, 76 ] 4 0.006% 99.619% ( 76, 110 ] 52 0.078% 99.697% ( 110, 170 ] 65 0.098% 99.795% ( 170, 250 ] 49 0.074% 99.869% ( 250, 380 ] 17 0.026% 99.895% ( 380, 580 ] 21 0.032% 99.926% ( 580, 870 ] 24 0.036% 99.962% ( 870, 1300 ] 7 0.011% 99.973% ( 1300, 1900 ] 6 0.009% 99.982% ( 1900, 2900 ] 6 0.009% 99.991% ( 2900, 4400 ] 9 0.014% 100.005% ( 4400, 6600 ] 3 0.005% 100.009% ** Level 4 read latency histogram (micros): Count: 25 Average: 1499.5600 StdDev: 1410.55 Min: 150 Median: 923.7500 Max: 5123 Percentiles: P50: 923.75 P75: 2150.00 P99: 5123.00 P99.9: 5123.00 P99.99: 5123.00 ------------------------------------------------------ ( 110, 170 ] 2 8.000% 8.000% ## ( 250, 380 ] 1 4.000% 12.000% # ( 380, 580 ] 5 20.000% 32.000% #### ( 580, 870 ] 4 16.000% 48.000% ### ( 870, 1300 ] 4 16.000% 64.000% ### ( 1300, 1900 ] 2 8.000% 72.000% ## ( 1900, 2900 ] 3 12.000% 84.000% ## ( 2900, 4400 ] 3 12.000% 96.000% ## ( 4400, 6600 ] 1 4.000% 100.000% # ** DB Stats ** Uptime(secs): 6.1 total, 2.9 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 325.30 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.544 H:M:S, 9.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 341.94 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.195 H:M:S, 6.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 2024/10/17-18:21:06 ... thread 0: (1000000,3000000) ops and (325998.3,336536.9) ops/second in (3.067501,8.914327) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/4 2.15 GB 188.8 1.7 0.0 1.7 4.4 2.8 0.0 1.6 116.3 310.0 14.63 13.55 46 0.318 1726K 1861 0.0 0.0 L1 2/0 50.36 MB 5.0 0.8 0.5 0.3 0.8 0.5 0.1 1.6 242.5 242.0 3.38 3.28 2 1.688 831K 1449 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.5 1.8 247.7 247.7 0.45 0.42 1 0.450 113K 40 0.0 0.0 L3 26/4 1.09 GB 1.0 1.5 0.6 0.9 1.5 0.6 0.0 2.5 211.5 211.3 7.50 6.64 4 1.874 1591K 801 0.0 0.0 L4 152/0 9.55 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 86.06 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1237/8 98.90 GB 0.0 4.1 1.2 3.0 6.9 3.9 1.4 2.5 162.5 271.5 25.95 23.90 53 0.490 4262K 4151 0.0 0.0 Int 0/0 0.00 KB 0.0 4.1 1.2 3.0 6.9 3.9 1.4 2.5 162.5 271.5 25.95 23.90 53 0.490 4262K 4151 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.1 1.2 3.0 4.1 1.2 0.0 0.0 235.4 235.1 17.91 16.76 13 1.378 4262K 4151 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 352.7 8.04 7.14 40 0.201 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.1 total, 8.9 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.88 GB write, 772.83 MB/s write, 4.12 GB read, 462.42 MB/s read, 26.0 seconds Interval compaction: 6.88 GB write, 790.38 MB/s write, 4.12 GB read, 472.92 MB/s read, 26.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 10, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 10, interval: 10 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 632205 Average: 2.2479 StdDev: 10.60 Min: 0 Median: 1.5223 Max: 4601 Percentiles: P50: 1.52 P75: 1.91 P99: 5.71 P99.9: 9.94 P99.99: 14.87 ------------------------------------------------------ [ 0, 1 ] 104847 16.584% 16.584% ### ( 1, 2 ] 404479 63.979% 80.563% ############# ( 2, 3 ] 75181 11.892% 92.455% ## ( 3, 4 ] 25614 4.052% 96.507% # ( 4, 6 ] 18476 2.922% 99.429% # ( 6, 10 ] 3019 0.478% 99.907% ( 10, 15 ] 540 0.085% 99.992% ( 15, 22 ] 55 0.009% 100.001% ( 22, 34 ] 12 0.002% 100.003% ( 34, 51 ] 4 0.001% 100.003% ( 51, 76 ] 5 0.001% 100.004% ( 76, 110 ] 1 0.000% 100.004% ( 110, 170 ] 16 0.003% 100.007% ( 170, 250 ] 21 0.003% 100.010% ( 250, 380 ] 18 0.003% 100.013% ( 380, 580 ] 7 0.001% 100.014% ( 580, 870 ] 17 0.003% 100.017% ( 870, 1300 ] 4 0.001% 100.018% ( 1300, 1900 ] 7 0.001% 100.019% ( 2900, 4400 ] 1 0.000% 100.019% ( 4400, 6600 ] 1 0.000% 100.019% ** Level 1 read latency histogram (micros): Count: 194728 Average: 2.4637 StdDev: 6.62 Min: 1 Median: 1.6748 Max: 1665 Percentiles: P50: 1.67 P75: 2.26 P99: 5.93 P99.9: 12.71 P99.99: 21.89 ------------------------------------------------------ [ 0, 1 ] 17119 8.791% 8.791% ## ( 1, 2 ] 118908 61.064% 69.855% ############ ( 2, 3 ] 38102 19.567% 89.422% #### ( 3, 4 ] 9757 5.011% 94.432% # ( 4, 6 ] 9201 4.725% 99.157% # ( 6, 10 ] 1274 0.654% 99.812% ( 10, 15 ] 318 0.163% 99.975% ( 15, 22 ] 30 0.015% 99.990% ( 22, 34 ] 1 0.001% 99.991% ( 34, 51 ] 1 0.001% 99.991% ( 51, 76 ] 2 0.001% 99.992% ( 110, 170 ] 2 0.001% 99.993% ( 170, 250 ] 1 0.001% 99.994% ( 250, 380 ] 5 0.003% 99.996% ( 380, 580 ] 1 0.001% 99.997% ( 580, 870 ] 3 0.002% 99.998% ( 870, 1300 ] 1 0.001% 99.999% ( 1300, 1900 ] 2 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 40582 Average: 2.4031 StdDev: 18.82 Min: 1 Median: 1.3320 Max: 2356 Percentiles: P50: 1.33 P75: 1.76 P99: 7.08 P99.9: 112.28 P99.99: 808.62 ------------------------------------------------------ [ 0, 1 ] 12396 30.546% 30.546% ###### ( 1, 2 ] 23781 58.600% 89.145% ############ ( 2, 3 ] 2651 6.532% 95.678% # ( 3, 4 ] 706 1.740% 97.418% ( 4, 6 ] 546 1.345% 98.763% ( 6, 10 ] 355 0.875% 99.638% ( 10, 15 ] 37 0.091% 99.729% ( 15, 22 ] 6 0.015% 99.744% ( 22, 34 ] 2 0.005% 99.749% ( 34, 51 ] 40 0.099% 99.847% ( 51, 76 ] 11 0.027% 99.874% ( 76, 110 ] 10 0.025% 99.899% ( 110, 170 ] 11 0.027% 99.926% ( 170, 250 ] 9 0.022% 99.948% ( 250, 380 ] 6 0.015% 99.963% ( 380, 580 ] 7 0.017% 99.980% ( 580, 870 ] 5 0.012% 99.993% ( 870, 1300 ] 1 0.002% 99.995% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 250771 Average: 3.7642 StdDev: 37.95 Min: 1 Median: 1.7094 Max: 5439 Percentiles: P50: 1.71 P75: 2.38 P99: 9.32 P99.9: 272.15 P99.99: 774.61 ------------------------------------------------------ [ 0, 1 ] 16932 6.752% 6.752% # ( 1, 2 ] 152884 60.966% 67.718% ############ ( 2, 3 ] 48275 19.251% 86.968% #### ( 3, 4 ] 14366 5.729% 92.697% # ( 4, 6 ] 11854 4.727% 97.424% # ( 6, 10 ] 4768 1.901% 99.325% ( 10, 15 ] 609 0.243% 99.568% ( 15, 22 ] 104 0.041% 99.610% ( 22, 34 ] 26 0.010% 99.620% ( 34, 51 ] 7 0.003% 99.623% ( 51, 76 ] 41 0.016% 99.639% ( 76, 110 ] 180 0.072% 99.711% ( 110, 170 ] 301 0.120% 99.831% ( 170, 250 ] 155 0.062% 99.893% ( 250, 380 ] 107 0.043% 99.935% ( 380, 580 ] 98 0.039% 99.974% ( 580, 870 ] 58 0.023% 99.998% ( 870, 1300 ] 14 0.006% 100.003% ( 1300, 1900 ] 9 0.004% 100.007% ( 1900, 2900 ] 8 0.003% 100.010% ( 2900, 4400 ] 9 0.004% 100.014% ( 4400, 6600 ] 3 0.001% 100.015% ** Level 4 read latency histogram (micros): Count: 25 Average: 1499.5600 StdDev: 1410.55 Min: 150 Median: 923.7500 Max: 5123 Percentiles: P50: 923.75 P75: 2150.00 P99: 5123.00 P99.9: 5123.00 P99.99: 5123.00 ------------------------------------------------------ ( 110, 170 ] 2 8.000% 8.000% ## ( 250, 380 ] 1 4.000% 12.000% # ( 380, 580 ] 5 20.000% 32.000% #### ( 580, 870 ] 4 16.000% 48.000% ### ( 870, 1300 ] 4 16.000% 64.000% ### ( 1300, 1900 ] 2 8.000% 72.000% ## ( 1900, 2900 ] 3 12.000% 84.000% ## ( 2900, 4400 ] 3 12.000% 96.000% ## ( 4400, 6600 ] 1 4.000% 100.000% # ** Level 5 read latency histogram (micros): Count: 20 Average: 921.3000 StdDev: 1049.10 Min: 100 Median: 380.0000 Max: 3736 Percentiles: P50: 380.00 P75: 1300.00 P99: 3736.00 P99.9: 3736.00 P99.99: 3736.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 3 15.000% 20.000% ### ( 170, 250 ] 5 25.000% 45.000% ##### ( 250, 380 ] 1 5.000% 50.000% # ( 380, 580 ] 1 5.000% 55.000% # ( 580, 870 ] 1 5.000% 60.000% # ( 870, 1300 ] 3 15.000% 75.000% ### ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 2 10.000% 90.000% ## ( 2900, 4400 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 9.1 total, 3.1 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 323.80 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.918 H:M:S, 10.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 320.85 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.374 H:M:S, 12.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/17-18:21:09 ... thread 0: (1000000,4000000) ops and (326726.4,334029.5) ops/second in (3.060665,11.974992) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/15 3.07 GB 31.5 2.5 0.0 2.5 6.1 3.7 0.0 1.7 123.4 308.4 20.42 18.97 62 0.329 2558K 2689 0.0 0.0 L1 2/2 50.36 MB 0.0 0.8 0.5 0.3 0.8 0.5 0.1 1.6 242.5 242.0 3.38 3.28 2 1.688 831K 1449 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.5 1.8 247.7 247.7 0.45 0.42 1 0.450 113K 40 0.0 0.0 L3 22/0 955.33 MB 1.0 1.5 0.6 0.9 1.5 0.6 0.0 2.5 211.5 211.3 7.50 6.64 4 1.874 1591K 801 0.0 0.0 L4 169/0 9.71 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.4 1.0 189.4 189.5 0.84 0.72 1 0.836 130K 0 0.0 0.0 L5 1044/0 86.06 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1254/17 99.82 GB 0.0 5.1 1.3 3.8 8.8 5.0 1.4 2.4 159.4 275.3 32.58 30.04 70 0.465 5224K 4979 0.0 0.0 Int 0/0 0.00 KB 0.0 5.1 1.3 3.8 8.8 5.0 1.4 2.4 159.4 275.3 32.58 30.04 70 0.465 5224K 4979 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.1 1.3 3.8 5.1 1.3 0.0 0.0 237.8 237.6 21.84 20.50 17 1.285 5224K 4979 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 352.1 10.74 9.54 53 0.203 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.2 total, 12.0 interval Flush(GB): cumulative 3.691, interval 3.691 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.76 GB write, 736.38 MB/s write, 5.07 GB read, 426.46 MB/s read, 32.6 seconds Interval compaction: 8.76 GB write, 748.83 MB/s write, 5.07 GB read, 433.67 MB/s read, 32.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 16, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 16, interval: 16 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 931886 Average: 2.2645 StdDev: 10.90 Min: 0 Median: 1.5108 Max: 4601 Percentiles: P50: 1.51 P75: 1.90 P99: 5.70 P99.9: 11.35 P99.99: 123.54 ------------------------------------------------------ [ 0, 1 ] 157614 16.913% 16.913% ### ( 1, 2 ] 603579 64.770% 81.683% ############# ( 2, 3 ] 106622 11.442% 93.125% ## ( 3, 4 ] 34364 3.688% 96.812% # ( 4, 6 ] 23997 2.575% 99.387% # ( 6, 10 ] 4543 0.488% 99.875% ( 10, 15 ] 878 0.094% 99.969% ( 15, 22 ] 90 0.010% 99.979% ( 22, 34 ] 14 0.002% 99.980% ( 34, 51 ] 5 0.001% 99.981% ( 51, 76 ] 6 0.001% 99.981% ( 76, 110 ] 51 0.005% 99.987% ( 110, 170 ] 141 0.015% 100.002% ( 170, 250 ] 63 0.007% 100.009% ( 250, 380 ] 42 0.005% 100.013% ( 380, 580 ] 24 0.003% 100.016% ( 580, 870 ] 27 0.003% 100.019% ( 870, 1300 ] 6 0.001% 100.019% ( 1300, 1900 ] 11 0.001% 100.020% ( 2900, 4400 ] 2 0.000% 100.021% ( 4400, 6600 ] 1 0.000% 100.021% ** Level 1 read latency histogram (micros): Count: 194729 Average: 2.4644 StdDev: 6.63 Min: 1 Median: 1.6749 Max: 1665 Percentiles: P50: 1.67 P75: 2.26 P99: 5.93 P99.9: 12.72 P99.99: 28.33 ------------------------------------------------------ [ 0, 1 ] 17119 8.791% 8.791% ## ( 1, 2 ] 118908 61.063% 69.855% ############ ( 2, 3 ] 38102 19.567% 89.421% #### ( 3, 4 ] 9757 5.011% 94.432% # ( 4, 6 ] 9201 4.725% 99.157% # ( 6, 10 ] 1274 0.654% 99.811% ( 10, 15 ] 318 0.163% 99.974% ( 15, 22 ] 30 0.015% 99.990% ( 22, 34 ] 1 0.001% 99.990% ( 34, 51 ] 1 0.001% 99.991% ( 51, 76 ] 2 0.001% 99.992% ( 110, 170 ] 3 0.002% 99.993% ( 170, 250 ] 1 0.001% 99.994% ( 250, 380 ] 5 0.003% 99.996% ( 380, 580 ] 1 0.001% 99.997% ( 580, 870 ] 3 0.002% 99.998% ( 870, 1300 ] 1 0.001% 99.999% ( 1300, 1900 ] 2 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 40582 Average: 2.4031 StdDev: 18.82 Min: 1 Median: 1.3320 Max: 2356 Percentiles: P50: 1.33 P75: 1.76 P99: 7.08 P99.9: 112.28 P99.99: 808.62 ------------------------------------------------------ [ 0, 1 ] 12396 30.546% 30.546% ###### ( 1, 2 ] 23781 58.600% 89.145% ############ ( 2, 3 ] 2651 6.532% 95.678% # ( 3, 4 ] 706 1.740% 97.418% ( 4, 6 ] 546 1.345% 98.763% ( 6, 10 ] 355 0.875% 99.638% ( 10, 15 ] 37 0.091% 99.729% ( 15, 22 ] 6 0.015% 99.744% ( 22, 34 ] 2 0.005% 99.749% ( 34, 51 ] 40 0.099% 99.847% ( 51, 76 ] 11 0.027% 99.874% ( 76, 110 ] 10 0.025% 99.899% ( 110, 170 ] 11 0.027% 99.926% ( 170, 250 ] 9 0.022% 99.948% ( 250, 380 ] 6 0.015% 99.963% ( 380, 580 ] 7 0.017% 99.980% ( 580, 870 ] 5 0.012% 99.993% ( 870, 1300 ] 1 0.002% 99.995% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 284156 Average: 3.6141 StdDev: 35.66 Min: 1 Median: 1.7105 Max: 5439 Percentiles: P50: 1.71 P75: 2.37 P99: 9.14 P99.9: 242.18 P99.99: 757.92 ------------------------------------------------------ [ 0, 1 ] 18485 6.505% 6.505% # ( 1, 2 ] 173947 61.215% 67.721% ############ ( 2, 3 ] 55679 19.595% 87.315% #### ( 3, 4 ] 15844 5.576% 92.891% # ( 4, 6 ] 13385 4.710% 97.601% # ( 6, 10 ] 5060 1.781% 99.382% ( 10, 15 ] 662 0.233% 99.615% ( 15, 22 ] 113 0.040% 99.655% ( 22, 34 ] 27 0.010% 99.664% ( 34, 51 ] 7 0.002% 99.667% ( 51, 76 ] 42 0.015% 99.682% ( 76, 110 ] 180 0.063% 99.745% ( 110, 170 ] 301 0.106% 99.851% ( 170, 250 ] 155 0.055% 99.905% ( 250, 380 ] 107 0.038% 99.943% ( 380, 580 ] 98 0.034% 99.977% ( 580, 870 ] 58 0.020% 99.998% ( 870, 1300 ] 14 0.005% 100.003% ( 1300, 1900 ] 9 0.003% 100.006% ( 1900, 2900 ] 8 0.003% 100.009% ( 2900, 4400 ] 9 0.003% 100.012% ( 4400, 6600 ] 3 0.001% 100.013% ** Level 4 read latency histogram (micros): Count: 110 Average: 349.0273 StdDev: 917.59 Min: 1 Median: 6.5333 Max: 5123 Percentiles: P50: 6.53 P75: 42.50 P99: 4350.00 P99.9: 5123.00 P99.99: 5123.00 ------------------------------------------------------ [ 0, 1 ] 28 25.455% 25.455% ##### ( 1, 2 ] 24 21.818% 47.273% #### ( 4, 6 ] 1 0.909% 48.182% ( 6, 10 ] 15 13.636% 61.818% ### ( 15, 22 ] 1 0.909% 62.727% ( 22, 34 ] 12 10.909% 73.636% ## ( 34, 51 ] 3 2.727% 76.364% # ( 110, 170 ] 2 1.818% 78.182% ( 170, 250 ] 1 0.909% 79.091% ( 250, 380 ] 1 0.909% 80.000% ( 380, 580 ] 5 4.545% 84.545% # ( 580, 870 ] 4 3.636% 88.182% # ( 870, 1300 ] 4 3.636% 91.818% # ( 1300, 1900 ] 2 1.818% 93.636% ( 1900, 2900 ] 3 2.727% 96.364% # ( 2900, 4400 ] 3 2.727% 99.091% # ( 4400, 6600 ] 1 0.909% 100.000% ** Level 5 read latency histogram (micros): Count: 20 Average: 921.3000 StdDev: 1049.10 Min: 100 Median: 380.0000 Max: 3736 Percentiles: P50: 380.00 P75: 1300.00 P99: 3736.00 P99.9: 3736.00 P99.99: 3736.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 3 15.000% 20.000% ### ( 170, 250 ] 5 25.000% 45.000% ##### ( 250, 380 ] 1 5.000% 50.000% # ( 380, 580 ] 1 5.000% 55.000% # ( 580, 870 ] 1 5.000% 60.000% # ( 870, 1300 ] 3 15.000% 75.000% ### ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 2 10.000% 90.000% ## ( 2900, 4400 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 12.2 total, 3.1 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 323.24 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.322 H:M:S, 10.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 321.56 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.405 H:M:S, 13.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/17-18:21:15 ... thread 0: (1000000,5000000) ops and (185435.2,287890.6) ops/second in (5.392718,17.367710) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/16 4.12 GB 62.9 3.8 0.0 3.8 8.5 4.7 0.0 1.8 135.5 306.5 28.35 26.48 84 0.338 3901K 4036 0.0 0.0 L1 2/2 50.36 MB 0.0 0.8 0.5 0.3 0.8 0.5 0.1 1.6 242.5 242.0 3.38 3.28 2 1.688 831K 1449 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 0.5 1.8 247.7 247.7 0.45 0.42 1 0.450 113K 40 0.0 0.0 L3 22/0 955.33 MB 1.0 1.5 0.6 0.9 1.5 0.6 0.0 2.5 211.5 211.3 7.50 6.64 4 1.874 1591K 801 0.0 0.0 L4 169/0 9.71 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.4 1.0 189.4 189.5 0.84 0.72 1 0.836 130K 0 0.0 0.0 L5 1044/0 86.06 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1256/18 100.87 GB 0.0 6.4 1.3 5.0 11.1 6.1 1.4 2.3 160.9 280.4 40.51 37.55 92 0.440 6568K 6326 0.0 0.0 Int 0/0 0.00 KB 0.0 6.4 1.3 5.0 11.1 6.1 1.4 2.3 160.9 280.4 40.51 37.55 92 0.440 6568K 6326 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.4 1.3 5.0 6.4 1.3 0.0 0.0 243.2 242.9 26.80 25.37 22 1.218 6568K 6326 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 353.7 13.71 12.17 70 0.196 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 17.6 total, 17.4 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.09 GB write, 646.54 MB/s write, 6.36 GB read, 370.87 MB/s read, 40.5 seconds Interval compaction: 11.09 GB write, 654.08 MB/s write, 6.36 GB read, 375.19 MB/s read, 40.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 9, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 9, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 16, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 9, total-stops: 16, interval: 25 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1575898 Average: 2.2605 StdDev: 11.13 Min: 0 Median: 1.5094 Max: 4601 Percentiles: P50: 1.51 P75: 1.89 P99: 5.71 P99.9: 11.12 P99.99: 97.26 ------------------------------------------------------ [ 0, 1 ] 259623 16.475% 16.475% ### ( 1, 2 ] 1037129 65.812% 82.287% ############# ( 2, 3 ] 178519 11.328% 93.615% ## ( 3, 4 ] 53830 3.416% 97.030% # ( 4, 6 ] 36336 2.306% 99.336% ( 6, 10 ] 8563 0.543% 99.880% ( 10, 15 ] 1445 0.092% 99.971% ( 15, 22 ] 163 0.010% 99.982% ( 22, 34 ] 17 0.001% 99.983% ( 34, 51 ] 12 0.001% 99.983% ( 51, 76 ] 10 0.001% 99.984% ( 76, 110 ] 151 0.010% 99.994% ( 110, 170 ] 210 0.013% 100.007% ( 170, 250 ] 107 0.007% 100.014% ( 250, 380 ] 59 0.004% 100.018% ( 380, 580 ] 41 0.003% 100.020% ( 580, 870 ] 49 0.003% 100.023% ( 870, 1300 ] 15 0.001% 100.024% ( 1300, 1900 ] 14 0.001% 100.025% ( 1900, 2900 ] 3 0.000% 100.025% ( 2900, 4400 ] 5 0.000% 100.026% ( 4400, 6600 ] 1 0.000% 100.026% ** Level 1 read latency histogram (micros): Count: 194729 Average: 2.4644 StdDev: 6.63 Min: 1 Median: 1.6749 Max: 1665 Percentiles: P50: 1.67 P75: 2.26 P99: 5.93 P99.9: 12.72 P99.99: 28.33 ------------------------------------------------------ [ 0, 1 ] 17119 8.791% 8.791% ## ( 1, 2 ] 118908 61.063% 69.855% ############ ( 2, 3 ] 38102 19.567% 89.421% #### ( 3, 4 ] 9757 5.011% 94.432% # ( 4, 6 ] 9201 4.725% 99.157% # ( 6, 10 ] 1274 0.654% 99.811% ( 10, 15 ] 318 0.163% 99.974% ( 15, 22 ] 30 0.015% 99.990% ( 22, 34 ] 1 0.001% 99.990% ( 34, 51 ] 1 0.001% 99.991% ( 51, 76 ] 2 0.001% 99.992% ( 110, 170 ] 3 0.002% 99.993% ( 170, 250 ] 1 0.001% 99.994% ( 250, 380 ] 5 0.003% 99.996% ( 380, 580 ] 1 0.001% 99.997% ( 580, 870 ] 3 0.002% 99.998% ( 870, 1300 ] 1 0.001% 99.999% ( 1300, 1900 ] 2 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 40582 Average: 2.4031 StdDev: 18.82 Min: 1 Median: 1.3320 Max: 2356 Percentiles: P50: 1.33 P75: 1.76 P99: 7.08 P99.9: 112.28 P99.99: 808.62 ------------------------------------------------------ [ 0, 1 ] 12396 30.546% 30.546% ###### ( 1, 2 ] 23781 58.600% 89.145% ############ ( 2, 3 ] 2651 6.532% 95.678% # ( 3, 4 ] 706 1.740% 97.418% ( 4, 6 ] 546 1.345% 98.763% ( 6, 10 ] 355 0.875% 99.638% ( 10, 15 ] 37 0.091% 99.729% ( 15, 22 ] 6 0.015% 99.744% ( 22, 34 ] 2 0.005% 99.749% ( 34, 51 ] 40 0.099% 99.847% ( 51, 76 ] 11 0.027% 99.874% ( 76, 110 ] 10 0.025% 99.899% ( 110, 170 ] 11 0.027% 99.926% ( 170, 250 ] 9 0.022% 99.948% ( 250, 380 ] 6 0.015% 99.963% ( 380, 580 ] 7 0.017% 99.980% ( 580, 870 ] 5 0.012% 99.993% ( 870, 1300 ] 1 0.002% 99.995% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 284156 Average: 3.6141 StdDev: 35.66 Min: 1 Median: 1.7105 Max: 5439 Percentiles: P50: 1.71 P75: 2.37 P99: 9.14 P99.9: 242.18 P99.99: 757.92 ------------------------------------------------------ [ 0, 1 ] 18485 6.505% 6.505% # ( 1, 2 ] 173947 61.215% 67.721% ############ ( 2, 3 ] 55679 19.595% 87.315% #### ( 3, 4 ] 15844 5.576% 92.891% # ( 4, 6 ] 13385 4.710% 97.601% # ( 6, 10 ] 5060 1.781% 99.382% ( 10, 15 ] 662 0.233% 99.615% ( 15, 22 ] 113 0.040% 99.655% ( 22, 34 ] 27 0.010% 99.664% ( 34, 51 ] 7 0.002% 99.667% ( 51, 76 ] 42 0.015% 99.682% ( 76, 110 ] 180 0.063% 99.745% ( 110, 170 ] 301 0.106% 99.851% ( 170, 250 ] 155 0.055% 99.905% ( 250, 380 ] 107 0.038% 99.943% ( 380, 580 ] 98 0.034% 99.977% ( 580, 870 ] 58 0.020% 99.998% ( 870, 1300 ] 14 0.005% 100.003% ( 1300, 1900 ] 9 0.003% 100.006% ( 1900, 2900 ] 8 0.003% 100.009% ( 2900, 4400 ] 9 0.003% 100.012% ( 4400, 6600 ] 3 0.001% 100.013% ** Level 4 read latency histogram (micros): Count: 110 Average: 349.0273 StdDev: 917.59 Min: 1 Median: 6.5333 Max: 5123 Percentiles: P50: 6.53 P75: 42.50 P99: 4350.00 P99.9: 5123.00 P99.99: 5123.00 ------------------------------------------------------ [ 0, 1 ] 28 25.455% 25.455% ##### ( 1, 2 ] 24 21.818% 47.273% #### ( 4, 6 ] 1 0.909% 48.182% ( 6, 10 ] 15 13.636% 61.818% ### ( 15, 22 ] 1 0.909% 62.727% ( 22, 34 ] 12 10.909% 73.636% ## ( 34, 51 ] 3 2.727% 76.364% # ( 110, 170 ] 2 1.818% 78.182% ( 170, 250 ] 1 0.909% 79.091% ( 250, 380 ] 1 0.909% 80.000% ( 380, 580 ] 5 4.545% 84.545% # ( 580, 870 ] 4 3.636% 88.182% # ( 870, 1300 ] 4 3.636% 91.818% # ( 1300, 1900 ] 2 1.818% 93.636% ( 1900, 2900 ] 3 2.727% 96.364% # ( 2900, 4400 ] 3 2.727% 99.091% # ( 4400, 6600 ] 1 0.909% 100.000% ** Level 5 read latency histogram (micros): Count: 20 Average: 921.3000 StdDev: 1049.10 Min: 100 Median: 380.0000 Max: 3736 Percentiles: P50: 380.00 P75: 1300.00 P99: 3736.00 P99.9: 3736.00 P99.99: 3736.00 ------------------------------------------------------ ( 76, 110 ] 1 5.000% 5.000% # ( 110, 170 ] 3 15.000% 20.000% ### ( 170, 250 ] 5 25.000% 45.000% ##### ( 250, 380 ] 1 5.000% 50.000% # ( 380, 580 ] 1 5.000% 55.000% # ( 580, 870 ] 1 5.000% 60.000% # ( 870, 1300 ] 3 15.000% 75.000% ### ( 1300, 1900 ] 1 5.000% 80.000% # ( 1900, 2900 ] 2 10.000% 90.000% ## ( 2900, 4400 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 17.6 total, 5.4 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 280.05 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.902 H:M:S, 22.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 182.50 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.580 H:M:S, 47.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/17-18:21:21 ... thread 0: (1000000,6000000) ops and (154839.0,251825.4) ops/second in (6.458320,23.826030) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/11 2.52 GB 25.2 5.0 0.0 5.0 10.6 5.6 0.0 1.9 143.5 302.6 35.98 33.59 99 0.363 5244K 8960 0.0 0.0 L1 16/16 1.47 GB 0.0 3.3 3.0 0.4 3.3 2.9 0.1 1.1 224.2 222.1 15.10 14.36 3 5.034 3438K 31K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 1.5 1.8 247.7 247.7 0.45 0.42 1 0.450 113K 40 0.0 0.0 L3 14/0 926.48 MB 0.9 1.5 0.6 0.9 1.5 0.6 1.0 2.5 211.5 211.3 7.50 6.64 4 1.874 1591K 801 0.0 0.0 L4 172/0 9.75 GB 1.0 0.3 0.3 0.0 0.3 0.3 1.3 1.1 195.8 195.3 1.66 1.46 2 0.832 300K 863 0.0 0.0 L5 1056/0 87.05 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1272/27 101.69 GB 0.0 10.3 3.9 6.4 15.9 9.5 5.3 2.8 174.2 267.9 60.70 56.47 109 0.557 10M 42K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.3 3.9 6.4 15.9 9.5 5.3 2.8 174.2 267.9 60.70 56.47 109 0.557 10M 42K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.3 3.9 6.4 10.3 3.9 0.0 0.0 237.9 236.9 44.45 42.06 26 1.710 10M 42K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.6 5.6 0.0 0.0 0.0 352.9 16.24 14.41 83 0.196 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 24.0 total, 23.8 interval Flush(GB): cumulative 5.598, interval 5.598 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 15.88 GB write, 676.79 MB/s write, 10.33 GB read, 440.02 MB/s read, 60.7 seconds Interval compaction: 15.88 GB write, 682.54 MB/s write, 10.33 GB read, 443.76 MB/s read, 60.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 11, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 11, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 11, total-stops: 20, interval: 31 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2219964 Average: 2.4560 StdDev: 13.45 Min: 0 Median: 1.5029 Max: 4601 Percentiles: P50: 1.50 P75: 1.88 P99: 5.88 P99.9: 104.19 P99.99: 216.19 ------------------------------------------------------ [ 0, 1 ] 370742 16.700% 16.700% ### ( 1, 2 ] 1470061 66.220% 82.920% ############# ( 2, 3 ] 240951 10.854% 93.774% ## ( 3, 4 ] 71409 3.217% 96.991% # ( 4, 6 ] 47368 2.134% 99.125% ( 6, 10 ] 14004 0.631% 99.755% ( 10, 15 ] 1879 0.085% 99.840% ( 15, 22 ] 218 0.010% 99.850% ( 22, 34 ] 20 0.001% 99.851% ( 34, 51 ] 13 0.001% 99.851% ( 51, 76 ] 38 0.002% 99.853% ( 76, 110 ] 1257 0.057% 99.910% ( 110, 170 ] 1548 0.070% 99.979% ( 170, 250 ] 407 0.018% 99.998% ( 250, 380 ] 242 0.011% 100.009% ( 380, 580 ] 178 0.008% 100.017% ( 580, 870 ] 112 0.005% 100.022% ( 870, 1300 ] 24 0.001% 100.023% ( 1300, 1900 ] 17 0.001% 100.024% ( 1900, 2900 ] 7 0.000% 100.024% ( 2900, 4400 ] 9 0.000% 100.024% ( 4400, 6600 ] 1 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 240337 Average: 2.8009 StdDev: 19.07 Min: 1 Median: 1.6539 Max: 3372 Percentiles: P50: 1.65 P75: 2.19 P99: 5.99 P99.9: 109.07 P99.99: 841.92 ------------------------------------------------------ [ 0, 1 ] 22706 9.448% 9.448% ## ( 1, 2 ] 149040 62.013% 71.460% ############ ( 2, 3 ] 44792 18.637% 90.098% #### ( 3, 4 ] 11237 4.676% 94.773% # ( 4, 6 ] 10234 4.258% 99.031% # ( 6, 10 ] 1549 0.645% 99.676% ( 10, 15 ] 372 0.155% 99.831% ( 15, 22 ] 38 0.016% 99.846% ( 22, 34 ] 2 0.001% 99.847% ( 34, 51 ] 1 0.000% 99.848% ( 51, 76 ] 7 0.003% 99.851% ( 76, 110 ] 122 0.051% 99.901% ( 110, 170 ] 116 0.048% 99.950% ( 170, 250 ] 30 0.012% 99.962% ( 250, 380 ] 28 0.012% 99.974% ( 380, 580 ] 20 0.008% 99.982% ( 580, 870 ] 21 0.009% 99.991% ( 870, 1300 ] 8 0.003% 99.994% ( 1300, 1900 ] 10 0.004% 99.998% ( 1900, 2900 ] 2 0.001% 99.999% ( 2900, 4400 ] 2 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 40582 Average: 2.4031 StdDev: 18.82 Min: 1 Median: 1.3320 Max: 2356 Percentiles: P50: 1.33 P75: 1.76 P99: 7.08 P99.9: 112.28 P99.99: 808.62 ------------------------------------------------------ [ 0, 1 ] 12396 30.546% 30.546% ###### ( 1, 2 ] 23781 58.600% 89.145% ############ ( 2, 3 ] 2651 6.532% 95.678% # ( 3, 4 ] 706 1.740% 97.418% ( 4, 6 ] 546 1.345% 98.763% ( 6, 10 ] 355 0.875% 99.638% ( 10, 15 ] 37 0.091% 99.729% ( 15, 22 ] 6 0.015% 99.744% ( 22, 34 ] 2 0.005% 99.749% ( 34, 51 ] 40 0.099% 99.847% ( 51, 76 ] 11 0.027% 99.874% ( 76, 110 ] 10 0.025% 99.899% ( 110, 170 ] 11 0.027% 99.926% ( 170, 250 ] 9 0.022% 99.948% ( 250, 380 ] 6 0.015% 99.963% ( 380, 580 ] 7 0.017% 99.980% ( 580, 870 ] 5 0.012% 99.993% ( 870, 1300 ] 1 0.002% 99.995% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 293934 Average: 3.7619 StdDev: 36.01 Min: 1 Median: 1.7067 Max: 5439 Percentiles: P50: 1.71 P75: 2.36 P99: 9.31 P99.9: 288.47 P99.99: 777.59 ------------------------------------------------------ [ 0, 1 ] 19894 6.768% 6.768% # ( 1, 2 ] 179804 61.172% 67.940% ############ ( 2, 3 ] 57007 19.394% 87.334% #### ( 3, 4 ] 16329 5.555% 92.890% # ( 4, 6 ] 13634 4.638% 97.528% # ( 6, 10 ] 5223 1.777% 99.305% ( 10, 15 ] 685 0.233% 99.538% ( 15, 22 ] 121 0.041% 99.579% ( 22, 34 ] 27 0.009% 99.588% ( 34, 51 ] 7 0.002% 99.591% ( 51, 76 ] 42 0.014% 99.605% ( 76, 110 ] 218 0.074% 99.679% ( 110, 170 ] 407 0.138% 99.818% ( 170, 250 ] 203 0.069% 99.887% ( 250, 380 ] 132 0.045% 99.932% ( 380, 580 ] 128 0.044% 99.975% ( 580, 870 ] 64 0.022% 99.997% ( 870, 1300 ] 16 0.005% 100.002% ( 1300, 1900 ] 10 0.003% 100.006% ( 1900, 2900 ] 8 0.003% 100.009% ( 2900, 4400 ] 9 0.003% 100.012% ( 4400, 6600 ] 3 0.001% 100.013% ** Level 4 read latency histogram (micros): Count: 130 Average: 312.7615 StdDev: 851.00 Min: 1 Median: 6.0000 Max: 5123 Percentiles: P50: 6.00 P75: 116.00 P99: 4250.00 P99.9: 5123.00 P99.99: 5123.00 ------------------------------------------------------ [ 0, 1 ] 38 29.231% 29.231% ###### ( 1, 2 ] 24 18.462% 47.692% #### ( 2, 3 ] 1 0.769% 48.462% ( 3, 4 ] 1 0.769% 49.231% ( 4, 6 ] 1 0.769% 50.000% ( 6, 10 ] 15 11.538% 61.538% ## ( 15, 22 ] 1 0.769% 62.308% ( 22, 34 ] 12 9.231% 71.538% ## ( 34, 51 ] 3 2.308% 73.846% ( 76, 110 ] 1 0.769% 74.615% ( 110, 170 ] 5 3.846% 78.462% # ( 170, 250 ] 1 0.769% 79.231% ( 250, 380 ] 2 1.538% 80.769% ( 380, 580 ] 8 6.154% 86.923% # ( 580, 870 ] 4 3.077% 90.000% # ( 870, 1300 ] 4 3.077% 93.077% # ( 1300, 1900 ] 2 1.538% 94.615% ( 1900, 2900 ] 3 2.308% 96.923% ( 2900, 4400 ] 3 2.308% 99.231% ( 4400, 6600 ] 1 0.769% 100.000% ** Level 5 read latency histogram (micros): Count: 80 Average: 750.4500 StdDev: 920.60 Min: 89 Median: 250.0000 Max: 3822 Percentiles: P50: 250.00 P75: 1085.00 P99: 3822.00 P99.9: 3822.00 P99.99: 3822.00 ------------------------------------------------------ ( 76, 110 ] 6 7.500% 7.500% ## ( 110, 170 ] 21 26.250% 33.750% ##### ( 170, 250 ] 13 16.250% 50.000% ### ( 250, 380 ] 6 7.500% 57.500% ## ( 380, 580 ] 4 5.000% 62.500% # ( 580, 870 ] 7 8.750% 71.250% ## ( 870, 1300 ] 6 7.500% 78.750% ## ( 1300, 1900 ] 6 7.500% 86.250% ## ( 1900, 2900 ] 6 7.500% 93.750% ## ( 2900, 4400 ] 5 6.250% 100.000% # ** DB Stats ** Uptime(secs): 24.0 total, 6.5 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 245.74 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.547 H:M:S, 31.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 152.39 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.645 H:M:S, 56.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/17-18:21:24 ... thread 0: (1000000,7000000) ops and (310974.5,258859.2) ops/second in (3.215698,27.041728) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/11 3.56 GB 132.2 5.8 0.0 5.8 12.5 6.6 0.0 1.9 141.9 303.1 42.16 39.38 116 0.363 6075K 9777 0.0 0.0 L1 16/16 1.47 GB 0.0 3.3 3.0 0.4 3.3 2.9 0.1 1.1 224.2 222.1 15.10 14.36 3 5.034 3438K 31K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 1.5 1.8 247.7 247.7 0.45 0.42 1 0.450 113K 40 0.0 0.0 L3 14/0 926.48 MB 0.9 1.5 0.6 0.9 1.5 0.6 1.0 2.5 211.5 211.3 7.50 6.64 4 1.874 1591K 801 0.0 0.0 L4 172/0 9.75 GB 1.0 0.3 0.3 0.0 0.3 0.3 1.3 1.1 195.8 195.3 1.66 1.46 2 0.832 300K 863 0.0 0.0 L5 1056/0 87.05 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1277/27 102.74 GB 0.0 11.1 3.9 7.2 17.7 10.5 5.3 2.7 170.4 271.5 66.87 62.26 126 0.531 11M 43K 0.0 0.0 Int 0/0 0.00 KB 0.0 11.1 3.9 7.2 17.7 10.5 5.3 2.7 170.4 271.5 66.87 62.26 126 0.531 11M 43K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.1 3.9 7.2 11.1 3.9 0.0 0.0 239.4 238.5 47.59 45.13 29 1.641 11M 43K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 352.8 19.28 17.13 97 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 27.2 total, 27.0 interval Flush(GB): cumulative 6.644, interval 6.644 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 17.73 GB write, 666.24 MB/s write, 11.13 GB read, 418.14 MB/s read, 66.9 seconds Interval compaction: 17.73 GB write, 671.23 MB/s write, 11.13 GB read, 421.27 MB/s read, 66.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 11, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 11, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 23, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 11, total-stops: 23, interval: 34 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2605530 Average: 2.4263 StdDev: 12.71 Min: 0 Median: 1.5114 Max: 4601 Percentiles: P50: 1.51 P75: 1.89 P99: 5.88 P99.9: 90.70 P99.99: 185.20 ------------------------------------------------------ [ 0, 1 ] 418170 16.049% 16.049% ### ( 1, 2 ] 1729745 66.387% 82.437% ############# ( 2, 3 ] 294244 11.293% 93.730% ## ( 3, 4 ] 85006 3.263% 96.992% # ( 4, 6 ] 55783 2.141% 99.133% ( 6, 10 ] 16845 0.647% 99.780% ( 10, 15 ] 2267 0.087% 99.867% ( 15, 22 ] 238 0.009% 99.876% ( 22, 34 ] 27 0.001% 99.877% ( 34, 51 ] 16 0.001% 99.878% ( 51, 76 ] 42 0.002% 99.879% ( 76, 110 ] 1257 0.048% 99.927% ( 110, 170 ] 1553 0.060% 99.987% ( 170, 250 ] 413 0.016% 100.003% ( 250, 380 ] 250 0.010% 100.013% ( 380, 580 ] 182 0.007% 100.019% ( 580, 870 ] 117 0.004% 100.024% ( 870, 1300 ] 25 0.001% 100.025% ( 1300, 1900 ] 21 0.001% 100.026% ( 1900, 2900 ] 8 0.000% 100.026% ( 2900, 4400 ] 9 0.000% 100.026% ( 4400, 6600 ] 1 0.000% 100.026% ** Level 1 read latency histogram (micros): Count: 263596 Average: 2.7824 StdDev: 18.33 Min: 1 Median: 1.6462 Max: 3372 Percentiles: P50: 1.65 P75: 2.15 P99: 6.28 P99.9: 108.84 P99.99: 809.80 ------------------------------------------------------ [ 0, 1 ] 24994 9.482% 9.482% ## ( 1, 2 ] 165286 62.704% 72.186% ############# ( 2, 3 ] 47872 18.161% 90.347% #### ( 3, 4 ] 11964 4.539% 94.886% # ( 4, 6 ] 10709 4.063% 98.949% # ( 6, 10 ] 1917 0.727% 99.676% ( 10, 15 ] 385 0.146% 99.822% ( 15, 22 ] 38 0.014% 99.836% ( 22, 34 ] 2 0.001% 99.837% ( 34, 51 ] 1 0.000% 99.838% ( 51, 76 ] 7 0.003% 99.840% ( 76, 110 ] 164 0.062% 99.903% ( 110, 170 ] 128 0.049% 99.951% ( 170, 250 ] 36 0.014% 99.965% ( 250, 380 ] 29 0.011% 99.976% ( 380, 580 ] 22 0.008% 99.984% ( 580, 870 ] 21 0.008% 99.992% ( 870, 1300 ] 8 0.003% 99.995% ( 1300, 1900 ] 10 0.004% 99.999% ( 1900, 2900 ] 2 0.001% 100.000% ( 2900, 4400 ] 2 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 40582 Average: 2.4031 StdDev: 18.82 Min: 1 Median: 1.3320 Max: 2356 Percentiles: P50: 1.33 P75: 1.76 P99: 7.08 P99.9: 112.28 P99.99: 808.62 ------------------------------------------------------ [ 0, 1 ] 12396 30.546% 30.546% ###### ( 1, 2 ] 23781 58.600% 89.145% ############ ( 2, 3 ] 2651 6.532% 95.678% # ( 3, 4 ] 706 1.740% 97.418% ( 4, 6 ] 546 1.345% 98.763% ( 6, 10 ] 355 0.875% 99.638% ( 10, 15 ] 37 0.091% 99.729% ( 15, 22 ] 6 0.015% 99.744% ( 22, 34 ] 2 0.005% 99.749% ( 34, 51 ] 40 0.099% 99.847% ( 51, 76 ] 11 0.027% 99.874% ( 76, 110 ] 10 0.025% 99.899% ( 110, 170 ] 11 0.027% 99.926% ( 170, 250 ] 9 0.022% 99.948% ( 250, 380 ] 6 0.015% 99.963% ( 380, 580 ] 7 0.017% 99.980% ( 580, 870 ] 5 0.012% 99.993% ( 870, 1300 ] 1 0.002% 99.995% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 293934 Average: 3.7619 StdDev: 36.01 Min: 1 Median: 1.7067 Max: 5439 Percentiles: P50: 1.71 P75: 2.36 P99: 9.31 P99.9: 288.47 P99.99: 777.59 ------------------------------------------------------ [ 0, 1 ] 19894 6.768% 6.768% # ( 1, 2 ] 179804 61.172% 67.940% ############ ( 2, 3 ] 57007 19.394% 87.334% #### ( 3, 4 ] 16329 5.555% 92.890% # ( 4, 6 ] 13634 4.638% 97.528% # ( 6, 10 ] 5223 1.777% 99.305% ( 10, 15 ] 685 0.233% 99.538% ( 15, 22 ] 121 0.041% 99.579% ( 22, 34 ] 27 0.009% 99.588% ( 34, 51 ] 7 0.002% 99.591% ( 51, 76 ] 42 0.014% 99.605% ( 76, 110 ] 218 0.074% 99.679% ( 110, 170 ] 407 0.138% 99.818% ( 170, 250 ] 203 0.069% 99.887% ( 250, 380 ] 132 0.045% 99.932% ( 380, 580 ] 128 0.044% 99.975% ( 580, 870 ] 64 0.022% 99.997% ( 870, 1300 ] 16 0.005% 100.002% ( 1300, 1900 ] 10 0.003% 100.006% ( 1900, 2900 ] 8 0.003% 100.009% ( 2900, 4400 ] 9 0.003% 100.012% ( 4400, 6600 ] 3 0.001% 100.013% ** Level 4 read latency histogram (micros): Count: 130 Average: 312.7615 StdDev: 851.00 Min: 1 Median: 6.0000 Max: 5123 Percentiles: P50: 6.00 P75: 116.00 P99: 4250.00 P99.9: 5123.00 P99.99: 5123.00 ------------------------------------------------------ [ 0, 1 ] 38 29.231% 29.231% ###### ( 1, 2 ] 24 18.462% 47.692% #### ( 2, 3 ] 1 0.769% 48.462% ( 3, 4 ] 1 0.769% 49.231% ( 4, 6 ] 1 0.769% 50.000% ( 6, 10 ] 15 11.538% 61.538% ## ( 15, 22 ] 1 0.769% 62.308% ( 22, 34 ] 12 9.231% 71.538% ## ( 34, 51 ] 3 2.308% 73.846% ( 76, 110 ] 1 0.769% 74.615% ( 110, 170 ] 5 3.846% 78.462% # ( 170, 250 ] 1 0.769% 79.231% ( 250, 380 ] 2 1.538% 80.769% ( 380, 580 ] 8 6.154% 86.923% # ( 580, 870 ] 4 3.077% 90.000% # ( 870, 1300 ] 4 3.077% 93.077% # ( 1300, 1900 ] 2 1.538% 94.615% ( 1900, 2900 ] 3 2.308% 96.923% ( 2900, 4400 ] 3 2.308% 99.231% ( 4400, 6600 ] 1 0.769% 100.000% ** Level 5 read latency histogram (micros): Count: 80 Average: 750.4500 StdDev: 920.60 Min: 89 Median: 250.0000 Max: 3822 Percentiles: P50: 250.00 P75: 1085.00 P99: 3822.00 P99.9: 3822.00 P99.99: 3822.00 ------------------------------------------------------ ( 76, 110 ] 6 7.500% 7.500% ## ( 110, 170 ] 21 26.250% 33.750% ##### ( 170, 250 ] 13 16.250% 50.000% ### ( 250, 380 ] 6 7.500% 57.500% ## ( 380, 580 ] 4 5.000% 62.500% # ( 580, 870 ] 7 8.750% 71.250% ## ( 870, 1300 ] 6 7.500% 78.750% ## ( 1300, 1900 ] 6 7.500% 86.250% ## ( 1900, 2900 ] 6 7.500% 93.750% ## ( 2900, 4400 ] 5 6.250% 100.000% # ** DB Stats ** Uptime(secs): 27.2 total, 3.2 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 252.86 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:8.118 H:M:S, 29.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 306.06 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.571 H:M:S, 17.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/10/17-18:21:38 ... thread 0: (1000000,8000000) ops and (71822.0,195288.5) ops/second in (13.923302,40.965030) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 6/6 2.51 GB 0.0 8.5 0.0 8.5 16.1 7.6 0.0 2.1 154.9 293.8 56.10 52.30 137 0.409 8825K 19K 0.0 0.0 L1 19/19 2.12 GB 0.0 6.8 5.0 1.8 6.7 4.9 0.1 1.4 219.8 217.3 31.69 29.71 4 7.924 6886K 31K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 2.8 1.8 247.7 247.7 0.45 0.42 1 0.450 113K 40 0.0 0.0 L3 39/9 2.22 GB 1.6 3.2 1.3 1.8 3.1 1.3 1.6 2.3 174.3 173.3 18.54 14.24 11 1.685 3263K 19K 0.0 0.0 L4 172/9 9.75 GB 1.0 0.3 0.3 0.0 0.3 0.3 1.3 1.1 195.8 195.3 1.66 1.46 2 0.832 300K 863 0.0 0.0 L5 1056/0 87.05 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1292/43 103.65 GB 0.0 18.9 6.7 12.2 26.4 14.2 7.2 3.5 178.2 249.1 108.44 98.13 155 0.700 19M 71K 0.0 0.0 Int 0/0 0.00 KB 0.0 18.9 6.7 12.2 26.4 14.2 7.2 3.5 178.2 249.1 108.44 98.13 155 0.700 19M 71K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 18.9 6.7 12.2 18.8 6.5 0.0 0.0 224.1 222.7 86.25 78.42 43 2.006 19M 71K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 352.0 22.19 19.72 112 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 41.2 total, 41.0 interval Flush(GB): cumulative 7.629, interval 7.629 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 26.38 GB write, 656.28 MB/s write, 18.87 GB read, 469.43 MB/s read, 108.4 seconds Interval compaction: 26.38 GB write, 659.52 MB/s write, 18.87 GB read, 471.75 MB/s read, 108.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 22, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 22, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 24, pending-compaction-bytes-delays: 11, pending-compaction-bytes-stops: 0, total-delays: 33, total-stops: 24, interval: 57 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3599906 Average: 2.8032 StdDev: 18.94 Min: 0 Median: 1.5203 Max: 7063 Percentiles: P50: 1.52 P75: 1.90 P99: 6.94 P99.9: 152.11 P99.99: 360.79 ------------------------------------------------------ [ 0, 1 ] 564034 15.668% 15.668% ### ( 1, 2 ] 2375209 65.980% 81.648% ############# ( 2, 3 ] 412720 11.465% 93.113% ## ( 3, 4 ] 122222 3.395% 96.508% # ( 4, 6 ] 83308 2.314% 98.822% ( 6, 10 ] 27151 0.754% 99.576% ( 10, 15 ] 3373 0.094% 99.670% ( 15, 22 ] 446 0.012% 99.682% ( 22, 34 ] 38 0.001% 99.683% ( 34, 51 ] 28 0.001% 99.684% ( 51, 76 ] 85 0.002% 99.686% ( 76, 110 ] 4408 0.122% 99.809% ( 110, 170 ] 4681 0.130% 99.939% ( 170, 250 ] 1175 0.033% 99.971% ( 250, 380 ] 785 0.022% 99.993% ( 380, 580 ] 606 0.017% 100.010% ( 580, 870 ] 301 0.008% 100.018% ( 870, 1300 ] 79 0.002% 100.021% ( 1300, 1900 ] 51 0.001% 100.022% ( 1900, 2900 ] 25 0.001% 100.023% ( 2900, 4400 ] 22 0.001% 100.023% ( 4400, 6600 ] 5 0.000% 100.024% ( 6600, 9900 ] 1 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 1037445 Average: 3.5141 StdDev: 29.41 Min: 1 Median: 1.7226 Max: 5305 Percentiles: P50: 1.72 P75: 2.50 P99: 8.85 P99.9: 96.89 P99.99: 146.68 ------------------------------------------------------ [ 0, 1 ] 87755 8.459% 8.459% ## ( 1, 2 ] 596453 57.492% 65.951% ########### ( 2, 3 ] 186801 18.006% 83.957% #### ( 3, 4 ] 71275 6.870% 90.827% # ( 4, 6 ] 70731 6.818% 97.645% # ( 6, 10 ] 19727 1.901% 99.547% ( 10, 15 ] 2622 0.253% 99.799% ( 15, 22 ] 391 0.038% 99.837% ( 22, 34 ] 24 0.002% 99.839% ( 34, 51 ] 7 0.001% 99.840% ( 51, 76 ] 98 0.009% 99.850% ( 76, 110 ] 852 0.082% 99.932% ( 110, 170 ] 990 0.095% 100.027% ( 170, 250 ] 382 0.037% 100.064% ( 250, 380 ] 379 0.037% 100.100% ( 380, 580 ] 310 0.030% 100.130% ( 580, 870 ] 155 0.015% 100.145% ( 870, 1300 ] 99 0.010% 100.155% ( 1300, 1900 ] 42 0.004% 100.159% ( 1900, 2900 ] 17 0.002% 100.160% ( 2900, 4400 ] 15 0.001% 100.162% ( 4400, 6600 ] 7 0.001% 100.163% ** Level 2 read latency histogram (micros): Count: 40582 Average: 2.4031 StdDev: 18.82 Min: 1 Median: 1.3320 Max: 2356 Percentiles: P50: 1.33 P75: 1.76 P99: 7.08 P99.9: 112.28 P99.99: 808.62 ------------------------------------------------------ [ 0, 1 ] 12396 30.546% 30.546% ###### ( 1, 2 ] 23781 58.600% 89.145% ############ ( 2, 3 ] 2651 6.532% 95.678% # ( 3, 4 ] 706 1.740% 97.418% ( 4, 6 ] 546 1.345% 98.763% ( 6, 10 ] 355 0.875% 99.638% ( 10, 15 ] 37 0.091% 99.729% ( 15, 22 ] 6 0.015% 99.744% ( 22, 34 ] 2 0.005% 99.749% ( 34, 51 ] 40 0.099% 99.847% ( 51, 76 ] 11 0.027% 99.874% ( 76, 110 ] 10 0.025% 99.899% ( 110, 170 ] 11 0.027% 99.926% ( 170, 250 ] 9 0.022% 99.948% ( 250, 380 ] 6 0.015% 99.963% ( 380, 580 ] 7 0.017% 99.980% ( 580, 870 ] 5 0.012% 99.993% ( 870, 1300 ] 1 0.002% 99.995% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 339398 Average: 4.6158 StdDev: 48.33 Min: 0 Median: 1.7486 Max: 8016 Percentiles: P50: 1.75 P75: 2.50 P99: 9.56 P99.9: 356.05 P99.99: 603.16 ------------------------------------------------------ [ 0, 1 ] 20307 5.983% 5.983% # ( 1, 2 ] 199558 58.798% 64.781% ############ ( 2, 3 ] 69069 20.350% 85.131% #### ( 3, 4 ] 22248 6.555% 91.686% # ( 4, 6 ] 19340 5.698% 97.385% # ( 6, 10 ] 6161 1.815% 99.200% ( 10, 15 ] 879 0.259% 99.459% ( 15, 22 ] 144 0.042% 99.501% ( 22, 34 ] 30 0.009% 99.510% ( 34, 51 ] 8 0.002% 99.513% ( 51, 76 ] 42 0.012% 99.525% ( 76, 110 ] 238 0.070% 99.595% ( 110, 170 ] 491 0.145% 99.740% ( 170, 250 ] 319 0.094% 99.834% ( 250, 380 ] 279 0.082% 99.916% ( 380, 580 ] 242 0.071% 99.987% ( 580, 870 ] 151 0.044% 100.032% ( 870, 1300 ] 46 0.014% 100.045% ( 1300, 1900 ] 32 0.009% 100.055% ( 1900, 2900 ] 17 0.005% 100.060% ( 2900, 4400 ] 16 0.005% 100.065% ( 4400, 6600 ] 4 0.001% 100.066% ( 6600, 9900 ] 1 0.000% 100.066% ** Level 4 read latency histogram (micros): Count: 130 Average: 312.7615 StdDev: 851.00 Min: 1 Median: 6.0000 Max: 5123 Percentiles: P50: 6.00 P75: 116.00 P99: 4250.00 P99.9: 5123.00 P99.99: 5123.00 ------------------------------------------------------ [ 0, 1 ] 38 29.231% 29.231% ###### ( 1, 2 ] 24 18.462% 47.692% #### ( 2, 3 ] 1 0.769% 48.462% ( 3, 4 ] 1 0.769% 49.231% ( 4, 6 ] 1 0.769% 50.000% ( 6, 10 ] 15 11.538% 61.538% ## ( 15, 22 ] 1 0.769% 62.308% ( 22, 34 ] 12 9.231% 71.538% ## ( 34, 51 ] 3 2.308% 73.846% ( 76, 110 ] 1 0.769% 74.615% ( 110, 170 ] 5 3.846% 78.462% # ( 170, 250 ] 1 0.769% 79.231% ( 250, 380 ] 2 1.538% 80.769% ( 380, 580 ] 8 6.154% 86.923% # ( 580, 870 ] 4 3.077% 90.000% # ( 870, 1300 ] 4 3.077% 93.077% # ( 1300, 1900 ] 2 1.538% 94.615% ( 1900, 2900 ] 3 2.308% 96.923% ( 2900, 4400 ] 3 2.308% 99.231% ( 4400, 6600 ] 1 0.769% 100.000% ** Level 5 read latency histogram (micros): Count: 80 Average: 750.4500 StdDev: 920.60 Min: 89 Median: 250.0000 Max: 3822 Percentiles: P50: 250.00 P75: 1085.00 P99: 3822.00 P99.9: 3822.00 P99.99: 3822.00 ------------------------------------------------------ ( 76, 110 ] 6 7.500% 7.500% ## ( 110, 170 ] 21 26.250% 33.750% ##### ( 170, 250 ] 13 16.250% 50.000% ### ( 250, 380 ] 6 7.500% 57.500% ## ( 380, 580 ] 4 5.000% 62.500% # ( 580, 870 ] 7 8.750% 71.250% ## ( 870, 1300 ] 6 7.500% 78.750% ## ( 1300, 1900 ] 6 7.500% 86.250% ## ( 1900, 2900 ] 6 7.500% 93.750% ## ( 2900, 4400 ] 5 6.250% 100.000% # ** DB Stats ** Uptime(secs): 41.2 total, 13.9 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 191.25 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:19.020 H:M:S, 46.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 70.69 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:10.902 H:M:S, 78.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/10/17-18:21:42 ... thread 0: (1000000,9000000) ops and (280552.0,202113.5) ops/second in (3.564401,44.529431) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/14 3.37 GB 31.5 8.8 0.0 8.8 17.3 8.5 0.0 2.0 149.1 292.7 60.40 55.75 150 0.403 9144K 19K 0.0 0.0 L1 19/19 2.12 GB 0.0 6.8 5.0 1.8 6.7 4.9 0.1 1.4 219.8 217.3 31.69 29.71 4 7.924 6886K 31K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.1 0.1 0.0 0.1 0.1 2.8 1.8 247.7 247.7 0.45 0.42 1 0.450 113K 40 0.0 0.0 L3 18/0 943.54 MB 0.9 3.2 1.3 1.8 3.1 1.3 1.6 2.3 174.3 173.3 18.54 14.24 11 1.685 3263K 19K 0.0 0.0 L4 176/0 9.75 GB 1.0 1.8 1.6 0.2 1.8 1.6 1.3 1.1 139.1 138.4 13.25 8.50 21 0.631 1802K 7693 0.0 0.0 L5 1072/0 88.34 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1300/33 104.50 GB 0.0 20.7 8.0 12.7 29.0 16.3 8.5 3.4 170.2 239.1 124.34 108.62 187 0.665 21M 79K 0.0 0.0 Int 0/0 0.00 KB 0.0 20.7 8.0 12.7 29.0 16.3 8.5 3.4 170.2 239.1 124.34 108.62 187 0.665 21M 79K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 20.7 8.0 12.7 20.5 7.8 0.0 0.0 213.7 212.4 99.01 86.61 63 1.572 21M 79K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.5 8.5 0.0 0.0 0.0 343.2 25.33 22.02 124 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 44.7 total, 44.5 interval Flush(GB): cumulative 8.490, interval 8.490 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 29.03 GB write, 664.46 MB/s write, 20.66 GB read, 472.98 MB/s read, 124.3 seconds Interval compaction: 29.03 GB write, 667.49 MB/s write, 20.66 GB read, 475.13 MB/s read, 124.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 22, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 22, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 39, pending-compaction-bytes-delays: 11, pending-compaction-bytes-stops: 0, total-delays: 33, total-stops: 39, interval: 72 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3944505 Average: 2.8107 StdDev: 22.20 Min: 0 Median: 1.5296 Max: 12066 Percentiles: P50: 1.53 P75: 1.91 P99: 6.98 P99.9: 148.57 P99.99: 346.67 ------------------------------------------------------ [ 0, 1 ] 597929 15.159% 15.159% ### ( 1, 2 ] 2594838 65.784% 80.942% ############# ( 2, 3 ] 469509 11.903% 92.845% ## ( 3, 4 ] 138250 3.505% 96.350% # ( 4, 6 ] 96874 2.456% 98.806% ( 6, 10 ] 31187 0.791% 99.596% ( 10, 15 ] 3772 0.096% 99.692% ( 15, 22 ] 472 0.012% 99.704% ( 22, 34 ] 43 0.001% 99.705% ( 34, 51 ] 29 0.001% 99.706% ( 51, 76 ] 87 0.002% 99.708% ( 76, 110 ] 4500 0.114% 99.822% ( 110, 170 ] 4780 0.121% 99.943% ( 170, 250 ] 1235 0.031% 99.975% ( 250, 380 ] 817 0.021% 99.995% ( 380, 580 ] 630 0.016% 100.011% ( 580, 870 ] 309 0.008% 100.019% ( 870, 1300 ] 89 0.002% 100.021% ( 1300, 1900 ] 53 0.001% 100.023% ( 1900, 2900 ] 26 0.001% 100.023% ( 2900, 4400 ] 36 0.001% 100.024% ( 4400, 6600 ] 8 0.000% 100.025% ( 6600, 9900 ] 4 0.000% 100.025% ( 9900, 14000 ] 1 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 1070149 Average: 3.6634 StdDev: 43.25 Min: 1 Median: 1.7269 Max: 18445 Percentiles: P50: 1.73 P75: 2.51 P99: 8.87 P99.9: 99.02 P99.99: 151.57 ------------------------------------------------------ [ 0, 1 ] 88466 8.267% 8.267% ## ( 1, 2 ] 614363 57.409% 65.676% ########### ( 2, 3 ] 194734 18.197% 83.873% #### ( 3, 4 ] 73844 6.900% 90.773% # ( 4, 6 ] 73328 6.852% 97.625% # ( 6, 10 ] 20506 1.916% 99.541% ( 10, 15 ] 2726 0.255% 99.796% ( 15, 22 ] 405 0.038% 99.834% ( 22, 34 ] 25 0.002% 99.836% ( 34, 51 ] 7 0.001% 99.837% ( 51, 76 ] 98 0.009% 99.846% ( 76, 110 ] 852 0.080% 99.926% ( 110, 170 ] 993 0.093% 100.019% ( 170, 250 ] 387 0.036% 100.055% ( 250, 380 ] 387 0.036% 100.091% ( 380, 580 ] 319 0.030% 100.121% ( 580, 870 ] 167 0.016% 100.136% ( 870, 1300 ] 112 0.010% 100.147% ( 1300, 1900 ] 54 0.005% 100.152% ( 1900, 2900 ] 22 0.002% 100.154% ( 2900, 4400 ] 21 0.002% 100.156% ( 4400, 6600 ] 14 0.001% 100.157% ( 6600, 9900 ] 3 0.000% 100.157% ( 9900, 14000 ] 2 0.000% 100.158% ( 14000, 22000 ] 1 0.000% 100.158% ** Level 2 read latency histogram (micros): Count: 40582 Average: 2.4031 StdDev: 18.82 Min: 1 Median: 1.3320 Max: 2356 Percentiles: P50: 1.33 P75: 1.76 P99: 7.08 P99.9: 112.28 P99.99: 808.62 ------------------------------------------------------ [ 0, 1 ] 12396 30.546% 30.546% ###### ( 1, 2 ] 23781 58.600% 89.145% ############ ( 2, 3 ] 2651 6.532% 95.678% # ( 3, 4 ] 706 1.740% 97.418% ( 4, 6 ] 546 1.345% 98.763% ( 6, 10 ] 355 0.875% 99.638% ( 10, 15 ] 37 0.091% 99.729% ( 15, 22 ] 6 0.015% 99.744% ( 22, 34 ] 2 0.005% 99.749% ( 34, 51 ] 40 0.099% 99.847% ( 51, 76 ] 11 0.027% 99.874% ( 76, 110 ] 10 0.025% 99.899% ( 110, 170 ] 11 0.027% 99.926% ( 170, 250 ] 9 0.022% 99.948% ( 250, 380 ] 6 0.015% 99.963% ( 380, 580 ] 7 0.017% 99.980% ( 580, 870 ] 5 0.012% 99.993% ( 870, 1300 ] 1 0.002% 99.995% ( 1300, 1900 ] 1 0.002% 99.998% ( 1900, 2900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 644266 Average: 5.0785 StdDev: 68.53 Min: 0 Median: 1.8537 Max: 11576 Percentiles: P50: 1.85 P75: 2.75 P99: 8.91 P99.9: 184.12 P99.99: 328.38 ------------------------------------------------------ [ 0, 1 ] 24109 3.742% 3.742% # ( 1, 2 ] 349077 54.182% 57.924% ########### ( 2, 3 ] 146919 22.804% 80.728% ##### ( 3, 4 ] 57796 8.971% 89.699% ## ( 4, 6 ] 52180 8.099% 97.798% ## ( 6, 10 ] 10652 1.653% 99.452% ( 10, 15 ] 1680 0.261% 99.712% ( 15, 22 ] 244 0.038% 99.750% ( 22, 34 ] 36 0.006% 99.756% ( 34, 51 ] 9 0.001% 99.757% ( 51, 76 ] 44 0.007% 99.764% ( 76, 110 ] 261 0.041% 99.805% ( 110, 170 ] 542 0.084% 99.889% ( 170, 250 ] 412 0.064% 99.953% ( 250, 380 ] 399 0.062% 100.015% ( 380, 580 ] 367 0.057% 100.072% ( 580, 870 ] 271 0.042% 100.114% ( 870, 1300 ] 145 0.023% 100.136% ( 1300, 1900 ] 60 0.009% 100.145% ( 1900, 2900 ] 49 0.008% 100.153% ( 2900, 4400 ] 90 0.014% 100.167% ( 4400, 6600 ] 16 0.002% 100.169% ( 6600, 9900 ] 4 0.001% 100.170% ( 9900, 14000 ] 2 0.000% 100.170% ** Level 4 read latency histogram (micros): Count: 325 Average: 215.3631 StdDev: 607.73 Min: 0 Median: 2.8542 Max: 5123 Percentiles: P50: 2.85 P75: 72.53 P99: 3275.00 P99.9: 5123.00 P99.99: 5123.00 ------------------------------------------------------ [ 0, 1 ] 60 18.462% 18.462% #### ( 1, 2 ] 82 25.231% 43.692% ##### ( 2, 3 ] 24 7.385% 51.077% # ( 3, 4 ] 8 2.462% 53.538% ( 4, 6 ] 8 2.462% 56.000% ( 6, 10 ] 22 6.769% 62.769% # ( 10, 15 ] 4 1.231% 64.000% ( 15, 22 ] 3 0.923% 64.923% ( 22, 34 ] 20 6.154% 71.077% # ( 34, 51 ] 5 1.538% 72.615% ( 51, 76 ] 9 2.769% 75.385% # ( 76, 110 ] 2 0.615% 76.000% ( 110, 170 ] 5 1.538% 77.538% ( 170, 250 ] 17 5.231% 82.769% # ( 250, 380 ] 8 2.462% 85.231% ( 380, 580 ] 9 2.769% 88.000% # ( 580, 870 ] 13 4.000% 92.000% # ( 870, 1300 ] 13 4.000% 96.000% # ( 1300, 1900 ] 5 1.538% 97.538% ( 1900, 2900 ] 4 1.231% 98.769% ( 2900, 4400 ] 3 0.923% 99.692% ( 4400, 6600 ] 1 0.308% 100.000% ** Level 5 read latency histogram (micros): Count: 160 Average: 1076.0000 StdDev: 1153.06 Min: 89 Median: 656.3158 Max: 5723 Percentiles: P50: 656.32 P75: 1637.50 P99: 4840.00 P99.9: 5723.00 P99.99: 5723.00 ------------------------------------------------------ ( 76, 110 ] 8 5.000% 5.000% # ( 110, 170 ] 22 13.750% 18.750% ### ( 170, 250 ] 19 11.875% 30.625% ## ( 250, 380 ] 16 10.000% 40.625% ## ( 380, 580 ] 10 6.250% 46.875% # ( 580, 870 ] 19 11.875% 58.750% ## ( 870, 1300 ] 17 10.625% 69.375% ## ( 1300, 1900 ] 16 10.000% 79.375% ## ( 1900, 2900 ] 17 10.625% 90.000% ## ( 2900, 4400 ] 14 8.750% 98.750% ## ( 4400, 6600 ] 2 1.250% 100.000% ** DB Stats ** Uptime(secs): 44.7 total, 3.6 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 198.01 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:19.796 H:M:S, 44.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 276.12 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.776 H:M:S, 21.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 6.404 micros/op 156150 ops/sec 60.051 seconds 9376999 operations; 151.3 MB/s Microseconds per write: Count: 9376999 Average: 6.4037 StdDev: 288.16 Min: 1 Median: 1.7526 Max: 515190 Percentiles: P50: 1.75 P75: 2.81 P99: 10.16 P99.9: 1195.32 P99.99: 2709.04 ------------------------------------------------------ [ 0, 1 ] 1176142 12.543% 12.543% ### ( 1, 2 ] 4667204 49.773% 62.316% ########## ( 2, 3 ] 1466493 15.639% 77.955% ### ( 3, 4 ] 1303526 13.901% 91.856% ### ( 4, 6 ] 531481 5.668% 97.524% # ( 6, 10 ] 136842 1.459% 98.984% ( 10, 15 ] 49265 0.525% 99.509% ( 15, 22 ] 10168 0.108% 99.617% ( 22, 34 ] 1838 0.020% 99.637% ( 34, 51 ] 199 0.002% 99.639% ( 51, 76 ] 42 0.000% 99.640% ( 76, 110 ] 6360 0.068% 99.707% ( 110, 170 ] 2704 0.029% 99.736% ( 170, 250 ] 251 0.003% 99.739% ( 250, 380 ] 218 0.002% 99.741% ( 380, 580 ] 10 0.000% 99.741% ( 580, 870 ] 17 0.000% 99.742% ( 870, 1300 ] 19644 0.209% 99.951% ( 1300, 1900 ] 32 0.000% 99.951% ( 1900, 2900 ] 4481 0.048% 99.999% ( 2900, 4400 ] 52 0.001% 100.000% ( 4400, 6600 ] 6 0.000% 100.000% ( 6600, 9900 ] 4 0.000% 100.000% ( 14000, 22000 ] 2 0.000% 100.000% ( 22000, 33000 ] 3 0.000% 100.000% ( 33000, 50000 ] 1 0.000% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% ( 170000, 250000 ] 12 0.000% 100.000% ( 380000, 570000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 203917 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 9376999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 9677062968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 444 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 33640438 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 27790772142 rocksdb.compact.write.bytes COUNT : 27603937111 rocksdb.flush.write.bytes COUNT : 9644401316 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 41 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 197 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 9566621696 rocksdb.memtable.garbage.bytes.at.flush COUNT : 3707904 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 27873315659 rocksdb.non.last.level.read.count COUNT : 6737852 rocksdb.block.checksum.compute.count COUNT : 6737005 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.411737 P95 : 3.984607 P99 : 8.850657 P100 : 515186.000000 COUNT : 9376999 SUM : 55563935 rocksdb.compaction.times.micros P50 : 990769.230769 P95 : 4090000.000000 P99 : 16593211.000000 P100 : 16593211.000000 COUNT : 66 SUM : 102193585 rocksdb.compaction.times.cpu_micros P50 : 950000.000000 P95 : 4090000.000000 P99 : 15346537.000000 P100 : 15346537.000000 COUNT : 66 SUM : 89726522 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 683.072289 P95 : 1083.045455 P99 : 1287.881818 P100 : 1680.000000 COUNT : 131 SUM : 88704 rocksdb.compaction.outfile.sync.micros P50 : 828.410853 P95 : 113562.500000 P99 : 158712.500000 P100 : 169683.000000 COUNT : 301 SUM : 4042024 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 100.920455 P95 : 485.000000 P99 : 1666.000000 P100 : 4026.000000 COUNT : 217 SUM : 40638 rocksdb.table.open.io.micros P50 : 1205.254237 P95 : 6574.117647 P99 : 11649.333333 P100 : 16160.000000 COUNT : 444 SUM : 837939 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.619287 P95 : 7.126750 P99 : 10.011513 P100 : 18451.000000 COUNT : 6735681 SUM : 35687906 rocksdb.read.block.get.micros P50 : 378.750000 P95 : 2792.682927 P99 : 4646.400000 P100 : 5799.000000 COUNT : 888 SUM : 668438 rocksdb.write.raw.block.micros P50 : 0.511825 P95 : 0.972468 P99 : 2.839287 P100 : 22301.000000 COUNT : 9021201 SUM : 16653515 rocksdb.numfiles.in.singlecompaction P50 : 3.296610 P95 : 9.900000 P99 : 25.000000 P100 : 25.000000 COUNT : 101 SUM : 416 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.562455 P95 : 1159.781325 P99 : 2423.086034 P100 : 514972.000000 COUNT : 219019 SUM : 33663172 rocksdb.sst.read.micros P50 : 1.555216 P95 : 4.090666 P99 : 8.169172 P100 : 18445.000000 COUNT : 6737933 SUM : 21616195 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 9376999 SUM : 9677062968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 212692.307692 P95 : 340566.666667 P99 : 511100.000000 P100 : 689419.000000 COUNT : 131 SUM : 26782512 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 282 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0