[2024-12-07 21:23:56.604389] Starting SPDK v25.01-pre git sha1 a2f5e1c2d / DPDK 23.11.0 initialization... [2024-12-07 21:23:56.604670] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid189070 ] Set seed to 1733603036597702 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-07 21:23:56.737794] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-07 21:23:56.761461] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-07 21:23:56.761504] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 189070' to capture a snapshot of events at runtime. [2024-12-07 21:23:56.761515] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-07 21:23:56.761525] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-07 21:23:56.761534] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid189070 for offline analysis/debug. [2024-12-07 21:23:56.762091] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sat Dec 7 21:24:01 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/07-21:24:03 ... thread 0: (1000000,1000000) ops and (349928.2,349928.2) ops/second in (2.857729,2.857729) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/9 630.02 MB 6.3 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 344.4 2.38 2.14 12 0.198 0 0 0.0 0.0 L2 3/2 237.23 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.0 1.8 246.8 246.6 1.41 1.36 2 0.707 354K 190 0.0 0.0 L3 11/0 881.89 MB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.66 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1040/0 85.57 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1180/11 96.94 GB 0.0 0.3 0.2 0.2 1.1 1.0 0.0 1.4 92.0 307.9 3.79 3.50 14 0.271 354K 190 0.0 0.0 Int 0/0 0.00 KB 0.0 0.3 0.2 0.2 1.1 1.0 0.0 1.4 92.0 307.9 3.79 3.50 14 0.271 354K 190 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 0.0 0.0 246.8 246.6 1.41 1.36 2 0.707 354K 190 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 344.4 2.38 2.14 12 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.1 total, 2.9 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.14 GB write, 378.68 MB/s write, 0.34 GB read, 113.14 MB/s read, 3.8 seconds Interval compaction: 1.14 GB write, 408.35 MB/s write, 0.34 GB read, 122.01 MB/s read, 3.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 2, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 2, interval: 2 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 124049 Average: 2.0367 StdDev: 18.10 Min: 1 Median: 1.3373 Max: 3879 Percentiles: P50: 1.34 P75: 1.76 P99: 5.25 P99.9: 9.82 P99.99: 32.79 ------------------------------------------------------ [ 0, 1 ] 37005 29.831% 29.831% ###### ( 1, 2 ] 74187 59.805% 89.636% ############ ( 2, 3 ] 7451 6.006% 95.642% # ( 3, 4 ] 3088 2.489% 98.131% ( 4, 6 ] 1725 1.391% 99.522% ( 6, 10 ] 493 0.397% 99.919% ( 10, 15 ] 72 0.058% 99.977% ( 15, 22 ] 14 0.011% 99.989% ( 22, 34 ] 4 0.003% 99.992% ( 51, 76 ] 3 0.002% 99.994% ( 76, 110 ] 1 0.001% 99.995% ( 110, 170 ] 6 0.005% 100.000% ( 170, 250 ] 4 0.003% 100.003% ( 250, 380 ] 7 0.006% 100.009% ( 380, 580 ] 1 0.001% 100.010% ( 870, 1300 ] 1 0.001% 100.010% ( 1300, 1900 ] 2 0.002% 100.012% ( 1900, 2900 ] 1 0.001% 100.013% ( 2900, 4400 ] 2 0.002% 100.015% ** Level 1 read latency histogram (micros): Count: 12317 Average: 3.0610 StdDev: 30.39 Min: 1 Median: 1.2983 Max: 2168 Percentiles: P50: 1.30 P75: 1.75 P99: 9.43 P99.9: 309.85 P99.99: 2168.00 ------------------------------------------------------ [ 0, 1 ] 4134 33.563% 33.563% ####### ( 1, 2 ] 6786 55.095% 88.658% ########### ( 2, 3 ] 871 7.072% 95.729% # ( 3, 4 ] 150 1.218% 96.947% ( 4, 6 ] 137 1.112% 98.060% ( 6, 10 ] 135 1.096% 99.156% ( 10, 15 ] 28 0.227% 99.383% ( 15, 22 ] 6 0.049% 99.432% ( 22, 34 ] 1 0.008% 99.440% ( 34, 51 ] 26 0.211% 99.651% ( 51, 76 ] 9 0.073% 99.724% ( 76, 110 ] 4 0.032% 99.756% ( 110, 170 ] 1 0.008% 99.765% ( 170, 250 ] 13 0.106% 99.870% ( 250, 380 ] 8 0.065% 99.935% ( 380, 580 ] 4 0.032% 99.968% ( 580, 870 ] 2 0.016% 99.984% ( 1900, 2900 ] 2 0.016% 100.000% ** Level 2 read latency histogram (micros): Count: 58866 Average: 1.9850 StdDev: 4.73 Min: 1 Median: 1.3700 Max: 876 Percentiles: P50: 1.37 P75: 1.78 P99: 5.46 P99.9: 11.65 P99.99: 214.54 ------------------------------------------------------ [ 0, 1 ] 16008 27.194% 27.194% ##### ( 1, 2 ] 36280 61.632% 88.825% ############ ( 2, 3 ] 3726 6.330% 95.155% # ( 3, 4 ] 1567 2.662% 97.817% # ( 4, 6 ] 954 1.621% 99.438% ( 6, 10 ] 255 0.433% 99.871% ( 10, 15 ] 58 0.099% 99.969% ( 15, 22 ] 7 0.012% 99.981% ( 22, 34 ] 2 0.003% 99.985% ( 34, 51 ] 1 0.002% 99.986% ( 110, 170 ] 3 0.005% 99.992% ( 170, 250 ] 2 0.003% 99.995% ( 250, 380 ] 4 0.007% 100.002% ( 870, 1300 ] 1 0.002% 100.003% ** Level 3 read latency histogram (micros): Count: 55 Average: 1611.6909 StdDev: 1460.30 Min: 108 Median: 853.8889 Max: 5967 Percentiles: P50: 853.89 P75: 2559.09 P99: 5967.00 P99.9: 5967.00 P99.99: 5967.00 ------------------------------------------------------ ( 76, 110 ] 1 1.818% 1.818% ( 170, 250 ] 2 3.636% 5.455% # ( 250, 380 ] 3 5.455% 10.909% # ( 380, 580 ] 13 23.636% 34.545% ##### ( 580, 870 ] 9 16.364% 50.909% ### ( 870, 1300 ] 4 7.273% 58.182% # ( 1300, 1900 ] 2 3.636% 61.818% # ( 1900, 2900 ] 11 20.000% 81.818% #### ( 2900, 4400 ] 7 12.727% 94.545% ### ( 4400, 6600 ] 3 5.455% 100.000% # ** Level 4 read latency histogram (micros): Count: 20 Average: 1597.9500 StdDev: 1227.35 Min: 213 Median: 1500.0000 Max: 4741 Percentiles: P50: 1500.00 P75: 2500.00 P99: 4741.00 P99.9: 4741.00 P99.99: 4741.00 ------------------------------------------------------ ( 170, 250 ] 2 10.000% 10.000% ## ( 250, 380 ] 2 10.000% 20.000% ## ( 380, 580 ] 2 10.000% 30.000% ## ( 580, 870 ] 1 5.000% 35.000% # ( 870, 1300 ] 2 10.000% 45.000% ## ( 1300, 1900 ] 3 15.000% 60.000% ### ( 1900, 2900 ] 5 25.000% 85.000% ##### ( 2900, 4400 ] 2 10.000% 95.000% ## ( 4400, 6600 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 3.1 total, 2.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 319.16 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.215 H:M:S, 7.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 344.07 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.215 H:M:S, 7.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/12/07-21:24:06 ... thread 0: (1000000,2000000) ops and (326352.1,337729.2) ops/second in (3.064175,5.921904) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/4 1.29 GB 100.7 0.8 0.0 0.8 2.6 1.8 0.0 1.4 98.2 317.3 8.34 7.72 29 0.287 831K 833 0.0 0.0 L1 1/0 3.03 MB 0.3 0.3 0.3 0.0 0.3 0.3 0.2 1.0 240.8 240.5 1.31 1.27 1 1.308 319K 406 0.0 0.0 L2 2/1 147.17 MB 1.0 0.3 0.2 0.2 0.3 0.2 0.4 1.8 246.8 246.6 1.41 1.36 2 0.707 354K 190 0.0 0.0 L3 20/11 1.09 GB 0.2 0.2 0.2 0.0 0.2 0.2 0.2 1.0 224.2 224.2 0.83 0.77 1 0.834 189K 0 0.0 0.0 L4 169/0 9.51 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.0 1.0 136.3 136.3 1.23 0.87 1 1.232 170K 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1246/16 97.92 GB 0.0 1.8 0.8 1.0 3.6 2.6 1.1 2.0 140.0 279.1 13.12 11.99 34 0.386 1866K 1429 0.0 0.0 Int 0/0 0.00 KB 0.0 1.8 0.8 1.0 3.6 2.6 1.1 2.0 140.0 279.1 13.12 11.99 34 0.386 1866K 1429 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.8 0.8 1.0 1.8 0.8 0.0 0.0 232.0 231.8 7.92 7.34 8 0.990 1866K 1429 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 351.1 5.20 4.65 26 0.200 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.1 total, 5.9 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.58 GB write, 595.86 MB/s write, 1.79 GB read, 298.94 MB/s read, 13.1 seconds Interval compaction: 3.58 GB write, 618.39 MB/s write, 1.79 GB read, 310.24 MB/s read, 13.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 5, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 5, interval: 5 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 380567 Average: 2.1911 StdDev: 13.65 Min: 1 Median: 1.4770 Max: 3879 Percentiles: P50: 1.48 P75: 1.86 P99: 5.56 P99.9: 9.96 P99.99: 20.42 ------------------------------------------------------ [ 0, 1 ] 72170 18.964% 18.964% #### ( 1, 2 ] 247633 65.069% 84.033% ############# ( 2, 3 ] 36890 9.693% 93.727% ## ( 3, 4 ] 14043 3.690% 97.417% # ( 4, 6 ] 7708 2.025% 99.442% ( 6, 10 ] 1761 0.463% 99.905% ( 10, 15 ] 294 0.077% 99.982% ( 15, 22 ] 40 0.011% 99.993% ( 22, 34 ] 9 0.002% 99.995% ( 34, 51 ] 5 0.001% 99.996% ( 51, 76 ] 5 0.001% 99.998% ( 76, 110 ] 1 0.000% 99.998% ( 110, 170 ] 10 0.003% 100.001% ( 170, 250 ] 12 0.003% 100.004% ( 250, 380 ] 13 0.003% 100.007% ( 380, 580 ] 3 0.001% 100.008% ( 580, 870 ] 5 0.001% 100.009% ( 870, 1300 ] 5 0.001% 100.011% ( 1300, 1900 ] 4 0.001% 100.012% ( 1900, 2900 ] 3 0.001% 100.012% ( 2900, 4400 ] 3 0.001% 100.013% ** Level 1 read latency histogram (micros): Count: 12337 Average: 3.2912 StdDev: 32.39 Min: 1 Median: 1.2987 Max: 2168 Percentiles: P50: 1.30 P75: 1.75 P99: 9.60 P99.9: 364.20 P99.99: 2168.00 ------------------------------------------------------ [ 0, 1 ] 4139 33.549% 33.549% ####### ( 1, 2 ] 6794 55.070% 88.620% ########### ( 2, 3 ] 871 7.060% 95.680% # ( 3, 4 ] 151 1.224% 96.904% ( 4, 6 ] 137 1.110% 98.014% ( 6, 10 ] 135 1.094% 99.108% ( 10, 15 ] 28 0.227% 99.335% ( 15, 22 ] 6 0.049% 99.384% ( 22, 34 ] 1 0.008% 99.392% ( 34, 51 ] 26 0.211% 99.603% ( 51, 76 ] 9 0.073% 99.676% ( 76, 110 ] 4 0.032% 99.708% ( 110, 170 ] 1 0.008% 99.716% ( 170, 250 ] 13 0.105% 99.822% ( 250, 380 ] 11 0.089% 99.911% ( 380, 580 ] 4 0.032% 99.943% ( 580, 870 ] 5 0.041% 99.984% ( 1900, 2900 ] 2 0.016% 100.000% ** Level 2 read latency histogram (micros): Count: 76086 Average: 2.0670 StdDev: 4.19 Min: 1 Median: 1.4336 Max: 876 Percentiles: P50: 1.43 P75: 1.83 P99: 5.60 P99.9: 11.49 P99.99: 157.83 ------------------------------------------------------ [ 0, 1 ] 17282 22.714% 22.714% ##### ( 1, 2 ] 47877 62.925% 85.639% ############# ( 2, 3 ] 6403 8.415% 94.054% ## ( 3, 4 ] 2580 3.391% 97.445% # ( 4, 6 ] 1479 1.944% 99.389% ( 6, 10 ] 366 0.481% 99.870% ( 10, 15 ] 77 0.101% 99.971% ( 15, 22 ] 9 0.012% 99.983% ( 22, 34 ] 2 0.003% 99.986% ( 34, 51 ] 1 0.001% 99.987% ( 110, 170 ] 3 0.004% 99.991% ( 170, 250 ] 2 0.003% 99.993% ( 250, 380 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 188798 Average: 3.4271 StdDev: 39.38 Min: 0 Median: 1.6327 Max: 5967 Percentiles: P50: 1.63 P75: 2.12 P99: 8.93 P99.9: 168.77 P99.99: 506.37 ------------------------------------------------------ [ 0, 1 ] 19738 10.455% 10.455% ## ( 1, 2 ] 118000 62.501% 72.955% ############# ( 2, 3 ] 31420 16.642% 89.597% ### ( 3, 4 ] 8618 4.565% 94.162% # ( 4, 6 ] 6925 3.668% 97.830% # ( 6, 10 ] 3012 1.595% 99.425% ( 10, 15 ] 420 0.222% 99.648% ( 15, 22 ] 86 0.046% 99.693% ( 22, 34 ] 27 0.014% 99.708% ( 34, 51 ] 8 0.004% 99.712% ( 51, 76 ] 4 0.002% 99.714% ( 76, 110 ] 122 0.065% 99.779% ( 110, 170 ] 234 0.124% 99.903% ( 170, 250 ] 85 0.045% 99.948% ( 250, 380 ] 46 0.024% 99.972% ( 380, 580 ] 54 0.029% 100.001% ( 580, 870 ] 23 0.012% 100.013% ( 870, 1300 ] 8 0.004% 100.017% ( 1300, 1900 ] 2 0.001% 100.018% ( 1900, 2900 ] 11 0.006% 100.024% ( 2900, 4400 ] 7 0.004% 100.028% ( 4400, 6600 ] 3 0.002% 100.029% ** Level 4 read latency histogram (micros): Count: 305 Average: 113.1148 StdDev: 505.21 Min: 0 Median: 1.8458 Max: 4741 Percentiles: P50: 1.85 P75: 5.09 P99: 2890.00 P99.9: 4741.00 P99.99: 4741.00 ------------------------------------------------------ [ 0, 1 ] 51 16.721% 16.721% ### ( 1, 2 ] 120 39.344% 56.066% ######## ( 2, 3 ] 36 11.803% 67.869% ## ( 3, 4 ] 13 4.262% 72.131% # ( 4, 6 ] 16 5.246% 77.377% # ( 6, 10 ] 28 9.180% 86.557% ## ( 10, 15 ] 11 3.607% 90.164% # ( 15, 22 ] 3 0.984% 91.148% ( 22, 34 ] 1 0.328% 91.475% ( 51, 76 ] 1 0.328% 91.803% ( 170, 250 ] 5 1.639% 93.443% ( 250, 380 ] 3 0.984% 94.426% ( 380, 580 ] 3 0.984% 95.410% ( 580, 870 ] 1 0.328% 95.738% ( 870, 1300 ] 2 0.656% 96.393% ( 1300, 1900 ] 3 0.984% 97.377% ( 1900, 2900 ] 5 1.639% 99.016% ( 2900, 4400 ] 2 0.656% 99.672% ( 4400, 6600 ] 1 0.328% 100.000% ** Level 5 read latency histogram (micros): Count: 20 Average: 701.2500 StdDev: 819.05 Min: 86 Median: 250.0000 Max: 3304 Percentiles: P50: 250.00 P75: 1300.00 P99: 3304.00 P99.9: 3304.00 P99.99: 3304.00 ------------------------------------------------------ ( 76, 110 ] 4 20.000% 20.000% #### ( 110, 170 ] 3 15.000% 35.000% ### ( 170, 250 ] 3 15.000% 50.000% ### ( 250, 380 ] 1 5.000% 55.000% # ( 380, 580 ] 2 10.000% 65.000% ## ( 580, 870 ] 1 5.000% 70.000% # ( 870, 1300 ] 1 5.000% 75.000% # ( 1300, 1900 ] 3 15.000% 90.000% ### ( 1900, 2900 ] 1 5.000% 95.000% # ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 6.1 total, 3.1 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 320.17 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.590 H:M:S, 9.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 321.19 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.375 H:M:S, 12.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/07-21:24:09 ... thread 0: (1000000,3000000) ops and (331401.5,335593.3) ops/second in (3.017488,8.939392) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/10 2.27 GB 12.6 1.9 0.0 1.9 4.7 2.8 0.0 1.7 126.7 310.4 15.42 14.40 47 0.328 1982K 2090 0.0 0.0 L1 1/1 3.03 MB 0.0 0.3 0.3 0.0 0.3 0.3 0.2 1.0 240.8 240.5 1.31 1.27 1 1.308 319K 406 0.0 0.0 L2 1/0 96.97 MB 1.0 0.3 0.2 0.2 0.3 0.2 0.4 1.8 246.8 246.6 1.41 1.36 2 0.707 354K 190 0.0 0.0 L3 20/0 906.54 MB 0.9 1.1 0.2 0.9 1.1 0.2 0.2 4.7 241.3 240.9 4.64 4.30 2 2.319 1136K 1741 0.0 0.0 L4 169/0 9.40 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.3 1.0 136.3 136.3 1.23 0.87 1 1.232 170K 0 0.0 0.0 L5 1048/0 86.25 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1251/11 98.90 GB 0.0 3.8 0.9 2.9 6.6 3.7 1.7 2.4 162.6 280.5 24.01 22.20 53 0.453 3964K 4427 0.0 0.0 Int 0/0 0.00 KB 0.0 3.8 0.9 2.9 6.6 3.7 1.7 2.4 162.6 280.5 24.01 22.20 53 0.453 3964K 4427 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.8 0.9 2.9 3.8 0.9 0.0 0.0 243.8 243.6 16.01 15.08 13 1.232 3964K 4427 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 354.4 8.00 7.12 40 0.200 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.2 total, 8.9 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.58 GB write, 734.74 MB/s write, 3.81 GB read, 425.94 MB/s read, 24.0 seconds Interval compaction: 6.58 GB write, 753.15 MB/s write, 3.81 GB read, 436.61 MB/s read, 24.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 7, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 7, interval: 7 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 639920 Average: 2.1735 StdDev: 12.58 Min: 1 Median: 1.4651 Max: 3879 Percentiles: P50: 1.47 P75: 1.85 P99: 5.54 P99.9: 10.70 P99.99: 79.03 ------------------------------------------------------ [ 0, 1 ] 126239 19.727% 19.727% #### ( 1, 2 ] 416489 65.085% 84.812% ############# ( 2, 3 ] 59332 9.272% 94.084% ## ( 3, 4 ] 22215 3.472% 97.555% # ( 4, 6 ] 12027 1.879% 99.435% ( 6, 10 ] 2900 0.453% 99.888% ( 10, 15 ] 564 0.088% 99.976% ( 15, 22 ] 60 0.009% 99.985% ( 22, 34 ] 13 0.002% 99.987% ( 34, 51 ] 7 0.001% 99.988% ( 51, 76 ] 7 0.001% 99.990% ( 76, 110 ] 45 0.007% 99.997% ( 110, 170 ] 29 0.005% 100.001% ( 170, 250 ] 23 0.004% 100.005% ( 250, 380 ] 20 0.003% 100.008% ( 380, 580 ] 9 0.001% 100.009% ( 580, 870 ] 11 0.002% 100.011% ( 870, 1300 ] 8 0.001% 100.012% ( 1300, 1900 ] 5 0.001% 100.013% ( 1900, 2900 ] 6 0.001% 100.014% ( 2900, 4400 ] 3 0.000% 100.014% ** Level 1 read latency histogram (micros): Count: 12338 Average: 3.2913 StdDev: 32.39 Min: 1 Median: 1.2988 Max: 2168 Percentiles: P50: 1.30 P75: 1.75 P99: 9.60 P99.9: 364.19 P99.99: 2168.00 ------------------------------------------------------ [ 0, 1 ] 4139 33.547% 33.547% ####### ( 1, 2 ] 6794 55.066% 88.612% ########### ( 2, 3 ] 871 7.059% 95.672% # ( 3, 4 ] 152 1.232% 96.904% ( 4, 6 ] 137 1.110% 98.014% ( 6, 10 ] 135 1.094% 99.108% ( 10, 15 ] 28 0.227% 99.335% ( 15, 22 ] 6 0.049% 99.384% ( 22, 34 ] 1 0.008% 99.392% ( 34, 51 ] 26 0.211% 99.603% ( 51, 76 ] 9 0.073% 99.676% ( 76, 110 ] 4 0.032% 99.708% ( 110, 170 ] 1 0.008% 99.716% ( 170, 250 ] 13 0.105% 99.822% ( 250, 380 ] 11 0.089% 99.911% ( 380, 580 ] 4 0.032% 99.943% ( 580, 870 ] 5 0.041% 99.984% ( 1900, 2900 ] 2 0.016% 100.000% ** Level 2 read latency histogram (micros): Count: 88547 Average: 2.0673 StdDev: 3.89 Min: 1 Median: 1.4430 Max: 876 Percentiles: P50: 1.44 P75: 1.83 P99: 5.55 P99.9: 10.94 P99.99: 132.91 ------------------------------------------------------ [ 0, 1 ] 19169 21.648% 21.648% #### ( 1, 2 ] 56665 63.994% 85.643% ############# ( 2, 3 ] 7625 8.611% 94.254% ## ( 3, 4 ] 2935 3.315% 97.569% # ( 4, 6 ] 1637 1.849% 99.417% ( 6, 10 ] 412 0.465% 99.883% ( 10, 15 ] 82 0.093% 99.975% ( 15, 22 ] 9 0.010% 99.985% ( 22, 34 ] 2 0.002% 99.988% ( 34, 51 ] 1 0.001% 99.989% ( 110, 170 ] 3 0.003% 99.992% ( 170, 250 ] 2 0.002% 99.994% ( 250, 380 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 266651 Average: 3.2663 StdDev: 34.14 Min: 0 Median: 1.6071 Max: 5967 Percentiles: P50: 1.61 P75: 2.00 P99: 8.79 P99.9: 177.20 P99.99: 542.04 ------------------------------------------------------ [ 0, 1 ] 29534 11.076% 11.076% ## ( 1, 2 ] 170954 64.112% 75.187% ############# ( 2, 3 ] 40272 15.103% 90.290% ### ( 3, 4 ] 11624 4.359% 94.650% # ( 4, 6 ] 8613 3.230% 97.880% # ( 6, 10 ] 4285 1.607% 99.487% ( 10, 15 ] 465 0.174% 99.661% ( 15, 22 ] 94 0.035% 99.696% ( 22, 34 ] 27 0.010% 99.706% ( 34, 51 ] 9 0.003% 99.710% ( 51, 76 ] 9 0.003% 99.713% ( 76, 110 ] 176 0.066% 99.779% ( 110, 170 ] 312 0.117% 99.896% ( 170, 250 ] 115 0.043% 99.939% ( 250, 380 ] 77 0.029% 99.968% ( 380, 580 ] 72 0.027% 99.995% ( 580, 870 ] 36 0.014% 100.009% ( 870, 1300 ] 9 0.003% 100.012% ( 1300, 1900 ] 2 0.001% 100.013% ( 1900, 2900 ] 11 0.004% 100.017% ( 2900, 4400 ] 7 0.003% 100.020% ( 4400, 6600 ] 3 0.001% 100.021% ** Level 4 read latency histogram (micros): Count: 305 Average: 113.1148 StdDev: 505.21 Min: 0 Median: 1.8458 Max: 4741 Percentiles: P50: 1.85 P75: 5.09 P99: 2890.00 P99.9: 4741.00 P99.99: 4741.00 ------------------------------------------------------ [ 0, 1 ] 51 16.721% 16.721% ### ( 1, 2 ] 120 39.344% 56.066% ######## ( 2, 3 ] 36 11.803% 67.869% ## ( 3, 4 ] 13 4.262% 72.131% # ( 4, 6 ] 16 5.246% 77.377% # ( 6, 10 ] 28 9.180% 86.557% ## ( 10, 15 ] 11 3.607% 90.164% # ( 15, 22 ] 3 0.984% 91.148% ( 22, 34 ] 1 0.328% 91.475% ( 51, 76 ] 1 0.328% 91.803% ( 170, 250 ] 5 1.639% 93.443% ( 250, 380 ] 3 0.984% 94.426% ( 380, 580 ] 3 0.984% 95.410% ( 580, 870 ] 1 0.328% 95.738% ( 870, 1300 ] 2 0.656% 96.393% ( 1300, 1900 ] 3 0.984% 97.377% ( 1900, 2900 ] 5 1.639% 99.016% ( 2900, 4400 ] 2 0.656% 99.672% ( 4400, 6600 ] 1 0.328% 100.000% ** Level 5 read latency histogram (micros): Count: 40 Average: 898.2250 StdDev: 1018.24 Min: 86 Median: 354.0000 Max: 3724 Percentiles: P50: 354.00 P75: 1500.00 P99: 3724.00 P99.9: 3724.00 P99.99: 3724.00 ------------------------------------------------------ ( 76, 110 ] 4 10.000% 10.000% ## ( 110, 170 ] 7 17.500% 27.500% #### ( 170, 250 ] 5 12.500% 40.000% ### ( 250, 380 ] 5 12.500% 52.500% ### ( 380, 580 ] 3 7.500% 60.000% ## ( 580, 870 ] 2 5.000% 65.000% # ( 870, 1300 ] 2 5.000% 70.000% # ( 1300, 1900 ] 6 15.000% 85.000% ### ( 1900, 2900 ] 2 5.000% 90.000% # ( 2900, 4400 ] 4 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 9.2 total, 3.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 322.14 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.962 H:M:S, 10.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 326.16 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.373 H:M:S, 12.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/07-21:24:13 ... thread 0: (1000000,4000000) ops and (312497.5,329505.1) ops/second in (3.200026,12.139418) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/14 3.26 GB 88.1 2.7 0.0 2.7 6.5 3.7 0.0 1.7 129.8 309.7 21.35 19.97 65 0.328 2814K 2987 0.0 0.0 L1 1/1 3.03 MB 0.0 0.3 0.3 0.0 0.3 0.3 0.2 1.0 240.8 240.5 1.31 1.27 1 1.308 319K 406 0.0 0.0 L2 1/0 96.97 MB 1.0 0.3 0.2 0.2 0.3 0.2 0.4 1.8 246.8 246.6 1.41 1.36 2 0.707 354K 190 0.0 0.0 L3 20/0 906.54 MB 0.9 1.1 0.2 0.9 1.1 0.2 0.2 4.7 241.3 240.9 4.64 4.30 2 2.319 1136K 1741 0.0 0.0 L4 169/0 9.40 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.3 1.0 136.3 136.3 1.23 0.87 1 1.232 170K 0 0.0 0.0 L5 1048/0 86.25 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1257/15 99.88 GB 0.0 4.6 0.9 3.7 8.4 4.6 1.7 2.2 157.7 285.9 29.94 27.77 71 0.422 4795K 5324 0.0 0.0 Int 0/0 0.00 KB 0.0 4.6 0.9 3.7 8.4 4.6 1.7 2.2 157.7 285.9 29.94 27.77 71 0.422 4795K 5324 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.6 0.9 3.7 4.6 0.9 0.0 0.0 247.2 246.9 19.11 18.12 16 1.194 4795K 5324 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 354.7 10.83 9.65 55 0.197 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.4 total, 12.1 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.36 GB write, 692.27 MB/s write, 4.61 GB read, 381.94 MB/s read, 29.9 seconds Interval compaction: 8.36 GB write, 705.05 MB/s write, 4.61 GB read, 388.99 MB/s read, 29.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 3, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 3, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 9, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 3, total-stops: 9, interval: 12 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1058236 Average: 2.1896 StdDev: 10.67 Min: 1 Median: 1.4853 Max: 3879 Percentiles: P50: 1.49 P75: 1.86 P99: 5.58 P99.9: 9.97 P99.99: 18.60 ------------------------------------------------------ [ 0, 1 ] 189535 17.910% 17.910% #### ( 1, 2 ] 699727 66.122% 84.032% ############# ( 2, 3 ] 105637 9.982% 94.015% ## ( 3, 4 ] 36976 3.494% 97.509% # ( 4, 6 ] 19989 1.889% 99.398% ( 6, 10 ] 5357 0.506% 99.904% ( 10, 15 ] 867 0.082% 99.986% ( 15, 22 ] 86 0.008% 99.994% ( 22, 34 ] 16 0.002% 99.996% ( 34, 51 ] 9 0.001% 99.997% ( 51, 76 ] 10 0.001% 99.997% ( 76, 110 ] 74 0.007% 100.004% ( 110, 170 ] 69 0.007% 100.011% ( 170, 250 ] 42 0.004% 100.015% ( 250, 380 ] 31 0.003% 100.018% ( 380, 580 ] 18 0.002% 100.020% ( 580, 870 ] 20 0.002% 100.021% ( 870, 1300 ] 14 0.001% 100.023% ( 1300, 1900 ] 7 0.001% 100.023% ( 1900, 2900 ] 6 0.001% 100.024% ( 2900, 4400 ] 3 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 12338 Average: 3.2913 StdDev: 32.39 Min: 1 Median: 1.2988 Max: 2168 Percentiles: P50: 1.30 P75: 1.75 P99: 9.60 P99.9: 364.19 P99.99: 2168.00 ------------------------------------------------------ [ 0, 1 ] 4139 33.547% 33.547% ####### ( 1, 2 ] 6794 55.066% 88.612% ########### ( 2, 3 ] 871 7.059% 95.672% # ( 3, 4 ] 152 1.232% 96.904% ( 4, 6 ] 137 1.110% 98.014% ( 6, 10 ] 135 1.094% 99.108% ( 10, 15 ] 28 0.227% 99.335% ( 15, 22 ] 6 0.049% 99.384% ( 22, 34 ] 1 0.008% 99.392% ( 34, 51 ] 26 0.211% 99.603% ( 51, 76 ] 9 0.073% 99.676% ( 76, 110 ] 4 0.032% 99.708% ( 110, 170 ] 1 0.008% 99.716% ( 170, 250 ] 13 0.105% 99.822% ( 250, 380 ] 11 0.089% 99.911% ( 380, 580 ] 4 0.032% 99.943% ( 580, 870 ] 5 0.041% 99.984% ( 1900, 2900 ] 2 0.016% 100.000% ** Level 2 read latency histogram (micros): Count: 88547 Average: 2.0673 StdDev: 3.89 Min: 1 Median: 1.4430 Max: 876 Percentiles: P50: 1.44 P75: 1.83 P99: 5.55 P99.9: 10.94 P99.99: 132.91 ------------------------------------------------------ [ 0, 1 ] 19169 21.648% 21.648% #### ( 1, 2 ] 56665 63.994% 85.643% ############# ( 2, 3 ] 7625 8.611% 94.254% ## ( 3, 4 ] 2935 3.315% 97.569% # ( 4, 6 ] 1637 1.849% 99.417% ( 6, 10 ] 412 0.465% 99.883% ( 10, 15 ] 82 0.093% 99.975% ( 15, 22 ] 9 0.010% 99.985% ( 22, 34 ] 2 0.002% 99.988% ( 34, 51 ] 1 0.001% 99.989% ( 110, 170 ] 3 0.003% 99.992% ( 170, 250 ] 2 0.002% 99.994% ( 250, 380 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 266651 Average: 3.2663 StdDev: 34.14 Min: 0 Median: 1.6071 Max: 5967 Percentiles: P50: 1.61 P75: 2.00 P99: 8.79 P99.9: 177.20 P99.99: 542.04 ------------------------------------------------------ [ 0, 1 ] 29534 11.076% 11.076% ## ( 1, 2 ] 170954 64.112% 75.187% ############# ( 2, 3 ] 40272 15.103% 90.290% ### ( 3, 4 ] 11624 4.359% 94.650% # ( 4, 6 ] 8613 3.230% 97.880% # ( 6, 10 ] 4285 1.607% 99.487% ( 10, 15 ] 465 0.174% 99.661% ( 15, 22 ] 94 0.035% 99.696% ( 22, 34 ] 27 0.010% 99.706% ( 34, 51 ] 9 0.003% 99.710% ( 51, 76 ] 9 0.003% 99.713% ( 76, 110 ] 176 0.066% 99.779% ( 110, 170 ] 312 0.117% 99.896% ( 170, 250 ] 115 0.043% 99.939% ( 250, 380 ] 77 0.029% 99.968% ( 380, 580 ] 72 0.027% 99.995% ( 580, 870 ] 36 0.014% 100.009% ( 870, 1300 ] 9 0.003% 100.012% ( 1300, 1900 ] 2 0.001% 100.013% ( 1900, 2900 ] 11 0.004% 100.017% ( 2900, 4400 ] 7 0.003% 100.020% ( 4400, 6600 ] 3 0.001% 100.021% ** Level 4 read latency histogram (micros): Count: 305 Average: 113.1148 StdDev: 505.21 Min: 0 Median: 1.8458 Max: 4741 Percentiles: P50: 1.85 P75: 5.09 P99: 2890.00 P99.9: 4741.00 P99.99: 4741.00 ------------------------------------------------------ [ 0, 1 ] 51 16.721% 16.721% ### ( 1, 2 ] 120 39.344% 56.066% ######## ( 2, 3 ] 36 11.803% 67.869% ## ( 3, 4 ] 13 4.262% 72.131% # ( 4, 6 ] 16 5.246% 77.377% # ( 6, 10 ] 28 9.180% 86.557% ## ( 10, 15 ] 11 3.607% 90.164% # ( 15, 22 ] 3 0.984% 91.148% ( 22, 34 ] 1 0.328% 91.475% ( 51, 76 ] 1 0.328% 91.803% ( 170, 250 ] 5 1.639% 93.443% ( 250, 380 ] 3 0.984% 94.426% ( 380, 580 ] 3 0.984% 95.410% ( 580, 870 ] 1 0.328% 95.738% ( 870, 1300 ] 2 0.656% 96.393% ( 1300, 1900 ] 3 0.984% 97.377% ( 1900, 2900 ] 5 1.639% 99.016% ( 2900, 4400 ] 2 0.656% 99.672% ( 4400, 6600 ] 1 0.328% 100.000% ** Level 5 read latency histogram (micros): Count: 40 Average: 898.2250 StdDev: 1018.24 Min: 86 Median: 354.0000 Max: 3724 Percentiles: P50: 354.00 P75: 1500.00 P99: 3724.00 P99.9: 3724.00 P99.99: 3724.00 ------------------------------------------------------ ( 76, 110 ] 4 10.000% 10.000% ## ( 110, 170 ] 7 17.500% 27.500% #### ( 170, 250 ] 5 12.500% 40.000% ### ( 250, 380 ] 5 12.500% 52.500% ### ( 380, 580 ] 3 7.500% 60.000% ## ( 580, 870 ] 2 5.000% 65.000% # ( 870, 1300 ] 2 5.000% 70.000% # ( 1300, 1900 ] 6 15.000% 85.000% ### ( 1900, 2900 ] 2 5.000% 90.000% # ( 2900, 4400 ] 4 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 12.4 total, 3.2 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 318.37 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.459 H:M:S, 11.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 307.55 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.496 H:M:S, 15.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/07-21:24:19 ... thread 0: (1000000,5000000) ops and (158330.2,270924.4) ops/second in (6.315913,18.455331) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/6 2.03 GB 75.5 3.7 0.0 3.7 8.4 4.7 0.0 1.8 135.6 307.2 27.87 26.11 84 0.332 3837K 4003 0.0 0.0 L1 6/0 696.80 MB 69.7 2.5 2.5 0.0 2.4 2.4 0.2 1.0 234.1 232.0 10.77 10.45 2 5.384 2559K 22K 0.0 0.0 L2 13/9 1.55 GB 4.6 0.3 0.2 0.2 0.3 0.2 1.8 1.8 246.8 246.6 1.41 1.36 2 0.707 354K 190 0.0 0.0 L3 20/9 906.54 MB 0.7 1.1 0.2 0.9 1.1 0.2 0.2 4.7 241.3 240.9 4.64 4.30 2 2.319 1136K 1741 0.0 0.0 L4 169/0 9.40 GB 1.0 0.2 0.2 0.0 0.2 0.2 0.3 1.0 136.3 136.3 1.23 0.87 1 1.232 170K 0 0.0 0.0 L5 1048/0 86.25 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1267/24 100.78 GB 0.0 7.8 3.0 4.7 12.4 7.7 3.1 2.7 172.8 276.4 45.93 43.09 91 0.505 8058K 28K 0.0 0.0 Int 0/0 0.00 KB 0.0 7.8 3.0 4.7 12.4 7.7 3.1 2.7 172.8 276.4 45.93 43.09 91 0.505 8058K 28K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 7.8 3.0 4.7 7.7 3.0 0.0 0.0 245.0 244.1 32.40 31.04 21 1.543 8058K 28K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 353.9 13.53 12.05 70 0.193 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 18.7 total, 18.5 interval Flush(GB): cumulative 4.676, interval 4.676 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.40 GB write, 679.59 MB/s write, 7.75 GB read, 424.84 MB/s read, 45.9 seconds Interval compaction: 12.40 GB write, 687.83 MB/s write, 7.75 GB read, 429.99 MB/s read, 45.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 9, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 9, interval: 22 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1790978 Average: 2.2005 StdDev: 10.85 Min: 0 Median: 1.4831 Max: 4124 Percentiles: P50: 1.48 P75: 1.86 P99: 5.65 P99.9: 10.31 P99.99: 89.47 ------------------------------------------------------ [ 0, 1 ] 316327 17.662% 17.662% #### ( 1, 2 ] 1198814 66.936% 84.599% ############# ( 2, 3 ] 173357 9.679% 94.278% ## ( 3, 4 ] 58904 3.289% 97.567% # ( 4, 6 ] 31147 1.739% 99.306% ( 6, 10 ] 10549 0.589% 99.895% ( 10, 15 ] 1447 0.081% 99.976% ( 15, 22 ] 137 0.008% 99.983% ( 22, 34 ] 18 0.001% 99.984% ( 34, 51 ] 13 0.001% 99.985% ( 51, 76 ] 15 0.001% 99.986% ( 76, 110 ] 179 0.010% 99.996% ( 110, 170 ] 187 0.010% 100.006% ( 170, 250 ] 98 0.005% 100.012% ( 250, 380 ] 62 0.003% 100.015% ( 380, 580 ] 50 0.003% 100.018% ( 580, 870 ] 40 0.002% 100.020% ( 870, 1300 ] 19 0.001% 100.021% ( 1300, 1900 ] 8 0.000% 100.022% ( 1900, 2900 ] 12 0.001% 100.023% ( 2900, 4400 ] 5 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 56822 Average: 4.6303 StdDev: 34.71 Min: 0 Median: 1.7429 Max: 2168 Percentiles: P50: 1.74 P75: 2.56 P99: 10.00 P99.9: 312.18 P99.99: 481.67 ------------------------------------------------------ [ 0, 1 ] 5020 8.835% 8.835% ## ( 1, 2 ] 31488 55.415% 64.250% ########### ( 2, 3 ] 10827 19.054% 83.304% #### ( 3, 4 ] 4037 7.105% 90.409% # ( 4, 6 ] 3823 6.728% 97.137% # ( 6, 10 ] 1060 1.865% 99.002% ( 10, 15 ] 170 0.299% 99.301% ( 15, 22 ] 42 0.074% 99.375% ( 22, 34 ] 3 0.005% 99.381% ( 34, 51 ] 27 0.048% 99.428% ( 51, 76 ] 9 0.016% 99.444% ( 76, 110 ] 45 0.079% 99.523% ( 110, 170 ] 103 0.181% 99.704% ( 170, 250 ] 83 0.146% 99.850% ( 250, 380 ] 61 0.107% 99.958% ( 380, 580 ] 38 0.067% 100.025% ( 580, 870 ] 25 0.044% 100.069% ( 870, 1300 ] 8 0.014% 100.083% ( 1300, 1900 ] 6 0.011% 100.093% ( 1900, 2900 ] 3 0.005% 100.099% ** Level 2 read latency histogram (micros): Count: 88547 Average: 2.0673 StdDev: 3.89 Min: 1 Median: 1.4430 Max: 876 Percentiles: P50: 1.44 P75: 1.83 P99: 5.55 P99.9: 10.94 P99.99: 132.91 ------------------------------------------------------ [ 0, 1 ] 19169 21.648% 21.648% #### ( 1, 2 ] 56665 63.994% 85.643% ############# ( 2, 3 ] 7625 8.611% 94.254% ## ( 3, 4 ] 2935 3.315% 97.569% # ( 4, 6 ] 1637 1.849% 99.417% ( 6, 10 ] 412 0.465% 99.883% ( 10, 15 ] 82 0.093% 99.975% ( 15, 22 ] 9 0.010% 99.985% ( 22, 34 ] 2 0.002% 99.988% ( 34, 51 ] 1 0.001% 99.989% ( 110, 170 ] 3 0.003% 99.992% ( 170, 250 ] 2 0.002% 99.994% ( 250, 380 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 266652 Average: 3.2673 StdDev: 34.14 Min: 0 Median: 1.6071 Max: 5967 Percentiles: P50: 1.61 P75: 2.00 P99: 8.79 P99.9: 177.89 P99.99: 542.04 ------------------------------------------------------ [ 0, 1 ] 29534 11.076% 11.076% ## ( 1, 2 ] 170954 64.111% 75.187% ############# ( 2, 3 ] 40272 15.103% 90.290% ### ( 3, 4 ] 11624 4.359% 94.649% # ( 4, 6 ] 8613 3.230% 97.879% # ( 6, 10 ] 4285 1.607% 99.486% ( 10, 15 ] 465 0.174% 99.661% ( 15, 22 ] 94 0.035% 99.696% ( 22, 34 ] 27 0.010% 99.706% ( 34, 51 ] 9 0.003% 99.709% ( 51, 76 ] 9 0.003% 99.713% ( 76, 110 ] 176 0.066% 99.779% ( 110, 170 ] 312 0.117% 99.896% ( 170, 250 ] 115 0.043% 99.939% ( 250, 380 ] 78 0.029% 99.968% ( 380, 580 ] 72 0.027% 99.995% ( 580, 870 ] 36 0.014% 100.009% ( 870, 1300 ] 9 0.003% 100.012% ( 1300, 1900 ] 2 0.001% 100.013% ( 1900, 2900 ] 11 0.004% 100.017% ( 2900, 4400 ] 7 0.003% 100.020% ( 4400, 6600 ] 3 0.001% 100.021% ** Level 4 read latency histogram (micros): Count: 305 Average: 113.1148 StdDev: 505.21 Min: 0 Median: 1.8458 Max: 4741 Percentiles: P50: 1.85 P75: 5.09 P99: 2890.00 P99.9: 4741.00 P99.99: 4741.00 ------------------------------------------------------ [ 0, 1 ] 51 16.721% 16.721% ### ( 1, 2 ] 120 39.344% 56.066% ######## ( 2, 3 ] 36 11.803% 67.869% ## ( 3, 4 ] 13 4.262% 72.131% # ( 4, 6 ] 16 5.246% 77.377% # ( 6, 10 ] 28 9.180% 86.557% ## ( 10, 15 ] 11 3.607% 90.164% # ( 15, 22 ] 3 0.984% 91.148% ( 22, 34 ] 1 0.328% 91.475% ( 51, 76 ] 1 0.328% 91.803% ( 170, 250 ] 5 1.639% 93.443% ( 250, 380 ] 3 0.984% 94.426% ( 380, 580 ] 3 0.984% 95.410% ( 580, 870 ] 1 0.328% 95.738% ( 870, 1300 ] 2 0.656% 96.393% ( 1300, 1900 ] 3 0.984% 97.377% ( 1900, 2900 ] 5 1.639% 99.016% ( 2900, 4400 ] 2 0.656% 99.672% ( 4400, 6600 ] 1 0.328% 100.000% ** Level 5 read latency histogram (micros): Count: 40 Average: 898.2250 StdDev: 1018.24 Min: 86 Median: 354.0000 Max: 3724 Percentiles: P50: 354.00 P75: 1500.00 P99: 3724.00 P99.9: 3724.00 P99.99: 3724.00 ------------------------------------------------------ ( 76, 110 ] 4 10.000% 10.000% ## ( 110, 170 ] 7 17.500% 27.500% #### ( 170, 250 ] 5 12.500% 40.000% ### ( 250, 380 ] 5 12.500% 52.500% ### ( 380, 580 ] 3 7.500% 60.000% ## ( 580, 870 ] 2 5.000% 65.000% # ( 870, 1300 ] 2 5.000% 70.000% # ( 1300, 1900 ] 6 15.000% 85.000% ### ( 1900, 2900 ] 2 5.000% 90.000% # ( 2900, 4400 ] 4 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 18.7 total, 6.3 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 263.42 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.917 H:M:S, 26.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 155.83 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.459 H:M:S, 54.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 2024/12/07-21:24:23 ... thread 0: (1000000,6000000) ops and (237426.2,264700.1) ops/second in (4.211835,22.667166) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/4 3.00 GB 276.2 5.8 0.0 5.8 11.5 5.6 0.0 2.0 146.7 288.5 40.79 37.31 101 0.404 6075K 12K 0.0 0.0 L1 2/0 181.69 MB 18.2 2.5 2.5 0.0 2.4 2.4 0.2 1.0 234.1 232.0 10.77 10.45 2 5.384 2559K 22K 0.0 0.0 L2 3/2 386.33 MB 1.3 0.3 0.2 0.2 0.3 0.2 2.3 1.8 246.8 246.6 1.41 1.36 2 0.707 354K 190 0.0 0.0 L3 30/8 1.36 GB 0.9 3.2 1.9 1.3 3.2 1.9 0.2 1.7 202.9 202.4 16.02 14.04 8 2.002 3300K 7072 0.0 0.0 L4 182/14 9.91 GB 1.0 0.7 0.7 0.0 0.7 0.7 0.9 1.1 124.6 124.4 6.07 3.60 11 0.552 742K 367 0.0 0.0 L5 1056/0 86.92 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1283/28 101.75 GB 0.0 12.6 5.2 7.3 18.2 10.9 5.0 3.2 171.3 247.9 75.07 66.76 124 0.605 13M 42K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.6 5.2 7.3 18.2 10.9 5.0 3.2 171.3 247.9 75.07 66.76 124 0.605 13M 42K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.6 5.2 7.3 12.5 5.2 0.0 0.0 223.2 222.4 57.62 52.09 41 1.405 13M 42K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 332.3 17.44 14.67 83 0.210 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 22.9 total, 22.7 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 18.18 GB write, 813.01 MB/s write, 12.56 GB read, 561.74 MB/s read, 75.1 seconds Interval compaction: 18.18 GB write, 821.04 MB/s write, 12.56 GB read, 567.29 MB/s read, 75.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 23, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 23, interval: 36 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2212967 Average: 2.3734 StdDev: 13.65 Min: 0 Median: 1.5371 Max: 4258 Percentiles: P50: 1.54 P75: 1.92 P99: 5.94 P99.9: 11.68 P99.99: 104.48 ------------------------------------------------------ [ 0, 1 ] 333266 15.060% 15.060% ### ( 1, 2 ] 1439651 65.055% 80.115% ############# ( 2, 3 ] 256498 11.591% 91.706% ## ( 3, 4 ] 97495 4.406% 96.111% # ( 4, 6 ] 65780 2.972% 99.084% # ( 6, 10 ] 17293 0.781% 99.865% ( 10, 15 ] 2301 0.104% 99.969% ( 15, 22 ] 241 0.011% 99.980% ( 22, 34 ] 23 0.001% 99.981% ( 34, 51 ] 15 0.001% 99.982% ( 51, 76 ] 17 0.001% 99.983% ( 76, 110 ] 199 0.009% 99.992% ( 110, 170 ] 251 0.011% 100.003% ( 170, 250 ] 145 0.007% 100.009% ( 250, 380 ] 113 0.005% 100.015% ( 380, 580 ] 109 0.005% 100.019% ( 580, 870 ] 76 0.003% 100.023% ( 870, 1300 ] 42 0.002% 100.025% ( 1300, 1900 ] 16 0.001% 100.025% ( 1900, 2900 ] 19 0.001% 100.026% ( 2900, 4400 ] 10 0.000% 100.027% ** Level 1 read latency histogram (micros): Count: 538833 Average: 4.6234 StdDev: 58.31 Min: 0 Median: 1.9405 Max: 9219 Percentiles: P50: 1.94 P75: 2.88 P99: 9.22 P99.9: 144.08 P99.99: 340.36 ------------------------------------------------------ [ 0, 1 ] 10524 1.953% 1.953% ( 1, 2 ] 275260 51.084% 53.038% ########## ( 2, 3 ] 134900 25.036% 78.073% ##### ( 3, 4 ] 51523 9.562% 87.635% ## ( 4, 6 ] 51049 9.474% 97.109% ## ( 6, 10 ] 12661 2.350% 99.459% ( 10, 15 ] 1827 0.339% 99.798% ( 15, 22 ] 259 0.048% 99.846% ( 22, 34 ] 19 0.004% 99.849% ( 34, 51 ] 32 0.006% 99.855% ( 51, 76 ] 11 0.002% 99.857% ( 76, 110 ] 95 0.018% 99.875% ( 110, 170 ] 238 0.044% 99.919% ( 170, 250 ] 216 0.040% 99.959% ( 250, 380 ] 239 0.044% 100.004% ( 380, 580 ] 245 0.045% 100.049% ( 580, 870 ] 191 0.035% 100.085% ( 870, 1300 ] 79 0.015% 100.099% ( 1300, 1900 ] 37 0.007% 100.106% ( 1900, 2900 ] 57 0.011% 100.117% ( 2900, 4400 ] 37 0.007% 100.124% ( 4400, 6600 ] 12 0.002% 100.126% ( 6600, 9900 ] 6 0.001% 100.127% ** Level 2 read latency histogram (micros): Count: 88547 Average: 2.0673 StdDev: 3.89 Min: 1 Median: 1.4430 Max: 876 Percentiles: P50: 1.44 P75: 1.83 P99: 5.55 P99.9: 10.94 P99.99: 132.91 ------------------------------------------------------ [ 0, 1 ] 19169 21.648% 21.648% #### ( 1, 2 ] 56665 63.994% 85.643% ############# ( 2, 3 ] 7625 8.611% 94.254% ## ( 3, 4 ] 2935 3.315% 97.569% # ( 4, 6 ] 1637 1.849% 99.417% ( 6, 10 ] 412 0.465% 99.883% ( 10, 15 ] 82 0.093% 99.975% ( 15, 22 ] 9 0.010% 99.985% ( 22, 34 ] 2 0.002% 99.988% ( 34, 51 ] 1 0.001% 99.989% ( 110, 170 ] 3 0.003% 99.992% ( 170, 250 ] 2 0.002% 99.994% ( 250, 380 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 502035 Average: 4.3564 StdDev: 51.71 Min: 0 Median: 1.7865 Max: 8242 Percentiles: P50: 1.79 P75: 2.63 P99: 8.99 P99.9: 197.92 P99.99: 402.32 ------------------------------------------------------ [ 0, 1 ] 31439 6.262% 6.262% # ( 1, 2 ] 279183 55.610% 61.873% ########### ( 2, 3 ] 104106 20.737% 82.609% #### ( 3, 4 ] 39270 7.822% 90.432% ## ( 4, 6 ] 36566 7.284% 97.715% # ( 6, 10 ] 8617 1.716% 99.432% ( 10, 15 ] 1340 0.267% 99.698% ( 15, 22 ] 197 0.039% 99.738% ( 22, 34 ] 36 0.007% 99.745% ( 34, 51 ] 14 0.003% 99.748% ( 51, 76 ] 12 0.002% 99.750% ( 76, 110 ] 216 0.043% 99.793% ( 110, 170 ] 441 0.088% 99.881% ( 170, 250 ] 275 0.055% 99.936% ( 250, 380 ] 244 0.049% 99.984% ( 380, 580 ] 258 0.051% 100.036% ( 580, 870 ] 173 0.034% 100.070% ( 870, 1300 ] 86 0.017% 100.087% ( 1300, 1900 ] 28 0.006% 100.093% ( 1900, 2900 ] 36 0.007% 100.100% ( 2900, 4400 ] 30 0.006% 100.106% ( 4400, 6600 ] 10 0.002% 100.108% ( 6600, 9900 ] 2 0.000% 100.108% ** Level 4 read latency histogram (micros): Count: 13738 Average: 111.6933 StdDev: 455.06 Min: 0 Median: 2.0935 Max: 7679 Percentiles: P50: 2.09 P75: 3.55 P99: 2522.20 P99.9: 4893.49 P99.99: 6325.14 ------------------------------------------------------ [ 0, 1 ] 298 2.169% 2.169% ( 1, 2 ] 6300 45.858% 48.027% ######### ( 2, 3 ] 2898 21.095% 69.122% #### ( 3, 4 ] 1472 10.715% 79.837% ## ( 4, 6 ] 942 6.857% 86.694% # ( 6, 10 ] 323 2.351% 89.045% ( 10, 15 ] 40 0.291% 89.336% ( 15, 22 ] 12 0.087% 89.423% ( 22, 34 ] 2 0.015% 89.438% ( 51, 76 ] 1 0.007% 89.445% ( 76, 110 ] 10 0.073% 89.518% ( 110, 170 ] 68 0.495% 90.013% ( 170, 250 ] 101 0.735% 90.748% ( 250, 380 ] 150 1.092% 91.840% ( 380, 580 ] 242 1.762% 93.602% ( 580, 870 ] 322 2.344% 95.946% ( 870, 1300 ] 240 1.747% 97.693% ( 1300, 1900 ] 123 0.895% 98.588% ( 1900, 2900 ] 91 0.662% 99.250% ( 2900, 4400 ] 85 0.619% 99.869% ( 4400, 6600 ] 19 0.138% 100.007% ( 6600, 9900 ] 3 0.022% 100.029% ** Level 5 read latency histogram (micros): Count: 80 Average: 1169.4375 StdDev: 1082.57 Min: 86 Median: 745.7143 Max: 4021 Percentiles: P50: 745.71 P75: 1900.00 P99: 4021.00 P99.9: 4021.00 P99.99: 4021.00 ------------------------------------------------------ ( 76, 110 ] 4 5.000% 5.000% # ( 110, 170 ] 8 10.000% 15.000% ## ( 170, 250 ] 6 7.500% 22.500% ## ( 250, 380 ] 7 8.750% 31.250% ## ( 380, 580 ] 11 13.750% 45.000% ### ( 580, 870 ] 7 8.750% 53.750% ## ( 870, 1300 ] 8 10.000% 63.750% ## ( 1300, 1900 ] 9 11.250% 75.000% ## ( 1900, 2900 ] 12 15.000% 90.000% ### ( 2900, 4400 ] 8 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 22.9 total, 4.2 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 257.94 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.275 H:M:S, 27.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 233.67 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.358 H:M:S, 32.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/12/07-21:24:26 ... thread 0: (1000000,7000000) ops and (312957.6,270662.3) ops/second in (3.195321,25.862487) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/8 3.93 GB 345.4 6.5 0.0 6.5 13.0 6.6 0.0 2.0 142.9 288.3 46.28 42.20 116 0.399 6715K 13K 0.0 0.0 L1 2/0 181.69 MB 18.2 2.5 2.5 0.0 2.4 2.4 0.2 1.0 234.1 232.0 10.77 10.45 2 5.384 2559K 22K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 2.3 1.8 246.8 246.6 1.41 1.36 2 0.707 354K 190 0.0 0.0 L3 23/0 980.08 MB 1.0 3.7 2.3 1.4 3.7 2.3 0.2 1.6 200.1 199.7 18.98 16.49 10 1.898 3856K 8354 0.0 0.0 L4 184/0 9.70 GB 1.0 1.6 1.5 0.1 1.6 1.5 0.9 1.1 137.6 137.4 11.72 7.65 25 0.469 1570K 817 0.0 0.0 L5 1068/0 87.91 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1294/8 102.67 GB 0.0 14.5 6.4 8.1 21.1 12.9 5.9 3.2 167.0 242.1 89.15 78.14 155 0.575 15M 44K 0.0 0.0 Int 0/0 0.00 KB 0.0 14.5 6.4 8.1 21.1 12.9 5.9 3.2 167.0 242.1 89.15 78.14 155 0.575 15M 44K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.5 6.4 8.1 14.5 6.4 0.0 0.0 216.6 215.9 68.75 61.03 59 1.165 15M 44K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 330.4 20.40 17.12 96 0.213 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 26.1 total, 25.9 interval Flush(GB): cumulative 6.583, interval 6.583 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 21.08 GB write, 827.43 MB/s write, 14.54 GB read, 570.85 MB/s read, 89.2 seconds Interval compaction: 21.08 GB write, 834.60 MB/s write, 14.54 GB read, 575.80 MB/s read, 89.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 30, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 30, interval: 43 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2456752 Average: 2.3680 StdDev: 13.51 Min: 0 Median: 1.5381 Max: 4258 Percentiles: P50: 1.54 P75: 1.92 P99: 5.93 P99.9: 11.79 P99.99: 99.98 ------------------------------------------------------ [ 0, 1 ] 371766 15.132% 15.132% ### ( 1, 2 ] 1591774 64.792% 79.924% ############# ( 2, 3 ] 287863 11.717% 91.641% ## ( 3, 4 ] 109297 4.449% 96.090% # ( 4, 6 ] 74055 3.014% 99.105% # ( 6, 10 ] 18577 0.756% 99.861% ( 10, 15 ] 2697 0.110% 99.971% ( 15, 22 ] 276 0.011% 99.982% ( 22, 34 ] 27 0.001% 99.983% ( 34, 51 ] 16 0.001% 99.984% ( 51, 76 ] 19 0.001% 99.984% ( 76, 110 ] 199 0.008% 99.992% ( 110, 170 ] 257 0.010% 100.003% ( 170, 250 ] 150 0.006% 100.009% ( 250, 380 ] 117 0.005% 100.014% ( 380, 580 ] 111 0.005% 100.018% ( 580, 870 ] 80 0.003% 100.022% ( 870, 1300 ] 48 0.002% 100.023% ( 1300, 1900 ] 18 0.001% 100.024% ( 1900, 2900 ] 20 0.001% 100.025% ( 2900, 4400 ] 11 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 599049 Average: 4.5148 StdDev: 56.79 Min: 0 Median: 1.9161 Max: 9219 Percentiles: P50: 1.92 P75: 2.84 P99: 9.20 P99.9: 148.13 P99.99: 357.02 ------------------------------------------------------ [ 0, 1 ] 13269 2.215% 2.215% ( 1, 2 ] 312479 52.163% 54.378% ########## ( 2, 3 ] 147159 24.565% 78.943% ##### ( 3, 4 ] 54942 9.172% 88.114% ## ( 4, 6 ] 54085 9.028% 97.143% ## ( 6, 10 ] 13914 2.323% 99.466% ( 10, 15 ] 2000 0.334% 99.800% ( 15, 22 ] 278 0.046% 99.846% ( 22, 34 ] 19 0.003% 99.849% ( 34, 51 ] 33 0.006% 99.855% ( 51, 76 ] 12 0.002% 99.857% ( 76, 110 ] 96 0.016% 99.873% ( 110, 170 ] 258 0.043% 99.916% ( 170, 250 ] 236 0.039% 99.955% ( 250, 380 ] 254 0.042% 99.997% ( 380, 580 ] 255 0.043% 100.040% ( 580, 870 ] 196 0.033% 100.073% ( 870, 1300 ] 85 0.014% 100.087% ( 1300, 1900 ] 42 0.007% 100.094% ( 1900, 2900 ] 63 0.011% 100.104% ( 2900, 4400 ] 38 0.006% 100.111% ( 4400, 6600 ] 13 0.002% 100.113% ( 6600, 9900 ] 6 0.001% 100.114% ** Level 2 read latency histogram (micros): Count: 88547 Average: 2.0673 StdDev: 3.89 Min: 1 Median: 1.4430 Max: 876 Percentiles: P50: 1.44 P75: 1.83 P99: 5.55 P99.9: 10.94 P99.99: 132.91 ------------------------------------------------------ [ 0, 1 ] 19169 21.648% 21.648% #### ( 1, 2 ] 56665 63.994% 85.643% ############# ( 2, 3 ] 7625 8.611% 94.254% ## ( 3, 4 ] 2935 3.315% 97.569% # ( 4, 6 ] 1637 1.849% 99.417% ( 6, 10 ] 412 0.465% 99.883% ( 10, 15 ] 82 0.093% 99.975% ( 15, 22 ] 9 0.010% 99.985% ( 22, 34 ] 2 0.002% 99.988% ( 34, 51 ] 1 0.001% 99.989% ( 110, 170 ] 3 0.003% 99.992% ( 170, 250 ] 2 0.002% 99.994% ( 250, 380 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 699417 Average: 4.0577 StdDev: 46.60 Min: 0 Median: 1.8243 Max: 8242 Percentiles: P50: 1.82 P75: 2.70 P99: 8.89 P99.9: 127.86 P99.99: 245.28 ------------------------------------------------------ [ 0, 1 ] 35304 5.048% 5.048% # ( 1, 2 ] 381420 54.534% 59.582% ########### ( 2, 3 ] 153458 21.941% 81.522% #### ( 3, 4 ] 58341 8.341% 89.864% ## ( 4, 6 ] 54899 7.849% 97.713% ## ( 6, 10 ] 12476 1.784% 99.497% ( 10, 15 ] 2057 0.294% 99.791% ( 15, 22 ] 284 0.041% 99.832% ( 22, 34 ] 43 0.006% 99.838% ( 34, 51 ] 15 0.002% 99.840% ( 51, 76 ] 16 0.002% 99.842% ( 76, 110 ] 259 0.037% 99.879% ( 110, 170 ] 489 0.070% 99.949% ( 170, 250 ] 304 0.043% 99.993% ( 250, 380 ] 261 0.037% 100.030% ( 380, 580 ] 272 0.039% 100.069% ( 580, 870 ] 182 0.026% 100.095% ( 870, 1300 ] 89 0.013% 100.108% ( 1300, 1900 ] 31 0.004% 100.112% ( 1900, 2900 ] 40 0.006% 100.118% ( 2900, 4400 ] 35 0.005% 100.123% ( 4400, 6600 ] 12 0.002% 100.124% ( 6600, 9900 ] 2 0.000% 100.125% ** Level 4 read latency histogram (micros): Count: 25685 Average: 73.6033 StdDev: 352.30 Min: 0 Median: 1.9766 Max: 7679 Percentiles: P50: 1.98 P75: 3.27 P99: 1677.95 P99.9: 4289.81 P99.99: 6207.46 ------------------------------------------------------ [ 0, 1 ] 653 2.542% 2.542% # ( 1, 2 ] 12481 48.593% 51.135% ########## ( 2, 3 ] 5430 21.141% 72.276% #### ( 3, 4 ] 2570 10.006% 82.281% ## ( 4, 6 ] 1620 6.307% 88.589% # ( 6, 10 ] 558 2.172% 90.761% ( 10, 15 ] 82 0.319% 91.080% ( 15, 22 ] 19 0.074% 91.154% ( 22, 34 ] 3 0.012% 91.166% ( 51, 76 ] 6 0.023% 91.189% ( 76, 110 ] 90 0.350% 91.540% ( 110, 170 ] 273 1.063% 92.603% ( 170, 250 ] 220 0.857% 93.459% ( 250, 380 ] 276 1.075% 94.534% ( 380, 580 ] 366 1.425% 95.959% ( 580, 870 ] 422 1.643% 97.602% ( 870, 1300 ] 276 1.075% 98.676% ( 1300, 1900 ] 132 0.514% 99.190% ( 1900, 2900 ] 98 0.382% 99.572% ( 2900, 4400 ] 91 0.354% 99.926% ( 4400, 6600 ] 20 0.078% 100.004% ( 6600, 9900 ] 3 0.012% 100.016% ** Level 5 read latency histogram (micros): Count: 140 Average: 1280.3429 StdDev: 1315.04 Min: 84 Median: 736.1538 Max: 7391 Percentiles: P50: 736.15 P75: 1955.56 P99: 6160.00 P99.9: 7391.00 P99.99: 7391.00 ------------------------------------------------------ ( 76, 110 ] 6 4.286% 4.286% # ( 110, 170 ] 11 7.857% 12.143% ## ( 170, 250 ] 16 11.429% 23.571% ## ( 250, 380 ] 13 9.286% 32.857% ## ( 380, 580 ] 17 12.143% 45.000% ## ( 580, 870 ] 13 9.286% 54.286% ## ( 870, 1300 ] 14 10.000% 64.286% ## ( 1300, 1900 ] 14 10.000% 74.286% ## ( 1900, 2900 ] 18 12.857% 87.143% ### ( 2900, 4400 ] 15 10.714% 97.857% ## ( 4400, 6600 ] 2 1.429% 99.286% ( 6600, 9900 ] 1 0.714% 100.000% ** DB Stats ** Uptime(secs): 26.1 total, 3.2 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 264.08 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.731 H:M:S, 25.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 308.00 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.456 H:M:S, 14.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/07-21:24:29 ... thread 0: (1000000,8000000) ops and (333386.0,277180.9) ops/second in (2.999526,28.862013) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/4 4.91 GB 471.2 7.6 0.0 7.6 15.1 7.6 0.0 2.0 145.1 289.9 53.41 48.94 134 0.399 7866K 14K 0.0 0.0 L1 2/0 181.69 MB 18.2 2.5 2.5 0.0 2.4 2.4 0.2 1.0 234.1 232.0 10.77 10.45 2 5.384 2559K 22K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 2.3 1.8 246.8 246.6 1.41 1.36 2 0.707 354K 190 0.0 0.0 L3 23/0 980.08 MB 1.0 3.7 2.3 1.4 3.7 2.3 0.2 1.6 200.1 199.7 18.98 16.49 10 1.898 3856K 8354 0.0 0.0 L4 184/0 9.70 GB 1.0 1.6 1.5 0.1 1.6 1.5 0.9 1.1 137.6 137.4 11.72 7.65 25 0.469 1570K 817 0.0 0.0 L5 1068/0 87.91 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1296/4 103.66 GB 0.0 15.7 6.4 9.2 23.2 13.9 5.9 3.1 166.4 246.4 96.28 84.89 173 0.557 16M 46K 0.0 0.0 Int 0/0 0.00 KB 0.0 15.7 6.4 9.2 23.2 13.9 5.9 3.1 166.4 246.4 96.28 84.89 173 0.557 16M 46K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 15.7 6.4 9.2 15.6 6.4 0.0 0.0 219.2 218.6 73.11 65.29 63 1.160 16M 46K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 334.4 23.17 19.60 110 0.211 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 29.1 total, 28.9 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 23.17 GB write, 815.70 MB/s write, 15.65 GB read, 550.97 MB/s read, 96.3 seconds Interval compaction: 23.17 GB write, 822.03 MB/s write, 15.65 GB read, 555.25 MB/s read, 96.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 34, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 34, interval: 47 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2696016 Average: 2.3255 StdDev: 13.34 Min: 0 Median: 1.5173 Max: 4258 Percentiles: P50: 1.52 P75: 1.91 P99: 5.89 P99.9: 11.69 P99.99: 98.96 ------------------------------------------------------ [ 0, 1 ] 449253 16.664% 16.664% ### ( 1, 2 ] 1737295 64.439% 81.103% ############# ( 2, 3 ] 297576 11.038% 92.141% ## ( 3, 4 ] 113639 4.215% 96.356% # ( 4, 6 ] 75538 2.802% 99.157% # ( 6, 10 ] 19033 0.706% 99.863% ( 10, 15 ] 2912 0.108% 99.971% ( 15, 22 ] 300 0.011% 99.983% ( 22, 34 ] 30 0.001% 99.984% ( 34, 51 ] 17 0.001% 99.984% ( 51, 76 ] 19 0.001% 99.985% ( 76, 110 ] 199 0.007% 99.992% ( 110, 170 ] 262 0.010% 100.002% ( 170, 250 ] 159 0.006% 100.008% ( 250, 380 ] 125 0.005% 100.013% ( 380, 580 ] 114 0.004% 100.017% ( 580, 870 ] 84 0.003% 100.020% ( 870, 1300 ] 51 0.002% 100.022% ( 1300, 1900 ] 21 0.001% 100.023% ( 1900, 2900 ] 21 0.001% 100.023% ( 2900, 4400 ] 12 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 599049 Average: 4.5148 StdDev: 56.79 Min: 0 Median: 1.9161 Max: 9219 Percentiles: P50: 1.92 P75: 2.84 P99: 9.20 P99.9: 148.13 P99.99: 357.02 ------------------------------------------------------ [ 0, 1 ] 13269 2.215% 2.215% ( 1, 2 ] 312479 52.163% 54.378% ########## ( 2, 3 ] 147159 24.565% 78.943% ##### ( 3, 4 ] 54942 9.172% 88.114% ## ( 4, 6 ] 54085 9.028% 97.143% ## ( 6, 10 ] 13914 2.323% 99.466% ( 10, 15 ] 2000 0.334% 99.800% ( 15, 22 ] 278 0.046% 99.846% ( 22, 34 ] 19 0.003% 99.849% ( 34, 51 ] 33 0.006% 99.855% ( 51, 76 ] 12 0.002% 99.857% ( 76, 110 ] 96 0.016% 99.873% ( 110, 170 ] 258 0.043% 99.916% ( 170, 250 ] 236 0.039% 99.955% ( 250, 380 ] 254 0.042% 99.997% ( 380, 580 ] 255 0.043% 100.040% ( 580, 870 ] 196 0.033% 100.073% ( 870, 1300 ] 85 0.014% 100.087% ( 1300, 1900 ] 42 0.007% 100.094% ( 1900, 2900 ] 63 0.011% 100.104% ( 2900, 4400 ] 38 0.006% 100.111% ( 4400, 6600 ] 13 0.002% 100.113% ( 6600, 9900 ] 6 0.001% 100.114% ** Level 2 read latency histogram (micros): Count: 88547 Average: 2.0673 StdDev: 3.89 Min: 1 Median: 1.4430 Max: 876 Percentiles: P50: 1.44 P75: 1.83 P99: 5.55 P99.9: 10.94 P99.99: 132.91 ------------------------------------------------------ [ 0, 1 ] 19169 21.648% 21.648% #### ( 1, 2 ] 56665 63.994% 85.643% ############# ( 2, 3 ] 7625 8.611% 94.254% ## ( 3, 4 ] 2935 3.315% 97.569% # ( 4, 6 ] 1637 1.849% 99.417% ( 6, 10 ] 412 0.465% 99.883% ( 10, 15 ] 82 0.093% 99.975% ( 15, 22 ] 9 0.010% 99.985% ( 22, 34 ] 2 0.002% 99.988% ( 34, 51 ] 1 0.001% 99.989% ( 110, 170 ] 3 0.003% 99.992% ( 170, 250 ] 2 0.002% 99.994% ( 250, 380 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 699417 Average: 4.0577 StdDev: 46.60 Min: 0 Median: 1.8243 Max: 8242 Percentiles: P50: 1.82 P75: 2.70 P99: 8.89 P99.9: 127.86 P99.99: 245.28 ------------------------------------------------------ [ 0, 1 ] 35304 5.048% 5.048% # ( 1, 2 ] 381420 54.534% 59.582% ########### ( 2, 3 ] 153458 21.941% 81.522% #### ( 3, 4 ] 58341 8.341% 89.864% ## ( 4, 6 ] 54899 7.849% 97.713% ## ( 6, 10 ] 12476 1.784% 99.497% ( 10, 15 ] 2057 0.294% 99.791% ( 15, 22 ] 284 0.041% 99.832% ( 22, 34 ] 43 0.006% 99.838% ( 34, 51 ] 15 0.002% 99.840% ( 51, 76 ] 16 0.002% 99.842% ( 76, 110 ] 259 0.037% 99.879% ( 110, 170 ] 489 0.070% 99.949% ( 170, 250 ] 304 0.043% 99.993% ( 250, 380 ] 261 0.037% 100.030% ( 380, 580 ] 272 0.039% 100.069% ( 580, 870 ] 182 0.026% 100.095% ( 870, 1300 ] 89 0.013% 100.108% ( 1300, 1900 ] 31 0.004% 100.112% ( 1900, 2900 ] 40 0.006% 100.118% ( 2900, 4400 ] 35 0.005% 100.123% ( 4400, 6600 ] 12 0.002% 100.124% ( 6600, 9900 ] 2 0.000% 100.125% ** Level 4 read latency histogram (micros): Count: 25685 Average: 73.6033 StdDev: 352.30 Min: 0 Median: 1.9766 Max: 7679 Percentiles: P50: 1.98 P75: 3.27 P99: 1677.95 P99.9: 4289.81 P99.99: 6207.46 ------------------------------------------------------ [ 0, 1 ] 653 2.542% 2.542% # ( 1, 2 ] 12481 48.593% 51.135% ########## ( 2, 3 ] 5430 21.141% 72.276% #### ( 3, 4 ] 2570 10.006% 82.281% ## ( 4, 6 ] 1620 6.307% 88.589% # ( 6, 10 ] 558 2.172% 90.761% ( 10, 15 ] 82 0.319% 91.080% ( 15, 22 ] 19 0.074% 91.154% ( 22, 34 ] 3 0.012% 91.166% ( 51, 76 ] 6 0.023% 91.189% ( 76, 110 ] 90 0.350% 91.540% ( 110, 170 ] 273 1.063% 92.603% ( 170, 250 ] 220 0.857% 93.459% ( 250, 380 ] 276 1.075% 94.534% ( 380, 580 ] 366 1.425% 95.959% ( 580, 870 ] 422 1.643% 97.602% ( 870, 1300 ] 276 1.075% 98.676% ( 1300, 1900 ] 132 0.514% 99.190% ( 1900, 2900 ] 98 0.382% 99.572% ( 2900, 4400 ] 91 0.354% 99.926% ( 4400, 6600 ] 20 0.078% 100.004% ( 6600, 9900 ] 3 0.012% 100.016% ** Level 5 read latency histogram (micros): Count: 140 Average: 1280.3429 StdDev: 1315.04 Min: 84 Median: 736.1538 Max: 7391 Percentiles: P50: 736.15 P75: 1955.56 P99: 6160.00 P99.9: 7391.00 P99.99: 7391.00 ------------------------------------------------------ ( 76, 110 ] 6 4.286% 4.286% # ( 110, 170 ] 11 7.857% 12.143% ## ( 170, 250 ] 16 11.429% 23.571% ## ( 250, 380 ] 13 9.286% 32.857% ## ( 380, 580 ] 17 12.143% 45.000% ## ( 580, 870 ] 13 9.286% 54.286% ## ( 870, 1300 ] 14 10.000% 64.286% ## ( 1300, 1900 ] 14 10.000% 74.286% ## ( 1900, 2900 ] 18 12.857% 87.143% ### ( 2900, 4400 ] 15 10.714% 97.857% ## ( 4400, 6600 ] 2 1.429% 99.286% ( 6600, 9900 ] 1 0.714% 100.000% ** DB Stats ** Uptime(secs): 29.1 total, 3.0 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 270.68 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.098 H:M:S, 24.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 328.12 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.367 H:M:S, 12.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/07-21:24:59 ... thread 0: (1000000,9000000) ops and (33646.1,153627.8) ops/second in (29.721132,58.583145) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 4/3 629.45 MB 6.3 8.6 0.0 8.6 17.1 8.5 0.0 2.0 145.4 288.5 60.64 55.30 152 0.399 8953K 15K 0.0 0.0 L1 5/5 517.20 MB 0.0 7.9 7.7 0.2 7.7 7.5 0.2 1.0 210.8 206.5 38.16 34.27 3 12.722 8168K 162K 0.0 0.0 L2 22/9 2.28 GB 15.6 0.3 0.2 0.2 0.3 0.2 7.1 1.8 246.8 246.6 1.41 1.36 2 0.707 354K 190 0.0 0.0 L3 45/9 3.43 GB 3.2 3.7 2.3 1.4 3.7 2.3 2.7 1.6 200.1 199.7 18.98 16.49 10 1.898 3856K 8354 0.0 0.0 L4 184/0 9.70 GB 1.0 1.6 1.5 0.1 1.6 1.5 0.9 1.1 137.6 137.4 11.72 7.65 25 0.469 1570K 817 0.0 0.0 L5 1068/0 87.91 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1328/26 104.44 GB 0.0 22.1 11.6 10.5 30.4 19.9 13.2 3.6 172.8 237.7 130.91 115.07 192 0.682 22M 187K 0.0 0.0 Int 0/0 0.00 KB 0.0 22.1 11.6 10.5 30.4 19.9 13.2 3.6 172.8 237.7 130.91 115.07 192 0.682 22M 187K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 22.1 11.6 10.5 21.9 11.4 0.0 0.0 216.5 214.7 104.48 93.02 68 1.537 22M 187K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.5 8.5 0.0 0.0 0.0 328.9 26.43 22.05 124 0.213 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 58.8 total, 58.6 interval Flush(GB): cumulative 8.490, interval 8.490 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 30.40 GB write, 529.26 MB/s write, 22.09 GB read, 384.65 MB/s read, 130.9 seconds Interval compaction: 30.40 GB write, 531.28 MB/s write, 22.09 GB read, 386.12 MB/s read, 130.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 23, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 23, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 36, pending-compaction-bytes-delays: 6, pending-compaction-bytes-stops: 0, total-delays: 29, total-stops: 36, interval: 65 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 4305301 Average: 2.9889 StdDev: 21.20 Min: 0 Median: 1.4542 Max: 7533 Percentiles: P50: 1.45 P75: 1.85 P99: 8.02 P99.9: 160.36 P99.99: 396.57 ------------------------------------------------------ [ 0, 1 ] 916467 21.287% 21.287% #### ( 1, 2 ] 2721584 63.215% 84.502% ############# ( 2, 3 ] 360201 8.366% 92.868% ## ( 3, 4 ] 149169 3.465% 96.333% # ( 4, 6 ] 101943 2.368% 98.701% ( 6, 10 ] 25560 0.594% 99.294% ( 10, 15 ] 4618 0.107% 99.402% ( 15, 22 ] 757 0.018% 99.419% ( 22, 34 ] 41 0.001% 99.420% ( 34, 51 ] 25 0.001% 99.421% ( 51, 76 ] 170 0.004% 99.425% ( 76, 110 ] 12723 0.296% 99.720% ( 110, 170 ] 9218 0.214% 99.934% ( 170, 250 ] 1345 0.031% 99.966% ( 250, 380 ] 990 0.023% 99.989% ( 380, 580 ] 718 0.017% 100.005% ( 580, 870 ] 274 0.006% 100.012% ( 870, 1300 ] 86 0.002% 100.014% ( 1300, 1900 ] 34 0.001% 100.014% ( 1900, 2900 ] 37 0.001% 100.015% ( 2900, 4400 ] 49 0.001% 100.016% ( 4400, 6600 ] 3 0.000% 100.017% ( 6600, 9900 ] 1 0.000% 100.017% ** Level 1 read latency histogram (micros): Count: 838571 Average: 5.7122 StdDev: 91.90 Min: 0 Median: 1.9197 Max: 9219 Percentiles: P50: 1.92 P75: 2.88 P99: 9.49 P99.9: 189.87 P99.99: 474.83 ------------------------------------------------------ [ 0, 1 ] 30719 3.663% 3.663% # ( 1, 2 ] 422510 50.385% 54.048% ########## ( 2, 3 ] 199695 23.814% 77.862% ##### ( 3, 4 ] 80273 9.573% 87.434% ## ( 4, 6 ] 80391 9.587% 97.021% ## ( 6, 10 ] 19047 2.271% 99.292% ( 10, 15 ] 3097 0.369% 99.661% ( 15, 22 ] 395 0.047% 99.709% ( 22, 34 ] 26 0.003% 99.712% ( 34, 51 ] 34 0.004% 99.716% ( 51, 76 ] 44 0.005% 99.721% ( 76, 110 ] 641 0.076% 99.797% ( 110, 170 ] 774 0.092% 99.890% ( 170, 250 ] 356 0.042% 99.932% ( 250, 380 ] 334 0.040% 99.972% ( 380, 580 ] 323 0.039% 100.010% ( 580, 870 ] 259 0.031% 100.041% ( 870, 1300 ] 119 0.014% 100.056% ( 1300, 1900 ] 75 0.009% 100.065% ( 1900, 2900 ] 96 0.011% 100.076% ( 2900, 4400 ] 180 0.021% 100.097% ( 4400, 6600 ] 69 0.008% 100.106% ( 6600, 9900 ] 29 0.003% 100.109% ** Level 2 read latency histogram (micros): Count: 88547 Average: 2.0673 StdDev: 3.89 Min: 1 Median: 1.4430 Max: 876 Percentiles: P50: 1.44 P75: 1.83 P99: 5.55 P99.9: 10.94 P99.99: 132.91 ------------------------------------------------------ [ 0, 1 ] 19169 21.648% 21.648% #### ( 1, 2 ] 56665 63.994% 85.643% ############# ( 2, 3 ] 7625 8.611% 94.254% ## ( 3, 4 ] 2935 3.315% 97.569% # ( 4, 6 ] 1637 1.849% 99.417% ( 6, 10 ] 412 0.465% 99.883% ( 10, 15 ] 82 0.093% 99.975% ( 15, 22 ] 9 0.010% 99.985% ( 22, 34 ] 2 0.002% 99.988% ( 34, 51 ] 1 0.001% 99.989% ( 110, 170 ] 3 0.003% 99.992% ( 170, 250 ] 2 0.002% 99.994% ( 250, 380 ] 4 0.005% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 781518 Average: 4.4917 StdDev: 58.27 Min: 0 Median: 1.8476 Max: 8671 Percentiles: P50: 1.85 P75: 2.75 P99: 9.14 P99.9: 159.17 P99.99: 310.89 ------------------------------------------------------ [ 0, 1 ] 36392 4.657% 4.657% # ( 1, 2 ] 418075 53.495% 58.152% ########### ( 2, 3 ] 174400 22.316% 80.467% #### ( 3, 4 ] 67877 8.685% 89.153% ## ( 4, 6 ] 65124 8.333% 97.486% ## ( 6, 10 ] 15083 1.930% 99.416% ( 10, 15 ] 2526 0.323% 99.739% ( 15, 22 ] 345 0.044% 99.783% ( 22, 34 ] 43 0.006% 99.788% ( 34, 51 ] 15 0.002% 99.790% ( 51, 76 ] 19 0.002% 99.793% ( 76, 110 ] 281 0.036% 99.829% ( 110, 170 ] 679 0.087% 99.916% ( 170, 250 ] 423 0.054% 99.970% ( 250, 380 ] 337 0.043% 100.013% ( 380, 580 ] 340 0.044% 100.056% ( 580, 870 ] 222 0.028% 100.085% ( 870, 1300 ] 110 0.014% 100.099% ( 1300, 1900 ] 46 0.006% 100.105% ( 1900, 2900 ] 51 0.007% 100.111% ( 2900, 4400 ] 65 0.008% 100.120% ( 4400, 6600 ] 29 0.004% 100.123% ( 6600, 9900 ] 3 0.000% 100.124% ** Level 4 read latency histogram (micros): Count: 25685 Average: 73.6033 StdDev: 352.30 Min: 0 Median: 1.9766 Max: 7679 Percentiles: P50: 1.98 P75: 3.27 P99: 1677.95 P99.9: 4289.81 P99.99: 6207.46 ------------------------------------------------------ [ 0, 1 ] 653 2.542% 2.542% # ( 1, 2 ] 12481 48.593% 51.135% ########## ( 2, 3 ] 5430 21.141% 72.276% #### ( 3, 4 ] 2570 10.006% 82.281% ## ( 4, 6 ] 1620 6.307% 88.589% # ( 6, 10 ] 558 2.172% 90.761% ( 10, 15 ] 82 0.319% 91.080% ( 15, 22 ] 19 0.074% 91.154% ( 22, 34 ] 3 0.012% 91.166% ( 51, 76 ] 6 0.023% 91.189% ( 76, 110 ] 90 0.350% 91.540% ( 110, 170 ] 273 1.063% 92.603% ( 170, 250 ] 220 0.857% 93.459% ( 250, 380 ] 276 1.075% 94.534% ( 380, 580 ] 366 1.425% 95.959% ( 580, 870 ] 422 1.643% 97.602% ( 870, 1300 ] 276 1.075% 98.676% ( 1300, 1900 ] 132 0.514% 99.190% ( 1900, 2900 ] 98 0.382% 99.572% ( 2900, 4400 ] 91 0.354% 99.926% ( 4400, 6600 ] 20 0.078% 100.004% ( 6600, 9900 ] 3 0.012% 100.016% ** Level 5 read latency histogram (micros): Count: 140 Average: 1280.3429 StdDev: 1315.04 Min: 84 Median: 736.1538 Max: 7391 Percentiles: P50: 736.15 P75: 1955.56 P99: 6160.00 P99.9: 7391.00 P99.99: 7391.00 ------------------------------------------------------ ( 76, 110 ] 6 4.286% 4.286% # ( 110, 170 ] 11 7.857% 12.143% ## ( 170, 250 ] 16 11.429% 23.571% ## ( 250, 380 ] 13 9.286% 32.857% ## ( 380, 580 ] 17 12.143% 45.000% ## ( 580, 870 ] 13 9.286% 54.286% ## ( 870, 1300 ] 14 10.000% 64.286% ## ( 1300, 1900 ] 14 10.000% 74.286% ## ( 1900, 2900 ] 18 12.857% 87.143% ### ( 2900, 4400 ] 15 10.714% 97.857% ## ( 4400, 6600 ] 2 1.429% 99.286% ( 6600, 9900 ] 1 0.714% 100.000% ** DB Stats ** Uptime(secs): 58.8 total, 29.7 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 150.62 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:33.104 H:M:S, 56.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 33.11 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:26.006 H:M:S, 87.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 6.570 micros/op 152195 ops/sec 60.126 seconds 9150999 operations; 147.5 MB/s Microseconds per write: Count: 9150999 Average: 6.5705 StdDev: 476.97 Min: 0 Median: 1.7713 Max: 971104 Percentiles: P50: 1.77 P75: 2.82 P99: 9.92 P99.9: 1221.29 P99.99: 2761.33 ------------------------------------------------------ [ 0, 1 ] 1050409 11.479% 11.479% ## ( 1, 2 ] 4570099 49.941% 61.420% ########## ( 2, 3 ] 1509786 16.499% 77.918% ### ( 3, 4 ] 1276614 13.951% 91.869% ### ( 4, 6 ] 524031 5.726% 97.595% # ( 6, 10 ] 131074 1.432% 99.028% ( 10, 15 ] 45845 0.501% 99.529% ( 15, 22 ] 10081 0.110% 99.639% ( 22, 34 ] 1968 0.022% 99.660% ( 34, 51 ] 267 0.003% 99.663% ( 51, 76 ] 26 0.000% 99.663% ( 76, 110 ] 6319 0.069% 99.732% ( 110, 170 ] 2478 0.027% 99.760% ( 170, 250 ] 275 0.003% 99.763% ( 250, 380 ] 201 0.002% 99.765% ( 380, 580 ] 31 0.000% 99.765% ( 580, 870 ] 16 0.000% 99.765% ( 870, 1300 ] 15090 0.165% 99.930% ( 1300, 1900 ] 32 0.000% 99.931% ( 1900, 2900 ] 6318 0.069% 100.000% ( 2900, 4400 ] 13 0.000% 100.000% ( 4400, 6600 ] 5 0.000% 100.000% ( 14000, 22000 ] 1 0.000% 100.000% ( 22000, 33000 ] 2 0.000% 100.000% ( 33000, 50000 ] 1 0.000% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% ( 170000, 250000 ] 11 0.000% 100.000% ( 250000, 380000 ] 1 0.000% 100.000% ( 380000, 570000 ] 1 0.000% 100.000% ( 570000, 860000 ] 1 0.000% 100.000% ( 860000, 1200000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 226532 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 9150999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 9443830968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 490 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 34205402 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 26803268109 rocksdb.compact.write.bytes COUNT : 26565258886 rocksdb.flush.write.bytes COUNT : 9314008673 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 36 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 263 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 9239033856 rocksdb.memtable.garbage.bytes.at.flush COUNT : 3718144 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 26884876335 rocksdb.non.last.level.read.count COUNT : 6494818 rocksdb.block.checksum.compute.count COUNT : 6493874 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.414936 P95 : 3.967098 P99 : 8.660187 P100 : 971100.000000 COUNT : 9150999 SUM : 55527943 rocksdb.compaction.times.micros P50 : 985000.000000 P95 : 3810000.000000 P99 : 24090000.000000 P100 : 27397277.000000 COUNT : 81 SUM : 119032566 rocksdb.compaction.times.cpu_micros P50 : 847916.666667 P95 : 3565000.000000 P99 : 23818471.000000 P100 : 23818471.000000 COUNT : 81 SUM : 100673477 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 698.289474 P95 : 8910.000000 P99 : 71750.000000 P100 : 95115.000000 COUNT : 126 SUM : 378795 rocksdb.compaction.outfile.sync.micros P50 : 1004.025974 P95 : 158000.000000 P99 : 248133.333333 P100 : 345121.000000 COUNT : 328 SUM : 6036185 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 114.736842 P95 : 841.000000 P99 : 3912.500000 P100 : 5226.000000 COUNT : 230 SUM : 67342 rocksdb.table.open.io.micros P50 : 1246.250000 P95 : 7245.652174 P99 : 10802.000000 P100 : 15580.000000 COUNT : 490 SUM : 945138 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.614883 P95 : 7.931872 P99 : 12.699953 P100 : 11585.000000 COUNT : 6492678 SUM : 40572848 rocksdb.read.block.get.micros P50 : 374.090909 P95 : 3104.545455 P99 : 4664.000000 P100 : 7465.000000 COUNT : 980 SUM : 769661 rocksdb.write.raw.block.micros P50 : 0.515915 P95 : 0.980239 P99 : 3.041955 P100 : 19602.000000 COUNT : 8692704 SUM : 18203229 rocksdb.numfiles.in.singlecompaction P50 : 3.034483 P95 : 5.250000 P99 : 14.250000 P100 : 19.000000 COUNT : 130 SUM : 451 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.574572 P95 : 1243.921154 P99 : 2640.477475 P100 : 970908.000000 COUNT : 166103 SUM : 34225256 rocksdb.sst.read.micros P50 : 1.584155 P95 : 4.706873 P99 : 9.209047 P100 : 10515.000000 COUNT : 6495335 SUM : 27270816 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 9150999 SUM : 9443830968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 213146.067416 P95 : 370058.823529 P99 : 834866.666667 P100 : 1103474.000000 COUNT : 126 SUM : 28008056 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 291 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0