[2024-11-27 08:57:15.457185] Starting SPDK v25.01-pre git sha1 2f2acf4eb / DPDK 23.11.0 initialization... [2024-11-27 08:57:15.457299] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1024867 ] Set seed to 1732694235454780 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-27 08:57:15.561270] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-27 08:57:15.584442] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-27 08:57:15.584487] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1024867' to capture a snapshot of events at runtime. [2024-11-27 08:57:15.584497] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-27 08:57:15.584508] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-27 08:57:15.584516] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1024867 for offline analysis/debug. [2024-11-27 08:57:15.585059] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Nov 27 08:57:19 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/27-08:57:22 ... thread 0: (1000000,1000000) ops and (334672.1,334672.1) ops/second in (2.987999,2.987999) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/9 693.01 MB 12.6 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 349.7 2.52 2.27 12 0.210 0 0 0.0 0.0 L2 3/2 237.01 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.0 1.8 249.0 248.9 1.40 1.35 2 0.700 353K 182 0.0 0.0 L3 8/0 692.92 MB 0.7 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1181/11 97.00 GB 0.0 0.3 0.2 0.2 1.2 1.0 0.0 1.4 88.9 313.7 3.92 3.61 14 0.280 353K 182 0.0 0.0 Int 0/0 0.00 KB 0.0 0.3 0.2 0.2 1.2 1.0 0.0 1.4 88.9 313.7 3.92 3.61 14 0.280 353K 182 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 0.0 0.0 249.0 248.9 1.40 1.35 2 0.700 353K 182 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 349.7 2.52 2.27 12 0.210 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.2 total, 3.0 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.20 GB write, 385.51 MB/s write, 0.34 GB read, 109.18 MB/s read, 3.9 seconds Interval compaction: 1.20 GB write, 411.52 MB/s write, 0.34 GB read, 116.55 MB/s read, 3.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 3, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 3, interval: 3 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 146012 Average: 2.0572 StdDev: 13.94 Min: 1 Median: 1.3808 Max: 3216 Percentiles: P50: 1.38 P75: 1.79 P99: 5.42 P99.9: 10.25 P99.99: 14.87 ------------------------------------------------------ [ 0, 1 ] 39188 26.839% 26.839% ##### ( 1, 2 ] 88809 60.823% 87.662% ############ ( 2, 3 ] 10451 7.158% 94.820% # ( 3, 4 ] 4228 2.896% 97.715% # ( 4, 6 ] 2652 1.816% 99.532% ( 6, 10 ] 533 0.365% 99.897% ( 10, 15 ] 142 0.097% 99.994% ( 15, 22 ] 11 0.008% 100.001% ( 22, 34 ] 2 0.001% 100.003% ( 34, 51 ] 1 0.001% 100.003% ( 51, 76 ] 1 0.001% 100.004% ( 110, 170 ] 5 0.003% 100.008% ( 170, 250 ] 6 0.004% 100.012% ( 250, 380 ] 5 0.003% 100.015% ( 380, 580 ] 2 0.001% 100.016% ( 580, 870 ] 1 0.001% 100.017% ( 870, 1300 ] 2 0.001% 100.018% ( 1300, 1900 ] 1 0.001% 100.019% ( 2900, 4400 ] 2 0.001% 100.021% ** Level 1 read latency histogram (micros): Count: 12255 Average: 3.0406 StdDev: 35.41 Min: 1 Median: 1.2768 Max: 3003 Percentiles: P50: 1.28 P75: 1.74 P99: 9.32 P99.9: 247.73 P99.99: 1764.70 ------------------------------------------------------ [ 0, 1 ] 4307 35.145% 35.145% ####### ( 1, 2 ] 6578 53.676% 88.821% ########### ( 2, 3 ] 841 6.863% 95.683% # ( 3, 4 ] 140 1.142% 96.826% ( 4, 6 ] 176 1.436% 98.262% ( 6, 10 ] 109 0.889% 99.151% ( 10, 15 ] 27 0.220% 99.372% ( 15, 22 ] 6 0.049% 99.421% ( 22, 34 ] 2 0.016% 99.437% ( 34, 51 ] 17 0.139% 99.576% ( 51, 76 ] 25 0.204% 99.780% ( 76, 110 ] 2 0.016% 99.796% ( 110, 170 ] 4 0.033% 99.829% ( 170, 250 ] 9 0.073% 99.902% ( 250, 380 ] 5 0.041% 99.943% ( 380, 580 ] 3 0.024% 99.967% ( 580, 870 ] 1 0.008% 99.976% ( 870, 1300 ] 1 0.008% 99.984% ( 1300, 1900 ] 1 0.008% 99.992% ( 2900, 4400 ] 1 0.008% 100.000% ** Level 2 read latency histogram (micros): Count: 70998 Average: 2.0059 StdDev: 5.25 Min: 0 Median: 1.3868 Max: 951 Percentiles: P50: 1.39 P75: 1.79 P99: 5.49 P99.9: 9.85 P99.99: 206.01 ------------------------------------------------------ [ 0, 1 ] 18331 25.819% 25.819% ##### ( 1, 2 ] 44389 62.521% 88.341% ############# ( 2, 3 ] 4548 6.406% 94.746% # ( 3, 4 ] 2066 2.910% 97.656% # ( 4, 6 ] 1281 1.804% 99.461% ( 6, 10 ] 325 0.458% 99.918% ( 10, 15 ] 41 0.058% 99.976% ( 15, 22 ] 6 0.008% 99.985% ( 22, 34 ] 1 0.001% 99.986% ( 51, 76 ] 1 0.001% 99.987% ( 110, 170 ] 2 0.003% 99.990% ( 170, 250 ] 2 0.003% 99.993% ( 250, 380 ] 4 0.006% 99.999% ( 580, 870 ] 1 0.001% 100.000% ( 870, 1300 ] 1 0.001% 100.001% ** Level 3 read latency histogram (micros): Count: 40 Average: 1674.9750 StdDev: 1240.45 Min: 197 Median: 1420.0000 Max: 4922 Percentiles: P50: 1420.00 P75: 2650.00 P99: 4922.00 P99.9: 4922.00 P99.99: 4922.00 ------------------------------------------------------ ( 170, 250 ] 5 12.500% 12.500% ### ( 250, 380 ] 3 7.500% 20.000% ## ( 380, 580 ] 1 2.500% 22.500% # ( 580, 870 ] 7 17.500% 40.000% #### ( 870, 1300 ] 3 7.500% 47.500% ## ( 1300, 1900 ] 5 12.500% 60.000% ### ( 1900, 2900 ] 8 20.000% 80.000% #### ( 2900, 4400 ] 7 17.500% 97.500% #### ( 4400, 6600 ] 1 2.500% 100.000% # ** Level 4 read latency histogram (micros): Count: 35 Average: 1665.3429 StdDev: 1401.13 Min: 190 Median: 1120.8333 Max: 5297 Percentiles: P50: 1120.83 P75: 2650.00 P99: 5297.00 P99.9: 5297.00 P99.99: 5297.00 ------------------------------------------------------ ( 170, 250 ] 1 2.857% 2.857% # ( 250, 380 ] 4 11.429% 14.286% ## ( 380, 580 ] 4 11.429% 25.714% ## ( 580, 870 ] 5 14.286% 40.000% ### ( 870, 1300 ] 6 17.143% 57.143% ### ( 1300, 1900 ] 4 11.429% 68.571% ## ( 1900, 2900 ] 3 8.571% 77.143% ## ( 2900, 4400 ] 5 14.286% 91.429% ### ( 4400, 6600 ] 3 8.571% 100.000% ## ** DB Stats ** Uptime(secs): 3.2 total, 3.0 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 308.40 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.423 H:M:S, 13.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 329.12 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.423 H:M:S, 14.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/27-08:57:25 ... thread 0: (1000000,2000000) ops and (330077.9,332359.1) ops/second in (3.029588,6.017587) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/9 1.35 GB 56.7 0.5 0.0 0.5 2.3 1.8 0.0 1.3 69.1 328.1 7.29 6.70 28 0.261 511K 573 0.0 0.0 L1 4/4 314.58 MB 0.0 0.3 0.3 0.0 0.3 0.3 0.2 1.0 239.7 239.4 1.31 1.28 1 1.314 319K 430 0.0 0.0 L2 1/0 49.57 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.0 1.8 249.0 248.9 1.40 1.35 2 0.700 353K 182 0.0 0.0 L3 17/0 880.36 MB 0.9 0.2 0.2 0.0 0.2 0.2 0.0 1.0 228.3 228.3 0.82 0.76 1 0.821 190K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1195/13 97.98 GB 0.0 1.3 0.7 0.6 3.2 2.5 0.2 1.7 125.1 299.5 10.83 10.08 32 0.338 1375K 1185 0.0 0.0 Int 0/0 0.00 KB 0.0 1.3 0.7 0.6 3.2 2.5 0.2 1.7 125.1 299.5 10.83 10.08 32 0.338 1375K 1185 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.3 0.7 0.6 1.3 0.7 0.0 0.0 249.3 249.0 5.44 5.25 6 0.906 1375K 1185 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 350.4 5.39 4.83 26 0.207 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.2 total, 6.0 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.17 GB write, 521.42 MB/s write, 1.32 GB read, 217.81 MB/s read, 10.8 seconds Interval compaction: 3.17 GB write, 538.89 MB/s write, 1.32 GB read, 225.11 MB/s read, 10.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 6, interval: 6 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 397367 Average: 2.0774 StdDev: 13.59 Min: 0 Median: 1.4174 Max: 3259 Percentiles: P50: 1.42 P75: 1.81 P99: 5.19 P99.9: 10.01 P99.99: 14.60 ------------------------------------------------------ [ 0, 1 ] 91858 23.117% 23.117% ##### ( 1, 2 ] 255955 64.413% 87.529% ############# ( 2, 3 ] 31652 7.965% 95.495% ## ( 3, 4 ] 10606 2.669% 98.164% # ( 4, 6 ] 5593 1.408% 99.571% ( 6, 10 ] 1306 0.329% 99.900% ( 10, 15 ] 389 0.098% 99.998% ( 15, 22 ] 25 0.006% 100.004% ( 22, 34 ] 7 0.002% 100.006% ( 34, 51 ] 2 0.001% 100.007% ( 51, 76 ] 3 0.001% 100.007% ( 110, 170 ] 10 0.003% 100.010% ( 170, 250 ] 11 0.003% 100.013% ( 250, 380 ] 11 0.003% 100.015% ( 380, 580 ] 7 0.002% 100.017% ( 580, 870 ] 6 0.002% 100.019% ( 870, 1300 ] 3 0.001% 100.019% ( 1300, 1900 ] 1 0.000% 100.020% ( 1900, 2900 ] 2 0.001% 100.020% ( 2900, 4400 ] 5 0.001% 100.021% ** Level 1 read latency histogram (micros): Count: 65986 Average: 2.2034 StdDev: 16.50 Min: 1 Median: 1.3964 Max: 3003 Percentiles: P50: 1.40 P75: 1.79 P99: 5.24 P99.9: 43.01 P99.99: 500.07 ------------------------------------------------------ [ 0, 1 ] 16324 24.739% 24.739% ##### ( 1, 2 ] 42053 63.730% 88.469% ############# ( 2, 3 ] 5208 7.893% 96.361% ## ( 3, 4 ] 1301 1.972% 98.333% ( 4, 6 ] 709 1.074% 99.407% ( 6, 10 ] 243 0.368% 99.776% ( 10, 15 ] 56 0.085% 99.861% ( 15, 22 ] 13 0.020% 99.880% ( 22, 34 ] 4 0.006% 99.886% ( 34, 51 ] 17 0.026% 99.912% ( 51, 76 ] 25 0.038% 99.950% ( 76, 110 ] 2 0.003% 99.953% ( 110, 170 ] 4 0.006% 99.959% ( 170, 250 ] 9 0.014% 99.973% ( 250, 380 ] 9 0.014% 99.986% ( 380, 580 ] 4 0.006% 99.992% ( 580, 870 ] 1 0.002% 99.994% ( 870, 1300 ] 1 0.002% 99.995% ( 1300, 1900 ] 2 0.003% 99.998% ( 2900, 4400 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 75843 Average: 2.0334 StdDev: 5.09 Min: 0 Median: 1.4064 Max: 951 Percentiles: P50: 1.41 P75: 1.81 P99: 5.56 P99.9: 9.87 P99.99: 186.63 ------------------------------------------------------ [ 0, 1 ] 18599 24.523% 24.523% ##### ( 1, 2 ] 47542 62.685% 87.208% ############# ( 2, 3 ] 5473 7.216% 94.424% # ( 3, 4 ] 2327 3.068% 97.492% # ( 4, 6 ] 1469 1.937% 99.429% ( 6, 10 ] 369 0.487% 99.916% ( 10, 15 ] 45 0.059% 99.975% ( 15, 22 ] 7 0.009% 99.984% ( 22, 34 ] 1 0.001% 99.985% ( 51, 76 ] 1 0.001% 99.987% ( 110, 170 ] 2 0.003% 99.989% ( 170, 250 ] 2 0.003% 99.992% ( 250, 380 ] 4 0.005% 99.997% ( 580, 870 ] 1 0.001% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 85 Average: 810.6000 StdDev: 1186.17 Min: 1 Median: 140.0000 Max: 4922 Percentiles: P50: 140.00 P75: 1273.12 P99: 4730.00 P99.9: 4922.00 P99.99: 4922.00 ------------------------------------------------------ [ 0, 1 ] 9 10.588% 10.588% ## ( 1, 2 ] 26 30.588% 41.176% ###### ( 4, 6 ] 4 4.706% 45.882% # ( 6, 10 ] 2 2.353% 48.235% ( 34, 51 ] 1 1.176% 49.412% ( 110, 170 ] 1 1.176% 50.588% ( 170, 250 ] 5 5.882% 56.471% # ( 250, 380 ] 3 3.529% 60.000% # ( 380, 580 ] 2 2.353% 62.353% ( 580, 870 ] 7 8.235% 70.588% ## ( 870, 1300 ] 4 4.706% 75.294% # ( 1300, 1900 ] 5 5.882% 81.176% # ( 1900, 2900 ] 8 9.412% 90.588% ## ( 2900, 4400 ] 7 8.235% 98.824% ## ( 4400, 6600 ] 1 1.176% 100.000% ** Level 4 read latency histogram (micros): Count: 35 Average: 1665.3429 StdDev: 1401.13 Min: 190 Median: 1120.8333 Max: 5297 Percentiles: P50: 1120.83 P75: 2650.00 P99: 5297.00 P99.9: 5297.00 P99.99: 5297.00 ------------------------------------------------------ ( 170, 250 ] 1 2.857% 2.857% # ( 250, 380 ] 4 11.429% 14.286% ## ( 380, 580 ] 4 11.429% 25.714% ## ( 580, 870 ] 5 14.286% 40.000% ### ( 870, 1300 ] 6 17.143% 57.143% ### ( 1300, 1900 ] 4 11.429% 68.571% ## ( 1900, 2900 ] 3 8.571% 77.143% ## ( 2900, 4400 ] 5 14.286% 91.429% ### ( 4400, 6600 ] 3 8.571% 100.000% ## ** DB Stats ** Uptime(secs): 6.2 total, 3.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 316.42 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.878 H:M:S, 14.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 324.87 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.455 H:M:S, 15.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/27-08:57:28 ... thread 0: (1000000,3000000) ops and (350937.5,338329.4) ops/second in (2.849510,8.867097) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/12 1.66 GB 0.0 1.3 0.0 1.3 4.0 2.7 0.0 1.5 102.6 317.6 12.89 11.93 44 0.293 1343K 1449 0.0 0.0 L1 4/4 363.05 MB 0.0 1.2 0.9 0.3 1.2 0.9 0.2 1.4 235.0 234.3 5.09 4.96 2 2.545 1214K 3423 0.0 0.0 L2 5/4 564.65 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.6 1.8 249.0 248.9 1.40 1.35 2 0.700 353K 182 0.0 0.0 L3 17/1 880.36 MB 0.8 0.2 0.2 0.0 0.2 0.2 0.0 1.0 228.3 228.3 0.82 0.76 1 0.821 190K 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.88 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1198/21 98.84 GB 0.0 3.0 1.2 1.8 5.7 3.9 0.7 2.1 151.2 288.2 20.20 19.00 49 0.412 3102K 5054 0.0 0.0 Int 0/0 0.00 KB 0.0 3.0 1.2 1.8 5.7 3.9 0.7 2.1 151.2 288.2 20.20 19.00 49 0.412 3102K 5054 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.0 1.2 1.8 3.0 1.2 0.0 0.0 248.3 247.9 12.30 11.96 10 1.230 3102K 5054 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.7 2.7 0.0 0.0 0.0 351.0 7.90 7.05 39 0.202 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.1 total, 8.9 interval Flush(GB): cumulative 2.707, interval 2.707 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.69 GB write, 641.85 MB/s write, 2.98 GB read, 336.83 MB/s read, 20.2 seconds Interval compaction: 5.69 GB write, 656.45 MB/s write, 2.98 GB read, 344.49 MB/s read, 20.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 9, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 9, interval: 9 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 659091 Average: 2.1225 StdDev: 11.74 Min: 0 Median: 1.4554 Max: 3259 Percentiles: P50: 1.46 P75: 1.84 P99: 5.37 P99.9: 10.34 P99.99: 14.57 ------------------------------------------------------ [ 0, 1 ] 131970 20.023% 20.023% #### ( 1, 2 ] 433871 65.829% 85.852% ############# ( 2, 3 ] 59458 9.021% 94.873% ## ( 3, 4 ] 19991 3.033% 97.906% # ( 4, 6 ] 10533 1.598% 99.504% ( 6, 10 ] 2564 0.389% 99.893% ( 10, 15 ] 700 0.106% 99.999% ( 15, 22 ] 38 0.006% 100.005% ( 22, 34 ] 13 0.002% 100.007% ( 34, 51 ] 3 0.000% 100.008% ( 51, 76 ] 4 0.001% 100.008% ( 110, 170 ] 16 0.002% 100.011% ( 170, 250 ] 19 0.003% 100.014% ( 250, 380 ] 17 0.003% 100.016% ( 380, 580 ] 8 0.001% 100.017% ( 580, 870 ] 11 0.002% 100.019% ( 870, 1300 ] 5 0.001% 100.020% ( 1300, 1900 ] 4 0.001% 100.020% ( 1900, 2900 ] 3 0.000% 100.021% ( 2900, 4400 ] 5 0.001% 100.022% ** Level 1 read latency histogram (micros): Count: 162262 Average: 2.2640 StdDev: 14.08 Min: 1 Median: 1.4976 Max: 3003 Percentiles: P50: 1.50 P75: 1.87 P99: 5.43 P99.9: 13.59 P99.99: 405.79 ------------------------------------------------------ [ 0, 1 ] 27158 16.737% 16.737% ### ( 1, 2 ] 108467 66.847% 83.584% ############# ( 2, 3 ] 18521 11.414% 94.998% ## ( 3, 4 ] 4652 2.867% 97.865% # ( 4, 6 ] 2582 1.591% 99.456% ( 6, 10 ] 605 0.373% 99.829% ( 10, 15 ] 161 0.099% 99.929% ( 15, 22 ] 21 0.013% 99.941% ( 22, 34 ] 7 0.004% 99.946% ( 34, 51 ] 17 0.010% 99.956% ( 51, 76 ] 25 0.015% 99.972% ( 76, 110 ] 2 0.001% 99.973% ( 110, 170 ] 4 0.002% 99.975% ( 170, 250 ] 9 0.006% 99.981% ( 250, 380 ] 15 0.009% 99.990% ( 380, 580 ] 6 0.004% 99.994% ( 580, 870 ] 4 0.002% 99.996% ( 870, 1300 ] 1 0.001% 99.997% ( 1300, 1900 ] 4 0.002% 99.999% ( 1900, 2900 ] 1 0.001% 100.000% ( 2900, 4400 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 75843 Average: 2.0334 StdDev: 5.09 Min: 0 Median: 1.4064 Max: 951 Percentiles: P50: 1.41 P75: 1.81 P99: 5.56 P99.9: 9.87 P99.99: 186.63 ------------------------------------------------------ [ 0, 1 ] 18599 24.523% 24.523% ##### ( 1, 2 ] 47542 62.685% 87.208% ############# ( 2, 3 ] 5473 7.216% 94.424% # ( 3, 4 ] 2327 3.068% 97.492% # ( 4, 6 ] 1469 1.937% 99.429% ( 6, 10 ] 369 0.487% 99.916% ( 10, 15 ] 45 0.059% 99.975% ( 15, 22 ] 7 0.009% 99.984% ( 22, 34 ] 1 0.001% 99.985% ( 51, 76 ] 1 0.001% 99.987% ( 110, 170 ] 2 0.003% 99.989% ( 170, 250 ] 2 0.003% 99.992% ( 250, 380 ] 4 0.005% 99.997% ( 580, 870 ] 1 0.001% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 15167 Average: 6.7104 StdDev: 107.37 Min: 1 Median: 1.5586 Max: 4922 Percentiles: P50: 1.56 P75: 1.93 P99: 5.71 P99.9: 2004.13 P99.99: 4289.28 ------------------------------------------------------ [ 0, 1 ] 1872 12.343% 12.343% ## ( 1, 2 ] 10224 67.410% 79.752% ############# ( 2, 3 ] 2117 13.958% 93.710% ### ( 3, 4 ] 542 3.574% 97.284% # ( 4, 6 ] 305 2.011% 99.295% ( 6, 10 ] 49 0.323% 99.618% ( 10, 15 ] 14 0.092% 99.710% ( 34, 51 ] 1 0.007% 99.716% ( 110, 170 ] 1 0.007% 99.723% ( 170, 250 ] 5 0.033% 99.756% ( 250, 380 ] 3 0.020% 99.776% ( 380, 580 ] 2 0.013% 99.789% ( 580, 870 ] 7 0.046% 99.835% ( 870, 1300 ] 4 0.026% 99.862% ( 1300, 1900 ] 5 0.033% 99.895% ( 1900, 2900 ] 8 0.053% 99.947% ( 2900, 4400 ] 7 0.046% 99.993% ( 4400, 6600 ] 1 0.007% 100.000% ** Level 4 read latency histogram (micros): Count: 35 Average: 1665.3429 StdDev: 1401.13 Min: 190 Median: 1120.8333 Max: 5297 Percentiles: P50: 1120.83 P75: 2650.00 P99: 5297.00 P99.9: 5297.00 P99.99: 5297.00 ------------------------------------------------------ ( 170, 250 ] 1 2.857% 2.857% # ( 250, 380 ] 4 11.429% 14.286% ## ( 380, 580 ] 4 11.429% 25.714% ## ( 580, 870 ] 5 14.286% 40.000% ### ( 870, 1300 ] 6 17.143% 57.143% ### ( 1300, 1900 ] 4 11.429% 68.571% ## ( 1900, 2900 ] 3 8.571% 77.143% ## ( 2900, 4400 ] 5 14.286% 91.429% ### ( 4400, 6600 ] 3 8.571% 100.000% ## ** DB Stats ** Uptime(secs): 9.1 total, 2.8 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 325.52 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.129 H:M:S, 12.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 345.38 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.250 H:M:S, 8.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/27-08:57:31 ... thread 0: (1000000,4000000) ops and (305816.4,329569.8) ops/second in (3.269936,12.137033) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/12 2.70 GB 100.7 2.1 0.0 2.1 5.8 3.8 0.0 1.6 111.3 311.0 19.24 17.88 61 0.315 2174K 2280 0.0 0.0 L1 4/4 363.05 MB 0.0 1.2 0.9 0.3 1.2 0.9 0.2 1.4 235.0 234.3 5.09 4.96 2 2.545 1214K 3423 0.0 0.0 L2 1/0 49.57 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.6 1.8 249.0 248.9 1.40 1.35 2 0.700 353K 182 0.0 0.0 L3 36/0 996.31 MB 1.0 0.8 0.7 0.1 0.8 0.7 0.0 1.2 213.2 212.9 3.90 3.52 2 1.950 844K 1034 0.0 0.0 L4 136/0 9.62 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1242/16 99.88 GB 0.0 4.4 1.7 2.7 8.2 5.5 1.4 2.2 152.5 281.9 29.63 27.71 67 0.442 4587K 6919 0.0 0.0 Int 0/0 0.00 KB 0.0 4.4 1.7 2.7 8.2 5.5 1.4 2.2 152.5 281.9 29.63 27.71 67 0.442 4587K 6919 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.4 1.7 2.7 4.4 1.7 0.0 0.0 242.3 242.0 18.64 17.90 14 1.332 4587K 6919 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 349.8 10.99 9.81 53 0.207 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.3 total, 12.1 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.16 GB write, 676.94 MB/s write, 4.41 GB read, 366.12 MB/s read, 29.6 seconds Interval compaction: 8.16 GB write, 688.19 MB/s write, 4.41 GB read, 372.20 MB/s read, 29.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 1, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 1, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 1, total-stops: 13, interval: 14 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1054069 Average: 2.2008 StdDev: 10.79 Min: 0 Median: 1.4975 Max: 3259 Percentiles: P50: 1.50 P75: 1.87 P99: 5.58 P99.9: 11.18 P99.99: 14.97 ------------------------------------------------------ [ 0, 1 ] 178998 16.982% 16.982% ### ( 1, 2 ] 699622 66.373% 83.355% ############# ( 2, 3 ] 113092 10.729% 94.084% ## ( 3, 4 ] 35486 3.367% 97.451% # ( 4, 6 ] 20638 1.958% 99.409% ( 6, 10 ] 4884 0.463% 99.872% ( 10, 15 ] 1250 0.119% 99.991% ( 15, 22 ] 82 0.008% 99.998% ( 22, 34 ] 19 0.002% 100.000% ( 34, 51 ] 7 0.001% 100.001% ( 51, 76 ] 10 0.001% 100.002% ( 76, 110 ] 32 0.003% 100.005% ( 110, 170 ] 76 0.007% 100.012% ( 170, 250 ] 38 0.004% 100.016% ( 250, 380 ] 29 0.003% 100.018% ( 380, 580 ] 17 0.002% 100.020% ( 580, 870 ] 20 0.002% 100.022% ( 870, 1300 ] 10 0.001% 100.023% ( 1300, 1900 ] 5 0.000% 100.023% ( 1900, 2900 ] 5 0.000% 100.024% ( 2900, 4400 ] 5 0.000% 100.024% ** Level 1 read latency histogram (micros): Count: 227061 Average: 2.3524 StdDev: 12.85 Min: 1 Median: 1.5415 Max: 3003 Percentiles: P50: 1.54 P75: 1.92 P99: 5.65 P99.9: 14.19 P99.99: 430.98 ------------------------------------------------------ [ 0, 1 ] 31230 13.754% 13.754% ### ( 1, 2 ] 151979 66.933% 80.687% ############# ( 2, 3 ] 29749 13.102% 93.789% ### ( 3, 4 ] 7829 3.448% 97.237% # ( 4, 6 ] 4846 2.134% 99.371% ( 6, 10 ] 1005 0.443% 99.814% ( 10, 15 ] 234 0.103% 99.917% ( 15, 22 ] 30 0.013% 99.930% ( 22, 34 ] 7 0.003% 99.933% ( 34, 51 ] 17 0.007% 99.941% ( 51, 76 ] 25 0.011% 99.952% ( 76, 110 ] 31 0.014% 99.965% ( 110, 170 ] 14 0.006% 99.971% ( 170, 250 ] 19 0.008% 99.980% ( 250, 380 ] 21 0.009% 99.989% ( 380, 580 ] 9 0.004% 99.993% ( 580, 870 ] 8 0.004% 99.996% ( 870, 1300 ] 2 0.001% 99.997% ( 1300, 1900 ] 4 0.002% 99.999% ( 1900, 2900 ] 1 0.000% 100.000% ( 2900, 4400 ] 1 0.000% 100.000% ** Level 2 read latency histogram (micros): Count: 75843 Average: 2.0334 StdDev: 5.09 Min: 0 Median: 1.4064 Max: 951 Percentiles: P50: 1.41 P75: 1.81 P99: 5.56 P99.9: 9.87 P99.99: 186.63 ------------------------------------------------------ [ 0, 1 ] 18599 24.523% 24.523% ##### ( 1, 2 ] 47542 62.685% 87.208% ############# ( 2, 3 ] 5473 7.216% 94.424% # ( 3, 4 ] 2327 3.068% 97.492% # ( 4, 6 ] 1469 1.937% 99.429% ( 6, 10 ] 369 0.487% 99.916% ( 10, 15 ] 45 0.059% 99.975% ( 15, 22 ] 7 0.009% 99.984% ( 22, 34 ] 1 0.001% 99.985% ( 51, 76 ] 1 0.001% 99.987% ( 110, 170 ] 2 0.003% 99.989% ( 170, 250 ] 2 0.003% 99.992% ( 250, 380 ] 4 0.005% 99.997% ( 580, 870 ] 1 0.001% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 32992 Average: 4.6472 StdDev: 73.33 Min: 0 Median: 1.6183 Max: 4922 Percentiles: P50: 1.62 P75: 2.00 P99: 7.11 P99.9: 580.29 P99.99: 3907.31 ------------------------------------------------------ [ 0, 1 ] 3072 9.311% 9.311% ## ( 1, 2 ] 21710 65.804% 75.115% ############# ( 2, 3 ] 5439 16.486% 91.601% ### ( 3, 4 ] 1449 4.392% 95.993% # ( 4, 6 ] 930 2.819% 98.812% # ( 6, 10 ] 223 0.676% 99.488% ( 10, 15 ] 54 0.164% 99.651% ( 15, 22 ] 11 0.033% 99.685% ( 22, 34 ] 2 0.006% 99.691% ( 34, 51 ] 10 0.030% 99.721% ( 51, 76 ] 26 0.079% 99.800% ( 110, 170 ] 9 0.027% 99.827% ( 170, 250 ] 10 0.030% 99.858% ( 250, 380 ] 7 0.021% 99.879% ( 380, 580 ] 7 0.021% 99.900% ( 580, 870 ] 8 0.024% 99.924% ( 870, 1300 ] 4 0.012% 99.936% ( 1300, 1900 ] 5 0.015% 99.952% ( 1900, 2900 ] 8 0.024% 99.976% ( 2900, 4400 ] 7 0.021% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ** Level 4 read latency histogram (micros): Count: 35 Average: 1665.3429 StdDev: 1401.13 Min: 190 Median: 1120.8333 Max: 5297 Percentiles: P50: 1120.83 P75: 2650.00 P99: 5297.00 P99.9: 5297.00 P99.99: 5297.00 ------------------------------------------------------ ( 170, 250 ] 1 2.857% 2.857% # ( 250, 380 ] 4 11.429% 14.286% ## ( 380, 580 ] 4 11.429% 25.714% ## ( 580, 870 ] 5 14.286% 40.000% ### ( 870, 1300 ] 6 17.143% 57.143% ### ( 1300, 1900 ] 4 11.429% 68.571% ## ( 1900, 2900 ] 3 8.571% 77.143% ## ( 2900, 4400 ] 5 14.286% 91.429% ### ( 4400, 6600 ] 3 8.571% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 764.9500 StdDev: 1011.55 Min: 86 Median: 250.0000 Max: 4038 Percentiles: P50: 250.00 P75: 870.00 P99: 4038.00 P99.9: 4038.00 P99.99: 4038.00 ------------------------------------------------------ ( 76, 110 ] 6 30.000% 30.000% ###### ( 110, 170 ] 2 10.000% 40.000% ## ( 170, 250 ] 2 10.000% 50.000% ## ( 380, 580 ] 4 20.000% 70.000% #### ( 580, 870 ] 1 5.000% 75.000% # ( 870, 1300 ] 1 5.000% 80.000% # ( 1300, 1900 ] 1 5.000% 85.000% # ( 1900, 2900 ] 2 10.000% 95.000% ## ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 12.3 total, 3.3 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 319.02 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.730 H:M:S, 14.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 300.98 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.602 H:M:S, 18.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/27-08:57:36 ... thread 0: (1000000,5000000) ops and (232145.5,304049.8) ops/second in (4.307644,16.444677) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/11 2.21 GB 6.3 3.1 0.0 3.1 7.8 4.7 0.0 1.7 123.9 308.4 25.93 24.21 80 0.324 3262K 3370 0.0 0.0 L1 16/16 1.75 GB 0.0 2.9 2.3 0.7 2.9 2.3 0.2 1.3 227.9 226.5 13.20 12.74 3 4.399 3053K 18K 0.0 0.0 L2 1/0 49.57 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.6 1.8 249.0 248.9 1.40 1.35 2 0.700 353K 182 0.0 0.0 L3 36/0 996.31 MB 1.0 0.8 0.7 0.1 0.8 0.7 0.0 1.2 213.2 212.9 3.90 3.52 2 1.950 844K 1034 0.0 0.0 L4 136/0 9.62 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1249/27 100.79 GB 0.0 7.2 3.1 4.1 11.9 7.8 1.4 2.5 166.5 273.8 44.43 41.82 87 0.511 7513K 22K 0.0 0.0 Int 0/0 0.00 KB 0.0 7.2 3.1 4.1 11.9 7.8 1.4 2.5 166.5 273.8 44.43 41.82 87 0.511 7513K 22K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 7.2 3.1 4.1 7.2 3.1 0.0 0.0 240.9 240.1 30.72 29.58 19 1.617 7513K 22K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 349.3 13.71 12.24 68 0.202 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.6 total, 16.4 interval Flush(GB): cumulative 4.676, interval 4.676 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.88 GB write, 730.68 MB/s write, 7.23 GB read, 444.48 MB/s read, 44.4 seconds Interval compaction: 11.88 GB write, 739.64 MB/s write, 7.23 GB read, 449.93 MB/s read, 44.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 8, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 8, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 8, total-stops: 13, interval: 21 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1456167 Average: 2.2513 StdDev: 11.44 Min: 0 Median: 1.4961 Max: 3259 Percentiles: P50: 1.50 P75: 1.87 P99: 5.63 P99.9: 12.01 P99.99: 127.01 ------------------------------------------------------ [ 0, 1 ] 246519 16.929% 16.929% ### ( 1, 2 ] 970796 66.668% 83.597% ############# ( 2, 3 ] 153703 10.555% 94.153% ## ( 3, 4 ] 48011 3.297% 97.450% # ( 4, 6 ] 27677 1.901% 99.350% ( 6, 10 ] 7348 0.505% 99.855% ( 10, 15 ] 1637 0.112% 99.967% ( 15, 22 ] 141 0.010% 99.977% ( 22, 34 ] 23 0.002% 99.979% ( 34, 51 ] 9 0.001% 99.979% ( 51, 76 ] 13 0.001% 99.980% ( 76, 110 ] 67 0.005% 99.985% ( 110, 170 ] 273 0.019% 100.003% ( 170, 250 ] 110 0.008% 100.011% ( 250, 380 ] 73 0.005% 100.016% ( 380, 580 ] 58 0.004% 100.020% ( 580, 870 ] 47 0.003% 100.023% ( 870, 1300 ] 17 0.001% 100.024% ( 1300, 1900 ] 8 0.001% 100.025% ( 1900, 2900 ] 6 0.000% 100.025% ( 2900, 4400 ] 7 0.000% 100.026% ** Level 1 read latency histogram (micros): Count: 316484 Average: 2.4735 StdDev: 14.46 Min: 0 Median: 1.5327 Max: 3003 Percentiles: P50: 1.53 P75: 1.91 P99: 5.92 P99.9: 78.24 P99.99: 622.62 ------------------------------------------------------ [ 0, 1 ] 45644 14.422% 14.422% ### ( 1, 2 ] 211355 66.782% 81.204% ############# ( 2, 3 ] 39215 12.391% 93.595% ## ( 3, 4 ] 10651 3.365% 96.961% # ( 4, 6 ] 6730 2.126% 99.087% ( 6, 10 ] 2137 0.675% 99.762% ( 10, 15 ] 323 0.102% 99.864% ( 15, 22 ] 43 0.014% 99.878% ( 22, 34 ] 10 0.003% 99.881% ( 34, 51 ] 18 0.006% 99.887% ( 51, 76 ] 36 0.011% 99.898% ( 76, 110 ] 99 0.031% 99.930% ( 110, 170 ] 73 0.023% 99.953% ( 170, 250 ] 42 0.013% 99.966% ( 250, 380 ] 39 0.012% 99.978% ( 380, 580 ] 36 0.011% 99.990% ( 580, 870 ] 16 0.005% 99.995% ( 870, 1300 ] 10 0.003% 99.998% ( 1300, 1900 ] 6 0.002% 100.000% ( 1900, 2900 ] 1 0.000% 100.000% ( 2900, 4400 ] 1 0.000% 100.000% ** Level 2 read latency histogram (micros): Count: 75843 Average: 2.0334 StdDev: 5.09 Min: 0 Median: 1.4064 Max: 951 Percentiles: P50: 1.41 P75: 1.81 P99: 5.56 P99.9: 9.87 P99.99: 186.63 ------------------------------------------------------ [ 0, 1 ] 18599 24.523% 24.523% ##### ( 1, 2 ] 47542 62.685% 87.208% ############# ( 2, 3 ] 5473 7.216% 94.424% # ( 3, 4 ] 2327 3.068% 97.492% # ( 4, 6 ] 1469 1.937% 99.429% ( 6, 10 ] 369 0.487% 99.916% ( 10, 15 ] 45 0.059% 99.975% ( 15, 22 ] 7 0.009% 99.984% ( 22, 34 ] 1 0.001% 99.985% ( 51, 76 ] 1 0.001% 99.987% ( 110, 170 ] 2 0.003% 99.989% ( 170, 250 ] 2 0.003% 99.992% ( 250, 380 ] 4 0.005% 99.997% ( 580, 870 ] 1 0.001% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 32992 Average: 4.6472 StdDev: 73.33 Min: 0 Median: 1.6183 Max: 4922 Percentiles: P50: 1.62 P75: 2.00 P99: 7.11 P99.9: 580.29 P99.99: 3907.31 ------------------------------------------------------ [ 0, 1 ] 3072 9.311% 9.311% ## ( 1, 2 ] 21710 65.804% 75.115% ############# ( 2, 3 ] 5439 16.486% 91.601% ### ( 3, 4 ] 1449 4.392% 95.993% # ( 4, 6 ] 930 2.819% 98.812% # ( 6, 10 ] 223 0.676% 99.488% ( 10, 15 ] 54 0.164% 99.651% ( 15, 22 ] 11 0.033% 99.685% ( 22, 34 ] 2 0.006% 99.691% ( 34, 51 ] 10 0.030% 99.721% ( 51, 76 ] 26 0.079% 99.800% ( 110, 170 ] 9 0.027% 99.827% ( 170, 250 ] 10 0.030% 99.858% ( 250, 380 ] 7 0.021% 99.879% ( 380, 580 ] 7 0.021% 99.900% ( 580, 870 ] 8 0.024% 99.924% ( 870, 1300 ] 4 0.012% 99.936% ( 1300, 1900 ] 5 0.015% 99.952% ( 1900, 2900 ] 8 0.024% 99.976% ( 2900, 4400 ] 7 0.021% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ** Level 4 read latency histogram (micros): Count: 35 Average: 1665.3429 StdDev: 1401.13 Min: 190 Median: 1120.8333 Max: 5297 Percentiles: P50: 1120.83 P75: 2650.00 P99: 5297.00 P99.9: 5297.00 P99.99: 5297.00 ------------------------------------------------------ ( 170, 250 ] 1 2.857% 2.857% # ( 250, 380 ] 4 11.429% 14.286% ## ( 380, 580 ] 4 11.429% 25.714% ## ( 580, 870 ] 5 14.286% 40.000% ### ( 870, 1300 ] 6 17.143% 57.143% ### ( 1300, 1900 ] 4 11.429% 68.571% ## ( 1900, 2900 ] 3 8.571% 77.143% ## ( 2900, 4400 ] 5 14.286% 91.429% ### ( 4400, 6600 ] 3 8.571% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 764.9500 StdDev: 1011.55 Min: 86 Median: 250.0000 Max: 4038 Percentiles: P50: 250.00 P75: 870.00 P99: 4038.00 P99.9: 4038.00 P99.99: 4038.00 ------------------------------------------------------ ( 76, 110 ] 6 30.000% 30.000% ###### ( 110, 170 ] 2 10.000% 40.000% ## ( 170, 250 ] 2 10.000% 50.000% ## ( 380, 580 ] 4 20.000% 70.000% #### ( 580, 870 ] 1 5.000% 75.000% # ( 870, 1300 ] 1 5.000% 80.000% # ( 1300, 1900 ] 1 5.000% 85.000% # ( 1900, 2900 ] 2 10.000% 95.000% ## ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 16.6 total, 4.3 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 295.59 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.404 H:M:S, 20.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 228.48 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.674 H:M:S, 38.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/27-08:57:39 ... thread 0: (1000000,6000000) ops and (293450.7,302230.4) ops/second in (3.407728,19.852405) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/15 3.20 GB 81.8 3.9 0.0 3.9 9.6 5.7 0.0 1.7 126.5 308.2 31.87 29.79 98 0.325 4093K 4152 0.0 0.0 L1 16/16 1.75 GB 0.0 2.9 2.3 0.7 2.9 2.3 0.2 1.3 227.9 226.5 13.20 12.74 3 4.399 3053K 18K 0.0 0.0 L2 1/0 49.57 MB 0.5 0.3 0.2 0.2 0.3 0.2 0.6 1.8 249.0 248.9 1.40 1.35 2 0.700 353K 182 0.0 0.0 L3 36/0 996.31 MB 1.0 0.8 0.7 0.1 0.8 0.7 0.0 1.2 213.2 212.9 3.90 3.52 2 1.950 844K 1034 0.0 0.0 L4 136/0 9.62 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1255/31 101.77 GB 0.0 8.0 3.1 4.9 13.7 8.8 1.4 2.4 163.2 277.7 50.37 47.40 105 0.480 8345K 23K 0.0 0.0 Int 0/0 0.00 KB 0.0 8.0 3.1 4.9 13.7 8.8 1.4 2.4 163.2 277.7 50.37 47.40 105 0.480 8345K 23K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 8.0 3.1 4.9 8.0 3.1 0.0 0.0 243.0 242.3 33.83 32.63 22 1.538 8345K 23K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 350.3 16.54 14.77 83 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.1 total, 19.9 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.66 GB write, 697.58 MB/s write, 8.03 GB read, 409.80 MB/s read, 50.4 seconds Interval compaction: 13.66 GB write, 704.67 MB/s write, 8.03 GB read, 413.96 MB/s read, 50.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 15, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 15, interval: 28 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1797409 Average: 2.2450 StdDev: 11.60 Min: 0 Median: 1.4978 Max: 4722 Percentiles: P50: 1.50 P75: 1.87 P99: 5.63 P99.9: 11.65 P99.99: 94.91 ------------------------------------------------------ [ 0, 1 ] 300970 16.745% 16.745% ### ( 1, 2 ] 1200689 66.801% 83.546% ############# ( 2, 3 ] 190256 10.585% 94.131% ## ( 3, 4 ] 59577 3.315% 97.445% # ( 4, 6 ] 34237 1.905% 99.350% ( 6, 10 ] 9218 0.513% 99.863% ( 10, 15 ] 2011 0.112% 99.975% ( 15, 22 ] 176 0.010% 99.985% ( 22, 34 ] 24 0.001% 99.986% ( 34, 51 ] 11 0.001% 99.987% ( 51, 76 ] 19 0.001% 99.988% ( 76, 110 ] 76 0.004% 99.992% ( 110, 170 ] 285 0.016% 100.008% ( 170, 250 ] 125 0.007% 100.015% ( 250, 380 ] 82 0.005% 100.019% ( 380, 580 ] 68 0.004% 100.023% ( 580, 870 ] 53 0.003% 100.026% ( 870, 1300 ] 19 0.001% 100.027% ( 1300, 1900 ] 12 0.001% 100.028% ( 1900, 2900 ] 6 0.000% 100.028% ( 2900, 4400 ] 8 0.000% 100.029% ( 4400, 6600 ] 1 0.000% 100.029% ** Level 1 read latency histogram (micros): Count: 393138 Average: 2.5215 StdDev: 15.63 Min: 0 Median: 1.5301 Max: 3003 Percentiles: P50: 1.53 P75: 1.90 P99: 6.49 P99.9: 101.20 P99.99: 605.74 ------------------------------------------------------ [ 0, 1 ] 57571 14.644% 14.644% ### ( 1, 2 ] 262209 66.696% 81.340% ############# ( 2, 3 ] 47843 12.170% 93.510% ## ( 3, 4 ] 12925 3.288% 96.798% # ( 4, 6 ] 8261 2.101% 98.899% ( 6, 10 ] 3261 0.829% 99.728% ( 10, 15 ] 436 0.111% 99.839% ( 15, 22 ] 55 0.014% 99.853% ( 22, 34 ] 12 0.003% 99.856% ( 34, 51 ] 18 0.005% 99.861% ( 51, 76 ] 36 0.009% 99.870% ( 76, 110 ] 159 0.040% 99.910% ( 110, 170 ] 147 0.037% 99.948% ( 170, 250 ] 62 0.016% 99.964% ( 250, 380 ] 49 0.012% 99.976% ( 380, 580 ] 53 0.013% 99.990% ( 580, 870 ] 19 0.005% 99.994% ( 870, 1300 ] 11 0.003% 99.997% ( 1300, 1900 ] 6 0.002% 99.999% ( 1900, 2900 ] 3 0.001% 99.999% ( 2900, 4400 ] 2 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 75843 Average: 2.0334 StdDev: 5.09 Min: 0 Median: 1.4064 Max: 951 Percentiles: P50: 1.41 P75: 1.81 P99: 5.56 P99.9: 9.87 P99.99: 186.63 ------------------------------------------------------ [ 0, 1 ] 18599 24.523% 24.523% ##### ( 1, 2 ] 47542 62.685% 87.208% ############# ( 2, 3 ] 5473 7.216% 94.424% # ( 3, 4 ] 2327 3.068% 97.492% # ( 4, 6 ] 1469 1.937% 99.429% ( 6, 10 ] 369 0.487% 99.916% ( 10, 15 ] 45 0.059% 99.975% ( 15, 22 ] 7 0.009% 99.984% ( 22, 34 ] 1 0.001% 99.985% ( 51, 76 ] 1 0.001% 99.987% ( 110, 170 ] 2 0.003% 99.989% ( 170, 250 ] 2 0.003% 99.992% ( 250, 380 ] 4 0.005% 99.997% ( 580, 870 ] 1 0.001% 99.999% ( 870, 1300 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 32992 Average: 4.6472 StdDev: 73.33 Min: 0 Median: 1.6183 Max: 4922 Percentiles: P50: 1.62 P75: 2.00 P99: 7.11 P99.9: 580.29 P99.99: 3907.31 ------------------------------------------------------ [ 0, 1 ] 3072 9.311% 9.311% ## ( 1, 2 ] 21710 65.804% 75.115% ############# ( 2, 3 ] 5439 16.486% 91.601% ### ( 3, 4 ] 1449 4.392% 95.993% # ( 4, 6 ] 930 2.819% 98.812% # ( 6, 10 ] 223 0.676% 99.488% ( 10, 15 ] 54 0.164% 99.651% ( 15, 22 ] 11 0.033% 99.685% ( 22, 34 ] 2 0.006% 99.691% ( 34, 51 ] 10 0.030% 99.721% ( 51, 76 ] 26 0.079% 99.800% ( 110, 170 ] 9 0.027% 99.827% ( 170, 250 ] 10 0.030% 99.858% ( 250, 380 ] 7 0.021% 99.879% ( 380, 580 ] 7 0.021% 99.900% ( 580, 870 ] 8 0.024% 99.924% ( 870, 1300 ] 4 0.012% 99.936% ( 1300, 1900 ] 5 0.015% 99.952% ( 1900, 2900 ] 8 0.024% 99.976% ( 2900, 4400 ] 7 0.021% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ** Level 4 read latency histogram (micros): Count: 35 Average: 1665.3429 StdDev: 1401.13 Min: 190 Median: 1120.8333 Max: 5297 Percentiles: P50: 1120.83 P75: 2650.00 P99: 5297.00 P99.9: 5297.00 P99.99: 5297.00 ------------------------------------------------------ ( 170, 250 ] 1 2.857% 2.857% # ( 250, 380 ] 4 11.429% 14.286% ## ( 380, 580 ] 4 11.429% 25.714% ## ( 580, 870 ] 5 14.286% 40.000% ### ( 870, 1300 ] 6 17.143% 57.143% ### ( 1300, 1900 ] 4 11.429% 68.571% ## ( 1900, 2900 ] 3 8.571% 77.143% ## ( 2900, 4400 ] 5 14.286% 91.429% ### ( 4400, 6600 ] 3 8.571% 100.000% ## ** Level 5 read latency histogram (micros): Count: 20 Average: 764.9500 StdDev: 1011.55 Min: 86 Median: 250.0000 Max: 4038 Percentiles: P50: 250.00 P75: 870.00 P99: 4038.00 P99.9: 4038.00 P99.99: 4038.00 ------------------------------------------------------ ( 76, 110 ] 6 30.000% 30.000% ###### ( 110, 170 ] 2 10.000% 40.000% ## ( 170, 250 ] 2 10.000% 50.000% ## ( 380, 580 ] 4 20.000% 70.000% #### ( 580, 870 ] 1 5.000% 75.000% # ( 870, 1300 ] 1 5.000% 80.000% # ( 1300, 1900 ] 1 5.000% 85.000% # ( 1900, 2900 ] 2 10.000% 95.000% ## ( 2900, 4400 ] 1 5.000% 100.000% # ** DB Stats ** Uptime(secs): 20.1 total, 3.4 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 294.44 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.226 H:M:S, 21.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 288.81 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.822 H:M:S, 24.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/27-08:57:57 ... thread 0: (1000000,7000000) ops and (54766.3,183670.2) ops/second in (18.259389,38.111794) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/12 2.03 GB 18.9 4.4 0.0 4.4 11.1 6.6 0.0 1.7 118.3 295.6 38.35 34.29 113 0.339 4605K 4667 0.0 0.0 L1 12/12 1.33 GB 0.0 6.8 4.4 2.4 6.8 4.3 0.2 1.5 224.2 221.6 31.25 29.44 4 7.812 7114K 80K 0.0 0.0 L2 1/1 49.57 MB 0.0 0.3 0.2 0.2 0.3 0.2 3.1 1.8 249.0 248.9 1.40 1.35 2 0.700 353K 182 0.0 0.0 L3 22/8 1.64 GB 1.0 2.2 1.8 0.4 2.2 1.8 1.4 1.2 156.7 156.3 14.63 10.28 11 1.330 2179K 1643 0.0 0.0 L4 161/0 9.43 GB 1.0 1.5 1.2 0.3 1.5 1.2 1.0 1.2 132.9 132.1 11.57 7.07 14 0.827 1562K 8669 0.0 0.0 L5 1072/0 88.22 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1282/33 102.68 GB 0.0 15.4 7.6 7.7 21.9 14.2 8.0 3.3 161.7 230.7 97.20 82.42 144 0.675 15M 95K 0.0 0.0 Int 0/0 0.00 KB 0.0 15.4 7.6 7.7 21.9 14.2 7.9 3.3 161.7 230.7 97.20 82.42 144 0.675 15M 95K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 15.4 7.6 7.7 15.3 7.5 0.0 0.0 206.7 205.4 76.06 65.00 48 1.585 15M 95K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 321.8 21.14 17.43 96 0.220 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 38.3 total, 38.1 interval Flush(GB): cumulative 6.644, interval 6.644 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 21.90 GB write, 585.25 MB/s write, 15.35 GB read, 410.29 MB/s read, 97.2 seconds Interval compaction: 21.90 GB write, 588.34 MB/s write, 15.35 GB read, 412.46 MB/s read, 97.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 17, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 17, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 36, pending-compaction-bytes-delays: 5, pending-compaction-bytes-stops: 0, total-delays: 22, total-stops: 36, interval: 58 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2533323 Average: 2.7629 StdDev: 28.59 Min: 0 Median: 1.4755 Max: 7798 Percentiles: P50: 1.48 P75: 1.87 P99: 5.97 P99.9: 126.79 P99.99: 424.50 ------------------------------------------------------ [ 0, 1 ] 498896 19.693% 19.693% #### ( 1, 2 ] 1614754 63.741% 83.434% ############# ( 2, 3 ] 250523 9.889% 93.323% ## ( 3, 4 ] 85113 3.360% 96.683% # ( 4, 6 ] 59508 2.349% 99.032% ( 6, 10 ] 14985 0.592% 99.623% ( 10, 15 ] 3029 0.120% 99.743% ( 15, 22 ] 375 0.015% 99.758% ( 22, 34 ] 34 0.001% 99.759% ( 34, 51 ] 14 0.001% 99.760% ( 51, 76 ] 117 0.005% 99.764% ( 76, 110 ] 2914 0.115% 99.879% ( 110, 170 ] 1893 0.075% 99.954% ( 170, 250 ] 443 0.017% 99.971% ( 250, 380 ] 390 0.015% 99.987% ( 380, 580 ] 376 0.015% 100.002% ( 580, 870 ] 237 0.009% 100.011% ( 870, 1300 ] 109 0.004% 100.015% ( 1300, 1900 ] 59 0.002% 100.018% ( 1900, 2900 ] 57 0.002% 100.020% ( 2900, 4400 ] 56 0.002% 100.022% ( 4400, 6600 ] 15 0.001% 100.023% ( 6600, 9900 ] 2 0.000% 100.023% ** Level 1 read latency histogram (micros): Count: 1194741 Average: 4.1331 StdDev: 64.68 Min: 0 Median: 1.5759 Max: 9470 Percentiles: P50: 1.58 P75: 2.07 P99: 8.37 P99.9: 139.44 P99.99: 569.25 ------------------------------------------------------ [ 0, 1 ] 206346 17.271% 17.271% ### ( 1, 2 ] 679008 56.833% 74.104% ########### ( 2, 3 ] 162912 13.636% 87.740% ### ( 3, 4 ] 64785 5.423% 93.163% # ( 4, 6 ] 61293 5.130% 98.293% # ( 6, 10 ] 14274 1.195% 99.488% ( 10, 15 ] 2565 0.215% 99.702% ( 15, 22 ] 404 0.034% 99.736% ( 22, 34 ] 34 0.003% 99.739% ( 34, 51 ] 31 0.003% 99.741% ( 51, 76 ] 143 0.012% 99.753% ( 76, 110 ] 1344 0.112% 99.866% ( 110, 170 ] 830 0.069% 99.935% ( 170, 250 ] 173 0.014% 99.950% ( 250, 380 ] 207 0.017% 99.967% ( 380, 580 ] 288 0.024% 99.991% ( 580, 870 ] 259 0.022% 100.013% ( 870, 1300 ] 159 0.013% 100.026% ( 1300, 1900 ] 116 0.010% 100.036% ( 1900, 2900 ] 146 0.012% 100.048% ( 2900, 4400 ] 143 0.012% 100.060% ( 4400, 6600 ] 41 0.003% 100.064% ( 6600, 9900 ] 11 0.001% 100.065% ** Level 2 read latency histogram (micros): Count: 76095 Average: 3.8236 StdDev: 84.86 Min: 0 Median: 1.4089 Max: 10549 Percentiles: P50: 1.41 P75: 1.81 P99: 5.66 P99.9: 32.86 P99.99: 4334.70 ------------------------------------------------------ [ 0, 1 ] 18599 24.442% 24.442% ##### ( 1, 2 ] 47566 62.509% 86.951% ############# ( 2, 3 ] 5566 7.315% 94.265% # ( 3, 4 ] 2354 3.094% 97.359% # ( 4, 6 ] 1503 1.975% 99.334% ( 6, 10 ] 377 0.495% 99.829% ( 10, 15 ] 46 0.060% 99.890% ( 15, 22 ] 7 0.009% 99.899% ( 22, 34 ] 1 0.001% 99.900% ( 51, 76 ] 1 0.001% 99.901% ( 110, 170 ] 4 0.005% 99.907% ( 170, 250 ] 4 0.005% 99.912% ( 250, 380 ] 5 0.007% 99.919% ( 380, 580 ] 11 0.014% 99.933% ( 580, 870 ] 10 0.013% 99.946% ( 870, 1300 ] 11 0.014% 99.961% ( 1300, 1900 ] 3 0.004% 99.965% ( 1900, 2900 ] 6 0.008% 99.972% ( 2900, 4400 ] 14 0.018% 99.991% ( 4400, 6600 ] 5 0.007% 99.997% ( 6600, 9900 ] 1 0.001% 99.999% ( 9900, 14000 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 498184 Average: 9.9052 StdDev: 130.72 Min: 0 Median: 1.9584 Max: 12118 Percentiles: P50: 1.96 P75: 2.96 P99: 9.72 P99.9: 749.50 P99.99: 1111.87 ------------------------------------------------------ [ 0, 1 ] 10095 2.026% 2.026% ( 1, 2 ] 249372 50.056% 52.083% ########## ( 2, 3 ] 118787 23.844% 75.927% ##### ( 3, 4 ] 53028 10.644% 86.571% ## ( 4, 6 ] 52167 10.471% 97.042% ## ( 6, 10 ] 10476 2.103% 99.145% ( 10, 15 ] 1896 0.381% 99.526% ( 15, 22 ] 219 0.044% 99.570% ( 22, 34 ] 28 0.006% 99.575% ( 34, 51 ] 20 0.004% 99.579% ( 51, 76 ] 31 0.006% 99.585% ( 76, 110 ] 33 0.007% 99.592% ( 110, 170 ] 148 0.030% 99.622% ( 170, 250 ] 215 0.043% 99.665% ( 250, 380 ] 332 0.067% 99.732% ( 380, 580 ] 508 0.102% 99.834% ( 580, 870 ] 566 0.114% 99.947% ( 870, 1300 ] 379 0.076% 100.023% ( 1300, 1900 ] 260 0.052% 100.075% ( 1900, 2900 ] 229 0.046% 100.121% ( 2900, 4400 ] 250 0.050% 100.172% ( 4400, 6600 ] 65 0.013% 100.185% ( 6600, 9900 ] 13 0.003% 100.187% ( 9900, 14000 ] 1 0.000% 100.187% ** Level 4 read latency histogram (micros): Count: 220 Average: 487.8909 StdDev: 1025.67 Min: 1 Median: 2.9167 Max: 6522 Percentiles: P50: 2.92 P75: 513.33 P99: 5390.00 P99.9: 6479.00 P99.99: 6522.00 ------------------------------------------------------ [ 0, 1 ] 29 13.182% 13.182% ### ( 1, 2 ] 59 26.818% 40.000% ##### ( 2, 3 ] 24 10.909% 50.909% ## ( 3, 4 ] 3 1.364% 52.273% ( 4, 6 ] 5 2.273% 54.545% ( 6, 10 ] 3 1.364% 55.909% ( 10, 15 ] 3 1.364% 57.273% ( 15, 22 ] 3 1.364% 58.636% ( 22, 34 ] 1 0.455% 59.091% ( 34, 51 ] 1 0.455% 59.545% ( 110, 170 ] 5 2.273% 61.818% ( 170, 250 ] 21 9.545% 71.364% ## ( 250, 380 ] 4 1.818% 73.182% ( 380, 580 ] 6 2.727% 75.909% # ( 580, 870 ] 14 6.364% 82.273% # ( 870, 1300 ] 14 6.364% 88.636% # ( 1300, 1900 ] 7 3.182% 91.818% # ( 1900, 2900 ] 7 3.182% 95.000% # ( 2900, 4400 ] 7 3.182% 98.182% # ( 4400, 6600 ] 4 1.818% 100.000% ** Level 5 read latency histogram (micros): Count: 140 Average: 1409.7714 StdDev: 1446.50 Min: 82 Median: 851.8750 Max: 6718 Percentiles: P50: 851.88 P75: 2066.67 P99: 6424.00 P99.9: 6718.00 P99.99: 6718.00 ------------------------------------------------------ ( 76, 110 ] 10 7.143% 7.143% # ( 110, 170 ] 9 6.429% 13.571% # ( 170, 250 ] 6 4.286% 17.857% # ( 250, 380 ] 13 9.286% 27.143% ## ( 380, 580 ] 17 12.143% 39.286% ## ( 580, 870 ] 16 11.429% 50.714% ## ( 870, 1300 ] 19 13.571% 64.286% ### ( 1300, 1900 ] 12 8.571% 72.857% ## ( 1900, 2900 ] 18 12.857% 85.714% ### ( 2900, 4400 ] 14 10.000% 95.714% ## ( 4400, 6600 ] 5 3.571% 99.286% # ( 6600, 9900 ] 1 0.714% 100.000% ** DB Stats ** Uptime(secs): 38.3 total, 18.3 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 179.81 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:19.532 H:M:S, 51.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 53.90 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:15.307 H:M:S, 83.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/27-08:58:01 ... thread 0: (1000000,8000000) ops and (321308.6,194061.4) ops/second in (3.112273,41.224067) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/12 2.95 GB 125.8 5.4 0.0 5.4 13.0 7.6 0.0 1.7 122.8 294.4 45.14 40.59 129 0.350 5628K 5990 0.0 0.0 L1 12/12 1.33 GB 0.0 6.8 4.4 2.4 6.8 4.3 0.2 1.5 224.2 221.6 31.25 29.44 4 7.812 7114K 80K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 3.1 1.8 249.0 248.9 1.40 1.35 2 0.700 353K 182 0.0 0.0 L3 14/0 981.86 MB 1.0 3.0 1.9 1.1 3.0 1.9 1.4 1.6 157.2 156.9 19.30 13.35 12 1.608 2933K 2993 0.0 0.0 L4 165/0 9.53 GB 1.0 1.5 1.2 0.3 1.5 1.2 1.7 1.2 132.9 132.1 11.57 7.07 14 0.827 1562K 8669 0.0 0.0 L5 1080/0 88.83 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1289/24 103.60 GB 0.0 17.1 7.7 9.4 24.5 15.2 9.3 3.2 160.8 231.2 108.66 91.79 161 0.675 17M 98K 0.0 0.0 Int 0/0 0.00 KB 0.0 17.1 7.7 9.4 24.5 15.2 9.3 3.2 160.8 231.2 108.66 91.79 161 0.675 17M 98K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 17.1 7.7 9.4 17.0 7.6 0.0 0.0 206.1 204.9 84.75 71.91 52 1.630 17M 98K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 324.2 23.90 19.88 109 0.219 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 41.4 total, 41.2 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 24.53 GB write, 606.27 MB/s write, 17.06 GB read, 421.72 MB/s read, 108.7 seconds Interval compaction: 24.53 GB write, 609.24 MB/s write, 17.06 GB read, 423.78 MB/s read, 108.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 17, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 17, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 40, pending-compaction-bytes-delays: 5, pending-compaction-bytes-stops: 0, total-delays: 22, total-stops: 40, interval: 62 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2940293 Average: 2.7199 StdDev: 26.83 Min: 0 Median: 1.4928 Max: 7798 Percentiles: P50: 1.49 P75: 1.88 P99: 5.99 P99.9: 111.79 P99.99: 341.46 ------------------------------------------------------ [ 0, 1 ] 540719 18.390% 18.390% #### ( 1, 2 ] 1886046 64.145% 82.535% ############# ( 2, 3 ] 309180 10.515% 93.050% ## ( 3, 4 ] 103378 3.516% 96.566% # ( 4, 6 ] 71956 2.447% 99.013% ( 6, 10 ] 18841 0.641% 99.654% ( 10, 15 ] 3645 0.124% 99.778% ( 15, 22 ] 427 0.015% 99.793% ( 22, 34 ] 35 0.001% 99.794% ( 34, 51 ] 14 0.000% 99.794% ( 51, 76 ] 120 0.004% 99.798% ( 76, 110 ] 2936 0.100% 99.898% ( 110, 170 ] 1938 0.066% 99.964% ( 170, 250 ] 474 0.016% 99.980% ( 250, 380 ] 415 0.014% 99.994% ( 380, 580 ] 399 0.014% 100.008% ( 580, 870 ] 250 0.009% 100.016% ( 870, 1300 ] 118 0.004% 100.020% ( 1300, 1900 ] 64 0.002% 100.023% ( 1900, 2900 ] 58 0.002% 100.024% ( 2900, 4400 ] 56 0.002% 100.026% ( 4400, 6600 ] 15 0.001% 100.027% ( 6600, 9900 ] 2 0.000% 100.027% ** Level 1 read latency histogram (micros): Count: 1219524 Average: 4.1091 StdDev: 64.06 Min: 0 Median: 1.5766 Max: 9470 Percentiles: P50: 1.58 P75: 2.06 P99: 8.37 P99.9: 141.12 P99.99: 570.54 ------------------------------------------------------ [ 0, 1 ] 208407 17.089% 17.089% ### ( 1, 2 ] 696038 57.075% 74.164% ########### ( 2, 3 ] 166720 13.671% 87.835% ### ( 3, 4 ] 65615 5.380% 93.215% # ( 4, 6 ] 61845 5.071% 98.286% # ( 6, 10 ] 14682 1.204% 99.490% ( 10, 15 ] 2595 0.213% 99.703% ( 15, 22 ] 406 0.033% 99.736% ( 22, 34 ] 34 0.003% 99.739% ( 34, 51 ] 31 0.003% 99.742% ( 51, 76 ] 143 0.012% 99.753% ( 76, 110 ] 1344 0.110% 99.864% ( 110, 170 ] 859 0.070% 99.934% ( 170, 250 ] 189 0.015% 99.949% ( 250, 380 ] 214 0.018% 99.967% ( 380, 580 ] 295 0.024% 99.991% ( 580, 870 ] 262 0.021% 100.013% ( 870, 1300 ] 160 0.013% 100.026% ( 1300, 1900 ] 116 0.010% 100.035% ( 1900, 2900 ] 146 0.012% 100.047% ( 2900, 4400 ] 143 0.012% 100.059% ( 4400, 6600 ] 41 0.003% 100.062% ( 6600, 9900 ] 11 0.001% 100.063% ** Level 2 read latency histogram (micros): Count: 88431 Average: 3.6723 StdDev: 78.81 Min: 0 Median: 1.4447 Max: 10549 Percentiles: P50: 1.44 P75: 1.84 P99: 5.90 P99.9: 112.85 P99.99: 4202.52 ------------------------------------------------------ [ 0, 1 ] 19399 21.937% 21.937% #### ( 1, 2 ] 55808 63.109% 85.046% ############# ( 2, 3 ] 7435 8.408% 93.454% ## ( 3, 4 ] 3058 3.458% 96.912% # ( 4, 6 ] 1939 2.193% 99.104% ( 6, 10 ] 614 0.694% 99.799% ( 10, 15 ] 77 0.087% 99.886% ( 15, 22 ] 8 0.009% 99.895% ( 22, 34 ] 1 0.001% 99.896% ( 34, 51 ] 1 0.001% 99.897% ( 51, 76 ] 1 0.001% 99.898% ( 76, 110 ] 1 0.001% 99.899% ( 110, 170 ] 12 0.014% 99.913% ( 170, 250 ] 5 0.006% 99.919% ( 250, 380 ] 7 0.008% 99.926% ( 380, 580 ] 13 0.015% 99.941% ( 580, 870 ] 11 0.012% 99.954% ( 870, 1300 ] 11 0.012% 99.966% ( 1300, 1900 ] 3 0.003% 99.969% ( 1900, 2900 ] 6 0.007% 99.976% ( 2900, 4400 ] 14 0.016% 99.992% ( 4400, 6600 ] 5 0.006% 99.998% ( 6600, 9900 ] 1 0.001% 99.999% ( 9900, 14000 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 543559 Average: 9.3771 StdDev: 125.32 Min: 0 Median: 1.9304 Max: 12118 Percentiles: P50: 1.93 P75: 2.92 P99: 9.67 P99.9: 731.96 P99.99: 1108.74 ------------------------------------------------------ [ 0, 1 ] 12721 2.340% 2.340% ( 1, 2 ] 278432 51.224% 53.564% ########## ( 2, 3 ] 126666 23.303% 76.867% ##### ( 3, 4 ] 55693 10.246% 87.113% ## ( 4, 6 ] 54122 9.957% 97.070% ## ( 6, 10 ] 11419 2.101% 99.171% ( 10, 15 ] 1970 0.362% 99.533% ( 15, 22 ] 223 0.041% 99.574% ( 22, 34 ] 28 0.005% 99.580% ( 34, 51 ] 21 0.004% 99.583% ( 51, 76 ] 32 0.006% 99.589% ( 76, 110 ] 43 0.008% 99.597% ( 110, 170 ] 203 0.037% 99.635% ( 170, 250 ] 249 0.046% 99.680% ( 250, 380 ] 361 0.066% 99.747% ( 380, 580 ] 528 0.097% 99.844% ( 580, 870 ] 581 0.107% 99.951% ( 870, 1300 ] 383 0.070% 100.021% ( 1300, 1900 ] 260 0.048% 100.069% ( 1900, 2900 ] 229 0.042% 100.111% ( 2900, 4400 ] 250 0.046% 100.157% ( 4400, 6600 ] 65 0.012% 100.169% ( 6600, 9900 ] 13 0.002% 100.172% ( 9900, 14000 ] 1 0.000% 100.172% ** Level 4 read latency histogram (micros): Count: 220 Average: 487.8909 StdDev: 1025.67 Min: 1 Median: 2.9167 Max: 6522 Percentiles: P50: 2.92 P75: 513.33 P99: 5390.00 P99.9: 6479.00 P99.99: 6522.00 ------------------------------------------------------ [ 0, 1 ] 29 13.182% 13.182% ### ( 1, 2 ] 59 26.818% 40.000% ##### ( 2, 3 ] 24 10.909% 50.909% ## ( 3, 4 ] 3 1.364% 52.273% ( 4, 6 ] 5 2.273% 54.545% ( 6, 10 ] 3 1.364% 55.909% ( 10, 15 ] 3 1.364% 57.273% ( 15, 22 ] 3 1.364% 58.636% ( 22, 34 ] 1 0.455% 59.091% ( 34, 51 ] 1 0.455% 59.545% ( 110, 170 ] 5 2.273% 61.818% ( 170, 250 ] 21 9.545% 71.364% ## ( 250, 380 ] 4 1.818% 73.182% ( 380, 580 ] 6 2.727% 75.909% # ( 580, 870 ] 14 6.364% 82.273% # ( 870, 1300 ] 14 6.364% 88.636% # ( 1300, 1900 ] 7 3.182% 91.818% # ( 1900, 2900 ] 7 3.182% 95.000% # ( 2900, 4400 ] 7 3.182% 98.182% # ( 4400, 6600 ] 4 1.818% 100.000% ** Level 5 read latency histogram (micros): Count: 180 Average: 1252.5056 StdDev: 1385.04 Min: 81 Median: 711.8182 Max: 6718 Percentiles: P50: 711.82 P75: 1787.50 P99: 6306.67 P99.9: 6718.00 P99.99: 6718.00 ------------------------------------------------------ ( 76, 110 ] 17 9.444% 9.444% ## ( 110, 170 ] 17 9.444% 18.889% ## ( 170, 250 ] 9 5.000% 23.889% # ( 250, 380 ] 18 10.000% 33.889% ## ( 380, 580 ] 19 10.556% 44.444% ## ( 580, 870 ] 22 12.222% 56.667% ## ( 870, 1300 ] 20 11.111% 67.778% ## ( 1300, 1900 ] 16 8.889% 76.667% ## ( 1900, 2900 ] 21 11.667% 88.333% ## ( 2900, 4400 ] 14 7.778% 96.111% ## ( 4400, 6600 ] 6 3.333% 99.444% # ( 6600, 9900 ] 1 0.556% 100.000% ** DB Stats ** Uptime(secs): 41.4 total, 3.1 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 190.06 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:19.960 H:M:S, 48.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 316.23 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.428 H:M:S, 13.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/27-08:58:05 ... thread 0: (1000000,9000000) ops and (229234.2,197427.2) ops/second in (4.362351,45.586418) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/8 3.93 GB 251.7 6.8 0.0 6.8 15.3 8.5 0.0 1.8 130.0 294.2 53.30 48.35 149 0.358 7036K 7490 0.0 0.0 L1 12/12 1.33 GB 0.0 6.8 4.4 2.4 6.8 4.3 0.2 1.5 224.2 221.6 31.25 29.44 4 7.812 7114K 80K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.3 0.2 0.2 0.3 0.2 3.1 1.8 249.0 248.9 1.40 1.35 2 0.700 353K 182 0.0 0.0 L3 14/0 981.86 MB 1.0 3.0 1.9 1.1 3.0 1.9 1.4 1.6 157.2 156.9 19.30 13.35 12 1.608 2933K 2993 0.0 0.0 L4 165/0 9.53 GB 1.0 1.5 1.2 0.3 1.5 1.2 1.7 1.2 132.9 132.1 11.57 7.07 14 0.827 1562K 8669 0.0 0.0 L5 1080/0 88.83 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1289/20 104.59 GB 0.0 18.4 7.7 10.7 26.9 16.1 9.3 3.1 161.4 235.5 116.82 99.55 181 0.645 19M 99K 0.0 0.0 Int 0/0 0.00 KB 0.0 18.4 7.7 10.7 26.9 16.1 9.3 3.1 161.4 235.5 116.82 99.55 181 0.645 19M 99K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 18.4 7.7 10.7 18.3 7.6 0.0 0.0 209.5 208.4 90.00 77.07 57 1.579 19M 99K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.6 8.6 0.0 0.0 0.0 326.5 26.82 22.49 124 0.216 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 45.8 total, 45.6 interval Flush(GB): cumulative 8.551, interval 8.551 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 26.86 GB write, 600.76 MB/s write, 18.41 GB read, 411.81 MB/s read, 116.8 seconds Interval compaction: 26.86 GB write, 603.42 MB/s write, 18.41 GB read, 413.63 MB/s read, 116.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 23, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 23, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 42, pending-compaction-bytes-delays: 5, pending-compaction-bytes-stops: 0, total-delays: 28, total-stops: 42, interval: 70 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3294331 Average: 2.6749 StdDev: 25.55 Min: 0 Median: 1.4947 Max: 7798 Percentiles: P50: 1.49 P75: 1.88 P99: 5.96 P99.9: 107.92 P99.99: 325.80 ------------------------------------------------------ [ 0, 1 ] 593875 18.027% 18.027% #### ( 1, 2 ] 2129022 64.627% 82.654% ############# ( 2, 3 ] 346816 10.528% 93.182% ## ( 3, 4 ] 114973 3.490% 96.672% # ( 4, 6 ] 78271 2.376% 99.048% ( 6, 10 ] 20682 0.628% 99.675% ( 10, 15 ] 3956 0.120% 99.796% ( 15, 22 ] 453 0.014% 99.809% ( 22, 34 ] 35 0.001% 99.810% ( 34, 51 ] 16 0.000% 99.811% ( 51, 76 ] 122 0.004% 99.815% ( 76, 110 ] 2999 0.091% 99.906% ( 110, 170 ] 2016 0.061% 99.967% ( 170, 250 ] 509 0.015% 99.982% ( 250, 380 ] 440 0.013% 99.996% ( 380, 580 ] 422 0.013% 100.008% ( 580, 870 ] 269 0.008% 100.017% ( 870, 1300 ] 122 0.004% 100.020% ( 1300, 1900 ] 66 0.002% 100.022% ( 1900, 2900 ] 59 0.002% 100.024% ( 2900, 4400 ] 56 0.002% 100.026% ( 4400, 6600 ] 15 0.000% 100.026% ( 6600, 9900 ] 2 0.000% 100.026% ** Level 1 read latency histogram (micros): Count: 1407672 Average: 3.9058 StdDev: 59.80 Min: 0 Median: 1.5717 Max: 9470 Percentiles: P50: 1.57 P75: 2.00 P99: 8.33 P99.9: 140.40 P99.99: 565.90 ------------------------------------------------------ [ 0, 1 ] 232267 16.500% 16.500% ### ( 1, 2 ] 824782 58.592% 75.092% ############ ( 2, 3 ] 189480 13.461% 88.553% ### ( 3, 4 ] 71201 5.058% 93.611% # ( 4, 6 ] 65741 4.670% 98.281% # ( 6, 10 ] 17374 1.234% 99.515% ( 10, 15 ] 2799 0.199% 99.714% ( 15, 22 ] 424 0.030% 99.744% ( 22, 34 ] 34 0.002% 99.746% ( 34, 51 ] 32 0.002% 99.749% ( 51, 76 ] 149 0.011% 99.759% ( 76, 110 ] 1465 0.104% 99.863% ( 110, 170 ] 1019 0.072% 99.936% ( 170, 250 ] 224 0.016% 99.952% ( 250, 380 ] 240 0.017% 99.969% ( 380, 580 ] 323 0.023% 99.992% ( 580, 870 ] 268 0.019% 100.011% ( 870, 1300 ] 162 0.012% 100.022% ( 1300, 1900 ] 117 0.008% 100.030% ( 1900, 2900 ] 146 0.010% 100.041% ( 2900, 4400 ] 144 0.010% 100.051% ( 4400, 6600 ] 41 0.003% 100.054% ( 6600, 9900 ] 11 0.001% 100.055% ** Level 2 read latency histogram (micros): Count: 88431 Average: 3.6723 StdDev: 78.81 Min: 0 Median: 1.4447 Max: 10549 Percentiles: P50: 1.44 P75: 1.84 P99: 5.90 P99.9: 112.85 P99.99: 4202.52 ------------------------------------------------------ [ 0, 1 ] 19399 21.937% 21.937% #### ( 1, 2 ] 55808 63.109% 85.046% ############# ( 2, 3 ] 7435 8.408% 93.454% ## ( 3, 4 ] 3058 3.458% 96.912% # ( 4, 6 ] 1939 2.193% 99.104% ( 6, 10 ] 614 0.694% 99.799% ( 10, 15 ] 77 0.087% 99.886% ( 15, 22 ] 8 0.009% 99.895% ( 22, 34 ] 1 0.001% 99.896% ( 34, 51 ] 1 0.001% 99.897% ( 51, 76 ] 1 0.001% 99.898% ( 76, 110 ] 1 0.001% 99.899% ( 110, 170 ] 12 0.014% 99.913% ( 170, 250 ] 5 0.006% 99.919% ( 250, 380 ] 7 0.008% 99.926% ( 380, 580 ] 13 0.015% 99.941% ( 580, 870 ] 11 0.012% 99.954% ( 870, 1300 ] 11 0.012% 99.966% ( 1300, 1900 ] 3 0.003% 99.969% ( 1900, 2900 ] 6 0.007% 99.976% ( 2900, 4400 ] 14 0.016% 99.992% ( 4400, 6600 ] 5 0.006% 99.998% ( 6600, 9900 ] 1 0.001% 99.999% ( 9900, 14000 ] 1 0.001% 100.000% ** Level 3 read latency histogram (micros): Count: 543559 Average: 9.3771 StdDev: 125.32 Min: 0 Median: 1.9304 Max: 12118 Percentiles: P50: 1.93 P75: 2.92 P99: 9.67 P99.9: 731.96 P99.99: 1108.74 ------------------------------------------------------ [ 0, 1 ] 12721 2.340% 2.340% ( 1, 2 ] 278432 51.224% 53.564% ########## ( 2, 3 ] 126666 23.303% 76.867% ##### ( 3, 4 ] 55693 10.246% 87.113% ## ( 4, 6 ] 54122 9.957% 97.070% ## ( 6, 10 ] 11419 2.101% 99.171% ( 10, 15 ] 1970 0.362% 99.533% ( 15, 22 ] 223 0.041% 99.574% ( 22, 34 ] 28 0.005% 99.580% ( 34, 51 ] 21 0.004% 99.583% ( 51, 76 ] 32 0.006% 99.589% ( 76, 110 ] 43 0.008% 99.597% ( 110, 170 ] 203 0.037% 99.635% ( 170, 250 ] 249 0.046% 99.680% ( 250, 380 ] 361 0.066% 99.747% ( 380, 580 ] 528 0.097% 99.844% ( 580, 870 ] 581 0.107% 99.951% ( 870, 1300 ] 383 0.070% 100.021% ( 1300, 1900 ] 260 0.048% 100.069% ( 1900, 2900 ] 229 0.042% 100.111% ( 2900, 4400 ] 250 0.046% 100.157% ( 4400, 6600 ] 65 0.012% 100.169% ( 6600, 9900 ] 13 0.002% 100.172% ( 9900, 14000 ] 1 0.000% 100.172% ** Level 4 read latency histogram (micros): Count: 220 Average: 487.8909 StdDev: 1025.67 Min: 1 Median: 2.9167 Max: 6522 Percentiles: P50: 2.92 P75: 513.33 P99: 5390.00 P99.9: 6479.00 P99.99: 6522.00 ------------------------------------------------------ [ 0, 1 ] 29 13.182% 13.182% ### ( 1, 2 ] 59 26.818% 40.000% ##### ( 2, 3 ] 24 10.909% 50.909% ## ( 3, 4 ] 3 1.364% 52.273% ( 4, 6 ] 5 2.273% 54.545% ( 6, 10 ] 3 1.364% 55.909% ( 10, 15 ] 3 1.364% 57.273% ( 15, 22 ] 3 1.364% 58.636% ( 22, 34 ] 1 0.455% 59.091% ( 34, 51 ] 1 0.455% 59.545% ( 110, 170 ] 5 2.273% 61.818% ( 170, 250 ] 21 9.545% 71.364% ## ( 250, 380 ] 4 1.818% 73.182% ( 380, 580 ] 6 2.727% 75.909% # ( 580, 870 ] 14 6.364% 82.273% # ( 870, 1300 ] 14 6.364% 88.636% # ( 1300, 1900 ] 7 3.182% 91.818% # ( 1900, 2900 ] 7 3.182% 95.000% # ( 2900, 4400 ] 7 3.182% 98.182% # ( 4400, 6600 ] 4 1.818% 100.000% ** Level 5 read latency histogram (micros): Count: 180 Average: 1252.5056 StdDev: 1385.04 Min: 81 Median: 711.8182 Max: 6718 Percentiles: P50: 711.82 P75: 1787.50 P99: 6306.67 P99.9: 6718.00 P99.99: 6718.00 ------------------------------------------------------ ( 76, 110 ] 17 9.444% 9.444% ## ( 110, 170 ] 17 9.444% 18.889% ## ( 170, 250 ] 9 5.000% 23.889% # ( 250, 380 ] 18 10.000% 33.889% ## ( 380, 580 ] 19 10.556% 44.444% ## ( 580, 870 ] 22 12.222% 56.667% ## ( 870, 1300 ] 20 11.111% 67.778% ## ( 1300, 1900 ] 16 8.889% 76.667% ## ( 1900, 2900 ] 21 11.667% 88.333% ## ( 2900, 4400 ] 14 7.778% 96.111% ## ( 4400, 6600 ] 6 3.333% 99.444% # ( 6600, 9900 ] 1 0.556% 100.000% ** DB Stats ** Uptime(secs): 45.8 total, 4.4 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 193.44 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:21.587 H:M:S, 47.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 225.61 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.627 H:M:S, 37.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 6.131 micros/op 163116 ops/sec 60.025 seconds 9790999 operations; 158.0 MB/s Microseconds per write: Count: 9790999 Average: 6.1306 StdDev: 411.69 Min: 1 Median: 1.7347 Max: 814879 Percentiles: P50: 1.73 P75: 2.71 P99: 8.21 P99.9: 1200.19 P99.99: 2754.39 ------------------------------------------------------ [ 0, 1 ] 1317143 13.453% 13.453% ### ( 1, 2 ] 4870376 49.743% 63.196% ########## ( 2, 3 ] 1620474 16.551% 79.747% ### ( 3, 4 ] 1342984 13.717% 93.463% ### ( 4, 6 ] 497981 5.086% 98.549% # ( 6, 10 ] 79704 0.814% 99.363% ( 10, 15 ] 22363 0.228% 99.592% ( 15, 22 ] 7076 0.072% 99.664% ( 22, 34 ] 1490 0.015% 99.679% ( 34, 51 ] 148 0.002% 99.681% ( 51, 76 ] 21 0.000% 99.681% ( 76, 110 ] 6778 0.069% 99.750% ( 110, 170 ] 2805 0.029% 99.779% ( 170, 250 ] 253 0.003% 99.781% ( 250, 380 ] 88 0.001% 99.782% ( 380, 580 ] 20 0.000% 99.783% ( 580, 870 ] 24 0.000% 99.783% ( 870, 1300 ] 14950 0.153% 99.935% ( 1300, 1900 ] 37 0.000% 99.936% ( 1900, 2900 ] 6209 0.063% 99.999% ( 2900, 4400 ] 38 0.000% 100.000% ( 4400, 6600 ] 11 0.000% 100.000% ( 9900, 14000 ] 3 0.000% 100.000% ( 14000, 22000 ] 3 0.000% 100.000% ( 33000, 50000 ] 1 0.000% 100.000% ( 50000, 75000 ] 2 0.000% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% ( 170000, 250000 ] 14 0.000% 100.000% ( 570000, 860000 ] 2 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 192389 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 9790999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 10104310968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 506 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 33650151 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 28850870674 rocksdb.compact.write.bytes COUNT : 28660902498 rocksdb.flush.write.bytes COUNT : 10040690139 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 55 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 275 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 9959706624 rocksdb.memtable.garbage.bytes.at.flush COUNT : 3849216 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 28941570342 rocksdb.non.last.level.read.count COUNT : 6991342 rocksdb.block.checksum.compute.count COUNT : 6990385 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.366457 P95 : 3.843798 P99 : 6.411331 P100 : 814877.000000 COUNT : 9790999 SUM : 55501057 rocksdb.compaction.times.micros P50 : 1011111.111111 P95 : 4520000.000000 P99 : 18049806.000000 P100 : 18049806.000000 COUNT : 78 SUM : 123264702 rocksdb.compaction.times.cpu_micros P50 : 860000.000000 P95 : 3880000.000000 P99 : 15760000.000000 P100 : 16698351.000000 COUNT : 78 SUM : 104555699 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 672.638889 P95 : 1273.538462 P99 : 119517.000000 P100 : 119517.000000 COUNT : 136 SUM : 349166 rocksdb.compaction.outfile.sync.micros P50 : 994.473684 P95 : 99125.000000 P99 : 153650.000000 P100 : 166529.000000 COUNT : 327 SUM : 4826037 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 105.410000 P95 : 1758.571429 P99 : 3767.500000 P100 : 4185.000000 COUNT : 253 SUM : 84788 rocksdb.table.open.io.micros P50 : 1375.000000 P95 : 7561.304348 P99 : 11919.250000 P100 : 15060.000000 COUNT : 506 SUM : 1044038 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.606940 P95 : 7.343221 P99 : 10.050885 P100 : 18402.000000 COUNT : 6988853 SUM : 40269969 rocksdb.read.block.get.micros P50 : 439.130435 P95 : 3441.176471 P99 : 5483.500000 P100 : 7459.000000 COUNT : 1012 SUM : 855334 rocksdb.write.raw.block.micros P50 : 0.512413 P95 : 0.973585 P99 : 2.886822 P100 : 13176.000000 COUNT : 9383040 SUM : 18294015 rocksdb.numfiles.in.singlecompaction P50 : 3.180556 P95 : 5.300000 P99 : 25.960000 P100 : 29.000000 COUNT : 134 SUM : 507 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.556113 P95 : 1177.190331 P99 : 2569.413381 P100 : 814692.000000 COUNT : 210967 SUM : 33661632 rocksdb.sst.read.micros P50 : 1.554446 P95 : 4.309598 P99 : 7.906261 P100 : 18396.000000 COUNT : 6991394 SUM : 25995172 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 9790999 SUM : 10104310968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 211632.653061 P95 : 373066.666667 P99 : 755600.000000 P100 : 958686.000000 COUNT : 136 SUM : 29237333 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 308 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0