[2024-12-10 12:17:35.000356] Starting SPDK v25.01-pre git sha1 52a413487 / DPDK 24.03.0 initialization... [2024-12-10 12:17:35.000621] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid4163016 ] Set seed to 1733829454993631 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-10 12:17:35.128407] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-10 12:17:35.226540] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-10 12:17:35.226581] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 4163016' to capture a snapshot of events at runtime. [2024-12-10 12:17:35.226592] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-10 12:17:35.226602] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-10 12:17:35.226610] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid4163016 for offline analysis/debug. [2024-12-10 12:17:35.227945] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Tue Dec 10 12:17:43 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/10-12:17:51 ... thread 0: (1000000,1000000) ops and (131439.2,131439.2) ops/second in (7.608081,7.608081) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/8 818.85 MB 6.3 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 116.4 7.03 2.42 9 0.781 0 0 0.0 0.0 L1 5/5 372.12 MB 0.0 0.4 0.3 0.1 0.4 0.3 0.0 1.3 101.2 101.1 3.68 1.78 1 3.681 378K 364 0.0 0.0 L2 11/10 1.35 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 39/9 2.51 GB 1.9 2.4 1.5 0.9 2.4 1.5 0.0 1.5 97.4 96.5 25.02 11.54 13 1.925 1020K 0 0.0 0.0 L4 200/12 11.50 GB 1.1 3.0 2.1 0.9 3.0 2.1 0.0 1.4 100.6 99.8 30.42 14.61 21 1.449 1364K 304 0.0 0.0 L5 1084/0 89.01 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1348/44 105.53 GB 0.0 5.7 3.9 1.8 6.5 4.7 0.0 8.1 88.7 100.4 66.15 30.34 44 1.503 2763K 668 0.0 0.0 Int 0/0 0.00 KB 0.0 5.7 3.9 1.8 6.5 4.7 0.0 8.1 88.7 100.4 66.15 30.34 44 1.503 2763K 668 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.7 3.9 1.8 5.7 3.9 0.0 0.0 99.3 98.5 59.12 27.92 35 1.689 2763K 668 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 116.4 7.03 2.42 9 0.781 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.8 total, 7.6 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.49 GB write, 854.35 MB/s write, 5.73 GB read, 755.12 MB/s read, 66.2 seconds Interval compaction: 6.49 GB write, 872.71 MB/s write, 5.73 GB read, 771.35 MB/s read, 66.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 25, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 25, interval: 25 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 129283 Average: 18.7096 StdDev: 203.68 Min: 1 Median: 2.1521 Max: 8689 Percentiles: P50: 2.15 P75: 3.05 P99: 14.56 P99.9: 3516.61 P99.99: 5851.51 ------------------------------------------------------ [ 0, 1 ] 2333 1.805% 1.805% ( 1, 2 ] 56630 43.803% 45.608% ######### ( 2, 3 ] 37342 28.884% 74.492% ###### ( 3, 4 ] 12597 9.744% 84.235% ## ( 4, 6 ] 14043 10.862% 95.098% ## ( 6, 10 ] 4203 3.251% 98.349% # ( 10, 15 ] 925 0.715% 99.064% ( 15, 22 ] 132 0.102% 99.166% ( 22, 34 ] 8 0.006% 99.172% ( 34, 51 ] 8 0.006% 99.179% ( 51, 76 ] 1 0.001% 99.179% ( 76, 110 ] 2 0.002% 99.181% ( 110, 170 ] 4 0.003% 99.184% ( 170, 250 ] 16 0.012% 99.196% ( 250, 380 ] 23 0.018% 99.214% ( 380, 580 ] 57 0.044% 99.258% ( 580, 870 ] 124 0.096% 99.354% ( 870, 1300 ] 175 0.135% 99.489% ( 1300, 1900 ] 230 0.178% 99.667% ( 1900, 2900 ] 239 0.185% 99.852% ( 2900, 4400 ] 155 0.120% 99.972% ( 4400, 6600 ] 38 0.029% 100.002% ( 6600, 9900 ] 3 0.002% 100.004% ** Level 1 read latency histogram (micros): Count: 89636 Average: 18.1497 StdDev: 213.27 Min: 1 Median: 2.1363 Max: 10541 Percentiles: P50: 2.14 P75: 3.08 P99: 13.79 P99.9: 3720.58 P99.99: 6529.34 ------------------------------------------------------ [ 0, 1 ] 1457 1.625% 1.625% ( 1, 2 ] 39931 44.548% 46.173% ######### ( 2, 3 ] 25165 28.075% 74.248% ###### ( 3, 4 ] 8340 9.304% 83.552% ## ( 4, 6 ] 10278 11.466% 95.019% ## ( 6, 10 ] 3021 3.370% 98.389% # ( 10, 15 ] 723 0.807% 99.196% ( 15, 22 ] 82 0.091% 99.287% ( 22, 34 ] 10 0.011% 99.298% ( 34, 51 ] 1 0.001% 99.299% ( 51, 76 ] 4 0.004% 99.304% ( 76, 110 ] 1 0.001% 99.305% ( 110, 170 ] 3 0.003% 99.308% ( 170, 250 ] 2 0.002% 99.311% ( 250, 380 ] 12 0.013% 99.324% ( 380, 580 ] 22 0.025% 99.348% ( 580, 870 ] 54 0.060% 99.409% ( 870, 1300 ] 101 0.113% 99.521% ( 1300, 1900 ] 119 0.133% 99.654% ( 1900, 2900 ] 158 0.176% 99.830% ( 2900, 4400 ] 114 0.127% 99.958% ( 4400, 6600 ] 30 0.033% 99.991% ( 6600, 9900 ] 7 0.008% 99.999% ( 9900, 14000 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 488752 Average: 20.1929 StdDev: 240.62 Min: 1 Median: 2.1883 Max: 41007 Percentiles: P50: 2.19 P75: 3.13 P99: 14.16 P99.9: 2865.85 P99.99: 3812.84 ------------------------------------------------------ [ 0, 1 ] 6391 1.308% 1.308% ( 1, 2 ] 211113 43.194% 44.502% ######### ( 2, 3 ] 142704 29.198% 73.700% ###### ( 3, 4 ] 48092 9.840% 83.539% ## ( 4, 6 ] 55183 11.291% 94.830% ## ( 6, 10 ] 16711 3.419% 98.249% # ( 10, 15 ] 4413 0.903% 99.152% ( 15, 22 ] 601 0.123% 99.275% ( 22, 34 ] 52 0.011% 99.286% ( 34, 51 ] 19 0.004% 99.289% ( 51, 76 ] 21 0.004% 99.294% ( 76, 110 ] 9 0.002% 99.296% ( 110, 170 ] 16 0.003% 99.299% ( 170, 250 ] 25 0.005% 99.304% ( 250, 380 ] 80 0.016% 99.320% ( 380, 580 ] 187 0.038% 99.359% ( 580, 870 ] 354 0.072% 99.431% ( 870, 1300 ] 577 0.118% 99.549% ( 1300, 1900 ] 790 0.162% 99.711% ( 1900, 2900 ] 959 0.196% 99.907% ( 2900, 4400 ] 669 0.137% 100.044% ( 4400, 6600 ] 195 0.040% 100.084% ( 6600, 9900 ] 41 0.008% 100.092% ( 9900, 14000 ] 10 0.002% 100.094% ( 33000, 50000 ] 2 0.000% 100.095% ** Level 3 read latency histogram (micros): Count: 754180 Average: 15.7648 StdDev: 196.05 Min: 1 Median: 2.2215 Max: 41027 Percentiles: P50: 2.22 P75: 3.11 P99: 12.07 P99.9: 1957.00 P99.99: 2593.14 ------------------------------------------------------ [ 0, 1 ] 7820 1.037% 1.037% ( 1, 2 ] 318042 42.171% 43.207% ######## ( 2, 3 ] 231232 30.660% 73.868% ###### ( 3, 4 ] 75572 10.020% 83.888% ## ( 4, 6 ] 87843 11.647% 95.535% ## ( 6, 10 ] 23774 3.152% 98.688% # ( 10, 15 ] 5681 0.753% 99.441% ( 15, 22 ] 714 0.095% 99.536% ( 22, 34 ] 65 0.009% 99.544% ( 34, 51 ] 32 0.004% 99.549% ( 51, 76 ] 31 0.004% 99.553% ( 76, 110 ] 26 0.003% 99.556% ( 110, 170 ] 22 0.003% 99.559% ( 170, 250 ] 43 0.006% 99.565% ( 250, 380 ] 86 0.011% 99.576% ( 380, 580 ] 196 0.026% 99.602% ( 580, 870 ] 463 0.061% 99.663% ( 870, 1300 ] 793 0.105% 99.769% ( 1300, 1900 ] 931 0.123% 99.892% ( 1900, 2900 ] 1067 0.141% 100.034% ( 2900, 4400 ] 782 0.104% 100.137% ( 4400, 6600 ] 223 0.030% 100.167% ( 6600, 9900 ] 36 0.005% 100.172% ( 33000, 50000 ] 1 0.000% 100.172% ** Level 4 read latency histogram (micros): Count: 246072 Average: 36.7072 StdDev: 317.26 Min: 0 Median: 2.1907 Max: 29291 Percentiles: P50: 2.19 P75: 3.12 P99: 1247.18 P99.9: 3923.83 P99.99: 4577.66 ------------------------------------------------------ [ 0, 1 ] 2959 1.202% 1.202% ( 1, 2 ] 106299 43.198% 44.401% ######### ( 2, 3 ] 72242 29.358% 73.759% ###### ( 3, 4 ] 24912 10.124% 83.883% ## ( 4, 6 ] 26331 10.701% 94.583% ## ( 6, 10 ] 7399 3.007% 97.590% # ( 10, 15 ] 1714 0.697% 98.287% ( 15, 22 ] 229 0.093% 98.380% ( 22, 34 ] 17 0.007% 98.387% ( 34, 51 ] 11 0.004% 98.391% ( 51, 76 ] 12 0.005% 98.396% ( 76, 110 ] 11 0.004% 98.400% ( 110, 170 ] 11 0.004% 98.405% ( 170, 250 ] 70 0.028% 98.433% ( 250, 380 ] 92 0.037% 98.471% ( 380, 580 ] 221 0.090% 98.561% ( 580, 870 ] 464 0.189% 98.749% ( 870, 1300 ] 706 0.287% 99.036% ( 1300, 1900 ] 836 0.340% 99.376% ( 1900, 2900 ] 851 0.346% 99.722% ( 2900, 4400 ] 646 0.263% 99.984% ( 4400, 6600 ] 203 0.082% 100.067% ( 6600, 9900 ] 45 0.018% 100.085% ( 22000, 33000 ] 1 0.000% 100.085% ** Level 5 read latency histogram (micros): Count: 9095 Average: 2381.0110 StdDev: 1881.90 Min: 115 Median: 1887.6435 Max: 50141 Percentiles: P50: 1887.64 P75: 3433.53 P99: 8715.36 P99.9: 13141.05 P99.99: 50141.00 ------------------------------------------------------ ( 110, 170 ] 22 0.242% 0.242% ( 170, 250 ] 62 0.682% 0.924% ( 250, 380 ] 207 2.276% 3.200% ( 380, 580 ] 496 5.454% 8.653% # ( 580, 870 ] 930 10.225% 18.879% ## ( 870, 1300 ] 1380 15.173% 34.052% ### ( 1300, 1900 ] 1481 16.284% 50.335% ### ( 1900, 2900 ] 1640 18.032% 68.367% #### ( 2900, 4400 ] 1696 18.648% 87.015% #### ( 4400, 6600 ] 958 10.533% 97.548% ## ( 6600, 9900 ] 206 2.265% 99.813% ( 9900, 14000 ] 10 0.110% 99.923% ( 14000, 22000 ] 2 0.022% 99.945% ( 22000, 33000 ] 3 0.033% 99.978% ( 33000, 50000 ] 1 0.011% 99.989% ( 50000, 75000 ] 1 0.011% 100.000% ** DB Stats ** Uptime(secs): 7.8 total, 7.6 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 126.61 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.762 H:M:S, 48.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 129.31 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.762 H:M:S, 49.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/12/10-12:17:58 ... thread 0: (1000000,2000000) ops and (135310.7,133346.8) ops/second in (7.390399,14.998480) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/9 1.41 GB 12.6 0.7 0.0 0.7 2.5 1.8 0.0 1.4 36.2 131.6 19.14 8.24 23 0.832 703K 892 0.0 0.0 L1 4/4 320.05 MB 0.0 1.1 0.6 0.5 1.1 0.6 0.0 1.7 109.8 109.5 10.23 5.32 2 5.113 1139K 2304 0.0 0.0 L2 1/1 117.19 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 19/3 1.23 GB 0.9 4.2 3.1 1.1 4.2 3.0 0.1 1.4 102.9 102.2 41.74 20.30 18 2.319 2172K 82 0.0 0.0 L4 184/15 10.05 GB 1.0 7.9 4.7 3.1 7.8 4.6 0.2 1.6 107.9 106.7 74.51 38.52 54 1.380 4307K 4286 0.0 0.0 L5 1136/0 93.32 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1355/32 106.44 GB 0.0 13.8 8.4 5.4 15.5 10.1 5.0 8.7 97.2 108.9 145.61 72.37 97 1.501 8322K 7564 0.0 0.0 Int 0/0 0.00 KB 0.0 13.8 8.4 5.4 15.5 10.1 5.0 8.7 97.2 108.9 145.61 72.37 97 1.501 8322K 7564 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.8 8.4 5.4 13.7 8.3 0.0 0.0 107.8 106.9 131.27 67.00 76 1.727 8322K 7564 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 127.4 14.34 5.37 21 0.683 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.2 total, 15.0 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 15.49 GB write, 1045.76 MB/s write, 13.82 GB read, 933.39 MB/s read, 145.6 seconds Interval compaction: 15.49 GB write, 1057.16 MB/s write, 13.82 GB read, 943.57 MB/s read, 145.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 66, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 66, interval: 66 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 502734 Average: 12.3792 StdDev: 162.32 Min: 1 Median: 2.0696 Max: 27640 Percentiles: P50: 2.07 P75: 2.97 P99: 12.28 P99.9: 2573.82 P99.99: 4380.15 ------------------------------------------------------ [ 0, 1 ] 10142 2.017% 2.017% ( 1, 2 ] 231471 46.042% 48.060% ######### ( 2, 3 ] 140209 27.889% 75.949% ###### ( 3, 4 ] 46915 9.332% 85.281% ## ( 4, 6 ] 52853 10.513% 95.794% ## ( 6, 10 ] 14648 2.914% 98.708% # ( 10, 15 ] 3217 0.640% 99.348% ( 15, 22 ] 388 0.077% 99.425% ( 22, 34 ] 20 0.004% 99.429% ( 34, 51 ] 18 0.004% 99.433% ( 51, 76 ] 17 0.003% 99.436% ( 76, 110 ] 6 0.001% 99.437% ( 110, 170 ] 5 0.001% 99.438% ( 170, 250 ] 47 0.009% 99.447% ( 250, 380 ] 110 0.022% 99.469% ( 380, 580 ] 287 0.057% 99.526% ( 580, 870 ] 499 0.099% 99.626% ( 870, 1300 ] 626 0.125% 99.750% ( 1300, 1900 ] 477 0.095% 99.845% ( 1900, 2900 ] 410 0.082% 99.927% ( 2900, 4400 ] 323 0.064% 99.991% ( 4400, 6600 ] 88 0.018% 100.008% ( 6600, 9900 ] 13 0.003% 100.011% ( 9900, 14000 ] 1 0.000% 100.011% ( 22000, 33000 ] 2 0.000% 100.012% ** Level 1 read latency histogram (micros): Count: 200564 Average: 16.9926 StdDev: 207.82 Min: 1 Median: 2.1795 Max: 24280 Percentiles: P50: 2.18 P75: 3.10 P99: 13.98 P99.9: 3428.35 P99.99: 5969.21 ------------------------------------------------------ [ 0, 1 ] 2246 1.120% 1.120% ( 1, 2 ] 87488 43.621% 44.741% ######### ( 2, 3 ] 58761 29.298% 74.039% ###### ( 3, 4 ] 19426 9.686% 83.724% ## ( 4, 6 ] 22574 11.255% 94.980% ## ( 6, 10 ] 6788 3.384% 98.364% # ( 10, 15 ] 1603 0.799% 99.163% ( 15, 22 ] 189 0.094% 99.258% ( 22, 34 ] 17 0.008% 99.266% ( 34, 51 ] 5 0.002% 99.269% ( 51, 76 ] 16 0.008% 99.277% ( 76, 110 ] 2 0.001% 99.278% ( 110, 170 ] 5 0.002% 99.280% ( 170, 250 ] 10 0.005% 99.285% ( 250, 380 ] 39 0.019% 99.304% ( 380, 580 ] 96 0.048% 99.352% ( 580, 870 ] 211 0.105% 99.458% ( 870, 1300 ] 293 0.146% 99.604% ( 1300, 1900 ] 249 0.124% 99.728% ( 1900, 2900 ] 270 0.135% 99.862% ( 2900, 4400 ] 217 0.108% 99.971% ( 4400, 6600 ] 56 0.028% 99.999% ( 6600, 9900 ] 13 0.006% 100.005% ( 9900, 14000 ] 2 0.001% 100.006% ( 14000, 22000 ] 1 0.000% 100.006% ( 22000, 33000 ] 1 0.000% 100.007% ** Level 2 read latency histogram (micros): Count: 641923 Average: 18.5944 StdDev: 226.93 Min: 1 Median: 2.2231 Max: 41007 Percentiles: P50: 2.22 P75: 3.18 P99: 13.93 P99.9: 2858.71 P99.99: 3922.01 ------------------------------------------------------ [ 0, 1 ] 7095 1.105% 1.105% ( 1, 2 ] 271038 42.223% 43.328% ######## ( 2, 3 ] 191979 29.907% 73.235% ###### ( 3, 4 ] 63983 9.967% 83.202% ## ( 4, 6 ] 74308 11.576% 94.778% ## ( 6, 10 ] 22563 3.515% 98.293% # ( 10, 15 ] 5772 0.899% 99.192% ( 15, 22 ] 757 0.118% 99.310% ( 22, 34 ] 64 0.010% 99.320% ( 34, 51 ] 28 0.004% 99.325% ( 51, 76 ] 39 0.006% 99.331% ( 76, 110 ] 12 0.002% 99.332% ( 110, 170 ] 18 0.003% 99.335% ( 170, 250 ] 34 0.005% 99.341% ( 250, 380 ] 112 0.017% 99.358% ( 380, 580 ] 278 0.043% 99.401% ( 580, 870 ] 491 0.076% 99.478% ( 870, 1300 ] 762 0.119% 99.597% ( 1300, 1900 ] 905 0.141% 99.738% ( 1900, 2900 ] 1088 0.169% 99.907% ( 2900, 4400 ] 782 0.122% 100.029% ( 4400, 6600 ] 238 0.037% 100.066% ( 6600, 9900 ] 50 0.008% 100.074% ( 9900, 14000 ] 13 0.002% 100.076% ( 33000, 50000 ] 2 0.000% 100.076% ** Level 3 read latency histogram (micros): Count: 1497749 Average: 13.5991 StdDev: 180.42 Min: 0 Median: 2.2530 Max: 41027 Percentiles: P50: 2.25 P75: 3.19 P99: 11.94 P99.9: 1614.87 P99.99: 2364.22 ------------------------------------------------------ [ 0, 1 ] 12778 0.853% 0.853% ( 1, 2 ] 618841 41.318% 42.171% ######## ( 2, 3 ] 463395 30.939% 73.111% ###### ( 3, 4 ] 152896 10.208% 83.319% ## ( 4, 6 ] 179158 11.962% 95.281% ## ( 6, 10 ] 51051 3.409% 98.689% # ( 10, 15 ] 12004 0.801% 99.491% ( 15, 22 ] 1405 0.094% 99.585% ( 22, 34 ] 113 0.008% 99.592% ( 34, 51 ] 97 0.006% 99.599% ( 51, 76 ] 108 0.007% 99.606% ( 76, 110 ] 43 0.003% 99.609% ( 110, 170 ] 47 0.003% 99.612% ( 170, 250 ] 85 0.006% 99.618% ( 250, 380 ] 214 0.014% 99.632% ( 380, 580 ] 560 0.037% 99.669% ( 580, 870 ] 1170 0.078% 99.747% ( 870, 1300 ] 1563 0.104% 99.852% ( 1300, 1900 ] 1382 0.092% 99.944% ( 1900, 2900 ] 1489 0.099% 100.043% ( 2900, 4400 ] 1252 0.084% 100.127% ( 4400, 6600 ] 359 0.024% 100.151% ( 6600, 9900 ] 68 0.005% 100.156% ( 9900, 14000 ] 12 0.001% 100.156% ( 14000, 22000 ] 5 0.000% 100.157% ( 22000, 33000 ] 1 0.000% 100.157% ( 33000, 50000 ] 1 0.000% 100.157% ** Level 4 read latency histogram (micros): Count: 911946 Average: 24.5687 StdDev: 259.81 Min: 0 Median: 2.2474 Max: 36435 Percentiles: P50: 2.25 P75: 3.23 P99: 504.58 P99.9: 2804.65 P99.99: 3628.98 ------------------------------------------------------ [ 0, 1 ] 8169 0.896% 0.896% ( 1, 2 ] 380059 41.676% 42.571% ######## ( 2, 3 ] 273832 30.027% 72.599% ###### ( 3, 4 ] 93556 10.259% 82.858% ## ( 4, 6 ] 106401 11.667% 94.525% ## ( 6, 10 ] 31197 3.421% 97.946% # ( 10, 15 ] 7200 0.790% 98.735% ( 15, 22 ] 790 0.087% 98.822% ( 22, 34 ] 66 0.007% 98.829% ( 34, 51 ] 73 0.008% 98.837% ( 51, 76 ] 104 0.011% 98.849% ( 76, 110 ] 27 0.003% 98.852% ( 110, 170 ] 42 0.005% 98.856% ( 170, 250 ] 232 0.025% 98.882% ( 250, 380 ] 417 0.046% 98.927% ( 380, 580 ] 1062 0.116% 99.044% ( 580, 870 ] 2009 0.220% 99.264% ( 870, 1300 ] 2428 0.266% 99.530% ( 1300, 1900 ] 1900 0.208% 99.739% ( 1900, 2900 ] 1625 0.178% 99.917% ( 2900, 4400 ] 1370 0.150% 100.067% ( 4400, 6600 ] 401 0.044% 100.111% ( 6600, 9900 ] 81 0.009% 100.120% ( 9900, 14000 ] 10 0.001% 100.121% ( 14000, 22000 ] 4 0.000% 100.122% ( 22000, 33000 ] 7 0.001% 100.122% ( 33000, 50000 ] 3 0.000% 100.123% ** Level 5 read latency histogram (micros): Count: 24739 Average: 1779.9372 StdDev: 1682.00 Min: 100 Median: 1221.1230 Max: 50141 Percentiles: P50: 1221.12 P75: 2439.75 P99: 7257.53 P99.9: 13333.48 P99.99: 43735.92 ------------------------------------------------------ ( 76, 110 ] 2 0.008% 0.008% ( 110, 170 ] 72 0.291% 0.299% ( 170, 250 ] 250 1.011% 1.310% ( 250, 380 ] 959 3.876% 5.186% # ( 380, 580 ] 2485 10.045% 15.231% ## ( 580, 870 ] 4446 17.972% 33.203% #### ( 870, 1300 ] 5089 20.571% 53.773% #### ( 1300, 1900 ] 3605 14.572% 68.346% ### ( 1900, 2900 ] 3050 12.329% 80.674% ## ( 2900, 4400 ] 3139 12.688% 93.363% ### ( 4400, 6600 ] 1344 5.433% 98.795% # ( 6600, 9900 ] 254 1.027% 99.822% ( 9900, 14000 ] 23 0.093% 99.915% ( 14000, 22000 ] 6 0.024% 99.939% ( 22000, 33000 ] 10 0.040% 99.980% ( 33000, 50000 ] 4 0.016% 99.996% ( 50000, 75000 ] 1 0.004% 100.000% ** DB Stats ** Uptime(secs): 15.2 total, 7.4 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 129.81 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.219 H:M:S, 47.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 133.17 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.457 H:M:S, 46.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 6 num-running-flushes: 1 2024/12/10-12:18:02 ... thread 0: (1000000,3000000) ops and (268633.3,160247.6) ops/second in (3.722547,18.721027) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/9 2.40 GB 125.8 1.8 0.0 1.8 4.6 2.8 0.0 1.7 68.4 171.0 27.61 15.81 42 0.657 1918K 2237 0.0 0.0 L1 4/4 320.05 MB 0.0 1.1 0.6 0.5 1.1 0.6 0.0 1.7 109.8 109.5 10.23 5.32 2 5.113 1139K 2304 0.0 0.0 L3 14/0 860.42 MB 0.9 4.3 3.2 1.1 4.3 3.1 0.1 1.3 103.6 102.9 42.61 20.85 19 2.243 2291K 82 0.0 0.0 L4 175/0 9.56 GB 1.0 8.7 5.0 3.7 8.6 4.9 0.4 1.7 110.1 108.8 81.05 42.74 57 1.422 4678K 4286 0.0 0.0 L5 1148/0 94.31 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1355/13 107.41 GB 0.0 16.0 8.9 7.1 18.6 11.5 5.8 6.7 101.3 117.9 161.51 84.72 120 1.346 10M 8909 0.0 0.0 Int 0/0 0.00 KB 0.0 16.0 8.9 7.1 18.6 11.5 5.8 6.7 101.3 117.9 161.51 84.72 120 1.346 10M 8909 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 16.0 8.9 7.1 15.8 8.7 0.0 0.0 113.8 112.8 143.72 76.51 84 1.711 10M 8909 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 159.4 17.79 8.22 36 0.494 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 18.9 total, 18.7 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 18.60 GB write, 1008.62 MB/s write, 15.97 GB read, 865.88 MB/s read, 161.5 seconds Interval compaction: 18.60 GB write, 1017.43 MB/s write, 15.97 GB read, 873.44 MB/s read, 161.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 73, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 73, interval: 73 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 866433 Average: 11.1176 StdDev: 128.95 Min: 1 Median: 1.8336 Max: 27640 Percentiles: P50: 1.83 P75: 2.72 P99: 172.99 P99.9: 1498.67 P99.99: 2990.88 ------------------------------------------------------ [ 0, 1 ] 59692 6.889% 6.889% # ( 1, 2 ] 448102 51.718% 58.607% ########## ( 2, 3 ] 197741 22.822% 81.430% ##### ( 3, 4 ] 61955 7.151% 88.580% # ( 4, 6 ] 63986 7.385% 95.965% # ( 6, 10 ] 17763 2.050% 98.016% ( 10, 15 ] 3734 0.431% 98.447% ( 15, 22 ] 494 0.057% 98.504% ( 22, 34 ] 36 0.004% 98.508% ( 34, 51 ] 24 0.003% 98.510% ( 51, 76 ] 22 0.003% 98.513% ( 76, 110 ] 346 0.040% 98.553% ( 110, 170 ] 3748 0.433% 98.985% ( 170, 250 ] 3393 0.392% 99.377% ( 250, 380 ] 1491 0.172% 99.549% ( 380, 580 ] 1292 0.149% 99.698% ( 580, 870 ] 870 0.100% 99.799% ( 870, 1300 ] 714 0.082% 99.881% ( 1300, 1900 ] 497 0.057% 99.938% ( 1900, 2900 ] 427 0.049% 99.988% ( 2900, 4400 ] 336 0.039% 100.027% ( 4400, 6600 ] 88 0.010% 100.037% ( 6600, 9900 ] 14 0.002% 100.038% ( 9900, 14000 ] 1 0.000% 100.038% ( 22000, 33000 ] 2 0.000% 100.039% ** Level 1 read latency histogram (micros): Count: 242262 Average: 15.5556 StdDev: 192.47 Min: 1 Median: 2.1916 Max: 24280 Percentiles: P50: 2.19 P75: 3.15 P99: 15.96 P99.9: 3177.02 P99.99: 5703.55 ------------------------------------------------------ [ 0, 1 ] 3006 1.241% 1.241% ( 1, 2 ] 104611 43.181% 44.422% ######### ( 2, 3 ] 70532 29.114% 73.536% ###### ( 3, 4 ] 23469 9.687% 83.223% ## ( 4, 6 ] 27905 11.519% 94.742% ## ( 6, 10 ] 8370 3.455% 98.197% # ( 10, 15 ] 1906 0.787% 98.983% ( 15, 22 ] 294 0.121% 99.105% ( 22, 34 ] 27 0.011% 99.116% ( 34, 51 ] 14 0.006% 99.122% ( 51, 76 ] 17 0.007% 99.129% ( 76, 110 ] 28 0.012% 99.140% ( 110, 170 ] 241 0.099% 99.240% ( 170, 250 ] 195 0.080% 99.320% ( 250, 380 ] 121 0.050% 99.370% ( 380, 580 ] 167 0.069% 99.439% ( 580, 870 ] 243 0.100% 99.539% ( 870, 1300 ] 307 0.127% 99.666% ( 1300, 1900 ] 253 0.104% 99.770% ( 1900, 2900 ] 272 0.112% 99.883% ( 2900, 4400 ] 226 0.093% 99.976% ( 4400, 6600 ] 57 0.024% 100.000% ( 6600, 9900 ] 14 0.006% 100.005% ( 9900, 14000 ] 2 0.001% 100.006% ( 14000, 22000 ] 1 0.000% 100.007% ( 22000, 33000 ] 1 0.000% 100.007% ** Level 2 read latency histogram (micros): Count: 641923 Average: 18.5944 StdDev: 226.93 Min: 1 Median: 2.2231 Max: 41007 Percentiles: P50: 2.22 P75: 3.18 P99: 13.93 P99.9: 2858.71 P99.99: 3922.01 ------------------------------------------------------ [ 0, 1 ] 7095 1.105% 1.105% ( 1, 2 ] 271038 42.223% 43.328% ######## ( 2, 3 ] 191979 29.907% 73.235% ###### ( 3, 4 ] 63983 9.967% 83.202% ## ( 4, 6 ] 74308 11.576% 94.778% ## ( 6, 10 ] 22563 3.515% 98.293% # ( 10, 15 ] 5772 0.899% 99.192% ( 15, 22 ] 757 0.118% 99.310% ( 22, 34 ] 64 0.010% 99.320% ( 34, 51 ] 28 0.004% 99.325% ( 51, 76 ] 39 0.006% 99.331% ( 76, 110 ] 12 0.002% 99.332% ( 110, 170 ] 18 0.003% 99.335% ( 170, 250 ] 34 0.005% 99.341% ( 250, 380 ] 112 0.017% 99.358% ( 380, 580 ] 278 0.043% 99.401% ( 580, 870 ] 491 0.076% 99.478% ( 870, 1300 ] 762 0.119% 99.597% ( 1300, 1900 ] 905 0.141% 99.738% ( 1900, 2900 ] 1088 0.169% 99.907% ( 2900, 4400 ] 782 0.122% 100.029% ( 4400, 6600 ] 238 0.037% 100.066% ( 6600, 9900 ] 50 0.008% 100.074% ( 9900, 14000 ] 13 0.002% 100.076% ( 33000, 50000 ] 2 0.000% 100.076% ** Level 3 read latency histogram (micros): Count: 1505310 Average: 13.6605 StdDev: 180.43 Min: 0 Median: 2.2534 Max: 41027 Percentiles: P50: 2.25 P75: 3.19 P99: 12.16 P99.9: 1613.08 P99.99: 2364.43 ------------------------------------------------------ [ 0, 1 ] 12837 0.853% 0.853% ( 1, 2 ] 621804 41.307% 42.160% ######## ( 2, 3 ] 465684 30.936% 73.096% ###### ( 3, 4 ] 153533 10.199% 83.296% ## ( 4, 6 ] 179923 11.953% 95.248% ## ( 6, 10 ] 51266 3.406% 98.654% # ( 10, 15 ] 12046 0.800% 99.454% ( 15, 22 ] 1408 0.094% 99.548% ( 22, 34 ] 113 0.008% 99.555% ( 34, 51 ] 97 0.006% 99.562% ( 51, 76 ] 108 0.007% 99.569% ( 76, 110 ] 73 0.005% 99.574% ( 110, 170 ] 252 0.017% 99.590% ( 170, 250 ] 256 0.017% 99.607% ( 250, 380 ] 296 0.020% 99.627% ( 380, 580 ] 620 0.041% 99.668% ( 580, 870 ] 1196 0.079% 99.748% ( 870, 1300 ] 1570 0.104% 99.852% ( 1300, 1900 ] 1385 0.092% 99.944% ( 1900, 2900 ] 1491 0.099% 100.043% ( 2900, 4400 ] 1252 0.083% 100.126% ( 4400, 6600 ] 359 0.024% 100.150% ( 6600, 9900 ] 69 0.005% 100.155% ( 9900, 14000 ] 13 0.001% 100.156% ( 14000, 22000 ] 5 0.000% 100.156% ( 22000, 33000 ] 1 0.000% 100.156% ( 33000, 50000 ] 1 0.000% 100.156% ** Level 4 read latency histogram (micros): Count: 961900 Average: 25.1819 StdDev: 254.98 Min: 0 Median: 2.2489 Max: 36435 Percentiles: P50: 2.25 P75: 3.25 P99: 522.81 P99.9: 2762.51 P99.99: 3593.44 ------------------------------------------------------ [ 0, 1 ] 8539 0.888% 0.888% ( 1, 2 ] 400832 41.671% 42.559% ######## ( 2, 3 ] 287570 29.896% 72.455% ###### ( 3, 4 ] 97218 10.107% 82.561% ## ( 4, 6 ] 110206 11.457% 94.019% ## ( 6, 10 ] 32416 3.370% 97.389% # ( 10, 15 ] 7463 0.776% 98.164% ( 15, 22 ] 817 0.085% 98.249% ( 22, 34 ] 67 0.007% 98.256% ( 34, 51 ] 73 0.008% 98.264% ( 51, 76 ] 106 0.011% 98.275% ( 76, 110 ] 231 0.024% 98.299% ( 110, 170 ] 2244 0.233% 98.532% ( 170, 250 ] 2082 0.216% 98.749% ( 250, 380 ] 1226 0.127% 98.876% ( 380, 580 ] 1668 0.173% 99.050% ( 580, 870 ] 2326 0.242% 99.291% ( 870, 1300 ] 2517 0.262% 99.553% ( 1300, 1900 ] 1920 0.200% 99.753% ( 1900, 2900 ] 1643 0.171% 99.923% ( 2900, 4400 ] 1384 0.144% 100.067% ( 4400, 6600 ] 401 0.042% 100.109% ( 6600, 9900 ] 83 0.009% 100.118% ( 9900, 14000 ] 10 0.001% 100.119% ( 14000, 22000 ] 4 0.000% 100.119% ( 22000, 33000 ] 7 0.001% 100.120% ( 33000, 50000 ] 3 0.000% 100.120% ** Level 5 read latency histogram (micros): Count: 62332 Average: 863.1732 StdDev: 1310.05 Min: 84 Median: 342.3207 Max: 50141 Percentiles: P50: 342.32 P75: 992.85 P99: 6075.51 P99.9: 9688.64 P99.99: 30543.48 ------------------------------------------------------ ( 76, 110 ] 1340 2.150% 2.150% ( 110, 170 ] 13759 22.074% 24.224% #### ( 170, 250 ] 11838 18.992% 43.215% #### ( 250, 380 ] 5955 9.554% 52.769% ## ( 380, 580 ] 6277 10.070% 62.839% ## ( 580, 870 ] 6005 9.634% 72.473% ## ( 870, 1300 ] 5513 8.845% 81.318% ## ( 1300, 1900 ] 3693 5.925% 87.243% # ( 1900, 2900 ] 3112 4.993% 92.235% # ( 2900, 4400 ] 3187 5.113% 97.348% # ( 4400, 6600 ] 1352 2.169% 99.517% ( 6600, 9900 ] 255 0.409% 99.926% ( 9900, 14000 ] 26 0.042% 99.968% ( 14000, 22000 ] 6 0.010% 99.978% ( 22000, 33000 ] 10 0.016% 99.994% ( 33000, 50000 ] 4 0.006% 100.000% ( 50000, 75000 ] 1 0.002% 100.002% ** DB Stats ** Uptime(secs): 18.9 total, 3.7 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 156.33 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.493 H:M:S, 39.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 264.39 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.274 H:M:S, 7.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/10-12:18:05 ... thread 0: (1000000,4000000) ops and (283420.8,179780.5) ops/second in (3.528323,22.249350) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/9 3.38 GB 226.5 2.8 0.0 2.8 6.6 3.7 0.0 1.8 83.1 193.2 34.88 22.49 62 0.563 2942K 3189 0.0 0.0 L1 4/4 320.05 MB 0.0 1.1 0.6 0.5 1.1 0.6 0.0 1.7 109.8 109.5 10.23 5.32 2 5.113 1139K 2304 0.0 0.0 L3 14/0 860.42 MB 0.9 4.3 3.2 1.1 4.3 3.1 0.1 1.3 103.6 102.9 42.61 20.85 19 2.243 2291K 82 0.0 0.0 L4 175/0 9.56 GB 1.0 8.7 5.0 3.7 8.6 4.9 0.4 1.7 110.1 108.8 81.05 42.74 57 1.422 4678K 4286 0.0 0.0 L5 1148/0 94.31 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1359/13 108.40 GB 0.0 17.0 8.9 8.1 20.6 12.5 5.8 5.5 102.9 124.8 168.77 91.40 140 1.206 11M 9861 0.0 0.0 Int 0/0 0.00 KB 0.0 17.0 8.9 8.1 20.6 12.5 5.8 5.5 102.9 124.8 168.77 91.40 140 1.206 11M 9861 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 17.0 8.9 8.1 16.8 8.7 0.0 0.0 117.6 116.6 147.69 80.37 88 1.678 11M 9861 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 182.3 21.08 11.03 52 0.405 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 22.4 total, 22.3 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 20.57 GB write, 939.75 MB/s write, 16.95 GB read, 774.55 MB/s read, 168.8 seconds Interval compaction: 20.57 GB write, 946.66 MB/s write, 16.95 GB read, 780.24 MB/s read, 168.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 1, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 1, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 73, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 1, total-stops: 73, interval: 74 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1156141 Average: 11.7768 StdDev: 115.88 Min: 1 Median: 1.7482 Max: 27640 Percentiles: P50: 1.75 P75: 2.58 P99: 222.44 P99.9: 1134.20 P99.99: 2436.91 ------------------------------------------------------ [ 0, 1 ] 116318 10.061% 10.061% ## ( 1, 2 ] 617146 53.380% 63.441% ########### ( 2, 3 ] 228926 19.801% 83.242% #### ( 3, 4 ] 71834 6.213% 89.455% # ( 4, 6 ] 70549 6.102% 95.557% # ( 6, 10 ] 19184 1.659% 97.216% ( 10, 15 ] 4000 0.346% 97.562% ( 15, 22 ] 562 0.049% 97.611% ( 22, 34 ] 53 0.005% 97.615% ( 34, 51 ] 27 0.002% 97.618% ( 51, 76 ] 26 0.002% 97.620% ( 76, 110 ] 899 0.078% 97.698% ( 110, 170 ] 9600 0.830% 98.528% ( 170, 250 ] 8324 0.720% 99.248% ( 250, 380 ] 3252 0.281% 99.529% ( 380, 580 ] 2494 0.216% 99.745% ( 580, 870 ] 1312 0.113% 99.859% ( 870, 1300 ] 781 0.068% 99.926% ( 1300, 1900 ] 501 0.043% 99.969% ( 1900, 2900 ] 444 0.038% 100.008% ( 2900, 4400 ] 339 0.029% 100.037% ( 4400, 6600 ] 88 0.008% 100.045% ( 6600, 9900 ] 14 0.001% 100.046% ( 9900, 14000 ] 1 0.000% 100.046% ( 22000, 33000 ] 2 0.000% 100.046% ** Level 1 read latency histogram (micros): Count: 257135 Average: 15.4159 StdDev: 187.26 Min: 1 Median: 2.2552 Max: 24280 Percentiles: P50: 2.26 P75: 3.33 P99: 117.99 P99.9: 3071.67 P99.99: 5607.55 ------------------------------------------------------ [ 0, 1 ] 3053 1.187% 1.187% ( 1, 2 ] 106438 41.394% 42.581% ######## ( 2, 3 ] 74753 29.071% 71.653% ###### ( 3, 4 ] 26135 10.164% 81.817% ## ( 4, 6 ] 32088 12.479% 94.296% ## ( 6, 10 ] 9330 3.628% 97.924% # ( 10, 15 ] 2146 0.835% 98.759% ( 15, 22 ] 422 0.164% 98.923% ( 22, 34 ] 45 0.018% 98.940% ( 34, 51 ] 14 0.005% 98.946% ( 51, 76 ] 17 0.007% 98.952% ( 76, 110 ] 61 0.024% 98.976% ( 110, 170 ] 463 0.180% 99.156% ( 170, 250 ] 386 0.150% 99.306% ( 250, 380 ] 189 0.074% 99.380% ( 380, 580 ] 220 0.086% 99.465% ( 580, 870 ] 258 0.100% 99.566% ( 870, 1300 ] 308 0.120% 99.685% ( 1300, 1900 ] 253 0.098% 99.784% ( 1900, 2900 ] 273 0.106% 99.890% ( 2900, 4400 ] 226 0.088% 99.978% ( 4400, 6600 ] 57 0.022% 100.000% ( 6600, 9900 ] 14 0.005% 100.005% ( 9900, 14000 ] 2 0.001% 100.006% ( 14000, 22000 ] 1 0.000% 100.007% ( 22000, 33000 ] 1 0.000% 100.007% ** Level 2 read latency histogram (micros): Count: 641923 Average: 18.5944 StdDev: 226.93 Min: 1 Median: 2.2231 Max: 41007 Percentiles: P50: 2.22 P75: 3.18 P99: 13.93 P99.9: 2858.71 P99.99: 3922.01 ------------------------------------------------------ [ 0, 1 ] 7095 1.105% 1.105% ( 1, 2 ] 271038 42.223% 43.328% ######## ( 2, 3 ] 191979 29.907% 73.235% ###### ( 3, 4 ] 63983 9.967% 83.202% ## ( 4, 6 ] 74308 11.576% 94.778% ## ( 6, 10 ] 22563 3.515% 98.293% # ( 10, 15 ] 5772 0.899% 99.192% ( 15, 22 ] 757 0.118% 99.310% ( 22, 34 ] 64 0.010% 99.320% ( 34, 51 ] 28 0.004% 99.325% ( 51, 76 ] 39 0.006% 99.331% ( 76, 110 ] 12 0.002% 99.332% ( 110, 170 ] 18 0.003% 99.335% ( 170, 250 ] 34 0.005% 99.341% ( 250, 380 ] 112 0.017% 99.358% ( 380, 580 ] 278 0.043% 99.401% ( 580, 870 ] 491 0.076% 99.478% ( 870, 1300 ] 762 0.119% 99.597% ( 1300, 1900 ] 905 0.141% 99.738% ( 1900, 2900 ] 1088 0.169% 99.907% ( 2900, 4400 ] 782 0.122% 100.029% ( 4400, 6600 ] 238 0.037% 100.066% ( 6600, 9900 ] 50 0.008% 100.074% ( 9900, 14000 ] 13 0.002% 100.076% ( 33000, 50000 ] 2 0.000% 100.076% ** Level 3 read latency histogram (micros): Count: 1506025 Average: 13.7414 StdDev: 180.46 Min: 0 Median: 2.2542 Max: 41027 Percentiles: P50: 2.25 P75: 3.19 P99: 12.41 P99.9: 1612.77 P99.99: 2364.38 ------------------------------------------------------ [ 0, 1 ] 12837 0.852% 0.852% ( 1, 2 ] 621811 41.288% 42.141% ######## ( 2, 3 ] 465721 30.924% 73.064% ###### ( 3, 4 ] 153553 10.196% 83.260% ## ( 4, 6 ] 179972 11.950% 95.211% ## ( 6, 10 ] 51271 3.404% 98.615% # ( 10, 15 ] 12048 0.800% 99.415% ( 15, 22 ] 1408 0.093% 99.508% ( 22, 34 ] 113 0.008% 99.516% ( 34, 51 ] 97 0.006% 99.522% ( 51, 76 ] 108 0.007% 99.529% ( 76, 110 ] 97 0.006% 99.536% ( 110, 170 ] 514 0.034% 99.570% ( 170, 250 ] 435 0.029% 99.599% ( 250, 380 ] 368 0.024% 99.623% ( 380, 580 ] 655 0.043% 99.667% ( 580, 870 ] 1216 0.081% 99.748% ( 870, 1300 ] 1573 0.104% 99.852% ( 1300, 1900 ] 1385 0.092% 99.944% ( 1900, 2900 ] 1491 0.099% 100.043% ( 2900, 4400 ] 1252 0.083% 100.126% ( 4400, 6600 ] 359 0.024% 100.150% ( 6600, 9900 ] 69 0.005% 100.155% ( 9900, 14000 ] 13 0.001% 100.155% ( 14000, 22000 ] 5 0.000% 100.156% ( 22000, 33000 ] 1 0.000% 100.156% ( 33000, 50000 ] 1 0.000% 100.156% ** Level 4 read latency histogram (micros): Count: 969238 Average: 26.6060 StdDev: 254.86 Min: 0 Median: 2.2616 Max: 36435 Percentiles: P50: 2.26 P75: 3.31 P99: 550.03 P99.9: 2758.39 P99.99: 3592.64 ------------------------------------------------------ [ 0, 1 ] 8539 0.881% 0.881% ( 1, 2 ] 400838 41.356% 42.237% ######## ( 2, 3 ] 287659 29.679% 71.916% ###### ( 3, 4 ] 97279 10.037% 81.953% ## ( 4, 6 ] 110302 11.380% 93.333% ## ( 6, 10 ] 32435 3.346% 96.679% # ( 10, 15 ] 7467 0.770% 97.450% ( 15, 22 ] 818 0.084% 97.534% ( 22, 34 ] 68 0.007% 97.541% ( 34, 51 ] 73 0.008% 97.549% ( 51, 76 ] 106 0.011% 97.560% ( 76, 110 ] 517 0.053% 97.613% ( 110, 170 ] 5324 0.549% 98.162% ( 170, 250 ] 4237 0.437% 98.599% ( 250, 380 ] 1981 0.204% 98.804% ( 380, 580 ] 2238 0.231% 99.035% ( 580, 870 ] 2507 0.259% 99.293% ( 870, 1300 ] 2540 0.262% 99.555% ( 1300, 1900 ] 1927 0.199% 99.754% ( 1900, 2900 ] 1647 0.170% 99.924% ( 2900, 4400 ] 1384 0.143% 100.067% ( 4400, 6600 ] 401 0.041% 100.108% ( 6600, 9900 ] 83 0.009% 100.117% ( 9900, 14000 ] 10 0.001% 100.118% ( 14000, 22000 ] 4 0.000% 100.118% ( 22000, 33000 ] 7 0.001% 100.119% ( 33000, 50000 ] 3 0.000% 100.119% ** Level 5 read latency histogram (micros): Count: 102681 Average: 609.9320 StdDev: 1071.83 Min: 83 Median: 225.7353 Max: 50141 Percentiles: P50: 225.74 P75: 551.38 P99: 5418.95 P99.9: 9166.48 P99.99: 26105.09 ------------------------------------------------------ ( 76, 110 ] 3059 2.979% 2.979% # ( 110, 170 ] 30913 30.106% 33.085% ###### ( 170, 250 ] 24930 24.279% 57.364% ##### ( 250, 380 ] 10144 9.879% 67.243% ## ( 380, 580 ] 9295 9.052% 76.296% ## ( 580, 870 ] 7009 6.826% 83.122% # ( 870, 1300 ] 5647 5.500% 88.621% # ( 1300, 1900 ] 3712 3.615% 92.236% # ( 1900, 2900 ] 3130 3.048% 95.284% # ( 2900, 4400 ] 3189 3.106% 98.390% # ( 4400, 6600 ] 1352 1.317% 99.707% ( 6600, 9900 ] 255 0.248% 99.955% ( 9900, 14000 ] 26 0.025% 99.981% ( 14000, 22000 ] 6 0.006% 99.986% ( 22000, 33000 ] 10 0.010% 99.996% ( 33000, 50000 ] 4 0.004% 100.000% ( 50000, 75000 ] 1 0.001% 100.001% ** DB Stats ** Uptime(secs): 22.4 total, 3.5 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 175.63 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.601 H:M:S, 33.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 278.94 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.108 H:M:S, 3.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/10-12:18:11 ... thread 0: (1000000,5000000) ops and (185789.5,180951.0) ops/second in (5.382436,27.631786) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/17 3.44 GB 0.0 3.8 0.0 3.8 8.5 4.7 0.0 1.8 92.9 208.3 42.03 29.07 82 0.513 3965K 4155 0.0 0.0 L1 13/13 1.23 GB 0.0 2.3 1.6 0.8 2.3 1.6 0.0 1.5 95.3 95.0 25.04 12.99 3 8.346 2423K 8839 0.0 0.0 L3 14/0 860.42 MB 0.9 4.3 3.2 1.1 4.3 3.1 0.1 1.3 103.6 102.9 42.61 20.85 19 2.243 2291K 82 0.0 0.0 L4 175/0 9.56 GB 1.0 8.7 5.0 3.7 8.6 4.9 0.4 1.7 110.1 108.8 81.05 42.74 57 1.422 4678K 4286 0.0 0.0 L5 1148/0 94.31 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1367/30 109.37 GB 0.0 19.2 9.8 9.4 23.8 14.4 5.8 5.0 102.9 127.6 190.73 105.66 161 1.185 13M 17K 0.0 0.0 Int 0/0 0.00 KB 0.0 19.2 9.8 9.4 23.8 14.4 5.8 5.0 102.9 127.6 190.73 105.66 161 1.185 13M 17K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 19.2 9.8 9.4 19.0 9.6 0.0 0.0 118.0 117.1 166.39 91.86 93 1.789 13M 17K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 199.3 24.34 13.80 68 0.358 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 27.8 total, 27.6 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 23.77 GB write, 875.50 MB/s write, 19.17 GB read, 706.31 MB/s read, 190.7 seconds Interval compaction: 23.77 GB write, 880.69 MB/s write, 19.17 GB read, 710.49 MB/s read, 190.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 10, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 10, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 73, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 10, total-stops: 73, interval: 83 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1440319 Average: 13.1985 StdDev: 108.32 Min: 0 Median: 1.7044 Max: 27640 Percentiles: P50: 1.70 P75: 2.52 P99: 239.73 P99.9: 932.65 P99.99: 2050.59 ------------------------------------------------------ [ 0, 1 ] 176830 12.277% 12.277% ## ( 1, 2 ] 771336 53.553% 65.830% ########### ( 2, 3 ] 255611 17.747% 83.577% #### ( 3, 4 ] 82806 5.749% 89.326% # ( 4, 6 ] 77006 5.346% 94.673% # ( 6, 10 ] 20620 1.432% 96.104% ( 10, 15 ] 4276 0.297% 96.401% ( 15, 22 ] 625 0.043% 96.445% ( 22, 34 ] 69 0.005% 96.449% ( 34, 51 ] 29 0.002% 96.451% ( 51, 76 ] 27 0.002% 96.453% ( 76, 110 ] 1958 0.136% 96.589% ( 110, 170 ] 20587 1.429% 98.019% ( 170, 250 ] 16217 1.126% 99.144% ( 250, 380 ] 5320 0.369% 99.514% ( 380, 580 ] 3735 0.259% 99.773% ( 580, 870 ] 1704 0.118% 99.891% ( 870, 1300 ] 842 0.058% 99.950% ( 1300, 1900 ] 508 0.035% 99.985% ( 1900, 2900 ] 458 0.032% 100.017% ( 2900, 4400 ] 346 0.024% 100.041% ( 4400, 6600 ] 89 0.006% 100.047% ( 6600, 9900 ] 14 0.001% 100.048% ( 9900, 14000 ] 1 0.000% 100.048% ( 22000, 33000 ] 2 0.000% 100.048% ** Level 1 read latency histogram (micros): Count: 282395 Average: 15.1494 StdDev: 179.46 Min: 1 Median: 2.2884 Max: 24280 Percentiles: P50: 2.29 P75: 3.47 P99: 157.25 P99.9: 2904.02 P99.99: 5510.05 ------------------------------------------------------ [ 0, 1 ] 4146 1.468% 1.468% ( 1, 2 ] 114106 40.407% 41.875% ######## ( 2, 3 ] 79560 28.173% 70.048% ###### ( 3, 4 ] 29680 10.510% 80.558% ## ( 4, 6 ] 37632 13.326% 93.884% ### ( 6, 10 ] 10457 3.703% 97.587% # ( 10, 15 ] 2391 0.847% 98.434% ( 15, 22 ] 571 0.202% 98.636% ( 22, 34 ] 63 0.022% 98.658% ( 34, 51 ] 14 0.005% 98.663% ( 51, 76 ] 17 0.006% 98.669% ( 76, 110 ] 145 0.051% 98.721% ( 110, 170 ] 1002 0.355% 99.075% ( 170, 250 ] 636 0.225% 99.301% ( 250, 380 ] 282 0.100% 99.400% ( 380, 580 ] 281 0.100% 99.500% ( 580, 870 ] 276 0.098% 99.598% ( 870, 1300 ] 321 0.114% 99.711% ( 1300, 1900 ] 259 0.092% 99.803% ( 1900, 2900 ] 273 0.097% 99.900% ( 2900, 4400 ] 226 0.080% 99.980% ( 4400, 6600 ] 57 0.020% 100.000% ( 6600, 9900 ] 14 0.005% 100.005% ( 9900, 14000 ] 2 0.001% 100.006% ( 14000, 22000 ] 1 0.000% 100.006% ( 22000, 33000 ] 1 0.000% 100.006% ** Level 2 read latency histogram (micros): Count: 641923 Average: 18.5944 StdDev: 226.93 Min: 1 Median: 2.2231 Max: 41007 Percentiles: P50: 2.22 P75: 3.18 P99: 13.93 P99.9: 2858.71 P99.99: 3922.01 ------------------------------------------------------ [ 0, 1 ] 7095 1.105% 1.105% ( 1, 2 ] 271038 42.223% 43.328% ######## ( 2, 3 ] 191979 29.907% 73.235% ###### ( 3, 4 ] 63983 9.967% 83.202% ## ( 4, 6 ] 74308 11.576% 94.778% ## ( 6, 10 ] 22563 3.515% 98.293% # ( 10, 15 ] 5772 0.899% 99.192% ( 15, 22 ] 757 0.118% 99.310% ( 22, 34 ] 64 0.010% 99.320% ( 34, 51 ] 28 0.004% 99.325% ( 51, 76 ] 39 0.006% 99.331% ( 76, 110 ] 12 0.002% 99.332% ( 110, 170 ] 18 0.003% 99.335% ( 170, 250 ] 34 0.005% 99.341% ( 250, 380 ] 112 0.017% 99.358% ( 380, 580 ] 278 0.043% 99.401% ( 580, 870 ] 491 0.076% 99.478% ( 870, 1300 ] 762 0.119% 99.597% ( 1300, 1900 ] 905 0.141% 99.738% ( 1900, 2900 ] 1088 0.169% 99.907% ( 2900, 4400 ] 782 0.122% 100.029% ( 4400, 6600 ] 238 0.037% 100.066% ( 6600, 9900 ] 50 0.008% 100.074% ( 9900, 14000 ] 13 0.002% 100.076% ( 33000, 50000 ] 2 0.000% 100.076% ** Level 3 read latency histogram (micros): Count: 1507328 Average: 13.8977 StdDev: 180.48 Min: 0 Median: 2.2556 Max: 41027 Percentiles: P50: 2.26 P75: 3.20 P99: 12.94 P99.9: 1612.20 P99.99: 2364.30 ------------------------------------------------------ [ 0, 1 ] 12837 0.852% 0.852% ( 1, 2 ] 621811 41.253% 42.104% ######## ( 2, 3 ] 465722 30.897% 73.001% ###### ( 3, 4 ] 153553 10.187% 83.188% ## ( 4, 6 ] 179972 11.940% 95.128% ## ( 6, 10 ] 51271 3.401% 98.530% # ( 10, 15 ] 12048 0.799% 99.329% ( 15, 22 ] 1408 0.093% 99.422% ( 22, 34 ] 113 0.007% 99.430% ( 34, 51 ] 97 0.006% 99.436% ( 51, 76 ] 108 0.007% 99.444% ( 76, 110 ] 168 0.011% 99.455% ( 110, 170 ] 1156 0.077% 99.531% ( 170, 250 ] 831 0.055% 99.586% ( 250, 380 ] 477 0.032% 99.618% ( 380, 580 ] 719 0.048% 99.666% ( 580, 870 ] 1234 0.082% 99.748% ( 870, 1300 ] 1575 0.104% 99.852% ( 1300, 1900 ] 1385 0.092% 99.944% ( 1900, 2900 ] 1491 0.099% 100.043% ( 2900, 4400 ] 1252 0.083% 100.126% ( 4400, 6600 ] 359 0.024% 100.150% ( 6600, 9900 ] 69 0.005% 100.154% ( 9900, 14000 ] 13 0.001% 100.155% ( 14000, 22000 ] 5 0.000% 100.156% ( 22000, 33000 ] 1 0.000% 100.156% ( 33000, 50000 ] 1 0.000% 100.156% ** Level 4 read latency histogram (micros): Count: 981426 Average: 28.7257 StdDev: 254.31 Min: 0 Median: 2.2827 Max: 36435 Percentiles: P50: 2.28 P75: 3.40 P99: 563.15 P99.9: 2751.68 P99.99: 3591.90 ------------------------------------------------------ [ 0, 1 ] 8539 0.870% 0.870% ( 1, 2 ] 400839 40.843% 41.713% ######## ( 2, 3 ] 287663 29.311% 71.023% ###### ( 3, 4 ] 97282 9.912% 80.936% ## ( 4, 6 ] 110307 11.239% 92.175% ## ( 6, 10 ] 32436 3.305% 95.480% # ( 10, 15 ] 7467 0.761% 96.241% ( 15, 22 ] 818 0.083% 96.324% ( 22, 34 ] 68 0.007% 96.331% ( 34, 51 ] 73 0.007% 96.339% ( 51, 76 ] 106 0.011% 96.349% ( 76, 110 ] 1234 0.126% 96.475% ( 110, 170 ] 11247 1.146% 97.621% ( 170, 250 ] 7942 0.809% 98.430% ( 250, 380 ] 2958 0.301% 98.732% ( 380, 580 ] 2875 0.293% 99.025% ( 580, 870 ] 2696 0.275% 99.299% ( 870, 1300 ] 2563 0.261% 99.561% ( 1300, 1900 ] 1928 0.196% 99.757% ( 1900, 2900 ] 1648 0.168% 99.925% ( 2900, 4400 ] 1385 0.141% 100.066% ( 4400, 6600 ] 401 0.041% 100.107% ( 6600, 9900 ] 83 0.008% 100.115% ( 9900, 14000 ] 10 0.001% 100.116% ( 14000, 22000 ] 4 0.000% 100.117% ( 22000, 33000 ] 7 0.001% 100.117% ( 33000, 50000 ] 3 0.000% 100.118% ** Level 5 read latency histogram (micros): Count: 171334 Average: 444.7954 StdDev: 857.58 Min: 83 Median: 194.1679 Max: 50141 Percentiles: P50: 194.17 P75: 343.44 P99: 4371.64 P99.9: 8278.03 P99.99: 17822.13 ------------------------------------------------------ ( 76, 110 ] 7136 4.165% 4.165% # ( 110, 170 ] 64810 37.827% 41.992% ######## ( 170, 250 ] 45419 26.509% 68.501% ##### ( 250, 380 ] 15492 9.042% 77.543% ## ( 380, 580 ] 12792 7.466% 85.009% # ( 580, 870 ] 8125 4.742% 89.751% # ( 870, 1300 ] 5826 3.400% 93.151% # ( 1300, 1900 ] 3733 2.179% 95.330% ( 1900, 2900 ] 3156 1.842% 97.172% ( 2900, 4400 ] 3192 1.863% 99.035% ( 4400, 6600 ] 1352 0.789% 99.824% ( 6600, 9900 ] 255 0.149% 99.973% ( 9900, 14000 ] 26 0.015% 99.988% ( 14000, 22000 ] 6 0.004% 99.992% ( 22000, 33000 ] 10 0.006% 99.998% ( 33000, 50000 ] 4 0.002% 100.000% ( 50000, 75000 ] 1 0.001% 100.001% ** DB Stats ** Uptime(secs): 27.8 total, 5.4 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 177.03 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.428 H:M:S, 33.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 182.85 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.827 H:M:S, 33.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/10-12:18:20 ... thread 0: (1000000,6000000) ops and (109704.3,163277.7) ops/second in (9.115416,36.747202) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/13 4.42 GB 125.4 5.8 0.0 5.8 11.6 5.7 0.0 2.0 112.5 222.4 53.21 39.64 103 0.517 6075K 9725 0.0 0.0 L1 13/13 1.23 GB 0.0 2.3 1.6 0.8 2.3 1.6 0.0 1.5 95.3 95.0 25.04 12.99 3 8.346 2423K 8839 0.0 0.0 L3 14/0 860.42 MB 0.9 4.3 3.2 1.1 4.3 3.1 0.1 1.3 103.6 102.9 42.61 20.85 19 2.243 2291K 82 0.0 0.0 L4 175/0 9.56 GB 1.0 8.7 5.0 3.7 8.6 4.9 0.4 1.7 110.1 108.8 81.05 42.74 57 1.422 4678K 4286 0.0 0.0 L5 1148/0 94.31 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1367/26 110.35 GB 0.0 21.2 9.8 11.4 26.8 15.4 5.8 4.7 107.5 135.8 201.91 116.22 182 1.109 15M 22K 0.0 0.0 Int 0/0 0.00 KB 0.0 21.2 9.8 11.4 26.8 15.4 5.8 4.7 107.5 135.8 201.91 116.22 182 1.109 15M 22K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 21.2 9.8 11.4 21.1 9.6 0.0 0.0 124.6 123.7 174.31 99.64 98 1.779 15M 22K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 212.2 27.60 16.59 84 0.329 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 36.9 total, 36.7 interval Flush(GB): cumulative 5.722, interval 5.722 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 26.77 GB write, 742.75 MB/s write, 21.20 GB read, 588.18 MB/s read, 201.9 seconds Interval compaction: 26.77 GB write, 746.06 MB/s write, 21.20 GB read, 590.80 MB/s read, 201.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 23, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 23, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 73, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 23, total-stops: 73, interval: 96 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2067563 Average: 14.0687 StdDev: 96.90 Min: 0 Median: 1.6506 Max: 27640 Percentiles: P50: 1.65 P75: 2.41 P99: 242.95 P99.9: 768.74 P99.99: 1437.00 ------------------------------------------------------ [ 0, 1 ] 312081 15.094% 15.094% ### ( 1, 2 ] 1109258 53.651% 68.745% ########### ( 2, 3 ] 312535 15.116% 83.861% ### ( 3, 4 ] 115303 5.577% 89.438% # ( 4, 6 ] 93679 4.531% 93.968% # ( 6, 10 ] 23498 1.137% 95.105% ( 10, 15 ] 5090 0.246% 95.351% ( 15, 22 ] 711 0.034% 95.385% ( 22, 34 ] 85 0.004% 95.390% ( 34, 51 ] 31 0.001% 95.391% ( 51, 76 ] 29 0.001% 95.392% ( 76, 110 ] 3755 0.182% 95.574% ( 110, 170 ] 42692 2.065% 97.639% ( 170, 250 ] 30859 1.493% 99.131% ( 250, 380 ] 8605 0.416% 99.548% ( 380, 580 ] 5796 0.280% 99.828% ( 580, 870 ] 2287 0.111% 99.939% ( 870, 1300 ] 941 0.046% 99.984% ( 1300, 1900 ] 531 0.026% 100.010% ( 1900, 2900 ] 494 0.024% 100.034% ( 2900, 4400 ] 363 0.018% 100.051% ( 4400, 6600 ] 90 0.004% 100.056% ( 6600, 9900 ] 14 0.001% 100.056% ( 9900, 14000 ] 1 0.000% 100.056% ( 22000, 33000 ] 2 0.000% 100.056% ** Level 1 read latency histogram (micros): Count: 304918 Average: 16.1400 StdDev: 173.93 Min: 1 Median: 2.3908 Max: 24280 Percentiles: P50: 2.39 P75: 3.74 P99: 210.37 P99.9: 2820.01 P99.99: 5423.12 ------------------------------------------------------ [ 0, 1 ] 4272 1.401% 1.401% ( 1, 2 ] 115923 38.018% 39.419% ######## ( 2, 3 ] 82565 27.078% 66.497% ##### ( 3, 4 ] 35089 11.508% 78.004% ## ( 4, 6 ] 45189 14.820% 92.824% ### ( 6, 10 ] 11565 3.793% 96.617% # ( 10, 15 ] 2706 0.887% 97.505% ( 15, 22 ] 820 0.269% 97.773% ( 22, 34 ] 79 0.026% 97.799% ( 34, 51 ] 18 0.006% 97.805% ( 51, 76 ] 18 0.006% 97.811% ( 76, 110 ] 317 0.104% 97.915% ( 110, 170 ] 2571 0.843% 98.758% ( 170, 250 ] 1462 0.479% 99.238% ( 250, 380 ] 470 0.154% 99.392% ( 380, 580 ] 398 0.131% 99.522% ( 580, 870 ] 310 0.102% 99.624% ( 870, 1300 ] 328 0.108% 99.732% ( 1300, 1900 ] 262 0.086% 99.818% ( 1900, 2900 ] 274 0.090% 99.908% ( 2900, 4400 ] 226 0.074% 99.982% ( 4400, 6600 ] 57 0.019% 100.000% ( 6600, 9900 ] 14 0.005% 100.005% ( 9900, 14000 ] 2 0.001% 100.006% ( 14000, 22000 ] 1 0.000% 100.006% ( 22000, 33000 ] 1 0.000% 100.006% ** Level 2 read latency histogram (micros): Count: 641923 Average: 18.5944 StdDev: 226.93 Min: 1 Median: 2.2231 Max: 41007 Percentiles: P50: 2.22 P75: 3.18 P99: 13.93 P99.9: 2858.71 P99.99: 3922.01 ------------------------------------------------------ [ 0, 1 ] 7095 1.105% 1.105% ( 1, 2 ] 271038 42.223% 43.328% ######## ( 2, 3 ] 191979 29.907% 73.235% ###### ( 3, 4 ] 63983 9.967% 83.202% ## ( 4, 6 ] 74308 11.576% 94.778% ## ( 6, 10 ] 22563 3.515% 98.293% # ( 10, 15 ] 5772 0.899% 99.192% ( 15, 22 ] 757 0.118% 99.310% ( 22, 34 ] 64 0.010% 99.320% ( 34, 51 ] 28 0.004% 99.325% ( 51, 76 ] 39 0.006% 99.331% ( 76, 110 ] 12 0.002% 99.332% ( 110, 170 ] 18 0.003% 99.335% ( 170, 250 ] 34 0.005% 99.341% ( 250, 380 ] 112 0.017% 99.358% ( 380, 580 ] 278 0.043% 99.401% ( 580, 870 ] 491 0.076% 99.478% ( 870, 1300 ] 762 0.119% 99.597% ( 1300, 1900 ] 905 0.141% 99.738% ( 1900, 2900 ] 1088 0.169% 99.907% ( 2900, 4400 ] 782 0.122% 100.029% ( 4400, 6600 ] 238 0.037% 100.066% ( 6600, 9900 ] 50 0.008% 100.074% ( 9900, 14000 ] 13 0.002% 100.076% ( 33000, 50000 ] 2 0.000% 100.076% ** Level 3 read latency histogram (micros): Count: 1509484 Average: 14.1490 StdDev: 180.53 Min: 0 Median: 2.2579 Max: 41027 Percentiles: P50: 2.26 P75: 3.21 P99: 13.83 P99.9: 1611.91 P99.99: 2364.51 ------------------------------------------------------ [ 0, 1 ] 12837 0.850% 0.850% ( 1, 2 ] 621811 41.194% 42.044% ######## ( 2, 3 ] 465722 30.853% 72.897% ###### ( 3, 4 ] 153553 10.173% 83.070% ## ( 4, 6 ] 179972 11.923% 94.992% ## ( 6, 10 ] 51271 3.397% 98.389% # ( 10, 15 ] 12048 0.798% 99.187% ( 15, 22 ] 1408 0.093% 99.280% ( 22, 34 ] 113 0.007% 99.288% ( 34, 51 ] 97 0.006% 99.294% ( 51, 76 ] 108 0.007% 99.301% ( 76, 110 ] 280 0.019% 99.320% ( 110, 170 ] 2314 0.153% 99.473% ( 170, 250 ] 1445 0.096% 99.569% ( 250, 380 ] 627 0.042% 99.611% ( 380, 580 ] 811 0.054% 99.664% ( 580, 870 ] 1257 0.083% 99.748% ( 870, 1300 ] 1580 0.105% 99.852% ( 1300, 1900 ] 1386 0.092% 99.944% ( 1900, 2900 ] 1492 0.099% 100.043% ( 2900, 4400 ] 1252 0.083% 100.126% ( 4400, 6600 ] 359 0.024% 100.150% ( 6600, 9900 ] 69 0.005% 100.154% ( 9900, 14000 ] 13 0.001% 100.155% ( 14000, 22000 ] 5 0.000% 100.155% ( 22000, 33000 ] 1 0.000% 100.155% ( 33000, 50000 ] 1 0.000% 100.156% ** Level 4 read latency histogram (micros): Count: 1002920 Average: 32.1580 StdDev: 253.15 Min: 0 Median: 2.3201 Max: 36435 Percentiles: P50: 2.32 P75: 3.57 P99: 571.97 P99.9: 2739.73 P99.99: 3590.16 ------------------------------------------------------ [ 0, 1 ] 8539 0.851% 0.851% ( 1, 2 ] 400839 39.967% 40.819% ######## ( 2, 3 ] 287663 28.683% 69.501% ###### ( 3, 4 ] 97282 9.700% 79.201% ## ( 4, 6 ] 110307 10.999% 90.200% ## ( 6, 10 ] 32436 3.234% 93.434% # ( 10, 15 ] 7467 0.745% 94.178% ( 15, 22 ] 818 0.082% 94.260% ( 22, 34 ] 68 0.007% 94.267% ( 34, 51 ] 73 0.007% 94.274% ( 51, 76 ] 106 0.011% 94.284% ( 76, 110 ] 2515 0.251% 94.535% ( 110, 170 ] 22489 2.242% 96.778% ( 170, 250 ] 14325 1.428% 98.206% ( 250, 380 ] 4350 0.434% 98.640% ( 380, 580 ] 3766 0.376% 99.015% ( 580, 870 ] 2958 0.295% 99.310% ( 870, 1300 ] 2598 0.259% 99.569% ( 1300, 1900 ] 1931 0.193% 99.762% ( 1900, 2900 ] 1653 0.165% 99.927% ( 2900, 4400 ] 1386 0.138% 100.065% ( 4400, 6600 ] 401 0.040% 100.105% ( 6600, 9900 ] 83 0.008% 100.113% ( 9900, 14000 ] 10 0.001% 100.114% ( 14000, 22000 ] 4 0.000% 100.114% ( 22000, 33000 ] 7 0.001% 100.115% ( 33000, 50000 ] 3 0.000% 100.115% ** Level 5 read latency histogram (micros): Count: 291192 Average: 339.2516 StdDev: 673.71 Min: 80 Median: 173.1085 Max: 50141 Percentiles: P50: 173.11 P75: 245.45 P99: 3809.88 P99.9: 6726.93 P99.99: 12561.97 ------------------------------------------------------ ( 76, 110 ] 14209 4.880% 4.880% # ( 110, 170 ] 128259 44.046% 48.926% ######### ( 170, 250 ] 80503 27.646% 76.572% ###### ( 250, 380 ] 23041 7.913% 84.484% ## ( 380, 580 ] 17763 6.100% 90.585% # ( 580, 870 ] 9561 3.283% 93.868% # ( 870, 1300 ] 6036 2.073% 95.941% ( 1300, 1900 ] 3781 1.298% 97.239% ( 1900, 2900 ] 3186 1.094% 98.333% ( 2900, 4400 ] 3200 1.099% 99.432% ( 4400, 6600 ] 1352 0.464% 99.897% ( 6600, 9900 ] 255 0.088% 99.984% ( 9900, 14000 ] 26 0.009% 99.993% ( 14000, 22000 ] 6 0.002% 99.995% ( 22000, 33000 ] 10 0.003% 99.999% ( 33000, 50000 ] 4 0.001% 100.000% ( 50000, 75000 ] 1 0.000% 100.000% ** DB Stats ** Uptime(secs): 36.9 total, 9.1 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 159.98 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:14.912 H:M:S, 40.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 107.97 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.484 H:M:S, 60.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 295.438 micros/op 13525 ops/sec 60.184 seconds 813996 operations; 13.1 MB/s (203999 of 203999 found) Microseconds per read: Count: 813996 Average: 295.4408 StdDev: 950.15 Min: 2 Median: 204.2617 Max: 94395 Percentiles: P50: 204.26 P75: 247.01 P99: 1812.87 P99.9: 17554.36 P99.99: 33431.85 ------------------------------------------------------ ( 1, 2 ] 1 0.000% 0.000% ( 2, 3 ] 18 0.002% 0.002% ( 3, 4 ] 36 0.004% 0.007% ( 4, 6 ] 74 0.009% 0.016% ( 6, 10 ] 268 0.033% 0.049% ( 10, 15 ] 2960 0.364% 0.412% ( 15, 22 ] 3841 0.472% 0.884% ( 22, 34 ] 9483 1.165% 2.049% ( 34, 51 ] 2750 0.338% 2.387% ( 51, 76 ] 378 0.046% 2.434% ( 76, 110 ] 1940 0.238% 2.672% ( 110, 170 ] 222148 27.291% 29.963% ##### ( 170, 250 ] 380836 46.786% 76.749% ######### ( 250, 380 ] 101431 12.461% 89.210% ## ( 380, 580 ] 49662 6.101% 95.311% # ( 580, 870 ] 18554 2.279% 97.590% ( 870, 1300 ] 8121 0.998% 98.588% ( 1300, 1900 ] 3925 0.482% 99.070% ( 1900, 2900 ] 2788 0.343% 99.413% ( 2900, 4400 ] 2522 0.310% 99.722% ( 4400, 6600 ] 801 0.098% 99.821% ( 6600, 9900 ] 156 0.019% 99.840% ( 9900, 14000 ] 170 0.021% 99.861% ( 14000, 22000 ] 718 0.088% 99.949% ( 22000, 33000 ] 332 0.041% 99.990% ( 33000, 50000 ] 63 0.008% 99.998% ( 50000, 75000 ] 18 0.002% 100.000% ( 75000, 110000 ] 2 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 15497183 rocksdb.bloom.filter.full.positive COUNT : 964904 rocksdb.bloom.filter.full.true.positive COUNT : 813630 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 366 rocksdb.memtable.miss COUNT : 813630 rocksdb.l0.hit COUNT : 33812 rocksdb.l1.hit COUNT : 8251 rocksdb.l2andup.hit COUNT : 771567 rocksdb.compaction.key.drop.new COUNT : 175952 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 6233042 rocksdb.number.keys.read COUNT : 813996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 6432499344 rocksdb.bytes.read COUNT : 813996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1702 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 37337530 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 24172490452 rocksdb.compact.write.bytes COUNT : 23990531866 rocksdb.flush.write.bytes COUNT : 6407678053 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 34 rocksdb.number.superversion_acquires COUNT : 1231 rocksdb.number.superversion_releases COUNT : 1177 rocksdb.number.superversion_cleanups COUNT : 307 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 310 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 6355870720 rocksdb.memtable.garbage.bytes.at.flush COUNT : 2337792 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 28028642881 rocksdb.non.last.level.read.count COUNT : 6628929 rocksdb.block.checksum.compute.count COUNT : 6830953 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 203.352464 P95 : 569.152597 P99 : 1811.865340 P100 : 94394.000000 COUNT : 813996 SUM : 239712942 rocksdb.db.write.micros P50 : 1.523652 P95 : 3.941927 P99 : 6.501353 P100 : 889343.000000 COUNT : 6233042 SUM : 52432488 rocksdb.compaction.times.micros P50 : 1422727.272727 P95 : 2935000.000000 P99 : 14080000.000000 P100 : 14811398.000000 COUNT : 99 SUM : 175265551 rocksdb.compaction.times.cpu_micros P50 : 846818.181818 P95 : 1839545.454545 P99 : 6533000.000000 P100 : 7672163.000000 COUNT : 99 SUM : 100582412 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 904.400000 P95 : 19200.000000 P99 : 117200.000000 P100 : 122140.000000 COUNT : 88 SUM : 453789 rocksdb.compaction.outfile.sync.micros P50 : 16962.962963 P95 : 199647.058824 P99 : 295933.333333 P100 : 904654.000000 COUNT : 294 SUM : 13818006 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 616.250000 P95 : 4358.333333 P99 : 8167.500000 P100 : 8406.000000 COUNT : 210 SUM : 263573 rocksdb.table.open.io.micros P50 : 11856.633663 P95 : 21561.581921 P99 : 30245.769231 P100 : 65614.000000 COUNT : 1702 SUM : 19336352 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.230861 P95 : 11.602239 P99 : 221.408131 P100 : 41031.000000 COUNT : 5853850 SUM : 105372058 rocksdb.read.block.get.micros P50 : 154.622261 P95 : 460.535877 P99 : 1780.183051 P100 : 50215.000000 COUNT : 975374 SUM : 204823317 rocksdb.write.raw.block.micros P50 : 0.538030 P95 : 1.418024 P99 : 3.855031 P100 : 49105.000000 COUNT : 7362543 SUM : 24921313 rocksdb.numfiles.in.singlecompaction P50 : 2.802632 P95 : 5.583333 P99 : 9.750000 P100 : 26.000000 COUNT : 125 SUM : 439 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.574840 P95 : 2237.846554 P99 : 2769.032538 P100 : 889164.000000 COUNT : 137949 SUM : 37345716 rocksdb.sst.read.micros P50 : 2.281159 P95 : 193.441466 P99 : 526.107073 P100 : 50141.000000 COUNT : 6628942 SUM : 293433721 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 813996 SUM : 813996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 6233042 SUM : 6432499344 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 223333.333333 P95 : 986285.714286 P99 : 1110715.000000 P100 : 1110715.000000 COUNT : 88 SUM : 28441920 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 337 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0