[2024-12-16 21:10:02.188627] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 23.11.0 initialization... [2024-12-16 21:10:02.188889] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2001123 ] Set seed to 1734379802181889 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-16 21:10:02.307520] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-16 21:10:02.515083] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-16 21:10:02.515127] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2001123' to capture a snapshot of events at runtime. [2024-12-16 21:10:02.515137] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2001123 for offline analysis/debug. [2024-12-16 21:10:02.515181] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Dec 16 21:10:11 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/16-21:10:18 ... thread 0: (1000000,1000000) ops and (132297.1,132297.1) ops/second in (7.558743,7.558743) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 6/4 566.86 MB 18.9 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 117.5 6.97 2.35 9 0.774 0 0 0.0 0.0 L1 2/1 175.14 MB 11.6 0.3 0.3 0.0 0.3 0.3 0.0 1.0 119.2 119.1 2.60 1.39 1 2.598 314K 358 0.0 0.0 L2 5/2 391.82 MB 1.3 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 72/8 4.85 GB 4.1 5.1 3.3 1.8 5.0 3.3 0.0 1.5 97.9 97.2 53.04 24.22 26 2.040 3020K 4003 0.0 0.0 L4 160/12 10.42 GB 1.0 0.9 0.7 0.1 0.8 0.7 0.3 1.2 95.9 95.1 9.15 4.14 6 1.525 107K 0 0.0 0.0 L5 1092/0 90.25 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1337/27 106.63 GB 0.0 6.2 4.3 1.9 7.0 5.1 0.6 8.7 88.9 99.7 71.76 32.11 42 1.708 3442K 4361 0.0 0.0 Int 0/0 0.00 KB 0.0 6.2 4.3 1.9 7.0 5.1 0.6 8.7 88.9 99.7 71.76 32.11 42 1.708 3442K 4361 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.2 4.3 1.9 6.2 4.3 0.0 0.0 98.4 97.7 64.79 29.75 33 1.963 3442K 4361 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 117.5 6.97 2.35 9 0.774 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.8 total, 7.6 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.98 GB write, 918.44 MB/s write, 6.23 GB read, 819.09 MB/s read, 71.8 seconds Interval compaction: 6.98 GB write, 945.89 MB/s write, 6.23 GB read, 843.57 MB/s read, 71.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 26, interval: 26 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 165117 Average: 8.7186 StdDev: 121.63 Min: 1 Median: 1.8889 Max: 13122 Percentiles: P50: 1.89 P75: 2.92 P99: 9.60 P99.9: 2133.77 P99.99: 4604.64 ------------------------------------------------------ [ 0, 1 ] 7306 4.425% 4.425% # ( 1, 2 ] 84655 51.270% 55.694% ########## ( 2, 3 ] 34469 20.876% 76.570% #### ( 3, 4 ] 15327 9.283% 85.852% ## ( 4, 6 ] 18465 11.183% 97.035% ## ( 6, 10 ] 3607 2.185% 99.220% ( 10, 15 ] 672 0.407% 99.627% ( 15, 22 ] 67 0.041% 99.668% ( 22, 34 ] 4 0.002% 99.670% ( 51, 76 ] 2 0.001% 99.671% ( 170, 250 ] 7 0.004% 99.675% ( 250, 380 ] 12 0.007% 99.683% ( 380, 580 ] 28 0.017% 99.700% ( 580, 870 ] 73 0.044% 99.744% ( 870, 1300 ] 90 0.055% 99.798% ( 1300, 1900 ] 141 0.085% 99.884% ( 1900, 2900 ] 115 0.070% 99.953% ( 2900, 4400 ] 59 0.036% 99.989% ( 4400, 6600 ] 16 0.010% 99.999% ( 6600, 9900 ] 1 0.001% 99.999% ( 9900, 14000 ] 1 0.001% 100.000% ** Level 1 read latency histogram (micros): Count: 44595 Average: 16.5857 StdDev: 192.45 Min: 1 Median: 1.9849 Max: 11616 Percentiles: P50: 1.98 P75: 3.12 P99: 13.54 P99.9: 3443.91 P99.99: 6014.55 ------------------------------------------------------ [ 0, 1 ] 1214 2.722% 2.722% # ( 1, 2 ] 21406 48.001% 50.723% ########## ( 2, 3 ] 10295 23.086% 73.809% ##### ( 3, 4 ] 4493 10.075% 83.884% ## ( 4, 6 ] 5297 11.878% 95.762% ## ( 6, 10 ] 1221 2.738% 98.500% # ( 10, 15 ] 315 0.706% 99.206% ( 15, 22 ] 35 0.078% 99.285% ( 22, 34 ] 4 0.009% 99.294% ( 110, 170 ] 1 0.002% 99.296% ( 170, 250 ] 4 0.009% 99.305% ( 250, 380 ] 4 0.009% 99.314% ( 380, 580 ] 18 0.040% 99.354% ( 580, 870 ] 38 0.085% 99.439% ( 870, 1300 ] 48 0.108% 99.547% ( 1300, 1900 ] 72 0.161% 99.708% ( 1900, 2900 ] 68 0.152% 99.861% ( 2900, 4400 ] 48 0.108% 99.969% ( 4400, 6600 ] 13 0.029% 99.998% ( 9900, 14000 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 762257 Average: 18.6005 StdDev: 219.03 Min: 1 Median: 1.9672 Max: 17320 Percentiles: P50: 1.97 P75: 3.02 P99: 13.25 P99.9: 2143.64 P99.99: 2614.82 ------------------------------------------------------ [ 0, 1 ] 25551 3.352% 3.352% # ( 1, 2 ] 367628 48.229% 51.581% ########## ( 2, 3 ] 176715 23.183% 74.764% ##### ( 3, 4 ] 72439 9.503% 84.267% ## ( 4, 6 ] 86824 11.390% 95.658% ## ( 6, 10 ] 21207 2.782% 98.440% # ( 10, 15 ] 6567 0.862% 99.301% ( 15, 22 ] 1024 0.134% 99.436% ( 22, 34 ] 104 0.014% 99.449% ( 34, 51 ] 34 0.004% 99.454% ( 51, 76 ] 33 0.004% 99.458% ( 76, 110 ] 8 0.001% 99.459% ( 110, 170 ] 28 0.004% 99.463% ( 170, 250 ] 68 0.009% 99.472% ( 250, 380 ] 145 0.019% 99.491% ( 380, 580 ] 278 0.036% 99.527% ( 580, 870 ] 488 0.064% 99.591% ( 870, 1300 ] 859 0.113% 99.704% ( 1300, 1900 ] 1140 0.150% 99.853% ( 1900, 2900 ] 1456 0.191% 100.044% ( 2900, 4400 ] 939 0.123% 100.168% ( 4400, 6600 ] 269 0.035% 100.203% ( 6600, 9900 ] 35 0.005% 100.208% ( 9900, 14000 ] 15 0.002% 100.210% ( 14000, 22000 ] 6 0.001% 100.210% ** Level 3 read latency histogram (micros): Count: 678717 Average: 17.9320 StdDev: 203.89 Min: 1 Median: 2.0296 Max: 14595 Percentiles: P50: 2.03 P75: 3.09 P99: 12.71 P99.9: 2171.88 P99.99: 2648.73 ------------------------------------------------------ [ 0, 1 ] 15625 2.302% 2.302% ( 1, 2 ] 318751 46.964% 49.266% ######### ( 2, 3 ] 168224 24.786% 74.051% ##### ( 3, 4 ] 68822 10.140% 84.191% ## ( 4, 6 ] 79891 11.771% 95.962% ## ( 6, 10 ] 18031 2.657% 98.619% # ( 10, 15 ] 4776 0.704% 99.323% ( 15, 22 ] 580 0.085% 99.408% ( 22, 34 ] 54 0.008% 99.416% ( 34, 51 ] 27 0.004% 99.420% ( 51, 76 ] 43 0.006% 99.426% ( 76, 110 ] 22 0.003% 99.430% ( 110, 170 ] 36 0.005% 99.435% ( 170, 250 ] 78 0.011% 99.446% ( 250, 380 ] 112 0.017% 99.463% ( 380, 580 ] 230 0.034% 99.497% ( 580, 870 ] 509 0.075% 99.572% ( 870, 1300 ] 801 0.118% 99.690% ( 1300, 1900 ] 1080 0.159% 99.849% ( 1900, 2900 ] 1281 0.189% 100.038% ( 2900, 4400 ] 781 0.115% 100.153% ( 4400, 6600 ] 195 0.029% 100.182% ( 6600, 9900 ] 28 0.004% 100.186% ( 9900, 14000 ] 2 0.000% 100.186% ( 14000, 22000 ] 3 0.000% 100.186% ** Level 4 read latency histogram (micros): Count: 90844 Average: 54.2224 StdDev: 390.89 Min: 1 Median: 2.0566 Max: 10639 Percentiles: P50: 2.06 P75: 3.21 P99: 1932.77 P99.9: 4810.21 P99.99: 6001.42 ------------------------------------------------------ [ 0, 1 ] 1932 2.127% 2.127% ( 1, 2 ] 42248 46.506% 48.633% ######### ( 2, 3 ] 21938 24.149% 72.782% ##### ( 3, 4 ] 9596 10.563% 83.345% ## ( 4, 6 ] 10337 11.379% 94.724% ## ( 6, 10 ] 1960 2.158% 96.881% ( 10, 15 ] 484 0.533% 97.414% ( 15, 22 ] 59 0.065% 97.479% ( 22, 34 ] 5 0.006% 97.485% ( 34, 51 ] 8 0.009% 97.494% ( 51, 76 ] 5 0.006% 97.499% ( 76, 110 ] 1 0.001% 97.500% ( 110, 170 ] 9 0.010% 97.510% ( 170, 250 ] 45 0.050% 97.560% ( 250, 380 ] 64 0.070% 97.630% ( 380, 580 ] 120 0.132% 97.762% ( 580, 870 ] 293 0.323% 98.085% ( 870, 1300 ] 387 0.426% 98.511% ( 1300, 1900 ] 431 0.474% 98.985% ( 1900, 2900 ] 444 0.489% 99.474% ( 2900, 4400 ] 360 0.396% 99.870% ( 4400, 6600 ] 151 0.166% 100.036% ( 6600, 9900 ] 17 0.019% 100.055% ( 9900, 14000 ] 1 0.001% 100.056% ** Level 5 read latency histogram (micros): Count: 9450 Average: 2290.0816 StdDev: 1692.45 Min: 84 Median: 1831.4945 Max: 29489 Percentiles: P50: 1831.49 P75: 3257.33 P99: 8345.38 P99.9: 11517.22 P99.99: 27802.50 ------------------------------------------------------ ( 76, 110 ] 6 0.063% 0.063% ( 110, 170 ] 17 0.180% 0.243% ( 170, 250 ] 80 0.847% 1.090% ( 250, 380 ] 230 2.434% 3.524% ( 380, 580 ] 541 5.725% 9.249% # ( 580, 870 ] 949 10.042% 19.291% ## ( 870, 1300 ] 1521 16.095% 35.386% ### ( 1300, 1900 ] 1559 16.497% 51.884% ### ( 1900, 2900 ] 1785 18.889% 70.772% #### ( 2900, 4400 ] 1677 17.746% 88.519% #### ( 4400, 6600 ] 899 9.513% 98.032% ## ( 6600, 9900 ] 173 1.831% 99.862% ( 9900, 14000 ] 9 0.095% 99.958% ( 14000, 22000 ] 2 0.021% 99.979% ( 22000, 33000 ] 2 0.021% 100.000% ** DB Stats ** Uptime(secs): 7.8 total, 7.6 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 126.39 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.739 H:M:S, 48.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 130.16 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.739 H:M:S, 49.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/12/16-21:10:26 ... thread 0: (1000000,2000000) ops and (139445.9,135777.5) ops/second in (7.171242,14.729985) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/9 1.54 GB 50.3 0.7 0.0 0.7 2.6 1.8 0.0 1.4 40.2 140.5 18.81 8.48 24 0.784 767K 1085 0.0 0.0 L1 1/1 115.77 MB 0.0 0.3 0.3 0.0 0.3 0.3 0.0 1.0 119.2 119.1 2.60 1.39 1 2.598 314K 358 0.0 0.0 L2 5/3 286.08 MB 0.7 0.3 0.1 0.3 0.3 0.1 0.3 5.3 118.4 118.1 2.68 1.49 1 2.676 321K 846 0.0 0.0 L3 36/25 1.97 GB 0.6 5.5 3.5 2.0 5.5 3.4 0.0 1.6 98.8 98.1 56.94 26.28 28 2.034 3460K 6948 0.0 0.0 L4 158/0 9.41 GB 1.0 6.3 3.8 2.6 6.2 3.6 0.3 1.7 114.8 112.6 56.66 30.21 43 1.318 2969K 0 0.0 0.0 L5 1140/0 94.18 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1352/38 107.50 GB 0.0 13.2 7.6 5.6 14.9 9.3 4.5 8.1 98.1 110.6 137.69 67.85 97 1.420 7833K 9237 0.0 0.0 Int 0/0 0.00 KB 0.0 13.2 7.6 5.6 14.9 9.3 4.5 8.1 98.1 110.6 137.69 67.85 97 1.420 7833K 9237 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.2 7.6 5.6 13.0 7.4 0.0 0.0 109.4 108.0 123.53 62.50 75 1.647 7833K 9237 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 133.5 14.16 5.34 22 0.644 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.0 total, 14.7 interval Flush(GB): cumulative 1.845, interval 1.845 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.88 GB write, 1018.57 MB/s write, 13.19 GB read, 903.15 MB/s read, 137.7 seconds Interval compaction: 14.88 GB write, 1034.19 MB/s write, 13.19 GB read, 917.01 MB/s read, 137.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 60, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 60, interval: 60 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 489413 Average: 10.7153 StdDev: 147.14 Min: 1 Median: 1.9034 Max: 32931 Percentiles: P50: 1.90 P75: 2.92 P99: 10.74 P99.9: 1997.23 P99.99: 4121.34 ------------------------------------------------------ [ 0, 1 ] 20156 4.118% 4.118% # ( 1, 2 ] 248550 50.785% 54.904% ########## ( 2, 3 ] 106785 21.819% 76.723% #### ( 3, 4 ] 43978 8.986% 85.709% ## ( 4, 6 ] 53473 10.926% 96.635% ## ( 6, 10 ] 11243 2.297% 98.932% ( 10, 15 ] 2281 0.466% 99.398% ( 15, 22 ] 228 0.047% 99.444% ( 22, 34 ] 16 0.003% 99.448% ( 34, 51 ] 23 0.005% 99.452% ( 51, 76 ] 27 0.006% 99.458% ( 76, 110 ] 4 0.001% 99.459% ( 110, 170 ] 24 0.005% 99.464% ( 170, 250 ] 71 0.015% 99.478% ( 250, 380 ] 165 0.034% 99.512% ( 380, 580 ] 354 0.072% 99.584% ( 580, 870 ] 555 0.113% 99.698% ( 870, 1300 ] 550 0.112% 99.810% ( 1300, 1900 ] 414 0.085% 99.895% ( 1900, 2900 ] 294 0.060% 99.955% ( 2900, 4400 ] 215 0.044% 99.999% ( 4400, 6600 ] 65 0.013% 100.012% ( 6600, 9900 ] 13 0.003% 100.015% ( 9900, 14000 ] 1 0.000% 100.015% ( 14000, 22000 ] 2 0.000% 100.015% ( 22000, 33000 ] 1 0.000% 100.015% ** Level 1 read latency histogram (micros): Count: 83749 Average: 15.8823 StdDev: 182.96 Min: 1 Median: 1.9949 Max: 18243 Percentiles: P50: 1.99 P75: 3.10 P99: 14.64 P99.9: 2926.79 P99.99: 5371.67 ------------------------------------------------------ [ 0, 1 ] 2091 2.497% 2.497% ( 1, 2 ] 39987 47.746% 50.243% ########## ( 2, 3 ] 19912 23.776% 74.019% ##### ( 3, 4 ] 8157 9.740% 83.759% ## ( 4, 6 ] 9847 11.758% 95.516% ## ( 6, 10 ] 2411 2.879% 98.395% # ( 10, 15 ] 547 0.653% 99.048% ( 15, 22 ] 60 0.072% 99.120% ( 22, 34 ] 8 0.010% 99.130% ( 34, 51 ] 4 0.005% 99.134% ( 51, 76 ] 8 0.010% 99.144% ( 76, 110 ] 2 0.002% 99.146% ( 110, 170 ] 10 0.012% 99.158% ( 170, 250 ] 12 0.014% 99.173% ( 250, 380 ] 25 0.030% 99.202% ( 380, 580 ] 76 0.091% 99.293% ( 580, 870 ] 145 0.173% 99.466% ( 870, 1300 ] 147 0.176% 99.642% ( 1300, 1900 ] 121 0.144% 99.786% ( 1900, 2900 ] 95 0.113% 99.900% ( 2900, 4400 ] 70 0.084% 99.983% ( 4400, 6600 ] 15 0.018% 100.001% ( 6600, 9900 ] 1 0.001% 100.002% ( 9900, 14000 ] 1 0.001% 100.004% ( 14000, 22000 ] 1 0.001% 100.005% ** Level 2 read latency histogram (micros): Count: 869449 Average: 17.4761 StdDev: 210.36 Min: 1 Median: 1.9771 Max: 17320 Percentiles: P50: 1.98 P75: 3.04 P99: 13.08 P99.9: 2140.74 P99.99: 2660.33 ------------------------------------------------------ [ 0, 1 ] 27288 3.139% 3.139% # ( 1, 2 ] 417006 47.962% 51.101% ########## ( 2, 3 ] 204146 23.480% 74.581% ##### ( 3, 4 ] 83501 9.604% 84.184% ## ( 4, 6 ] 100077 11.510% 95.695% ## ( 6, 10 ] 24235 2.787% 98.482% # ( 10, 15 ] 7306 0.840% 99.323% ( 15, 22 ] 1126 0.130% 99.452% ( 22, 34 ] 114 0.013% 99.465% ( 34, 51 ] 43 0.005% 99.470% ( 51, 76 ] 50 0.006% 99.476% ( 76, 110 ] 10 0.001% 99.477% ( 110, 170 ] 33 0.004% 99.481% ( 170, 250 ] 75 0.009% 99.489% ( 250, 380 ] 159 0.018% 99.508% ( 380, 580 ] 324 0.037% 99.545% ( 580, 870 ] 583 0.067% 99.612% ( 870, 1300 ] 945 0.109% 99.721% ( 1300, 1900 ] 1196 0.138% 99.858% ( 1900, 2900 ] 1506 0.173% 100.032% ( 2900, 4400 ] 984 0.113% 100.145% ( 4400, 6600 ] 289 0.033% 100.178% ( 6600, 9900 ] 36 0.004% 100.182% ( 9900, 14000 ] 16 0.002% 100.184% ( 14000, 22000 ] 6 0.001% 100.185% ** Level 3 read latency histogram (micros): Count: 1654288 Average: 14.2453 StdDev: 180.07 Min: 1 Median: 2.0453 Max: 36113 Percentiles: P50: 2.05 P75: 3.12 P99: 11.86 P99.9: 1686.14 P99.99: 2366.10 ------------------------------------------------------ [ 0, 1 ] 32598 1.971% 1.971% ( 1, 2 ] 775862 46.900% 48.871% ######### ( 2, 3 ] 412687 24.947% 73.817% ##### ( 3, 4 ] 165663 10.014% 83.831% ## ( 4, 6 ] 201180 12.161% 95.992% ## ( 6, 10 ] 45639 2.759% 98.751% # ( 10, 15 ] 11083 0.670% 99.421% ( 15, 22 ] 1298 0.078% 99.500% ( 22, 34 ] 119 0.007% 99.507% ( 34, 51 ] 107 0.006% 99.513% ( 51, 76 ] 146 0.009% 99.522% ( 76, 110 ] 37 0.002% 99.524% ( 110, 170 ] 96 0.006% 99.530% ( 170, 250 ] 229 0.014% 99.544% ( 250, 380 ] 477 0.029% 99.573% ( 380, 580 ] 892 0.054% 99.627% ( 580, 870 ] 1536 0.093% 99.720% ( 870, 1300 ] 1835 0.111% 99.831% ( 1300, 1900 ] 1788 0.108% 99.939% ( 1900, 2900 ] 1827 0.110% 100.049% ( 2900, 4400 ] 1425 0.086% 100.135% ( 4400, 6600 ] 387 0.023% 100.159% ( 6600, 9900 ] 61 0.004% 100.162% ( 9900, 14000 ] 8 0.000% 100.163% ( 14000, 22000 ] 4 0.000% 100.163% ( 22000, 33000 ] 2 0.000% 100.163% ( 33000, 50000 ] 1 0.000% 100.163% ** Level 4 read latency histogram (micros): Count: 675248 Average: 21.3295 StdDev: 230.70 Min: 1 Median: 2.0631 Max: 23481 Percentiles: P50: 2.06 P75: 3.22 P99: 235.38 P99.9: 2631.63 P99.99: 3565.03 ------------------------------------------------------ [ 0, 1 ] 13283 1.967% 1.967% ( 1, 2 ] 313985 46.499% 48.466% ######### ( 2, 3 ] 164139 24.308% 72.774% ##### ( 3, 4 ] 69157 10.242% 83.016% ## ( 4, 6 ] 83467 12.361% 95.377% ## ( 6, 10 ] 18873 2.795% 98.172% # ( 10, 15 ] 4669 0.691% 98.863% ( 15, 22 ] 510 0.076% 98.939% ( 22, 34 ] 43 0.006% 98.945% ( 34, 51 ] 39 0.006% 98.951% ( 51, 76 ] 72 0.011% 98.962% ( 76, 110 ] 22 0.003% 98.965% ( 110, 170 ] 51 0.008% 98.973% ( 170, 250 ] 227 0.034% 99.006% ( 250, 380 ] 373 0.055% 99.061% ( 380, 580 ] 812 0.120% 99.182% ( 580, 870 ] 1403 0.208% 99.389% ( 870, 1300 ] 1584 0.235% 99.624% ( 1300, 1900 ] 1157 0.171% 99.795% ( 1900, 2900 ] 966 0.143% 99.938% ( 2900, 4400 ] 786 0.116% 100.055% ( 4400, 6600 ] 314 0.047% 100.101% ( 6600, 9900 ] 46 0.007% 100.108% ( 9900, 14000 ] 10 0.001% 100.110% ( 14000, 22000 ] 4 0.001% 100.110% ( 22000, 33000 ] 2 0.000% 100.110% ** Level 5 read latency histogram (micros): Count: 26681 Average: 1629.2658 StdDev: 1520.28 Min: 84 Median: 1120.7705 Max: 37175 Percentiles: P50: 1120.77 P75: 2174.96 P99: 6558.21 P99.9: 9859.42 P99.99: 25662.72 ------------------------------------------------------ ( 76, 110 ] 25 0.094% 0.094% ( 110, 170 ] 227 0.851% 0.944% ( 170, 250 ] 603 2.260% 3.205% ( 250, 380 ] 1484 5.562% 8.767% # ( 380, 580 ] 3115 11.675% 20.442% ## ( 580, 870 ] 4893 18.339% 38.780% #### ( 870, 1300 ] 5133 19.238% 58.019% #### ( 1300, 1900 ] 3688 13.823% 71.841% ### ( 1900, 2900 ] 3065 11.488% 83.329% ## ( 2900, 4400 ] 2900 10.869% 94.198% ## ( 4400, 6600 ] 1306 4.895% 99.093% # ( 6600, 9900 ] 218 0.817% 99.910% ( 9900, 14000 ] 10 0.037% 99.948% ( 14000, 22000 ] 10 0.037% 99.985% ( 22000, 33000 ] 4 0.015% 100.000% ( 33000, 50000 ] 2 0.007% 100.007% ** DB Stats ** Uptime(secs): 15.0 total, 7.2 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 131.59 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.055 H:M:S, 47.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 137.24 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.317 H:M:S, 46.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 5 num-running-flushes: 1 2024/12/16-21:10:31 ... thread 0: (1000000,3000000) ops and (173204.4,146316.4) ops/second in (5.773526,20.503511) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/9 1.78 GB 37.8 1.5 0.0 1.5 4.3 2.8 0.0 1.6 56.0 156.8 28.11 14.40 39 0.721 1598K 1963 0.0 0.0 L1 5/5 429.68 MB 0.0 1.2 1.0 0.1 1.1 1.0 0.0 1.1 130.6 130.2 9.04 5.51 2 4.520 1199K 3274 0.0 0.0 L2 1/0 49.66 MB 0.5 0.3 0.1 0.3 0.3 0.1 0.7 5.3 118.4 118.1 2.68 1.49 1 2.676 321K 846 0.0 0.0 L3 19/2 1.09 GB 0.9 7.9 4.1 3.8 7.9 4.0 0.1 1.9 109.1 108.4 74.36 37.80 35 2.124 5729K 14K 0.0 0.0 L4 175/9 10.20 GB 1.0 9.0 5.0 4.0 8.8 4.8 0.6 1.8 118.8 116.4 77.48 43.37 62 1.250 4492K 4199 0.0 0.0 L5 1148/0 94.86 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1359/25 108.40 GB 0.0 19.9 10.2 9.8 22.4 12.7 6.0 8.1 106.4 119.9 191.66 102.57 139 1.379 13M 25K 0.0 0.0 Int 0/0 0.00 KB 0.0 19.9 10.2 9.8 22.4 12.7 6.0 8.1 106.4 119.9 191.66 102.57 139 1.379 13M 25K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 19.9 10.2 9.8 19.7 9.9 0.0 0.0 118.5 117.1 172.13 94.59 105 1.639 13M 25K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 145.1 19.53 7.99 34 0.574 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.7 total, 20.5 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 22.44 GB write, 1108.58 MB/s write, 19.91 GB read, 983.54 MB/s read, 191.7 seconds Interval compaction: 22.44 GB write, 1120.80 MB/s write, 19.91 GB read, 994.38 MB/s read, 191.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 85, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 85, interval: 85 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 843901 Average: 9.5517 StdDev: 140.00 Min: 1 Median: 1.8845 Max: 32931 Percentiles: P50: 1.88 P75: 2.88 P99: 9.92 P99.9: 1682.44 P99.99: 3728.49 ------------------------------------------------------ [ 0, 1 ] 36843 4.366% 4.366% # ( 1, 2 ] 435406 51.594% 55.960% ########## ( 2, 3 ] 183101 21.697% 77.657% #### ( 3, 4 ] 74362 8.812% 86.469% ## ( 4, 6 ] 88648 10.505% 96.973% ## ( 6, 10 ] 17474 2.071% 99.044% ( 10, 15 ] 3374 0.400% 99.444% ( 15, 22 ] 331 0.039% 99.483% ( 22, 34 ] 26 0.003% 99.486% ( 34, 51 ] 35 0.004% 99.490% ( 51, 76 ] 39 0.005% 99.495% ( 76, 110 ] 13 0.002% 99.497% ( 110, 170 ] 99 0.012% 99.508% ( 170, 250 ] 219 0.026% 99.534% ( 250, 380 ] 381 0.045% 99.579% ( 380, 580 ] 657 0.078% 99.657% ( 580, 870 ] 907 0.107% 99.765% ( 870, 1300 ] 805 0.095% 99.860% ( 1300, 1900 ] 532 0.063% 99.923% ( 1900, 2900 ] 386 0.046% 99.969% ( 2900, 4400 ] 327 0.039% 100.008% ( 4400, 6600 ] 87 0.010% 100.018% ( 6600, 9900 ] 17 0.002% 100.020% ( 9900, 14000 ] 1 0.000% 100.020% ( 14000, 22000 ] 4 0.000% 100.021% ( 22000, 33000 ] 4 0.000% 100.021% ** Level 1 read latency histogram (micros): Count: 258523 Average: 11.2332 StdDev: 160.55 Min: 1 Median: 1.9764 Max: 18243 Percentiles: P50: 1.98 P75: 2.98 P99: 11.94 P99.9: 1895.06 P99.99: 3945.77 ------------------------------------------------------ [ 0, 1 ] 5446 2.107% 2.107% ( 1, 2 ] 126810 49.052% 51.158% ########## ( 2, 3 ] 62948 24.349% 75.507% ##### ( 3, 4 ] 24780 9.585% 85.093% ## ( 4, 6 ] 28978 11.209% 96.302% ## ( 6, 10 ] 6420 2.483% 98.785% ( 10, 15 ] 1433 0.554% 99.339% ( 15, 22 ] 146 0.056% 99.396% ( 22, 34 ] 10 0.004% 99.400% ( 34, 51 ] 28 0.011% 99.410% ( 51, 76 ] 22 0.009% 99.419% ( 76, 110 ] 5 0.002% 99.421% ( 110, 170 ] 39 0.015% 99.436% ( 170, 250 ] 84 0.032% 99.469% ( 250, 380 ] 153 0.059% 99.528% ( 380, 580 ] 219 0.085% 99.612% ( 580, 870 ] 284 0.110% 99.722% ( 870, 1300 ] 276 0.107% 99.829% ( 1300, 1900 ] 185 0.072% 99.901% ( 1900, 2900 ] 144 0.056% 99.956% ( 2900, 4400 ] 125 0.048% 100.005% ( 4400, 6600 ] 44 0.017% 100.022% ( 6600, 9900 ] 6 0.002% 100.024% ( 9900, 14000 ] 3 0.001% 100.025% ( 14000, 22000 ] 5 0.002% 100.027% ** Level 2 read latency histogram (micros): Count: 875301 Average: 17.5142 StdDev: 210.48 Min: 1 Median: 1.9776 Max: 17320 Percentiles: P50: 1.98 P75: 3.05 P99: 13.11 P99.9: 2147.33 P99.99: 2665.94 ------------------------------------------------------ [ 0, 1 ] 27344 3.124% 3.124% # ( 1, 2 ] 419719 47.951% 51.075% ########## ( 2, 3 ] 205609 23.490% 74.565% ##### ( 3, 4 ] 84112 9.609% 84.175% ## ( 4, 6 ] 100795 11.515% 95.690% ## ( 6, 10 ] 24402 2.788% 98.478% # ( 10, 15 ] 7343 0.839% 99.317% ( 15, 22 ] 1131 0.129% 99.446% ( 22, 34 ] 114 0.013% 99.459% ( 34, 51 ] 43 0.005% 99.464% ( 51, 76 ] 53 0.006% 99.470% ( 76, 110 ] 10 0.001% 99.471% ( 110, 170 ] 33 0.004% 99.475% ( 170, 250 ] 75 0.009% 99.484% ( 250, 380 ] 161 0.018% 99.502% ( 380, 580 ] 332 0.038% 99.540% ( 580, 870 ] 604 0.069% 99.609% ( 870, 1300 ] 963 0.110% 99.719% ( 1300, 1900 ] 1207 0.138% 99.857% ( 1900, 2900 ] 1519 0.174% 100.031% ( 2900, 4400 ] 988 0.113% 100.143% ( 4400, 6600 ] 290 0.033% 100.177% ( 6600, 9900 ] 37 0.004% 100.181% ( 9900, 14000 ] 16 0.002% 100.183% ( 14000, 22000 ] 6 0.001% 100.183% ** Level 3 read latency histogram (micros): Count: 2232015 Average: 13.0729 StdDev: 181.13 Min: 1 Median: 2.0213 Max: 47318 Percentiles: P50: 2.02 P75: 3.06 P99: 11.37 P99.9: 1482.12 P99.99: 2198.54 ------------------------------------------------------ [ 0, 1 ] 43303 1.940% 1.940% ( 1, 2 ] 1060885 47.530% 49.470% ########## ( 2, 3 ] 555905 24.906% 74.376% ##### ( 3, 4 ] 220694 9.888% 84.264% ## ( 4, 6 ] 265441 11.892% 96.157% ## ( 6, 10 ] 59611 2.671% 98.827% # ( 10, 15 ] 14107 0.632% 99.459% ( 15, 22 ] 1586 0.071% 99.530% ( 22, 34 ] 140 0.006% 99.537% ( 34, 51 ] 142 0.006% 99.543% ( 51, 76 ] 197 0.009% 99.552% ( 76, 110 ] 50 0.002% 99.554% ( 110, 170 ] 183 0.008% 99.562% ( 170, 250 ] 430 0.019% 99.581% ( 250, 380 ] 808 0.036% 99.618% ( 380, 580 ] 1318 0.059% 99.677% ( 580, 870 ] 2095 0.094% 99.771% ( 870, 1300 ] 2280 0.102% 99.873% ( 1300, 1900 ] 2003 0.090% 99.963% ( 1900, 2900 ] 2056 0.092% 100.055% ( 2900, 4400 ] 1667 0.075% 100.129% ( 4400, 6600 ] 459 0.021% 100.150% ( 6600, 9900 ] 74 0.003% 100.153% ( 9900, 14000 ] 11 0.000% 100.154% ( 14000, 22000 ] 17 0.001% 100.154% ( 22000, 33000 ] 4 0.000% 100.155% ( 33000, 50000 ] 4 0.000% 100.155% ** Level 4 read latency histogram (micros): Count: 1131596 Average: 20.2155 StdDev: 255.96 Min: 1 Median: 2.0329 Max: 49655 Percentiles: P50: 2.03 P75: 3.15 P99: 210.08 P99.9: 2332.86 P99.99: 3244.06 ------------------------------------------------------ [ 0, 1 ] 22303 1.971% 1.971% ( 1, 2 ] 534458 47.230% 49.201% ######### ( 2, 3 ] 274789 24.283% 73.485% ##### ( 3, 4 ] 113192 10.003% 83.488% ## ( 4, 6 ] 135176 11.946% 95.433% ## ( 6, 10 ] 31138 2.752% 98.185% # ( 10, 15 ] 7580 0.670% 98.855% ( 15, 22 ] 834 0.074% 98.928% ( 22, 34 ] 64 0.006% 98.934% ( 34, 51 ] 61 0.005% 98.939% ( 51, 76 ] 122 0.011% 98.950% ( 76, 110 ] 52 0.005% 98.955% ( 110, 170 ] 230 0.020% 98.975% ( 170, 250 ] 561 0.050% 99.025% ( 250, 380 ] 965 0.085% 99.110% ( 380, 580 ] 1758 0.155% 99.265% ( 580, 870 ] 2513 0.222% 99.487% ( 870, 1300 ] 2486 0.220% 99.707% ( 1300, 1900 ] 1608 0.142% 99.849% ( 1900, 2900 ] 1327 0.117% 99.967% ( 2900, 4400 ] 1159 0.102% 100.069% ( 4400, 6600 ] 445 0.039% 100.108% ( 6600, 9900 ] 65 0.006% 100.114% ( 9900, 14000 ] 14 0.001% 100.115% ( 14000, 22000 ] 30 0.003% 100.118% ( 22000, 33000 ] 9 0.001% 100.119% ( 33000, 50000 ] 6 0.001% 100.119% ** Level 5 read latency histogram (micros): Count: 43739 Average: 1384.1178 StdDev: 1510.65 Min: 84 Median: 893.4262 Max: 57962 Percentiles: P50: 893.43 P75: 1730.88 P99: 6376.03 P99.9: 9835.11 P99.99: 39849.30 ------------------------------------------------------ ( 76, 110 ] 88 0.201% 0.201% ( 110, 170 ] 904 2.067% 2.268% ( 170, 250 ] 1997 4.566% 6.834% # ( 250, 380 ] 3586 8.199% 15.032% ## ( 380, 580 ] 6296 14.394% 29.427% ### ( 580, 870 ] 8571 19.596% 49.023% #### ( 870, 1300 ] 7847 17.941% 66.963% #### ( 1300, 1900 ] 4895 11.191% 78.155% ## ( 1900, 2900 ] 3970 9.077% 87.231% ## ( 2900, 4400 ] 3759 8.594% 95.825% ## ( 4400, 6600 ] 1546 3.535% 99.360% # ( 6600, 9900 ] 241 0.551% 99.911% ( 9900, 14000 ] 10 0.023% 99.934% ( 14000, 22000 ] 13 0.030% 99.963% ( 22000, 33000 ] 8 0.018% 99.982% ( 33000, 50000 ] 9 0.021% 100.002% ( 50000, 75000 ] 1 0.002% 100.005% ** DB Stats ** Uptime(secs): 20.7 total, 5.8 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 142.42 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:8.871 H:M:S, 42.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 170.46 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.815 H:M:S, 31.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 2024/12/16-21:10:35 ... thread 0: (1000000,4000000) ops and (288459.3,166873.8) ops/second in (3.466694,23.970205) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/9 2.77 GB 144.7 2.6 0.0 2.6 6.3 3.7 0.0 1.7 74.5 182.6 35.52 21.24 59 0.602 2686K 3081 0.0 0.0 L1 5/5 429.68 MB 0.0 1.2 1.0 0.1 1.1 1.0 0.0 1.1 130.6 130.2 9.04 5.51 2 4.520 1199K 3274 0.0 0.0 L2 1/0 49.66 MB 0.5 0.3 0.1 0.3 0.3 0.1 0.7 5.3 118.4 118.1 2.68 1.49 1 2.676 321K 846 0.0 0.0 L3 17/0 916.20 MB 0.9 7.9 4.1 3.8 7.9 4.0 0.1 1.9 109.1 108.4 74.36 37.80 35 2.124 5729K 14K 0.0 0.0 L4 165/0 9.41 GB 1.0 9.7 5.2 4.5 9.5 5.0 0.6 1.8 121.3 119.0 81.58 46.42 64 1.275 5142K 8070 0.0 0.0 L5 1160/0 95.84 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1363/14 109.38 GB 0.0 21.6 10.4 11.3 25.1 13.9 7.0 6.7 109.1 126.7 203.17 112.46 161 1.262 15M 30K 0.0 0.0 Int 0/0 0.00 KB 0.0 21.6 10.4 11.3 25.1 13.9 7.0 6.7 109.1 126.7 203.17 112.46 161 1.262 15M 30K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 21.6 10.4 11.3 21.4 10.1 0.0 0.0 122.9 121.5 180.34 101.66 111 1.625 15M 30K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 168.3 22.83 10.80 50 0.457 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 24.2 total, 24.0 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 25.15 GB write, 1064.13 MB/s write, 21.64 GB read, 915.71 MB/s read, 203.2 seconds Interval compaction: 25.15 GB write, 1074.16 MB/s write, 21.64 GB read, 924.34 MB/s read, 203.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 85, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 85, interval: 85 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1101359 Average: 10.9199 StdDev: 126.99 Min: 1 Median: 1.7866 Max: 32931 Percentiles: P50: 1.79 P75: 2.72 P99: 195.89 P99.9: 1281.83 P99.99: 2949.08 ------------------------------------------------------ [ 0, 1 ] 90661 8.232% 8.232% ## ( 1, 2 ] 584785 53.097% 61.328% ########### ( 2, 3 ] 207762 18.864% 80.193% #### ( 3, 4 ] 82251 7.468% 87.661% # ( 4, 6 ] 94589 8.588% 96.249% ## ( 6, 10 ] 18631 1.692% 97.941% ( 10, 15 ] 3599 0.327% 98.268% ( 15, 22 ] 365 0.033% 98.301% ( 22, 34 ] 35 0.003% 98.304% ( 34, 51 ] 35 0.003% 98.307% ( 51, 76 ] 41 0.004% 98.311% ( 76, 110 ] 514 0.047% 98.357% ( 110, 170 ] 5437 0.494% 98.851% ( 170, 250 ] 5072 0.461% 99.312% ( 250, 380 ] 2284 0.207% 99.519% ( 380, 580 ] 1952 0.177% 99.696% ( 580, 870 ] 1376 0.125% 99.821% ( 870, 1300 ] 908 0.082% 99.904% ( 1300, 1900 ] 540 0.049% 99.953% ( 1900, 2900 ] 402 0.037% 99.989% ( 2900, 4400 ] 332 0.030% 100.019% ( 4400, 6600 ] 87 0.008% 100.027% ( 6600, 9900 ] 17 0.002% 100.029% ( 9900, 14000 ] 1 0.000% 100.029% ( 14000, 22000 ] 4 0.000% 100.029% ( 22000, 33000 ] 4 0.000% 100.030% ** Level 1 read latency histogram (micros): Count: 277239 Average: 11.2136 StdDev: 155.67 Min: 1 Median: 1.9843 Max: 18243 Percentiles: P50: 1.98 P75: 3.00 P99: 13.82 P99.9: 1837.60 P99.99: 3923.31 ------------------------------------------------------ [ 0, 1 ] 6068 2.189% 2.189% ( 1, 2 ] 134661 48.572% 50.761% ########## ( 2, 3 ] 67106 24.205% 74.966% ##### ( 3, 4 ] 27007 9.741% 84.707% ## ( 4, 6 ] 31350 11.308% 96.015% ## ( 6, 10 ] 7054 2.544% 98.560% # ( 10, 15 ] 1596 0.576% 99.135% ( 15, 22 ] 216 0.078% 99.213% ( 22, 34 ] 24 0.009% 99.222% ( 34, 51 ] 29 0.010% 99.232% ( 51, 76 ] 22 0.008% 99.240% ( 76, 110 ] 28 0.010% 99.250% ( 110, 170 ] 276 0.100% 99.350% ( 170, 250 ] 264 0.095% 99.445% ( 250, 380 ] 233 0.084% 99.529% ( 380, 580 ] 280 0.101% 99.630% ( 580, 870 ] 299 0.108% 99.738% ( 870, 1300 ] 283 0.102% 99.840% ( 1300, 1900 ] 185 0.067% 99.907% ( 1900, 2900 ] 145 0.052% 99.959% ( 2900, 4400 ] 125 0.045% 100.004% ( 4400, 6600 ] 44 0.016% 100.020% ( 6600, 9900 ] 6 0.002% 100.022% ( 9900, 14000 ] 3 0.001% 100.023% ( 14000, 22000 ] 5 0.002% 100.025% ** Level 2 read latency histogram (micros): Count: 875301 Average: 17.5142 StdDev: 210.48 Min: 1 Median: 1.9776 Max: 17320 Percentiles: P50: 1.98 P75: 3.05 P99: 13.11 P99.9: 2147.33 P99.99: 2665.94 ------------------------------------------------------ [ 0, 1 ] 27344 3.124% 3.124% # ( 1, 2 ] 419719 47.951% 51.075% ########## ( 2, 3 ] 205609 23.490% 74.565% ##### ( 3, 4 ] 84112 9.609% 84.175% ## ( 4, 6 ] 100795 11.515% 95.690% ## ( 6, 10 ] 24402 2.788% 98.478% # ( 10, 15 ] 7343 0.839% 99.317% ( 15, 22 ] 1131 0.129% 99.446% ( 22, 34 ] 114 0.013% 99.459% ( 34, 51 ] 43 0.005% 99.464% ( 51, 76 ] 53 0.006% 99.470% ( 76, 110 ] 10 0.001% 99.471% ( 110, 170 ] 33 0.004% 99.475% ( 170, 250 ] 75 0.009% 99.484% ( 250, 380 ] 161 0.018% 99.502% ( 380, 580 ] 332 0.038% 99.540% ( 580, 870 ] 604 0.069% 99.609% ( 870, 1300 ] 963 0.110% 99.719% ( 1300, 1900 ] 1207 0.138% 99.857% ( 1900, 2900 ] 1519 0.174% 100.031% ( 2900, 4400 ] 988 0.113% 100.143% ( 4400, 6600 ] 290 0.033% 100.177% ( 6600, 9900 ] 37 0.004% 100.181% ( 9900, 14000 ] 16 0.002% 100.183% ( 14000, 22000 ] 6 0.001% 100.183% ** Level 3 read latency histogram (micros): Count: 2266792 Average: 13.0037 StdDev: 179.83 Min: 1 Median: 2.0089 Max: 47318 Percentiles: P50: 2.01 P75: 3.04 P99: 11.57 P99.9: 1471.41 P99.99: 2196.22 ------------------------------------------------------ [ 0, 1 ] 45383 2.002% 2.002% ( 1, 2 ] 1083011 47.777% 49.779% ########## ( 2, 3 ] 562071 24.796% 74.575% ##### ( 3, 4 ] 222176 9.801% 84.377% ## ( 4, 6 ] 266780 11.769% 96.146% ## ( 6, 10 ] 60238 2.657% 98.803% # ( 10, 15 ] 14181 0.626% 99.429% ( 15, 22 ] 1590 0.070% 99.499% ( 22, 34 ] 140 0.006% 99.505% ( 34, 51 ] 142 0.006% 99.511% ( 51, 76 ] 199 0.009% 99.520% ( 76, 110 ] 90 0.004% 99.524% ( 110, 170 ] 528 0.023% 99.547% ( 170, 250 ] 695 0.031% 99.578% ( 250, 380 ] 917 0.040% 99.618% ( 380, 580 ] 1393 0.061% 99.680% ( 580, 870 ] 2124 0.094% 99.774% ( 870, 1300 ] 2295 0.101% 99.875% ( 1300, 1900 ] 2003 0.088% 99.963% ( 1900, 2900 ] 2057 0.091% 100.054% ( 2900, 4400 ] 1667 0.074% 100.127% ( 4400, 6600 ] 459 0.020% 100.148% ( 6600, 9900 ] 74 0.003% 100.151% ( 9900, 14000 ] 11 0.000% 100.151% ( 14000, 22000 ] 17 0.001% 100.152% ( 22000, 33000 ] 4 0.000% 100.152% ( 33000, 50000 ] 4 0.000% 100.153% ** Level 4 read latency histogram (micros): Count: 1180998 Average: 20.7193 StdDev: 251.35 Min: 1 Median: 2.0109 Max: 49655 Percentiles: P50: 2.01 P75: 3.14 P99: 312.82 P99.9: 2290.23 P99.99: 3226.01 ------------------------------------------------------ [ 0, 1 ] 26674 2.259% 2.259% ( 1, 2 ] 560759 47.482% 49.740% ######### ( 2, 3 ] 282461 23.917% 73.658% ##### ( 3, 4 ] 115122 9.748% 83.405% ## ( 4, 6 ] 137166 11.614% 95.020% ## ( 6, 10 ] 31928 2.703% 97.723% # ( 10, 15 ] 7688 0.651% 98.374% ( 15, 22 ] 843 0.071% 98.446% ( 22, 34 ] 64 0.005% 98.451% ( 34, 51 ] 62 0.005% 98.456% ( 51, 76 ] 122 0.010% 98.467% ( 76, 110 ] 306 0.026% 98.493% ( 110, 170 ] 2527 0.214% 98.707% ( 170, 250 ] 2615 0.221% 98.928% ( 250, 380 ] 1761 0.149% 99.077% ( 380, 580 ] 2326 0.197% 99.274% ( 580, 870 ] 2728 0.231% 99.505% ( 870, 1300 ] 2529 0.214% 99.719% ( 1300, 1900 ] 1617 0.137% 99.856% ( 1900, 2900 ] 1330 0.113% 99.969% ( 2900, 4400 ] 1159 0.098% 100.067% ( 4400, 6600 ] 445 0.038% 100.104% ( 6600, 9900 ] 65 0.006% 100.110% ( 9900, 14000 ] 14 0.001% 100.111% ( 14000, 22000 ] 30 0.003% 100.114% ( 22000, 33000 ] 9 0.001% 100.114% ( 33000, 50000 ] 6 0.001% 100.115% ** Level 5 read latency histogram (micros): Count: 81503 Average: 853.5806 StdDev: 1250.57 Min: 84 Median: 409.6470 Max: 57962 Percentiles: P50: 409.65 P75: 989.69 P99: 5837.21 P99.9: 9304.32 P99.99: 31418.34 ------------------------------------------------------ ( 76, 110 ] 1498 1.838% 1.838% ( 110, 170 ] 15168 18.610% 20.448% #### ( 170, 250 ] 14171 17.387% 37.835% ### ( 250, 380 ] 8495 10.423% 48.258% ## ( 380, 580 ] 9576 11.749% 60.008% ## ( 580, 870 ] 9956 12.216% 72.223% ## ( 870, 1300 ] 8131 9.976% 82.199% ## ( 1300, 1900 ] 4926 6.044% 88.243% # ( 1900, 2900 ] 3990 4.896% 93.139% # ( 2900, 4400 ] 3767 4.622% 97.761% # ( 4400, 6600 ] 1546 1.897% 99.658% ( 6600, 9900 ] 241 0.296% 99.953% ( 9900, 14000 ] 10 0.012% 99.966% ( 14000, 22000 ] 13 0.016% 99.982% ( 22000, 33000 ] 8 0.010% 99.991% ( 33000, 50000 ] 9 0.011% 100.002% ( 50000, 75000 ] 1 0.001% 100.004% ** DB Stats ** Uptime(secs): 24.2 total, 3.5 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 162.69 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:8.871 H:M:S, 36.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 283.90 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/16-21:10:39 ... thread 0: (1000000,5000000) ops and (243491.3,178080.9) ops/second in (4.106923,28.077128) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/9 3.75 GB 245.4 3.6 0.0 3.6 8.3 4.7 0.0 1.8 85.7 199.3 42.65 27.76 79 0.540 3709K 4057 0.0 0.0 L1 5/5 429.68 MB 0.0 1.2 1.0 0.1 1.1 1.0 0.0 1.1 130.6 130.2 9.04 5.51 2 4.520 1199K 3274 0.0 0.0 L2 1/0 49.66 MB 0.5 0.3 0.1 0.3 0.3 0.1 0.7 5.3 118.4 118.1 2.68 1.49 1 2.676 321K 846 0.0 0.0 L3 17/0 916.20 MB 0.9 7.9 4.1 3.8 7.9 4.0 0.1 1.9 109.1 108.4 74.36 37.80 35 2.124 5729K 14K 0.0 0.0 L4 165/0 9.41 GB 1.0 9.7 5.2 4.5 9.5 5.0 0.6 1.8 121.3 119.0 81.58 46.42 64 1.275 5142K 8070 0.0 0.0 L5 1160/0 95.84 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1367/14 110.36 GB 0.0 22.6 10.4 12.3 27.1 14.8 7.0 5.7 110.2 132.0 210.30 118.98 181 1.162 16M 31K 0.0 0.0 Int 0/0 0.00 KB 0.0 22.6 10.4 12.3 27.1 14.8 7.0 5.7 110.2 132.0 210.30 118.98 181 1.162 16M 31K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 22.6 10.4 12.3 22.4 10.1 0.0 0.0 125.7 124.4 184.25 105.48 115 1.602 16M 31K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 186.2 26.06 13.50 66 0.395 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 28.3 total, 28.1 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 27.11 GB write, 980.92 MB/s write, 22.62 GB read, 818.46 MB/s read, 210.3 seconds Interval compaction: 27.11 GB write, 988.81 MB/s write, 22.62 GB read, 825.05 MB/s read, 210.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 85, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 85, interval: 89 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1381845 Average: 12.3118 StdDev: 117.73 Min: 1 Median: 1.7183 Max: 32931 Percentiles: P50: 1.72 P75: 2.60 P99: 233.31 P99.9: 1108.47 P99.99: 2497.67 ------------------------------------------------------ [ 0, 1 ] 159225 11.523% 11.523% ## ( 1, 2 ] 740185 53.565% 65.088% ########### ( 2, 3 ] 229200 16.587% 81.674% ### ( 3, 4 ] 90044 6.516% 88.190% # ( 4, 6 ] 99916 7.231% 95.421% # ( 6, 10 ] 19685 1.425% 96.846% ( 10, 15 ] 3868 0.280% 97.125% ( 15, 22 ] 404 0.029% 97.155% ( 22, 34 ] 42 0.003% 97.158% ( 34, 51 ] 35 0.003% 97.160% ( 51, 76 ] 51 0.004% 97.164% ( 76, 110 ] 1311 0.095% 97.259% ( 110, 170 ] 14524 1.051% 98.310% ( 170, 250 ] 12051 0.872% 99.182% ( 250, 380 ] 4252 0.308% 99.490% ( 380, 580 ] 3300 0.239% 99.728% ( 580, 870 ] 1826 0.132% 99.861% ( 870, 1300 ] 983 0.071% 99.932% ( 1300, 1900 ] 553 0.040% 99.972% ( 1900, 2900 ] 423 0.031% 100.002% ( 2900, 4400 ] 337 0.024% 100.027% ( 4400, 6600 ] 87 0.006% 100.033% ( 6600, 9900 ] 17 0.001% 100.034% ( 9900, 14000 ] 1 0.000% 100.034% ( 14000, 22000 ] 4 0.000% 100.035% ( 22000, 33000 ] 4 0.000% 100.035% ** Level 1 read latency histogram (micros): Count: 289450 Average: 11.4850 StdDev: 152.97 Min: 1 Median: 2.0344 Max: 18243 Percentiles: P50: 2.03 P75: 3.16 P99: 68.61 P99.9: 1801.24 P99.99: 3896.66 ------------------------------------------------------ [ 0, 1 ] 6119 2.114% 2.114% ( 1, 2 ] 136205 47.056% 49.170% ######### ( 2, 3 ] 69882 24.143% 73.314% ##### ( 3, 4 ] 29997 10.363% 83.677% ## ( 4, 6 ] 34385 11.879% 95.556% ## ( 6, 10 ] 7753 2.679% 98.235% # ( 10, 15 ] 1788 0.618% 98.853% ( 15, 22 ] 341 0.118% 98.970% ( 22, 34 ] 40 0.014% 98.984% ( 34, 51 ] 30 0.010% 98.995% ( 51, 76 ] 22 0.008% 99.002% ( 76, 110 ] 65 0.022% 99.025% ( 110, 170 ] 642 0.222% 99.247% ( 170, 250 ] 516 0.178% 99.425% ( 250, 380 ] 293 0.101% 99.526% ( 380, 580 ] 325 0.112% 99.638% ( 580, 870 ] 317 0.110% 99.748% ( 870, 1300 ] 286 0.099% 99.847% ( 1300, 1900 ] 185 0.064% 99.911% ( 1900, 2900 ] 147 0.051% 99.961% ( 2900, 4400 ] 125 0.043% 100.004% ( 4400, 6600 ] 44 0.015% 100.020% ( 6600, 9900 ] 6 0.002% 100.022% ( 9900, 14000 ] 3 0.001% 100.023% ( 14000, 22000 ] 5 0.002% 100.025% ** Level 2 read latency histogram (micros): Count: 875301 Average: 17.5142 StdDev: 210.48 Min: 1 Median: 1.9776 Max: 17320 Percentiles: P50: 1.98 P75: 3.05 P99: 13.11 P99.9: 2147.33 P99.99: 2665.94 ------------------------------------------------------ [ 0, 1 ] 27344 3.124% 3.124% # ( 1, 2 ] 419719 47.951% 51.075% ########## ( 2, 3 ] 205609 23.490% 74.565% ##### ( 3, 4 ] 84112 9.609% 84.175% ## ( 4, 6 ] 100795 11.515% 95.690% ## ( 6, 10 ] 24402 2.788% 98.478% # ( 10, 15 ] 7343 0.839% 99.317% ( 15, 22 ] 1131 0.129% 99.446% ( 22, 34 ] 114 0.013% 99.459% ( 34, 51 ] 43 0.005% 99.464% ( 51, 76 ] 53 0.006% 99.470% ( 76, 110 ] 10 0.001% 99.471% ( 110, 170 ] 33 0.004% 99.475% ( 170, 250 ] 75 0.009% 99.484% ( 250, 380 ] 161 0.018% 99.502% ( 380, 580 ] 332 0.038% 99.540% ( 580, 870 ] 604 0.069% 99.609% ( 870, 1300 ] 963 0.110% 99.719% ( 1300, 1900 ] 1207 0.138% 99.857% ( 1900, 2900 ] 1519 0.174% 100.031% ( 2900, 4400 ] 988 0.113% 100.143% ( 4400, 6600 ] 290 0.033% 100.177% ( 6600, 9900 ] 37 0.004% 100.181% ( 9900, 14000 ] 16 0.002% 100.183% ( 14000, 22000 ] 6 0.001% 100.183% ** Level 3 read latency histogram (micros): Count: 2267992 Average: 13.1076 StdDev: 179.86 Min: 1 Median: 2.0100 Max: 47318 Percentiles: P50: 2.01 P75: 3.05 P99: 11.99 P99.9: 1471.05 P99.99: 2196.16 ------------------------------------------------------ [ 0, 1 ] 45383 2.001% 2.001% ( 1, 2 ] 1083011 47.752% 49.753% ########## ( 2, 3 ] 562071 24.783% 74.536% ##### ( 3, 4 ] 222176 9.796% 84.332% ## ( 4, 6 ] 266780 11.763% 96.095% ## ( 6, 10 ] 60238 2.656% 98.751% # ( 10, 15 ] 14181 0.625% 99.376% ( 15, 22 ] 1590 0.070% 99.446% ( 22, 34 ] 140 0.006% 99.452% ( 34, 51 ] 142 0.006% 99.459% ( 51, 76 ] 199 0.009% 99.467% ( 76, 110 ] 152 0.007% 99.474% ( 110, 170 ] 1070 0.047% 99.521% ( 170, 250 ] 1073 0.047% 99.569% ( 250, 380 ] 1023 0.045% 99.614% ( 380, 580 ] 1477 0.065% 99.679% ( 580, 870 ] 2148 0.095% 99.773% ( 870, 1300 ] 2299 0.101% 99.875% ( 1300, 1900 ] 2003 0.088% 99.963% ( 1900, 2900 ] 2057 0.091% 100.054% ( 2900, 4400 ] 1667 0.074% 100.127% ( 4400, 6600 ] 459 0.020% 100.148% ( 6600, 9900 ] 74 0.003% 100.151% ( 9900, 14000 ] 11 0.000% 100.151% ( 14000, 22000 ] 17 0.001% 100.152% ( 22000, 33000 ] 4 0.000% 100.152% ( 33000, 50000 ] 4 0.000% 100.152% ** Level 4 read latency histogram (micros): Count: 1189872 Average: 22.0196 StdDev: 251.14 Min: 1 Median: 2.0265 Max: 49655 Percentiles: P50: 2.03 P75: 3.19 P99: 361.96 P99.9: 2285.86 P99.99: 3224.87 ------------------------------------------------------ [ 0, 1 ] 26674 2.242% 2.242% ( 1, 2 ] 560782 47.130% 49.371% ######### ( 2, 3 ] 282617 23.752% 73.123% ##### ( 3, 4 ] 115235 9.685% 82.808% ## ( 4, 6 ] 137330 11.542% 94.349% ## ( 6, 10 ] 31942 2.684% 97.034% # ( 10, 15 ] 7695 0.647% 97.681% ( 15, 22 ] 843 0.071% 97.752% ( 22, 34 ] 64 0.005% 97.757% ( 34, 51 ] 62 0.005% 97.762% ( 51, 76 ] 122 0.010% 97.772% ( 76, 110 ] 731 0.061% 97.834% ( 110, 170 ] 6348 0.534% 98.367% ( 170, 250 ] 5351 0.450% 98.817% ( 250, 380 ] 2528 0.212% 99.029% ( 380, 580 ] 2778 0.233% 99.263% ( 580, 870 ] 2893 0.243% 99.506% ( 870, 1300 ] 2553 0.215% 99.721% ( 1300, 1900 ] 1619 0.136% 99.857% ( 1900, 2900 ] 1335 0.112% 99.969% ( 2900, 4400 ] 1159 0.097% 100.066% ( 4400, 6600 ] 445 0.037% 100.104% ( 6600, 9900 ] 65 0.005% 100.109% ( 9900, 14000 ] 14 0.001% 100.110% ( 14000, 22000 ] 30 0.003% 100.113% ( 22000, 33000 ] 9 0.001% 100.114% ( 33000, 50000 ] 6 0.001% 100.114% ** Level 5 read latency histogram (micros): Count: 130518 Average: 611.2130 StdDev: 1040.00 Min: 81 Median: 233.6573 Max: 57962 Percentiles: P50: 233.66 P75: 609.79 P99: 5138.29 P99.9: 8619.46 P99.99: 23303.77 ------------------------------------------------------ ( 76, 110 ] 3879 2.972% 2.972% # ( 110, 170 ] 37752 28.925% 31.897% ###### ( 170, 250 ] 29694 22.751% 54.648% ##### ( 250, 380 ] 12869 9.860% 64.508% ## ( 380, 580 ] 12574 9.634% 74.141% ## ( 580, 870 ] 10907 8.357% 82.498% ## ( 870, 1300 ] 8278 6.342% 88.841% # ( 1300, 1900 ] 4949 3.792% 92.632% # ( 1900, 2900 ] 4014 3.075% 95.708% # ( 2900, 4400 ] 3778 2.895% 98.602% # ( 4400, 6600 ] 1546 1.185% 99.787% ( 6600, 9900 ] 241 0.185% 99.972% ( 9900, 14000 ] 10 0.008% 99.979% ( 14000, 22000 ] 13 0.010% 99.989% ( 22000, 33000 ] 8 0.006% 99.995% ( 33000, 50000 ] 9 0.007% 100.002% ( 50000, 75000 ] 1 0.001% 100.003% ** DB Stats ** Uptime(secs): 28.3 total, 4.1 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 173.85 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.472 H:M:S, 33.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 239.64 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.602 H:M:S, 14.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/16-21:10:49 ... thread 0: (1000000,6000000) ops and (98528.9,156959.4) ops/second in (10.149306,38.226434) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/4 3.55 GB 338.8 5.8 0.0 5.8 11.5 5.6 0.0 2.0 110.3 216.8 54.26 38.79 98 0.554 6075K 15K 0.0 0.0 L1 14/0 1.52 GB 155.2 2.7 2.1 0.5 2.7 2.1 0.0 1.2 91.9 91.5 29.83 14.89 3 9.945 2785K 13K 0.0 0.0 L2 1/0 49.66 MB 0.5 0.3 0.1 0.3 0.3 0.1 0.7 5.3 118.4 118.1 2.68 1.49 1 2.676 321K 846 0.0 0.0 L3 17/0 916.20 MB 0.9 7.9 4.1 3.8 7.9 4.0 0.1 1.9 109.1 108.4 74.36 37.80 35 2.124 5729K 14K 0.0 0.0 L4 165/0 9.41 GB 1.0 9.7 5.2 4.5 9.5 5.0 0.6 1.8 121.3 119.0 81.58 46.42 64 1.275 5142K 8070 0.0 0.0 L5 1160/0 95.84 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1371/4 111.26 GB 0.0 26.4 11.5 15.0 31.8 16.9 7.0 5.6 111.5 134.2 242.71 139.39 201 1.207 20M 52K 0.0 0.0 Int 0/0 0.00 KB 0.0 26.4 11.5 15.0 31.8 16.9 7.0 5.6 111.5 134.2 242.71 139.39 201 1.207 20M 52K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 26.4 11.5 15.0 26.2 11.2 0.0 0.0 126.7 125.4 213.63 123.34 120 1.780 20M 52K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 199.3 29.07 16.06 81 0.359 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 38.5 total, 38.2 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 31.82 GB write, 847.24 MB/s write, 26.42 GB read, 703.65 MB/s read, 242.7 seconds Interval compaction: 31.82 GB write, 852.25 MB/s write, 26.42 GB read, 707.81 MB/s read, 242.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 12, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 12, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 85, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 12, total-stops: 85, interval: 97 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2179171 Average: 12.9532 StdDev: 99.99 Min: 1 Median: 1.6099 Max: 32931 Percentiles: P50: 1.61 P75: 2.29 P99: 236.26 P99.9: 795.56 P99.99: 1573.57 ------------------------------------------------------ [ 0, 1 ] 368131 16.893% 16.893% ### ( 1, 2 ] 1182871 54.281% 71.174% ########### ( 2, 3 ] 285817 13.116% 84.290% ### ( 3, 4 ] 112131 5.146% 89.435% # ( 4, 6 ] 111899 5.135% 94.570% # ( 6, 10 ] 23246 1.067% 95.637% ( 10, 15 ] 4687 0.215% 95.852% ( 15, 22 ] 556 0.026% 95.878% ( 22, 34 ] 71 0.003% 95.881% ( 34, 51 ] 38 0.002% 95.883% ( 51, 76 ] 53 0.002% 95.885% ( 76, 110 ] 3932 0.180% 96.066% ( 110, 170 ] 41029 1.883% 97.948% ( 170, 250 ] 27671 1.270% 99.218% ( 250, 380 ] 7715 0.354% 99.572% ( 380, 580 ] 5320 0.244% 99.816% ( 580, 870 ] 2455 0.113% 99.929% ( 870, 1300 ] 1068 0.049% 99.978% ( 1300, 1900 ] 577 0.026% 100.004% ( 1900, 2900 ] 441 0.020% 100.025% ( 2900, 4400 ] 358 0.016% 100.041% ( 4400, 6600 ] 90 0.004% 100.045% ( 6600, 9900 ] 17 0.001% 100.046% ( 9900, 14000 ] 1 0.000% 100.046% ( 14000, 22000 ] 4 0.000% 100.046% ( 22000, 33000 ] 4 0.000% 100.046% ** Level 1 read latency histogram (micros): Count: 309496 Average: 12.3680 StdDev: 149.74 Min: 1 Median: 2.0106 Max: 18243 Percentiles: P50: 2.01 P75: 3.18 P99: 171.50 P99.9: 1760.22 P99.99: 3876.79 ------------------------------------------------------ [ 0, 1 ] 7879 2.546% 2.546% # ( 1, 2 ] 146101 47.206% 49.752% ######### ( 2, 3 ] 72468 23.415% 73.167% ##### ( 3, 4 ] 31361 10.133% 83.300% ## ( 4, 6 ] 36057 11.650% 94.950% ## ( 6, 10 ] 8204 2.651% 97.601% # ( 10, 15 ] 1855 0.599% 98.200% ( 15, 22 ] 384 0.124% 98.324% ( 22, 34 ] 46 0.015% 98.339% ( 34, 51 ] 30 0.010% 98.349% ( 51, 76 ] 23 0.007% 98.356% ( 76, 110 ] 194 0.063% 98.419% ( 110, 170 ] 1778 0.574% 98.993% ( 170, 250 ] 1124 0.363% 99.356% ( 250, 380 ] 451 0.146% 99.502% ( 380, 580 ] 431 0.139% 99.641% ( 580, 870 ] 352 0.114% 99.755% ( 870, 1300 ] 302 0.098% 99.853% ( 1300, 1900 ] 191 0.062% 99.914% ( 1900, 2900 ] 152 0.049% 99.963% ( 2900, 4400 ] 126 0.041% 100.004% ( 4400, 6600 ] 44 0.014% 100.018% ( 6600, 9900 ] 6 0.002% 100.020% ( 9900, 14000 ] 3 0.001% 100.021% ( 14000, 22000 ] 5 0.002% 100.023% ** Level 2 read latency histogram (micros): Count: 875301 Average: 17.5142 StdDev: 210.48 Min: 1 Median: 1.9776 Max: 17320 Percentiles: P50: 1.98 P75: 3.05 P99: 13.11 P99.9: 2147.33 P99.99: 2665.94 ------------------------------------------------------ [ 0, 1 ] 27344 3.124% 3.124% # ( 1, 2 ] 419719 47.951% 51.075% ########## ( 2, 3 ] 205609 23.490% 74.565% ##### ( 3, 4 ] 84112 9.609% 84.175% ## ( 4, 6 ] 100795 11.515% 95.690% ## ( 6, 10 ] 24402 2.788% 98.478% # ( 10, 15 ] 7343 0.839% 99.317% ( 15, 22 ] 1131 0.129% 99.446% ( 22, 34 ] 114 0.013% 99.459% ( 34, 51 ] 43 0.005% 99.464% ( 51, 76 ] 53 0.006% 99.470% ( 76, 110 ] 10 0.001% 99.471% ( 110, 170 ] 33 0.004% 99.475% ( 170, 250 ] 75 0.009% 99.484% ( 250, 380 ] 161 0.018% 99.502% ( 380, 580 ] 332 0.038% 99.540% ( 580, 870 ] 604 0.069% 99.609% ( 870, 1300 ] 963 0.110% 99.719% ( 1300, 1900 ] 1207 0.138% 99.857% ( 1900, 2900 ] 1519 0.174% 100.031% ( 2900, 4400 ] 988 0.113% 100.143% ( 4400, 6600 ] 290 0.033% 100.177% ( 6600, 9900 ] 37 0.004% 100.181% ( 9900, 14000 ] 16 0.002% 100.183% ( 14000, 22000 ] 6 0.001% 100.183% ** Level 3 read latency histogram (micros): Count: 2271059 Average: 13.3518 StdDev: 179.92 Min: 1 Median: 2.0127 Max: 47318 Percentiles: P50: 2.01 P75: 3.06 P99: 13.06 P99.9: 1470.43 P99.99: 2196.35 ------------------------------------------------------ [ 0, 1 ] 45383 1.998% 1.998% ( 1, 2 ] 1083011 47.687% 49.686% ########## ( 2, 3 ] 562071 24.749% 74.435% ##### ( 3, 4 ] 222176 9.783% 84.218% ## ( 4, 6 ] 266780 11.747% 95.965% ## ( 6, 10 ] 60238 2.652% 98.617% # ( 10, 15 ] 14181 0.624% 99.242% ( 15, 22 ] 1590 0.070% 99.312% ( 22, 34 ] 140 0.006% 99.318% ( 34, 51 ] 142 0.006% 99.324% ( 51, 76 ] 199 0.009% 99.333% ( 76, 110 ] 354 0.016% 99.349% ( 110, 170 ] 2638 0.116% 99.465% ( 170, 250 ] 1965 0.087% 99.551% ( 250, 380 ] 1223 0.054% 99.605% ( 380, 580 ] 1620 0.071% 99.676% ( 580, 870 ] 2199 0.097% 99.773% ( 870, 1300 ] 2309 0.102% 99.875% ( 1300, 1900 ] 2003 0.088% 99.963% ( 1900, 2900 ] 2058 0.091% 100.054% ( 2900, 4400 ] 1667 0.073% 100.127% ( 4400, 6600 ] 459 0.020% 100.147% ( 6600, 9900 ] 74 0.003% 100.151% ( 9900, 14000 ] 11 0.000% 100.151% ( 14000, 22000 ] 17 0.001% 100.152% ( 22000, 33000 ] 4 0.000% 100.152% ( 33000, 50000 ] 4 0.000% 100.152% ** Level 4 read latency histogram (micros): Count: 1213383 Average: 25.3473 StdDev: 250.45 Min: 1 Median: 2.0680 Max: 49655 Percentiles: P50: 2.07 P75: 3.35 P99: 431.46 P99.9: 2273.32 P99.99: 3224.86 ------------------------------------------------------ [ 0, 1 ] 26674 2.198% 2.198% ( 1, 2 ] 560794 46.217% 48.416% ######### ( 2, 3 ] 282693 23.298% 71.714% ##### ( 3, 4 ] 115308 9.503% 81.217% ## ( 4, 6 ] 137403 11.324% 92.541% ## ( 6, 10 ] 31953 2.633% 95.174% # ( 10, 15 ] 7697 0.634% 95.808% ( 15, 22 ] 843 0.069% 95.878% ( 22, 34 ] 64 0.005% 95.883% ( 34, 51 ] 62 0.005% 95.888% ( 51, 76 ] 122 0.010% 95.898% ( 76, 110 ] 2070 0.171% 96.069% ( 110, 170 ] 18404 1.517% 97.586% ( 170, 250 ] 11961 0.986% 98.571% ( 250, 380 ] 4197 0.346% 98.917% ( 380, 580 ] 3903 0.322% 99.239% ( 580, 870 ] 3257 0.268% 99.507% ( 870, 1300 ] 2632 0.217% 99.724% ( 1300, 1900 ] 1632 0.134% 99.859% ( 1900, 2900 ] 1341 0.111% 99.969% ( 2900, 4400 ] 1162 0.096% 100.065% ( 4400, 6600 ] 445 0.037% 100.102% ( 6600, 9900 ] 65 0.005% 100.107% ( 9900, 14000 ] 14 0.001% 100.108% ( 14000, 22000 ] 30 0.002% 100.111% ( 22000, 33000 ] 9 0.001% 100.111% ( 33000, 50000 ] 6 0.000% 100.112% ** Level 5 read latency histogram (micros): Count: 259474 Average: 404.2321 StdDev: 771.40 Min: 81 Median: 186.9440 Max: 57962 Percentiles: P50: 186.94 P75: 321.35 P99: 4094.56 P99.9: 6839.98 P99.99: 14032.37 ------------------------------------------------------ ( 76, 110 ] 11572 4.460% 4.460% # ( 110, 170 ] 103992 40.078% 44.538% ######## ( 170, 250 ] 66917 25.789% 70.327% ##### ( 250, 380 ] 22091 8.514% 78.841% ## ( 380, 580 ] 18620 7.176% 86.017% # ( 580, 870 ] 13004 5.012% 91.029% # ( 870, 1300 ] 8613 3.319% 94.348% # ( 1300, 1900 ] 5002 1.928% 96.276% ( 1900, 2900 ] 4050 1.561% 97.837% ( 2900, 4400 ] 3790 1.461% 99.297% ( 4400, 6600 ] 1546 0.596% 99.893% ( 6600, 9900 ] 241 0.093% 99.986% ( 9900, 14000 ] 10 0.004% 99.990% ( 14000, 22000 ] 13 0.005% 99.995% ( 22000, 33000 ] 8 0.003% 99.998% ( 33000, 50000 ] 9 0.003% 100.002% ( 50000, 75000 ] 1 0.000% 100.002% ** DB Stats ** Uptime(secs): 38.5 total, 10.1 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 153.56 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:15.899 H:M:S, 41.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 96.97 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:6.426 H:M:S, 63.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 315.895 micros/op 12650 ops/sec 60.153 seconds 760996 operations; 12.3 MB/s (189999 of 189999 found) Microseconds per read: Count: 760996 Average: 315.8968 StdDev: 992.30 Min: 2 Median: 201.9672 Max: 85005 Percentiles: P50: 201.97 P75: 247.88 P99: 2481.22 P99.9: 17627.70 P99.99: 37337.27 ------------------------------------------------------ ( 1, 2 ] 2 0.000% 0.000% ( 2, 3 ] 22 0.003% 0.003% ( 3, 4 ] 33 0.004% 0.007% ( 4, 6 ] 79 0.010% 0.018% ( 6, 10 ] 307 0.040% 0.058% ( 10, 15 ] 2930 0.385% 0.443% ( 15, 22 ] 3578 0.470% 0.913% ( 22, 34 ] 8025 1.055% 1.968% ( 34, 51 ] 2047 0.269% 2.237% ( 51, 76 ] 276 0.036% 2.273% ( 76, 110 ] 2937 0.386% 2.659% ( 110, 170 ] 227812 29.936% 32.595% ###### ( 170, 250 ] 331465 43.557% 76.152% ######### ( 250, 380 ] 87210 11.460% 87.612% ## ( 380, 580 ] 45786 6.017% 93.628% # ( 580, 870 ] 21372 2.808% 96.437% # ( 870, 1300 ] 11459 1.506% 97.943% ( 1300, 1900 ] 5720 0.752% 98.694% ( 1900, 2900 ] 4002 0.526% 99.220% ( 2900, 4400 ] 3255 0.428% 99.648% ( 4400, 6600 ] 1171 0.154% 99.802% ( 6600, 9900 ] 211 0.028% 99.830% ( 9900, 14000 ] 195 0.026% 99.855% ( 14000, 22000 ] 752 0.099% 99.954% ( 22000, 33000 ] 254 0.033% 99.987% ( 33000, 50000 ] 78 0.010% 99.998% ( 50000, 75000 ] 16 0.002% 100.000% ( 75000, 110000 ] 2 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 12635628 rocksdb.bloom.filter.full.positive COUNT : 883898 rocksdb.bloom.filter.full.true.positive COUNT : 760636 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 360 rocksdb.memtable.miss COUNT : 760636 rocksdb.l0.hit COUNT : 27235 rocksdb.l1.hit COUNT : 8192 rocksdb.l2andup.hit COUNT : 725209 rocksdb.compaction.key.drop.new COUNT : 302466 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 6107888 rocksdb.number.keys.read COUNT : 760996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 6303340416 rocksdb.bytes.read COUNT : 760996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1749 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 36883214 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 29814722799 rocksdb.compact.write.bytes COUNT : 29502972868 rocksdb.flush.write.bytes COUNT : 6275433758 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 28 rocksdb.number.superversion_acquires COUNT : 1284 rocksdb.number.superversion_releases COUNT : 1248 rocksdb.number.superversion_cleanups COUNT : 325 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 378 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 6224874496 rocksdb.memtable.garbage.bytes.at.flush COUNT : 2455552 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 33492900995 rocksdb.non.last.level.read.count COUNT : 7950658 rocksdb.block.checksum.compute.count COUNT : 8115150 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 201.094240 P95 : 720.683803 P99 : 2480.195146 P100 : 85003.000000 COUNT : 760996 SUM : 239774078 rocksdb.db.write.micros P50 : 1.536148 P95 : 4.235493 P99 : 8.494593 P100 : 847646.000000 COUNT : 6107888 SUM : 52171851 rocksdb.compaction.times.micros P50 : 1334615.384615 P95 : 3565000.000000 P99 : 6384500.000000 P100 : 20793588.000000 COUNT : 121 SUM : 214560855 rocksdb.compaction.times.cpu_micros P50 : 839285.714286 P95 : 1847500.000000 P99 : 6038000.000000 P100 : 9379337.000000 COUNT : 121 SUM : 124248185 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 1073.684211 P95 : 107666.666667 P99 : 270800.000000 P100 : 373834.000000 COUNT : 84 SUM : 1479005 rocksdb.compaction.outfile.sync.micros P50 : 27316.666667 P95 : 204260.869565 P99 : 339050.000000 P100 : 741674.000000 COUNT : 363 SUM : 19950185 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 551.428571 P95 : 4285.714286 P99 : 6277.333333 P100 : 7170.000000 COUNT : 232 SUM : 282463 rocksdb.table.open.io.micros P50 : 11243.357271 P95 : 21206.113537 P99 : 28740.250000 P100 : 33729.000000 COUNT : 1749 SUM : 18580065 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.993871 P95 : 9.805124 P99 : 192.029827 P100 : 47321.000000 COUNT : 7220577 SUM : 111069184 rocksdb.read.block.get.micros P50 : 156.077666 P95 : 578.244780 P99 : 2377.208690 P100 : 57968.000000 COUNT : 892836 SUM : 209115298 rocksdb.write.raw.block.micros P50 : 0.526862 P95 : 1.030799 P99 : 3.501455 P100 : 91901.000000 COUNT : 8665901 SUM : 28099596 rocksdb.numfiles.in.singlecompaction P50 : 2.428571 P95 : 5.666667 P99 : 12.500000 P100 : 26.000000 COUNT : 150 SUM : 491 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.578498 P95 : 1917.823016 P99 : 2706.112727 P100 : 847477.000000 COUNT : 142306 SUM : 36897476 rocksdb.sst.read.micros P50 : 2.000596 P95 : 169.257563 P99 : 528.085719 P100 : 57962.000000 COUNT : 7950710 SUM : 301979780 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 760996 SUM : 760996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 6107888 SUM : 6303340416 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 232222.222222 P95 : 1041333.333333 P99 : 1081019.000000 P100 : 1081019.000000 COUNT : 84 SUM : 29674961 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 380 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0