[2024-11-29 04:49:00.492584] Starting SPDK v25.01-pre git sha1 35cd3e84d / DPDK 23.11.0 initialization... [2024-11-29 04:49:00.492910] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1371421 ] Set seed to 1732852140485382 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-29 04:49:00.640399] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-29 04:49:00.665613] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-29 04:49:00.665671] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1371421' to capture a snapshot of events at runtime. [2024-11-29 04:49:00.665684] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-29 04:49:00.665697] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-29 04:49:00.665706] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1371421 for offline analysis/debug. [2024-11-29 04:49:00.666251] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Fri Nov 29 04:49:02 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/11/29-04:49:10 ... thread 0: (1000000,1000000) ops and (129337.7,129337.7) ops/second in (7.731696,7.731696) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/6 1.85 GB 56.6 0.4 0.0 0.4 1.2 0.9 0.0 1.4 39.7 132.3 9.52 4.31 12 0.793 383K 588 0.0 0.0 L1 7/7 850.31 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 25/8 2.25 GB 1.3 2.3 1.4 1.0 2.3 1.4 0.3 1.7 114.5 113.4 21.01 12.12 12 1.751 1503K 5657 0.0 0.0 L4 212/38 11.58 GB 1.0 3.5 2.1 1.4 3.5 2.1 0.1 1.6 117.8 116.1 30.45 17.70 26 1.171 2007K 0 0.0 0.0 L5 1120/0 90.06 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1373/59 106.57 GB 0.0 6.2 3.5 2.7 7.0 4.3 0.3 8.1 104.5 117.7 60.98 34.14 50 1.220 3894K 6245 0.0 0.0 Int 0/0 0.00 KB 0.0 6.2 3.5 2.7 7.0 4.3 0.3 8.1 104.5 117.7 60.98 34.14 50 1.220 3894K 6245 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.2 3.5 2.7 6.1 3.5 0.0 0.0 118.2 116.7 53.92 31.48 39 1.383 3894K 6245 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 125.0 7.05 2.66 11 0.641 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.9 total, 7.7 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.01 GB write, 906.14 MB/s write, 6.22 GB read, 804.51 MB/s read, 61.0 seconds Interval compaction: 7.01 GB write, 928.01 MB/s write, 6.22 GB read, 823.93 MB/s read, 61.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 22, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 22, interval: 22 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 353100 Average: 14.9256 StdDev: 232.14 Min: 1 Median: 2.0493 Max: 30067 Percentiles: P50: 2.05 P75: 3.14 P99: 13.79 P99.9: 3267.44 P99.99: 5277.63 ------------------------------------------------------ [ 0, 1 ] 8540 2.419% 2.419% ( 1, 2 ] 163709 46.363% 48.782% ######### ( 2, 3 ] 87202 24.696% 73.478% ##### ( 3, 4 ] 39710 11.246% 84.724% ## ( 4, 6 ] 39510 11.189% 95.914% ## ( 6, 10 ] 9220 2.611% 98.525% # ( 10, 15 ] 2213 0.627% 99.152% ( 15, 22 ] 296 0.084% 99.235% ( 22, 34 ] 75 0.021% 99.257% ( 34, 51 ] 85 0.024% 99.281% ( 51, 76 ] 70 0.020% 99.300% ( 76, 110 ] 27 0.008% 99.308% ( 110, 170 ] 42 0.012% 99.320% ( 170, 250 ] 155 0.044% 99.364% ( 250, 380 ] 318 0.090% 99.454% ( 380, 580 ] 474 0.134% 99.588% ( 580, 870 ] 393 0.111% 99.700% ( 870, 1300 ] 213 0.060% 99.760% ( 1300, 1900 ] 155 0.044% 99.804% ( 1900, 2900 ] 250 0.071% 99.875% ( 2900, 4400 ] 367 0.104% 99.978% ( 4400, 6600 ] 102 0.029% 100.007% ( 6600, 9900 ] 38 0.011% 100.018% ( 9900, 14000 ] 5 0.001% 100.020% ( 14000, 22000 ] 6 0.002% 100.021% ( 22000, 33000 ] 7 0.002% 100.023% ** Level 1 read latency histogram (micros): Count: 11705 Average: 40.6957 StdDev: 481.37 Min: 1 Median: 2.2871 Max: 30272 Percentiles: P50: 2.29 P75: 3.51 P99: 769.55 P99.9: 6480.69 P99.99: 21318.00 ------------------------------------------------------ [ 0, 1 ] 97 0.829% 0.829% ( 1, 2 ] 4861 41.529% 42.358% ######## ( 2, 3 ] 3116 26.621% 68.979% ##### ( 3, 4 ] 1395 11.918% 80.897% ## ( 4, 6 ] 1455 12.431% 93.328% ## ( 6, 10 ] 388 3.315% 96.642% # ( 10, 15 ] 76 0.649% 97.292% ( 15, 22 ] 14 0.120% 97.411% ( 34, 51 ] 1 0.009% 97.420% ( 51, 76 ] 4 0.034% 97.454% ( 76, 110 ] 5 0.043% 97.497% ( 110, 170 ] 8 0.068% 97.565% ( 170, 250 ] 17 0.145% 97.710% ( 250, 380 ] 39 0.333% 98.044% ( 380, 580 ] 76 0.649% 98.693% ( 580, 870 ] 55 0.470% 99.163% ( 870, 1300 ] 22 0.188% 99.351% ( 1300, 1900 ] 14 0.120% 99.470% ( 1900, 2900 ] 21 0.179% 99.650% ( 2900, 4400 ] 17 0.145% 99.795% ( 4400, 6600 ] 13 0.111% 99.906% ( 6600, 9900 ] 8 0.068% 99.974% ( 14000, 22000 ] 2 0.017% 99.991% ( 22000, 33000 ] 1 0.009% 100.000% ** Level 2 read latency histogram (micros): Count: 299409 Average: 12.8810 StdDev: 232.28 Min: 1 Median: 2.2160 Max: 36733 Percentiles: P50: 2.22 P75: 3.44 P99: 12.65 P99.9: 1501.34 P99.99: 2890.05 ------------------------------------------------------ [ 0, 1 ] 5352 1.788% 1.788% ( 1, 2 ] 128451 42.902% 44.689% ######### ( 2, 3 ] 73613 24.586% 69.275% ##### ( 3, 4 ] 39144 13.074% 82.349% ### ( 4, 6 ] 38398 12.825% 95.173% ### ( 6, 10 ] 10115 3.378% 98.552% # ( 10, 15 ] 2533 0.846% 99.398% ( 15, 22 ] 420 0.140% 99.538% ( 22, 34 ] 74 0.025% 99.563% ( 34, 51 ] 76 0.025% 99.588% ( 51, 76 ] 62 0.021% 99.609% ( 76, 110 ] 35 0.012% 99.621% ( 110, 170 ] 25 0.008% 99.629% ( 170, 250 ] 68 0.023% 99.652% ( 250, 380 ] 119 0.040% 99.691% ( 380, 580 ] 210 0.070% 99.762% ( 580, 870 ] 196 0.065% 99.827% ( 870, 1300 ] 181 0.060% 99.887% ( 1300, 1900 ] 115 0.038% 99.926% ( 1900, 2900 ] 195 0.065% 99.991% ( 2900, 4400 ] 224 0.075% 100.066% ( 4400, 6600 ] 79 0.026% 100.092% ( 6600, 9900 ] 21 0.007% 100.099% ( 9900, 14000 ] 7 0.002% 100.102% ( 14000, 22000 ] 7 0.002% 100.104% ( 22000, 33000 ] 5 0.002% 100.106% ( 33000, 50000 ] 1 0.000% 100.106% ** Level 3 read latency histogram (micros): Count: 836233 Average: 8.5784 StdDev: 176.72 Min: 1 Median: 2.1431 Max: 51208 Percentiles: P50: 2.14 P75: 3.21 P99: 9.83 P99.9: 414.90 P99.99: 768.61 ------------------------------------------------------ [ 0, 1 ] 11573 1.384% 1.384% ( 1, 2 ] 375055 44.851% 46.234% ######### ( 2, 3 ] 220121 26.323% 72.557% ##### ( 3, 4 ] 99181 11.860% 84.418% ## ( 4, 6 ] 101912 12.187% 96.605% ## ( 6, 10 ] 20945 2.505% 99.110% # ( 10, 15 ] 4674 0.559% 99.669% ( 15, 22 ] 609 0.073% 99.741% ( 22, 34 ] 142 0.017% 99.758% ( 34, 51 ] 198 0.024% 99.782% ( 51, 76 ] 198 0.024% 99.806% ( 76, 110 ] 75 0.009% 99.815% ( 110, 170 ] 59 0.007% 99.822% ( 170, 250 ] 175 0.021% 99.843% ( 250, 380 ] 385 0.046% 99.889% ( 380, 580 ] 543 0.065% 99.954% ( 580, 870 ] 468 0.056% 100.010% ( 870, 1300 ] 319 0.038% 100.048% ( 1300, 1900 ] 226 0.027% 100.075% ( 1900, 2900 ] 254 0.030% 100.105% ( 2900, 4400 ] 346 0.041% 100.146% ( 4400, 6600 ] 101 0.012% 100.159% ( 6600, 9900 ] 20 0.002% 100.161% ( 9900, 14000 ] 8 0.001% 100.162% ( 14000, 22000 ] 9 0.001% 100.163% ( 22000, 33000 ] 8 0.001% 100.164% ( 33000, 50000 ] 2 0.000% 100.164% ( 50000, 75000 ] 1 0.000% 100.164% ** Level 4 read latency histogram (micros): Count: 581210 Average: 20.4272 StdDev: 308.57 Min: 1 Median: 2.1735 Max: 72042 Percentiles: P50: 2.17 P75: 3.28 P99: 302.73 P99.9: 1854.60 P99.99: 2820.07 ------------------------------------------------------ [ 0, 1 ] 7612 1.310% 1.310% ( 1, 2 ] 256509 44.134% 45.443% ######### ( 2, 3 ] 152657 26.265% 71.709% ##### ( 3, 4 ] 67473 11.609% 83.318% ## ( 4, 6 ] 69157 11.899% 95.217% ## ( 6, 10 ] 16753 2.882% 98.099% # ( 10, 15 ] 3366 0.579% 98.678% ( 15, 22 ] 442 0.076% 98.754% ( 22, 34 ] 73 0.013% 98.767% ( 34, 51 ] 138 0.024% 98.790% ( 51, 76 ] 155 0.027% 98.817% ( 76, 110 ] 48 0.008% 98.825% ( 110, 170 ] 123 0.021% 98.847% ( 170, 250 ] 456 0.078% 98.925% ( 250, 380 ] 1082 0.186% 99.111% ( 380, 580 ] 1699 0.292% 99.403% ( 580, 870 ] 1505 0.259% 99.662% ( 870, 1300 ] 917 0.158% 99.820% ( 1300, 1900 ] 505 0.087% 99.907% ( 1900, 2900 ] 527 0.091% 99.998% ( 2900, 4400 ] 642 0.110% 100.108% ( 4400, 6600 ] 184 0.032% 100.140% ( 6600, 9900 ] 49 0.008% 100.148% ( 9900, 14000 ] 10 0.002% 100.150% ( 14000, 22000 ] 20 0.003% 100.153% ( 22000, 33000 ] 17 0.003% 100.156% ( 33000, 50000 ] 2 0.000% 100.157% ( 50000, 75000 ] 3 0.001% 100.157% ** Level 5 read latency histogram (micros): Count: 18852 Average: 1113.6342 StdDev: 1886.98 Min: 93 Median: 601.7621 Max: 67266 Percentiles: P50: 601.76 P75: 1153.06 P99: 6284.00 P99.9: 26719.00 P99.99: 60574.00 ------------------------------------------------------ ( 76, 110 ] 22 0.117% 0.117% ( 110, 170 ] 342 1.814% 1.931% ( 170, 250 ] 1262 6.694% 8.625% # ( 250, 380 ] 3080 16.338% 24.963% ### ( 380, 580 ] 4450 23.605% 48.568% ##### ( 580, 870 ] 3598 19.086% 67.653% #### ( 870, 1300 ] 2104 11.161% 78.814% ## ( 1300, 1900 ] 1149 6.095% 84.909% # ( 1900, 2900 ] 1002 5.315% 90.224% # ( 2900, 4400 ] 1240 6.578% 96.801% # ( 4400, 6600 ] 484 2.567% 99.369% # ( 6600, 9900 ] 87 0.461% 99.830% ( 9900, 14000 ] 3 0.016% 99.846% ( 14000, 22000 ] 5 0.027% 99.873% ( 22000, 33000 ] 12 0.064% 99.936% ( 33000, 50000 ] 8 0.042% 99.979% ( 50000, 75000 ] 5 0.027% 100.005% ** DB Stats ** Uptime(secs): 7.9 total, 7.7 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 124.26 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.167 H:M:S, 27.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 127.24 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.167 H:M:S, 28.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/11/29-04:49:14 ... thread 0: (1000000,2000000) ops and (211936.8,160641.5) ops/second in (4.718388,12.450084) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/6 2.83 GB 163.6 1.5 0.0 1.5 3.4 1.8 0.0 1.8 79.8 175.6 19.72 12.32 31 0.636 1599K 1965 0.0 0.0 L1 7/7 850.31 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 13/0 994.70 MB 1.0 2.3 1.4 1.0 2.3 1.4 0.3 1.7 114.5 113.4 21.01 12.12 12 1.751 1503K 5657 0.0 0.0 L4 176/0 9.44 GB 1.0 7.0 3.4 3.6 6.9 3.3 0.1 2.0 118.8 116.7 60.51 36.21 38 1.592 3132K 6537 0.0 0.0 L5 1161/0 93.41 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1369/13 107.48 GB 0.0 10.9 4.8 6.1 12.6 6.5 3.7 6.8 110.3 127.5 101.24 60.66 81 1.250 6235K 14K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.9 4.8 6.1 12.6 6.5 3.7 6.8 110.3 127.5 101.24 60.66 81 1.250 6235K 14K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.9 4.8 6.1 10.8 4.7 0.0 0.0 124.5 122.8 89.70 55.11 55 1.631 6235K 14K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 163.9 11.53 5.55 26 0.444 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.6 total, 12.5 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.61 GB write, 1021.36 MB/s write, 10.91 GB read, 883.72 MB/s read, 101.2 seconds Interval compaction: 12.61 GB write, 1036.67 MB/s write, 10.91 GB read, 896.97 MB/s read, 101.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 32, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 32, interval: 32 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 663858 Average: 12.4463 StdDev: 221.51 Min: 1 Median: 1.8104 Max: 55196 Percentiles: P50: 1.81 P75: 2.73 P99: 146.85 P99.9: 1980.74 P99.99: 4004.84 ------------------------------------------------------ [ 0, 1 ] 46672 7.030% 7.030% # ( 1, 2 ] 352016 53.026% 60.056% ########### ( 2, 3 ] 135074 20.347% 80.403% #### ( 3, 4 ] 55554 8.368% 88.771% ## ( 4, 6 ] 50572 7.618% 96.389% ## ( 6, 10 ] 12474 1.879% 98.268% ( 10, 15 ] 2937 0.442% 98.711% ( 15, 22 ] 382 0.058% 98.768% ( 22, 34 ] 86 0.013% 98.781% ( 34, 51 ] 96 0.014% 98.796% ( 51, 76 ] 80 0.012% 98.808% ( 76, 110 ] 241 0.036% 98.844% ( 110, 170 ] 1689 0.254% 99.098% ( 170, 250 ] 2178 0.328% 99.427% ( 250, 380 ] 1136 0.171% 99.598% ( 380, 580 ] 921 0.139% 99.736% ( 580, 870 ] 579 0.087% 99.824% ( 870, 1300 ] 292 0.044% 99.868% ( 1300, 1900 ] 193 0.029% 99.897% ( 1900, 2900 ] 299 0.045% 99.942% ( 2900, 4400 ] 438 0.066% 100.008% ( 4400, 6600 ] 115 0.017% 100.025% ( 6600, 9900 ] 46 0.007% 100.032% ( 9900, 14000 ] 8 0.001% 100.033% ( 14000, 22000 ] 13 0.002% 100.035% ( 22000, 33000 ] 10 0.002% 100.037% ( 33000, 50000 ] 2 0.000% 100.037% ( 50000, 75000 ] 1 0.000% 100.037% ** Level 1 read latency histogram (micros): Count: 131446 Average: 10.5333 StdDev: 211.89 Min: 1 Median: 1.9046 Max: 30272 Percentiles: P50: 1.90 P75: 2.84 P99: 13.57 P99.9: 1962.29 P99.99: 9355.60 ------------------------------------------------------ [ 0, 1 ] 3748 2.851% 2.851% # ( 1, 2 ] 68514 52.123% 54.975% ########## ( 2, 3 ] 31417 23.901% 78.876% ##### ( 3, 4 ] 11655 8.867% 87.742% ## ( 4, 6 ] 10798 8.215% 95.957% ## ( 6, 10 ] 3530 2.686% 98.643% # ( 10, 15 ] 658 0.501% 99.143% ( 15, 22 ] 105 0.080% 99.223% ( 22, 34 ] 3 0.002% 99.226% ( 34, 51 ] 18 0.014% 99.239% ( 51, 76 ] 17 0.013% 99.252% ( 76, 110 ] 24 0.018% 99.270% ( 110, 170 ] 182 0.138% 99.409% ( 170, 250 ] 214 0.163% 99.572% ( 250, 380 ] 125 0.095% 99.667% ( 380, 580 ] 123 0.094% 99.760% ( 580, 870 ] 91 0.069% 99.830% ( 870, 1300 ] 65 0.049% 99.879% ( 1300, 1900 ] 25 0.019% 99.898% ( 1900, 2900 ] 41 0.031% 99.929% ( 2900, 4400 ] 47 0.036% 99.965% ( 4400, 6600 ] 22 0.017% 99.982% ( 6600, 9900 ] 13 0.010% 99.992% ( 9900, 14000 ] 4 0.003% 99.995% ( 14000, 22000 ] 5 0.004% 99.998% ( 22000, 33000 ] 2 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 342913 Average: 12.4004 StdDev: 237.42 Min: 1 Median: 2.1990 Max: 36733 Percentiles: P50: 2.20 P75: 3.40 P99: 12.27 P99.9: 1427.56 P99.99: 2987.13 ------------------------------------------------------ [ 0, 1 ] 6000 1.750% 1.750% ( 1, 2 ] 148571 43.326% 45.076% ######### ( 2, 3 ] 84843 24.742% 69.818% ##### ( 3, 4 ] 43925 12.809% 82.627% ### ( 4, 6 ] 43507 12.687% 95.315% ### ( 6, 10 ] 11375 3.317% 98.632% # ( 10, 15 ] 2776 0.810% 99.441% ( 15, 22 ] 457 0.133% 99.575% ( 22, 34 ] 74 0.022% 99.596% ( 34, 51 ] 88 0.026% 99.622% ( 51, 76 ] 70 0.020% 99.642% ( 76, 110 ] 37 0.011% 99.653% ( 110, 170 ] 25 0.007% 99.660% ( 170, 250 ] 68 0.020% 99.680% ( 250, 380 ] 123 0.036% 99.716% ( 380, 580 ] 219 0.064% 99.780% ( 580, 870 ] 204 0.059% 99.839% ( 870, 1300 ] 183 0.053% 99.893% ( 1300, 1900 ] 118 0.034% 99.927% ( 1900, 2900 ] 202 0.059% 99.986% ( 2900, 4400 ] 236 0.069% 100.055% ( 4400, 6600 ] 82 0.024% 100.079% ( 6600, 9900 ] 25 0.007% 100.086% ( 9900, 14000 ] 8 0.002% 100.088% ( 14000, 22000 ] 10 0.003% 100.091% ( 22000, 33000 ] 8 0.002% 100.094% ( 33000, 50000 ] 1 0.000% 100.094% ** Level 3 read latency histogram (micros): Count: 966681 Average: 9.0316 StdDev: 190.96 Min: 1 Median: 2.1372 Max: 51208 Percentiles: P50: 2.14 P75: 3.20 P99: 9.94 P99.9: 462.38 P99.99: 811.77 ------------------------------------------------------ [ 0, 1 ] 13610 1.408% 1.408% ( 1, 2 ] 434860 44.985% 46.393% ######### ( 2, 3 ] 254166 26.293% 72.685% ##### ( 3, 4 ] 113261 11.716% 84.402% ## ( 4, 6 ] 116754 12.078% 96.480% ## ( 6, 10 ] 24764 2.562% 99.041% # ( 10, 15 ] 5395 0.558% 99.600% ( 15, 22 ] 739 0.076% 99.676% ( 22, 34 ] 157 0.016% 99.692% ( 34, 51 ] 237 0.025% 99.717% ( 51, 76 ] 233 0.024% 99.741% ( 76, 110 ] 108 0.011% 99.752% ( 110, 170 ] 236 0.024% 99.776% ( 170, 250 ] 401 0.041% 99.818% ( 250, 380 ] 512 0.053% 99.871% ( 380, 580 ] 683 0.071% 99.942% ( 580, 870 ] 586 0.061% 100.002% ( 870, 1300 ] 356 0.037% 100.039% ( 1300, 1900 ] 251 0.026% 100.065% ( 1900, 2900 ] 282 0.029% 100.094% ( 2900, 4400 ] 397 0.041% 100.135% ( 4400, 6600 ] 121 0.013% 100.148% ( 6600, 9900 ] 35 0.004% 100.151% ( 9900, 14000 ] 13 0.001% 100.153% ( 14000, 22000 ] 19 0.002% 100.155% ( 22000, 33000 ] 11 0.001% 100.156% ( 33000, 50000 ] 3 0.000% 100.156% ( 50000, 75000 ] 1 0.000% 100.156% ** Level 4 read latency histogram (micros): Count: 942099 Average: 19.3366 StdDev: 316.87 Min: 1 Median: 2.1387 Max: 84026 Percentiles: P50: 2.14 P75: 3.22 P99: 237.76 P99.9: 1368.05 P99.99: 2311.80 ------------------------------------------------------ [ 0, 1 ] 12086 1.283% 1.283% ( 1, 2 ] 424860 45.097% 46.380% ######### ( 2, 3 ] 245917 26.103% 72.483% ##### ( 3, 4 ] 105943 11.245% 83.729% ## ( 4, 6 ] 106339 11.287% 95.016% ## ( 6, 10 ] 26888 2.854% 97.870% # ( 10, 15 ] 5303 0.563% 98.433% ( 15, 22 ] 734 0.078% 98.511% ( 22, 34 ] 98 0.010% 98.521% ( 34, 51 ] 260 0.028% 98.549% ( 51, 76 ] 246 0.026% 98.575% ( 76, 110 ] 255 0.027% 98.602% ( 110, 170 ] 1695 0.180% 98.782% ( 170, 250 ] 2425 0.257% 99.039% ( 250, 380 ] 2104 0.223% 99.263% ( 380, 580 ] 2657 0.282% 99.545% ( 580, 870 ] 2108 0.224% 99.768% ( 870, 1300 ] 1167 0.124% 99.892% ( 1300, 1900 ] 634 0.067% 99.960% ( 1900, 2900 ] 694 0.074% 100.033% ( 2900, 4400 ] 911 0.097% 100.130% ( 4400, 6600 ] 260 0.028% 100.158% ( 6600, 9900 ] 73 0.008% 100.165% ( 9900, 14000 ] 25 0.003% 100.168% ( 14000, 22000 ] 41 0.004% 100.172% ( 22000, 33000 ] 24 0.003% 100.175% ( 33000, 50000 ] 7 0.001% 100.176% ( 50000, 75000 ] 4 0.000% 100.176% ( 75000, 110000 ] 1 0.000% 100.176% ** Level 5 read latency histogram (micros): Count: 56247 Average: 610.7824 StdDev: 1502.29 Min: 84 Median: 265.3540 Max: 67266 Percentiles: P50: 265.35 P75: 550.77 P99: 5077.47 P99.9: 9829.38 P99.99: 46919.00 ------------------------------------------------------ ( 76, 110 ] 1360 2.418% 2.418% ( 110, 170 ] 11584 20.595% 23.013% #### ( 170, 250 ] 14192 25.232% 48.244% ##### ( 250, 380 ] 8361 14.865% 63.109% ### ( 380, 580 ] 7833 13.926% 77.035% ### ( 580, 870 ] 5250 9.334% 86.369% ## ( 870, 1300 ] 2637 4.688% 91.057% # ( 1300, 1900 ] 1361 2.420% 93.477% ( 1900, 2900 ] 1305 2.320% 95.797% ( 2900, 4400 ] 1622 2.884% 98.681% # ( 4400, 6600 ] 583 1.036% 99.717% ( 6600, 9900 ] 105 0.187% 99.904% ( 9900, 14000 ] 8 0.014% 99.918% ( 14000, 22000 ] 10 0.018% 99.936% ( 22000, 33000 ] 14 0.025% 99.961% ( 33000, 50000 ] 20 0.036% 99.996% ( 50000, 75000 ] 10 0.018% 100.014% ** DB Stats ** Uptime(secs): 12.6 total, 4.7 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 155.74 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.918 H:M:S, 23.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 208.59 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.752 H:M:S, 15.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/29-04:49:18 ... thread 0: (1000000,3000000) ops and (268977.7,185553.3) ops/second in (3.717780,16.167864) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/4 2.83 GB 264.3 2.5 0.0 2.5 5.3 2.8 0.0 1.9 96.4 204.4 26.80 18.99 51 0.525 2622K 2987 0.0 0.0 L1 19/0 1.81 GB 185.5 1.8 1.0 0.8 1.8 1.0 0.0 1.8 124.5 123.9 14.98 9.26 1 14.977 1893K 9081 0.0 0.0 L3 13/0 994.70 MB 1.0 2.3 1.4 1.0 2.3 1.4 0.3 1.7 114.5 113.4 21.01 12.12 12 1.751 1503K 5657 0.0 0.0 L4 176/0 9.44 GB 1.0 7.0 3.4 3.6 6.9 3.3 0.1 2.0 118.8 116.7 60.51 36.21 38 1.592 3132K 6537 0.0 0.0 L5 1161/0 93.41 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1383/4 108.46 GB 0.0 13.7 5.8 7.9 16.4 8.5 3.7 5.8 113.9 136.1 123.29 76.58 102 1.209 9151K 24K 0.0 0.0 Int 0/0 0.00 KB 0.0 13.7 5.8 7.9 16.4 8.5 3.7 5.8 113.9 136.1 123.29 76.58 102 1.209 9151K 24K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.7 5.8 7.9 13.6 5.6 0.0 0.0 129.3 127.8 108.59 68.18 60 1.810 9151K 24K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 197.2 14.70 8.40 42 0.350 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.4 total, 16.2 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.38 GB write, 1025.80 MB/s write, 13.71 GB read, 858.44 MB/s read, 123.3 seconds Interval compaction: 16.38 GB write, 1037.64 MB/s write, 13.71 GB read, 868.35 MB/s read, 123.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 32, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 32, interval: 32 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 946782 Average: 10.6577 StdDev: 186.29 Min: 0 Median: 1.6939 Max: 55196 Percentiles: P50: 1.69 P75: 2.48 P99: 167.44 P99.9: 1040.86 P99.99: 3514.12 ------------------------------------------------------ [ 0, 1 ] 112239 11.855% 11.855% ## ( 1, 2 ] 520434 54.969% 66.824% ########### ( 2, 3 ] 162012 17.112% 83.935% ### ( 3, 4 ] 64207 6.782% 90.717% # ( 4, 6 ] 55493 5.861% 96.578% # ( 6, 10 ] 13970 1.476% 98.054% ( 10, 15 ] 3238 0.342% 98.396% ( 15, 22 ] 434 0.046% 98.442% ( 22, 34 ] 109 0.012% 98.453% ( 34, 51 ] 97 0.010% 98.463% ( 51, 76 ] 82 0.009% 98.472% ( 76, 110 ] 581 0.061% 98.533% ( 110, 170 ] 4616 0.488% 99.021% ( 170, 250 ] 4748 0.501% 99.522% ( 250, 380 ] 1724 0.182% 99.704% ( 380, 580 ] 1127 0.119% 99.824% ( 580, 870 ] 608 0.064% 99.888% ( 870, 1300 ] 295 0.031% 99.919% ( 1300, 1900 ] 196 0.021% 99.940% ( 1900, 2900 ] 299 0.032% 99.971% ( 2900, 4400 ] 438 0.046% 100.017% ( 4400, 6600 ] 115 0.012% 100.030% ( 6600, 9900 ] 46 0.005% 100.034% ( 9900, 14000 ] 8 0.001% 100.035% ( 14000, 22000 ] 13 0.001% 100.037% ( 22000, 33000 ] 10 0.001% 100.038% ( 33000, 50000 ] 2 0.000% 100.038% ( 50000, 75000 ] 1 0.000% 100.038% ** Level 1 read latency histogram (micros): Count: 217622 Average: 8.2773 StdDev: 165.45 Min: 1 Median: 1.8495 Max: 30272 Percentiles: P50: 1.85 P75: 2.75 P99: 15.50 P99.9: 971.31 P99.99: 6914.21 ------------------------------------------------------ [ 0, 1 ] 8795 4.041% 4.041% # ( 1, 2 ] 117737 54.102% 58.143% ########### ( 2, 3 ] 49056 22.542% 80.685% ##### ( 3, 4 ] 17646 8.109% 88.793% ## ( 4, 6 ] 15744 7.235% 96.028% # ( 6, 10 ] 5505 2.530% 98.558% # ( 10, 15 ] 948 0.436% 98.993% ( 15, 22 ] 209 0.096% 99.089% ( 22, 34 ] 22 0.010% 99.099% ( 34, 51 ] 19 0.009% 99.108% ( 51, 76 ] 18 0.008% 99.116% ( 76, 110 ] 95 0.044% 99.160% ( 110, 170 ] 591 0.272% 99.432% ( 170, 250 ] 552 0.254% 99.685% ( 250, 380 ] 205 0.094% 99.779% ( 380, 580 ] 145 0.067% 99.846% ( 580, 870 ] 99 0.045% 99.892% ( 870, 1300 ] 78 0.036% 99.927% ( 1300, 1900 ] 25 0.011% 99.939% ( 1900, 2900 ] 41 0.019% 99.958% ( 2900, 4400 ] 47 0.022% 99.979% ( 4400, 6600 ] 22 0.010% 99.989% ( 6600, 9900 ] 13 0.006% 99.995% ( 9900, 14000 ] 4 0.002% 99.997% ( 14000, 22000 ] 5 0.002% 100.000% ( 22000, 33000 ] 2 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 342913 Average: 12.4004 StdDev: 237.42 Min: 1 Median: 2.1990 Max: 36733 Percentiles: P50: 2.20 P75: 3.40 P99: 12.27 P99.9: 1427.56 P99.99: 2987.13 ------------------------------------------------------ [ 0, 1 ] 6000 1.750% 1.750% ( 1, 2 ] 148571 43.326% 45.076% ######### ( 2, 3 ] 84843 24.742% 69.818% ##### ( 3, 4 ] 43925 12.809% 82.627% ### ( 4, 6 ] 43507 12.687% 95.315% ### ( 6, 10 ] 11375 3.317% 98.632% # ( 10, 15 ] 2776 0.810% 99.441% ( 15, 22 ] 457 0.133% 99.575% ( 22, 34 ] 74 0.022% 99.596% ( 34, 51 ] 88 0.026% 99.622% ( 51, 76 ] 70 0.020% 99.642% ( 76, 110 ] 37 0.011% 99.653% ( 110, 170 ] 25 0.007% 99.660% ( 170, 250 ] 68 0.020% 99.680% ( 250, 380 ] 123 0.036% 99.716% ( 380, 580 ] 219 0.064% 99.780% ( 580, 870 ] 204 0.059% 99.839% ( 870, 1300 ] 183 0.053% 99.893% ( 1300, 1900 ] 118 0.034% 99.927% ( 1900, 2900 ] 202 0.059% 99.986% ( 2900, 4400 ] 236 0.069% 100.055% ( 4400, 6600 ] 82 0.024% 100.079% ( 6600, 9900 ] 25 0.007% 100.086% ( 9900, 14000 ] 8 0.002% 100.088% ( 14000, 22000 ] 10 0.003% 100.091% ( 22000, 33000 ] 8 0.002% 100.094% ( 33000, 50000 ] 1 0.000% 100.094% ** Level 3 read latency histogram (micros): Count: 967421 Average: 9.1723 StdDev: 190.97 Min: 1 Median: 2.1387 Max: 51208 Percentiles: P50: 2.14 P75: 3.20 P99: 10.31 P99.9: 467.06 P99.99: 812.72 ------------------------------------------------------ [ 0, 1 ] 13610 1.407% 1.407% ( 1, 2 ] 434860 44.950% 46.357% ######### ( 2, 3 ] 254166 26.273% 72.630% ##### ( 3, 4 ] 113261 11.708% 84.337% ## ( 4, 6 ] 116754 12.069% 96.406% ## ( 6, 10 ] 24764 2.560% 98.966% # ( 10, 15 ] 5395 0.558% 99.523% ( 15, 22 ] 739 0.076% 99.600% ( 22, 34 ] 157 0.016% 99.616% ( 34, 51 ] 237 0.024% 99.640% ( 51, 76 ] 233 0.024% 99.665% ( 76, 110 ] 152 0.016% 99.680% ( 110, 170 ] 542 0.056% 99.736% ( 170, 250 ] 691 0.071% 99.808% ( 250, 380 ] 587 0.061% 99.868% ( 380, 580 ] 702 0.073% 99.941% ( 580, 870 ] 591 0.061% 100.002% ( 870, 1300 ] 356 0.037% 100.039% ( 1300, 1900 ] 252 0.026% 100.065% ( 1900, 2900 ] 282 0.029% 100.094% ( 2900, 4400 ] 397 0.041% 100.135% ( 4400, 6600 ] 121 0.013% 100.148% ( 6600, 9900 ] 35 0.004% 100.151% ( 9900, 14000 ] 13 0.001% 100.153% ( 14000, 22000 ] 19 0.002% 100.155% ( 22000, 33000 ] 11 0.001% 100.156% ( 33000, 50000 ] 3 0.000% 100.156% ( 50000, 75000 ] 1 0.000% 100.156% ** Level 4 read latency histogram (micros): Count: 949881 Average: 20.6398 StdDev: 315.99 Min: 1 Median: 2.1542 Max: 84026 Percentiles: P50: 2.15 P75: 3.28 P99: 272.64 P99.9: 1360.68 P99.99: 2310.68 ------------------------------------------------------ [ 0, 1 ] 12087 1.272% 1.272% ( 1, 2 ] 424901 44.732% 46.004% ######### ( 2, 3 ] 246067 25.905% 71.910% ##### ( 3, 4 ] 106053 11.165% 83.074% ## ( 4, 6 ] 106474 11.209% 94.284% ## ( 6, 10 ] 26916 2.834% 97.117% # ( 10, 15 ] 5313 0.559% 97.677% ( 15, 22 ] 737 0.078% 97.754% ( 22, 34 ] 99 0.010% 97.765% ( 34, 51 ] 260 0.027% 97.792% ( 51, 76 ] 246 0.026% 97.818% ( 76, 110 ] 603 0.063% 97.881% ( 110, 170 ] 4931 0.519% 98.400% ( 170, 250 ] 5210 0.548% 98.949% ( 250, 380 ] 2786 0.293% 99.242% ( 380, 580 ] 2881 0.303% 99.546% ( 580, 870 ] 2135 0.225% 99.770% ( 870, 1300 ] 1168 0.123% 99.893% ( 1300, 1900 ] 634 0.067% 99.960% ( 1900, 2900 ] 694 0.073% 100.033% ( 2900, 4400 ] 911 0.096% 100.129% ( 4400, 6600 ] 260 0.027% 100.156% ( 6600, 9900 ] 73 0.008% 100.164% ( 9900, 14000 ] 25 0.003% 100.167% ( 14000, 22000 ] 41 0.004% 100.171% ( 22000, 33000 ] 24 0.003% 100.173% ( 33000, 50000 ] 7 0.001% 100.174% ( 50000, 75000 ] 4 0.000% 100.175% ( 75000, 110000 ] 1 0.000% 100.175% ** Level 5 read latency histogram (micros): Count: 106215 Average: 412.7466 StdDev: 1114.47 Min: 84 Median: 207.7716 Max: 67266 Percentiles: P50: 207.77 P75: 338.63 P99: 4096.53 P99.9: 8007.53 P99.99: 35871.72 ------------------------------------------------------ ( 76, 110 ] 4007 3.773% 3.773% # ( 110, 170 ] 33227 31.283% 35.055% ###### ( 170, 250 ] 33620 31.653% 66.708% ###### ( 250, 380 ] 12918 12.162% 78.870% ## ( 380, 580 ] 9320 8.775% 87.645% ## ( 580, 870 ] 5449 5.130% 92.775% # ( 870, 1300 ] 2647 2.492% 95.267% ( 1300, 1900 ] 1366 1.286% 96.553% ( 1900, 2900 ] 1305 1.229% 97.782% ( 2900, 4400 ] 1622 1.527% 99.309% ( 4400, 6600 ] 583 0.549% 99.858% ( 6600, 9900 ] 105 0.099% 99.957% ( 9900, 14000 ] 8 0.008% 99.964% ( 14000, 22000 ] 10 0.009% 99.974% ( 22000, 33000 ] 14 0.013% 99.987% ( 33000, 50000 ] 20 0.019% 100.006% ( 50000, 75000 ] 10 0.009% 100.015% ** DB Stats ** Uptime(secs): 16.4 total, 3.7 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 180.51 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.918 H:M:S, 17.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 264.72 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 2024/11/29-04:49:22 ... thread 0: (1000000,4000000) ops and (271284.3,201470.4) ops/second in (3.686170,19.854034) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/4 3.75 GB 358.7 3.5 0.0 3.5 7.3 3.7 0.0 1.9 107.4 222.3 33.43 25.29 70 0.478 3646K 3986 0.0 0.0 L1 19/0 1.81 GB 185.5 1.8 1.0 0.8 1.8 1.0 0.0 1.8 124.5 123.9 14.98 9.26 1 14.977 1893K 9081 0.0 0.0 L3 13/0 994.70 MB 1.0 2.3 1.4 1.0 2.3 1.4 0.3 1.7 114.5 113.4 21.01 12.12 12 1.751 1503K 5657 0.0 0.0 L4 176/0 9.44 GB 1.0 7.0 3.4 3.6 6.9 3.3 0.1 2.0 118.8 116.7 60.51 36.21 38 1.592 3132K 6537 0.0 0.0 L5 1161/0 93.41 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1386/4 109.38 GB 0.0 14.7 5.8 8.9 18.3 9.4 3.7 4.9 115.8 144.2 129.92 82.89 121 1.074 10M 25K 0.0 0.0 Int 0/0 0.00 KB 0.0 14.7 5.8 8.9 18.3 9.4 3.7 4.9 115.8 144.2 129.92 82.89 121 1.074 10M 25K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.7 5.8 8.9 14.5 5.6 0.0 0.0 133.8 132.4 112.46 71.99 64 1.757 10M 25K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 220.2 17.45 10.91 57 0.306 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.0 total, 19.9 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 18.29 GB write, 934.53 MB/s write, 14.70 GB read, 750.85 MB/s read, 129.9 seconds Interval compaction: 18.29 GB write, 943.31 MB/s write, 14.70 GB read, 757.91 MB/s read, 129.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 1, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 1, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 32, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 1, total-stops: 32, interval: 33 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1211579 Average: 9.3435 StdDev: 165.05 Min: 0 Median: 1.6176 Max: 55196 Percentiles: P50: 1.62 P75: 2.26 P99: 159.98 P99.9: 770.26 P99.99: 3193.98 ------------------------------------------------------ [ 0, 1 ] 192540 15.892% 15.892% ### ( 1, 2 ] 669145 55.229% 71.121% ########### ( 2, 3 ] 180401 14.890% 86.011% ### ( 3, 4 ] 71459 5.898% 91.909% # ( 4, 6 ] 60066 4.958% 96.866% # ( 6, 10 ] 15065 1.243% 98.110% ( 10, 15 ] 3538 0.292% 98.402% ( 15, 22 ] 490 0.040% 98.442% ( 22, 34 ] 139 0.011% 98.454% ( 34, 51 ] 98 0.008% 98.462% ( 51, 76 ] 83 0.007% 98.469% ( 76, 110 ] 857 0.071% 98.539% ( 110, 170 ] 6702 0.553% 99.092% ( 170, 250 ] 6235 0.515% 99.607% ( 250, 380 ] 1953 0.161% 99.768% ( 380, 580 ] 1190 0.098% 99.866% ( 580, 870 ] 621 0.051% 99.918% ( 870, 1300 ] 296 0.024% 99.942% ( 1300, 1900 ] 196 0.016% 99.958% ( 1900, 2900 ] 299 0.025% 99.983% ( 2900, 4400 ] 438 0.036% 100.019% ( 4400, 6600 ] 115 0.009% 100.029% ( 6600, 9900 ] 46 0.004% 100.032% ( 9900, 14000 ] 8 0.001% 100.033% ( 14000, 22000 ] 13 0.001% 100.034% ( 22000, 33000 ] 10 0.001% 100.035% ( 33000, 50000 ] 2 0.000% 100.035% ( 50000, 75000 ] 1 0.000% 100.035% ** Level 1 read latency histogram (micros): Count: 219391 Average: 9.1153 StdDev: 165.29 Min: 1 Median: 1.8568 Max: 30272 Percentiles: P50: 1.86 P75: 2.77 P99: 144.46 P99.9: 961.56 P99.99: 6869.31 ------------------------------------------------------ [ 0, 1 ] 8795 4.009% 4.009% # ( 1, 2 ] 117761 53.676% 57.685% ########### ( 2, 3 ] 49184 22.418% 80.104% #### ( 3, 4 ] 17815 8.120% 88.224% ## ( 4, 6 ] 16042 7.312% 95.536% # ( 6, 10 ] 5536 2.523% 98.059% # ( 10, 15 ] 962 0.438% 98.498% ( 15, 22 ] 216 0.098% 98.596% ( 22, 34 ] 24 0.011% 98.607% ( 34, 51 ] 19 0.009% 98.616% ( 51, 76 ] 18 0.008% 98.624% ( 76, 110 ] 152 0.069% 98.693% ( 110, 170 ] 1172 0.534% 99.227% ( 170, 250 ] 916 0.418% 99.645% ( 250, 380 ] 268 0.122% 99.767% ( 380, 580 ] 171 0.078% 99.845% ( 580, 870 ] 104 0.047% 99.892% ( 870, 1300 ] 78 0.036% 99.928% ( 1300, 1900 ] 25 0.011% 99.939% ( 1900, 2900 ] 41 0.019% 99.958% ( 2900, 4400 ] 47 0.021% 99.979% ( 4400, 6600 ] 22 0.010% 99.990% ( 6600, 9900 ] 13 0.006% 99.995% ( 9900, 14000 ] 4 0.002% 99.997% ( 14000, 22000 ] 5 0.002% 100.000% ( 22000, 33000 ] 2 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 342913 Average: 12.4004 StdDev: 237.42 Min: 1 Median: 2.1990 Max: 36733 Percentiles: P50: 2.20 P75: 3.40 P99: 12.27 P99.9: 1427.56 P99.99: 2987.13 ------------------------------------------------------ [ 0, 1 ] 6000 1.750% 1.750% ( 1, 2 ] 148571 43.326% 45.076% ######### ( 2, 3 ] 84843 24.742% 69.818% ##### ( 3, 4 ] 43925 12.809% 82.627% ### ( 4, 6 ] 43507 12.687% 95.315% ### ( 6, 10 ] 11375 3.317% 98.632% # ( 10, 15 ] 2776 0.810% 99.441% ( 15, 22 ] 457 0.133% 99.575% ( 22, 34 ] 74 0.022% 99.596% ( 34, 51 ] 88 0.026% 99.622% ( 51, 76 ] 70 0.020% 99.642% ( 76, 110 ] 37 0.011% 99.653% ( 110, 170 ] 25 0.007% 99.660% ( 170, 250 ] 68 0.020% 99.680% ( 250, 380 ] 123 0.036% 99.716% ( 380, 580 ] 219 0.064% 99.780% ( 580, 870 ] 204 0.059% 99.839% ( 870, 1300 ] 183 0.053% 99.893% ( 1300, 1900 ] 118 0.034% 99.927% ( 1900, 2900 ] 202 0.059% 99.986% ( 2900, 4400 ] 236 0.069% 100.055% ( 4400, 6600 ] 82 0.024% 100.079% ( 6600, 9900 ] 25 0.007% 100.086% ( 9900, 14000 ] 8 0.002% 100.088% ( 14000, 22000 ] 10 0.003% 100.091% ( 22000, 33000 ] 8 0.002% 100.094% ( 33000, 50000 ] 1 0.000% 100.094% ** Level 3 read latency histogram (micros): Count: 968208 Average: 9.3092 StdDev: 190.96 Min: 1 Median: 2.1402 Max: 51208 Percentiles: P50: 2.14 P75: 3.21 P99: 11.03 P99.9: 470.50 P99.99: 812.97 ------------------------------------------------------ [ 0, 1 ] 13610 1.406% 1.406% ( 1, 2 ] 434860 44.914% 46.320% ######### ( 2, 3 ] 254166 26.251% 72.571% ##### ( 3, 4 ] 113261 11.698% 84.269% ## ( 4, 6 ] 116754 12.059% 96.328% ## ( 6, 10 ] 24764 2.558% 98.885% # ( 10, 15 ] 5395 0.557% 99.442% ( 15, 22 ] 739 0.076% 99.519% ( 22, 34 ] 157 0.016% 99.535% ( 34, 51 ] 237 0.024% 99.559% ( 51, 76 ] 233 0.024% 99.584% ( 76, 110 ] 203 0.021% 99.605% ( 110, 170 ] 953 0.098% 99.703% ( 170, 250 ] 953 0.098% 99.801% ( 250, 380 ] 629 0.065% 99.866% ( 380, 580 ] 720 0.074% 99.941% ( 580, 870 ] 594 0.061% 100.002% ( 870, 1300 ] 356 0.037% 100.039% ( 1300, 1900 ] 252 0.026% 100.065% ( 1900, 2900 ] 282 0.029% 100.094% ( 2900, 4400 ] 397 0.041% 100.135% ( 4400, 6600 ] 121 0.012% 100.147% ( 6600, 9900 ] 35 0.004% 100.151% ( 9900, 14000 ] 13 0.001% 100.152% ( 14000, 22000 ] 19 0.002% 100.154% ( 22000, 33000 ] 11 0.001% 100.156% ( 33000, 50000 ] 3 0.000% 100.156% ( 50000, 75000 ] 1 0.000% 100.156% ** Level 4 read latency histogram (micros): Count: 957884 Average: 21.9172 StdDev: 315.03 Min: 1 Median: 2.1704 Max: 84026 Percentiles: P50: 2.17 P75: 3.33 P99: 290.33 P99.9: 1353.11 P99.99: 2309.53 ------------------------------------------------------ [ 0, 1 ] 12087 1.262% 1.262% ( 1, 2 ] 424909 44.359% 45.621% ######### ( 2, 3 ] 246091 25.691% 71.312% ##### ( 3, 4 ] 106088 11.075% 82.387% ## ( 4, 6 ] 106523 11.121% 93.508% ## ( 6, 10 ] 26918 2.810% 96.318% # ( 10, 15 ] 5314 0.555% 96.873% ( 15, 22 ] 738 0.077% 96.950% ( 22, 34 ] 99 0.010% 96.960% ( 34, 51 ] 260 0.027% 96.987% ( 51, 76 ] 246 0.026% 97.013% ( 76, 110 ] 1138 0.119% 97.132% ( 110, 170 ] 8943 0.934% 98.066% ( 170, 250 ] 7959 0.831% 98.896% ( 250, 380 ] 3198 0.334% 99.230% ( 380, 580 ] 3039 0.317% 99.548% ( 580, 870 ] 2152 0.225% 99.772% ( 870, 1300 ] 1168 0.122% 99.894% ( 1300, 1900 ] 634 0.066% 99.960% ( 1900, 2900 ] 694 0.072% 100.033% ( 2900, 4400 ] 911 0.095% 100.128% ( 4400, 6600 ] 260 0.027% 100.155% ( 6600, 9900 ] 73 0.008% 100.163% ( 9900, 14000 ] 25 0.003% 100.165% ( 14000, 22000 ] 41 0.004% 100.170% ( 22000, 33000 ] 24 0.003% 100.172% ( 33000, 50000 ] 7 0.001% 100.173% ( 50000, 75000 ] 4 0.000% 100.173% ( 75000, 110000 ] 1 0.000% 100.173% ** Level 5 read latency histogram (micros): Count: 157062 Average: 335.9509 StdDev: 923.87 Min: 84 Median: 188.5908 Max: 67266 Percentiles: P50: 188.59 P75: 249.73 P99: 3623.53 P99.9: 6565.80 P99.99: 29302.27 ------------------------------------------------------ ( 76, 110 ] 7402 4.713% 4.713% # ( 110, 170 ] 59190 37.686% 42.399% ######## ( 170, 250 ] 51376 32.711% 75.109% ####### ( 250, 380 ] 15627 9.950% 85.059% ## ( 380, 580 ] 10230 6.513% 91.572% # ( 580, 870 ] 5563 3.542% 95.114% # ( 870, 1300 ] 2650 1.687% 96.801% ( 1300, 1900 ] 1366 0.870% 97.671% ( 1900, 2900 ] 1305 0.831% 98.502% ( 2900, 4400 ] 1622 1.033% 99.535% ( 4400, 6600 ] 583 0.371% 99.906% ( 6600, 9900 ] 105 0.067% 99.973% ( 9900, 14000 ] 8 0.005% 99.978% ( 14000, 22000 ] 10 0.006% 99.984% ( 22000, 33000 ] 14 0.009% 99.993% ( 33000, 50000 ] 20 0.013% 100.006% ( 50000, 75000 ] 10 0.006% 100.012% ** DB Stats ** Uptime(secs): 20.0 total, 3.7 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 196.42 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.952 H:M:S, 14.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 267.00 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.034 H:M:S, 0.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 239.908 micros/op 16665 ops/sec 60.063 seconds 1000996 operations; 16.1 MB/s (251999 of 251999 found) Microseconds per read: Count: 1000996 Average: 239.9116 StdDev: 726.10 Min: 2 Median: 198.1721 Max: 106864 Percentiles: P50: 198.17 P75: 236.40 P99: 812.52 P99.9: 7928.72 P99.99: 31033.59 ------------------------------------------------------ ( 1, 2 ] 4 0.000% 0.000% ( 2, 3 ] 48 0.005% 0.005% ( 3, 4 ] 57 0.006% 0.011% ( 4, 6 ] 80 0.008% 0.019% ( 6, 10 ] 364 0.036% 0.055% ( 10, 15 ] 4672 0.467% 0.522% ( 15, 22 ] 4581 0.458% 0.980% ( 22, 34 ] 8892 0.888% 1.868% ( 34, 51 ] 3690 0.369% 2.237% ( 51, 76 ] 504 0.050% 2.287% ( 76, 110 ] 8953 0.894% 3.181% ( 110, 170 ] 284222 28.394% 31.575% ###### ( 170, 250 ] 523727 52.321% 83.896% ########## ( 250, 380 ] 102521 10.242% 94.138% ## ( 380, 580 ] 41143 4.110% 98.248% # ( 580, 870 ] 9389 0.938% 99.186% ( 870, 1300 ] 2715 0.271% 99.457% ( 1300, 1900 ] 1111 0.111% 99.568% ( 1900, 2900 ] 1050 0.105% 99.673% ( 2900, 4400 ] 1357 0.136% 99.809% ( 4400, 6600 ] 762 0.076% 99.885% ( 6600, 9900 ] 380 0.038% 99.923% ( 9900, 14000 ] 267 0.027% 99.949% ( 14000, 22000 ] 287 0.029% 99.978% ( 22000, 33000 ] 146 0.015% 99.993% ( 33000, 50000 ] 38 0.004% 99.996% ( 50000, 75000 ] 24 0.002% 99.999% ( 75000, 110000 ] 12 0.001% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 19298734 rocksdb.bloom.filter.full.positive COUNT : 1188085 rocksdb.bloom.filter.full.true.positive COUNT : 1000583 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 413 rocksdb.memtable.miss COUNT : 1000583 rocksdb.l0.hit COUNT : 38853 rocksdb.l1.hit COUNT : 17141 rocksdb.l2andup.hit COUNT : 944589 rocksdb.compaction.key.drop.new COUNT : 205493 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 4302164 rocksdb.number.keys.read COUNT : 1000996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 4439833248 rocksdb.bytes.read COUNT : 1000996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1623 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 40916906 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 17909003682 rocksdb.compact.write.bytes COUNT : 17702256688 rocksdb.flush.write.bytes COUNT : 4425965233 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 27 rocksdb.number.superversion_acquires COUNT : 829 rocksdb.number.superversion_releases COUNT : 800 rocksdb.number.superversion_cleanups COUNT : 208 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 234 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 4390111232 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1559552 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 22756991993 rocksdb.non.last.level.read.count COUNT : 5355882 rocksdb.block.checksum.compute.count COUNT : 5540313 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 197.639653 P95 : 419.604311 P99 : 811.758567 P100 : 106862.000000 COUNT : 1000996 SUM : 239376223 rocksdb.db.write.micros P50 : 1.637956 P95 : 5.515086 P99 : 14.061884 P100 : 695296.000000 COUNT : 4302164 SUM : 54124649 rocksdb.compaction.times.micros P50 : 1280769.230769 P95 : 3775000.000000 P99 : 14976504.000000 P100 : 14976504.000000 COUNT : 65 SUM : 113418501 rocksdb.compaction.times.cpu_micros P50 : 955625.000000 P95 : 1821250.000000 P99 : 7655000.000000 P100 : 9258887.000000 COUNT : 65 SUM : 72927896 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 725.000000 P95 : 6435.000000 P99 : 9207.000000 P100 : 9528.000000 COUNT : 63 SUM : 88063 rocksdb.compaction.outfile.sync.micros P50 : 2787.096774 P95 : 8819.250000 P99 : 11837.250000 P100 : 13010.000000 COUNT : 211 SUM : 686194 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 165.263158 P95 : 3853.571429 P99 : 5038.000000 P100 : 5077.000000 COUNT : 142 SUM : 105916 rocksdb.table.open.io.micros P50 : 5534.700315 P95 : 19212.195122 P99 : 29122.500000 P100 : 98770.000000 COUNT : 1623 SUM : 11780608 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.361591 P95 : 14.860467 P99 : 207.427421 P100 : 84033.000000 COUNT : 4338522 SUM : 76665739 rocksdb.read.block.get.micros P50 : 147.747851 P95 : 249.012553 P99 : 719.800670 P100 : 75470.000000 COUNT : 1200172 SUM : 203559386 rocksdb.write.raw.block.micros P50 : 0.550348 P95 : 1.619496 P99 : 4.385143 P100 : 152066.000000 COUNT : 5359672 SUM : 19531276 rocksdb.numfiles.in.singlecompaction P50 : 3.037037 P95 : 5.818182 P99 : 36.000000 P100 : 36.000000 COUNT : 80 SUM : 314 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.600679 P95 : 2132.135011 P99 : 2747.066107 P100 : 695096.000000 COUNT : 153948 SUM : 40935494 rocksdb.sst.read.micros P50 : 2.510129 P95 : 193.650626 P99 : 330.597746 P100 : 84026.000000 COUNT : 5355931 SUM : 261637588 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1000996 SUM : 1000996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 4302164 SUM : 4439833248 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 221304.347826 P95 : 932250.000000 P99 : 1026406.000000 P100 : 1026406.000000 COUNT : 63 SUM : 18709474 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 239 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0