[2024-11-20 18:33:08.969547] Starting SPDK v24.09.1-pre git sha1 b18e1bd62 / DPDK 23.11.0 initialization... [2024-11-20 18:33:08.969810] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3290173 ] Set seed to 1732123988962861 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-20 18:33:09.083244] app.c: 917:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-20 18:33:09.122630] app.c: 610:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-20 18:33:09.122676] app.c: 611:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3290173' to capture a snapshot of events at runtime. [2024-11-20 18:33:09.122686] app.c: 616:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-20 18:33:09.122697] app.c: 617:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-20 18:33:09.122705] app.c: 618:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3290173 for offline analysis/debug. [2024-11-20 18:33:09.122796] reactor.c: 990:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Nov 20 18:33:13 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/20-18:33:21 ... thread 0: (1000000,1000000) ops and (122115.6,122115.6) ops/second in (8.188959,8.188959) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/7 2.07 GB 56.6 0.4 0.0 0.4 1.2 0.9 0.0 1.4 43.9 146.3 8.61 4.05 12 0.717 383K 541 0.0 0.0 L1 9/9 1.10 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 20/1 1.06 GB 1.0 1.6 1.0 0.7 1.6 0.9 0.1 1.7 108.4 107.3 15.27 7.80 8 1.908 690K 0 0.0 0.0 L4 163/3 9.67 GB 1.0 5.1 3.4 1.7 5.1 3.4 0.0 1.5 108.9 108.0 48.28 24.28 33 1.463 1780K 0 0.0 0.0 L5 1108/0 92.03 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1311/20 105.93 GB 0.0 7.1 4.4 2.8 7.9 5.2 3.6 9.2 101.0 112.4 72.15 36.13 53 1.361 2854K 541 0.0 0.0 Int 0/0 0.00 KB 0.0 7.1 4.4 2.8 7.9 5.2 3.6 9.2 101.0 112.4 72.15 36.13 53 1.361 2854K 541 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 7.1 4.4 2.8 7.1 4.3 0.0 0.0 110.4 109.5 66.02 33.63 42 1.572 2854K 541 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 143.8 6.13 2.51 11 0.557 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.4 total, 8.2 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.92 GB write, 970.09 MB/s write, 7.12 GB read, 871.62 MB/s read, 72.2 seconds Interval compaction: 7.92 GB write, 990.58 MB/s write, 7.12 GB read, 890.03 MB/s read, 72.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 25, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 7, total-stops: 25, interval: 31 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 402718 Average: 10.4951 StdDev: 138.48 Min: 1 Median: 1.9733 Max: 24830 Percentiles: P50: 1.97 P75: 2.93 P99: 11.60 P99.9: 1883.22 P99.99: 3729.82 ------------------------------------------------------ [ 0, 1 ] 11565 2.872% 2.872% # ( 1, 2 ] 194994 48.419% 51.291% ########## ( 2, 3 ] 102357 25.417% 76.708% ##### ( 3, 4 ] 35756 8.879% 85.586% ## ( 4, 6 ] 42482 10.549% 96.135% ## ( 6, 10 ] 10930 2.714% 98.849% # ( 10, 15 ] 1897 0.471% 99.320% ( 15, 22 ] 296 0.074% 99.394% ( 22, 34 ] 32 0.008% 99.402% ( 34, 51 ] 21 0.005% 99.407% ( 51, 76 ] 24 0.006% 99.413% ( 76, 110 ] 44 0.011% 99.424% ( 110, 170 ] 163 0.040% 99.464% ( 170, 250 ] 178 0.044% 99.509% ( 250, 380 ] 213 0.053% 99.561% ( 380, 580 ] 310 0.077% 99.638% ( 580, 870 ] 393 0.098% 99.736% ( 870, 1300 ] 394 0.098% 99.834% ( 1300, 1900 ] 276 0.069% 99.902% ( 1900, 2900 ] 238 0.059% 99.962% ( 2900, 4400 ] 211 0.052% 100.014% ( 4400, 6600 ] 46 0.011% 100.025% ( 6600, 9900 ] 6 0.001% 100.027% ( 22000, 33000 ] 1 0.000% 100.027% ** Level 1 read latency histogram (micros): Count: 241 Average: 1209.7220 StdDev: 1931.93 Min: 79 Median: 598.5897 Max: 13210 Percentiles: P50: 598.59 P75: 1179.47 P99: 11529.75 P99.9: 13210.00 P99.99: 13210.00 ------------------------------------------------------ ( 76, 110 ] 6 2.490% 2.490% ( 110, 170 ] 25 10.373% 12.863% ## ( 170, 250 ] 27 11.203% 24.066% ## ( 250, 380 ] 23 9.544% 33.610% ## ( 380, 580 ] 37 15.353% 48.963% ### ( 580, 870 ] 39 16.183% 65.145% ### ( 870, 1300 ] 33 13.693% 78.838% ### ( 1300, 1900 ] 15 6.224% 85.062% # ( 1900, 2900 ] 12 4.979% 90.041% # ( 2900, 4400 ] 13 5.394% 95.436% # ( 4400, 6600 ] 2 0.830% 96.266% ( 6600, 9900 ] 5 2.075% 98.340% ( 9900, 14000 ] 4 1.660% 100.000% ** Level 2 read latency histogram (micros): Count: 157560 Average: 10.9632 StdDev: 144.01 Min: 1 Median: 2.0622 Max: 11868 Percentiles: P50: 2.06 P75: 3.24 P99: 12.16 P99.9: 1914.85 P99.99: 3600.67 ------------------------------------------------------ [ 0, 1 ] 3812 2.419% 2.419% ( 1, 2 ] 72630 46.097% 48.516% ######### ( 2, 3 ] 37588 23.856% 72.372% ##### ( 3, 4 ] 17136 10.876% 83.248% ## ( 4, 6 ] 19076 12.107% 95.355% ## ( 6, 10 ] 5333 3.385% 98.740% # ( 10, 15 ] 949 0.602% 99.342% ( 15, 22 ] 138 0.088% 99.430% ( 22, 34 ] 25 0.016% 99.446% ( 34, 51 ] 11 0.007% 99.453% ( 51, 76 ] 14 0.009% 99.462% ( 76, 110 ] 13 0.008% 99.470% ( 110, 170 ] 34 0.022% 99.492% ( 170, 250 ] 51 0.032% 99.524% ( 250, 380 ] 79 0.050% 99.574% ( 380, 580 ] 115 0.073% 99.647% ( 580, 870 ] 143 0.091% 99.738% ( 870, 1300 ] 151 0.096% 99.834% ( 1300, 1900 ] 103 0.065% 99.899% ( 1900, 2900 ] 97 0.062% 99.961% ( 2900, 4400 ] 99 0.063% 100.023% ( 4400, 6600 ] 16 0.010% 100.034% ( 6600, 9900 ] 7 0.004% 100.038% ( 9900, 14000 ] 1 0.001% 100.039% ** Level 3 read latency histogram (micros): Count: 849224 Average: 11.7920 StdDev: 151.03 Min: 1 Median: 1.9896 Max: 15296 Percentiles: P50: 1.99 P75: 2.99 P99: 9.94 P99.9: 1176.91 P99.99: 1697.91 ------------------------------------------------------ [ 0, 1 ] 14002 1.649% 1.649% ( 1, 2 ] 414945 48.862% 50.510% ########## ( 2, 3 ] 210249 24.758% 75.268% ##### ( 3, 4 ] 87979 10.360% 85.628% ## ( 4, 6 ] 92329 10.872% 96.500% ## ( 6, 10 ] 21539 2.536% 99.037% # ( 10, 15 ] 4133 0.487% 99.523% ( 15, 22 ] 556 0.065% 99.589% ( 22, 34 ] 63 0.007% 99.596% ( 34, 51 ] 31 0.004% 99.600% ( 51, 76 ] 47 0.006% 99.605% ( 76, 110 ] 25 0.003% 99.608% ( 110, 170 ] 82 0.010% 99.618% ( 170, 250 ] 150 0.018% 99.636% ( 250, 380 ] 272 0.032% 99.668% ( 380, 580 ] 485 0.057% 99.725% ( 580, 870 ] 809 0.095% 99.820% ( 870, 1300 ] 951 0.112% 99.932% ( 1300, 1900 ] 742 0.087% 100.019% ( 1900, 2900 ] 676 0.080% 100.099% ( 2900, 4400 ] 546 0.064% 100.163% ( 4400, 6600 ] 124 0.015% 100.178% ( 6600, 9900 ] 26 0.003% 100.181% ( 9900, 14000 ] 2 0.000% 100.181% ( 14000, 22000 ] 2 0.000% 100.181% ** Level 4 read latency histogram (micros): Count: 456075 Average: 22.6431 StdDev: 223.48 Min: 1 Median: 2.0125 Max: 25676 Percentiles: P50: 2.01 P75: 3.09 P99: 483.01 P99.9: 2645.40 P99.99: 3493.07 ------------------------------------------------------ [ 0, 1 ] 7413 1.625% 1.625% ( 1, 2 ] 219234 48.070% 49.695% ########## ( 2, 3 ] 111073 24.354% 74.049% ##### ( 3, 4 ] 47873 10.497% 84.546% ## ( 4, 6 ] 49911 10.944% 95.490% ## ( 6, 10 ] 11690 2.563% 98.053% # ( 10, 15 ] 2192 0.481% 98.533% ( 15, 22 ] 279 0.061% 98.595% ( 22, 34 ] 30 0.007% 98.601% ( 34, 51 ] 17 0.004% 98.605% ( 51, 76 ] 38 0.008% 98.613% ( 76, 110 ] 94 0.021% 98.634% ( 110, 170 ] 374 0.082% 98.716% ( 170, 250 ] 414 0.091% 98.807% ( 250, 380 ] 480 0.105% 98.912% ( 380, 580 ] 781 0.171% 99.083% ( 580, 870 ] 1113 0.244% 99.327% ( 870, 1300 ] 1221 0.268% 99.595% ( 1300, 1900 ] 894 0.196% 99.791% ( 1900, 2900 ] 668 0.146% 99.937% ( 2900, 4400 ] 608 0.133% 100.071% ( 4400, 6600 ] 165 0.036% 100.107% ( 6600, 9900 ] 24 0.005% 100.112% ( 9900, 14000 ] 4 0.001% 100.113% ( 14000, 22000 ] 1 0.000% 100.113% ( 22000, 33000 ] 1 0.000% 100.113% ** Level 5 read latency histogram (micros): Count: 20127 Average: 1184.7620 StdDev: 1331.61 Min: 77 Median: 699.7345 Max: 22776 Percentiles: P50: 699.73 P75: 1527.42 P99: 6228.90 P99.9: 9376.19 P99.99: 13986.98 ------------------------------------------------------ ( 76, 110 ] 402 1.997% 1.997% ( 110, 170 ] 1927 9.574% 11.572% ## ( 170, 250 ] 2013 10.001% 21.573% ## ( 250, 380 ] 1954 9.708% 31.281% ## ( 380, 580 ] 2578 12.809% 44.090% ### ( 580, 870 ] 2881 14.314% 58.404% ### ( 870, 1300 ] 2672 13.276% 71.680% ### ( 1300, 1900 ] 1763 8.759% 80.439% ## ( 1900, 2900 ] 1575 7.825% 88.265% ## ( 2900, 4400 ] 1637 8.133% 96.398% ## ( 4400, 6600 ] 630 3.130% 99.528% # ( 6600, 9900 ] 89 0.442% 99.970% ( 9900, 14000 ] 4 0.020% 99.990% ( 14000, 22000 ] 2 0.010% 100.000% ( 22000, 33000 ] 1 0.005% 100.005% ** DB Stats ** Uptime(secs): 8.4 total, 8.2 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 117.67 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.409 H:M:S, 52.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 120.14 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.409 H:M:S, 53.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/20-18:33:24 ... thread 0: (1000000,2000000) ops and (290867.9,172014.2) ops/second in (3.437987,11.626946) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/7 3.05 GB 163.6 1.4 0.0 1.4 3.3 1.8 0.0 1.8 91.4 210.4 15.86 10.81 32 0.496 1471K 1718 0.0 0.0 L1 9/9 1.10 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 19/0 988.78 MB 1.0 1.6 1.0 0.7 1.6 0.9 0.1 1.7 108.4 107.3 15.27 7.80 8 1.908 690K 0 0.0 0.0 L4 160/0 9.40 GB 1.0 5.3 3.5 1.8 5.3 3.5 0.0 1.5 110.7 109.8 49.29 25.19 34 1.450 1923K 0 0.0 0.0 L5 1112/0 92.40 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.8 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1315/16 106.91 GB 0.0 8.4 4.5 3.9 10.1 6.2 3.9 5.5 106.5 129.2 80.41 43.79 74 1.087 4084K 1718 0.0 0.0 Int 0/0 0.00 KB 0.0 8.4 4.5 3.9 10.1 6.2 3.9 5.5 106.5 129.2 80.41 43.79 74 1.087 4084K 1718 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 8.4 4.5 3.9 8.3 4.4 0.0 0.0 120.2 119.3 71.22 38.64 47 1.515 4084K 1718 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 205.6 9.19 5.15 27 0.340 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 11.8 total, 11.6 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.14 GB write, 880.20 MB/s write, 8.36 GB read, 725.41 MB/s read, 80.4 seconds Interval compaction: 10.14 GB write, 893.30 MB/s write, 8.36 GB read, 736.20 MB/s read, 80.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 25, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 7, total-stops: 25, interval: 31 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 709632 Average: 9.7031 StdDev: 109.23 Min: 1 Median: 1.7595 Max: 24830 Percentiles: P50: 1.76 P75: 2.61 P99: 157.06 P99.9: 1150.73 P99.99: 2661.28 ------------------------------------------------------ [ 0, 1 ] 64082 9.030% 9.030% ## ( 1, 2 ] 382817 53.946% 62.976% ########### ( 2, 3 ] 140248 19.763% 82.740% #### ( 3, 4 ] 46752 6.588% 89.328% # ( 4, 6 ] 49917 7.034% 96.362% # ( 6, 10 ] 12381 1.745% 98.107% ( 10, 15 ] 2206 0.311% 98.418% ( 15, 22 ] 342 0.048% 98.466% ( 22, 34 ] 34 0.005% 98.471% ( 34, 51 ] 21 0.003% 98.474% ( 51, 76 ] 27 0.004% 98.477% ( 76, 110 ] 928 0.131% 98.608% ( 110, 170 ] 3548 0.500% 99.108% ( 170, 250 ] 2183 0.308% 99.416% ( 250, 380 ] 1259 0.177% 99.593% ( 380, 580 ] 1163 0.164% 99.757% ( 580, 870 ] 703 0.099% 99.856% ( 870, 1300 ] 480 0.068% 99.924% ( 1300, 1900 ] 284 0.040% 99.964% ( 1900, 2900 ] 247 0.035% 99.999% ( 2900, 4400 ] 211 0.030% 100.028% ( 4400, 6600 ] 46 0.006% 100.035% ( 6600, 9900 ] 6 0.001% 100.036% ( 22000, 33000 ] 1 0.000% 100.036% ** Level 1 read latency histogram (micros): Count: 97427 Average: 7.6716 StdDev: 115.88 Min: 1 Median: 1.9007 Max: 13210 Percentiles: P50: 1.90 P75: 2.78 P99: 103.12 P99.9: 847.52 P99.99: 5783.03 ------------------------------------------------------ [ 0, 1 ] 3302 3.389% 3.389% # ( 1, 2 ] 50416 51.747% 55.137% ########## ( 2, 3 ] 24864 25.521% 80.657% ##### ( 3, 4 ] 7300 7.493% 88.150% # ( 4, 6 ] 7508 7.706% 95.856% ## ( 6, 10 ] 2531 2.598% 98.454% # ( 10, 15 ] 338 0.347% 98.801% ( 15, 22 ] 103 0.106% 98.907% ( 22, 34 ] 4 0.004% 98.911% ( 76, 110 ] 110 0.113% 99.024% ( 110, 170 ] 363 0.373% 99.396% ( 170, 250 ] 210 0.216% 99.612% ( 250, 380 ] 112 0.115% 99.727% ( 380, 580 ] 105 0.108% 99.835% ( 580, 870 ] 70 0.072% 99.907% ( 870, 1300 ] 41 0.042% 99.949% ( 1300, 1900 ] 15 0.015% 99.964% ( 1900, 2900 ] 12 0.012% 99.976% ( 2900, 4400 ] 13 0.013% 99.990% ( 4400, 6600 ] 2 0.002% 99.992% ( 6600, 9900 ] 5 0.005% 99.997% ( 9900, 14000 ] 4 0.004% 100.001% ** Level 2 read latency histogram (micros): Count: 157626 Average: 11.0530 StdDev: 144.11 Min: 1 Median: 2.0631 Max: 11868 Percentiles: P50: 2.06 P75: 3.24 P99: 12.50 P99.9: 1914.16 P99.99: 3600.57 ------------------------------------------------------ [ 0, 1 ] 3812 2.418% 2.418% ( 1, 2 ] 72630 46.077% 48.496% ######### ( 2, 3 ] 37588 23.846% 72.342% ##### ( 3, 4 ] 17136 10.871% 83.213% ## ( 4, 6 ] 19076 12.102% 95.315% ## ( 6, 10 ] 5333 3.383% 98.699% # ( 10, 15 ] 949 0.602% 99.301% ( 15, 22 ] 138 0.088% 99.388% ( 22, 34 ] 25 0.016% 99.404% ( 34, 51 ] 11 0.007% 99.411% ( 51, 76 ] 14 0.009% 99.420% ( 76, 110 ] 26 0.016% 99.437% ( 110, 170 ] 63 0.040% 99.477% ( 170, 250 ] 59 0.037% 99.514% ( 250, 380 ] 86 0.055% 99.569% ( 380, 580 ] 120 0.076% 99.645% ( 580, 870 ] 145 0.092% 99.737% ( 870, 1300 ] 153 0.097% 99.834% ( 1300, 1900 ] 103 0.065% 99.899% ( 1900, 2900 ] 97 0.062% 99.961% ( 2900, 4400 ] 99 0.063% 100.023% ( 4400, 6600 ] 16 0.010% 100.034% ( 6600, 9900 ] 7 0.004% 100.038% ( 9900, 14000 ] 1 0.001% 100.039% ** Level 3 read latency histogram (micros): Count: 874137 Average: 11.6857 StdDev: 149.04 Min: 1 Median: 1.9819 Max: 15296 Percentiles: P50: 1.98 P75: 2.98 P99: 10.11 P99.9: 1166.32 P99.99: 1695.08 ------------------------------------------------------ [ 0, 1 ] 14979 1.714% 1.714% ( 1, 2 ] 429849 49.174% 50.888% ########## ( 2, 3 ] 215298 24.630% 75.517% ##### ( 3, 4 ] 89522 10.241% 85.759% ## ( 4, 6 ] 93631 10.711% 96.470% ## ( 6, 10 ] 22026 2.520% 98.990% # ( 10, 15 ] 4163 0.476% 99.466% ( 15, 22 ] 567 0.065% 99.531% ( 22, 34 ] 63 0.007% 99.538% ( 34, 51 ] 31 0.004% 99.541% ( 51, 76 ] 48 0.005% 99.547% ( 76, 110 ] 105 0.012% 99.559% ( 110, 170 ] 324 0.037% 99.596% ( 170, 250 ] 282 0.032% 99.628% ( 250, 380 ] 348 0.040% 99.668% ( 380, 580 ] 540 0.062% 99.730% ( 580, 870 ] 826 0.094% 99.824% ( 870, 1300 ] 959 0.110% 99.934% ( 1300, 1900 ] 742 0.085% 100.019% ( 1900, 2900 ] 676 0.077% 100.096% ( 2900, 4400 ] 546 0.062% 100.159% ( 4400, 6600 ] 124 0.014% 100.173% ( 6600, 9900 ] 26 0.003% 100.176% ( 9900, 14000 ] 2 0.000% 100.176% ( 14000, 22000 ] 2 0.000% 100.176% ** Level 4 read latency histogram (micros): Count: 481213 Average: 24.4414 StdDev: 219.80 Min: 1 Median: 2.0141 Max: 25676 Percentiles: P50: 2.01 P75: 3.14 P99: 533.43 P99.9: 2608.45 P99.99: 3484.40 ------------------------------------------------------ [ 0, 1 ] 8234 1.711% 1.711% ( 1, 2 ] 230743 47.950% 49.661% ########## ( 2, 3 ] 115198 23.939% 73.600% ##### ( 3, 4 ] 49248 10.234% 83.835% ## ( 4, 6 ] 50903 10.578% 94.413% ## ( 6, 10 ] 12017 2.497% 96.910% ( 10, 15 ] 2233 0.464% 97.374% ( 15, 22 ] 283 0.059% 97.433% ( 22, 34 ] 30 0.006% 97.439% ( 34, 51 ] 17 0.004% 97.443% ( 51, 76 ] 38 0.008% 97.450% ( 76, 110 ] 870 0.181% 97.631% ( 110, 170 ] 2752 0.572% 98.203% ( 170, 250 ] 1647 0.342% 98.545% ( 250, 380 ] 1153 0.240% 98.785% ( 380, 580 ] 1349 0.280% 99.065% ( 580, 870 ] 1364 0.283% 99.349% ( 870, 1300 ] 1273 0.265% 99.613% ( 1300, 1900 ] 903 0.188% 99.801% ( 1900, 2900 ] 673 0.140% 99.941% ( 2900, 4400 ] 608 0.126% 100.067% ( 4400, 6600 ] 165 0.034% 100.101% ( 6600, 9900 ] 24 0.005% 100.106% ( 9900, 14000 ] 4 0.001% 100.107% ( 14000, 22000 ] 1 0.000% 100.107% ( 22000, 33000 ] 1 0.000% 100.108% ** Level 5 read latency histogram (micros): Count: 60434 Average: 547.5608 StdDev: 902.58 Min: 75 Median: 218.1393 Max: 22776 Percentiles: P50: 218.14 P75: 520.03 P99: 4817.86 P99.9: 7844.58 P99.99: 9861.31 ------------------------------------------------------ ( 51, 76 ] 3 0.005% 0.005% ( 76, 110 ] 5526 9.144% 9.149% ## ( 110, 170 ] 18398 30.443% 39.592% ###### ( 170, 250 ] 10454 17.298% 56.890% ### ( 250, 380 ] 6513 10.777% 67.667% ## ( 380, 580 ] 6331 10.476% 78.143% ## ( 580, 870 ] 4382 7.251% 85.394% # ( 870, 1300 ] 3048 5.044% 90.438% # ( 1300, 1900 ] 1820 3.012% 93.449% # ( 1900, 2900 ] 1594 2.638% 96.087% # ( 2900, 4400 ] 1642 2.717% 98.804% # ( 4400, 6600 ] 630 1.042% 99.846% ( 6600, 9900 ] 89 0.147% 99.993% ( 9900, 14000 ] 4 0.007% 100.000% ( 14000, 22000 ] 2 0.003% 100.003% ( 22000, 33000 ] 1 0.002% 100.005% ** DB Stats ** Uptime(secs): 11.8 total, 3.4 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 166.78 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.409 H:M:S, 37.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 286.27 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/20-18:33:28 ... thread 0: (1000000,3000000) ops and (291820.2,199286.4) ops/second in (3.426767,15.053713) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/7 3.98 GB 258.0 2.4 0.0 2.4 5.2 2.8 0.0 1.9 108.2 232.9 22.71 17.17 51 0.445 2494K 2669 0.0 0.0 L1 9/9 1.10 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 19/0 988.78 MB 1.0 1.6 1.0 0.7 1.6 0.9 0.1 1.7 108.4 107.3 15.27 7.80 8 1.908 690K 0 0.0 0.0 L4 160/0 9.40 GB 1.0 5.3 3.5 1.8 5.3 3.5 0.0 1.5 110.7 109.8 49.29 25.19 34 1.450 1923K 0 0.0 0.0 L5 1112/0 92.40 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.8 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1318/16 107.83 GB 0.0 9.3 4.5 4.9 12.1 7.2 3.9 4.4 109.7 141.4 87.26 50.16 93 0.938 5107K 2669 0.0 0.0 Int 0/0 0.00 KB 0.0 9.3 4.5 4.9 12.1 7.2 3.9 4.4 109.7 141.4 87.26 50.16 93 0.938 5107K 2669 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.3 4.5 4.9 9.3 4.4 0.0 0.0 127.4 126.6 75.09 42.42 51 1.472 5107K 2669 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 232.8 12.18 7.74 42 0.290 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.2 total, 15.1 interval Flush(GB): cumulative 2.769, interval 2.769 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.05 GB write, 810.32 MB/s write, 9.34 GB read, 628.37 MB/s read, 87.3 seconds Interval compaction: 12.05 GB write, 819.63 MB/s write, 9.34 GB read, 635.59 MB/s read, 87.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 25, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 7, total-stops: 25, interval: 31 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 983714 Average: 9.6881 StdDev: 96.67 Min: 1 Median: 1.6752 Max: 24830 Percentiles: P50: 1.68 P75: 2.41 P99: 185.48 P99.9: 865.79 P99.99: 2029.99 ------------------------------------------------------ [ 0, 1 ] 122767 12.480% 12.480% ## ( 1, 2 ] 546605 55.565% 68.045% ########### ( 2, 3 ] 168119 17.090% 85.136% ### ( 3, 4 ] 54851 5.576% 90.712% # ( 4, 6 ] 55060 5.597% 96.309% # ( 6, 10 ] 13457 1.368% 97.677% ( 10, 15 ] 2428 0.247% 97.923% ( 15, 22 ] 367 0.037% 97.961% ( 22, 34 ] 35 0.004% 97.964% ( 34, 51 ] 23 0.002% 97.967% ( 51, 76 ] 29 0.003% 97.970% ( 76, 110 ] 1932 0.196% 98.166% ( 110, 170 ] 7362 0.748% 98.914% ( 170, 250 ] 4356 0.443% 99.357% ( 250, 380 ] 2339 0.238% 99.595% ( 380, 580 ] 2003 0.204% 99.799% ( 580, 870 ] 1013 0.103% 99.902% ( 870, 1300 ] 543 0.055% 99.957% ( 1300, 1900 ] 295 0.030% 99.987% ( 1900, 2900 ] 251 0.026% 100.012% ( 2900, 4400 ] 213 0.022% 100.034% ( 4400, 6600 ] 46 0.005% 100.039% ( 6600, 9900 ] 6 0.001% 100.039% ( 22000, 33000 ] 1 0.000% 100.039% ** Level 1 read latency histogram (micros): Count: 191019 Average: 6.2582 StdDev: 85.15 Min: 1 Median: 1.9219 Max: 13210 Percentiles: P50: 1.92 P75: 2.79 P99: 21.83 P99.9: 600.45 P99.99: 3489.81 ------------------------------------------------------ [ 0, 1 ] 7046 3.689% 3.689% # ( 1, 2 ] 95960 50.236% 53.924% ########## ( 2, 3 ] 51171 26.788% 80.713% ##### ( 3, 4 ] 14206 7.437% 88.150% # ( 4, 6 ] 14943 7.823% 95.973% ## ( 6, 10 ] 4858 2.543% 98.516% # ( 10, 15 ] 715 0.374% 98.890% ( 15, 22 ] 215 0.113% 99.003% ( 22, 34 ] 14 0.007% 99.010% ( 34, 51 ] 2 0.001% 99.011% ( 51, 76 ] 1 0.001% 99.012% ( 76, 110 ] 234 0.123% 99.134% ( 110, 170 ] 687 0.360% 99.494% ( 170, 250 ] 414 0.217% 99.711% ( 250, 380 ] 193 0.101% 99.812% ( 380, 580 ] 162 0.085% 99.896% ( 580, 870 ] 99 0.052% 99.948% ( 870, 1300 ] 47 0.025% 99.973% ( 1300, 1900 ] 15 0.008% 99.981% ( 1900, 2900 ] 12 0.006% 99.987% ( 2900, 4400 ] 15 0.008% 99.995% ( 4400, 6600 ] 2 0.001% 99.996% ( 6600, 9900 ] 5 0.003% 99.998% ( 9900, 14000 ] 4 0.002% 100.001% ** Level 2 read latency histogram (micros): Count: 157699 Average: 11.1578 StdDev: 144.21 Min: 1 Median: 2.0640 Max: 11868 Percentiles: P50: 2.06 P75: 3.25 P99: 12.88 P99.9: 1913.41 P99.99: 3600.46 ------------------------------------------------------ [ 0, 1 ] 3812 2.417% 2.417% ( 1, 2 ] 72630 46.056% 48.473% ######### ( 2, 3 ] 37588 23.835% 72.309% ##### ( 3, 4 ] 17136 10.866% 83.175% ## ( 4, 6 ] 19076 12.096% 95.271% ## ( 6, 10 ] 5333 3.382% 98.653% # ( 10, 15 ] 949 0.602% 99.255% ( 15, 22 ] 138 0.088% 99.342% ( 22, 34 ] 25 0.016% 99.358% ( 34, 51 ] 11 0.007% 99.365% ( 51, 76 ] 14 0.009% 99.374% ( 76, 110 ] 35 0.022% 99.396% ( 110, 170 ] 91 0.058% 99.454% ( 170, 250 ] 72 0.046% 99.500% ( 250, 380 ] 100 0.063% 99.563% ( 380, 580 ] 125 0.079% 99.642% ( 580, 870 ] 147 0.093% 99.736% ( 870, 1300 ] 155 0.098% 99.834% ( 1300, 1900 ] 103 0.065% 99.899% ( 1900, 2900 ] 97 0.062% 99.961% ( 2900, 4400 ] 99 0.063% 100.023% ( 4400, 6600 ] 16 0.010% 100.034% ( 6600, 9900 ] 7 0.004% 100.038% ( 9900, 14000 ] 1 0.001% 100.039% ** Level 3 read latency histogram (micros): Count: 874784 Average: 11.8381 StdDev: 149.17 Min: 1 Median: 1.9827 Max: 15296 Percentiles: P50: 1.98 P75: 2.98 P99: 10.88 P99.9: 1166.89 P99.99: 1695.84 ------------------------------------------------------ [ 0, 1 ] 14979 1.712% 1.712% ( 1, 2 ] 429849 49.138% 50.850% ########## ( 2, 3 ] 215298 24.612% 75.462% ##### ( 3, 4 ] 89522 10.234% 85.695% ## ( 4, 6 ] 93631 10.703% 96.399% ## ( 6, 10 ] 22026 2.518% 98.916% # ( 10, 15 ] 4163 0.476% 99.392% ( 15, 22 ] 567 0.065% 99.457% ( 22, 34 ] 63 0.007% 99.464% ( 34, 51 ] 31 0.004% 99.468% ( 51, 76 ] 48 0.005% 99.473% ( 76, 110 ] 176 0.020% 99.493% ( 110, 170 ] 605 0.069% 99.563% ( 170, 250 ] 433 0.049% 99.612% ( 250, 380 ] 419 0.048% 99.660% ( 380, 580 ] 587 0.067% 99.727% ( 580, 870 ] 848 0.097% 99.824% ( 870, 1300 ] 962 0.110% 99.934% ( 1300, 1900 ] 742 0.085% 100.019% ( 1900, 2900 ] 676 0.077% 100.096% ( 2900, 4400 ] 547 0.063% 100.159% ( 4400, 6600 ] 124 0.014% 100.173% ( 6600, 9900 ] 26 0.003% 100.176% ( 9900, 14000 ] 2 0.000% 100.176% ( 14000, 22000 ] 2 0.000% 100.176% ** Level 4 read latency histogram (micros): Count: 487985 Average: 27.0056 StdDev: 220.33 Min: 1 Median: 2.0432 Max: 25676 Percentiles: P50: 2.04 P75: 3.24 P99: 565.28 P99.9: 2602.37 P99.99: 3481.78 ------------------------------------------------------ [ 0, 1 ] 8234 1.687% 1.687% ( 1, 2 ] 230772 47.291% 48.978% ######### ( 2, 3 ] 115323 23.632% 72.611% ##### ( 3, 4 ] 49301 10.103% 82.714% ## ( 4, 6 ] 50967 10.444% 93.158% ## ( 6, 10 ] 12020 2.463% 95.621% ( 10, 15 ] 2234 0.458% 96.079% ( 15, 22 ] 283 0.058% 96.137% ( 22, 34 ] 30 0.006% 96.143% ( 34, 51 ] 17 0.003% 96.147% ( 51, 76 ] 38 0.008% 96.154% ( 76, 110 ] 1762 0.361% 96.515% ( 110, 170 ] 5487 1.124% 97.640% ( 170, 250 ] 3063 0.628% 98.268% ( 250, 380 ] 1827 0.374% 98.642% ( 380, 580 ] 1886 0.386% 99.028% ( 580, 870 ] 1557 0.319% 99.348% ( 870, 1300 ] 1309 0.268% 99.616% ( 1300, 1900 ] 908 0.186% 99.802% ( 1900, 2900 ] 682 0.140% 99.942% ( 2900, 4400 ] 609 0.125% 100.066% ( 4400, 6600 ] 165 0.034% 100.100% ( 6600, 9900 ] 24 0.005% 100.105% ( 9900, 14000 ] 4 0.001% 100.106% ( 14000, 22000 ] 1 0.000% 100.106% ( 22000, 33000 ] 1 0.000% 100.106% ** Level 5 read latency histogram (micros): Count: 102452 Average: 410.6909 StdDev: 720.13 Min: 74 Median: 183.5318 Max: 22776 Percentiles: P50: 183.53 P75: 364.82 P99: 4125.72 P99.9: 6566.99 P99.99: 9668.44 ------------------------------------------------------ ( 51, 76 ] 7 0.007% 0.007% ( 76, 110 ] 11317 11.046% 11.053% ## ( 110, 170 ] 36606 35.730% 46.783% ####### ( 170, 250 ] 19486 19.020% 65.803% #### ( 250, 380 ] 10669 10.414% 76.216% ## ( 380, 580 ] 9622 9.392% 85.608% ## ( 580, 870 ] 5629 5.494% 91.102% # ( 870, 1300 ] 3284 3.205% 94.308% # ( 1300, 1900 ] 1843 1.799% 96.106% ( 1900, 2900 ] 1617 1.578% 97.685% ( 2900, 4400 ] 1649 1.610% 99.294% ( 4400, 6600 ] 630 0.615% 99.909% ( 6600, 9900 ] 89 0.087% 99.996% ( 9900, 14000 ] 4 0.004% 100.000% ( 14000, 22000 ] 2 0.002% 100.002% ( 22000, 33000 ] 1 0.001% 100.003% ** DB Stats ** Uptime(secs): 15.2 total, 3.4 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 193.88 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.409 H:M:S, 29.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 287.21 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 228.559 micros/op 17486 ops/sec 60.103 seconds 1050996 operations; 16.9 MB/s (258999 of 258999 found) Microseconds per read: Count: 1050996 Average: 228.5618 StdDev: 599.05 Min: 2 Median: 168.0347 Max: 63880 Percentiles: P50: 168.03 P75: 229.06 P99: 1002.85 P99.9: 10244.73 P99.99: 24973.77 ------------------------------------------------------ ( 1, 2 ] 11 0.001% 0.001% ( 2, 3 ] 64 0.006% 0.007% ( 3, 4 ] 65 0.006% 0.013% ( 4, 6 ] 104 0.010% 0.023% ( 6, 10 ] 276 0.026% 0.049% ( 10, 15 ] 2307 0.220% 0.269% ( 15, 22 ] 4202 0.400% 0.669% ( 22, 34 ] 8373 0.797% 1.465% ( 34, 51 ] 2923 0.278% 1.744% ( 51, 76 ] 217 0.021% 1.764% ( 76, 110 ] 25148 2.393% 4.157% ( 110, 170 ] 498124 47.395% 51.552% ######### ( 170, 250 ] 333824 31.763% 83.315% ###### ( 250, 380 ] 107284 10.208% 93.523% ## ( 380, 580 ] 39851 3.792% 97.315% # ( 580, 870 ] 15758 1.499% 98.814% ( 870, 1300 ] 6328 0.602% 99.416% ( 1300, 1900 ] 2317 0.220% 99.637% ( 1900, 2900 ] 1275 0.121% 99.758% ( 2900, 4400 ] 960 0.091% 99.849% ( 4400, 6600 ] 297 0.028% 99.877% ( 6600, 9900 ] 199 0.019% 99.896% ( 9900, 14000 ] 452 0.043% 99.939% ( 14000, 22000 ] 500 0.048% 99.987% ( 22000, 33000 ] 118 0.011% 99.998% ( 33000, 50000 ] 16 0.002% 100.000% ( 50000, 75000 ] 3 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 14641591 rocksdb.bloom.filter.full.positive COUNT : 1192968 rocksdb.bloom.filter.full.true.positive COUNT : 1050526 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 470 rocksdb.memtable.miss COUNT : 1050526 rocksdb.l0.hit COUNT : 38774 rocksdb.l1.hit COUNT : 17382 rocksdb.l2andup.hit COUNT : 994370 rocksdb.compaction.key.drop.new COUNT : 133082 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3906732 rocksdb.number.keys.read COUNT : 1050996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 4031747424 rocksdb.bytes.read COUNT : 1050996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1552 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 45179149 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 18319440933 rocksdb.compact.write.bytes COUNT : 18177819561 rocksdb.flush.write.bytes COUNT : 4029680361 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 30 rocksdb.number.superversion_acquires COUNT : 828 rocksdb.number.superversion_releases COUNT : 793 rocksdb.number.superversion_cleanups COUNT : 206 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 223 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3997005824 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1392640 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 22703354484 rocksdb.non.last.level.read.count COUNT : 5342621 rocksdb.block.checksum.compute.count COUNT : 5639649 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 167.116651 P95 : 456.531675 P99 : 1001.209949 P100 : 63878.000000 COUNT : 1050996 SUM : 239281970 rocksdb.db.write.micros P50 : 1.561379 P95 : 4.935367 P99 : 14.313910 P100 : 765185.000000 COUNT : 3906732 SUM : 55159271 rocksdb.compaction.times.micros P50 : 1307692.307692 P95 : 2855555.555556 P99 : 16560000.000000 P100 : 19350762.000000 COUNT : 68 SUM : 115401030 rocksdb.compaction.times.cpu_micros P50 : 882666.666667 P95 : 1655000.000000 P99 : 11144000.000000 P100 : 12037139.000000 COUNT : 68 SUM : 73695346 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 799.696970 P95 : 113000.000000 P99 : 304600.000000 P100 : 372229.000000 COUNT : 58 SUM : 776789 rocksdb.compaction.outfile.sync.micros P50 : 33000.000000 P95 : 227066.666667 P99 : 352440.000000 P100 : 590663.000000 COUNT : 206 SUM : 13415955 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 178.888889 P95 : 4121.428571 P99 : 6397.600000 P100 : 7323.000000 COUNT : 146 SUM : 119856 rocksdb.table.open.io.micros P50 : 8933.974359 P95 : 19253.608247 P99 : 21813.608247 P100 : 28308.000000 COUNT : 1552 SUM : 13907345 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.973045 P95 : 94.291920 P99 : 176.184484 P100 : 24834.000000 COUNT : 4435820 SUM : 74390756 rocksdb.read.block.get.micros P50 : 140.154953 P95 : 337.318324 P99 : 894.762733 P100 : 25681.000000 COUNT : 1202256 SUM : 204699946 rocksdb.write.raw.block.micros P50 : 0.530302 P95 : 1.192388 P99 : 3.734638 P100 : 14589.000000 COUNT : 5379102 SUM : 13675052 rocksdb.numfiles.in.singlecompaction P50 : 2.807692 P95 : 3.988571 P99 : 23.440000 P100 : 26.000000 COUNT : 88 SUM : 310 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.576456 P95 : 1278.606741 P99 : 2685.290253 P100 : 765001.000000 COUNT : 226161 SUM : 45199286 rocksdb.sst.read.micros P50 : 2.356445 P95 : 168.558184 P99 : 463.695611 P100 : 25676.000000 COUNT : 5342637 SUM : 263652108 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1050996 SUM : 1050996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3906732 SUM : 4031747424 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 217346.938776 P95 : 871333.333333 P99 : 1026570.000000 P100 : 1026570.000000 COUNT : 58 SUM : 15429847 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 223 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0