[2024-11-15 11:10:20.782455] Starting SPDK v25.01-pre git sha1 4bd31eb0a / DPDK 24.03.0 initialization... [2024-11-15 11:10:20.782616] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1419563 ] Set seed to 1731665420778898 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-15 11:10:20.922632] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-15 11:10:21.025628] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-15 11:10:21.025687] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1419563' to capture a snapshot of events at runtime. [2024-11-15 11:10:21.025700] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-15 11:10:21.025714] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-15 11:10:21.025724] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1419563 for offline analysis/debug. [2024-11-15 11:10:21.026922] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Fri Nov 15 11:10:24 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/11/15-11:10:30 ... thread 0: (1000000,1000000) ops and (158335.5,158335.5) ops/second in (6.315704,6.315704) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/13 1.32 GB 25.1 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 149.5 5.89 2.65 11 0.535 0 0 0.0 0.0 L1 4/4 281.78 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 24/6 1.31 GB 1.0 2.3 1.7 0.6 2.2 1.6 0.0 1.3 118.6 116.6 19.64 11.48 11 1.785 1776K 8464 0.0 0.0 L4 155/24 10.82 GB 0.9 3.3 1.7 1.5 3.1 1.5 0.4 1.8 132.5 124.3 25.22 16.25 15 1.681 710K 0 0.0 0.0 L5 136/0 10.52 GB 0.1 0.0 0.0 0.0 0.0 0.0 1.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 335/47 24.24 GB 0.0 5.5 3.4 2.2 6.2 4.0 1.6 7.2 111.7 124.2 50.75 30.37 37 1.372 2487K 8464 0.0 0.0 Int 0/0 0.00 KB 0.0 5.5 3.4 2.2 6.2 4.0 1.6 7.2 111.7 124.2 50.75 30.37 37 1.372 2487K 8464 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.5 3.4 2.2 5.3 3.1 0.0 0.0 126.4 120.9 44.86 27.73 26 1.725 2487K 8464 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 149.5 5.89 2.65 11 0.535 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.4 total, 6.3 interval Flush(GB): cumulative 0.859, interval 0.859 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.16 GB write, 985.32 MB/s write, 5.54 GB read, 886.11 MB/s read, 50.7 seconds Interval compaction: 6.16 GB write, 998.03 MB/s write, 5.54 GB read, 897.54 MB/s read, 50.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 20, interval: 20 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 51525 Average: 222.3301 StdDev: 1007.65 Min: 1 Median: 2.6367 Max: 61966 Percentiles: P50: 2.64 P75: 5.88 P99: 3457.16 P99.9: 7529.02 P99.99: 44117.29 ------------------------------------------------------ [ 0, 1 ] 758 1.471% 1.471% ( 1, 2 ] 17104 33.196% 34.667% ####### ( 2, 3 ] 12408 24.082% 58.748% ##### ( 3, 4 ] 4507 8.747% 67.495% ## ( 4, 6 ] 4107 7.971% 75.466% ## ( 6, 10 ] 1160 2.251% 77.718% ( 10, 15 ] 345 0.670% 78.387% ( 15, 22 ] 50 0.097% 78.484% ( 22, 34 ] 14 0.027% 78.511% ( 34, 51 ] 4 0.008% 78.519% ( 51, 76 ] 6 0.012% 78.531% ( 76, 110 ] 1 0.002% 78.533% ( 110, 170 ] 68 0.132% 78.665% ( 170, 250 ] 372 0.722% 79.387% ( 250, 380 ] 1214 2.356% 81.743% ( 380, 580 ] 2967 5.758% 87.501% # ( 580, 870 ] 3249 6.306% 93.807% # ( 870, 1300 ] 1587 3.080% 96.887% # ( 1300, 1900 ] 410 0.796% 97.683% ( 1900, 2900 ] 470 0.912% 98.595% ( 2900, 4400 ] 562 1.091% 99.686% ( 4400, 6600 ] 104 0.202% 99.887% ( 6600, 9900 ] 23 0.045% 99.932% ( 9900, 14000 ] 5 0.010% 99.942% ( 14000, 22000 ] 11 0.021% 99.963% ( 22000, 33000 ] 6 0.012% 99.975% ( 33000, 50000 ] 12 0.023% 99.998% ( 50000, 75000 ] 2 0.004% 100.002% ** Level 1 read latency histogram (micros): Count: 263 Average: 1033.4791 StdDev: 1013.57 Min: 107 Median: 692.4157 Max: 5097 Percentiles: P50: 692.42 P75: 1016.59 P99: 5097.00 P99.9: 5097.00 P99.99: 5097.00 ------------------------------------------------------ ( 76, 110 ] 1 0.380% 0.380% ( 110, 170 ] 2 0.760% 1.141% ( 170, 250 ] 5 1.901% 3.042% ( 250, 380 ] 22 8.365% 11.407% ## ( 380, 580 ] 67 25.475% 36.882% ##### ( 580, 870 ] 89 33.840% 70.722% ####### ( 870, 1300 ] 33 12.548% 83.270% ### ( 1300, 1900 ] 8 3.042% 86.312% # ( 1900, 2900 ] 12 4.563% 90.875% # ( 2900, 4400 ] 19 7.224% 98.099% # ( 4400, 6600 ] 5 1.901% 100.000% ** Level 2 read latency histogram (micros): Count: 318649 Average: 12.2803 StdDev: 204.42 Min: 1 Median: 2.2348 Max: 42756 Percentiles: P50: 2.23 P75: 3.22 P99: 13.04 P99.9: 1636.97 P99.99: 3082.52 ------------------------------------------------------ [ 0, 1 ] 3416 1.072% 1.072% ( 1, 2 ] 133860 42.009% 43.081% ######## ( 2, 3 ] 93916 29.473% 72.554% ###### ( 3, 4 ] 35475 11.133% 83.687% ## ( 4, 6 ] 37835 11.874% 95.560% ## ( 6, 10 ] 9376 2.942% 98.503% # ( 10, 15 ] 2606 0.818% 99.321% ( 15, 22 ] 361 0.113% 99.434% ( 22, 34 ] 53 0.017% 99.450% ( 34, 51 ] 78 0.024% 99.475% ( 51, 76 ] 74 0.023% 99.498% ( 76, 110 ] 25 0.008% 99.506% ( 110, 170 ] 18 0.006% 99.512% ( 170, 250 ] 49 0.015% 99.527% ( 250, 380 ] 213 0.067% 99.594% ( 380, 580 ] 374 0.117% 99.711% ( 580, 870 ] 323 0.101% 99.813% ( 870, 1300 ] 184 0.058% 99.870% ( 1300, 1900 ] 168 0.053% 99.923% ( 1900, 2900 ] 186 0.058% 99.981% ( 2900, 4400 ] 223 0.070% 100.051% ( 4400, 6600 ] 73 0.023% 100.074% ( 6600, 9900 ] 12 0.004% 100.078% ( 9900, 14000 ] 5 0.002% 100.080% ( 14000, 22000 ] 4 0.001% 100.081% ( 22000, 33000 ] 3 0.001% 100.082% ( 33000, 50000 ] 1 0.000% 100.082% ** Level 3 read latency histogram (micros): Count: 575183 Average: 14.5048 StdDev: 265.23 Min: 1 Median: 2.2749 Max: 59043 Percentiles: P50: 2.27 P75: 3.25 P99: 13.85 P99.9: 974.07 P99.99: 1410.19 ------------------------------------------------------ [ 0, 1 ] 5272 0.917% 0.917% ( 1, 2 ] 233916 40.668% 41.585% ######## ( 2, 3 ] 176072 30.611% 72.196% ###### ( 3, 4 ] 65154 11.328% 83.524% ## ( 4, 6 ] 69277 12.044% 95.568% ## ( 6, 10 ] 16859 2.931% 98.499% # ( 10, 15 ] 3746 0.651% 99.150% ( 15, 22 ] 530 0.092% 99.243% ( 22, 34 ] 97 0.017% 99.259% ( 34, 51 ] 193 0.034% 99.293% ( 51, 76 ] 168 0.029% 99.322% ( 76, 110 ] 39 0.007% 99.329% ( 110, 170 ] 59 0.010% 99.339% ( 170, 250 ] 159 0.028% 99.367% ( 250, 380 ] 531 0.092% 99.459% ( 380, 580 ] 1136 0.198% 99.657% ( 580, 870 ] 1250 0.217% 99.874% ( 870, 1300 ] 619 0.108% 99.982% ( 1300, 1900 ] 264 0.046% 100.027% ( 1900, 2900 ] 317 0.055% 100.083% ( 2900, 4400 ] 367 0.064% 100.146% ( 4400, 6600 ] 96 0.017% 100.163% ( 6600, 9900 ] 32 0.006% 100.169% ( 9900, 14000 ] 8 0.001% 100.170% ( 14000, 22000 ] 21 0.004% 100.174% ( 22000, 33000 ] 12 0.002% 100.176% ( 33000, 50000 ] 6 0.001% 100.177% ( 50000, 75000 ] 1 0.000% 100.177% ** Level 4 read latency histogram (micros): Count: 632058 Average: 26.0817 StdDev: 335.05 Min: 1 Median: 2.3175 Max: 61591 Percentiles: P50: 2.32 P75: 3.39 P99: 544.22 P99.9: 1278.40 P99.99: 1764.32 ------------------------------------------------------ [ 0, 1 ] 5911 0.935% 0.935% ( 1, 2 ] 250180 39.582% 40.517% ######## ( 2, 3 ] 188770 29.866% 70.383% ###### ( 3, 4 ] 75453 11.938% 82.321% ## ( 4, 6 ] 76606 12.120% 94.441% ## ( 6, 10 ] 18814 2.977% 97.417% # ( 10, 15 ] 4379 0.693% 98.110% ( 15, 22 ] 487 0.077% 98.187% ( 22, 34 ] 160 0.025% 98.213% ( 34, 51 ] 226 0.036% 98.248% ( 51, 76 ] 171 0.027% 98.275% ( 76, 110 ] 74 0.012% 98.287% ( 110, 170 ] 131 0.021% 98.308% ( 170, 250 ] 407 0.064% 98.372% ( 250, 380 ] 1415 0.224% 98.596% ( 380, 580 ] 3111 0.492% 99.088% ( 580, 870 ] 3448 0.546% 99.634% ( 870, 1300 ] 1773 0.281% 99.914% ( 1300, 1900 ] 620 0.098% 100.012% ( 1900, 2900 ] 681 0.108% 100.120% ( 2900, 4400 ] 789 0.125% 100.245% ( 4400, 6600 ] 179 0.028% 100.273% ( 6600, 9900 ] 46 0.007% 100.281% ( 9900, 14000 ] 9 0.001% 100.282% ( 14000, 22000 ] 28 0.004% 100.286% ( 22000, 33000 ] 19 0.003% 100.289% ( 33000, 50000 ] 9 0.001% 100.291% ( 50000, 75000 ] 2 0.000% 100.291% ** Level 5 read latency histogram (micros): Count: 7796 Average: 1032.6814 StdDev: 1707.03 Min: 101 Median: 671.0733 Max: 49248 Percentiles: P50: 671.07 P75: 1029.74 P99: 5521.58 P99.9: 33433.50 P99.99: 48343.35 ------------------------------------------------------ ( 76, 110 ] 2 0.026% 0.026% ( 110, 170 ] 46 0.590% 0.616% ( 170, 250 ] 239 3.066% 3.681% # ( 250, 380 ] 857 10.993% 14.674% ## ( 380, 580 ] 2043 26.206% 40.880% ##### ( 580, 870 ] 2264 29.041% 69.920% ###### ( 870, 1300 ] 1066 13.674% 83.594% ### ( 1300, 1900 ] 333 4.271% 87.866% # ( 1900, 2900 ] 368 4.720% 92.586% # ( 2900, 4400 ] 446 5.721% 98.307% # ( 4400, 6600 ] 106 1.360% 99.666% ( 6600, 9900 ] 11 0.141% 99.808% ( 9900, 14000 ] 3 0.038% 99.846% ( 14000, 22000 ] 2 0.026% 99.872% ( 22000, 33000 ] 2 0.026% 99.897% ( 33000, 50000 ] 8 0.103% 100.000% ** DB Stats ** Uptime(secs): 6.4 total, 6.3 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 153.79 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.559 H:M:S, 24.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 155.75 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.559 H:M:S, 24.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 9 num-running-flushes: 1 2024/11/15-11:10:36 ... thread 0: (1000000,2000000) ops and (158570.7,158453.0) ops/second in (6.306334,12.622038) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/9 2.36 GB 162.8 1.3 0.0 1.3 3.2 1.9 0.0 1.7 66.3 163.9 19.92 11.00 33 0.604 1341K 6703 0.0 0.0 L1 4/4 281.78 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 18/0 959.30 MB 1.0 2.3 1.7 0.6 2.2 1.6 0.0 1.3 118.6 116.6 19.64 11.48 11 1.785 1776K 8464 0.0 0.0 L4 145/0 9.65 GB 1.0 5.5 2.1 3.4 4.9 1.5 0.4 2.3 146.7 131.3 38.26 25.67 21 1.822 1096K 0 0.0 0.0 L5 152/0 11.69 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 337/13 24.91 GB 0.0 9.0 3.8 5.3 10.3 5.0 2.8 5.4 119.0 135.9 77.81 48.15 65 1.197 4215K 15K 0.0 0.0 Int 0/0 0.00 KB 0.0 9.0 3.8 5.3 10.3 5.0 2.8 5.4 119.0 135.9 77.81 48.15 65 1.197 4215K 15K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.0 3.8 5.3 8.4 3.1 0.0 0.0 135.7 126.4 68.26 42.51 37 1.845 4215K 15K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.9 1.9 0.0 0.0 0.0 204.0 9.56 5.64 28 0.341 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.7 total, 12.6 interval Flush(GB): cumulative 1.904, interval 1.904 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.33 GB write, 832.61 MB/s write, 9.04 GB read, 728.91 MB/s read, 77.8 seconds Interval compaction: 10.33 GB write, 837.99 MB/s write, 9.04 GB read, 733.62 MB/s read, 77.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 12, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 12, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 12, total-stops: 20, interval: 32 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 499137 Average: 56.8902 StdDev: 374.38 Min: 0 Median: 1.7110 Max: 61966 Percentiles: P50: 1.71 P75: 2.82 P99: 748.68 P99.9: 2558.51 P99.99: 3518.13 ------------------------------------------------------ [ 0, 1 ] 66306 13.284% 13.284% ### ( 1, 2 ] 257751 51.639% 64.923% ########## ( 2, 3 ] 60979 12.217% 77.140% ## ( 3, 4 ] 20757 4.159% 81.299% # ( 4, 6 ] 14042 2.813% 84.112% # ( 6, 10 ] 4764 0.954% 85.067% ( 10, 15 ] 1196 0.240% 85.306% ( 15, 22 ] 228 0.046% 85.352% ( 22, 34 ] 59 0.012% 85.364% ( 34, 51 ] 11 0.002% 85.366% ( 51, 76 ] 12 0.002% 85.368% ( 76, 110 ] 296 0.059% 85.428% ( 110, 170 ] 10944 2.193% 87.620% ( 170, 250 ] 34197 6.851% 94.471% # ( 250, 380 ] 12845 2.573% 97.045% # ( 380, 580 ] 7155 1.433% 98.478% ( 580, 870 ] 4478 0.897% 99.376% ( 870, 1300 ] 1798 0.360% 99.736% ( 1300, 1900 ] 460 0.092% 99.828% ( 1900, 2900 ] 548 0.110% 99.938% ( 2900, 4400 ] 636 0.127% 100.065% ( 4400, 6600 ] 107 0.021% 100.087% ( 6600, 9900 ] 31 0.006% 100.093% ( 9900, 14000 ] 7 0.001% 100.094% ( 14000, 22000 ] 20 0.004% 100.098% ( 22000, 33000 ] 16 0.003% 100.101% ( 33000, 50000 ] 12 0.002% 100.104% ( 50000, 75000 ] 2 0.000% 100.104% ** Level 1 read latency histogram (micros): Count: 1341 Average: 412.7435 StdDev: 811.71 Min: 97 Median: 232.2010 Max: 20321 Percentiles: P50: 232.20 P75: 360.16 P99: 3893.95 P99.9: 8774.70 P99.99: 20321.00 ------------------------------------------------------ ( 76, 110 ] 4 0.298% 0.298% ( 110, 170 ] 179 13.348% 13.647% ### ( 170, 250 ] 627 46.756% 60.403% ######### ( 250, 380 ] 231 17.226% 77.629% ### ( 380, 580 ] 114 8.501% 86.130% ## ( 580, 870 ] 105 7.830% 93.960% ## ( 870, 1300 ] 35 2.610% 96.570% # ( 1300, 1900 ] 8 0.597% 97.166% ( 1900, 2900 ] 12 0.895% 98.061% ( 2900, 4400 ] 19 1.417% 99.478% ( 4400, 6600 ] 5 0.373% 99.851% ( 6600, 9900 ] 1 0.075% 99.925% ( 14000, 22000 ] 1 0.075% 100.000% ** Level 2 read latency histogram (micros): Count: 318649 Average: 12.2803 StdDev: 204.42 Min: 1 Median: 2.2348 Max: 42756 Percentiles: P50: 2.23 P75: 3.22 P99: 13.04 P99.9: 1636.97 P99.99: 3082.52 ------------------------------------------------------ [ 0, 1 ] 3416 1.072% 1.072% ( 1, 2 ] 133860 42.009% 43.081% ######## ( 2, 3 ] 93916 29.473% 72.554% ###### ( 3, 4 ] 35475 11.133% 83.687% ## ( 4, 6 ] 37835 11.874% 95.560% ## ( 6, 10 ] 9376 2.942% 98.503% # ( 10, 15 ] 2606 0.818% 99.321% ( 15, 22 ] 361 0.113% 99.434% ( 22, 34 ] 53 0.017% 99.450% ( 34, 51 ] 78 0.024% 99.475% ( 51, 76 ] 74 0.023% 99.498% ( 76, 110 ] 25 0.008% 99.506% ( 110, 170 ] 18 0.006% 99.512% ( 170, 250 ] 49 0.015% 99.527% ( 250, 380 ] 213 0.067% 99.594% ( 380, 580 ] 374 0.117% 99.711% ( 580, 870 ] 323 0.101% 99.813% ( 870, 1300 ] 184 0.058% 99.870% ( 1300, 1900 ] 168 0.053% 99.923% ( 1900, 2900 ] 186 0.058% 99.981% ( 2900, 4400 ] 223 0.070% 100.051% ( 4400, 6600 ] 73 0.023% 100.074% ( 6600, 9900 ] 12 0.004% 100.078% ( 9900, 14000 ] 5 0.002% 100.080% ( 14000, 22000 ] 4 0.001% 100.081% ( 22000, 33000 ] 3 0.001% 100.082% ( 33000, 50000 ] 1 0.000% 100.082% ** Level 3 read latency histogram (micros): Count: 629965 Average: 15.6886 StdDev: 257.78 Min: 1 Median: 2.2599 Max: 59043 Percentiles: P50: 2.26 P75: 3.25 P99: 218.67 P99.9: 943.16 P99.99: 1389.23 ------------------------------------------------------ [ 0, 1 ] 6890 1.094% 1.094% ( 1, 2 ] 258959 41.107% 42.201% ######## ( 2, 3 ] 189059 30.011% 72.212% ###### ( 3, 4 ] 69472 11.028% 83.240% ## ( 4, 6 ] 73111 11.606% 94.845% ## ( 6, 10 ] 18172 2.885% 97.730% # ( 10, 15 ] 3973 0.631% 98.360% ( 15, 22 ] 562 0.089% 98.450% ( 22, 34 ] 100 0.016% 98.465% ( 34, 51 ] 204 0.032% 98.498% ( 51, 76 ] 174 0.028% 98.525% ( 76, 110 ] 71 0.011% 98.537% ( 110, 170 ] 1007 0.160% 98.697% ( 170, 250 ] 3142 0.499% 99.195% ( 250, 380 ] 1493 0.237% 99.432% ( 380, 580 ] 1475 0.234% 99.666% ( 580, 870 ] 1363 0.216% 99.883% ( 870, 1300 ] 635 0.101% 99.984% ( 1300, 1900 ] 269 0.043% 100.026% ( 1900, 2900 ] 321 0.051% 100.077% ( 2900, 4400 ] 372 0.059% 100.136% ( 4400, 6600 ] 98 0.016% 100.152% ( 6600, 9900 ] 32 0.005% 100.157% ( 9900, 14000 ] 10 0.002% 100.159% ( 14000, 22000 ] 23 0.004% 100.162% ( 22000, 33000 ] 12 0.002% 100.164% ( 33000, 50000 ] 6 0.001% 100.165% ( 50000, 75000 ] 1 0.000% 100.165% ** Level 4 read latency histogram (micros): Count: 916968 Average: 29.6245 StdDev: 302.53 Min: 0 Median: 2.2950 Max: 61591 Percentiles: P50: 2.29 P75: 3.48 P99: 466.34 P99.9: 1116.04 P99.99: 1301.21 ------------------------------------------------------ [ 0, 1 ] 11043 1.204% 1.204% ( 1, 2 ] 371357 40.498% 41.703% ######## ( 2, 3 ] 257933 28.129% 69.832% ###### ( 3, 4 ] 98549 10.747% 80.579% ## ( 4, 6 ] 96933 10.571% 91.150% ## ( 6, 10 ] 25362 2.766% 93.916% # ( 10, 15 ] 5803 0.633% 94.549% ( 15, 22 ] 610 0.067% 94.615% ( 22, 34 ] 185 0.020% 94.635% ( 34, 51 ] 261 0.028% 94.664% ( 51, 76 ] 215 0.023% 94.687% ( 76, 110 ] 316 0.034% 94.722% ( 110, 170 ] 7227 0.788% 95.510% ( 170, 250 ] 21860 2.384% 97.894% ( 250, 380 ] 7829 0.854% 98.748% ( 380, 580 ] 5363 0.585% 99.332% ( 580, 870 ] 4103 0.447% 99.780% ( 870, 1300 ] 1926 0.210% 99.990% ( 1300, 1900 ] 646 0.070% 100.060% ( 1900, 2900 ] 725 0.079% 100.139% ( 2900, 4400 ] 831 0.091% 100.230% ( 4400, 6600 ] 182 0.020% 100.250% ( 6600, 9900 ] 63 0.007% 100.257% ( 9900, 14000 ] 25 0.003% 100.259% ( 14000, 22000 ] 37 0.004% 100.263% ( 22000, 33000 ] 25 0.003% 100.266% ( 33000, 50000 ] 9 0.001% 100.267% ( 50000, 75000 ] 2 0.000% 100.267% ** Level 5 read latency histogram (micros): Count: 36720 Average: 412.0544 StdDev: 909.51 Min: 91 Median: 232.2114 Max: 49248 Percentiles: P50: 232.21 P75: 373.88 P99: 3682.80 P99.9: 6524.92 P99.99: 42197.00 ------------------------------------------------------ ( 76, 110 ] 127 0.346% 0.346% ( 110, 170 ] 5440 14.815% 15.161% ### ( 170, 250 ] 16451 44.801% 59.962% ######### ( 250, 380 ] 5795 15.782% 75.743% ### ( 380, 580 ] 3690 10.049% 85.792% ## ( 580, 870 ] 2713 7.388% 93.181% # ( 870, 1300 ] 1154 3.143% 96.324% # ( 1300, 1900 ] 351 0.956% 97.279% ( 1900, 2900 ] 386 1.051% 98.331% ( 2900, 4400 ] 471 1.283% 99.613% ( 4400, 6600 ] 109 0.297% 99.910% ( 6600, 9900 ] 13 0.035% 99.946% ( 9900, 14000 ] 3 0.008% 99.954% ( 14000, 22000 ] 4 0.011% 99.965% ( 22000, 33000 ] 5 0.014% 99.978% ( 33000, 50000 ] 8 0.022% 100.000% ** DB Stats ** Uptime(secs): 12.7 total, 6.3 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 154.92 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.994 H:M:S, 31.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 156.07 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.435 H:M:S, 38.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 2024/11/15-11:10:44 ... thread 0: (1000000,3000000) ops and (134777.1,149687.9) ops/second in (7.419657,20.041695) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/12 3.27 GB 131.4 2.6 0.0 2.6 5.4 2.8 0.0 1.9 79.3 165.7 33.33 19.09 53 0.629 2682K 13K 0.0 0.0 L1 4/4 281.78 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 18/0 959.30 MB 1.0 2.3 1.7 0.6 2.2 1.6 0.0 1.3 118.6 116.6 19.64 11.48 11 1.785 1776K 8464 0.0 0.0 L4 145/0 9.65 GB 1.0 5.5 2.1 3.4 4.9 1.5 0.4 2.3 146.7 131.3 38.26 25.67 21 1.822 1096K 0 0.0 0.0 L5 152/0 11.69 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 337/16 25.82 GB 0.0 10.3 3.8 6.6 12.5 6.0 2.8 4.4 116.0 140.7 91.23 56.24 85 1.073 5556K 21K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.3 3.8 6.6 12.5 6.0 2.8 4.4 116.0 140.7 91.23 56.24 85 1.073 5556K 21K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.3 3.8 6.6 9.7 3.1 0.0 0.0 134.3 126.2 78.80 48.07 42 1.876 5556K 21K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 232.9 12.42 8.17 43 0.289 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.1 total, 20.0 interval Flush(GB): cumulative 2.825, interval 2.825 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.54 GB write, 637.84 MB/s write, 10.33 GB read, 525.81 MB/s read, 91.2 seconds Interval compaction: 12.54 GB write, 640.43 MB/s write, 10.33 GB read, 527.95 MB/s read, 91.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 18, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 18, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 20, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 18, total-stops: 20, interval: 38 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 927922 Average: 44.3498 StdDev: 279.20 Min: 0 Median: 1.6604 Max: 61966 Percentiles: P50: 1.66 P75: 2.74 P99: 535.50 P99.9: 1398.82 P99.99: 2706.18 ------------------------------------------------------ [ 0, 1 ] 153974 16.593% 16.593% ### ( 1, 2 ] 469384 50.584% 67.178% ########## ( 2, 3 ] 97597 10.518% 77.696% ## ( 3, 4 ] 39829 4.292% 81.988% # ( 4, 6 ] 23210 2.501% 84.489% # ( 6, 10 ] 7092 0.764% 85.254% ( 10, 15 ] 1817 0.196% 85.449% ( 15, 22 ] 366 0.039% 85.489% ( 22, 34 ] 102 0.011% 85.500% ( 34, 51 ] 12 0.001% 85.501% ( 51, 76 ] 14 0.002% 85.503% ( 76, 110 ] 1208 0.130% 85.633% ( 110, 170 ] 30686 3.307% 88.940% # ( 170, 250 ] 69837 7.526% 96.466% ## ( 250, 380 ] 17119 1.845% 98.311% ( 380, 580 ] 8226 0.886% 99.197% ( 580, 870 ] 4623 0.498% 99.695% ( 870, 1300 ] 1821 0.196% 99.892% ( 1300, 1900 ] 468 0.050% 99.942% ( 1900, 2900 ] 551 0.059% 100.002% ( 2900, 4400 ] 636 0.069% 100.070% ( 4400, 6600 ] 107 0.012% 100.082% ( 6600, 9900 ] 31 0.003% 100.085% ( 9900, 14000 ] 7 0.001% 100.086% ( 14000, 22000 ] 20 0.002% 100.088% ( 22000, 33000 ] 16 0.002% 100.090% ( 33000, 50000 ] 12 0.001% 100.091% ( 50000, 75000 ] 2 0.000% 100.091% ** Level 1 read latency histogram (micros): Count: 39697 Average: 24.5795 StdDev: 170.45 Min: 1 Median: 2.1594 Max: 20321 Percentiles: P50: 2.16 P75: 2.98 P99: 353.48 P99.9: 1697.65 P99.99: 5293.29 ------------------------------------------------------ [ 0, 1 ] 879 2.214% 2.214% ( 1, 2 ] 17039 42.923% 45.137% ######### ( 2, 3 ] 12112 30.511% 75.648% ###### ( 3, 4 ] 3282 8.268% 83.916% ## ( 4, 6 ] 2242 5.648% 89.563% # ( 6, 10 ] 950 2.393% 91.957% ( 10, 15 ] 146 0.368% 92.324% ( 15, 22 ] 42 0.106% 92.430% ( 22, 34 ] 11 0.028% 92.458% ( 34, 51 ] 1 0.003% 92.460% ( 76, 110 ] 35 0.088% 92.549% ( 110, 170 ] 739 1.862% 94.410% ( 170, 250 ] 1550 3.905% 98.315% # ( 250, 380 ] 343 0.864% 99.179% ( 380, 580 ] 140 0.353% 99.531% ( 580, 870 ] 107 0.270% 99.801% ( 870, 1300 ] 35 0.088% 99.889% ( 1300, 1900 ] 8 0.020% 99.909% ( 1900, 2900 ] 12 0.030% 99.940% ( 2900, 4400 ] 19 0.048% 99.987% ( 4400, 6600 ] 5 0.013% 100.000% ( 6600, 9900 ] 1 0.003% 100.003% ( 14000, 22000 ] 1 0.003% 100.005% ** Level 2 read latency histogram (micros): Count: 318649 Average: 12.2803 StdDev: 204.42 Min: 1 Median: 2.2348 Max: 42756 Percentiles: P50: 2.23 P75: 3.22 P99: 13.04 P99.9: 1636.97 P99.99: 3082.52 ------------------------------------------------------ [ 0, 1 ] 3416 1.072% 1.072% ( 1, 2 ] 133860 42.009% 43.081% ######## ( 2, 3 ] 93916 29.473% 72.554% ###### ( 3, 4 ] 35475 11.133% 83.687% ## ( 4, 6 ] 37835 11.874% 95.560% ## ( 6, 10 ] 9376 2.942% 98.503% # ( 10, 15 ] 2606 0.818% 99.321% ( 15, 22 ] 361 0.113% 99.434% ( 22, 34 ] 53 0.017% 99.450% ( 34, 51 ] 78 0.024% 99.475% ( 51, 76 ] 74 0.023% 99.498% ( 76, 110 ] 25 0.008% 99.506% ( 110, 170 ] 18 0.006% 99.512% ( 170, 250 ] 49 0.015% 99.527% ( 250, 380 ] 213 0.067% 99.594% ( 380, 580 ] 374 0.117% 99.711% ( 580, 870 ] 323 0.101% 99.813% ( 870, 1300 ] 184 0.058% 99.870% ( 1300, 1900 ] 168 0.053% 99.923% ( 1900, 2900 ] 186 0.058% 99.981% ( 2900, 4400 ] 223 0.070% 100.051% ( 4400, 6600 ] 73 0.023% 100.074% ( 6600, 9900 ] 12 0.004% 100.078% ( 9900, 14000 ] 5 0.002% 100.080% ( 14000, 22000 ] 4 0.001% 100.081% ( 22000, 33000 ] 3 0.001% 100.082% ( 33000, 50000 ] 1 0.000% 100.082% ** Level 3 read latency histogram (micros): Count: 637421 Average: 17.7398 StdDev: 257.05 Min: 1 Median: 2.2796 Max: 59043 Percentiles: P50: 2.28 P75: 3.33 P99: 242.32 P99.9: 940.94 P99.99: 1387.56 ------------------------------------------------------ [ 0, 1 ] 6890 1.081% 1.081% ( 1, 2 ] 258959 40.626% 41.707% ######## ( 2, 3 ] 189059 29.660% 71.367% ###### ( 3, 4 ] 69472 10.899% 82.266% ## ( 4, 6 ] 73111 11.470% 93.736% ## ( 6, 10 ] 18172 2.851% 96.587% # ( 10, 15 ] 3973 0.623% 97.210% ( 15, 22 ] 562 0.088% 97.298% ( 22, 34 ] 100 0.016% 97.314% ( 34, 51 ] 204 0.032% 97.346% ( 51, 76 ] 174 0.027% 97.373% ( 76, 110 ] 200 0.031% 97.404% ( 110, 170 ] 3520 0.552% 97.957% ( 170, 250 ] 7357 1.154% 99.111% ( 250, 380 ] 1960 0.307% 99.418% ( 380, 580 ] 1591 0.250% 99.668% ( 580, 870 ] 1374 0.216% 99.883% ( 870, 1300 ] 640 0.100% 99.984% ( 1300, 1900 ] 269 0.042% 100.026% ( 1900, 2900 ] 321 0.050% 100.076% ( 2900, 4400 ] 372 0.058% 100.135% ( 4400, 6600 ] 98 0.015% 100.150% ( 6600, 9900 ] 32 0.005% 100.155% ( 9900, 14000 ] 10 0.002% 100.157% ( 14000, 22000 ] 23 0.004% 100.160% ( 22000, 33000 ] 12 0.002% 100.162% ( 33000, 50000 ] 6 0.001% 100.163% ( 50000, 75000 ] 1 0.000% 100.163% ** Level 4 read latency histogram (micros): Count: 973036 Average: 38.9064 StdDev: 296.45 Min: 0 Median: 2.4034 Max: 61591 Percentiles: P50: 2.40 P75: 3.91 P99: 469.64 P99.9: 1105.91 P99.99: 1299.71 ------------------------------------------------------ [ 0, 1 ] 11043 1.135% 1.135% ( 1, 2 ] 371379 38.167% 39.302% ######## ( 2, 3 ] 258038 26.519% 65.821% ##### ( 3, 4 ] 98608 10.134% 75.955% ## ( 4, 6 ] 97051 9.974% 85.929% ## ( 6, 10 ] 25374 2.608% 88.537% # ( 10, 15 ] 5814 0.598% 89.134% ( 15, 22 ] 611 0.063% 89.197% ( 22, 34 ] 185 0.019% 89.216% ( 34, 51 ] 261 0.027% 89.243% ( 51, 76 ] 215 0.022% 89.265% ( 76, 110 ] 1230 0.126% 89.391% ( 110, 170 ] 25767 2.648% 92.039% # ( 170, 250 ] 53575 5.506% 97.545% # ( 250, 380 ] 11325 1.164% 98.709% ( 380, 580 ] 6313 0.649% 99.358% ( 580, 870 ] 4208 0.432% 99.790% ( 870, 1300 ] 1943 0.200% 99.990% ( 1300, 1900 ] 650 0.067% 100.057% ( 1900, 2900 ] 725 0.075% 100.131% ( 2900, 4400 ] 831 0.085% 100.217% ( 4400, 6600 ] 182 0.019% 100.236% ( 6600, 9900 ] 63 0.006% 100.242% ( 9900, 14000 ] 25 0.003% 100.245% ( 14000, 22000 ] 37 0.004% 100.248% ( 22000, 33000 ] 25 0.003% 100.251% ( 33000, 50000 ] 9 0.001% 100.252% ( 50000, 75000 ] 2 0.000% 100.252% ** Level 5 read latency histogram (micros): Count: 77546 Average: 296.1132 StdDev: 636.87 Min: 90 Median: 208.2574 Max: 49248 Percentiles: P50: 208.26 P75: 247.38 P99: 2479.12 P99.9: 5700.91 P99.99: 33521.47 ------------------------------------------------------ ( 76, 110 ] 764 0.985% 0.985% ( 110, 170 ] 19051 24.567% 25.553% ##### ( 170, 250 ] 39643 51.122% 76.674% ########## ( 250, 380 ] 8411 10.846% 87.521% ## ( 380, 580 ] 4380 5.648% 93.169% # ( 580, 870 ] 2783 3.589% 96.758% # ( 870, 1300 ] 1163 1.500% 98.258% ( 1300, 1900 ] 352 0.454% 98.712% ( 1900, 2900 ] 386 0.498% 99.210% ( 2900, 4400 ] 471 0.607% 99.817% ( 4400, 6600 ] 109 0.141% 99.957% ( 6600, 9900 ] 13 0.017% 99.974% ( 9900, 14000 ] 3 0.004% 99.978% ( 14000, 22000 ] 4 0.005% 99.983% ( 22000, 33000 ] 5 0.006% 99.990% ( 33000, 50000 ] 8 0.010% 100.000% ** DB Stats ** Uptime(secs): 20.1 total, 7.4 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 146.71 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.713 H:M:S, 38.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 132.65 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.719 H:M:S, 50.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 355.258 micros/op 11197 ops/sec 20.183 seconds 225996 operations; 10.8 MB/s (55999 of 55999 found) Microseconds per read: Count: 225996 Average: 355.2645 StdDev: 978.88 Min: 2 Median: 234.5872 Max: 100981 Percentiles: P50: 234.59 P75: 350.08 P99: 2498.50 P99.9: 12618.16 P99.99: 40687.25 ------------------------------------------------------ ( 1, 2 ] 8 0.004% 0.004% ( 2, 3 ] 47 0.021% 0.024% ( 3, 4 ] 79 0.035% 0.059% ( 4, 6 ] 164 0.073% 0.132% ( 6, 10 ] 347 0.154% 0.285% ( 10, 15 ] 1891 0.837% 1.122% ( 15, 22 ] 2937 1.300% 2.422% ( 22, 34 ] 4913 2.174% 4.596% ( 34, 51 ] 1948 0.862% 5.458% ( 51, 76 ] 277 0.123% 5.580% ( 76, 110 ] 101 0.045% 5.625% ( 110, 170 ] 16493 7.298% 12.923% # ( 170, 250 ] 103789 45.925% 58.848% ######### ( 250, 380 ] 47415 20.980% 79.828% #### ( 380, 580 ] 26145 11.569% 91.397% ## ( 580, 870 ] 11096 4.910% 96.307% # ( 870, 1300 ] 4201 1.859% 98.166% ( 1300, 1900 ] 1344 0.595% 98.761% ( 1900, 2900 ] 904 0.400% 99.161% ( 2900, 4400 ] 1140 0.504% 99.665% ( 4400, 6600 ] 308 0.136% 99.801% ( 6600, 9900 ] 164 0.073% 99.874% ( 9900, 14000 ] 89 0.039% 99.913% ( 14000, 22000 ] 106 0.047% 99.960% ( 22000, 33000 ] 57 0.025% 99.985% ( 33000, 50000 ] 23 0.010% 99.996% ( 50000, 75000 ] 6 0.003% 99.998% ( 75000, 110000 ] 4 0.002% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 3961983 rocksdb.bloom.filter.full.positive COUNT : 263580 rocksdb.bloom.filter.full.true.positive COUNT : 225148 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 848 rocksdb.memtable.miss COUNT : 225148 rocksdb.l0.hit COUNT : 26077 rocksdb.l1.hit COUNT : 2744 rocksdb.l2andup.hit COUNT : 196327 rocksdb.compaction.key.drop.new COUNT : 666126 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3039818 rocksdb.number.keys.read COUNT : 225996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3137092176 rocksdb.bytes.read COUNT : 225996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 482 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 7712772 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 12058905675 rocksdb.compact.write.bytes COUNT : 11371965453 rocksdb.flush.write.bytes COUNT : 3033813046 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 23 rocksdb.number.superversion_acquires COUNT : 583 rocksdb.number.superversion_releases COUNT : 563 rocksdb.number.superversion_cleanups COUNT : 143 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3014069248 rocksdb.memtable.garbage.bytes.at.flush COUNT : 5501952 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 12488466893 rocksdb.non.last.level.read.count COUNT : 2989039 rocksdb.block.checksum.compute.count COUNT : 3189041 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 233.804566 P95 : 792.110085 P99 : 2497.834254 P100 : 100979.000000 COUNT : 225996 SUM : 80070003 rocksdb.db.write.micros P50 : 1.577551 P95 : 5.132115 P99 : 8.427265 P100 : 476667.000000 COUNT : 3039818 SUM : 16188281 rocksdb.compaction.times.micros P50 : 1585000.000000 P95 : 2890000.000000 P99 : 6735149.000000 P100 : 6735149.000000 COUNT : 42 SUM : 78804516 rocksdb.compaction.times.cpu_micros P50 : 1086666.666667 P95 : 1793149.000000 P99 : 1793149.000000 P100 : 1793149.000000 COUNT : 42 SUM : 48065324 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 1037.222222 P95 : 6094.000000 P99 : 10337.000000 P100 : 10337.000000 COUNT : 43 SUM : 79348 rocksdb.compaction.outfile.sync.micros P50 : 1677.777778 P95 : 8066.666667 P99 : 18800.000000 P100 : 27836.000000 COUNT : 140 SUM : 383980 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 232.857143 P95 : 1330.000000 P99 : 2930.000000 P100 : 4173.000000 COUNT : 98 SUM : 42887 rocksdb.table.open.io.micros P50 : 6069.879518 P95 : 17790.476190 P99 : 21462.857143 P100 : 23188.000000 COUNT : 482 SUM : 3197437 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.279584 P95 : 12.621591 P99 : 310.539200 P100 : 61973.000000 COUNT : 2921934 SUM : 58885915 rocksdb.read.block.get.micros P50 : 203.705279 P95 : 645.302340 P99 : 2026.844181 P100 : 61596.000000 COUNT : 266838 SUM : 71500598 rocksdb.write.raw.block.micros P50 : 0.538699 P95 : 1.467154 P99 : 4.372119 P100 : 88509.000000 COUNT : 3501031 SUM : 12271942 rocksdb.numfiles.in.singlecompaction P50 : 3.100000 P95 : 5.800000 P99 : 8.880000 P100 : 9.000000 COUNT : 56 SUM : 198 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.559938 P95 : 1129.138347 P99 : 2150.651163 P100 : 471978.000000 COUNT : 48933 SUM : 7716603 rocksdb.sst.read.micros P50 : 2.100942 P95 : 221.178632 P99 : 550.912201 P100 : 61966.000000 COUNT : 2989420 SUM : 118796659 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 225996 SUM : 225996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3039818 SUM : 3137092176 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 227333.333333 P95 : 652166.666667 P99 : 763908.000000 P100 : 763908.000000 COUNT : 43 SUM : 12422399 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 146 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0