[2024-11-29 18:38:25.471370] Starting SPDK v25.01-pre git sha1 35cd3e84d / DPDK 22.11.4 initialization... [2024-11-29 18:38:25.471465] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3097944 ] Set seed to 1732901905469028 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-29 18:38:25.561903] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-29 18:38:25.583117] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-29 18:38:25.583158] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3097944' to capture a snapshot of events at runtime. [2024-11-29 18:38:25.583168] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-29 18:38:25.583178] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-29 18:38:25.583186] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3097944 for offline analysis/debug. [2024-11-29 18:38:25.583753] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Fri Nov 29 18:38:29 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/29-18:38:37 ... thread 0: (1000000,1000000) ops and (130247.8,130247.8) ops/second in (7.677675,7.677675) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 8/6 1.72 GB 50.3 0.4 0.0 0.4 1.2 0.8 0.0 1.5 39.0 123.4 9.69 3.73 10 0.969 383K 534 0.0 0.0 L1 5/5 603.83 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 99.70 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 30/9 2.17 GB 1.7 2.4 1.6 0.8 2.4 1.6 0.0 1.5 100.1 99.1 24.98 11.61 9 2.776 1628K 265 0.0 0.0 L4 194/15 11.88 GB 1.1 3.6 2.5 1.1 3.6 2.5 0.0 1.4 106.7 105.8 34.86 17.26 25 1.394 1995K 404 0.0 0.0 L5 1100/0 90.43 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1338/35 106.89 GB 0.0 6.4 4.1 2.3 7.2 4.9 0.0 9.0 94.9 105.8 69.53 32.60 44 1.580 4008K 1203 0.0 0.0 Int 0/0 0.00 KB 0.0 6.4 4.1 2.3 7.2 4.9 0.0 9.0 94.9 105.8 69.53 32.60 44 1.580 4008K 1203 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.4 4.1 2.3 6.4 4.1 0.0 0.0 105.3 104.3 62.69 30.36 35 1.791 4008K 1203 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 119.7 6.84 2.24 9 0.760 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.9 total, 7.7 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.19 GB write, 934.85 MB/s write, 6.44 GB read, 838.27 MB/s read, 69.5 seconds Interval compaction: 7.19 GB write, 958.37 MB/s write, 6.44 GB read, 859.35 MB/s read, 69.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 26, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 26, interval: 26 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 334234 Average: 10.2567 StdDev: 159.30 Min: 1 Median: 1.9371 Max: 35958 Percentiles: P50: 1.94 P75: 2.96 P99: 9.79 P99.9: 2015.78 P99.99: 4003.09 ------------------------------------------------------ [ 0, 1 ] 9928 2.970% 2.970% # ( 1, 2 ] 167747 50.188% 53.159% ########## ( 2, 3 ] 76164 22.788% 75.946% ##### ( 3, 4 ] 34460 10.310% 86.257% ## ( 4, 6 ] 35309 10.564% 96.821% ## ( 6, 10 ] 7681 2.298% 99.119% ( 10, 15 ] 1251 0.374% 99.493% ( 15, 22 ] 147 0.044% 99.537% ( 22, 34 ] 33 0.010% 99.547% ( 34, 51 ] 16 0.005% 99.552% ( 51, 76 ] 26 0.008% 99.560% ( 76, 110 ] 11 0.003% 99.563% ( 110, 170 ] 23 0.007% 99.570% ( 170, 250 ] 59 0.018% 99.587% ( 250, 380 ] 88 0.026% 99.614% ( 380, 580 ] 174 0.052% 99.666% ( 580, 870 ] 293 0.088% 99.753% ( 870, 1300 ] 305 0.091% 99.845% ( 1300, 1900 ] 166 0.050% 99.894% ( 1900, 2900 ] 188 0.056% 99.951% ( 2900, 4400 ] 183 0.055% 100.005% ( 4400, 6600 ] 49 0.015% 100.020% ( 6600, 9900 ] 11 0.003% 100.023% ( 9900, 14000 ] 5 0.001% 100.025% ( 14000, 22000 ] 1 0.000% 100.025% ( 33000, 50000 ] 1 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 65501 Average: 14.9790 StdDev: 203.44 Min: 1 Median: 1.9735 Max: 16555 Percentiles: P50: 1.97 P75: 2.98 P99: 12.70 P99.9: 3384.34 P99.99: 8226.38 ------------------------------------------------------ [ 0, 1 ] 1200 1.832% 1.832% ( 1, 2 ] 32411 49.482% 51.314% ########## ( 2, 3 ] 15891 24.261% 75.574% ##### ( 3, 4 ] 6644 10.143% 85.718% ## ( 4, 6 ] 6825 10.420% 96.137% ## ( 6, 10 ] 1713 2.615% 98.753% # ( 10, 15 ] 300 0.458% 99.211% ( 15, 22 ] 49 0.075% 99.286% ( 22, 34 ] 3 0.005% 99.290% ( 34, 51 ] 5 0.008% 99.298% ( 51, 76 ] 3 0.005% 99.302% ( 76, 110 ] 5 0.008% 99.310% ( 110, 170 ] 3 0.005% 99.315% ( 170, 250 ] 6 0.009% 99.324% ( 250, 380 ] 28 0.043% 99.366% ( 380, 580 ] 46 0.070% 99.437% ( 580, 870 ] 79 0.121% 99.557% ( 870, 1300 ] 94 0.144% 99.701% ( 1300, 1900 ] 65 0.099% 99.800% ( 1900, 2900 ] 50 0.076% 99.876% ( 2900, 4400 ] 48 0.073% 99.950% ( 4400, 6600 ] 23 0.035% 99.985% ( 6600, 9900 ] 7 0.011% 99.995% ( 9900, 14000 ] 2 0.003% 99.998% ( 14000, 22000 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 358814 Average: 12.8684 StdDev: 159.09 Min: 1 Median: 2.0153 Max: 11487 Percentiles: P50: 2.02 P75: 3.11 P99: 11.89 P99.9: 1457.25 P99.99: 2083.52 ------------------------------------------------------ [ 0, 1 ] 8735 2.434% 2.434% ( 1, 2 ] 169345 47.196% 49.630% ######### ( 2, 3 ] 86659 24.152% 73.782% ##### ( 3, 4 ] 39021 10.875% 84.657% ## ( 4, 6 ] 40185 11.199% 95.856% ## ( 6, 10 ] 10546 2.939% 98.795% # ( 10, 15 ] 1945 0.542% 99.337% ( 15, 22 ] 365 0.102% 99.439% ( 22, 34 ] 62 0.017% 99.456% ( 34, 51 ] 22 0.006% 99.462% ( 51, 76 ] 18 0.005% 99.467% ( 76, 110 ] 5 0.001% 99.469% ( 110, 170 ] 46 0.013% 99.482% ( 170, 250 ] 90 0.025% 99.507% ( 250, 380 ] 158 0.044% 99.551% ( 380, 580 ] 274 0.076% 99.627% ( 580, 870 ] 399 0.111% 99.738% ( 870, 1300 ] 484 0.135% 99.873% ( 1300, 1900 ] 367 0.102% 99.975% ( 1900, 2900 ] 284 0.079% 100.055% ( 2900, 4400 ] 222 0.062% 100.116% ( 4400, 6600 ] 75 0.021% 100.137% ( 6600, 9900 ] 13 0.004% 100.141% ( 9900, 14000 ] 2 0.001% 100.142% ** Level 3 read latency histogram (micros): Count: 699674 Average: 11.5039 StdDev: 189.05 Min: 1 Median: 1.9832 Max: 64673 Percentiles: P50: 1.98 P75: 2.99 P99: 9.78 P99.9: 1144.90 P99.99: 1684.39 ------------------------------------------------------ [ 0, 1 ] 12561 1.795% 1.795% ( 1, 2 ] 343049 49.030% 50.825% ########## ( 2, 3 ] 170337 24.345% 75.170% ##### ( 3, 4 ] 75076 10.730% 85.900% ## ( 4, 6 ] 76259 10.899% 96.800% ## ( 6, 10 ] 16299 2.330% 99.129% ( 10, 15 ] 2928 0.418% 99.548% ( 15, 22 ] 379 0.054% 99.602% ( 22, 34 ] 31 0.004% 99.606% ( 34, 51 ] 35 0.005% 99.611% ( 51, 76 ] 53 0.008% 99.619% ( 76, 110 ] 25 0.004% 99.622% ( 110, 170 ] 50 0.007% 99.630% ( 170, 250 ] 98 0.014% 99.644% ( 250, 380 ] 208 0.030% 99.673% ( 380, 580 ] 434 0.062% 99.735% ( 580, 870 ] 647 0.092% 99.828% ( 870, 1300 ] 792 0.113% 99.941% ( 1300, 1900 ] 537 0.077% 100.018% ( 1900, 2900 ] 421 0.060% 100.078% ( 2900, 4400 ] 466 0.067% 100.144% ( 4400, 6600 ] 97 0.014% 100.158% ( 6600, 9900 ] 14 0.002% 100.160% ( 9900, 14000 ] 3 0.000% 100.161% ( 14000, 22000 ] 3 0.000% 100.161% ( 22000, 33000 ] 3 0.000% 100.162% ( 33000, 50000 ] 2 0.000% 100.162% ( 50000, 75000 ] 1 0.000% 100.162% ** Level 4 read latency histogram (micros): Count: 359601 Average: 26.1814 StdDev: 288.28 Min: 1 Median: 1.9996 Max: 40337 Percentiles: P50: 2.00 P75: 3.08 P99: 634.23 P99.9: 3184.05 P99.99: 4016.74 ------------------------------------------------------ [ 0, 1 ] 6734 1.873% 1.873% ( 1, 2 ] 173138 48.147% 50.020% ########## ( 2, 3 ] 86634 24.092% 74.112% ##### ( 3, 4 ] 38720 10.767% 84.879% ## ( 4, 6 ] 38352 10.665% 95.544% ## ( 6, 10 ] 9459 2.630% 98.175% # ( 10, 15 ] 1510 0.420% 98.595% ( 15, 22 ] 195 0.054% 98.649% ( 22, 34 ] 21 0.006% 98.655% ( 34, 51 ] 18 0.005% 98.660% ( 51, 76 ] 37 0.010% 98.670% ( 76, 110 ] 8 0.002% 98.672% ( 110, 170 ] 40 0.011% 98.683% ( 170, 250 ] 152 0.042% 98.726% ( 250, 380 ] 249 0.069% 98.795% ( 380, 580 ] 571 0.159% 98.954% ( 580, 870 ] 893 0.248% 99.202% ( 870, 1300 ] 1035 0.288% 99.490% ( 1300, 1900 ] 762 0.212% 99.702% ( 1900, 2900 ] 603 0.168% 99.869% ( 2900, 4400 ] 583 0.162% 100.031% ( 4400, 6600 ] 172 0.048% 100.079% ( 6600, 9900 ] 40 0.011% 100.090% ( 9900, 14000 ] 4 0.001% 100.091% ( 14000, 22000 ] 7 0.002% 100.093% ( 22000, 33000 ] 1 0.000% 100.094% ( 33000, 50000 ] 4 0.001% 100.095% ** Level 5 read latency histogram (micros): Count: 12968 Average: 1713.7728 StdDev: 1636.36 Min: 86 Median: 1173.7988 Max: 44096 Percentiles: P50: 1173.80 P75: 2372.35 P99: 6576.64 P99.9: 13559.02 P99.99: 36984.80 ------------------------------------------------------ ( 76, 110 ] 7 0.054% 0.054% ( 110, 170 ] 74 0.571% 0.625% ( 170, 250 ] 185 1.427% 2.051% ( 250, 380 ] 630 4.858% 6.909% # ( 380, 580 ] 1421 10.958% 17.867% ## ( 580, 870 ] 2377 18.330% 36.197% #### ( 870, 1300 ] 2535 19.548% 55.745% #### ( 1300, 1900 ] 1789 13.795% 69.540% ### ( 1900, 2900 ] 1501 11.575% 81.115% ## ( 2900, 4400 ] 1698 13.094% 94.209% ### ( 4400, 6600 ] 629 4.850% 99.059% # ( 6600, 9900 ] 102 0.787% 99.846% ( 9900, 14000 ] 9 0.069% 99.915% ( 14000, 22000 ] 7 0.054% 99.969% ( 22000, 33000 ] 3 0.023% 99.992% ( 33000, 50000 ] 3 0.023% 100.015% ** DB Stats ** Uptime(secs): 7.9 total, 7.7 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 125.01 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.749 H:M:S, 47.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 128.14 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.749 H:M:S, 48.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/11/29-18:38:42 ... thread 0: (1000000,2000000) ops and (188013.8,153888.4) ops/second in (5.318759,12.996434) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/6 2.76 GB 163.6 1.6 0.0 1.6 3.4 1.8 0.0 1.9 76.4 164.5 21.43 11.78 29 0.739 1662K 1992 0.0 0.0 L1 5/5 603.83 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 99.70 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 12/0 971.52 MB 1.0 2.4 1.6 0.8 2.4 1.6 0.0 1.5 100.1 99.1 24.98 11.61 9 2.776 1628K 265 0.0 0.0 L4 172/0 9.48 GB 1.0 7.1 3.7 3.4 7.0 3.6 0.1 1.9 115.5 113.9 62.55 33.24 41 1.525 2855K 404 0.0 0.0 L5 1144/0 94.00 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1345/11 107.87 GB 0.0 11.1 5.3 5.8 12.8 7.0 3.6 6.9 104.3 120.5 108.96 56.63 79 1.379 6147K 2661 0.0 0.0 Int 0/0 0.00 KB 0.0 11.1 5.3 5.8 12.8 7.0 3.6 6.9 104.3 120.5 108.96 56.63 79 1.379 6147K 2661 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.1 5.3 5.8 11.0 5.2 0.0 0.0 117.5 116.2 96.69 51.30 55 1.758 6147K 2661 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 154.1 12.27 5.33 24 0.511 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 13.2 total, 13.0 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.82 GB write, 995.06 MB/s write, 11.10 GB read, 861.37 MB/s read, 109.0 seconds Interval compaction: 12.82 GB write, 1009.85 MB/s write, 11.10 GB read, 874.17 MB/s read, 109.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 34, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 34, interval: 34 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 661842 Average: 11.3318 StdDev: 143.59 Min: 1 Median: 1.8062 Max: 35958 Percentiles: P50: 1.81 P75: 2.74 P99: 136.30 P99.9: 1677.02 P99.99: 3658.73 ------------------------------------------------------ [ 0, 1 ] 45604 6.890% 6.890% # ( 1, 2 ] 353890 53.470% 60.361% ########### ( 2, 3 ] 131335 19.844% 80.205% #### ( 3, 4 ] 54951 8.303% 88.508% ## ( 4, 6 ] 53991 8.158% 96.665% ## ( 6, 10 ] 11960 1.807% 98.472% ( 10, 15 ] 1961 0.296% 98.769% ( 15, 22 ] 230 0.035% 98.803% ( 22, 34 ] 37 0.006% 98.809% ( 34, 51 ] 26 0.004% 98.813% ( 51, 76 ] 44 0.007% 98.820% ( 76, 110 ] 430 0.065% 98.884% ( 110, 170 ] 1744 0.264% 99.148% ( 170, 250 ] 1253 0.189% 99.337% ( 250, 380 ] 841 0.127% 99.464% ( 380, 580 ] 1017 0.154% 99.618% ( 580, 870 ] 931 0.141% 99.759% ( 870, 1300 ] 719 0.109% 99.867% ( 1300, 1900 ] 344 0.052% 99.919% ( 1900, 2900 ] 310 0.047% 99.966% ( 2900, 4400 ] 312 0.047% 100.013% ( 4400, 6600 ] 81 0.012% 100.026% ( 6600, 9900 ] 14 0.002% 100.028% ( 9900, 14000 ] 5 0.001% 100.028% ( 14000, 22000 ] 2 0.000% 100.029% ( 33000, 50000 ] 1 0.000% 100.029% ** Level 1 read latency histogram (micros): Count: 153772 Average: 11.0365 StdDev: 158.01 Min: 1 Median: 2.0845 Max: 16555 Percentiles: P50: 2.08 P75: 3.07 P99: 13.26 P99.9: 2349.03 P99.99: 6390.81 ------------------------------------------------------ [ 0, 1 ] 2441 1.587% 1.587% ( 1, 2 ] 71002 46.174% 47.761% ######### ( 2, 3 ] 40731 26.488% 74.249% ##### ( 3, 4 ] 15649 10.177% 84.426% ## ( 4, 6 ] 17221 11.199% 95.625% ## ( 6, 10 ] 4661 3.031% 98.656% # ( 10, 15 ] 811 0.527% 99.183% ( 15, 22 ] 193 0.126% 99.309% ( 22, 34 ] 16 0.010% 99.319% ( 34, 51 ] 6 0.004% 99.323% ( 51, 76 ] 13 0.008% 99.331% ( 76, 110 ] 42 0.027% 99.359% ( 110, 170 ] 134 0.087% 99.446% ( 170, 250 ] 78 0.051% 99.497% ( 250, 380 ] 81 0.053% 99.549% ( 380, 580 ] 114 0.074% 99.623% ( 580, 870 ] 147 0.096% 99.719% ( 870, 1300 ] 149 0.097% 99.816% ( 1300, 1900 ] 97 0.063% 99.879% ( 1900, 2900 ] 74 0.048% 99.927% ( 2900, 4400 ] 75 0.049% 99.976% ( 4400, 6600 ] 25 0.016% 99.992% ( 6600, 9900 ] 8 0.005% 99.997% ( 9900, 14000 ] 3 0.002% 99.999% ( 14000, 22000 ] 2 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 358865 Average: 12.9141 StdDev: 159.18 Min: 1 Median: 2.0156 Max: 11487 Percentiles: P50: 2.02 P75: 3.11 P99: 12.02 P99.9: 1458.80 P99.99: 2086.36 ------------------------------------------------------ [ 0, 1 ] 8735 2.434% 2.434% ( 1, 2 ] 169345 47.189% 49.623% ######### ( 2, 3 ] 86659 24.148% 73.771% ##### ( 3, 4 ] 39021 10.873% 84.645% ## ( 4, 6 ] 40185 11.198% 95.842% ## ( 6, 10 ] 10546 2.939% 98.781% # ( 10, 15 ] 1945 0.542% 99.323% ( 15, 22 ] 365 0.102% 99.425% ( 22, 34 ] 62 0.017% 99.442% ( 34, 51 ] 22 0.006% 99.448% ( 51, 76 ] 18 0.005% 99.453% ( 76, 110 ] 8 0.002% 99.456% ( 110, 170 ] 60 0.017% 99.472% ( 170, 250 ] 107 0.030% 99.502% ( 250, 380 ] 161 0.045% 99.547% ( 380, 580 ] 280 0.078% 99.625% ( 580, 870 ] 405 0.113% 99.738% ( 870, 1300 ] 485 0.135% 99.873% ( 1300, 1900 ] 367 0.102% 99.975% ( 1900, 2900 ] 285 0.079% 100.055% ( 2900, 4400 ] 222 0.062% 100.116% ( 4400, 6600 ] 75 0.021% 100.137% ( 6600, 9900 ] 13 0.004% 100.141% ( 9900, 14000 ] 2 0.001% 100.142% ** Level 3 read latency histogram (micros): Count: 965024 Average: 10.5163 StdDev: 169.86 Min: 1 Median: 1.9757 Max: 64673 Percentiles: P50: 1.98 P75: 2.98 P99: 9.78 P99.9: 1067.82 P99.99: 1610.92 ------------------------------------------------------ [ 0, 1 ] 17672 1.831% 1.831% ( 1, 2 ] 476417 49.368% 51.200% ########## ( 2, 3 ] 233382 24.184% 75.384% ##### ( 3, 4 ] 101495 10.517% 85.901% ## ( 4, 6 ] 104066 10.784% 96.685% ## ( 6, 10 ] 23620 2.448% 99.132% ( 10, 15 ] 3961 0.410% 99.543% ( 15, 22 ] 525 0.054% 99.597% ( 22, 34 ] 43 0.004% 99.602% ( 34, 51 ] 57 0.006% 99.608% ( 51, 76 ] 81 0.008% 99.616% ( 76, 110 ] 71 0.007% 99.623% ( 110, 170 ] 187 0.019% 99.643% ( 170, 250 ] 215 0.022% 99.665% ( 250, 380 ] 301 0.031% 99.696% ( 380, 580 ] 609 0.063% 99.759% ( 580, 870 ] 896 0.093% 99.852% ( 870, 1300 ] 1002 0.104% 99.956% ( 1300, 1900 ] 632 0.065% 100.022% ( 1900, 2900 ] 512 0.053% 100.075% ( 2900, 4400 ] 541 0.056% 100.131% ( 4400, 6600 ] 116 0.012% 100.143% ( 6600, 9900 ] 18 0.002% 100.145% ( 9900, 14000 ] 3 0.000% 100.145% ( 14000, 22000 ] 3 0.000% 100.145% ( 22000, 33000 ] 3 0.000% 100.145% ( 33000, 50000 ] 2 0.000% 100.146% ( 50000, 75000 ] 1 0.000% 100.146% ** Level 4 read latency histogram (micros): Count: 886346 Average: 18.9320 StdDev: 217.38 Min: 0 Median: 1.9776 Max: 40337 Percentiles: P50: 1.98 P75: 3.00 P99: 207.03 P99.9: 1847.14 P99.99: 2647.43 ------------------------------------------------------ [ 0, 1 ] 15291 1.725% 1.725% ( 1, 2 ] 437666 49.379% 51.104% ########## ( 2, 3 ] 212840 24.013% 75.117% ##### ( 3, 4 ] 90621 10.224% 85.341% ## ( 4, 6 ] 91387 10.311% 95.652% ## ( 6, 10 ] 23229 2.621% 98.272% # ( 10, 15 ] 3739 0.422% 98.694% ( 15, 22 ] 486 0.055% 98.749% ( 22, 34 ] 55 0.006% 98.755% ( 34, 51 ] 59 0.007% 98.762% ( 51, 76 ] 89 0.010% 98.772% ( 76, 110 ] 317 0.036% 98.808% ( 110, 170 ] 1217 0.137% 98.945% ( 170, 250 ] 1051 0.119% 99.064% ( 250, 380 ] 1010 0.114% 99.178% ( 380, 580 ] 1541 0.174% 99.351% ( 580, 870 ] 1931 0.218% 99.569% ( 870, 1300 ] 1861 0.210% 99.779% ( 1300, 1900 ] 1173 0.132% 99.912% ( 1900, 2900 ] 929 0.105% 100.016% ( 2900, 4400 ] 928 0.105% 100.121% ( 4400, 6600 ] 257 0.029% 100.150% ( 6600, 9900 ] 50 0.006% 100.156% ( 9900, 14000 ] 7 0.001% 100.157% ( 14000, 22000 ] 7 0.001% 100.157% ( 22000, 33000 ] 1 0.000% 100.158% ( 33000, 50000 ] 4 0.000% 100.158% ** Level 5 read latency histogram (micros): Count: 43885 Average: 856.8173 StdDev: 1197.98 Min: 75 Median: 440.2562 Max: 44096 Percentiles: P50: 440.26 P75: 1011.93 P99: 5718.11 P99.9: 9260.88 P99.99: 20611.50 ------------------------------------------------------ ( 51, 76 ] 2 0.005% 0.005% ( 76, 110 ] 2257 5.143% 5.148% # ( 110, 170 ] 8061 18.368% 23.516% #### ( 170, 250 ] 5598 12.756% 36.272% ### ( 250, 380 ] 4378 9.976% 46.248% ## ( 380, 580 ] 5465 12.453% 58.701% ## ( 580, 870 ] 5612 12.788% 71.489% ### ( 870, 1300 ] 4668 10.637% 82.126% ## ( 1300, 1900 ] 2694 6.139% 88.265% # ( 1900, 2900 ] 2025 4.614% 92.879% # ( 2900, 4400 ] 2232 5.086% 97.965% # ( 4400, 6600 ] 758 1.727% 99.692% ( 6600, 9900 ] 113 0.257% 99.950% ( 9900, 14000 ] 11 0.025% 99.975% ( 14000, 22000 ] 8 0.018% 99.993% ( 22000, 33000 ] 3 0.007% 100.000% ( 33000, 50000 ] 3 0.007% 100.007% ** DB Stats ** Uptime(secs): 13.2 total, 5.3 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 149.22 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.349 H:M:S, 40.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 185.05 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.600 H:M:S, 30.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/29-18:38:46 ... thread 0: (1000000,3000000) ops and (291255.4,182594.5) ops/second in (3.433413,16.429847) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/8 2.83 GB 239.1 2.3 0.0 2.3 5.2 2.8 0.0 1.8 87.3 192.9 27.41 17.28 48 0.571 2430K 2752 0.0 0.0 L1 17/0 1.50 GB 153.8 1.5 0.9 0.6 1.5 0.9 0.0 1.6 119.2 118.7 12.96 7.43 1 12.955 1568K 6406 0.0 0.0 L2 1/0 99.70 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 12/0 971.52 MB 1.0 2.4 1.6 0.8 2.4 1.6 0.0 1.5 100.1 99.1 24.98 11.61 9 2.776 1628K 265 0.0 0.0 L4 172/0 9.48 GB 1.0 7.1 3.7 3.4 7.0 3.6 0.1 1.9 115.5 113.9 62.55 33.24 41 1.525 2855K 404 0.0 0.0 L5 1144/0 94.00 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1362/8 108.85 GB 0.0 13.3 6.2 7.1 16.0 8.9 3.6 5.7 106.8 128.4 127.90 69.56 99 1.292 8483K 9827 0.0 0.0 Int 0/0 0.00 KB 0.0 13.3 6.2 7.1 16.0 8.9 3.6 5.7 106.8 128.4 127.90 69.56 99 1.292 8483K 9827 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.3 6.2 7.1 13.2 6.1 0.0 0.0 121.5 120.3 112.47 61.49 59 1.906 8483K 9827 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 187.9 15.42 8.07 40 0.386 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.6 total, 16.4 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.04 GB write, 988.10 MB/s write, 13.34 GB read, 821.82 MB/s read, 127.9 seconds Interval compaction: 16.04 GB write, 999.71 MB/s write, 13.34 GB read, 831.48 MB/s read, 127.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 34, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 34, interval: 34 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 914867 Average: 9.4291 StdDev: 123.20 Min: 0 Median: 1.6648 Max: 35958 Percentiles: P50: 1.66 P75: 2.42 P99: 128.15 P99.9: 1250.42 P99.99: 3284.62 ------------------------------------------------------ [ 0, 1 ] 126593 13.837% 13.837% ### ( 1, 2 ] 497684 54.400% 68.237% ########### ( 2, 3 ] 147223 16.092% 84.329% ### ( 3, 4 ] 60317 6.593% 90.922% # ( 4, 6 ] 57124 6.244% 97.166% # ( 6, 10 ] 12572 1.374% 98.540% ( 10, 15 ] 2169 0.237% 98.777% ( 15, 22 ] 247 0.027% 98.804% ( 22, 34 ] 42 0.005% 98.809% ( 34, 51 ] 29 0.003% 98.812% ( 51, 76 ] 60 0.007% 98.819% ( 76, 110 ] 731 0.080% 98.899% ( 110, 170 ] 3065 0.335% 99.234% ( 170, 250 ] 2029 0.222% 99.455% ( 250, 380 ] 1167 0.128% 99.583% ( 380, 580 ] 1227 0.134% 99.717% ( 580, 870 ] 1022 0.112% 99.829% ( 870, 1300 ] 736 0.080% 99.909% ( 1300, 1900 ] 347 0.038% 99.947% ( 1900, 2900 ] 311 0.034% 99.981% ( 2900, 4400 ] 314 0.034% 100.016% ( 4400, 6600 ] 81 0.009% 100.024% ( 6600, 9900 ] 14 0.002% 100.026% ( 9900, 14000 ] 5 0.001% 100.026% ( 14000, 22000 ] 2 0.000% 100.027% ( 33000, 50000 ] 1 0.000% 100.027% ** Level 1 read latency histogram (micros): Count: 155113 Average: 12.3841 StdDev: 158.88 Min: 1 Median: 2.1005 Max: 16555 Percentiles: P50: 2.10 P75: 3.13 P99: 141.19 P99.9: 2338.49 P99.99: 6379.01 ------------------------------------------------------ [ 0, 1 ] 2441 1.574% 1.574% ( 1, 2 ] 71017 45.784% 47.358% ######### ( 2, 3 ] 40800 26.303% 73.661% ##### ( 3, 4 ] 15737 10.146% 83.807% ## ( 4, 6 ] 17309 11.159% 94.966% ## ( 6, 10 ] 4673 3.013% 97.978% # ( 10, 15 ] 817 0.527% 98.505% ( 15, 22 ] 194 0.125% 98.630% ( 22, 34 ] 16 0.010% 98.640% ( 34, 51 ] 6 0.004% 98.644% ( 51, 76 ] 13 0.008% 98.653% ( 76, 110 ] 215 0.139% 98.791% ( 110, 170 ] 623 0.402% 99.193% ( 170, 250 ] 275 0.177% 99.370% ( 250, 380 ] 166 0.107% 99.477% ( 380, 580 ] 195 0.126% 99.603% ( 580, 870 ] 170 0.110% 99.712% ( 870, 1300 ] 157 0.101% 99.814% ( 1300, 1900 ] 101 0.065% 99.879% ( 1900, 2900 ] 75 0.048% 99.927% ( 2900, 4400 ] 75 0.048% 99.976% ( 4400, 6600 ] 25 0.016% 99.992% ( 6600, 9900 ] 8 0.005% 99.997% ( 9900, 14000 ] 3 0.002% 99.999% ( 14000, 22000 ] 2 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 358938 Average: 12.9549 StdDev: 159.20 Min: 1 Median: 2.0160 Max: 11487 Percentiles: P50: 2.02 P75: 3.11 P99: 12.20 P99.9: 1458.68 P99.99: 2086.34 ------------------------------------------------------ [ 0, 1 ] 8735 2.434% 2.434% ( 1, 2 ] 169345 47.179% 49.613% ######### ( 2, 3 ] 86659 24.143% 73.756% ##### ( 3, 4 ] 39021 10.871% 84.627% ## ( 4, 6 ] 40185 11.196% 95.823% ## ( 6, 10 ] 10546 2.938% 98.761% # ( 10, 15 ] 1945 0.542% 99.303% ( 15, 22 ] 365 0.102% 99.405% ( 22, 34 ] 62 0.017% 99.422% ( 34, 51 ] 22 0.006% 99.428% ( 51, 76 ] 18 0.005% 99.433% ( 76, 110 ] 22 0.006% 99.439% ( 110, 170 ] 87 0.024% 99.463% ( 170, 250 ] 122 0.034% 99.497% ( 250, 380 ] 170 0.047% 99.545% ( 380, 580 ] 284 0.079% 99.624% ( 580, 870 ] 408 0.114% 99.738% ( 870, 1300 ] 486 0.135% 99.873% ( 1300, 1900 ] 367 0.102% 99.975% ( 1900, 2900 ] 285 0.079% 100.055% ( 2900, 4400 ] 222 0.062% 100.116% ( 4400, 6600 ] 75 0.021% 100.137% ( 6600, 9900 ] 13 0.004% 100.141% ( 9900, 14000 ] 2 0.001% 100.142% ** Level 3 read latency histogram (micros): Count: 965720 Average: 10.6600 StdDev: 169.93 Min: 1 Median: 1.9764 Max: 64673 Percentiles: P50: 1.98 P75: 2.99 P99: 9.90 P99.9: 1068.45 P99.99: 1610.85 ------------------------------------------------------ [ 0, 1 ] 17672 1.830% 1.830% ( 1, 2 ] 476417 49.333% 51.163% ########## ( 2, 3 ] 233382 24.167% 75.329% ##### ( 3, 4 ] 101495 10.510% 85.839% ## ( 4, 6 ] 104066 10.776% 96.615% ## ( 6, 10 ] 23620 2.446% 99.061% ( 10, 15 ] 3961 0.410% 99.471% ( 15, 22 ] 525 0.054% 99.526% ( 22, 34 ] 43 0.004% 99.530% ( 34, 51 ] 57 0.006% 99.536% ( 51, 76 ] 81 0.008% 99.544% ( 76, 110 ] 166 0.017% 99.561% ( 110, 170 ] 509 0.053% 99.614% ( 170, 250 ] 350 0.036% 99.650% ( 250, 380 ] 355 0.037% 99.687% ( 380, 580 ] 674 0.070% 99.757% ( 580, 870 ] 917 0.095% 99.852% ( 870, 1300 ] 1006 0.104% 99.956% ( 1300, 1900 ] 632 0.065% 100.022% ( 1900, 2900 ] 512 0.053% 100.075% ( 2900, 4400 ] 541 0.056% 100.131% ( 4400, 6600 ] 116 0.012% 100.143% ( 6600, 9900 ] 18 0.002% 100.144% ( 9900, 14000 ] 3 0.000% 100.145% ( 14000, 22000 ] 3 0.000% 100.145% ( 22000, 33000 ] 3 0.000% 100.145% ( 33000, 50000 ] 2 0.000% 100.146% ( 50000, 75000 ] 1 0.000% 100.146% ** Level 4 read latency histogram (micros): Count: 893394 Average: 20.2061 StdDev: 217.45 Min: 0 Median: 1.9857 Max: 40337 Percentiles: P50: 1.99 P75: 3.04 P99: 291.08 P99.9: 1846.36 P99.99: 2648.83 ------------------------------------------------------ [ 0, 1 ] 15291 1.712% 1.712% ( 1, 2 ] 437683 48.991% 50.703% ########## ( 2, 3 ] 212996 23.841% 74.544% ##### ( 3, 4 ] 90794 10.163% 84.707% ## ( 4, 6 ] 91652 10.259% 94.965% ## ( 6, 10 ] 23272 2.605% 97.570% # ( 10, 15 ] 3749 0.420% 97.990% ( 15, 22 ] 488 0.055% 98.045% ( 22, 34 ] 55 0.006% 98.051% ( 34, 51 ] 59 0.007% 98.057% ( 51, 76 ] 89 0.010% 98.067% ( 76, 110 ] 1362 0.152% 98.220% ( 110, 170 ] 4236 0.474% 98.694% ( 170, 250 ] 2248 0.252% 98.946% ( 250, 380 ] 1538 0.172% 99.118% ( 380, 580 ] 1934 0.216% 99.334% ( 580, 870 ] 2094 0.234% 99.569% ( 870, 1300 ] 1887 0.211% 99.780% ( 1300, 1900 ] 1179 0.132% 99.912% ( 1900, 2900 ] 933 0.104% 100.016% ( 2900, 4400 ] 929 0.104% 100.120% ( 4400, 6600 ] 257 0.029% 100.149% ( 6600, 9900 ] 50 0.006% 100.155% ( 9900, 14000 ] 7 0.001% 100.155% ( 14000, 22000 ] 7 0.001% 100.156% ( 22000, 33000 ] 1 0.000% 100.156% ( 33000, 50000 ] 4 0.000% 100.157% ** Level 5 read latency histogram (micros): Count: 87538 Average: 529.8600 StdDev: 915.86 Min: 74 Median: 203.9695 Max: 44096 Percentiles: P50: 203.97 P75: 523.83 P99: 4448.24 P99.9: 7956.85 P99.99: 15246.20 ------------------------------------------------------ ( 51, 76 ] 5 0.006% 0.006% ( 76, 110 ] 9302 10.626% 10.632% ## ( 110, 170 ] 28692 32.777% 43.409% ####### ( 170, 250 ] 13591 15.526% 58.934% ### ( 250, 380 ] 7900 9.025% 67.959% ## ( 380, 580 ] 8572 9.792% 77.751% ## ( 580, 870 ] 6679 7.630% 85.381% ## ( 870, 1300 ] 4904 5.602% 90.983% # ( 1300, 1900 ] 2723 3.111% 94.094% # ( 1900, 2900 ] 2041 2.332% 96.426% ( 2900, 4400 ] 2238 2.557% 98.982% # ( 4400, 6600 ] 758 0.866% 99.848% ( 6600, 9900 ] 113 0.129% 99.977% ( 9900, 14000 ] 11 0.013% 99.990% ( 14000, 22000 ] 8 0.009% 99.999% ( 22000, 33000 ] 3 0.003% 100.002% ( 33000, 50000 ] 3 0.003% 100.006% ** DB Stats ** Uptime(secs): 16.6 total, 3.4 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 177.60 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.349 H:M:S, 32.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 286.65 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 220.341 micros/op 18132 ops/sec 60.168 seconds 1090996 operations; 17.6 MB/s (275999 of 275999 found) Microseconds per read: Count: 1090996 Average: 220.3420 StdDev: 651.21 Min: 2 Median: 161.0403 Max: 70954 Percentiles: P50: 161.04 P75: 217.83 P99: 1066.27 P99.9: 10266.11 P99.99: 28311.38 ------------------------------------------------------ ( 1, 2 ] 18 0.002% 0.002% ( 2, 3 ] 71 0.007% 0.008% ( 3, 4 ] 77 0.007% 0.015% ( 4, 6 ] 67 0.006% 0.021% ( 6, 10 ] 480 0.044% 0.065% ( 10, 15 ] 5351 0.490% 0.556% ( 15, 22 ] 6389 0.586% 1.141% ( 22, 34 ] 8148 0.747% 1.888% ( 34, 51 ] 3187 0.292% 2.180% ( 51, 76 ] 435 0.040% 2.220% ( 76, 110 ] 40294 3.693% 5.914% # ( 110, 170 ] 565413 51.825% 57.739% ########## ( 170, 250 ] 315010 28.874% 86.613% ###### ( 250, 380 ] 90677 8.311% 94.924% ## ( 380, 580 ] 29916 2.742% 97.666% # ( 580, 870 ] 11960 1.096% 98.762% ( 870, 1300 ] 5681 0.521% 99.283% ( 1300, 1900 ] 2696 0.247% 99.530% ( 1900, 2900 ] 1784 0.164% 99.694% ( 2900, 4400 ] 1510 0.138% 99.832% ( 4400, 6600 ] 486 0.045% 99.877% ( 6600, 9900 ] 215 0.020% 99.896% ( 9900, 14000 ] 448 0.041% 99.937% ( 14000, 22000 ] 493 0.045% 99.983% ( 22000, 33000 ] 141 0.013% 99.996% ( 33000, 50000 ] 40 0.004% 99.999% ( 50000, 75000 ] 9 0.001% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 19567216 rocksdb.bloom.filter.full.positive COUNT : 1281977 rocksdb.bloom.filter.full.true.positive COUNT : 1090554 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 442 rocksdb.memtable.miss COUNT : 1090554 rocksdb.l0.hit COUNT : 38802 rocksdb.l1.hit COUNT : 15882 rocksdb.l2andup.hit COUNT : 1035870 rocksdb.compaction.key.drop.new COUNT : 184115 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3859014 rocksdb.number.keys.read COUNT : 1090996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3982502448 rocksdb.bytes.read COUNT : 1090996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1575 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 45123094 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 17572989468 rocksdb.compact.write.bytes COUNT : 17387790724 rocksdb.flush.write.bytes COUNT : 3963455919 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 24 rocksdb.number.superversion_acquires COUNT : 745 rocksdb.number.superversion_releases COUNT : 720 rocksdb.number.superversion_cleanups COUNT : 189 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 211 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3931496448 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1528832 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 22502104670 rocksdb.non.last.level.read.count COUNT : 5295043 rocksdb.block.checksum.compute.count COUNT : 5552572 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 160.340975 P95 : 383.562917 P99 : 1065.063982 P100 : 70950.000000 COUNT : 1090996 SUM : 239456058 rocksdb.db.write.micros P50 : 1.576502 P95 : 4.836171 P99 : 12.991393 P100 : 891721.000000 COUNT : 3859014 SUM : 55150821 rocksdb.compaction.times.micros P50 : 1550000.000000 P95 : 3547500.000000 P99 : 11354000.000000 P100 : 12955147.000000 COUNT : 63 SUM : 116342438 rocksdb.compaction.times.cpu_micros P50 : 902500.000000 P95 : 1832916.666667 P99 : 7432475.000000 P100 : 7432475.000000 COUNT : 63 SUM : 65247650 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 806.956522 P95 : 97750.000000 P99 : 430191.000000 P100 : 430191.000000 COUNT : 54 SUM : 1272150 rocksdb.compaction.outfile.sync.micros P50 : 53240.740741 P95 : 242285.714286 P99 : 552900.000000 P100 : 952355.000000 COUNT : 227 SUM : 18482964 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 363.750000 P95 : 4572.857143 P99 : 5576.000000 P100 : 5576.000000 COUNT : 129 SUM : 136565 rocksdb.table.open.io.micros P50 : 9201.644737 P95 : 20173.160173 P99 : 26336.538462 P100 : 28049.000000 COUNT : 1575 SUM : 14737513 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.281265 P95 : 13.002322 P99 : 171.346017 P100 : 64680.000000 COUNT : 4257162 SUM : 70740773 rocksdb.read.block.get.micros P50 : 132.661888 P95 : 250.641701 P99 : 989.074019 P100 : 44101.000000 COUNT : 1293853 SUM : 203994628 rocksdb.write.raw.block.micros P50 : 0.557494 P95 : 1.657905 P99 : 4.029422 P100 : 60237.000000 COUNT : 5171740 SUM : 16572710 rocksdb.numfiles.in.singlecompaction P50 : 2.913043 P95 : 5.680000 P99 : 24.880000 P100 : 32.000000 COUNT : 76 SUM : 288 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.601810 P95 : 2235.829224 P99 : 2767.996366 P100 : 891549.000000 COUNT : 153783 SUM : 45139262 rocksdb.sst.read.micros P50 : 2.676694 P95 : 165.153110 P99 : 447.707116 P100 : 64673.000000 COUNT : 5295109 SUM : 258281163 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1090996 SUM : 1090996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3859014 SUM : 3982502448 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 226842.105263 P95 : 1016400.000000 P99 : 1142963.000000 P100 : 1142963.000000 COUNT : 54 SUM : 18161178 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 217 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0