[2024-07-15 15:17:15.307476] Starting SPDK v24.09-pre git sha1 248c547d0 / DPDK 24.03.0 initialization... [2024-07-15 15:17:15.307639] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2059191 ] Set seed to 1721049435303863 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-15 15:17:15.440713] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-15 15:17:15.659676] app.c: 603:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-15 15:17:15.659727] app.c: 604:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2059191' to capture a snapshot of events at runtime. [2024-07-15 15:17:15.659740] app.c: 609:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-15 15:17:15.659753] app.c: 610:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-15 15:17:15.659763] app.c: 611:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2059191 for offline analysis/debug. [2024-07-15 15:17:15.659861] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Jul 15 15:17:19 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/07/15-15:17:25 ... thread 0: (1000000,1000000) ops and (157549.7,157549.7) ops/second in (6.347204,6.347204) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/13 1.25 GB 18.9 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 141.4 5.78 2.52 11 0.526 0 0 0.0 0.0 L1 5/5 336.00 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 25/7 1.42 GB 1.0 2.2 1.6 0.5 2.1 1.6 0.0 1.3 105.3 103.4 21.01 11.95 10 2.101 1541K 9811 0.0 0.0 L4 161/28 11.33 GB 0.9 3.2 1.7 1.5 3.0 1.5 0.2 1.7 130.2 122.4 25.22 15.80 16 1.576 693K 0 0.0 0.0 L5 128/0 9.90 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 335/53 24.24 GB 0.0 5.4 3.4 2.0 5.9 3.9 1.2 7.4 105.7 116.8 52.01 30.27 37 1.406 2235K 9811 0.0 0.0 Int 0/0 0.00 KB 0.0 5.4 3.4 2.0 5.9 3.9 1.2 7.4 105.7 116.8 52.01 30.27 37 1.406 2235K 9811 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.4 3.4 2.0 5.1 3.1 0.0 0.0 118.9 113.8 46.23 27.75 26 1.778 2235K 9811 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 141.4 5.78 2.52 11 0.526 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.4 total, 6.4 interval Flush(GB): cumulative 0.798, interval 0.798 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.93 GB write, 944.49 MB/s write, 5.37 GB read, 854.04 MB/s read, 52.0 seconds Interval compaction: 5.93 GB write, 957.00 MB/s write, 5.37 GB read, 865.35 MB/s read, 52.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 13, interval: 13 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 117969 Average: 92.8408 StdDev: 586.30 Min: 1 Median: 2.2997 Max: 43075 Percentiles: P50: 2.30 P75: 3.64 P99: 1686.83 P99.9: 5500.55 P99.99: 21291.64 ------------------------------------------------------ [ 0, 1 ] 2078 1.761% 1.761% ( 1, 2 ] 47276 40.075% 41.836% ######## ( 2, 3 ] 32135 27.240% 69.077% ##### ( 3, 4 ] 10934 9.269% 78.345% ## ( 4, 6 ] 11249 9.536% 87.881% ## ( 6, 10 ] 3124 2.648% 90.529% # ( 10, 15 ] 627 0.531% 91.060% ( 15, 22 ] 99 0.084% 91.144% ( 22, 34 ] 9 0.008% 91.152% ( 34, 51 ] 18 0.015% 91.167% ( 51, 76 ] 18 0.015% 91.182% ( 76, 110 ] 1 0.001% 91.183% ( 110, 170 ] 48 0.041% 91.224% ( 170, 250 ] 290 0.246% 91.470% ( 250, 380 ] 945 0.801% 92.271% ( 380, 580 ] 2705 2.293% 94.564% ( 580, 870 ] 3394 2.877% 97.441% # ( 870, 1300 ] 1584 1.343% 98.784% ( 1300, 1900 ] 396 0.336% 99.119% ( 1900, 2900 ] 412 0.349% 99.469% ( 2900, 4400 ] 447 0.379% 99.847% ( 4400, 6600 ] 124 0.105% 99.953% ( 6600, 9900 ] 27 0.023% 99.975% ( 9900, 14000 ] 9 0.008% 99.983% ( 14000, 22000 ] 9 0.008% 99.991% ( 22000, 33000 ] 13 0.011% 100.002% ( 33000, 50000 ] 6 0.005% 100.007% ** Level 1 read latency histogram (micros): Count: 516 Average: 846.5465 StdDev: 1103.45 Min: 2 Median: 558.6885 Max: 7178 Percentiles: P50: 558.69 P75: 853.50 P99: 6020.67 P99.9: 7178.00 P99.99: 7178.00 ------------------------------------------------------ ( 1, 2 ] 20 3.876% 3.876% # ( 2, 3 ] 46 8.915% 12.791% ## ( 3, 4 ] 6 1.163% 13.953% ( 4, 6 ] 13 2.519% 16.473% # ( 6, 10 ] 5 0.969% 17.442% ( 10, 15 ] 1 0.194% 17.636% ( 110, 170 ] 3 0.581% 18.217% ( 170, 250 ] 7 1.357% 19.574% ( 250, 380 ] 49 9.496% 29.070% ## ( 380, 580 ] 122 23.643% 52.713% ##### ( 580, 870 ] 123 23.837% 76.550% ##### ( 870, 1300 ] 50 9.690% 86.240% ## ( 1300, 1900 ] 22 4.264% 90.504% # ( 1900, 2900 ] 15 2.907% 93.411% # ( 2900, 4400 ] 21 4.070% 97.481% # ( 4400, 6600 ] 12 2.326% 99.806% ( 6600, 9900 ] 2 0.388% 100.194% ** Level 2 read latency histogram (micros): Count: 328437 Average: 15.1063 StdDev: 252.64 Min: 1 Median: 2.1915 Max: 28364 Percentiles: P50: 2.19 P75: 3.15 P99: 12.69 P99.9: 2351.82 P99.99: 3803.99 ------------------------------------------------------ [ 0, 1 ] 3656 1.113% 1.113% ( 1, 2 ] 142395 43.355% 44.468% ######### ( 2, 3 ] 94849 28.879% 73.347% ###### ( 3, 4 ] 35621 10.846% 84.193% ## ( 4, 6 ] 38216 11.636% 95.829% ## ( 6, 10 ] 9262 2.820% 98.649% # ( 10, 15 ] 2146 0.653% 99.302% ( 15, 22 ] 325 0.099% 99.401% ( 22, 34 ] 57 0.017% 99.418% ( 34, 51 ] 60 0.018% 99.437% ( 51, 76 ] 48 0.015% 99.451% ( 76, 110 ] 12 0.004% 99.455% ( 110, 170 ] 11 0.003% 99.458% ( 170, 250 ] 50 0.015% 99.474% ( 250, 380 ] 187 0.057% 99.531% ( 380, 580 ] 391 0.119% 99.650% ( 580, 870 ] 363 0.111% 99.760% ( 870, 1300 ] 203 0.062% 99.822% ( 1300, 1900 ] 154 0.047% 99.869% ( 1900, 2900 ] 227 0.069% 99.938% ( 2900, 4400 ] 284 0.086% 100.024% ( 4400, 6600 ] 100 0.030% 100.055% ( 6600, 9900 ] 34 0.010% 100.065% ( 9900, 14000 ] 13 0.004% 100.069% ( 14000, 22000 ] 24 0.007% 100.076% ( 22000, 33000 ] 3 0.001% 100.077% ** Level 3 read latency histogram (micros): Count: 485935 Average: 14.1973 StdDev: 224.28 Min: 1 Median: 2.2225 Max: 42768 Percentiles: P50: 2.22 P75: 3.12 P99: 13.56 P99.9: 1099.18 P99.99: 1763.17 ------------------------------------------------------ [ 0, 1 ] 4071 0.838% 0.838% ( 1, 2 ] 206005 42.394% 43.231% ######## ( 2, 3 ] 147824 30.421% 73.652% ###### ( 3, 4 ] 52709 10.847% 84.499% ## ( 4, 6 ] 55563 11.434% 95.933% ## ( 6, 10 ] 13288 2.735% 98.668% # ( 10, 15 ] 2270 0.467% 99.135% ( 15, 22 ] 279 0.057% 99.192% ( 22, 34 ] 110 0.023% 99.215% ( 34, 51 ] 175 0.036% 99.251% ( 51, 76 ] 74 0.015% 99.266% ( 76, 110 ] 28 0.006% 99.272% ( 110, 170 ] 46 0.009% 99.281% ( 170, 250 ] 125 0.026% 99.307% ( 250, 380 ] 380 0.078% 99.385% ( 380, 580 ] 1010 0.208% 99.593% ( 580, 870 ] 1200 0.247% 99.840% ( 870, 1300 ] 548 0.113% 99.953% ( 1300, 1900 ] 235 0.048% 100.001% ( 1900, 2900 ] 258 0.053% 100.054% ( 2900, 4400 ] 309 0.064% 100.118% ( 4400, 6600 ] 81 0.017% 100.134% ( 6600, 9900 ] 26 0.005% 100.140% ( 9900, 14000 ] 14 0.003% 100.143% ( 14000, 22000 ] 13 0.003% 100.145% ( 22000, 33000 ] 5 0.001% 100.146% ( 33000, 50000 ] 2 0.000% 100.147% ** Level 4 read latency histogram (micros): Count: 529455 Average: 28.3969 StdDev: 334.95 Min: 1 Median: 2.2618 Max: 42917 Percentiles: P50: 2.26 P75: 3.27 P99: 630.52 P99.9: 1647.14 P99.99: 2228.16 ------------------------------------------------------ [ 0, 1 ] 4426 0.836% 0.836% ( 1, 2 ] 218878 41.340% 42.176% ######## ( 2, 3 ] 158217 29.883% 72.059% ###### ( 3, 4 ] 58125 10.978% 83.037% ## ( 4, 6 ] 60556 11.437% 94.475% ## ( 6, 10 ] 15084 2.849% 97.324% # ( 10, 15 ] 2710 0.512% 97.836% ( 15, 22 ] 344 0.065% 97.901% ( 22, 34 ] 126 0.024% 97.924% ( 34, 51 ] 209 0.039% 97.964% ( 51, 76 ] 142 0.027% 97.991% ( 76, 110 ] 56 0.011% 98.001% ( 110, 170 ] 83 0.016% 98.017% ( 170, 250 ] 370 0.070% 98.087% ( 250, 380 ] 1220 0.230% 98.317% ( 380, 580 ] 3030 0.572% 98.890% ( 580, 870 ] 3355 0.634% 99.523% ( 870, 1300 ] 1626 0.307% 99.830% ( 1300, 1900 ] 637 0.120% 99.951% ( 1900, 2900 ] 634 0.120% 100.070% ( 2900, 4400 ] 645 0.122% 100.192% ( 4400, 6600 ] 181 0.034% 100.226% ( 6600, 9900 ] 43 0.008% 100.235% ( 9900, 14000 ] 18 0.003% 100.238% ( 14000, 22000 ] 25 0.005% 100.243% ( 22000, 33000 ] 14 0.003% 100.245% ( 33000, 50000 ] 9 0.002% 100.247% ** Level 5 read latency histogram (micros): Count: 7415 Average: 1115.5125 StdDev: 2128.03 Min: 90 Median: 676.3007 Max: 44790 Percentiles: P50: 676.30 P75: 1025.91 P99: 6530.70 P99.9: 38540.45 P99.99: 44790.00 ------------------------------------------------------ ( 76, 110 ] 5 0.067% 0.067% ( 110, 170 ] 39 0.526% 0.593% ( 170, 250 ] 218 2.940% 3.533% # ( 250, 380 ] 756 10.196% 13.729% ## ( 380, 580 ] 1944 26.217% 39.946% ##### ( 580, 870 ] 2245 30.276% 70.223% ###### ( 870, 1300 ] 977 13.176% 83.399% ### ( 1300, 1900 ] 334 4.504% 87.903% # ( 1900, 2900 ] 325 4.383% 92.286% # ( 2900, 4400 ] 401 5.408% 97.694% # ( 4400, 6600 ] 100 1.349% 99.042% ( 6600, 9900 ] 27 0.364% 99.407% ( 9900, 14000 ] 6 0.081% 99.488% ( 14000, 22000 ] 18 0.243% 99.730% ( 22000, 33000 ] 9 0.121% 99.852% ( 33000, 50000 ] 11 0.148% 100.000% ** DB Stats ** Uptime(secs): 6.4 total, 6.4 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 152.96 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.485 H:M:S, 23.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 154.97 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.485 H:M:S, 23.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/07/15-15:17:31 ... thread 0: (1000000,2000000) ops and (172743.6,164797.2) ops/second in (5.788928,12.136132) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/9 1.84 GB 125.2 1.1 0.0 1.1 2.9 1.8 0.0 1.6 50.4 134.1 22.46 10.19 31 0.725 1149K 6156 0.0 0.0 L1 9/0 791.93 MB 79.2 0.8 0.5 0.3 0.8 0.4 0.0 1.7 70.9 69.9 11.33 4.75 1 11.331 816K 11K 0.0 0.0 L3 18/0 999.83 MB 1.0 2.2 1.6 0.5 2.1 1.6 0.0 1.3 105.3 103.4 21.01 11.95 10 2.101 1541K 9811 0.0 0.0 L4 142/0 9.55 GB 1.0 5.8 2.2 3.6 5.2 1.5 0.2 2.4 148.2 132.1 40.06 26.36 23 1.742 1150K 0 0.0 0.0 L5 154/0 11.69 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 338/9 24.82 GB 0.0 9.8 4.3 5.6 11.0 5.4 3.0 6.0 106.3 118.8 94.86 53.26 65 1.459 4657K 27K 0.0 0.0 Int 0/0 0.00 KB 0.0 9.8 4.3 5.6 11.0 5.4 3.0 6.0 106.3 118.8 94.86 53.26 65 1.459 4657K 27K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.8 4.3 5.6 9.2 3.6 0.0 0.0 118.4 110.2 85.16 47.78 38 2.241 4657K 27K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 194.6 9.70 5.48 27 0.359 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.2 total, 12.1 interval Flush(GB): cumulative 1.843, interval 1.843 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.00 GB write, 921.93 MB/s write, 9.85 GB read, 825.05 MB/s read, 94.9 seconds Interval compaction: 11.00 GB write, 928.32 MB/s write, 9.85 GB read, 830.77 MB/s read, 94.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 16, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 16, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 14, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 16, total-stops: 14, interval: 30 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 483453 Average: 53.5520 StdDev: 321.01 Min: 0 Median: 1.7462 Max: 43075 Percentiles: P50: 1.75 P75: 2.79 P99: 840.89 P99.9: 2854.34 P99.99: 3865.84 ------------------------------------------------------ [ 0, 1 ] 57191 11.830% 11.830% ## ( 1, 2 ] 247312 51.155% 62.985% ########## ( 2, 3 ] 73783 15.262% 78.247% ### ( 3, 4 ] 22508 4.656% 82.902% # ( 4, 6 ] 18792 3.887% 86.789% # ( 6, 10 ] 6677 1.381% 88.171% ( 10, 15 ] 966 0.200% 88.370% ( 15, 22 ] 173 0.036% 88.406% ( 22, 34 ] 27 0.006% 88.412% ( 34, 51 ] 24 0.005% 88.417% ( 51, 76 ] 23 0.005% 88.421% ( 76, 110 ] 177 0.037% 88.458% ( 110, 170 ] 7074 1.463% 89.921% ( 170, 250 ] 23020 4.762% 94.683% # ( 250, 380 ] 9274 1.918% 96.601% ( 380, 580 ] 6583 1.362% 97.963% ( 580, 870 ] 5574 1.153% 99.116% ( 870, 1300 ] 2496 0.516% 99.632% ( 1300, 1900 ] 680 0.141% 99.773% ( 1900, 2900 ] 645 0.133% 99.906% ( 2900, 4400 ] 630 0.130% 100.036% ( 4400, 6600 ] 144 0.030% 100.066% ( 6600, 9900 ] 28 0.006% 100.072% ( 9900, 14000 ] 9 0.002% 100.074% ( 14000, 22000 ] 9 0.002% 100.076% ( 22000, 33000 ] 13 0.003% 100.078% ( 33000, 50000 ] 6 0.001% 100.080% ** Level 1 read latency histogram (micros): Count: 87130 Average: 13.3712 StdDev: 118.34 Min: 1 Median: 2.6900 Max: 7178 Percentiles: P50: 2.69 P75: 4.20 P99: 247.28 P99.9: 1316.31 P99.99: 4819.28 ------------------------------------------------------ [ 0, 1 ] 585 0.671% 0.671% ( 1, 2 ] 24431 28.040% 28.711% ###### ( 2, 3 ] 26884 30.855% 59.566% ###### ( 3, 4 ] 11808 13.552% 73.118% ### ( 4, 6 ] 16567 19.014% 92.132% #### ( 6, 10 ] 3698 4.244% 96.377% # ( 10, 15 ] 752 0.863% 97.240% ( 15, 22 ] 378 0.434% 97.674% ( 22, 34 ] 63 0.072% 97.746% ( 34, 51 ] 15 0.017% 97.763% ( 51, 76 ] 7 0.008% 97.771% ( 76, 110 ] 13 0.015% 97.786% ( 110, 170 ] 283 0.325% 98.111% ( 170, 250 ] 802 0.920% 99.031% ( 250, 380 ] 304 0.349% 99.380% ( 380, 580 ] 213 0.244% 99.625% ( 580, 870 ] 173 0.199% 99.823% ( 870, 1300 ] 66 0.076% 99.899% ( 1300, 1900 ] 32 0.037% 99.936% ( 1900, 2900 ] 18 0.021% 99.956% ( 2900, 4400 ] 27 0.031% 99.987% ( 4400, 6600 ] 12 0.014% 100.001% ( 6600, 9900 ] 2 0.002% 100.003% ** Level 2 read latency histogram (micros): Count: 328437 Average: 15.1063 StdDev: 252.64 Min: 1 Median: 2.1915 Max: 28364 Percentiles: P50: 2.19 P75: 3.15 P99: 12.69 P99.9: 2351.82 P99.99: 3803.99 ------------------------------------------------------ [ 0, 1 ] 3656 1.113% 1.113% ( 1, 2 ] 142395 43.355% 44.468% ######### ( 2, 3 ] 94849 28.879% 73.347% ###### ( 3, 4 ] 35621 10.846% 84.193% ## ( 4, 6 ] 38216 11.636% 95.829% ## ( 6, 10 ] 9262 2.820% 98.649% # ( 10, 15 ] 2146 0.653% 99.302% ( 15, 22 ] 325 0.099% 99.401% ( 22, 34 ] 57 0.017% 99.418% ( 34, 51 ] 60 0.018% 99.437% ( 51, 76 ] 48 0.015% 99.451% ( 76, 110 ] 12 0.004% 99.455% ( 110, 170 ] 11 0.003% 99.458% ( 170, 250 ] 50 0.015% 99.474% ( 250, 380 ] 187 0.057% 99.531% ( 380, 580 ] 391 0.119% 99.650% ( 580, 870 ] 363 0.111% 99.760% ( 870, 1300 ] 203 0.062% 99.822% ( 1300, 1900 ] 154 0.047% 99.869% ( 1900, 2900 ] 227 0.069% 99.938% ( 2900, 4400 ] 284 0.086% 100.024% ( 4400, 6600 ] 100 0.030% 100.055% ( 6600, 9900 ] 34 0.010% 100.065% ( 9900, 14000 ] 13 0.004% 100.069% ( 14000, 22000 ] 24 0.007% 100.076% ( 22000, 33000 ] 3 0.001% 100.077% ** Level 3 read latency histogram (micros): Count: 570001 Average: 15.1314 StdDev: 212.09 Min: 1 Median: 2.2283 Max: 42768 Percentiles: P50: 2.23 P75: 3.17 P99: 220.01 P99.9: 1121.20 P99.99: 1805.15 ------------------------------------------------------ [ 0, 1 ] 5161 0.905% 0.905% ( 1, 2 ] 240735 42.234% 43.140% ######## ( 2, 3 ] 171300 30.053% 73.192% ###### ( 3, 4 ] 61431 10.777% 83.970% ## ( 4, 6 ] 64598 11.333% 95.302% ## ( 6, 10 ] 15554 2.729% 98.031% # ( 10, 15 ] 2617 0.459% 98.490% ( 15, 22 ] 327 0.057% 98.548% ( 22, 34 ] 118 0.021% 98.568% ( 34, 51 ] 186 0.033% 98.601% ( 51, 76 ] 91 0.016% 98.617% ( 76, 110 ] 49 0.009% 98.626% ( 110, 170 ] 768 0.135% 98.760% ( 170, 250 ] 2185 0.383% 99.144% ( 250, 380 ] 1098 0.193% 99.336% ( 380, 580 ] 1359 0.238% 99.575% ( 580, 870 ] 1455 0.255% 99.830% ( 870, 1300 ] 683 0.120% 99.950% ( 1300, 1900 ] 272 0.048% 99.998% ( 1900, 2900 ] 296 0.052% 100.049% ( 2900, 4400 ] 334 0.059% 100.108% ( 4400, 6600 ] 82 0.014% 100.122% ( 6600, 9900 ] 27 0.005% 100.127% ( 9900, 14000 ] 14 0.002% 100.130% ( 14000, 22000 ] 13 0.002% 100.132% ( 22000, 33000 ] 5 0.001% 100.133% ( 33000, 50000 ] 2 0.000% 100.133% ** Level 4 read latency histogram (micros): Count: 952903 Average: 27.3448 StdDev: 263.76 Min: 1 Median: 2.2861 Max: 42917 Percentiles: P50: 2.29 P75: 3.43 P99: 488.68 P99.9: 1145.68 P99.99: 1349.54 ------------------------------------------------------ [ 0, 1 ] 8835 0.927% 0.927% ( 1, 2 ] 389651 40.891% 41.818% ######## ( 2, 3 ] 272480 28.595% 70.413% ###### ( 3, 4 ] 102729 10.781% 81.193% ## ( 4, 6 ] 105019 11.021% 92.214% ## ( 6, 10 ] 26387 2.769% 94.984% # ( 10, 15 ] 4522 0.475% 95.458% ( 15, 22 ] 580 0.061% 95.519% ( 22, 34 ] 162 0.017% 95.536% ( 34, 51 ] 265 0.028% 95.564% ( 51, 76 ] 188 0.020% 95.583% ( 76, 110 ] 243 0.026% 95.609% ( 110, 170 ] 5828 0.612% 96.221% ( 170, 250 ] 16884 1.772% 97.992% ( 250, 380 ] 6620 0.695% 98.687% ( 380, 580 ] 5486 0.576% 99.263% ( 580, 870 ] 4660 0.489% 99.752% ( 870, 1300 ] 2201 0.231% 99.983% ( 1300, 1900 ] 820 0.086% 100.069% ( 1900, 2900 ] 803 0.084% 100.153% ( 2900, 4400 ] 787 0.083% 100.236% ( 4400, 6600 ] 204 0.021% 100.257% ( 6600, 9900 ] 47 0.005% 100.262% ( 9900, 14000 ] 18 0.002% 100.264% ( 14000, 22000 ] 25 0.003% 100.267% ( 22000, 33000 ] 14 0.001% 100.268% ( 33000, 50000 ] 9 0.001% 100.269% ** Level 5 read latency histogram (micros): Count: 30160 Average: 481.5891 StdDev: 1139.19 Min: 90 Median: 240.9618 Max: 44790 Percentiles: P50: 240.96 P75: 472.40 P99: 3977.90 P99.9: 17484.44 P99.99: 44790.00 ------------------------------------------------------ ( 76, 110 ] 141 0.468% 0.468% ( 110, 170 ] 4112 13.634% 14.101% ### ( 170, 250 ] 12206 40.471% 54.572% ######## ( 250, 380 ] 4568 15.146% 69.718% ### ( 380, 580 ] 3448 11.432% 81.151% ## ( 580, 870 ] 2926 9.702% 90.852% ## ( 870, 1300 ] 1289 4.274% 95.126% # ( 1300, 1900 ] 447 1.482% 96.608% ( 1900, 2900 ] 393 1.303% 97.911% ( 2900, 4400 ] 457 1.515% 99.426% ( 4400, 6600 ] 102 0.338% 99.765% ( 6600, 9900 ] 27 0.090% 99.854% ( 9900, 14000 ] 6 0.020% 99.874% ( 14000, 22000 ] 18 0.060% 99.934% ( 22000, 33000 ] 9 0.030% 99.964% ( 33000, 50000 ] 11 0.036% 100.000% ** DB Stats ** Uptime(secs): 12.2 total, 5.8 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 161.03 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.268 H:M:S, 26.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 170.01 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.784 H:M:S, 30.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-15:17:35 ... thread 0: (1000000,3000000) ops and (263789.6,188359.0) ops/second in (3.790900,15.927032) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/9 2.81 GB 225.3 2.1 0.0 2.1 4.9 2.8 0.0 1.7 73.0 171.4 29.29 16.60 51 0.574 2171K 11K 0.0 0.0 L1 9/0 791.93 MB 79.2 0.8 0.5 0.3 0.8 0.4 0.0 1.7 70.9 69.9 11.33 4.75 1 11.331 816K 11K 0.0 0.0 L3 18/0 999.83 MB 1.0 2.2 1.6 0.5 2.1 1.6 0.0 1.3 105.3 103.4 21.01 11.95 10 2.101 1541K 9811 0.0 0.0 L4 142/0 9.55 GB 1.0 5.8 2.2 3.6 5.2 1.5 0.2 2.4 148.2 132.1 40.06 26.36 23 1.742 1150K 0 0.0 0.0 L5 154/0 11.69 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 342/9 25.80 GB 0.0 10.8 4.3 6.6 13.0 6.4 3.0 4.6 109.1 130.6 101.69 59.66 85 1.196 5680K 32K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.8 4.3 6.6 13.0 6.4 3.0 4.6 109.1 130.6 101.69 59.66 85 1.196 5680K 32K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.8 4.3 6.6 10.1 3.6 0.0 0.0 124.7 116.7 88.96 51.52 42 2.118 5680K 32K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 227.4 12.73 8.14 43 0.296 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.0 total, 15.9 interval Flush(GB): cumulative 2.826, interval 2.826 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.97 GB write, 829.09 MB/s write, 10.83 GB read, 692.60 MB/s read, 101.7 seconds Interval compaction: 12.97 GB write, 833.47 MB/s write, 10.83 GB read, 696.26 MB/s read, 101.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 17, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 17, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 14, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 17, total-stops: 14, interval: 31 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 775138 Average: 40.6652 StdDev: 256.74 Min: 0 Median: 1.6359 Max: 43075 Percentiles: P50: 1.64 P75: 2.46 P99: 682.85 P99.9: 2092.03 P99.99: 3320.21 ------------------------------------------------------ [ 0, 1 ] 124701 16.088% 16.088% ### ( 1, 2 ] 413366 53.328% 69.416% ########### ( 2, 3 ] 95081 12.266% 81.682% ## ( 3, 4 ] 28834 3.720% 85.402% # ( 4, 6 ] 23065 2.976% 88.377% # ( 6, 10 ] 7728 0.997% 89.374% ( 10, 15 ] 1240 0.160% 89.534% ( 15, 22 ] 215 0.028% 89.562% ( 22, 34 ] 41 0.005% 89.567% ( 34, 51 ] 29 0.004% 89.571% ( 51, 76 ] 25 0.003% 89.574% ( 76, 110 ] 438 0.057% 89.631% ( 110, 170 ] 13871 1.789% 91.420% ( 170, 250 ] 37708 4.865% 96.285% # ( 250, 380 ] 11776 1.519% 97.804% ( 380, 580 ] 7258 0.936% 98.741% ( 580, 870 ] 5669 0.731% 99.472% ( 870, 1300 ] 2513 0.324% 99.796% ( 1300, 1900 ] 681 0.088% 99.884% ( 1900, 2900 ] 645 0.083% 99.967% ( 2900, 4400 ] 630 0.081% 100.049% ( 4400, 6600 ] 144 0.019% 100.067% ( 6600, 9900 ] 28 0.004% 100.071% ( 9900, 14000 ] 9 0.001% 100.072% ( 14000, 22000 ] 9 0.001% 100.073% ( 22000, 33000 ] 13 0.002% 100.075% ( 33000, 50000 ] 6 0.001% 100.075% ** Level 1 read latency histogram (micros): Count: 89805 Average: 18.1190 StdDev: 120.86 Min: 1 Median: 2.7353 Max: 7178 Percentiles: P50: 2.74 P75: 4.41 P99: 307.60 P99.9: 1288.75 P99.99: 4770.24 ------------------------------------------------------ [ 0, 1 ] 585 0.651% 0.651% ( 1, 2 ] 24461 27.238% 27.889% ##### ( 2, 3 ] 27006 30.072% 57.961% ###### ( 3, 4 ] 11884 13.233% 71.194% ### ( 4, 6 ] 16719 18.617% 89.811% #### ( 6, 10 ] 3713 4.135% 93.946% # ( 10, 15 ] 760 0.846% 94.792% ( 15, 22 ] 379 0.422% 95.214% ( 22, 34 ] 64 0.071% 95.285% ( 34, 51 ] 15 0.017% 95.302% ( 51, 76 ] 7 0.008% 95.310% ( 76, 110 ] 35 0.039% 95.349% ( 110, 170 ] 939 1.046% 96.394% ( 170, 250 ] 2114 2.354% 98.748% ( 250, 380 ] 510 0.568% 99.316% ( 380, 580 ] 277 0.308% 99.625% ( 580, 870 ] 180 0.200% 99.825% ( 870, 1300 ] 69 0.077% 99.902% ( 1300, 1900 ] 32 0.036% 99.938% ( 1900, 2900 ] 18 0.020% 99.958% ( 2900, 4400 ] 27 0.030% 99.988% ( 4400, 6600 ] 12 0.013% 100.001% ( 6600, 9900 ] 2 0.002% 100.003% ** Level 2 read latency histogram (micros): Count: 328437 Average: 15.1063 StdDev: 252.64 Min: 1 Median: 2.1915 Max: 28364 Percentiles: P50: 2.19 P75: 3.15 P99: 12.69 P99.9: 2351.82 P99.99: 3803.99 ------------------------------------------------------ [ 0, 1 ] 3656 1.113% 1.113% ( 1, 2 ] 142395 43.355% 44.468% ######### ( 2, 3 ] 94849 28.879% 73.347% ###### ( 3, 4 ] 35621 10.846% 84.193% ## ( 4, 6 ] 38216 11.636% 95.829% ## ( 6, 10 ] 9262 2.820% 98.649% # ( 10, 15 ] 2146 0.653% 99.302% ( 15, 22 ] 325 0.099% 99.401% ( 22, 34 ] 57 0.017% 99.418% ( 34, 51 ] 60 0.018% 99.437% ( 51, 76 ] 48 0.015% 99.451% ( 76, 110 ] 12 0.004% 99.455% ( 110, 170 ] 11 0.003% 99.458% ( 170, 250 ] 50 0.015% 99.474% ( 250, 380 ] 187 0.057% 99.531% ( 380, 580 ] 391 0.119% 99.650% ( 580, 870 ] 363 0.111% 99.760% ( 870, 1300 ] 203 0.062% 99.822% ( 1300, 1900 ] 154 0.047% 99.869% ( 1900, 2900 ] 227 0.069% 99.938% ( 2900, 4400 ] 284 0.086% 100.024% ( 4400, 6600 ] 100 0.030% 100.055% ( 6600, 9900 ] 34 0.010% 100.065% ( 9900, 14000 ] 13 0.004% 100.069% ( 14000, 22000 ] 24 0.007% 100.076% ( 22000, 33000 ] 3 0.001% 100.077% ** Level 3 read latency histogram (micros): Count: 573521 Average: 16.2859 StdDev: 212.02 Min: 1 Median: 2.2386 Max: 42768 Percentiles: P50: 2.24 P75: 3.21 P99: 242.08 P99.9: 1119.25 P99.99: 1804.37 ------------------------------------------------------ [ 0, 1 ] 5161 0.900% 0.900% ( 1, 2 ] 240735 41.975% 42.875% ######## ( 2, 3 ] 171300 29.868% 72.743% ###### ( 3, 4 ] 61431 10.711% 83.454% ## ( 4, 6 ] 64598 11.263% 94.718% ## ( 6, 10 ] 15554 2.712% 97.430% # ( 10, 15 ] 2617 0.456% 97.886% ( 15, 22 ] 327 0.057% 97.943% ( 22, 34 ] 118 0.021% 97.963% ( 34, 51 ] 186 0.032% 97.996% ( 51, 76 ] 91 0.016% 98.012% ( 76, 110 ] 87 0.015% 98.027% ( 110, 170 ] 1759 0.307% 98.334% ( 170, 250 ] 4242 0.740% 99.073% ( 250, 380 ] 1423 0.248% 99.321% ( 380, 580 ] 1451 0.253% 99.574% ( 580, 870 ] 1471 0.256% 99.831% ( 870, 1300 ] 684 0.119% 99.950% ( 1300, 1900 ] 272 0.047% 99.998% ( 1900, 2900 ] 296 0.052% 100.049% ( 2900, 4400 ] 334 0.058% 100.107% ( 4400, 6600 ] 82 0.014% 100.122% ( 6600, 9900 ] 27 0.005% 100.126% ( 9900, 14000 ] 14 0.002% 100.129% ( 14000, 22000 ] 13 0.002% 100.131% ( 22000, 33000 ] 5 0.001% 100.132% ( 33000, 50000 ] 2 0.000% 100.132% ** Level 4 read latency histogram (micros): Count: 981566 Average: 32.4323 StdDev: 261.82 Min: 1 Median: 2.3386 Max: 42917 Percentiles: P50: 2.34 P75: 3.63 P99: 494.84 P99.9: 1141.14 P99.99: 1348.06 ------------------------------------------------------ [ 0, 1 ] 8835 0.900% 0.900% ( 1, 2 ] 389660 39.698% 40.598% ######## ( 2, 3 ] 272537 27.766% 68.363% ###### ( 3, 4 ] 102781 10.471% 78.835% ## ( 4, 6 ] 105084 10.706% 89.540% ## ( 6, 10 ] 26389 2.688% 92.229% # ( 10, 15 ] 4526 0.461% 92.690% ( 15, 22 ] 583 0.059% 92.749% ( 22, 34 ] 162 0.017% 92.766% ( 34, 51 ] 265 0.027% 92.793% ( 51, 76 ] 188 0.019% 92.812% ( 76, 110 ] 588 0.060% 92.872% ( 110, 170 ] 13977 1.424% 94.296% ( 170, 250 ] 33300 3.393% 97.688% # ( 250, 380 ] 9275 0.945% 98.633% ( 380, 580 ] 6270 0.639% 99.272% ( 580, 870 ] 4769 0.486% 99.758% ( 870, 1300 ] 2213 0.225% 99.983% ( 1300, 1900 ] 822 0.084% 100.067% ( 1900, 2900 ] 803 0.082% 100.149% ( 2900, 4400 ] 787 0.080% 100.229% ( 4400, 6600 ] 204 0.021% 100.250% ( 6600, 9900 ] 47 0.005% 100.255% ( 9900, 14000 ] 18 0.002% 100.256% ( 14000, 22000 ] 25 0.003% 100.259% ( 22000, 33000 ] 14 0.001% 100.260% ( 33000, 50000 ] 9 0.001% 100.261% ** Level 5 read latency histogram (micros): Count: 49483 Average: 372.7918 StdDev: 900.77 Min: 90 Median: 220.5383 Max: 44790 Percentiles: P50: 220.54 P75: 326.39 P99: 3343.67 P99.9: 9229.86 P99.99: 42352.63 ------------------------------------------------------ ( 76, 110 ] 371 0.750% 0.750% ( 110, 170 ] 9607 19.415% 20.165% #### ( 170, 250 ] 23370 47.228% 67.393% ######### ( 250, 380 ] 6406 12.946% 80.339% ### ( 380, 580 ] 3961 8.005% 88.343% ## ( 580, 870 ] 2997 6.057% 94.400% # ( 870, 1300 ] 1300 2.627% 97.027% # ( 1300, 1900 ] 448 0.905% 97.933% ( 1900, 2900 ] 393 0.794% 98.727% ( 2900, 4400 ] 457 0.924% 99.650% ( 4400, 6600 ] 102 0.206% 99.857% ( 6600, 9900 ] 27 0.055% 99.911% ( 9900, 14000 ] 6 0.012% 99.923% ( 14000, 22000 ] 18 0.036% 99.960% ( 22000, 33000 ] 9 0.018% 99.978% ( 33000, 50000 ] 11 0.022% 100.000% ** DB Stats ** Uptime(secs): 16.0 total, 3.8 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 184.37 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.337 H:M:S, 20.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 259.62 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.069 H:M:S, 1.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 373.769 micros/op 10680 ops/sec 20.129 seconds 214996 operations; 10.3 MB/s (53999 of 53999 found) Microseconds per read: Count: 214996 Average: 373.7774 StdDev: 1092.20 Min: 2 Median: 237.7853 Max: 84221 Percentiles: P50: 237.79 P75: 359.38 P99: 2729.37 P99.9: 14523.66 P99.99: 46523.04 ------------------------------------------------------ ( 1, 2 ] 15 0.007% 0.007% ( 2, 3 ] 67 0.031% 0.038% ( 3, 4 ] 85 0.040% 0.078% ( 4, 6 ] 180 0.084% 0.161% ( 6, 10 ] 319 0.148% 0.310% ( 10, 15 ] 1471 0.684% 0.994% ( 15, 22 ] 3138 1.460% 2.454% ( 22, 34 ] 4641 2.159% 4.612% ( 34, 51 ] 2237 1.040% 5.653% ( 51, 76 ] 351 0.163% 5.816% ( 76, 110 ] 112 0.052% 5.868% ( 110, 170 ] 14559 6.772% 12.640% # ( 170, 250 ] 94797 44.092% 56.732% ######### ( 250, 380 ] 46678 21.711% 78.443% #### ( 380, 580 ] 25171 11.708% 90.151% ## ( 580, 870 ] 11741 5.461% 95.612% # ( 870, 1300 ] 4686 2.180% 97.792% ( 1300, 1900 ] 1699 0.790% 98.582% ( 1900, 2900 ] 1084 0.504% 99.086% ( 2900, 4400 ] 1172 0.545% 99.631% ( 4400, 6600 ] 332 0.154% 99.786% ( 6600, 9900 ] 146 0.068% 99.853% ( 9900, 14000 ] 93 0.043% 99.897% ( 14000, 22000 ] 107 0.050% 99.947% ( 22000, 33000 ] 76 0.035% 99.982% ( 33000, 50000 ] 22 0.010% 99.992% ( 50000, 75000 ] 12 0.006% 99.998% ( 75000, 110000 ] 5 0.002% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 3697338 rocksdb.bloom.filter.full.positive COUNT : 250348 rocksdb.bloom.filter.full.true.positive COUNT : 214091 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 905 rocksdb.memtable.miss COUNT : 214091 rocksdb.l0.hit COUNT : 24010 rocksdb.l1.hit COUNT : 6060 rocksdb.l2andup.hit COUNT : 184021 rocksdb.compaction.key.drop.new COUNT : 733246 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3644544 rocksdb.number.keys.read COUNT : 214996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3761169408 rocksdb.bytes.read COUNT : 214996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 505 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 5077235 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 12970053526 rocksdb.compact.write.bytes COUNT : 12209496716 rocksdb.flush.write.bytes COUNT : 3693586664 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 27 rocksdb.number.superversion_acquires COUNT : 673 rocksdb.number.superversion_releases COUNT : 654 rocksdb.number.superversion_cleanups COUNT : 166 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 79 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3669408768 rocksdb.memtable.garbage.bytes.at.flush COUNT : 6565888 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 13101018042 rocksdb.non.last.level.read.count COUNT : 3135857 rocksdb.block.checksum.compute.count COUNT : 3395460 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 237.082070 P95 : 836.833803 P99 : 2729.057301 P100 : 84218.000000 COUNT : 214996 SUM : 80176154 rocksdb.db.write.micros P50 : 1.617049 P95 : 5.144581 P99 : 7.855891 P100 : 696670.000000 COUNT : 3644544 SUM : 15218792 rocksdb.compaction.times.micros P50 : 1643333.333333 P95 : 5125000.000000 P99 : 11331152.000000 P100 : 11331152.000000 COUNT : 45 SUM : 91930898 rocksdb.compaction.times.cpu_micros P50 : 1110952.380952 P95 : 1845312.500000 P99 : 4753795.000000 P100 : 4753795.000000 COUNT : 45 SUM : 54410042 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 1317.647059 P95 : 7177.500000 P99 : 10364.000000 P100 : 10364.000000 COUNT : 53 SUM : 89651 rocksdb.compaction.outfile.sync.micros P50 : 2166.666667 P95 : 7920.000000 P99 : 41840.000000 P100 : 50916.000000 COUNT : 148 SUM : 511877 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 197.200000 P95 : 966.750000 P99 : 4317.500000 P100 : 8999.000000 COUNT : 111 SUM : 45596 rocksdb.table.open.io.micros P50 : 5882.608696 P95 : 21192.307692 P99 : 62375.000000 P100 : 70414.000000 COUNT : 505 SUM : 3958320 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.172846 P95 : 10.009527 P99 : 288.663074 P100 : 42503.000000 COUNT : 3141794 SUM : 58652413 rocksdb.read.block.get.micros P50 : 205.490864 P95 : 730.122989 P99 : 2223.357271 P100 : 44856.000000 COUNT : 253478 SUM : 71033080 rocksdb.write.raw.block.micros P50 : 0.536286 P95 : 1.397874 P99 : 3.899912 P100 : 76493.000000 COUNT : 3853590 SUM : 14483400 rocksdb.numfiles.in.singlecompaction P50 : 3.159091 P95 : 5.788889 P99 : 8.820000 P100 : 10.000000 COUNT : 59 SUM : 216 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.558996 P95 : 1106.515832 P99 : 1282.328203 P100 : 696449.000000 COUNT : 27762 SUM : 5079595 rocksdb.sst.read.micros P50 : 2.013305 P95 : 213.991084 P99 : 576.227826 P100 : 44790.000000 COUNT : 3136368 SUM : 116136163 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 214996 SUM : 214996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3644544 SUM : 3761169408 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 225789.473684 P95 : 671500.000000 P99 : 1069562.000000 P100 : 1069562.000000 COUNT : 53 SUM : 14718930 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 162 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0