[2024-07-24 23:58:58.915942] Starting SPDK v24.05.1-pre git sha1 241d0f3c9 / DPDK 22.11.4 initialization... [2024-07-24 23:58:58.916261] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3107231 ] Set seed to 1721858338908778 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-24 23:58:59.161586] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-24 23:58:59.210943] app.c: 604:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-24 23:58:59.211004] app.c: 605:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3107231' to capture a snapshot of events at runtime. [2024-07-24 23:58:59.211024] app.c: 610:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-24 23:58:59.211044] app.c: 611:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-24 23:58:59.211059] app.c: 612:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3107231 for offline analysis/debug. [2024-07-24 23:58:59.211192] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Jul 24 23:59:03 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/07/24-23:59:12 ... thread 0: (1000000,1000000) ops and (108741.8,108741.8) ops/second in (9.196098,9.196098) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/8 2.70 GB 62.9 0.6 0.0 0.6 1.4 0.9 0.0 1.6 69.5 177.5 8.16 5.91 15 0.544 575K 626 0.0 0.0 L1 14/14 1.43 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 973.12 MB 1.0 1.3 0.6 0.7 1.3 0.5 0.1 2.3 98.0 96.9 13.54 8.67 7 1.934 221K 0 0.0 0.0 L4 160/0 9.72 GB 1.0 3.6 2.5 1.1 3.6 2.5 0.0 1.4 94.9 94.3 39.34 23.04 28 1.405 1622K 292 0.0 0.0 L5 1112/0 90.49 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1313/22 105.29 GB 0.0 5.5 3.1 2.4 6.3 3.9 2.6 7.3 92.2 106.0 61.04 37.62 50 1.221 2419K 918 0.0 0.0 Int 0/0 0.00 KB 0.0 5.5 3.1 2.4 6.3 3.9 2.6 7.3 92.2 106.0 61.04 37.62 50 1.221 2419K 918 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.5 3.1 2.4 5.5 3.0 0.0 0.0 101.0 100.3 55.69 34.43 37 1.505 2419K 918 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 165.1 5.34 3.19 13 0.411 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.4 total, 9.2 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.32 GB write, 685.01 MB/s write, 5.49 GB read, 595.66 MB/s read, 61.0 seconds Interval compaction: 6.32 GB write, 703.47 MB/s write, 5.49 GB read, 611.71 MB/s read, 61.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 14, pending-compaction-bytes-stops: 0, total-delays: 14, total-stops: 13, interval: 26 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 444742 Average: 10.4426 StdDev: 166.01 Min: 1 Median: 2.3298 Max: 35934 Percentiles: P50: 2.33 P75: 3.19 P99: 139.64 P99.9: 832.02 P99.99: 2307.89 ------------------------------------------------------ [ 0, 1 ] 11492 2.584% 2.584% # ( 1, 2 ] 159647 35.897% 38.481% ####### ( 2, 3 ] 155349 34.930% 73.411% ####### ( 3, 4 ] 36910 8.299% 81.710% ## ( 4, 6 ] 48341 10.869% 92.579% ## ( 6, 10 ] 20882 4.695% 97.275% # ( 10, 15 ] 5407 1.216% 98.490% ( 15, 22 ] 968 0.218% 98.708% ( 22, 34 ] 288 0.065% 98.773% ( 34, 51 ] 53 0.012% 98.785% ( 51, 76 ] 27 0.006% 98.791% ( 76, 110 ] 148 0.033% 98.824% ( 110, 170 ] 1592 0.358% 99.182% ( 170, 250 ] 1662 0.374% 99.556% ( 250, 380 ] 752 0.169% 99.725% ( 380, 580 ] 566 0.127% 99.852% ( 580, 870 ] 250 0.056% 99.908% ( 870, 1300 ] 171 0.038% 99.947% ( 1300, 1900 ] 159 0.036% 99.982% ( 1900, 2900 ] 92 0.021% 100.003% ( 2900, 4400 ] 168 0.038% 100.041% ( 4400, 6600 ] 52 0.012% 100.053% ( 6600, 9900 ] 16 0.004% 100.056% ( 9900, 14000 ] 3 0.001% 100.057% ( 14000, 22000 ] 6 0.001% 100.058% ( 22000, 33000 ] 4 0.001% 100.059% ( 33000, 50000 ] 1 0.000% 100.059% ** Level 1 read latency histogram (micros): Count: 821 Average: 654.0987 StdDev: 2076.46 Min: 89 Median: 213.1193 Max: 21658 Percentiles: P50: 213.12 P75: 383.98 P99: 15432.00 P99.9: 21343.20 P99.99: 21658.00 ------------------------------------------------------ ( 76, 110 ] 21 2.558% 2.558% # ( 110, 170 ] 272 33.130% 35.688% ####### ( 170, 250 ] 218 26.553% 62.241% ##### ( 250, 380 ] 103 12.546% 74.787% ### ( 380, 580 ] 88 10.719% 85.505% ## ( 580, 870 ] 40 4.872% 90.378% # ( 870, 1300 ] 28 3.410% 93.788% # ( 1300, 1900 ] 12 1.462% 95.250% ( 1900, 2900 ] 11 1.340% 96.590% ( 2900, 4400 ] 5 0.609% 97.199% ( 4400, 6600 ] 7 0.853% 98.051% ( 6600, 9900 ] 5 0.609% 98.660% ( 9900, 14000 ] 1 0.122% 98.782% ( 14000, 22000 ] 10 1.218% 100.000% ** Level 2 read latency histogram (micros): Count: 16581 Average: 16.1080 StdDev: 194.97 Min: 1 Median: 2.8171 Max: 13320 Percentiles: P50: 2.82 P75: 4.69 P99: 130.38 P99.9: 3266.32 P99.99: 6310.44 ------------------------------------------------------ [ 0, 1 ] 28 0.169% 0.169% ( 1, 2 ] 3109 18.750% 18.919% #### ( 2, 3 ] 6307 38.038% 56.957% ######## ( 3, 4 ] 1822 10.988% 67.945% ## ( 4, 6 ] 3371 20.330% 88.276% #### ( 6, 10 ] 1364 8.226% 96.502% ## ( 10, 15 ] 338 2.038% 98.540% ( 15, 22 ] 46 0.277% 98.818% ( 22, 34 ] 13 0.078% 98.896% ( 34, 51 ] 6 0.036% 98.933% ( 76, 110 ] 1 0.006% 98.939% ( 110, 170 ] 30 0.181% 99.119% ( 170, 250 ] 24 0.145% 99.264% ( 250, 380 ] 19 0.115% 99.379% ( 380, 580 ] 17 0.103% 99.481% ( 580, 870 ] 18 0.109% 99.590% ( 870, 1300 ] 21 0.127% 99.717% ( 1300, 1900 ] 16 0.096% 99.813% ( 1900, 2900 ] 11 0.066% 99.879% ( 2900, 4400 ] 14 0.084% 99.964% ( 4400, 6600 ] 5 0.030% 99.994% ( 9900, 14000 ] 1 0.006% 100.000% ** Level 3 read latency histogram (micros): Count: 586121 Average: 11.2155 StdDev: 205.52 Min: 1 Median: 2.7112 Max: 52615 Percentiles: P50: 2.71 P75: 4.24 P99: 12.22 P99.9: 20.73 P99.99: 33.31 ------------------------------------------------------ [ 0, 1 ] 2267 0.387% 0.387% ( 1, 2 ] 131989 22.519% 22.906% ##### ( 2, 3 ] 223289 38.096% 61.002% ######## ( 3, 4 ] 68838 11.745% 72.747% ## ( 4, 6 ] 109245 18.639% 91.385% #### ( 6, 10 ] 41230 7.034% 98.420% # ( 10, 15 ] 7662 1.307% 99.727% ( 15, 22 ] 1239 0.211% 99.938% ( 22, 34 ] 322 0.055% 99.993% ( 34, 51 ] 79 0.013% 100.007% ( 51, 76 ] 74 0.013% 100.019% ( 76, 110 ] 66 0.011% 100.031% ( 110, 170 ] 215 0.037% 100.067% ( 170, 250 ] 221 0.038% 100.105% ( 250, 380 ] 148 0.025% 100.130% ( 380, 580 ] 176 0.030% 100.160% ( 580, 870 ] 232 0.040% 100.200% ( 870, 1300 ] 338 0.058% 100.257% ( 1300, 1900 ] 253 0.043% 100.301% ( 1900, 2900 ] 212 0.036% 100.337% ( 2900, 4400 ] 317 0.054% 100.391% ( 4400, 6600 ] 139 0.024% 100.415% ( 6600, 9900 ] 47 0.008% 100.423% ( 9900, 14000 ] 11 0.002% 100.424% ( 14000, 22000 ] 11 0.002% 100.426% ( 22000, 33000 ] 4 0.001% 100.427% ( 33000, 50000 ] 1 0.000% 100.427% ( 50000, 75000 ] 1 0.000% 100.427% ** Level 4 read latency histogram (micros): Count: 291587 Average: 35.7185 StdDev: 410.91 Min: 1 Median: 2.7348 Max: 84370 Percentiles: P50: 2.73 P75: 4.38 P99: 689.31 P99.9: 3281.13 P99.99: 4025.26 ------------------------------------------------------ [ 0, 1 ] 1331 0.456% 0.456% ( 1, 2 ] 65440 22.443% 22.899% #### ( 2, 3 ] 107550 36.884% 59.784% ####### ( 3, 4 ] 34797 11.934% 71.717% ## ( 4, 6 ] 50651 17.371% 89.088% ### ( 6, 10 ] 18298 6.275% 95.363% # ( 10, 15 ] 4389 1.505% 96.869% ( 15, 22 ] 697 0.239% 97.108% ( 22, 34 ] 173 0.059% 97.167% ( 34, 51 ] 52 0.018% 97.185% ( 51, 76 ] 40 0.014% 97.198% ( 76, 110 ] 189 0.065% 97.263% ( 110, 170 ] 1751 0.601% 97.864% ( 170, 250 ] 1452 0.498% 98.362% ( 250, 380 ] 749 0.257% 98.619% ( 380, 580 ] 798 0.274% 98.892% ( 580, 870 ] 836 0.287% 99.179% ( 870, 1300 ] 826 0.283% 99.462% ( 1300, 1900 ] 661 0.227% 99.689% ( 1900, 2900 ] 482 0.165% 99.854% ( 2900, 4400 ] 529 0.181% 100.036% ( 4400, 6600 ] 252 0.086% 100.122% ( 6600, 9900 ] 71 0.024% 100.146% ( 9900, 14000 ] 8 0.003% 100.149% ( 14000, 22000 ] 12 0.004% 100.153% ( 22000, 33000 ] 8 0.003% 100.156% ( 33000, 50000 ] 3 0.001% 100.157% ( 50000, 75000 ] 1 0.000% 100.157% ( 75000, 110000 ] 1 0.000% 100.158% ** Level 5 read latency histogram (micros): Count: 42695 Average: 598.9801 StdDev: 1302.07 Min: 84 Median: 216.2427 Max: 77668 Percentiles: P50: 216.24 P75: 438.42 P99: 6075.74 P99.9: 10705.57 P99.99: 32670.61 ------------------------------------------------------ ( 76, 110 ] 1163 2.724% 2.724% # ( 110, 170 ] 13740 32.182% 34.906% ###### ( 170, 250 ] 11149 26.113% 61.019% ##### ( 250, 380 ] 4850 11.360% 72.378% ## ( 380, 580 ] 3832 8.975% 81.354% ## ( 580, 870 ] 2012 4.712% 86.066% # ( 870, 1300 ] 1564 3.663% 89.729% # ( 1300, 1900 ] 1219 2.855% 92.585% # ( 1900, 2900 ] 995 2.330% 94.915% ( 2900, 4400 ] 1220 2.857% 97.773% # ( 4400, 6600 ] 688 1.611% 99.384% ( 6600, 9900 ] 215 0.504% 99.888% ( 9900, 14000 ] 27 0.063% 99.951% ( 14000, 22000 ] 8 0.019% 99.970% ( 22000, 33000 ] 9 0.021% 99.991% ( 33000, 50000 ] 4 0.009% 100.000% ( 75000, 110000 ] 1 0.002% 100.002% ** DB Stats ** Uptime(secs): 9.4 total, 9.2 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 104.19 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.337 H:M:S, 35.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 106.98 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.337 H:M:S, 36.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/24-23:59:17 ... thread 0: (1000000,2000000) ops and (216812.8,144839.7) ops/second in (4.612273,13.808371) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/12 3.69 GB 138.5 1.3 0.0 1.3 3.1 1.8 0.0 1.7 82.1 199.2 16.12 13.14 34 0.474 1343K 1392 0.0 0.0 L1 14/14 1.43 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 973.12 MB 1.0 1.3 0.6 0.7 1.3 0.5 0.1 2.3 98.0 96.9 13.54 8.67 7 1.934 221K 0 0.0 0.0 L4 160/0 9.72 GB 1.0 3.6 2.5 1.1 3.6 2.5 0.0 1.4 94.9 94.3 39.34 23.04 28 1.405 1622K 292 0.0 0.0 L5 1112/0 90.49 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1320/26 106.28 GB 0.0 6.2 3.1 3.1 8.0 4.9 2.6 4.4 92.5 119.3 69.00 44.85 69 1.000 3187K 1684 0.0 0.0 Int 0/0 0.00 KB 0.0 6.2 3.1 3.1 8.0 4.9 2.6 4.4 92.5 119.3 69.00 44.85 69 1.000 3187K 1684 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.2 3.1 3.1 6.2 3.0 0.0 0.0 107.3 106.6 59.51 38.12 40 1.488 3187K 1684 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 199.0 9.50 6.73 29 0.327 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.1 total, 13.8 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.04 GB write, 585.70 MB/s write, 6.23 GB read, 454.02 MB/s read, 69.0 seconds Interval compaction: 8.04 GB write, 596.21 MB/s write, 6.23 GB read, 462.16 MB/s read, 69.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 14, pending-compaction-bytes-stops: 0, total-delays: 14, total-stops: 13, interval: 26 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 789929 Average: 10.5177 StdDev: 128.79 Min: 1 Median: 2.2594 Max: 35934 Percentiles: P50: 2.26 P75: 3.00 P99: 207.20 P99.9: 566.27 P99.99: 1146.44 ------------------------------------------------------ [ 0, 1 ] 21116 2.673% 2.673% # ( 1, 2 ] 304453 38.542% 41.215% ######## ( 2, 3 ] 267525 33.867% 75.082% ####### ( 3, 4 ] 71041 8.993% 84.075% ## ( 4, 6 ] 73137 9.259% 93.334% ## ( 6, 10 ] 26226 3.320% 96.654% # ( 10, 15 ] 7703 0.975% 97.629% ( 15, 22 ] 1286 0.163% 97.792% ( 22, 34 ] 451 0.057% 97.849% ( 34, 51 ] 63 0.008% 97.857% ( 51, 76 ] 35 0.004% 97.861% ( 76, 110 ] 385 0.049% 97.910% ( 110, 170 ] 5724 0.725% 98.635% ( 170, 250 ] 6205 0.786% 99.420% ( 250, 380 ] 2394 0.303% 99.723% ( 380, 580 ] 1499 0.190% 99.913% ( 580, 870 ] 482 0.061% 99.974% ( 870, 1300 ] 196 0.025% 99.999% ( 1300, 1900 ] 161 0.020% 100.019% ( 1900, 2900 ] 101 0.013% 100.032% ( 2900, 4400 ] 171 0.022% 100.054% ( 4400, 6600 ] 53 0.007% 100.061% ( 6600, 9900 ] 16 0.002% 100.063% ( 9900, 14000 ] 3 0.000% 100.063% ( 14000, 22000 ] 6 0.001% 100.064% ( 22000, 33000 ] 4 0.001% 100.064% ( 33000, 50000 ] 1 0.000% 100.064% ** Level 1 read latency histogram (micros): Count: 31302 Average: 28.8030 StdDev: 354.64 Min: 1 Median: 2.6952 Max: 21658 Percentiles: P50: 2.70 P75: 4.49 P99: 379.99 P99.9: 2508.91 P99.99: 18695.84 ------------------------------------------------------ [ 0, 1 ] 281 0.898% 0.898% ( 1, 2 ] 7956 25.417% 26.315% ##### ( 2, 3 ] 10664 34.068% 60.383% ####### ( 3, 4 ] 3223 10.296% 70.679% ## ( 4, 6 ] 5484 17.520% 88.199% #### ( 6, 10 ] 1038 3.316% 91.515% # ( 10, 15 ] 434 1.386% 92.901% ( 15, 22 ] 149 0.476% 93.377% ( 22, 34 ] 35 0.112% 93.489% ( 34, 51 ] 4 0.013% 93.502% ( 76, 110 ] 63 0.201% 93.703% ( 110, 170 ] 784 2.505% 96.208% # ( 170, 250 ] 634 2.025% 98.233% ( 250, 380 ] 240 0.767% 99.000% ( 380, 580 ] 172 0.549% 99.550% ( 580, 870 ] 61 0.195% 99.744% ( 870, 1300 ] 29 0.093% 99.837% ( 1300, 1900 ] 13 0.042% 99.879% ( 1900, 2900 ] 11 0.035% 99.914% ( 2900, 4400 ] 5 0.016% 99.930% ( 4400, 6600 ] 7 0.022% 99.952% ( 6600, 9900 ] 5 0.016% 99.968% ( 9900, 14000 ] 1 0.003% 99.971% ( 14000, 22000 ] 10 0.032% 100.003% ** Level 2 read latency histogram (micros): Count: 16678 Average: 17.1853 StdDev: 195.04 Min: 1 Median: 2.8248 Max: 13320 Percentiles: P50: 2.82 P75: 4.74 P99: 207.36 P99.9: 3255.93 P99.99: 6306.17 ------------------------------------------------------ [ 0, 1 ] 28 0.168% 0.168% ( 1, 2 ] 3109 18.641% 18.809% #### ( 2, 3 ] 6307 37.816% 56.625% ######## ( 3, 4 ] 1822 10.925% 67.550% ## ( 4, 6 ] 3371 20.212% 87.762% #### ( 6, 10 ] 1364 8.178% 95.941% ## ( 10, 15 ] 338 2.027% 97.967% ( 15, 22 ] 46 0.276% 98.243% ( 22, 34 ] 13 0.078% 98.321% ( 34, 51 ] 6 0.036% 98.357% ( 76, 110 ] 2 0.012% 98.369% ( 110, 170 ] 80 0.480% 98.849% ( 170, 250 ] 54 0.324% 99.173% ( 250, 380 ] 28 0.168% 99.340% ( 380, 580 ] 24 0.144% 99.484% ( 580, 870 ] 18 0.108% 99.592% ( 870, 1300 ] 21 0.126% 99.718% ( 1300, 1900 ] 16 0.096% 99.814% ( 1900, 2900 ] 11 0.066% 99.880% ( 2900, 4400 ] 14 0.084% 99.964% ( 4400, 6600 ] 5 0.030% 99.994% ( 9900, 14000 ] 1 0.006% 100.000% ** Level 3 read latency histogram (micros): Count: 586907 Average: 11.4862 StdDev: 205.57 Min: 1 Median: 2.7130 Max: 52615 Percentiles: P50: 2.71 P75: 4.25 P99: 12.73 P99.9: 108.89 P99.99: 167.41 ------------------------------------------------------ [ 0, 1 ] 2267 0.386% 0.386% ( 1, 2 ] 131989 22.489% 22.875% #### ( 2, 3 ] 223289 38.045% 60.920% ######## ( 3, 4 ] 68838 11.729% 72.649% ## ( 4, 6 ] 109245 18.614% 91.263% #### ( 6, 10 ] 41230 7.025% 98.288% # ( 10, 15 ] 7662 1.305% 99.593% ( 15, 22 ] 1239 0.211% 99.804% ( 22, 34 ] 322 0.055% 99.859% ( 34, 51 ] 79 0.013% 99.873% ( 51, 76 ] 74 0.013% 99.885% ( 76, 110 ] 89 0.015% 99.900% ( 110, 170 ] 549 0.094% 99.994% ( 170, 250 ] 488 0.083% 100.077% ( 250, 380 ] 246 0.042% 100.119% ( 380, 580 ] 227 0.039% 100.158% ( 580, 870 ] 243 0.041% 100.199% ( 870, 1300 ] 338 0.058% 100.257% ( 1300, 1900 ] 254 0.043% 100.300% ( 1900, 2900 ] 213 0.036% 100.336% ( 2900, 4400 ] 317 0.054% 100.390% ( 4400, 6600 ] 139 0.024% 100.414% ( 6600, 9900 ] 47 0.008% 100.422% ( 9900, 14000 ] 11 0.002% 100.424% ( 14000, 22000 ] 11 0.002% 100.426% ( 22000, 33000 ] 4 0.001% 100.426% ( 33000, 50000 ] 1 0.000% 100.427% ( 50000, 75000 ] 1 0.000% 100.427% ** Level 4 read latency histogram (micros): Count: 300059 Average: 40.2810 StdDev: 406.60 Min: 1 Median: 2.7718 Max: 84370 Percentiles: P50: 2.77 P75: 4.61 P99: 690.54 P99.9: 3259.95 P99.99: 4025.69 ------------------------------------------------------ [ 0, 1 ] 1331 0.444% 0.444% ( 1, 2 ] 65491 21.826% 22.270% #### ( 2, 3 ] 107813 35.931% 58.200% ####### ( 3, 4 ] 34915 11.636% 69.836% ## ( 4, 6 ] 50842 16.944% 86.780% ### ( 6, 10 ] 18319 6.105% 92.885% # ( 10, 15 ] 4396 1.465% 94.350% ( 15, 22 ] 702 0.234% 94.584% ( 22, 34 ] 174 0.058% 94.642% ( 34, 51 ] 52 0.017% 94.660% ( 51, 76 ] 40 0.013% 94.673% ( 76, 110 ] 448 0.149% 94.822% ( 110, 170 ] 5134 1.711% 96.533% ( 170, 250 ] 4024 1.341% 97.874% ( 250, 380 ] 1616 0.539% 98.413% ( 380, 580 ] 1397 0.466% 98.879% ( 580, 870 ] 956 0.319% 99.197% ( 870, 1300 ] 835 0.278% 99.475% ( 1300, 1900 ] 664 0.221% 99.697% ( 1900, 2900 ] 483 0.161% 99.858% ( 2900, 4400 ] 529 0.176% 100.034% ( 4400, 6600 ] 253 0.084% 100.118% ( 6600, 9900 ] 71 0.024% 100.142% ( 9900, 14000 ] 8 0.003% 100.145% ( 14000, 22000 ] 12 0.004% 100.149% ( 22000, 33000 ] 8 0.003% 100.151% ( 33000, 50000 ] 3 0.001% 100.152% ( 50000, 75000 ] 1 0.000% 100.153% ( 75000, 110000 ] 1 0.000% 100.153% ** Level 5 read latency histogram (micros): Count: 99601 Average: 378.1245 StdDev: 878.24 Min: 83 Median: 192.7385 Max: 77668 Percentiles: P50: 192.74 P75: 288.25 P99: 4342.34 P99.9: 9077.29 P99.99: 23270.99 ------------------------------------------------------ ( 76, 110 ] 3098 3.110% 3.110% # ( 110, 170 ] 38146 38.299% 41.409% ######## ( 170, 250 ] 30104 30.225% 71.634% ###### ( 250, 380 ] 11396 11.442% 83.075% ## ( 380, 580 ] 7886 7.918% 90.993% ## ( 580, 870 ] 2954 2.966% 93.959% # ( 870, 1300 ] 1616 1.622% 95.581% ( 1300, 1900 ] 1226 1.231% 96.812% ( 1900, 2900 ] 1003 1.007% 97.819% ( 2900, 4400 ] 1223 1.228% 99.047% ( 4400, 6600 ] 688 0.691% 99.738% ( 6600, 9900 ] 215 0.216% 99.954% ( 9900, 14000 ] 27 0.027% 99.981% ( 14000, 22000 ] 8 0.008% 99.989% ( 22000, 33000 ] 9 0.009% 99.998% ( 33000, 50000 ] 4 0.004% 100.002% ( 75000, 110000 ] 1 0.001% 100.003% ** DB Stats ** Uptime(secs): 14.1 total, 4.6 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 140.01 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.337 H:M:S, 23.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 213.39 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/07/24-23:59:50 ... thread 0: (1000000,3000000) ops and (30207.8,63948.9) ops/second in (33.104066,46.912437) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/10 2.77 GB 107.0 2.5 0.0 2.5 5.3 2.8 0.0 1.9 88.5 185.5 29.19 23.96 54 0.541 2622K 2580 0.0 0.0 L1 6/0 754.43 MB 75.4 3.3 1.8 1.4 3.2 1.8 0.0 1.8 82.9 82.0 40.40 22.20 1 40.398 3402K 36K 0.0 0.0 L3 23/6 1.40 GB 1.0 2.9 1.4 1.5 2.9 1.4 1.8 2.0 105.7 104.7 27.92 19.55 16 1.745 1385K 4019 0.0 0.0 L4 172/18 10.25 GB 1.0 7.1 4.5 2.6 7.1 4.5 0.0 1.6 100.3 99.4 72.77 46.28 53 1.373 4344K 7533 0.0 0.0 L5 1128/0 91.97 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1343/34 107.11 GB 0.0 15.8 7.8 8.0 18.4 10.5 5.8 6.7 95.0 110.9 170.28 112.00 124 1.373 11M 50K 0.0 0.0 Int 0/0 0.00 KB 0.0 15.8 7.8 8.0 18.4 10.5 5.8 6.7 95.0 110.9 170.28 112.00 124 1.373 11M 50K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 15.8 7.8 8.0 15.7 7.7 0.0 0.0 103.9 103.1 155.67 102.13 80 1.946 11M 50K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 194.1 14.61 9.86 44 0.332 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 47.2 total, 46.9 interval Flush(GB): cumulative 2.769, interval 2.769 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 18.44 GB write, 400.45 MB/s write, 15.80 GB read, 343.08 MB/s read, 170.3 seconds Interval compaction: 18.44 GB write, 402.56 MB/s write, 15.80 GB read, 344.89 MB/s read, 170.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 2, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 2, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 50, pending-compaction-bytes-stops: 0, total-delays: 52, total-stops: 13, interval: 64 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1472122 Average: 31.0581 StdDev: 191.98 Min: 1 Median: 2.3229 Max: 48579 Percentiles: P50: 2.32 P75: 3.75 P99: 354.63 P99.9: 1217.03 P99.99: 2227.57 ------------------------------------------------------ [ 0, 1 ] 49634 3.372% 3.372% # ( 1, 2 ] 554473 37.665% 41.036% ######## ( 2, 3 ] 408607 27.756% 68.793% ###### ( 3, 4 ] 121102 8.226% 77.019% ## ( 4, 6 ] 116026 7.882% 84.901% ## ( 6, 10 ] 34653 2.354% 87.255% ( 10, 15 ] 10674 0.725% 87.980% ( 15, 22 ] 1750 0.119% 88.099% ( 22, 34 ] 605 0.041% 88.140% ( 34, 51 ] 72 0.005% 88.145% ( 51, 76 ] 68 0.005% 88.149% ( 76, 110 ] 5381 0.366% 88.515% ( 110, 170 ] 84331 5.729% 94.243% # ( 170, 250 ] 61429 4.173% 98.416% # ( 250, 380 ] 10691 0.726% 99.142% ( 380, 580 ] 6431 0.437% 99.579% ( 580, 870 ] 3163 0.215% 99.794% ( 870, 1300 ] 1944 0.132% 99.926% ( 1300, 1900 ] 805 0.055% 99.981% ( 1900, 2900 ] 442 0.030% 100.011% ( 2900, 4400 ] 532 0.036% 100.047% ( 4400, 6600 ] 181 0.012% 100.059% ( 6600, 9900 ] 43 0.003% 100.062% ( 9900, 14000 ] 8 0.001% 100.063% ( 14000, 22000 ] 16 0.001% 100.064% ( 22000, 33000 ] 19 0.001% 100.065% ( 33000, 50000 ] 4 0.000% 100.065% ** Level 1 read latency histogram (micros): Count: 870307 Average: 10.4229 StdDev: 184.56 Min: 1 Median: 2.6707 Max: 36637 Percentiles: P50: 2.67 P75: 4.24 P99: 133.07 P99.9: 362.51 P99.99: 576.52 ------------------------------------------------------ [ 0, 1 ] 19092 2.194% 2.194% ( 1, 2 ] 227907 26.187% 28.381% ##### ( 2, 3 ] 280532 32.234% 60.614% ###### ( 3, 4 ] 105637 12.138% 72.752% ## ( 4, 6 ] 160265 18.415% 91.167% #### ( 6, 10 ] 50078 5.754% 96.921% # ( 10, 15 ] 11379 1.307% 98.229% ( 15, 22 ] 3166 0.364% 98.592% ( 22, 34 ] 618 0.071% 98.663% ( 34, 51 ] 173 0.020% 98.683% ( 51, 76 ] 143 0.016% 98.700% ( 76, 110 ] 510 0.059% 98.758% ( 110, 170 ] 5473 0.629% 99.387% ( 170, 250 ] 3799 0.437% 99.824% ( 250, 380 ] 768 0.088% 99.912% ( 380, 580 ] 692 0.080% 99.991% ( 580, 870 ] 530 0.061% 100.052% ( 870, 1300 ] 483 0.055% 100.108% ( 1300, 1900 ] 202 0.023% 100.131% ( 1900, 2900 ] 133 0.015% 100.146% ( 2900, 4400 ] 177 0.020% 100.167% ( 4400, 6600 ] 64 0.007% 100.174% ( 6600, 9900 ] 23 0.003% 100.177% ( 9900, 14000 ] 13 0.001% 100.178% ( 14000, 22000 ] 35 0.004% 100.182% ( 22000, 33000 ] 10 0.001% 100.183% ( 33000, 50000 ] 2 0.000% 100.183% ** Level 2 read latency histogram (micros): Count: 45042 Average: 14.7027 StdDev: 226.08 Min: 1 Median: 2.7484 Max: 26398 Percentiles: P50: 2.75 P75: 4.48 P99: 202.64 P99.9: 1798.95 P99.99: 6415.13 ------------------------------------------------------ [ 0, 1 ] 151 0.335% 0.335% ( 1, 2 ] 9763 21.675% 22.011% #### ( 2, 3 ] 16845 37.398% 59.409% ####### ( 3, 4 ] 5073 11.263% 70.672% ## ( 4, 6 ] 8162 18.121% 88.793% #### ( 6, 10 ] 3139 6.969% 95.762% # ( 10, 15 ] 695 1.543% 97.305% ( 15, 22 ] 76 0.169% 97.473% ( 22, 34 ] 16 0.036% 97.509% ( 34, 51 ] 13 0.029% 97.538% ( 51, 76 ] 4 0.009% 97.547% ( 76, 110 ] 32 0.071% 97.618% ( 110, 170 ] 501 1.112% 98.730% ( 170, 250 ] 298 0.662% 99.392% ( 250, 380 ] 74 0.164% 99.556% ( 380, 580 ] 67 0.149% 99.705% ( 580, 870 ] 36 0.080% 99.785% ( 870, 1300 ] 32 0.071% 99.856% ( 1300, 1900 ] 24 0.053% 99.909% ( 1900, 2900 ] 13 0.029% 99.938% ( 2900, 4400 ] 18 0.040% 99.978% ( 4400, 6600 ] 6 0.013% 99.991% ( 9900, 14000 ] 1 0.002% 99.993% ( 14000, 22000 ] 1 0.002% 99.996% ( 22000, 33000 ] 2 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 1040219 Average: 11.1790 StdDev: 207.57 Min: 1 Median: 2.7418 Max: 52615 Percentiles: P50: 2.74 P75: 4.34 P99: 14.54 P99.9: 222.32 P99.99: 276.38 ------------------------------------------------------ [ 0, 1 ] 3123 0.300% 0.300% ( 1, 2 ] 217287 20.889% 21.189% #### ( 2, 3 ] 404043 38.842% 60.031% ######## ( 3, 4 ] 122083 11.736% 71.767% ## ( 4, 6 ] 196305 18.872% 90.639% #### ( 6, 10 ] 75206 7.230% 97.869% # ( 10, 15 ] 12970 1.247% 99.115% ( 15, 22 ] 1907 0.183% 99.299% ( 22, 34 ] 479 0.046% 99.345% ( 34, 51 ] 177 0.017% 99.362% ( 51, 76 ] 176 0.017% 99.379% ( 76, 110 ] 352 0.034% 99.413% ( 110, 170 ] 3534 0.340% 99.752% ( 170, 250 ] 2353 0.226% 99.978% ( 250, 380 ] 601 0.058% 100.036% ( 380, 580 ] 530 0.051% 100.087% ( 580, 870 ] 497 0.048% 100.135% ( 870, 1300 ] 581 0.056% 100.191% ( 1300, 1900 ] 360 0.035% 100.225% ( 1900, 2900 ] 277 0.027% 100.252% ( 2900, 4400 ] 397 0.038% 100.290% ( 4400, 6600 ] 173 0.017% 100.307% ( 6600, 9900 ] 60 0.006% 100.313% ( 9900, 14000 ] 19 0.002% 100.314% ( 14000, 22000 ] 33 0.003% 100.318% ( 22000, 33000 ] 13 0.001% 100.319% ( 33000, 50000 ] 2 0.000% 100.319% ( 50000, 75000 ] 1 0.000% 100.319% ** Level 4 read latency histogram (micros): Count: 837363 Average: 38.3241 StdDev: 347.99 Min: 1 Median: 2.8700 Max: 84370 Percentiles: P50: 2.87 P75: 5.10 P99: 461.18 P99.9: 1682.14 P99.99: 2346.30 ------------------------------------------------------ [ 0, 1 ] 2336 0.279% 0.279% ( 1, 2 ] 160036 19.112% 19.391% #### ( 2, 3 ] 294599 35.182% 54.573% ####### ( 3, 4 ] 92582 11.056% 65.629% ## ( 4, 6 ] 142108 16.971% 82.600% ### ( 6, 10 ] 52945 6.323% 88.923% # ( 10, 15 ] 12252 1.463% 90.386% ( 15, 22 ] 1818 0.217% 90.603% ( 22, 34 ] 502 0.060% 90.663% ( 34, 51 ] 205 0.024% 90.687% ( 51, 76 ] 227 0.027% 90.715% ( 76, 110 ] 2729 0.326% 91.040% ( 110, 170 ] 37837 4.519% 95.559% # ( 170, 250 ] 22950 2.741% 98.300% # ( 250, 380 ] 4426 0.529% 98.828% ( 380, 580 ] 3551 0.424% 99.252% ( 580, 870 ] 2681 0.320% 99.573% ( 870, 1300 ] 2003 0.239% 99.812% ( 1300, 1900 ] 1166 0.139% 99.951% ( 1900, 2900 ] 740 0.088% 100.039% ( 2900, 4400 ] 904 0.108% 100.147% ( 4400, 6600 ] 372 0.044% 100.192% ( 6600, 9900 ] 110 0.013% 100.205% ( 9900, 14000 ] 34 0.004% 100.209% ( 14000, 22000 ] 44 0.005% 100.214% ( 22000, 33000 ] 31 0.004% 100.218% ( 33000, 50000 ] 7 0.001% 100.219% ( 50000, 75000 ] 1 0.000% 100.219% ( 75000, 110000 ] 1 0.000% 100.219% ** Level 5 read latency histogram (micros): Count: 503497 Average: 244.8568 StdDev: 554.96 Min: 83 Median: 164.0310 Max: 77668 Percentiles: P50: 164.03 P75: 220.70 P99: 1874.72 P99.9: 6340.57 P99.99: 21259.63 ------------------------------------------------------ ( 76, 110 ] 18059 3.587% 3.587% # ( 110, 170 ] 259506 51.541% 55.127% ########## ( 170, 250 ] 157896 31.360% 86.487% ###### ( 250, 380 ] 28314 5.623% 92.111% # ( 380, 580 ] 18587 3.692% 95.802% # ( 580, 870 ] 8504 1.689% 97.491% ( 870, 1300 ] 5050 1.003% 98.494% ( 1300, 1900 ] 2658 0.528% 99.022% ( 1900, 2900 ] 1632 0.324% 99.346% ( 2900, 4400 ] 1946 0.386% 99.733% ( 4400, 6600 ] 954 0.189% 99.922% ( 6600, 9900 ] 262 0.052% 99.974% ( 9900, 14000 ] 36 0.007% 99.982% ( 14000, 22000 ] 47 0.009% 99.991% ( 22000, 33000 ] 37 0.007% 99.998% ( 33000, 50000 ] 14 0.003% 100.001% ( 75000, 110000 ] 1 0.000% 100.001% ** DB Stats ** Uptime(secs): 47.2 total, 33.1 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 62.60 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:30.169 H:M:S, 64.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 29.73 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:26.832 H:M:S, 81.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 8 num-running-flushes: 1 2024/07/24-23:59:56 ... thread 0: (1000000,4000000) ops and (177277.0,76113.2) ops/second in (5.640890,52.553327) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/10 3.75 GB 207.7 3.5 0.0 3.5 7.3 3.7 0.0 1.9 93.9 194.3 38.25 31.72 74 0.517 3646K 3553 0.0 0.0 L1 6/0 754.43 MB 75.4 3.3 1.8 1.4 3.2 1.8 0.0 1.8 82.9 82.0 40.40 22.20 1 40.398 3402K 36K 0.0 0.0 L3 17/0 965.66 MB 1.0 2.9 1.4 1.5 2.9 1.4 1.8 2.0 105.7 104.7 27.92 19.55 16 1.745 1385K 4019 0.0 0.0 L4 167/0 9.57 GB 1.0 8.5 5.0 3.5 8.4 4.9 0.0 1.7 102.5 101.6 84.66 55.48 59 1.435 5554K 16K 0.0 0.0 L5 1140/0 93.07 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1348/10 108.08 GB 0.0 18.1 8.3 9.9 21.7 11.9 6.9 5.8 97.1 116.4 191.23 128.95 150 1.275 13M 60K 0.0 0.0 Int 0/0 0.00 KB 0.0 18.1 8.3 9.9 21.7 11.9 6.9 5.8 97.1 116.4 191.23 128.95 150 1.275 13M 60K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 18.1 8.3 9.9 18.0 8.1 0.0 0.0 107.7 106.9 172.41 115.97 90 1.916 13M 60K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 204.2 18.82 12.98 60 0.314 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 52.8 total, 52.6 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 21.74 GB write, 421.67 MB/s write, 18.14 GB read, 351.73 MB/s read, 191.2 seconds Interval compaction: 21.74 GB write, 423.66 MB/s write, 18.14 GB read, 353.39 MB/s read, 191.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 3, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 3, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 50, pending-compaction-bytes-stops: 0, total-delays: 53, total-stops: 13, interval: 65 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1761986 Average: 30.6987 StdDev: 187.37 Min: 0 Median: 2.2275 Max: 48579 Percentiles: P50: 2.23 P75: 3.57 P99: 365.80 P99.9: 1188.66 P99.99: 2173.07 ------------------------------------------------------ [ 0, 1 ] 72862 4.135% 4.135% # ( 1, 2 ] 700679 39.766% 43.902% ######## ( 2, 3 ] 472305 26.805% 70.707% ##### ( 3, 4 ] 132527 7.521% 78.228% ## ( 4, 6 ] 127764 7.251% 85.480% # ( 6, 10 ] 37293 2.117% 87.596% ( 10, 15 ] 11100 0.630% 88.226% ( 15, 22 ] 1786 0.101% 88.327% ( 22, 34 ] 614 0.035% 88.362% ( 34, 51 ] 77 0.004% 88.367% ( 51, 76 ] 81 0.005% 88.371% ( 76, 110 ] 6005 0.341% 88.712% ( 110, 170 ] 95818 5.438% 94.150% # ( 170, 250 ] 72908 4.138% 98.288% # ( 250, 380 ] 14087 0.799% 99.087% ( 380, 580 ] 8669 0.492% 99.579% ( 580, 870 ] 4010 0.228% 99.807% ( 870, 1300 ] 2213 0.126% 99.933% ( 1300, 1900 ] 882 0.050% 99.983% ( 1900, 2900 ] 479 0.027% 100.010% ( 2900, 4400 ] 624 0.035% 100.045% ( 4400, 6600 ] 189 0.011% 100.056% ( 6600, 9900 ] 47 0.003% 100.059% ( 9900, 14000 ] 10 0.001% 100.059% ( 14000, 22000 ] 20 0.001% 100.060% ( 22000, 33000 ] 22 0.001% 100.062% ( 33000, 50000 ] 4 0.000% 100.062% ** Level 1 read latency histogram (micros): Count: 896571 Average: 10.6575 StdDev: 187.05 Min: 1 Median: 2.6656 Max: 36637 Percentiles: P50: 2.67 P75: 4.23 P99: 140.11 P99.9: 387.51 P99.99: 606.91 ------------------------------------------------------ [ 0, 1 ] 19187 2.140% 2.140% ( 1, 2 ] 235726 26.292% 28.432% ##### ( 2, 3 ] 290504 32.402% 60.834% ###### ( 3, 4 ] 108242 12.073% 72.907% ## ( 4, 6 ] 163697 18.258% 91.165% #### ( 6, 10 ] 50908 5.678% 96.843% # ( 10, 15 ] 11697 1.305% 98.147% ( 15, 22 ] 3190 0.356% 98.503% ( 22, 34 ] 618 0.069% 98.572% ( 34, 51 ] 176 0.020% 98.592% ( 51, 76 ] 147 0.016% 98.608% ( 76, 110 ] 535 0.060% 98.668% ( 110, 170 ] 5934 0.662% 99.330% ( 170, 250 ] 4181 0.466% 99.796% ( 250, 380 ] 903 0.101% 99.897% ( 380, 580 ] 784 0.087% 99.984% ( 580, 870 ] 564 0.063% 100.047% ( 870, 1300 ] 495 0.055% 100.102% ( 1300, 1900 ] 207 0.023% 100.125% ( 1900, 2900 ] 136 0.015% 100.141% ( 2900, 4400 ] 183 0.020% 100.161% ( 4400, 6600 ] 65 0.007% 100.168% ( 6600, 9900 ] 24 0.003% 100.171% ( 9900, 14000 ] 14 0.002% 100.172% ( 14000, 22000 ] 39 0.004% 100.177% ( 22000, 33000 ] 10 0.001% 100.178% ( 33000, 50000 ] 2 0.000% 100.178% ** Level 2 read latency histogram (micros): Count: 45042 Average: 14.7027 StdDev: 226.08 Min: 1 Median: 2.7484 Max: 26398 Percentiles: P50: 2.75 P75: 4.48 P99: 202.64 P99.9: 1798.95 P99.99: 6415.13 ------------------------------------------------------ [ 0, 1 ] 151 0.335% 0.335% ( 1, 2 ] 9763 21.675% 22.011% #### ( 2, 3 ] 16845 37.398% 59.409% ####### ( 3, 4 ] 5073 11.263% 70.672% ## ( 4, 6 ] 8162 18.121% 88.793% #### ( 6, 10 ] 3139 6.969% 95.762% # ( 10, 15 ] 695 1.543% 97.305% ( 15, 22 ] 76 0.169% 97.473% ( 22, 34 ] 16 0.036% 97.509% ( 34, 51 ] 13 0.029% 97.538% ( 51, 76 ] 4 0.009% 97.547% ( 76, 110 ] 32 0.071% 97.618% ( 110, 170 ] 501 1.112% 98.730% ( 170, 250 ] 298 0.662% 99.392% ( 250, 380 ] 74 0.164% 99.556% ( 380, 580 ] 67 0.149% 99.705% ( 580, 870 ] 36 0.080% 99.785% ( 870, 1300 ] 32 0.071% 99.856% ( 1300, 1900 ] 24 0.053% 99.909% ( 1900, 2900 ] 13 0.029% 99.938% ( 2900, 4400 ] 18 0.040% 99.978% ( 4400, 6600 ] 6 0.013% 99.991% ( 9900, 14000 ] 1 0.002% 99.993% ( 14000, 22000 ] 1 0.002% 99.996% ( 22000, 33000 ] 2 0.004% 100.000% ** Level 3 read latency histogram (micros): Count: 1103570 Average: 10.9911 StdDev: 205.10 Min: 1 Median: 2.7491 Max: 52615 Percentiles: P50: 2.75 P75: 4.36 P99: 14.56 P99.9: 225.70 P99.99: 297.31 ------------------------------------------------------ [ 0, 1 ] 3291 0.298% 0.298% ( 1, 2 ] 229674 20.812% 21.110% #### ( 2, 3 ] 425611 38.567% 59.677% ######## ( 3, 4 ] 131601 11.925% 71.602% ## ( 4, 6 ] 210845 19.106% 90.708% #### ( 6, 10 ] 78957 7.155% 97.862% # ( 10, 15 ] 13761 1.247% 99.109% ( 15, 22 ] 2003 0.182% 99.291% ( 22, 34 ] 525 0.048% 99.338% ( 34, 51 ] 183 0.017% 99.355% ( 51, 76 ] 187 0.017% 99.372% ( 76, 110 ] 364 0.033% 99.405% ( 110, 170 ] 3723 0.337% 99.742% ( 170, 250 ] 2501 0.227% 99.969% ( 250, 380 ] 642 0.058% 100.027% ( 380, 580 ] 584 0.053% 100.080% ( 580, 870 ] 529 0.048% 100.128% ( 870, 1300 ] 599 0.054% 100.182% ( 1300, 1900 ] 365 0.033% 100.215% ( 1900, 2900 ] 279 0.025% 100.240% ( 2900, 4400 ] 400 0.036% 100.277% ( 4400, 6600 ] 173 0.016% 100.292% ( 6600, 9900 ] 60 0.005% 100.298% ( 9900, 14000 ] 22 0.002% 100.300% ( 14000, 22000 ] 36 0.003% 100.303% ( 22000, 33000 ] 13 0.001% 100.304% ( 33000, 50000 ] 2 0.000% 100.304% ( 50000, 75000 ] 1 0.000% 100.305% ** Level 4 read latency histogram (micros): Count: 979515 Average: 35.9465 StdDev: 331.55 Min: 1 Median: 2.8556 Max: 84370 Percentiles: P50: 2.86 P75: 5.02 P99: 423.36 P99.9: 1500.58 P99.99: 2021.44 ------------------------------------------------------ [ 0, 1 ] 2825 0.288% 0.288% ( 1, 2 ] 192542 19.657% 19.945% #### ( 2, 3 ] 344073 35.127% 55.072% ####### ( 3, 4 ] 109620 11.191% 66.263% ## ( 4, 6 ] 167117 17.061% 83.325% ### ( 6, 10 ] 59305 6.055% 89.379% # ( 10, 15 ] 14169 1.447% 90.826% ( 15, 22 ] 2175 0.222% 91.048% ( 22, 34 ] 646 0.066% 91.114% ( 34, 51 ] 243 0.025% 91.138% ( 51, 76 ] 253 0.026% 91.164% ( 76, 110 ] 2913 0.297% 91.462% ( 110, 170 ] 41375 4.224% 95.686% # ( 170, 250 ] 26135 2.668% 98.354% # ( 250, 380 ] 5403 0.552% 98.905% ( 380, 580 ] 4271 0.436% 99.342% ( 580, 870 ] 2977 0.304% 99.645% ( 870, 1300 ] 2097 0.214% 99.860% ( 1300, 1900 ] 1186 0.121% 99.981% ( 1900, 2900 ] 758 0.077% 100.058% ( 2900, 4400 ] 942 0.096% 100.154% ( 4400, 6600 ] 376 0.038% 100.193% ( 6600, 9900 ] 116 0.012% 100.204% ( 9900, 14000 ] 38 0.004% 100.208% ( 14000, 22000 ] 52 0.005% 100.214% ( 22000, 33000 ] 32 0.003% 100.217% ( 33000, 50000 ] 7 0.001% 100.218% ( 50000, 75000 ] 1 0.000% 100.218% ( 75000, 110000 ] 1 0.000% 100.218% ** Level 5 read latency histogram (micros): Count: 560860 Average: 244.8562 StdDev: 541.22 Min: 83 Median: 165.4577 Max: 77668 Percentiles: P50: 165.46 P75: 223.25 P99: 1803.55 P99.9: 6252.95 P99.99: 20909.83 ------------------------------------------------------ ( 76, 110 ] 19287 3.439% 3.439% # ( 110, 170 ] 282532 50.375% 53.814% ########## ( 170, 250 ] 178505 31.827% 85.641% ###### ( 250, 380 ] 34430 6.139% 91.779% # ( 380, 580 ] 22629 4.035% 95.814% # ( 580, 870 ] 10025 1.787% 97.602% ( 870, 1300 ] 5518 0.984% 98.585% ( 1300, 1900 ] 2772 0.494% 99.080% ( 1900, 2900 ] 1714 0.306% 99.385% ( 2900, 4400 ] 2072 0.369% 99.755% ( 4400, 6600 ] 969 0.173% 99.927% ( 6600, 9900 ] 266 0.047% 99.975% ( 9900, 14000 ] 41 0.007% 99.982% ( 14000, 22000 ] 52 0.009% 99.991% ( 22000, 33000 ] 41 0.007% 99.999% ( 33000, 50000 ] 14 0.002% 100.001% ( 75000, 110000 ] 1 0.000% 100.001% ** DB Stats ** Uptime(secs): 52.8 total, 5.6 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 74.55 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:30.431 H:M:S, 57.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 174.52 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.262 H:M:S, 4.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 301.167 micros/op 13254 ops/sec 60.208 seconds 797996 operations; 12.8 MB/s (199999 of 199999 found) Microseconds per read: Count: 797996 Average: 301.1694 StdDev: 954.70 Min: 2 Median: 214.1765 Max: 115395 Percentiles: P50: 214.18 P75: 273.55 P99: 1393.20 P99.9: 15928.35 P99.99: 36142.53 ------------------------------------------------------ ( 1, 2 ] 3 0.000% 0.000% ( 2, 3 ] 33 0.004% 0.005% ( 3, 4 ] 56 0.007% 0.012% ( 4, 6 ] 77 0.010% 0.021% ( 6, 10 ] 193 0.024% 0.045% ( 10, 15 ] 1839 0.230% 0.276% ( 15, 22 ] 2919 0.366% 0.642% ( 22, 34 ] 5770 0.723% 1.365% ( 34, 51 ] 2909 0.365% 1.729% ( 51, 76 ] 505 0.063% 1.792% ( 76, 110 ] 734 0.092% 1.884% ( 110, 170 ] 164421 20.604% 22.489% #### ( 170, 250 ] 397567 49.821% 72.309% ########## ( 250, 380 ] 118538 14.854% 87.164% ### ( 380, 580 ] 64033 8.024% 95.188% ## ( 580, 870 ] 22228 2.785% 97.974% # ( 870, 1300 ] 7694 0.964% 98.938% ( 1300, 1900 ] 3200 0.401% 99.339% ( 1900, 2900 ] 1614 0.202% 99.541% ( 2900, 4400 ] 1533 0.192% 99.733% ( 4400, 6600 ] 652 0.082% 99.815% ( 6600, 9900 ] 256 0.032% 99.847% ( 9900, 14000 ] 303 0.038% 99.885% ( 14000, 22000 ] 502 0.063% 99.948% ( 22000, 33000 ] 325 0.041% 99.988% ( 33000, 50000 ] 66 0.008% 99.997% ( 50000, 75000 ] 18 0.002% 99.999% ( 75000, 110000 ] 7 0.001% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 13331585 rocksdb.bloom.filter.full.positive COUNT : 927848 rocksdb.bloom.filter.full.true.positive COUNT : 797564 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 432 rocksdb.memtable.miss COUNT : 797564 rocksdb.l0.hit COUNT : 32195 rocksdb.l1.hit COUNT : 9905 rocksdb.l2andup.hit COUNT : 755464 rocksdb.compaction.key.drop.new COUNT : 162728 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 4640669 rocksdb.number.keys.read COUNT : 797996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 4789170408 rocksdb.bytes.read COUNT : 797996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1647 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 34004063 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 22023542552 rocksdb.compact.write.bytes COUNT : 21843811000 rocksdb.flush.write.bytes COUNT : 4756539074 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 32 rocksdb.number.superversion_acquires COUNT : 1049 rocksdb.number.superversion_releases COUNT : 1005 rocksdb.number.superversion_cleanups COUNT : 257 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 155 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 4717816832 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1504256 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 24933829859 rocksdb.non.last.level.read.count COUNT : 5882865 rocksdb.block.checksum.compute.count COUNT : 6272210 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 213.539238 P95 : 574.792345 P99 : 1391.332289 P100 : 115393.000000 COUNT : 797996 SUM : 239667557 rocksdb.db.write.micros P50 : 2.207509 P95 : 8.175436 P99 : 14.817173 P100 : 636866.000000 COUNT : 4640669 SUM : 51875809 rocksdb.compaction.times.micros P50 : 1430263.157895 P95 : 2717500.000000 P99 : 34120000.000000 P100 : 40398272.000000 COUNT : 93 SUM : 175826878 rocksdb.compaction.times.cpu_micros P50 : 1058333.333333 P95 : 1884833.333333 P99 : 22200207.000000 P100 : 22200207.000000 COUNT : 93 SUM : 119333284 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 1094.555556 P95 : 5995.000000 P99 : 11175.000000 P100 : 11175.000000 COUNT : 71 SUM : 112128 rocksdb.compaction.outfile.sync.micros P50 : 3609.090909 P95 : 12268.888889 P99 : 25080.000000 P100 : 80740.000000 COUNT : 272 SUM : 1326720 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 351.111111 P95 : 4934.285714 P99 : 8954.000000 P100 : 60497.000000 COUNT : 186 SUM : 247618 rocksdb.table.open.io.micros P50 : 9185.820896 P95 : 23051.875000 P99 : 32110.375000 P100 : 106193.000000 COUNT : 1647 SUM : 16941547 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 5.317460 P95 : 14.470932 P99 : 230.426955 P100 : 52624.000000 COUNT : 5333819 SUM : 98111758 rocksdb.read.block.get.micros P50 : 167.118964 P95 : 478.944880 P99 : 1277.094306 P100 : 84604.000000 COUNT : 937160 SUM : 206522075 rocksdb.write.raw.block.micros P50 : 0.599698 P95 : 1.871444 P99 : 5.119439 P100 : 56438.000000 COUNT : 6446039 SUM : 23208156 rocksdb.numfiles.in.singlecompaction P50 : 2.793103 P95 : 4.320000 P99 : 9.040000 P100 : 18.000000 COUNT : 124 SUM : 408 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.563282 P95 : 1187.840648 P99 : 2576.695881 P100 : 636461.000000 COUNT : 219645 SUM : 34023107 rocksdb.sst.read.micros P50 : 2.748075 P95 : 217.901729 P99 : 519.445906 P100 : 84370.000000 COUNT : 5883564 SUM : 281414354 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 797996 SUM : 797996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 4640669 SUM : 4789170408 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 239268.292683 P95 : 736750.000000 P99 : 958600.000000 P100 : 1074825.000000 COUNT : 71 SUM : 21625283 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 297 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0