[2024-12-08 07:48:58.436912] Starting SPDK v24.09.1-pre git sha1 b18e1bd62 / DPDK 23.11.0 initialization... [2024-12-08 07:48:58.437053] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1459015 ] Set seed to 1733640538433430 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-08 07:48:58.540625] app.c: 917:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-08 07:48:58.581041] app.c: 610:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-08 07:48:58.581085] app.c: 611:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1459015' to capture a snapshot of events at runtime. [2024-12-08 07:48:58.581096] app.c: 616:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-08 07:48:58.581107] app.c: 617:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-08 07:48:58.581114] app.c: 618:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1459015 for offline analysis/debug. [2024-12-08 07:48:58.581196] reactor.c: 990:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sun Dec 8 07:49:02 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/08-07:49:10 ... thread 0: (1000000,1000000) ops and (138436.4,138436.4) ops/second in (7.223535,7.223535) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/13 1.96 GB 37.7 0.4 0.0 0.4 1.2 0.8 0.0 1.5 40.7 128.9 9.28 3.91 10 0.928 383K 562 0.0 0.0 L1 7/7 776.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 128.78 MB 1.3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 29/8 2.29 GB 1.4 2.3 1.5 0.8 2.3 1.5 0.0 1.5 103.8 102.9 23.11 11.56 12 1.926 484K 0 0.0 0.0 L4 196/23 11.43 GB 1.1 3.2 2.1 1.1 3.2 2.1 0.0 1.5 107.8 106.5 30.81 15.60 27 1.141 1206K 0 0.0 0.0 L5 1088/0 89.57 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1335/51 106.13 GB 0.0 6.0 3.6 2.3 6.7 4.4 0.0 8.4 96.5 108.5 63.20 31.07 49 1.290 2074K 562 0.0 0.0 Int 0/0 0.00 KB 0.0 6.0 3.6 2.3 6.7 4.4 0.0 8.4 96.5 108.5 63.20 31.07 49 1.290 2074K 562 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.0 3.6 2.3 5.9 3.6 0.0 0.0 107.7 106.7 56.58 28.66 40 1.414 2074K 562 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 123.6 6.62 2.41 9 0.736 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.4 total, 7.2 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.69 GB write, 928.49 MB/s write, 5.95 GB read, 825.62 MB/s read, 63.2 seconds Interval compaction: 6.69 GB write, 948.78 MB/s write, 5.95 GB read, 843.66 MB/s read, 63.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 33, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 33, interval: 33 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 211231 Average: 21.7946 StdDev: 209.55 Min: 1 Median: 1.9077 Max: 10460 Percentiles: P50: 1.91 P75: 2.93 P99: 630.65 P99.9: 3296.74 P99.99: 4393.52 ------------------------------------------------------ [ 0, 1 ] 7712 3.651% 3.651% # ( 1, 2 ] 107854 51.060% 54.711% ########## ( 2, 3 ] 46297 21.918% 76.628% #### ( 3, 4 ] 20364 9.641% 86.269% ## ( 4, 6 ] 20574 9.740% 96.009% ## ( 6, 10 ] 4545 2.152% 98.161% ( 10, 15 ] 843 0.399% 98.560% ( 15, 22 ] 106 0.050% 98.610% ( 22, 34 ] 18 0.009% 98.619% ( 34, 51 ] 8 0.004% 98.622% ( 51, 76 ] 2 0.001% 98.623% ( 76, 110 ] 3 0.001% 98.625% ( 110, 170 ] 31 0.015% 98.639% ( 170, 250 ] 62 0.029% 98.669% ( 250, 380 ] 188 0.089% 98.758% ( 380, 580 ] 402 0.190% 98.948% ( 580, 870 ] 628 0.297% 99.245% ( 870, 1300 ] 629 0.298% 99.543% ( 1300, 1900 ] 408 0.193% 99.736% ( 1900, 2900 ] 277 0.131% 99.867% ( 2900, 4400 ] 260 0.123% 99.991% ( 4400, 6600 ] 62 0.029% 100.020% ( 6600, 9900 ] 10 0.005% 100.025% ( 9900, 14000 ] 1 0.000% 100.025% ** Level 1 read latency histogram (micros): Count: 34535 Average: 25.0239 StdDev: 439.36 Min: 1 Median: 1.9580 Max: 34692 Percentiles: P50: 1.96 P75: 2.94 P99: 407.56 P99.9: 3984.14 P99.99: 26252.87 ------------------------------------------------------ [ 0, 1 ] 796 2.305% 2.305% ( 1, 2 ] 17193 49.784% 52.089% ########## ( 2, 3 ] 8395 24.309% 76.398% ##### ( 3, 4 ] 3296 9.544% 85.942% ## ( 4, 6 ] 3363 9.738% 95.680% ## ( 6, 10 ] 914 2.647% 98.326% # ( 10, 15 ] 172 0.498% 98.824% ( 15, 22 ] 19 0.055% 98.879% ( 22, 34 ] 3 0.009% 98.888% ( 34, 51 ] 2 0.006% 98.894% ( 76, 110 ] 1 0.003% 98.897% ( 110, 170 ] 1 0.003% 98.900% ( 170, 250 ] 10 0.029% 98.929% ( 250, 380 ] 19 0.055% 98.984% ( 380, 580 ] 41 0.119% 99.102% ( 580, 870 ] 66 0.191% 99.293% ( 870, 1300 ] 81 0.235% 99.528% ( 1300, 1900 ] 62 0.180% 99.708% ( 1900, 2900 ] 39 0.113% 99.820% ( 2900, 4400 ] 38 0.110% 99.931% ( 4400, 6600 ] 15 0.043% 99.974% ( 6600, 9900 ] 1 0.003% 99.977% ( 9900, 14000 ] 2 0.006% 99.983% ( 14000, 22000 ] 1 0.003% 99.986% ( 22000, 33000 ] 4 0.012% 99.997% ( 33000, 50000 ] 1 0.003% 100.000% ** Level 2 read latency histogram (micros): Count: 213865 Average: 15.2589 StdDev: 226.32 Min: 1 Median: 1.9828 Max: 33987 Percentiles: P50: 1.98 P75: 3.02 P99: 13.04 P99.9: 2961.16 P99.99: 5384.54 ------------------------------------------------------ [ 0, 1 ] 4696 2.196% 2.196% ( 1, 2 ] 104030 48.643% 50.839% ########## ( 2, 3 ] 51169 23.926% 74.764% ##### ( 3, 4 ] 22467 10.505% 85.270% ## ( 4, 6 ] 22598 10.566% 95.836% ## ( 6, 10 ] 6021 2.815% 98.651% # ( 10, 15 ] 1224 0.572% 99.224% ( 15, 22 ] 171 0.080% 99.304% ( 22, 34 ] 22 0.010% 99.314% ( 34, 51 ] 12 0.006% 99.320% ( 51, 76 ] 15 0.007% 99.327% ( 76, 110 ] 8 0.004% 99.330% ( 110, 170 ] 23 0.011% 99.341% ( 170, 250 ] 37 0.017% 99.358% ( 250, 380 ] 68 0.032% 99.390% ( 380, 580 ] 165 0.077% 99.467% ( 580, 870 ] 221 0.103% 99.571% ( 870, 1300 ] 263 0.123% 99.694% ( 1300, 1900 ] 241 0.113% 99.806% ( 1900, 2900 ] 193 0.090% 99.897% ( 2900, 4400 ] 175 0.082% 99.978% ( 4400, 6600 ] 55 0.026% 100.004% ( 6600, 9900 ] 20 0.009% 100.014% ( 9900, 14000 ] 2 0.001% 100.014% ( 14000, 22000 ] 5 0.002% 100.017% ( 33000, 50000 ] 2 0.001% 100.018% ** Level 3 read latency histogram (micros): Count: 771271 Average: 11.5198 StdDev: 186.57 Min: 1 Median: 1.9624 Max: 37113 Percentiles: P50: 1.96 P75: 2.97 P99: 9.66 P99.9: 832.66 P99.99: 1230.42 ------------------------------------------------------ [ 0, 1 ] 16543 2.145% 2.145% ( 1, 2 ] 383504 49.724% 51.869% ########## ( 2, 3 ] 184804 23.961% 75.830% ##### ( 3, 4 ] 81287 10.539% 86.369% ## ( 4, 6 ] 80657 10.458% 96.827% ## ( 6, 10 ] 18332 2.377% 99.203% ( 10, 15 ] 3478 0.451% 99.654% ( 15, 22 ] 458 0.059% 99.714% ( 22, 34 ] 32 0.004% 99.718% ( 34, 51 ] 32 0.004% 99.722% ( 51, 76 ] 43 0.006% 99.728% ( 76, 110 ] 33 0.004% 99.732% ( 110, 170 ] 43 0.006% 99.737% ( 170, 250 ] 89 0.012% 99.749% ( 250, 380 ] 205 0.027% 99.776% ( 380, 580 ] 404 0.052% 99.828% ( 580, 870 ] 639 0.083% 99.911% ( 870, 1300 ] 730 0.095% 100.005% ( 1300, 1900 ] 562 0.073% 100.078% ( 1900, 2900 ] 477 0.062% 100.140% ( 2900, 4400 ] 460 0.060% 100.200% ( 4400, 6600 ] 137 0.018% 100.218% ( 6600, 9900 ] 41 0.005% 100.223% ( 9900, 14000 ] 5 0.001% 100.224% ( 14000, 22000 ] 12 0.002% 100.225% ( 22000, 33000 ] 6 0.001% 100.226% ( 33000, 50000 ] 1 0.000% 100.226% ** Level 4 read latency histogram (micros): Count: 386772 Average: 24.2569 StdDev: 251.25 Min: 1 Median: 1.9750 Max: 54370 Percentiles: P50: 1.98 P75: 3.02 P99: 627.85 P99.9: 2561.35 P99.99: 3341.26 ------------------------------------------------------ [ 0, 1 ] 7830 2.024% 2.024% ( 1, 2 ] 190311 49.205% 51.229% ########## ( 2, 3 ] 90976 23.522% 74.751% ##### ( 3, 4 ] 41447 10.716% 85.467% ## ( 4, 6 ] 39486 10.209% 95.677% ## ( 6, 10 ] 9514 2.460% 98.136% ( 10, 15 ] 1699 0.439% 98.576% ( 15, 22 ] 215 0.056% 98.631% ( 22, 34 ] 19 0.005% 98.636% ( 34, 51 ] 26 0.007% 98.643% ( 51, 76 ] 20 0.005% 98.648% ( 76, 110 ] 14 0.004% 98.652% ( 110, 170 ] 45 0.012% 98.663% ( 170, 250 ] 134 0.035% 98.698% ( 250, 380 ] 299 0.077% 98.775% ( 380, 580 ] 669 0.173% 98.948% ( 580, 870 ] 1226 0.317% 99.265% ( 870, 1300 ] 1253 0.324% 99.589% ( 1300, 1900 ] 818 0.211% 99.801% ( 1900, 2900 ] 584 0.151% 99.952% ( 2900, 4400 ] 511 0.132% 100.084% ( 4400, 6600 ] 151 0.039% 100.123% ( 6600, 9900 ] 28 0.007% 100.130% ( 9900, 14000 ] 1 0.000% 100.130% ( 14000, 22000 ] 2 0.001% 100.131% ( 22000, 33000 ] 2 0.001% 100.131% ( 50000, 75000 ] 1 0.000% 100.132% ** Level 5 read latency histogram (micros): Count: 11568 Average: 1762.9321 StdDev: 2025.03 Min: 81 Median: 1186.7521 Max: 80396 Percentiles: P50: 1186.75 P75: 2397.72 P99: 7068.31 P99.9: 20485.33 P99.99: 73040.00 ------------------------------------------------------ ( 76, 110 ] 9 0.078% 0.078% ( 110, 170 ] 44 0.380% 0.458% ( 170, 250 ] 177 1.530% 1.988% ( 250, 380 ] 537 4.642% 6.630% # ( 380, 580 ] 1217 10.520% 17.151% ## ( 580, 870 ] 2133 18.439% 35.590% #### ( 870, 1300 ] 2263 19.563% 55.152% #### ( 1300, 1900 ] 1642 14.194% 69.346% ### ( 1900, 2900 ] 1314 11.359% 80.705% ## ( 2900, 4400 ] 1457 12.595% 93.300% ### ( 4400, 6600 ] 643 5.558% 98.859% # ( 6600, 9900 ] 115 0.994% 99.853% ( 9900, 14000 ] 3 0.026% 99.879% ( 14000, 22000 ] 3 0.026% 99.905% ( 22000, 33000 ] 7 0.061% 99.965% ( 33000, 50000 ] 1 0.009% 99.974% ( 50000, 75000 ] 2 0.017% 99.991% ( 75000, 110000 ] 1 0.009% 100.000% ** DB Stats ** Uptime(secs): 7.4 total, 7.2 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 133.30 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.228 H:M:S, 43.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 136.19 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.228 H:M:S, 44.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/12/08-07:49:14 ... thread 0: (1000000,2000000) ops and (220683.7,170141.7) ops/second in (4.531372,11.754907) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/8 3.01 GB 188.8 1.5 0.0 1.5 3.4 1.8 0.0 1.8 85.3 187.5 18.46 11.38 30 0.615 1599K 2079 0.0 0.0 L1 7/7 776.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 953.47 MB 1.0 2.5 1.6 0.9 2.5 1.6 0.0 1.5 105.4 104.6 24.24 12.28 13 1.864 513K 0 0.0 0.0 L4 166/0 9.62 GB 1.0 5.7 3.5 2.2 5.6 3.4 0.0 1.6 119.3 117.9 48.58 26.91 39 1.246 2696K 172 0.0 0.0 L5 1128/0 92.83 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1334/15 107.15 GB 0.0 9.7 5.1 4.6 11.5 6.9 3.3 6.2 108.8 128.5 91.28 50.57 82 1.113 4808K 2251 0.0 0.0 Int 0/0 0.00 KB 0.0 9.7 5.1 4.6 11.5 6.9 3.3 6.2 108.8 128.5 91.28 50.57 82 1.113 4808K 2251 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.7 5.1 4.6 9.6 5.0 0.0 0.0 123.4 122.3 80.44 45.22 57 1.411 4808K 2251 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 174.3 10.84 5.35 25 0.434 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 11.9 total, 11.8 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.45 GB write, 984.10 MB/s write, 9.70 GB read, 833.29 MB/s read, 91.3 seconds Interval compaction: 11.45 GB write, 997.31 MB/s write, 9.70 GB read, 844.48 MB/s read, 91.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 3, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 3, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 43, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 3, total-stops: 43, interval: 46 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 534516 Average: 20.6703 StdDev: 168.99 Min: 1 Median: 1.6950 Max: 10460 Percentiles: P50: 1.70 P75: 2.55 P99: 473.32 P99.9: 2420.44 P99.99: 3909.95 ------------------------------------------------------ [ 0, 1 ] 66825 12.502% 12.502% ### ( 1, 2 ] 288382 53.952% 66.454% ########### ( 2, 3 ] 83041 15.536% 81.990% ### ( 3, 4 ] 35446 6.631% 88.621% # ( 4, 6 ] 32540 6.088% 94.709% # ( 6, 10 ] 7233 1.353% 96.062% ( 10, 15 ] 1241 0.232% 96.294% ( 15, 22 ] 170 0.032% 96.326% ( 22, 34 ] 20 0.004% 96.330% ( 34, 51 ] 11 0.002% 96.332% ( 51, 76 ] 6 0.001% 96.333% ( 76, 110 ] 1773 0.332% 96.665% ( 110, 170 ] 6432 1.203% 97.868% ( 170, 250 ] 3295 0.616% 98.484% ( 250, 380 ] 1825 0.341% 98.826% ( 380, 580 ] 1996 0.373% 99.199% ( 580, 870 ] 1590 0.297% 99.497% ( 870, 1300 ] 1246 0.233% 99.730% ( 1300, 1900 ] 669 0.125% 99.855% ( 1900, 2900 ] 464 0.087% 99.942% ( 2900, 4400 ] 384 0.072% 100.014% ( 4400, 6600 ] 97 0.018% 100.032% ( 6600, 9900 ] 10 0.002% 100.034% ( 9900, 14000 ] 1 0.000% 100.034% ** Level 1 read latency histogram (micros): Count: 51253 Average: 21.3124 StdDev: 364.91 Min: 1 Median: 2.2698 Max: 34692 Percentiles: P50: 2.27 P75: 3.62 P99: 289.31 P99.9: 3470.92 P99.99: 20997.60 ------------------------------------------------------ [ 0, 1 ] 941 1.836% 1.836% ( 1, 2 ] 21192 41.348% 43.184% ######## ( 2, 3 ] 12948 25.263% 68.447% ##### ( 3, 4 ] 5416 10.567% 79.014% ## ( 4, 6 ] 7418 14.473% 93.487% ### ( 6, 10 ] 1961 3.826% 97.313% # ( 10, 15 ] 373 0.728% 98.041% ( 15, 22 ] 119 0.232% 98.273% ( 22, 34 ] 13 0.025% 98.299% ( 34, 51 ] 2 0.004% 98.303% ( 76, 110 ] 60 0.117% 98.420% ( 110, 170 ] 191 0.373% 98.792% ( 170, 250 ] 85 0.166% 98.958% ( 250, 380 ] 71 0.139% 99.097% ( 380, 580 ] 85 0.166% 99.262% ( 580, 870 ] 103 0.201% 99.463% ( 870, 1300 ] 94 0.183% 99.647% ( 1300, 1900 ] 71 0.139% 99.785% ( 1900, 2900 ] 42 0.082% 99.867% ( 2900, 4400 ] 44 0.086% 99.953% ( 4400, 6600 ] 15 0.029% 99.982% ( 6600, 9900 ] 1 0.002% 99.984% ( 9900, 14000 ] 2 0.004% 99.988% ( 14000, 22000 ] 1 0.002% 99.990% ( 22000, 33000 ] 4 0.008% 99.998% ( 33000, 50000 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 246576 Average: 14.8309 StdDev: 218.21 Min: 1 Median: 1.9887 Max: 33987 Percentiles: P50: 1.99 P75: 3.02 P99: 12.79 P99.9: 2955.96 P99.99: 5443.02 ------------------------------------------------------ [ 0, 1 ] 5030 2.040% 2.040% ( 1, 2 ] 119608 48.508% 50.547% ########## ( 2, 3 ] 59876 24.283% 74.830% ##### ( 3, 4 ] 25912 10.509% 85.339% ## ( 4, 6 ] 26036 10.559% 95.898% ## ( 6, 10 ] 6893 2.795% 98.694% # ( 10, 15 ] 1353 0.549% 99.242% ( 15, 22 ] 184 0.075% 99.317% ( 22, 34 ] 23 0.009% 99.326% ( 34, 51 ] 15 0.006% 99.332% ( 51, 76 ] 17 0.007% 99.339% ( 76, 110 ] 9 0.004% 99.343% ( 110, 170 ] 26 0.011% 99.354% ( 170, 250 ] 42 0.017% 99.371% ( 250, 380 ] 81 0.033% 99.403% ( 380, 580 ] 190 0.077% 99.480% ( 580, 870 ] 257 0.104% 99.585% ( 870, 1300 ] 290 0.118% 99.702% ( 1300, 1900 ] 261 0.106% 99.808% ( 1900, 2900 ] 219 0.089% 99.897% ( 2900, 4400 ] 199 0.081% 99.978% ( 4400, 6600 ] 64 0.026% 100.004% ( 6600, 9900 ] 20 0.008% 100.012% ( 9900, 14000 ] 2 0.001% 100.013% ( 14000, 22000 ] 5 0.002% 100.015% ( 33000, 50000 ] 2 0.001% 100.015% ** Level 3 read latency histogram (micros): Count: 1000539 Average: 10.6192 StdDev: 170.72 Min: 1 Median: 1.9567 Max: 37113 Percentiles: P50: 1.96 P75: 2.95 P99: 9.65 P99.9: 786.42 P99.99: 1209.13 ------------------------------------------------------ [ 0, 1 ] 20043 2.003% 2.003% ( 1, 2 ] 501971 50.170% 52.173% ########## ( 2, 3 ] 241033 24.090% 76.264% ##### ( 3, 4 ] 103210 10.315% 86.579% ## ( 4, 6 ] 102795 10.274% 96.853% ## ( 6, 10 ] 23514 2.350% 99.203% ( 10, 15 ] 4265 0.426% 99.629% ( 15, 22 ] 577 0.058% 99.687% ( 22, 34 ] 44 0.004% 99.691% ( 34, 51 ] 42 0.004% 99.696% ( 51, 76 ] 57 0.006% 99.701% ( 76, 110 ] 90 0.009% 99.710% ( 110, 170 ] 266 0.027% 99.737% ( 170, 250 ] 216 0.022% 99.759% ( 250, 380 ] 316 0.032% 99.790% ( 380, 580 ] 535 0.053% 99.844% ( 580, 870 ] 793 0.079% 99.923% ( 870, 1300 ] 852 0.085% 100.008% ( 1300, 1900 ] 661 0.066% 100.074% ( 1900, 2900 ] 546 0.055% 100.129% ( 2900, 4400 ] 539 0.054% 100.183% ( 4400, 6600 ] 151 0.015% 100.198% ( 6600, 9900 ] 42 0.004% 100.202% ( 9900, 14000 ] 5 0.000% 100.202% ( 14000, 22000 ] 12 0.001% 100.203% ( 22000, 33000 ] 6 0.001% 100.204% ( 33000, 50000 ] 1 0.000% 100.204% ** Level 4 read latency histogram (micros): Count: 571260 Average: 22.2084 StdDev: 222.44 Min: 1 Median: 1.9755 Max: 54370 Percentiles: P50: 1.98 P75: 3.04 P99: 485.20 P99.9: 2213.01 P99.99: 2950.83 ------------------------------------------------------ [ 0, 1 ] 11097 1.943% 1.943% ( 1, 2 ] 281425 49.264% 51.206% ########## ( 2, 3 ] 133789 23.420% 74.626% ##### ( 3, 4 ] 59979 10.499% 85.126% ## ( 4, 6 ] 57404 10.049% 95.175% ## ( 6, 10 ] 13628 2.386% 97.560% ( 10, 15 ] 2349 0.411% 97.971% ( 15, 22 ] 316 0.055% 98.027% ( 22, 34 ] 30 0.005% 98.032% ( 34, 51 ] 36 0.006% 98.038% ( 51, 76 ] 45 0.008% 98.046% ( 76, 110 ] 565 0.099% 98.145% ( 110, 170 ] 2133 0.373% 98.518% ( 170, 250 ] 1225 0.214% 98.733% ( 250, 380 ] 851 0.149% 98.882% ( 380, 580 ] 1284 0.225% 99.107% ( 580, 870 ] 1719 0.301% 99.407% ( 870, 1300 ] 1603 0.281% 99.688% ( 1300, 1900 ] 986 0.173% 99.861% ( 1900, 2900 ] 718 0.126% 99.986% ( 2900, 4400 ] 616 0.108% 100.094% ( 4400, 6600 ] 165 0.029% 100.123% ( 6600, 9900 ] 29 0.005% 100.128% ( 9900, 14000 ] 1 0.000% 100.128% ( 14000, 22000 ] 2 0.000% 100.129% ( 22000, 33000 ] 2 0.000% 100.129% ( 50000, 75000 ] 1 0.000% 100.129% ** Level 5 read latency histogram (micros): Count: 49149 Average: 657.5617 StdDev: 1225.78 Min: 74 Median: 225.2452 Max: 80396 Percentiles: P50: 225.25 P75: 691.88 P99: 5455.72 P99.9: 8880.41 P99.99: 25276.59 ------------------------------------------------------ ( 51, 76 ] 3 0.006% 0.006% ( 76, 110 ] 4322 8.794% 8.800% ## ( 110, 170 ] 15236 31.000% 39.799% ###### ( 170, 250 ] 7260 14.771% 54.571% ### ( 250, 380 ] 3937 8.010% 62.581% ## ( 380, 580 ] 4507 9.170% 71.751% ## ( 580, 870 ] 4139 8.421% 80.173% ## ( 870, 1300 ] 3427 6.973% 87.145% # ( 1300, 1900 ] 2135 4.344% 91.489% # ( 1900, 2900 ] 1660 3.377% 94.867% # ( 2900, 4400 ] 1698 3.455% 98.321% # ( 4400, 6600 ] 695 1.414% 99.735% ( 6600, 9900 ] 117 0.238% 99.974% ( 9900, 14000 ] 3 0.006% 99.980% ( 14000, 22000 ] 3 0.006% 99.986% ( 22000, 33000 ] 7 0.014% 100.000% ( 33000, 50000 ] 1 0.002% 100.002% ( 50000, 75000 ] 2 0.004% 100.006% ( 75000, 110000 ] 1 0.002% 100.008% ** DB Stats ** Uptime(secs): 11.9 total, 4.5 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 165.21 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.142 H:M:S, 34.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 217.20 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.914 H:M:S, 20.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/08-07:49:23 ... thread 0: (1000000,3000000) ops and (114260.9,146292.9) ops/second in (8.751897,20.506804) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/12 3.92 GB 257.1 3.7 0.0 3.7 6.4 2.8 0.0 2.3 127.0 222.0 29.74 21.97 49 0.607 3836K 11K 0.0 0.0 L1 7/7 776.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 953.47 MB 1.0 2.5 1.6 0.9 2.5 1.6 0.0 1.5 105.4 104.6 24.24 12.28 13 1.864 513K 0 0.0 0.0 L4 166/0 9.62 GB 1.0 5.7 3.5 2.2 5.6 3.4 0.0 1.6 119.3 117.9 48.58 26.91 39 1.246 2696K 172 0.0 0.0 L5 1128/0 92.83 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1334/19 108.06 GB 0.0 11.8 5.1 6.7 14.5 7.8 3.3 5.2 118.3 144.9 102.56 61.16 101 1.015 7046K 11K 0.0 0.0 Int 0/0 0.00 KB 0.0 11.8 5.1 6.7 14.5 7.8 3.3 5.2 118.3 144.9 102.56 61.16 101 1.015 7046K 11K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.8 5.1 6.7 11.7 5.0 0.0 0.0 136.6 135.4 88.83 53.32 61 1.456 7046K 11K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 206.5 13.73 7.84 40 0.343 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.7 total, 20.5 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.52 GB write, 719.24 MB/s write, 11.85 GB read, 587.03 MB/s read, 102.6 seconds Interval compaction: 14.52 GB write, 724.77 MB/s write, 11.85 GB read, 591.55 MB/s read, 102.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 11, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 11, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 43, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 11, total-stops: 43, interval: 54 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1202984 Average: 19.7714 StdDev: 122.64 Min: 0 Median: 1.5368 Max: 10460 Percentiles: P50: 1.54 P75: 2.06 P99: 313.21 P99.9: 1236.79 P99.99: 2302.65 ------------------------------------------------------ [ 0, 1 ] 261146 21.708% 21.708% #### ( 1, 2 ] 634024 52.704% 74.412% ########### ( 2, 3 ] 122569 10.189% 84.601% ## ( 3, 4 ] 49074 4.079% 88.681% # ( 4, 6 ] 39270 3.264% 91.945% # ( 6, 10 ] 9050 0.752% 92.697% ( 10, 15 ] 1799 0.150% 92.847% ( 15, 22 ] 243 0.020% 92.867% ( 22, 34 ] 25 0.002% 92.869% ( 34, 51 ] 12 0.001% 92.870% ( 51, 76 ] 24 0.002% 92.872% ( 76, 110 ] 12134 1.009% 93.881% ( 110, 170 ] 43512 3.617% 97.498% # ( 170, 250 ] 15460 1.285% 98.783% ( 250, 380 ] 5374 0.447% 99.230% ( 380, 580 ] 4627 0.385% 99.614% ( 580, 870 ] 2296 0.191% 99.805% ( 870, 1300 ] 1340 0.111% 99.916% ( 1300, 1900 ] 688 0.057% 99.974% ( 1900, 2900 ] 491 0.041% 100.014% ( 2900, 4400 ] 400 0.033% 100.048% ( 4400, 6600 ] 99 0.008% 100.056% ( 6600, 9900 ] 10 0.001% 100.057% ( 9900, 14000 ] 1 0.000% 100.057% ** Level 1 read latency histogram (micros): Count: 52585 Average: 24.9680 StdDev: 361.30 Min: 1 Median: 2.3212 Max: 34692 Percentiles: P50: 2.32 P75: 3.80 P99: 384.60 P99.9: 3425.51 P99.99: 19932.00 ------------------------------------------------------ [ 0, 1 ] 941 1.789% 1.789% ( 1, 2 ] 21192 40.300% 42.090% ######## ( 2, 3 ] 12948 24.623% 66.713% ##### ( 3, 4 ] 5416 10.300% 77.012% ## ( 4, 6 ] 7418 14.107% 91.119% ### ( 6, 10 ] 1961 3.729% 94.848% # ( 10, 15 ] 373 0.709% 95.558% ( 15, 22 ] 119 0.226% 95.784% ( 22, 34 ] 13 0.025% 95.809% ( 34, 51 ] 2 0.004% 95.812% ( 76, 110 ] 334 0.635% 96.448% ( 110, 170 ] 890 1.692% 98.140% ( 170, 250 ] 324 0.616% 98.756% ( 250, 380 ] 125 0.238% 98.994% ( 380, 580 ] 137 0.261% 99.255% ( 580, 870 ] 116 0.221% 99.475% ( 870, 1300 ] 95 0.181% 99.656% ( 1300, 1900 ] 71 0.135% 99.791% ( 1900, 2900 ] 42 0.080% 99.871% ( 2900, 4400 ] 44 0.084% 99.954% ( 4400, 6600 ] 15 0.029% 99.983% ( 6600, 9900 ] 1 0.002% 99.985% ( 9900, 14000 ] 2 0.004% 99.989% ( 14000, 22000 ] 1 0.002% 99.990% ( 22000, 33000 ] 4 0.008% 99.998% ( 33000, 50000 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 246576 Average: 14.8309 StdDev: 218.21 Min: 1 Median: 1.9887 Max: 33987 Percentiles: P50: 1.99 P75: 3.02 P99: 12.79 P99.9: 2955.96 P99.99: 5443.02 ------------------------------------------------------ [ 0, 1 ] 5030 2.040% 2.040% ( 1, 2 ] 119608 48.508% 50.547% ########## ( 2, 3 ] 59876 24.283% 74.830% ##### ( 3, 4 ] 25912 10.509% 85.339% ## ( 4, 6 ] 26036 10.559% 95.898% ## ( 6, 10 ] 6893 2.795% 98.694% # ( 10, 15 ] 1353 0.549% 99.242% ( 15, 22 ] 184 0.075% 99.317% ( 22, 34 ] 23 0.009% 99.326% ( 34, 51 ] 15 0.006% 99.332% ( 51, 76 ] 17 0.007% 99.339% ( 76, 110 ] 9 0.004% 99.343% ( 110, 170 ] 26 0.011% 99.354% ( 170, 250 ] 42 0.017% 99.371% ( 250, 380 ] 81 0.033% 99.403% ( 380, 580 ] 190 0.077% 99.480% ( 580, 870 ] 257 0.104% 99.585% ( 870, 1300 ] 290 0.118% 99.702% ( 1300, 1900 ] 261 0.106% 99.808% ( 1900, 2900 ] 219 0.089% 99.897% ( 2900, 4400 ] 199 0.081% 99.978% ( 4400, 6600 ] 64 0.026% 100.004% ( 6600, 9900 ] 20 0.008% 100.012% ( 9900, 14000 ] 2 0.001% 100.013% ( 14000, 22000 ] 5 0.002% 100.015% ( 33000, 50000 ] 2 0.001% 100.015% ** Level 3 read latency histogram (micros): Count: 1002411 Average: 10.9153 StdDev: 170.76 Min: 1 Median: 1.9585 Max: 37113 Percentiles: P50: 1.96 P75: 2.95 P99: 9.96 P99.9: 788.88 P99.99: 1209.35 ------------------------------------------------------ [ 0, 1 ] 20043 1.999% 1.999% ( 1, 2 ] 501976 50.077% 52.076% ########## ( 2, 3 ] 241051 24.047% 76.123% ##### ( 3, 4 ] 103216 10.297% 86.420% ## ( 4, 6 ] 102802 10.255% 96.676% ## ( 6, 10 ] 23516 2.346% 99.022% ( 10, 15 ] 4266 0.426% 99.447% ( 15, 22 ] 577 0.058% 99.505% ( 22, 34 ] 44 0.004% 99.509% ( 34, 51 ] 42 0.004% 99.513% ( 51, 76 ] 57 0.006% 99.519% ( 76, 110 ] 403 0.040% 99.559% ( 110, 170 ] 1249 0.125% 99.684% ( 170, 250 ] 562 0.056% 99.740% ( 250, 380 ] 400 0.040% 99.780% ( 380, 580 ] 619 0.062% 99.842% ( 580, 870 ] 813 0.081% 99.923% ( 870, 1300 ] 855 0.085% 100.008% ( 1300, 1900 ] 661 0.066% 100.074% ( 1900, 2900 ] 546 0.054% 100.128% ( 2900, 4400 ] 539 0.054% 100.182% ( 4400, 6600 ] 151 0.015% 100.197% ( 6600, 9900 ] 42 0.004% 100.201% ( 9900, 14000 ] 5 0.000% 100.202% ( 14000, 22000 ] 12 0.001% 100.203% ( 22000, 33000 ] 6 0.001% 100.204% ( 33000, 50000 ] 1 0.000% 100.204% ** Level 4 read latency histogram (micros): Count: 591267 Average: 26.5815 StdDev: 220.90 Min: 1 Median: 2.0211 Max: 54370 Percentiles: P50: 2.02 P75: 3.27 P99: 517.25 P99.9: 2188.70 P99.99: 2943.52 ------------------------------------------------------ [ 0, 1 ] 11098 1.877% 1.877% ( 1, 2 ] 281695 47.643% 49.520% ########## ( 2, 3 ] 134530 22.753% 72.272% ##### ( 3, 4 ] 60499 10.232% 82.505% ## ( 4, 6 ] 57973 9.805% 92.309% ## ( 6, 10 ] 13689 2.315% 94.625% ( 10, 15 ] 2367 0.400% 95.025% ( 15, 22 ] 317 0.054% 95.079% ( 22, 34 ] 30 0.005% 95.084% ( 34, 51 ] 36 0.006% 95.090% ( 51, 76 ] 48 0.008% 95.098% ( 76, 110 ] 3715 0.628% 95.726% ( 110, 170 ] 12046 2.037% 97.763% ( 170, 250 ] 4206 0.711% 98.475% ( 250, 380 ] 1752 0.296% 98.771% ( 380, 580 ] 1972 0.334% 99.105% ( 580, 870 ] 1876 0.317% 99.422% ( 870, 1300 ] 1622 0.274% 99.696% ( 1300, 1900 ] 996 0.168% 99.865% ( 1900, 2900 ] 723 0.122% 99.987% ( 2900, 4400 ] 616 0.104% 100.091% ( 4400, 6600 ] 165 0.028% 100.119% ( 6600, 9900 ] 29 0.005% 100.124% ( 9900, 14000 ] 1 0.000% 100.124% ( 14000, 22000 ] 2 0.000% 100.124% ( 22000, 33000 ] 2 0.000% 100.125% ( 50000, 75000 ] 1 0.000% 100.125% ** Level 5 read latency histogram (micros): Count: 175073 Average: 307.9424 StdDev: 691.71 Min: 73 Median: 153.2283 Max: 80396 Percentiles: P50: 153.23 P75: 226.54 P99: 3581.90 P99.9: 6450.99 P99.99: 9716.46 ------------------------------------------------------ ( 51, 76 ] 18 0.010% 0.010% ( 76, 110 ] 26429 15.096% 15.106% ### ( 110, 170 ] 84791 48.432% 63.538% ########## ( 170, 250 ] 28394 16.218% 79.756% ### ( 250, 380 ] 10555 6.029% 85.785% # ( 380, 580 ] 9429 5.386% 91.171% # ( 580, 870 ] 5460 3.119% 94.290% # ( 870, 1300 ] 3602 2.057% 96.347% ( 1300, 1900 ] 2168 1.238% 97.586% ( 1900, 2900 ] 1703 0.973% 98.558% ( 2900, 4400 ] 1701 0.972% 99.530% ( 4400, 6600 ] 695 0.397% 99.927% ( 6600, 9900 ] 117 0.067% 99.994% ( 9900, 14000 ] 3 0.002% 99.995% ( 14000, 22000 ] 3 0.002% 99.997% ( 22000, 33000 ] 7 0.004% 100.001% ( 33000, 50000 ] 1 0.001% 100.002% ( 50000, 75000 ] 2 0.001% 100.003% ( 75000, 110000 ] 1 0.001% 100.003% ** DB Stats ** Uptime(secs): 20.7 total, 8.8 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 142.87 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.219 H:M:S, 44.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 112.45 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.078 H:M:S, 58.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/08-07:49:43 ... thread 0: (1000000,4000000) ops and (49277.6,98039.2) ops/second in (20.293192,40.799996) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/8 4.96 GB 388.2 6.5 0.0 6.5 10.2 3.8 0.0 2.7 151.3 240.1 43.71 35.29 72 0.607 6713K 23K 0.0 0.0 L1 7/7 776.22 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 953.47 MB 1.0 2.5 1.6 0.9 2.5 1.6 0.0 1.5 105.4 104.6 24.24 12.28 13 1.864 513K 0 0.0 0.0 L4 166/0 9.62 GB 1.0 5.7 3.5 2.2 5.6 3.4 0.0 1.6 119.3 117.9 48.58 26.91 39 1.246 2696K 172 0.0 0.0 L5 1128/0 92.83 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1330/15 109.09 GB 0.0 14.6 5.1 9.5 18.3 8.8 3.3 4.8 128.4 161.0 116.53 74.48 124 0.940 9923K 23K 0.0 0.0 Int 0/0 0.00 KB 0.0 14.6 5.1 9.5 18.3 8.8 3.3 4.8 128.4 161.0 116.53 74.48 124 0.940 9923K 23K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.6 5.1 9.5 14.5 5.0 0.0 0.0 150.6 149.4 99.39 63.69 67 1.483 9923K 23K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 227.9 17.14 10.79 57 0.301 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 41.0 total, 40.8 interval Flush(GB): cumulative 3.814, interval 3.814 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 18.32 GB write, 457.93 MB/s write, 14.61 GB read, 365.36 MB/s read, 116.5 seconds Interval compaction: 18.32 GB write, 459.70 MB/s write, 14.61 GB read, 366.78 MB/s read, 116.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 19, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 19, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 43, pending-compaction-bytes-delays: 4, pending-compaction-bytes-stops: 0, total-delays: 23, total-stops: 43, interval: 66 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2088372 Average: 24.2057 StdDev: 105.33 Min: 0 Median: 1.5116 Max: 10460 Percentiles: P50: 1.51 P75: 2.12 P99: 280.68 P99.9: 875.56 P99.99: 1685.01 ------------------------------------------------------ [ 0, 1 ] 517851 24.797% 24.797% ##### ( 1, 2 ] 1028759 49.261% 74.058% ########## ( 2, 3 ] 162753 7.793% 81.851% ## ( 3, 4 ] 66504 3.184% 85.036% # ( 4, 6 ] 49097 2.351% 87.387% ( 6, 10 ] 10557 0.506% 87.892% ( 10, 15 ] 2407 0.115% 88.008% ( 15, 22 ] 314 0.015% 88.023% ( 22, 34 ] 28 0.001% 88.024% ( 34, 51 ] 14 0.001% 88.025% ( 51, 76 ] 56 0.003% 88.027% ( 76, 110 ] 45382 2.173% 90.201% ( 110, 170 ] 144048 6.898% 97.098% # ( 170, 250 ] 37325 1.787% 98.885% ( 250, 380 ] 10143 0.486% 99.371% ( 380, 580 ] 7917 0.379% 99.750% ( 580, 870 ] 3111 0.149% 99.899% ( 870, 1300 ] 1440 0.069% 99.968% ( 1300, 1900 ] 714 0.034% 100.002% ( 1900, 2900 ] 556 0.027% 100.029% ( 2900, 4400 ] 452 0.022% 100.051% ( 4400, 6600 ] 101 0.005% 100.055% ( 6600, 9900 ] 10 0.000% 100.056% ( 9900, 14000 ] 1 0.000% 100.056% ** Level 1 read latency histogram (micros): Count: 90041 Average: 21.4765 StdDev: 278.10 Min: 1 Median: 1.8599 Max: 34692 Percentiles: P50: 1.86 P75: 3.01 P99: 244.33 P99.9: 2387.42 P99.99: 6599.40 ------------------------------------------------------ [ 0, 1 ] 8392 9.320% 9.320% ## ( 1, 2 ] 42598 47.310% 56.630% ######### ( 2, 3 ] 16502 18.327% 74.957% #### ( 3, 4 ] 6003 6.667% 81.624% # ( 4, 6 ] 7692 8.543% 90.167% ## ( 6, 10 ] 2417 2.684% 92.851% # ( 10, 15 ] 387 0.430% 93.281% ( 15, 22 ] 119 0.132% 93.413% ( 22, 34 ] 13 0.014% 93.427% ( 34, 51 ] 2 0.002% 93.430% ( 76, 110 ] 1255 1.394% 94.823% ( 110, 170 ] 3073 3.413% 98.236% # ( 170, 250 ] 740 0.822% 99.058% ( 250, 380 ] 217 0.241% 99.299% ( 380, 580 ] 216 0.240% 99.539% ( 580, 870 ] 136 0.151% 99.690% ( 870, 1300 ] 96 0.107% 99.797% ( 1300, 1900 ] 72 0.080% 99.877% ( 1900, 2900 ] 43 0.048% 99.924% ( 2900, 4400 ] 44 0.049% 99.973% ( 4400, 6600 ] 15 0.017% 99.990% ( 6600, 9900 ] 1 0.001% 99.991% ( 9900, 14000 ] 2 0.002% 99.993% ( 14000, 22000 ] 1 0.001% 99.994% ( 22000, 33000 ] 4 0.004% 99.999% ( 33000, 50000 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 246576 Average: 14.8309 StdDev: 218.21 Min: 1 Median: 1.9887 Max: 33987 Percentiles: P50: 1.99 P75: 3.02 P99: 12.79 P99.9: 2955.96 P99.99: 5443.02 ------------------------------------------------------ [ 0, 1 ] 5030 2.040% 2.040% ( 1, 2 ] 119608 48.508% 50.547% ########## ( 2, 3 ] 59876 24.283% 74.830% ##### ( 3, 4 ] 25912 10.509% 85.339% ## ( 4, 6 ] 26036 10.559% 95.898% ## ( 6, 10 ] 6893 2.795% 98.694% # ( 10, 15 ] 1353 0.549% 99.242% ( 15, 22 ] 184 0.075% 99.317% ( 22, 34 ] 23 0.009% 99.326% ( 34, 51 ] 15 0.006% 99.332% ( 51, 76 ] 17 0.007% 99.339% ( 76, 110 ] 9 0.004% 99.343% ( 110, 170 ] 26 0.011% 99.354% ( 170, 250 ] 42 0.017% 99.371% ( 250, 380 ] 81 0.033% 99.403% ( 380, 580 ] 190 0.077% 99.480% ( 580, 870 ] 257 0.104% 99.585% ( 870, 1300 ] 290 0.118% 99.702% ( 1300, 1900 ] 261 0.106% 99.808% ( 1900, 2900 ] 219 0.089% 99.897% ( 2900, 4400 ] 199 0.081% 99.978% ( 4400, 6600 ] 64 0.026% 100.004% ( 6600, 9900 ] 20 0.008% 100.012% ( 9900, 14000 ] 2 0.001% 100.013% ( 14000, 22000 ] 5 0.002% 100.015% ( 33000, 50000 ] 2 0.001% 100.015% ** Level 3 read latency histogram (micros): Count: 1007248 Average: 11.5874 StdDev: 170.72 Min: 1 Median: 1.9634 Max: 37113 Percentiles: P50: 1.96 P75: 2.97 P99: 18.65 P99.9: 791.86 P99.99: 1211.03 ------------------------------------------------------ [ 0, 1 ] 20043 1.990% 1.990% ( 1, 2 ] 501977 49.836% 51.826% ########## ( 2, 3 ] 241053 23.932% 75.758% ##### ( 3, 4 ] 103216 10.247% 86.006% ## ( 4, 6 ] 102804 10.206% 96.212% ## ( 6, 10 ] 23516 2.335% 98.547% ( 10, 15 ] 4266 0.424% 98.970% ( 15, 22 ] 577 0.057% 99.027% ( 22, 34 ] 44 0.004% 99.032% ( 34, 51 ] 42 0.004% 99.036% ( 51, 76 ] 58 0.006% 99.042% ( 76, 110 ] 1384 0.137% 99.179% ( 110, 170 ] 4275 0.424% 99.604% ( 170, 250 ] 1162 0.115% 99.719% ( 250, 380 ] 509 0.051% 99.769% ( 380, 580 ] 704 0.070% 99.839% ( 580, 870 ] 836 0.083% 99.922% ( 870, 1300 ] 859 0.085% 100.008% ( 1300, 1900 ] 663 0.066% 100.073% ( 1900, 2900 ] 547 0.054% 100.128% ( 2900, 4400 ] 539 0.054% 100.181% ( 4400, 6600 ] 151 0.015% 100.196% ( 6600, 9900 ] 42 0.004% 100.200% ( 9900, 14000 ] 5 0.000% 100.201% ( 14000, 22000 ] 12 0.001% 100.202% ( 22000, 33000 ] 6 0.001% 100.203% ( 33000, 50000 ] 1 0.000% 100.203% ** Level 4 read latency histogram (micros): Count: 642754 Average: 36.5487 StdDev: 215.89 Min: 1 Median: 2.2121 Max: 54370 Percentiles: P50: 2.21 P75: 3.90 P99: 522.50 P99.9: 2131.21 P99.99: 2938.10 ------------------------------------------------------ [ 0, 1 ] 11098 1.727% 1.727% ( 1, 2 ] 281729 43.832% 45.558% ######### ( 2, 3 ] 134617 20.944% 66.502% #### ( 3, 4 ] 60575 9.424% 75.926% ## ( 4, 6 ] 58071 9.035% 84.961% ## ( 6, 10 ] 13700 2.131% 87.092% ( 10, 15 ] 2375 0.370% 87.462% ( 15, 22 ] 317 0.049% 87.511% ( 22, 34 ] 30 0.005% 87.516% ( 34, 51 ] 36 0.006% 87.522% ( 51, 76 ] 58 0.009% 87.531% ( 76, 110 ] 14249 2.217% 89.747% ( 110, 170 ] 43647 6.791% 96.538% # ( 170, 250 ] 10583 1.647% 98.185% ( 250, 380 ] 3099 0.482% 98.667% ( 380, 580 ] 3007 0.468% 99.135% ( 580, 870 ] 2094 0.326% 99.460% ( 870, 1300 ] 1654 0.257% 99.718% ( 1300, 1900 ] 1003 0.156% 99.874% ( 1900, 2900 ] 732 0.114% 99.988% ( 2900, 4400 ] 619 0.096% 100.084% ( 4400, 6600 ] 165 0.026% 100.110% ( 6600, 9900 ] 29 0.005% 100.114% ( 9900, 14000 ] 1 0.000% 100.114% ( 14000, 22000 ] 2 0.000% 100.115% ( 22000, 33000 ] 2 0.000% 100.115% ( 50000, 75000 ] 1 0.000% 100.115% ** Level 5 read latency histogram (micros): Count: 497706 Average: 206.7676 StdDev: 423.01 Min: 71 Median: 142.8460 Max: 80396 Percentiles: P50: 142.85 P75: 169.27 P99: 1721.39 P99.9: 5401.22 P99.99: 8536.08 ------------------------------------------------------ ( 51, 76 ] 106 0.021% 0.021% ( 76, 110 ] 94076 18.902% 18.923% #### ( 110, 170 ] 282540 56.768% 75.692% ########### ( 170, 250 ] 68582 13.780% 89.471% ### ( 250, 380 ] 19339 3.886% 93.357% # ( 380, 580 ] 15862 3.187% 96.544% # ( 580, 870 ] 6895 1.385% 97.929% ( 870, 1300 ] 3768 0.757% 98.686% ( 1300, 1900 ] 2224 0.447% 99.133% ( 1900, 2900 ] 1773 0.356% 99.489% ( 2900, 4400 ] 1728 0.347% 99.837% ( 4400, 6600 ] 695 0.140% 99.976% ( 6600, 9900 ] 118 0.024% 100.000% ( 9900, 14000 ] 3 0.001% 100.001% ( 14000, 22000 ] 3 0.001% 100.001% ( 22000, 33000 ] 7 0.001% 100.003% ( 33000, 50000 ] 1 0.000% 100.003% ( 50000, 75000 ] 2 0.000% 100.003% ( 75000, 110000 ] 1 0.000% 100.003% ** DB Stats ** Uptime(secs): 41.0 total, 20.3 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 96.11 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:25.381 H:M:S, 62.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 48.50 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:16.161 H:M:S, 79.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 228.101 micros/op 17525 ops/sec 60.142 seconds 1053996 operations; 17.0 MB/s (254999 of 254999 found) Microseconds per read: Count: 1053996 Average: 228.1032 StdDev: 722.49 Min: 2 Median: 167.3483 Max: 183198 Percentiles: P50: 167.35 P75: 227.78 P99: 924.82 P99.9: 10700.04 P99.99: 27897.90 ------------------------------------------------------ ( 1, 2 ] 6 0.001% 0.001% ( 2, 3 ] 43 0.004% 0.005% ( 3, 4 ] 69 0.007% 0.011% ( 4, 6 ] 78 0.007% 0.019% ( 6, 10 ] 460 0.044% 0.062% ( 10, 15 ] 5440 0.516% 0.578% ( 15, 22 ] 3806 0.361% 0.939% ( 22, 34 ] 10306 0.978% 1.917% ( 34, 51 ] 2540 0.241% 2.158% ( 51, 76 ] 327 0.031% 2.189% ( 76, 110 ] 29875 2.834% 5.024% # ( 110, 170 ] 495967 47.056% 52.080% ######### ( 170, 250 ] 334486 31.735% 83.815% ###### ( 250, 380 ] 103661 9.835% 93.650% ## ( 380, 580 ] 41261 3.915% 97.564% # ( 580, 870 ] 14487 1.374% 98.939% ( 870, 1300 ] 5052 0.479% 99.418% ( 1300, 1900 ] 1999 0.190% 99.608% ( 1900, 2900 ] 1446 0.137% 99.745% ( 2900, 4400 ] 1032 0.098% 99.843% ( 4400, 6600 ] 330 0.031% 99.874% ( 6600, 9900 ] 191 0.018% 99.892% ( 9900, 14000 ] 410 0.039% 99.931% ( 14000, 22000 ] 543 0.052% 99.983% ( 22000, 33000 ] 141 0.013% 99.996% ( 33000, 50000 ] 27 0.003% 99.999% ( 50000, 75000 ] 9 0.001% 100.000% ( 110000, 170000 ] 3 0.000% 100.000% ( 170000, 250000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 14570663 rocksdb.bloom.filter.full.positive COUNT : 1195419 rocksdb.bloom.filter.full.true.positive COUNT : 1053585 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 411 rocksdb.memtable.miss COUNT : 1053585 rocksdb.l0.hit COUNT : 45025 rocksdb.l1.hit COUNT : 11964 rocksdb.l2andup.hit COUNT : 996596 rocksdb.compaction.key.drop.new COUNT : 144290 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 4088148 rocksdb.number.keys.read COUNT : 1053996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 4218968736 rocksdb.bytes.read COUNT : 1053996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1580 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 43957739 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 18398364607 rocksdb.compact.write.bytes COUNT : 18242720080 rocksdb.flush.write.bytes COUNT : 4161729194 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 35 rocksdb.number.superversion_acquires COUNT : 793 rocksdb.number.superversion_releases COUNT : 766 rocksdb.number.superversion_cleanups COUNT : 198 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 238 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 4128056320 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1499136 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 22767014072 rocksdb.non.last.level.read.count COUNT : 5356924 rocksdb.block.checksum.compute.count COUNT : 5660262 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 166.445252 P95 : 447.452379 P99 : 922.952771 P100 : 183196.000000 COUNT : 1053996 SUM : 239477894 rocksdb.db.write.micros P50 : 1.560096 P95 : 5.030318 P99 : 13.599622 P100 : 855226.000000 COUNT : 4088148 SUM : 54621518 rocksdb.compaction.times.micros P50 : 1300000.000000 P95 : 4740000.000000 P99 : 38120000.000000 P100 : 45427355.000000 COUNT : 68 SUM : 144815609 rocksdb.compaction.times.cpu_micros P50 : 860000.000000 P95 : 3740000.000000 P99 : 11144000.000000 P100 : 13228101.000000 COUNT : 68 SUM : 76919028 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 813.870968 P95 : 94250.000000 P99 : 385555.000000 P100 : 385555.000000 COUNT : 58 SUM : 884434 rocksdb.compaction.outfile.sync.micros P50 : 23833.333333 P95 : 255777.777778 P99 : 366711.111111 P100 : 892519.000000 COUNT : 192 SUM : 11882036 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 236.666667 P95 : 4447.142857 P99 : 6169.428571 P100 : 6308.000000 COUNT : 137 SUM : 163316 rocksdb.table.open.io.micros P50 : 9496.073903 P95 : 20000.000000 P99 : 23728.571429 P100 : 155920.000000 COUNT : 1580 SUM : 15361340 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.801068 P95 : 106.954137 P99 : 224.341001 P100 : 37116.000000 COUNT : 4454027 SUM : 83518297 rocksdb.read.block.get.micros P50 : 140.159412 P95 : 324.361660 P99 : 854.122810 P100 : 87688.000000 COUNT : 1204645 SUM : 204179676 rocksdb.write.raw.block.micros P50 : 0.536354 P95 : 1.378027 P99 : 3.453672 P100 : 22206.000000 COUNT : 5426404 SUM : 15338696 rocksdb.numfiles.in.singlecompaction P50 : 2.909091 P95 : 6.000000 P99 : 24.400000 P100 : 31.000000 COUNT : 80 SUM : 310 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.569237 P95 : 1257.066821 P99 : 2663.195284 P100 : 855040.000000 COUNT : 232103 SUM : 43983679 rocksdb.sst.read.micros P50 : 2.057340 P95 : 174.993616 P99 : 474.552273 P100 : 80396.000000 COUNT : 5356964 SUM : 271562408 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1053996 SUM : 1053996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 4088148 SUM : 4218968736 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 221555.555556 P95 : 953500.000000 P99 : 1092575.000000 P100 : 1092575.000000 COUNT : 58 SUM : 17378506 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 240 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0