[2024-11-28 19:53:41.228454] Starting SPDK v25.01-pre git sha1 35cd3e84d / DPDK 24.11.0-rc4 initialization... [2024-11-28 19:53:41.228610] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3784799 ] Set seed to 1732820021224883 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-28 19:53:41.398415] pci_dpdk.c: 37:dpdk_pci_init: *NOTICE*: In-development DPDK 24.11.0-rc4 is used. There is no support for it in SPDK. Enabled only for validation. [2024-11-28 19:53:41.451637] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-28 19:53:41.475405] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-28 19:53:41.475445] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3784799' to capture a snapshot of events at runtime. [2024-11-28 19:53:41.475457] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-28 19:53:41.475466] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-28 19:53:41.475475] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3784799 for offline analysis/debug. [2024-11-28 19:53:41.476010] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Thu Nov 28 19:53:45 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/28-19:53:53 ... thread 0: (1000000,1000000) ops and (134428.4,134428.4) ops/second in (7.438906,7.438906) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 8/7 755.80 MB 6.3 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 121.2 6.76 2.48 9 0.751 0 0 0.0 0.0 L1 4/4 256.30 MB 0.0 0.3 0.2 0.1 0.3 0.2 0.0 1.5 102.4 102.3 2.50 1.18 1 2.505 260K 269 0.0 0.0 L2 3/1 299.64 MB 1.7 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 54/9 3.33 GB 2.6 3.0 2.0 1.0 3.0 2.0 0.0 1.5 98.2 97.2 31.71 14.46 14 2.265 1803K 0 0.0 0.0 L4 197/13 11.94 GB 1.2 2.9 2.2 0.8 2.9 2.1 0.1 1.3 109.0 108.3 27.43 13.87 19 1.444 931K 0 0.0 0.0 L5 1092/0 89.81 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1358/34 106.37 GB 0.0 6.2 4.4 1.8 7.0 5.1 0.1 8.7 93.0 104.2 68.40 31.98 43 1.591 2995K 269 0.0 0.0 Int 0/0 0.00 KB 0.0 6.2 4.4 1.8 7.0 5.1 0.1 8.7 93.0 104.2 68.40 31.98 43 1.591 2995K 269 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.2 4.4 1.8 6.2 4.3 0.0 0.0 103.2 102.3 61.65 29.51 34 1.813 2995K 269 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 121.2 6.76 2.48 9 0.751 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.6 total, 7.4 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.96 GB write, 940.66 MB/s write, 6.21 GB read, 839.69 MB/s read, 68.4 seconds Interval compaction: 6.96 GB write, 957.82 MB/s write, 6.21 GB read, 855.01 MB/s read, 68.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 29, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 29, interval: 29 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 94006 Average: 26.5226 StdDev: 224.94 Min: 1 Median: 1.8612 Max: 8272 Percentiles: P50: 1.86 P75: 2.87 P99: 869.94 P99.9: 3713.31 P99.99: 5886.84 ------------------------------------------------------ [ 0, 1 ] 4557 4.848% 4.848% # ( 1, 2 ] 49289 52.432% 57.279% ########## ( 2, 3 ] 19059 20.274% 77.554% #### ( 3, 4 ] 8729 9.286% 86.839% ## ( 4, 6 ] 8490 9.031% 95.870% ## ( 6, 10 ] 1856 1.974% 97.845% ( 10, 15 ] 281 0.299% 98.144% ( 15, 22 ] 53 0.056% 98.200% ( 22, 34 ] 3 0.003% 98.203% ( 34, 51 ] 3 0.003% 98.206% ( 51, 76 ] 3 0.003% 98.210% ( 76, 110 ] 3 0.003% 98.213% ( 110, 170 ] 8 0.009% 98.221% ( 170, 250 ] 39 0.041% 98.263% ( 250, 380 ] 119 0.127% 98.389% ( 380, 580 ] 237 0.252% 98.642% ( 580, 870 ] 339 0.361% 99.002% ( 870, 1300 ] 336 0.357% 99.360% ( 1300, 1900 ] 252 0.268% 99.628% ( 1900, 2900 ] 181 0.193% 99.820% ( 2900, 4400 ] 142 0.151% 99.971% ( 4400, 6600 ] 29 0.031% 100.002% ( 6600, 9900 ] 1 0.001% 100.003% ** Level 1 read latency histogram (micros): Count: 56711 Average: 13.9212 StdDev: 166.25 Min: 0 Median: 1.9157 Max: 7926 Percentiles: P50: 1.92 P75: 2.92 P99: 12.44 P99.9: 2909.63 P99.99: 5792.36 ------------------------------------------------------ [ 0, 1 ] 1486 2.620% 2.620% # ( 1, 2 ] 29343 51.741% 54.362% ########## ( 2, 3 ] 12779 22.534% 76.895% ##### ( 3, 4 ] 5349 9.432% 86.327% ## ( 4, 6 ] 5649 9.961% 96.288% ## ( 6, 10 ] 1416 2.497% 98.785% ( 10, 15 ] 250 0.441% 99.226% ( 15, 22 ] 26 0.046% 99.272% ( 22, 34 ] 4 0.007% 99.279% ( 34, 51 ] 1 0.002% 99.281% ( 51, 76 ] 3 0.005% 99.286% ( 76, 110 ] 2 0.004% 99.289% ( 110, 170 ] 7 0.012% 99.302% ( 170, 250 ] 7 0.012% 99.314% ( 250, 380 ] 19 0.034% 99.348% ( 380, 580 ] 42 0.074% 99.422% ( 580, 870 ] 68 0.120% 99.542% ( 870, 1300 ] 80 0.141% 99.683% ( 1300, 1900 ] 80 0.141% 99.824% ( 1900, 2900 ] 43 0.076% 99.899% ( 2900, 4400 ] 45 0.079% 99.979% ( 4400, 6600 ] 10 0.018% 99.996% ( 6600, 9900 ] 2 0.004% 100.000% ** Level 2 read latency histogram (micros): Count: 404959 Average: 13.0705 StdDev: 162.60 Min: 1 Median: 1.9500 Max: 9220 Percentiles: P50: 1.95 P75: 2.98 P99: 11.70 P99.9: 1573.87 P99.99: 2475.52 ------------------------------------------------------ [ 0, 1 ] 12095 2.987% 2.987% # ( 1, 2 ] 200399 49.486% 52.473% ########## ( 2, 3 ] 93388 23.061% 75.534% ##### ( 3, 4 ] 40733 10.059% 85.593% ## ( 4, 6 ] 41589 10.270% 95.863% ## ( 6, 10 ] 11974 2.957% 98.819% # ( 10, 15 ] 2151 0.531% 99.351% ( 15, 22 ] 312 0.077% 99.428% ( 22, 34 ] 61 0.015% 99.443% ( 34, 51 ] 39 0.010% 99.452% ( 51, 76 ] 25 0.006% 99.458% ( 76, 110 ] 19 0.005% 99.463% ( 110, 170 ] 41 0.010% 99.473% ( 170, 250 ] 79 0.020% 99.493% ( 250, 380 ] 184 0.045% 99.538% ( 380, 580 ] 310 0.077% 99.615% ( 580, 870 ] 479 0.118% 99.733% ( 870, 1300 ] 514 0.127% 99.860% ( 1300, 1900 ] 355 0.088% 99.948% ( 1900, 2900 ] 298 0.074% 100.021% ( 2900, 4400 ] 312 0.077% 100.098% ( 4400, 6600 ] 103 0.025% 100.124% ( 6600, 9900 ] 11 0.003% 100.126% ** Level 3 read latency histogram (micros): Count: 800920 Average: 13.2888 StdDev: 163.36 Min: 1 Median: 1.9418 Max: 12134 Percentiles: P50: 1.94 P75: 2.92 P99: 9.86 P99.9: 923.49 P99.99: 1198.77 ------------------------------------------------------ [ 0, 1 ] 15919 1.988% 1.988% ( 1, 2 ] 408307 50.980% 52.967% ########## ( 2, 3 ] 192014 23.974% 76.942% ##### ( 3, 4 ] 79942 9.981% 86.923% ## ( 4, 6 ] 76785 9.587% 96.510% ## ( 6, 10 ] 20648 2.578% 99.088% # ( 10, 15 ] 3507 0.438% 99.526% ( 15, 22 ] 462 0.058% 99.583% ( 22, 34 ] 38 0.005% 99.588% ( 34, 51 ] 44 0.005% 99.594% ( 51, 76 ] 37 0.005% 99.598% ( 76, 110 ] 26 0.003% 99.602% ( 110, 170 ] 88 0.011% 99.613% ( 170, 250 ] 173 0.022% 99.634% ( 250, 380 ] 328 0.041% 99.675% ( 380, 580 ] 647 0.081% 99.756% ( 580, 870 ] 1014 0.127% 99.883% ( 870, 1300 ] 1126 0.141% 100.023% ( 1300, 1900 ] 823 0.103% 100.126% ( 1900, 2900 ] 641 0.080% 100.206% ( 2900, 4400 ] 591 0.074% 100.280% ( 4400, 6600 ] 162 0.020% 100.300% ( 6600, 9900 ] 31 0.004% 100.304% ( 9900, 14000 ] 4 0.000% 100.304% ** Level 4 read latency histogram (micros): Count: 249706 Average: 27.8100 StdDev: 240.11 Min: 1 Median: 1.9450 Max: 9976 Percentiles: P50: 1.95 P75: 2.96 P99: 799.26 P99.9: 2825.30 P99.99: 3632.67 ------------------------------------------------------ [ 0, 1 ] 5316 2.129% 2.129% ( 1, 2 ] 126490 50.656% 52.784% ########## ( 2, 3 ] 58068 23.255% 76.039% ##### ( 3, 4 ] 25908 10.375% 86.414% ## ( 4, 6 ] 23196 9.289% 95.704% ## ( 6, 10 ] 5300 2.122% 97.826% ( 10, 15 ] 935 0.374% 98.201% ( 15, 22 ] 117 0.047% 98.248% ( 22, 34 ] 13 0.005% 98.253% ( 34, 51 ] 16 0.006% 98.259% ( 51, 76 ] 17 0.007% 98.266% ( 76, 110 ] 18 0.007% 98.273% ( 110, 170 ] 56 0.022% 98.296% ( 170, 250 ] 141 0.056% 98.352% ( 250, 380 ] 320 0.128% 98.480% ( 380, 580 ] 623 0.249% 98.730% ( 580, 870 ] 894 0.358% 99.088% ( 870, 1300 ] 984 0.394% 99.482% ( 1300, 1900 ] 603 0.241% 99.723% ( 1900, 2900 ] 478 0.191% 99.915% ( 2900, 4400 ] 387 0.155% 100.070% ( 4400, 6600 ] 105 0.042% 100.112% ( 6600, 9900 ] 16 0.006% 100.118% ( 9900, 14000 ] 1 0.000% 100.119% ** Level 5 read latency histogram (micros): Count: 13568 Average: 1598.8399 StdDev: 1401.67 Min: 81 Median: 1104.0214 Max: 12297 Percentiles: P50: 1104.02 P75: 2170.04 P99: 6486.78 P99.9: 9611.49 P99.99: 12297.00 ------------------------------------------------------ ( 76, 110 ] 14 0.103% 0.103% ( 110, 170 ] 132 0.973% 1.076% ( 170, 250 ] 270 1.990% 3.066% ( 250, 380 ] 804 5.926% 8.992% # ( 380, 580 ] 1660 12.235% 21.226% ## ( 580, 870 ] 2483 18.300% 39.527% #### ( 870, 1300 ] 2611 19.244% 58.771% #### ( 1300, 1900 ] 1791 13.200% 71.971% ### ( 1900, 2900 ] 1522 11.218% 83.188% ## ( 2900, 4400 ] 1543 11.372% 94.561% ## ( 4400, 6600 ] 635 4.680% 99.241% # ( 6600, 9900 ] 98 0.722% 99.963% ( 9900, 14000 ] 5 0.037% 100.000% ** DB Stats ** Uptime(secs): 7.6 total, 7.4 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 129.90 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.510 H:M:S, 46.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 132.25 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.510 H:M:S, 47.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/11/28-19:53:58 ... thread 0: (1000000,2000000) ops and (170310.5,150256.9) ops/second in (5.871628,13.310534) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/8 1.41 GB 31.5 0.7 0.0 0.7 2.5 1.8 0.0 1.4 40.9 148.6 16.95 8.18 24 0.706 703K 849 0.0 0.0 L1 6/6 570.02 MB 0.0 0.8 0.5 0.3 0.8 0.5 0.0 1.7 86.3 86.1 9.59 3.86 2 4.796 840K 1400 0.0 0.0 L2 1/0 42.07 MB 0.4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 20/1 1.03 GB 0.9 3.4 2.3 1.2 3.4 2.3 0.0 1.5 100.5 99.5 35.05 16.37 16 2.191 1956K 0 0.0 0.0 L4 182/4 9.80 GB 1.0 7.4 4.3 3.1 7.3 4.2 0.4 1.7 117.6 116.0 64.37 35.27 49 1.314 3077K 301 0.0 0.0 L5 1148/0 94.43 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1369/19 107.27 GB 0.0 12.3 7.1 5.2 14.0 8.8 5.1 7.8 100.1 113.5 125.97 63.68 91 1.384 6577K 2550 0.0 0.0 Int 0/0 0.00 KB 0.0 12.3 7.1 5.2 14.0 8.8 5.1 7.8 100.1 113.5 125.97 63.68 91 1.384 6577K 2550 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.3 7.1 5.2 12.2 7.0 0.0 0.0 111.3 110.0 113.37 58.32 69 1.643 6577K 2550 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 145.0 12.60 5.35 22 0.573 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 13.4 total, 13.3 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.96 GB write, 1063.25 MB/s write, 12.32 GB read, 938.14 MB/s read, 126.0 seconds Interval compaction: 13.96 GB write, 1074.09 MB/s write, 12.32 GB read, 947.70 MB/s read, 126.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 53, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 53, interval: 53 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 402304 Average: 16.3038 StdDev: 173.34 Min: 1 Median: 1.7940 Max: 15391 Percentiles: P50: 1.79 P75: 2.71 P99: 241.29 P99.9: 2615.71 P99.99: 4087.59 ------------------------------------------------------ [ 0, 1 ] 23562 5.857% 5.857% # ( 1, 2 ] 223656 55.594% 61.451% ########### ( 2, 3 ] 76518 19.020% 80.470% #### ( 3, 4 ] 33483 8.323% 88.793% ## ( 4, 6 ] 32451 8.066% 96.860% ## ( 6, 10 ] 6947 1.727% 98.586% ( 10, 15 ] 1025 0.255% 98.841% ( 15, 22 ] 131 0.033% 98.874% ( 22, 34 ] 8 0.002% 98.876% ( 34, 51 ] 12 0.003% 98.879% ( 51, 76 ] 14 0.003% 98.882% ( 76, 110 ] 37 0.009% 98.891% ( 110, 170 ] 191 0.047% 98.939% ( 170, 250 ] 276 0.069% 99.007% ( 250, 380 ] 351 0.087% 99.095% ( 380, 580 ] 643 0.160% 99.255% ( 580, 870 ] 848 0.211% 99.465% ( 870, 1300 ] 888 0.221% 99.686% ( 1300, 1900 ] 583 0.145% 99.831% ( 1900, 2900 ] 388 0.096% 99.927% ( 2900, 4400 ] 318 0.079% 100.006% ( 4400, 6600 ] 80 0.020% 100.026% ( 6600, 9900 ] 4 0.001% 100.027% ( 9900, 14000 ] 1 0.000% 100.028% ( 14000, 22000 ] 2 0.000% 100.028% ** Level 1 read latency histogram (micros): Count: 113645 Average: 14.6776 StdDev: 170.07 Min: 0 Median: 1.8801 Max: 16275 Percentiles: P50: 1.88 P75: 2.86 P99: 53.29 P99.9: 2810.06 P99.99: 5863.06 ------------------------------------------------------ [ 0, 1 ] 3550 3.124% 3.124% # ( 1, 2 ] 60530 53.262% 56.386% ########### ( 2, 3 ] 24717 21.749% 78.135% #### ( 3, 4 ] 10104 8.891% 87.026% ## ( 4, 6 ] 10233 9.004% 96.031% ## ( 6, 10 ] 2848 2.506% 98.537% # ( 10, 15 ] 469 0.413% 98.949% ( 15, 22 ] 48 0.042% 98.992% ( 22, 34 ] 4 0.004% 98.995% ( 34, 51 ] 6 0.005% 99.000% ( 51, 76 ] 6 0.005% 99.006% ( 76, 110 ] 25 0.022% 99.028% ( 110, 170 ] 65 0.057% 99.085% ( 170, 250 ] 92 0.081% 99.166% ( 250, 380 ] 93 0.082% 99.248% ( 380, 580 ] 135 0.119% 99.366% ( 580, 870 ] 177 0.156% 99.522% ( 870, 1300 ] 201 0.177% 99.699% ( 1300, 1900 ] 152 0.134% 99.833% ( 1900, 2900 ] 85 0.075% 99.908% ( 2900, 4400 ] 82 0.072% 99.980% ( 4400, 6600 ] 19 0.017% 99.996% ( 6600, 9900 ] 4 0.004% 100.000% ( 14000, 22000 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 452763 Average: 12.8390 StdDev: 160.05 Min: 1 Median: 1.9510 Max: 9220 Percentiles: P50: 1.95 P75: 2.97 P99: 11.58 P99.9: 1643.89 P99.99: 2620.25 ------------------------------------------------------ [ 0, 1 ] 12634 2.790% 2.790% # ( 1, 2 ] 224764 49.643% 52.433% ########## ( 2, 3 ] 105061 23.204% 75.638% ##### ( 3, 4 ] 45460 10.041% 85.678% ## ( 4, 6 ] 46348 10.237% 95.915% ## ( 6, 10 ] 13227 2.921% 98.836% # ( 10, 15 ] 2346 0.518% 99.354% ( 15, 22 ] 344 0.076% 99.430% ( 22, 34 ] 63 0.014% 99.444% ( 34, 51 ] 44 0.010% 99.454% ( 51, 76 ] 26 0.006% 99.460% ( 76, 110 ] 21 0.005% 99.464% ( 110, 170 ] 46 0.010% 99.475% ( 170, 250 ] 86 0.019% 99.494% ( 250, 380 ] 195 0.043% 99.537% ( 380, 580 ] 337 0.074% 99.611% ( 580, 870 ] 527 0.116% 99.727% ( 870, 1300 ] 560 0.124% 99.851% ( 1300, 1900 ] 386 0.085% 99.936% ( 1900, 2900 ] 337 0.074% 100.011% ( 2900, 4400 ] 343 0.076% 100.087% ( 4400, 6600 ] 109 0.024% 100.111% ( 6600, 9900 ] 11 0.002% 100.113% ** Level 3 read latency histogram (micros): Count: 1339917 Average: 12.2248 StdDev: 159.06 Min: 1 Median: 1.9371 Max: 18986 Percentiles: P50: 1.94 P75: 2.92 P99: 9.78 P99.9: 894.41 P99.99: 1188.54 ------------------------------------------------------ [ 0, 1 ] 26789 1.999% 1.999% ( 1, 2 ] 686339 51.223% 53.222% ########## ( 2, 3 ] 317838 23.721% 76.943% ##### ( 3, 4 ] 132198 9.866% 86.809% ## ( 4, 6 ] 130041 9.705% 96.514% ## ( 6, 10 ] 35280 2.633% 99.147% # ( 10, 15 ] 5539 0.413% 99.560% ( 15, 22 ] 718 0.054% 99.614% ( 22, 34 ] 49 0.004% 99.617% ( 34, 51 ] 71 0.005% 99.623% ( 51, 76 ] 80 0.006% 99.629% ( 76, 110 ] 45 0.003% 99.632% ( 110, 170 ] 162 0.012% 99.644% ( 170, 250 ] 250 0.019% 99.663% ( 250, 380 ] 480 0.036% 99.699% ( 380, 580 ] 1012 0.076% 99.774% ( 580, 870 ] 1586 0.118% 99.893% ( 870, 1300 ] 1763 0.132% 100.024% ( 1300, 1900 ] 1198 0.089% 100.114% ( 1900, 2900 ] 918 0.069% 100.182% ( 2900, 4400 ] 846 0.063% 100.245% ( 4400, 6600 ] 243 0.018% 100.263% ( 6600, 9900 ] 45 0.003% 100.267% ( 9900, 14000 ] 7 0.001% 100.267% ( 14000, 22000 ] 9 0.001% 100.268% ** Level 4 read latency histogram (micros): Count: 827615 Average: 19.9718 StdDev: 209.26 Min: 1 Median: 1.9311 Max: 21134 Percentiles: P50: 1.93 P75: 2.93 P99: 315.51 P99.9: 2043.11 P99.99: 2807.84 ------------------------------------------------------ [ 0, 1 ] 17456 2.109% 2.109% ( 1, 2 ] 425700 51.437% 53.546% ########## ( 2, 3 ] 190916 23.068% 76.614% ##### ( 3, 4 ] 82004 9.908% 86.523% ## ( 4, 6 ] 77807 9.401% 95.924% ## ( 6, 10 ] 20232 2.445% 98.369% ( 10, 15 ] 2984 0.361% 98.729% ( 15, 22 ] 394 0.048% 98.777% ( 22, 34 ] 31 0.004% 98.781% ( 34, 51 ] 59 0.007% 98.788% ( 51, 76 ] 64 0.008% 98.796% ( 76, 110 ] 121 0.015% 98.810% ( 110, 170 ] 493 0.060% 98.870% ( 170, 250 ] 647 0.078% 98.948% ( 250, 380 ] 857 0.104% 99.051% ( 380, 580 ] 1434 0.173% 99.225% ( 580, 870 ] 2047 0.247% 99.472% ( 870, 1300 ] 2087 0.252% 99.724% ( 1300, 1900 ] 1316 0.159% 99.883% ( 1900, 2900 ] 974 0.118% 100.001% ( 2900, 4400 ] 842 0.102% 100.103% ( 4400, 6600 ] 286 0.035% 100.137% ( 6600, 9900 ] 41 0.005% 100.142% ( 9900, 14000 ] 5 0.001% 100.143% ( 14000, 22000 ] 9 0.001% 100.144% ** Level 5 read latency histogram (micros): Count: 34747 Average: 1131.3790 StdDev: 1253.73 Min: 76 Median: 729.2025 Max: 22050 Percentiles: P50: 729.20 P75: 1367.84 P99: 6106.42 P99.9: 9537.08 P99.99: 20200.22 ------------------------------------------------------ ( 51, 76 ] 1 0.003% 0.003% ( 76, 110 ] 572 1.646% 1.649% ( 110, 170 ] 2714 7.811% 9.460% ## ( 170, 250 ] 2998 8.628% 18.088% ## ( 250, 380 ] 3299 9.494% 27.582% ## ( 380, 580 ] 4754 13.682% 41.264% ### ( 580, 870 ] 5900 16.980% 58.244% ### ( 870, 1300 ] 5447 15.676% 73.920% ### ( 1300, 1900 ] 3319 9.552% 83.472% ## ( 1900, 2900 ] 2386 6.867% 90.339% # ( 2900, 4400 ] 2313 6.657% 96.995% # ( 4400, 6600 ] 898 2.584% 99.580% # ( 6600, 9900 ] 125 0.360% 99.940% ( 9900, 14000 ] 9 0.026% 99.965% ( 14000, 22000 ] 11 0.032% 99.997% ( 22000, 33000 ] 1 0.003% 100.000% ** DB Stats ** Uptime(secs): 13.4 total, 5.9 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 146.37 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.557 H:M:S, 41.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 167.62 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.048 H:M:S, 34.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/28-19:54:02 ... thread 0: (1000000,3000000) ops and (284714.8,178329.2) ops/second in (3.512287,16.822821) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/8 2.40 GB 132.2 1.7 0.0 1.7 4.4 2.8 0.0 1.6 70.2 187.1 24.24 14.90 44 0.551 1726K 1811 0.0 0.0 L1 6/6 570.02 MB 0.0 0.8 0.5 0.3 0.8 0.5 0.0 1.7 86.3 86.1 9.59 3.86 2 4.796 840K 1400 0.0 0.0 L2 1/0 42.07 MB 0.4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 19/0 921.72 MB 0.9 3.4 2.3 1.2 3.4 2.3 0.0 1.5 100.5 99.5 35.05 16.37 16 2.191 1956K 0 0.0 0.0 L4 180/0 9.55 GB 1.0 7.7 4.5 3.3 7.6 4.4 0.4 1.7 119.5 117.7 66.31 36.81 50 1.326 3140K 301 0.0 0.0 L5 1152/0 94.80 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1374/14 108.24 GB 0.0 13.6 7.2 6.4 16.3 9.9 5.4 5.9 103.4 123.2 135.19 71.94 112 1.207 7665K 3512 0.0 0.0 Int 0/0 0.00 KB 0.0 13.6 7.2 6.4 16.3 9.9 5.4 5.9 103.4 123.2 135.19 71.94 112 1.207 7665K 3512 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.6 7.2 6.4 13.5 7.1 0.0 0.0 117.1 115.9 119.27 63.75 74 1.612 7665K 3512 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 178.1 15.91 8.19 38 0.419 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 17.0 total, 16.8 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.26 GB write, 981.98 MB/s write, 13.64 GB read, 823.79 MB/s read, 135.2 seconds Interval compaction: 16.26 GB write, 989.90 MB/s write, 13.64 GB read, 830.44 MB/s read, 135.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 53, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 53, interval: 53 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 727735 Average: 12.8269 StdDev: 133.08 Min: 1 Median: 1.6540 Max: 15391 Percentiles: P50: 1.65 P75: 2.33 P99: 208.44 P99.9: 1659.66 P99.99: 2988.74 ------------------------------------------------------ [ 0, 1 ] 89924 12.357% 12.357% ## ( 1, 2 ] 418899 57.562% 69.919% ############ ( 2, 3 ] 111189 15.279% 85.197% ### ( 3, 4 ] 44536 6.120% 91.317% # ( 4, 6 ] 39056 5.367% 96.684% # ( 6, 10 ] 8755 1.203% 97.887% ( 10, 15 ] 1386 0.190% 98.078% ( 15, 22 ] 205 0.028% 98.106% ( 22, 34 ] 15 0.002% 98.108% ( 34, 51 ] 13 0.002% 98.110% ( 51, 76 ] 18 0.002% 98.112% ( 76, 110 ] 1091 0.150% 98.262% ( 110, 170 ] 4135 0.568% 98.830% ( 170, 250 ] 2574 0.354% 99.184% ( 250, 380 ] 1334 0.183% 99.367% ( 380, 580 ] 1435 0.197% 99.564% ( 580, 870 ] 1136 0.156% 99.721% ( 870, 1300 ] 953 0.131% 99.851% ( 1300, 1900 ] 591 0.081% 99.933% ( 1900, 2900 ] 399 0.055% 99.987% ( 2900, 4400 ] 325 0.045% 100.032% ( 4400, 6600 ] 80 0.011% 100.043% ( 6600, 9900 ] 4 0.001% 100.044% ( 9900, 14000 ] 1 0.000% 100.044% ( 14000, 22000 ] 2 0.000% 100.044% ** Level 1 read latency histogram (micros): Count: 166132 Average: 12.0430 StdDev: 141.93 Min: 0 Median: 1.8953 Max: 16275 Percentiles: P50: 1.90 P75: 2.92 P99: 135.04 P99.9: 2192.56 P99.99: 5139.52 ------------------------------------------------------ [ 0, 1 ] 6289 3.786% 3.786% # ( 1, 2 ] 85752 51.617% 55.402% ########## ( 2, 3 ] 35379 21.296% 76.698% #### ( 3, 4 ] 15062 9.066% 85.764% ## ( 4, 6 ] 16122 9.704% 95.469% ## ( 6, 10 ] 4574 2.753% 98.222% # ( 10, 15 ] 760 0.457% 98.679% ( 15, 22 ] 180 0.108% 98.788% ( 22, 34 ] 15 0.009% 98.797% ( 34, 51 ] 6 0.004% 98.800% ( 51, 76 ] 6 0.004% 98.804% ( 76, 110 ] 140 0.084% 98.888% ( 110, 170 ] 445 0.268% 99.156% ( 170, 250 ] 280 0.169% 99.325% ( 250, 380 ] 168 0.101% 99.426% ( 380, 580 ] 215 0.129% 99.555% ( 580, 870 ] 194 0.117% 99.672% ( 870, 1300 ] 202 0.122% 99.794% ( 1300, 1900 ] 152 0.091% 99.885% ( 1900, 2900 ] 85 0.051% 99.936% ( 2900, 4400 ] 83 0.050% 99.986% ( 4400, 6600 ] 19 0.011% 99.998% ( 6600, 9900 ] 4 0.002% 100.000% ( 14000, 22000 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 452788 Average: 12.8491 StdDev: 160.05 Min: 1 Median: 1.9510 Max: 9220 Percentiles: P50: 1.95 P75: 2.97 P99: 11.63 P99.9: 1643.85 P99.99: 2620.24 ------------------------------------------------------ [ 0, 1 ] 12634 2.790% 2.790% # ( 1, 2 ] 224764 49.640% 52.430% ########## ( 2, 3 ] 105061 23.203% 75.633% ##### ( 3, 4 ] 45460 10.040% 85.673% ## ( 4, 6 ] 46348 10.236% 95.910% ## ( 6, 10 ] 13227 2.921% 98.831% # ( 10, 15 ] 2346 0.518% 99.349% ( 15, 22 ] 344 0.076% 99.425% ( 22, 34 ] 63 0.014% 99.439% ( 34, 51 ] 44 0.010% 99.449% ( 51, 76 ] 26 0.006% 99.454% ( 76, 110 ] 25 0.006% 99.460% ( 110, 170 ] 56 0.012% 99.472% ( 170, 250 ] 93 0.021% 99.493% ( 250, 380 ] 198 0.044% 99.536% ( 380, 580 ] 337 0.074% 99.611% ( 580, 870 ] 528 0.117% 99.727% ( 870, 1300 ] 560 0.124% 99.851% ( 1300, 1900 ] 386 0.085% 99.936% ( 1900, 2900 ] 337 0.074% 100.011% ( 2900, 4400 ] 343 0.076% 100.087% ( 4400, 6600 ] 109 0.024% 100.111% ( 6600, 9900 ] 11 0.002% 100.113% ** Level 3 read latency histogram (micros): Count: 1340673 Average: 12.3252 StdDev: 159.12 Min: 1 Median: 1.9376 Max: 18986 Percentiles: P50: 1.94 P75: 2.92 P99: 9.85 P99.9: 895.62 P99.99: 1189.08 ------------------------------------------------------ [ 0, 1 ] 26789 1.998% 1.998% ( 1, 2 ] 686347 51.194% 53.192% ########## ( 2, 3 ] 317883 23.711% 76.903% ##### ( 3, 4 ] 132214 9.862% 86.765% ## ( 4, 6 ] 130060 9.701% 96.466% ## ( 6, 10 ] 35285 2.632% 99.098% # ( 10, 15 ] 5539 0.413% 99.511% ( 15, 22 ] 718 0.054% 99.565% ( 22, 34 ] 49 0.004% 99.568% ( 34, 51 ] 71 0.005% 99.573% ( 51, 76 ] 80 0.006% 99.579% ( 76, 110 ] 140 0.010% 99.590% ( 110, 170 ] 443 0.033% 99.623% ( 170, 250 ] 392 0.029% 99.652% ( 250, 380 ] 543 0.041% 99.693% ( 380, 580 ] 1064 0.079% 99.772% ( 580, 870 ] 1610 0.120% 99.892% ( 870, 1300 ] 1768 0.132% 100.024% ( 1300, 1900 ] 1199 0.089% 100.113% ( 1900, 2900 ] 918 0.068% 100.182% ( 2900, 4400 ] 846 0.063% 100.245% ( 4400, 6600 ] 243 0.018% 100.263% ( 6600, 9900 ] 45 0.003% 100.267% ( 9900, 14000 ] 7 0.001% 100.267% ( 14000, 22000 ] 9 0.001% 100.268% ** Level 4 read latency histogram (micros): Count: 840991 Average: 21.4017 StdDev: 208.83 Min: 1 Median: 1.9380 Max: 21134 Percentiles: P50: 1.94 P75: 2.95 P99: 405.69 P99.9: 2031.16 P99.99: 2806.66 ------------------------------------------------------ [ 0, 1 ] 17603 2.093% 2.093% ( 1, 2 ] 429532 51.075% 53.168% ########## ( 2, 3 ] 192554 22.896% 76.064% ##### ( 3, 4 ] 82471 9.806% 85.870% ## ( 4, 6 ] 78247 9.304% 95.174% ## ( 6, 10 ] 20361 2.421% 97.595% ( 10, 15 ] 3001 0.357% 97.952% ( 15, 22 ] 397 0.047% 97.999% ( 22, 34 ] 31 0.004% 98.003% ( 34, 51 ] 59 0.007% 98.010% ( 51, 76 ] 64 0.008% 98.018% ( 76, 110 ] 989 0.118% 98.135% ( 110, 170 ] 3333 0.396% 98.532% ( 170, 250 ] 2135 0.254% 98.785% ( 250, 380 ] 1543 0.183% 98.969% ( 380, 580 ] 2033 0.242% 99.211% ( 580, 870 ] 2225 0.265% 99.475% ( 870, 1300 ] 2121 0.252% 99.727% ( 1300, 1900 ] 1323 0.157% 99.885% ( 1900, 2900 ] 976 0.116% 100.001% ( 2900, 4400 ] 843 0.100% 100.101% ( 4400, 6600 ] 286 0.034% 100.135% ( 6600, 9900 ] 41 0.005% 100.140% ( 9900, 14000 ] 5 0.001% 100.141% ( 14000, 22000 ] 9 0.001% 100.142% ** Level 5 read latency histogram (micros): Count: 76578 Average: 631.6596 StdDev: 967.51 Min: 75 Median: 247.0577 Max: 22050 Percentiles: P50: 247.06 P75: 692.09 P99: 5074.26 P99.9: 8353.54 P99.99: 14976.15 ------------------------------------------------------ ( 51, 76 ] 8 0.010% 0.010% ( 76, 110 ] 5856 7.647% 7.658% ## ( 110, 170 ] 20797 27.158% 34.815% ##### ( 170, 250 ] 12072 15.764% 50.580% ### ( 250, 380 ] 7754 10.126% 60.705% ## ( 380, 580 ] 8200 10.708% 71.413% ## ( 580, 870 ] 7106 9.279% 80.693% ## ( 870, 1300 ] 5655 7.385% 88.078% # ( 1300, 1900 ] 3356 4.382% 92.460% # ( 1900, 2900 ] 2405 3.141% 95.601% # ( 2900, 4400 ] 2328 3.040% 98.641% # ( 4400, 6600 ] 898 1.173% 99.813% ( 6600, 9900 ] 125 0.163% 99.976% ( 9900, 14000 ] 9 0.012% 99.988% ( 14000, 22000 ] 11 0.014% 100.003% ( 22000, 33000 ] 1 0.001% 100.004% ** DB Stats ** Uptime(secs): 17.0 total, 3.5 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 174.09 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.557 H:M:S, 32.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 280.21 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/28-19:54:06 ... thread 0: (1000000,4000000) ops and (269385.7,194789.7) ops/second in (3.712149,20.534970) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/8 3.44 GB 239.2 2.9 0.0 2.9 6.7 3.8 0.0 1.8 91.1 211.2 32.50 22.64 66 0.492 3006K 3053 0.0 0.0 L1 6/6 570.02 MB 0.0 0.8 0.5 0.3 0.8 0.5 0.0 1.7 86.3 86.1 9.59 3.86 2 4.796 840K 1400 0.0 0.0 L2 1/0 42.07 MB 0.4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 19/0 921.72 MB 0.9 3.4 2.3 1.2 3.4 2.3 0.0 1.5 100.5 99.5 35.05 16.37 16 2.191 1956K 0 0.0 0.0 L4 180/0 9.55 GB 1.0 7.7 4.5 3.3 7.6 4.4 0.4 1.7 119.5 117.7 66.31 36.81 50 1.326 3140K 301 0.0 0.0 L5 1152/0 94.80 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1376/14 109.29 GB 0.0 14.9 7.2 7.6 18.5 10.9 5.4 4.9 106.2 132.3 143.45 79.68 134 1.070 8944K 4754 0.0 0.0 Int 0/0 0.00 KB 0.0 14.9 7.2 7.6 18.5 10.9 5.4 4.9 106.2 132.3 143.45 79.68 134 1.070 8944K 4754 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.9 7.2 7.6 14.7 7.1 0.0 0.0 122.7 121.5 124.09 68.48 79 1.571 8944K 4754 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 201.8 19.36 11.20 55 0.352 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.7 total, 20.5 interval Flush(GB): cumulative 3.814, interval 3.814 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 18.54 GB write, 918.34 MB/s write, 14.87 GB read, 736.82 MB/s read, 143.4 seconds Interval compaction: 18.54 GB write, 924.41 MB/s write, 14.87 GB read, 741.69 MB/s read, 143.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 53, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 53, interval: 57 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1101180 Average: 10.9553 StdDev: 111.28 Min: 0 Median: 1.6000 Max: 15391 Percentiles: P50: 1.60 P75: 2.11 P99: 190.94 P99.9: 1156.59 P99.99: 2046.77 ------------------------------------------------------ [ 0, 1 ] 162562 14.763% 14.763% ### ( 1, 2 ] 646751 58.733% 73.495% ############ ( 2, 3 ] 152918 13.887% 87.382% ### ( 3, 4 ] 57269 5.201% 92.583% # ( 4, 6 ] 46084 4.185% 96.767% # ( 6, 10 ] 11183 1.016% 97.783% ( 10, 15 ] 1787 0.162% 97.945% ( 15, 22 ] 276 0.025% 97.970% ( 22, 34 ] 27 0.002% 97.973% ( 34, 51 ] 15 0.001% 97.974% ( 51, 76 ] 21 0.002% 97.976% ( 76, 110 ] 2175 0.198% 98.174% ( 110, 170 ] 7921 0.719% 98.893% ( 170, 250 ] 4509 0.409% 99.302% ( 250, 380 ] 2295 0.208% 99.511% ( 380, 580 ] 2226 0.202% 99.713% ( 580, 870 ] 1395 0.127% 99.840% ( 870, 1300 ] 999 0.091% 99.930% ( 1300, 1900 ] 598 0.054% 99.985% ( 1900, 2900 ] 408 0.037% 100.022% ( 2900, 4400 ] 330 0.030% 100.052% ( 4400, 6600 ] 81 0.007% 100.059% ( 6600, 9900 ] 4 0.000% 100.059% ( 9900, 14000 ] 1 0.000% 100.059% ( 14000, 22000 ] 2 0.000% 100.060% ** Level 1 read latency histogram (micros): Count: 218589 Average: 10.6691 StdDev: 124.85 Min: 0 Median: 1.8470 Max: 16275 Percentiles: P50: 1.85 P75: 2.83 P99: 144.31 P99.9: 1787.15 P99.99: 4416.34 ------------------------------------------------------ [ 0, 1 ] 10602 4.850% 4.850% # ( 1, 2 ] 116515 53.303% 58.153% ########### ( 2, 3 ] 44619 20.412% 78.566% #### ( 3, 4 ] 18149 8.303% 86.869% ## ( 4, 6 ] 18897 8.645% 95.513% ## ( 6, 10 ] 5715 2.614% 98.128% # ( 10, 15 ] 899 0.411% 98.539% ( 15, 22 ] 230 0.105% 98.644% ( 22, 34 ] 17 0.008% 98.652% ( 34, 51 ] 6 0.003% 98.655% ( 51, 76 ] 6 0.003% 98.658% ( 76, 110 ] 258 0.118% 98.776% ( 110, 170 ] 857 0.392% 99.168% ( 170, 250 ] 503 0.230% 99.398% ( 250, 380 ] 251 0.115% 99.513% ( 380, 580 ] 294 0.134% 99.647% ( 580, 870 ] 224 0.102% 99.750% ( 870, 1300 ] 205 0.094% 99.844% ( 1300, 1900 ] 152 0.070% 99.913% ( 1900, 2900 ] 85 0.039% 99.952% ( 2900, 4400 ] 83 0.038% 99.990% ( 4400, 6600 ] 19 0.009% 99.999% ( 6600, 9900 ] 4 0.002% 100.000% ( 14000, 22000 ] 1 0.000% 100.001% ** Level 2 read latency histogram (micros): Count: 452808 Average: 12.8591 StdDev: 160.06 Min: 1 Median: 1.9511 Max: 9220 Percentiles: P50: 1.95 P75: 2.97 P99: 11.68 P99.9: 1643.82 P99.99: 2620.24 ------------------------------------------------------ [ 0, 1 ] 12634 2.790% 2.790% # ( 1, 2 ] 224764 49.638% 52.428% ########## ( 2, 3 ] 105061 23.202% 75.630% ##### ( 3, 4 ] 45460 10.040% 85.670% ## ( 4, 6 ] 46348 10.236% 95.905% ## ( 6, 10 ] 13227 2.921% 98.826% # ( 10, 15 ] 2346 0.518% 99.345% ( 15, 22 ] 344 0.076% 99.421% ( 22, 34 ] 63 0.014% 99.434% ( 34, 51 ] 44 0.010% 99.444% ( 51, 76 ] 26 0.006% 99.450% ( 76, 110 ] 27 0.006% 99.456% ( 110, 170 ] 65 0.014% 99.470% ( 170, 250 ] 95 0.021% 99.491% ( 250, 380 ] 201 0.044% 99.536% ( 380, 580 ] 341 0.075% 99.611% ( 580, 870 ] 528 0.117% 99.727% ( 870, 1300 ] 560 0.124% 99.851% ( 1300, 1900 ] 386 0.085% 99.936% ( 1900, 2900 ] 337 0.074% 100.011% ( 2900, 4400 ] 343 0.076% 100.087% ( 4400, 6600 ] 109 0.024% 100.111% ( 6600, 9900 ] 11 0.002% 100.113% ** Level 3 read latency histogram (micros): Count: 1341524 Average: 12.4539 StdDev: 159.21 Min: 1 Median: 1.9383 Max: 18986 Percentiles: P50: 1.94 P75: 2.92 P99: 9.94 P99.9: 896.57 P99.99: 1189.55 ------------------------------------------------------ [ 0, 1 ] 26789 1.997% 1.997% ( 1, 2 ] 686348 51.162% 53.159% ########## ( 2, 3 ] 317891 23.696% 76.855% ##### ( 3, 4 ] 132218 9.856% 86.711% ## ( 4, 6 ] 130066 9.695% 96.406% ## ( 6, 10 ] 35286 2.630% 99.036% # ( 10, 15 ] 5539 0.413% 99.449% ( 15, 22 ] 718 0.054% 99.503% ( 22, 34 ] 49 0.004% 99.507% ( 34, 51 ] 71 0.005% 99.512% ( 51, 76 ] 80 0.006% 99.518% ( 76, 110 ] 254 0.019% 99.537% ( 110, 170 ] 803 0.060% 99.597% ( 170, 250 ] 539 0.040% 99.637% ( 250, 380 ] 645 0.048% 99.685% ( 380, 580 ] 1139 0.085% 99.770% ( 580, 870 ] 1638 0.122% 99.892% ( 870, 1300 ] 1772 0.132% 100.024% ( 1300, 1900 ] 1199 0.089% 100.113% ( 1900, 2900 ] 919 0.069% 100.182% ( 2900, 4400 ] 846 0.063% 100.245% ( 4400, 6600 ] 243 0.018% 100.263% ( 6600, 9900 ] 45 0.003% 100.266% ( 9900, 14000 ] 7 0.001% 100.267% ( 14000, 22000 ] 9 0.001% 100.268% ** Level 4 read latency histogram (micros): Count: 848718 Average: 23.0839 StdDev: 209.13 Min: 1 Median: 1.9469 Max: 21134 Percentiles: P50: 1.95 P75: 2.98 P99: 458.37 P99.9: 2026.82 P99.99: 2806.25 ------------------------------------------------------ [ 0, 1 ] 17603 2.074% 2.074% ( 1, 2 ] 429555 50.612% 52.686% ########## ( 2, 3 ] 192649 22.699% 75.385% ##### ( 3, 4 ] 82518 9.723% 85.108% ## ( 4, 6 ] 78284 9.224% 94.332% ## ( 6, 10 ] 20370 2.400% 96.732% ( 10, 15 ] 3002 0.354% 97.085% ( 15, 22 ] 398 0.047% 97.132% ( 22, 34 ] 31 0.004% 97.136% ( 34, 51 ] 59 0.007% 97.143% ( 51, 76 ] 64 0.008% 97.150% ( 76, 110 ] 1990 0.234% 97.385% ( 110, 170 ] 6633 0.782% 98.166% ( 170, 250 ] 3728 0.439% 98.606% ( 250, 380 ] 2310 0.272% 98.878% ( 380, 580 ] 2646 0.312% 99.190% ( 580, 870 ] 2413 0.284% 99.474% ( 870, 1300 ] 2168 0.255% 99.729% ( 1300, 1900 ] 1324 0.156% 99.885% ( 1900, 2900 ] 980 0.115% 100.001% ( 2900, 4400 ] 843 0.099% 100.100% ( 4400, 6600 ] 286 0.034% 100.134% ( 6600, 9900 ] 41 0.005% 100.139% ( 9900, 14000 ] 5 0.001% 100.139% ( 14000, 22000 ] 9 0.001% 100.140% ** Level 5 read latency histogram (micros): Count: 120915 Average: 478.3564 StdDev: 801.39 Min: 73 Median: 201.9154 Max: 22050 Percentiles: P50: 201.92 P75: 465.83 P99: 4289.22 P99.9: 7077.44 P99.99: 10769.43 ------------------------------------------------------ ( 51, 76 ] 16 0.013% 0.013% ( 76, 110 ] 12093 10.001% 10.014% ## ( 110, 170 ] 39859 32.964% 42.979% ####### ( 170, 250 ] 21280 17.599% 60.578% #### ( 250, 380 ] 12329 10.196% 70.775% ## ( 380, 580 ] 11905 9.846% 80.620% ## ( 580, 870 ] 8355 6.910% 87.530% # ( 870, 1300 ] 5906 4.884% 92.415% # ( 1300, 1900 ] 3385 2.799% 95.214% # ( 1900, 2900 ] 2419 2.001% 97.215% ( 2900, 4400 ] 2331 1.928% 99.142% ( 4400, 6600 ] 898 0.743% 99.885% ( 6600, 9900 ] 125 0.103% 99.988% ( 9900, 14000 ] 9 0.007% 99.996% ( 14000, 22000 ] 11 0.009% 100.005% ( 22000, 33000 ] 1 0.001% 100.006% ** DB Stats ** Uptime(secs): 20.7 total, 3.7 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 190.43 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.822 H:M:S, 28.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 265.13 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.265 H:M:S, 7.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 2024/11/28-19:54:11 ... thread 0: (1000000,5000000) ops and (186198.6,193008.6) ops/second in (5.370610,25.905580) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/7 3.57 GB 207.7 3.9 0.0 3.9 8.7 4.8 0.0 1.8 100.2 224.2 39.60 29.22 86 0.460 4029K 4015 0.0 0.0 L1 10/0 1.18 GB 120.9 2.2 1.3 0.9 2.2 1.3 0.0 1.7 107.7 107.3 21.16 11.45 3 7.053 2313K 9171 0.0 0.0 L3 20/1 1.07 GB 1.0 4.1 2.6 1.5 4.0 2.5 0.0 1.6 109.1 108.2 38.08 19.19 19 2.004 2546K 231 0.0 0.0 L4 182/4 9.65 GB 1.0 7.7 4.5 3.3 7.6 4.4 0.5 1.7 119.5 117.7 66.31 36.81 50 1.326 3140K 301 0.0 0.0 L5 1152/0 94.80 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1382/12 110.26 GB 0.0 17.9 8.4 9.5 22.5 13.0 5.5 4.7 111.0 139.7 165.15 96.68 158 1.045 12M 13K 0.0 0.0 Int 0/0 0.00 KB 0.0 17.9 8.4 9.5 22.5 13.0 5.5 4.7 111.0 139.7 165.15 96.68 158 1.045 12M 13K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 17.9 8.4 9.5 17.7 8.2 0.0 0.0 128.5 127.4 142.56 82.68 87 1.639 12M 13K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.8 4.8 0.0 0.0 0.0 217.6 22.58 14.00 71 0.318 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 26.0 total, 25.9 interval Flush(GB): cumulative 4.799, interval 4.799 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 22.53 GB write, 885.95 MB/s write, 17.90 GB read, 703.63 MB/s read, 165.1 seconds Interval compaction: 22.53 GB write, 890.59 MB/s write, 17.90 GB read, 707.32 MB/s read, 165.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 11, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 11, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 53, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 11, total-stops: 53, interval: 64 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1601492 Average: 10.2189 StdDev: 95.77 Min: 0 Median: 1.5782 Max: 15391 Percentiles: P50: 1.58 P75: 2.00 P99: 190.36 P99.9: 860.41 P99.99: 1600.84 ------------------------------------------------------ [ 0, 1 ] 251543 15.707% 15.707% ### ( 1, 2 ] 949865 59.311% 75.018% ############ ( 2, 3 ] 211004 13.175% 88.194% ### ( 3, 4 ] 76869 4.800% 92.993% # ( 4, 6 ] 58008 3.622% 96.615% # ( 6, 10 ] 13393 0.836% 97.452% ( 10, 15 ] 2141 0.134% 97.585% ( 15, 22 ] 352 0.022% 97.607% ( 22, 34 ] 40 0.002% 97.610% ( 34, 51 ] 17 0.001% 97.611% ( 51, 76 ] 28 0.002% 97.613% ( 76, 110 ] 4494 0.281% 97.893% ( 110, 170 ] 15756 0.984% 98.877% ( 170, 250 ] 7734 0.483% 99.360% ( 250, 380 ] 3677 0.230% 99.590% ( 380, 580 ] 3260 0.204% 99.793% ( 580, 870 ] 1769 0.110% 99.904% ( 870, 1300 ] 1077 0.067% 99.971% ( 1300, 1900 ] 610 0.038% 100.009% ( 1900, 2900 ] 427 0.027% 100.036% ( 2900, 4400 ] 332 0.021% 100.056% ( 4400, 6600 ] 81 0.005% 100.062% ( 6600, 9900 ] 4 0.000% 100.062% ( 9900, 14000 ] 1 0.000% 100.062% ( 14000, 22000 ] 2 0.000% 100.062% ** Level 1 read latency histogram (micros): Count: 294591 Average: 9.3471 StdDev: 108.83 Min: 0 Median: 1.8382 Max: 16275 Percentiles: P50: 1.84 P75: 2.78 P99: 137.29 P99.9: 1486.19 P99.99: 4192.91 ------------------------------------------------------ [ 0, 1 ] 13818 4.691% 4.691% # ( 1, 2 ] 159248 54.057% 58.748% ########### ( 2, 3 ] 61570 20.900% 79.648% #### ( 3, 4 ] 23841 8.093% 87.741% ## ( 4, 6 ] 23688 8.041% 95.782% ## ( 6, 10 ] 7036 2.388% 98.170% ( 10, 15 ] 1084 0.368% 98.538% ( 15, 22 ] 303 0.103% 98.641% ( 22, 34 ] 20 0.007% 98.648% ( 34, 51 ] 6 0.002% 98.650% ( 51, 76 ] 7 0.002% 98.652% ( 76, 110 ] 401 0.136% 98.788% ( 110, 170 ] 1370 0.465% 99.254% ( 170, 250 ] 680 0.231% 99.484% ( 250, 380 ] 348 0.118% 99.602% ( 380, 580 ] 358 0.122% 99.724% ( 580, 870 ] 256 0.087% 99.811% ( 870, 1300 ] 214 0.073% 99.884% ( 1300, 1900 ] 156 0.053% 99.937% ( 1900, 2900 ] 86 0.029% 99.966% ( 2900, 4400 ] 83 0.028% 99.994% ( 4400, 6600 ] 19 0.006% 100.000% ( 6600, 9900 ] 4 0.001% 100.002% ( 14000, 22000 ] 1 0.000% 100.002% ** Level 2 read latency histogram (micros): Count: 463518 Average: 12.6613 StdDev: 158.25 Min: 1 Median: 1.9440 Max: 9220 Percentiles: P50: 1.94 P75: 2.96 P99: 11.68 P99.9: 1627.17 P99.99: 2617.06 ------------------------------------------------------ [ 0, 1 ] 13125 2.832% 2.832% # ( 1, 2 ] 231600 49.966% 52.797% ########## ( 2, 3 ] 107128 23.112% 75.909% ##### ( 3, 4 ] 46015 9.927% 85.837% ## ( 4, 6 ] 46790 10.095% 95.931% ## ( 6, 10 ] 13432 2.898% 98.829% # ( 10, 15 ] 2366 0.510% 99.339% ( 15, 22 ] 348 0.075% 99.414% ( 22, 34 ] 63 0.014% 99.428% ( 34, 51 ] 44 0.009% 99.438% ( 51, 76 ] 26 0.006% 99.443% ( 76, 110 ] 38 0.008% 99.451% ( 110, 170 ] 113 0.024% 99.476% ( 170, 250 ] 109 0.024% 99.499% ( 250, 380 ] 204 0.044% 99.543% ( 380, 580 ] 350 0.076% 99.619% ( 580, 870 ] 533 0.115% 99.734% ( 870, 1300 ] 560 0.121% 99.855% ( 1300, 1900 ] 386 0.083% 99.938% ( 1900, 2900 ] 337 0.073% 100.011% ( 2900, 4400 ] 343 0.074% 100.085% ( 4400, 6600 ] 109 0.024% 100.108% ( 6600, 9900 ] 11 0.002% 100.110% ** Level 3 read latency histogram (micros): Count: 1438402 Average: 12.0220 StdDev: 154.03 Min: 1 Median: 1.9215 Max: 18986 Percentiles: P50: 1.92 P75: 2.89 P99: 10.24 P99.9: 876.64 P99.99: 1188.33 ------------------------------------------------------ [ 0, 1 ] 30726 2.136% 2.136% ( 1, 2 ] 747146 51.943% 54.079% ########## ( 2, 3 ] 337667 23.475% 77.554% ##### ( 3, 4 ] 137521 9.561% 87.115% ## ( 4, 6 ] 134017 9.317% 96.432% ## ( 6, 10 ] 36675 2.550% 98.982% # ( 10, 15 ] 5668 0.394% 99.376% ( 15, 22 ] 746 0.052% 99.427% ( 22, 34 ] 50 0.003% 99.431% ( 34, 51 ] 72 0.005% 99.436% ( 51, 76 ] 81 0.006% 99.442% ( 76, 110 ] 484 0.034% 99.475% ( 110, 170 ] 1425 0.099% 99.574% ( 170, 250 ] 880 0.061% 99.635% ( 250, 380 ] 819 0.057% 99.692% ( 380, 580 ] 1268 0.088% 99.781% ( 580, 870 ] 1692 0.118% 99.898% ( 870, 1300 ] 1786 0.124% 100.022% ( 1300, 1900 ] 1201 0.083% 100.106% ( 1900, 2900 ] 919 0.064% 100.170% ( 2900, 4400 ] 846 0.059% 100.229% ( 4400, 6600 ] 243 0.017% 100.245% ( 6600, 9900 ] 45 0.003% 100.249% ( 9900, 14000 ] 7 0.000% 100.249% ( 14000, 22000 ] 9 0.001% 100.250% ** Level 4 read latency histogram (micros): Count: 874156 Average: 25.2037 StdDev: 207.95 Min: 1 Median: 1.9554 Max: 21134 Percentiles: P50: 1.96 P75: 3.04 P99: 497.39 P99.9: 2001.47 P99.99: 2801.00 ------------------------------------------------------ [ 0, 1 ] 18204 2.082% 2.082% ( 1, 2 ] 438436 50.155% 52.238% ########## ( 2, 3 ] 195510 22.366% 74.603% #### ( 3, 4 ] 83291 9.528% 84.132% ## ( 4, 6 ] 78882 9.024% 93.155% ## ( 6, 10 ] 20660 2.363% 95.519% ( 10, 15 ] 3037 0.347% 95.866% ( 15, 22 ] 408 0.047% 95.913% ( 22, 34 ] 31 0.004% 95.916% ( 34, 51 ] 59 0.007% 95.923% ( 51, 76 ] 64 0.007% 95.930% ( 76, 110 ] 3575 0.409% 96.339% ( 110, 170 ] 11740 1.343% 97.682% ( 170, 250 ] 6048 0.692% 98.374% ( 250, 380 ] 3422 0.391% 98.766% ( 380, 580 ] 3490 0.399% 99.165% ( 580, 870 ] 2749 0.314% 99.479% ( 870, 1300 ] 2245 0.257% 99.736% ( 1300, 1900 ] 1332 0.152% 99.889% ( 1900, 2900 ] 984 0.113% 100.001% ( 2900, 4400 ] 843 0.096% 100.098% ( 4400, 6600 ] 286 0.033% 100.130% ( 6600, 9900 ] 41 0.005% 100.135% ( 9900, 14000 ] 5 0.001% 100.136% ( 14000, 22000 ] 9 0.001% 100.137% ** Level 5 read latency histogram (micros): Count: 184983 Average: 385.3143 StdDev: 667.76 Min: 73 Median: 175.1606 Max: 22050 Percentiles: P50: 175.16 P75: 353.11 P99: 3878.53 P99.9: 6484.89 P99.99: 9705.16 ------------------------------------------------------ ( 51, 76 ] 23 0.012% 0.012% ( 76, 110 ] 21558 11.654% 11.666% ## ( 110, 170 ] 68728 37.154% 48.820% ####### ( 170, 250 ] 33842 18.295% 67.115% #### ( 250, 380 ] 18393 9.943% 77.058% ## ( 380, 580 ] 16692 9.024% 86.081% ## ( 580, 870 ] 10208 5.518% 91.600% # ( 870, 1300 ] 6290 3.400% 95.000% # ( 1300, 1900 ] 3426 1.852% 96.852% ( 1900, 2900 ] 2448 1.323% 98.176% ( 2900, 4400 ] 2341 1.266% 99.441% ( 4400, 6600 ] 898 0.485% 99.926% ( 6600, 9900 ] 127 0.069% 99.995% ( 9900, 14000 ] 9 0.005% 100.000% ( 14000, 22000 ] 11 0.006% 100.006% ( 22000, 33000 ] 1 0.001% 100.006% ** DB Stats ** Uptime(secs): 26.0 total, 5.4 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 188.95 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.552 H:M:S, 29.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 183.26 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.730 H:M:S, 32.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 2024/11/28-19:54:17 ... thread 0: (1000000,6000000) ops and (183988.1,191444.3) ops/second in (5.435135,31.340715) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/9 4.42 GB 319.9 6.2 0.0 6.2 11.8 5.6 0.0 2.1 120.4 230.9 52.31 40.53 104 0.503 6395K 15K 0.0 0.0 L1 10/0 1.18 GB 120.9 2.2 1.3 0.9 2.2 1.3 0.0 1.7 107.7 107.3 21.16 11.45 3 7.053 2313K 9171 0.0 0.0 L3 19/0 964.10 MB 1.0 4.1 2.6 1.5 4.0 2.5 0.0 1.6 109.1 108.2 38.08 19.19 19 2.004 2546K 231 0.0 0.0 L4 179/0 9.46 GB 1.0 8.0 4.6 3.4 7.9 4.5 0.5 1.7 121.4 119.6 67.78 38.17 51 1.329 3189K 301 0.0 0.0 L5 1156/0 95.11 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1381/9 111.10 GB 0.0 20.5 8.5 12.0 26.0 14.0 5.8 4.6 116.9 148.2 179.33 109.34 177 1.013 14M 25K 0.0 0.0 Int 0/0 0.00 KB 0.0 20.5 8.5 12.0 26.0 14.0 5.8 4.6 116.9 148.2 179.33 109.34 177 1.013 14M 25K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 20.5 8.5 12.0 20.3 8.3 0.0 0.0 136.2 135.0 153.86 92.87 92 1.672 14M 25K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 227.6 25.47 16.47 85 0.300 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 31.5 total, 31.3 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 25.95 GB write, 844.17 MB/s write, 20.47 GB read, 665.89 MB/s read, 179.3 seconds Interval compaction: 25.95 GB write, 847.83 MB/s write, 20.47 GB read, 668.78 MB/s read, 179.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 20, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 20, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 53, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 20, total-stops: 53, interval: 73 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2180337 Average: 9.5976 StdDev: 85.18 Min: 0 Median: 1.5462 Max: 15391 Percentiles: P50: 1.55 P75: 1.96 P99: 186.76 P99.9: 746.27 P99.99: 1264.78 ------------------------------------------------------ [ 0, 1 ] 378865 17.376% 17.376% ### ( 1, 2 ] 1302271 59.728% 77.104% ############ ( 2, 3 ] 263820 12.100% 89.204% ## ( 3, 4 ] 93037 4.267% 93.471% # ( 4, 6 ] 67447 3.093% 96.565% # ( 6, 10 ] 16913 0.776% 97.341% ( 10, 15 ] 2544 0.117% 97.457% ( 15, 22 ] 405 0.019% 97.476% ( 22, 34 ] 46 0.002% 97.478% ( 34, 51 ] 17 0.001% 97.479% ( 51, 76 ] 38 0.002% 97.480% ( 76, 110 ] 6799 0.312% 97.792% ( 110, 170 ] 23974 1.100% 98.892% ( 170, 250 ] 11252 0.516% 99.408% ( 250, 380 ] 5163 0.237% 99.645% ( 380, 580 ] 4345 0.199% 99.844% ( 580, 870 ] 2129 0.098% 99.942% ( 870, 1300 ] 1148 0.053% 99.994% ( 1300, 1900 ] 619 0.028% 100.023% ( 1900, 2900 ] 439 0.020% 100.043% ( 2900, 4400 ] 338 0.016% 100.058% ( 4400, 6600 ] 83 0.004% 100.062% ( 6600, 9900 ] 4 0.000% 100.062% ( 9900, 14000 ] 1 0.000% 100.062% ( 14000, 22000 ] 2 0.000% 100.062% ** Level 1 read latency histogram (micros): Count: 296114 Average: 10.2584 StdDev: 109.86 Min: 0 Median: 1.8429 Max: 16275 Percentiles: P50: 1.84 P75: 2.80 P99: 163.91 P99.9: 1490.82 P99.99: 4190.16 ------------------------------------------------------ [ 0, 1 ] 13818 4.666% 4.666% # ( 1, 2 ] 159253 53.781% 58.447% ########### ( 2, 3 ] 61609 20.806% 79.253% #### ( 3, 4 ] 23886 8.066% 87.320% ## ( 4, 6 ] 23744 8.019% 95.338% ## ( 6, 10 ] 7043 2.378% 97.717% ( 10, 15 ] 1088 0.367% 98.084% ( 15, 22 ] 304 0.103% 98.187% ( 22, 34 ] 20 0.007% 98.194% ( 34, 51 ] 6 0.002% 98.196% ( 51, 76 ] 7 0.002% 98.198% ( 76, 110 ] 594 0.201% 98.399% ( 110, 170 ] 1982 0.669% 99.068% ( 170, 250 ] 974 0.329% 99.397% ( 250, 380 ] 470 0.159% 99.556% ( 380, 580 ] 461 0.156% 99.711% ( 580, 870 ] 288 0.097% 99.809% ( 870, 1300 ] 220 0.074% 99.883% ( 1300, 1900 ] 160 0.054% 99.937% ( 1900, 2900 ] 86 0.029% 99.966% ( 2900, 4400 ] 83 0.028% 99.994% ( 4400, 6600 ] 19 0.006% 100.000% ( 6600, 9900 ] 4 0.001% 100.002% ( 14000, 22000 ] 1 0.000% 100.002% ** Level 2 read latency histogram (micros): Count: 463518 Average: 12.6613 StdDev: 158.25 Min: 1 Median: 1.9440 Max: 9220 Percentiles: P50: 1.94 P75: 2.96 P99: 11.68 P99.9: 1627.17 P99.99: 2617.06 ------------------------------------------------------ [ 0, 1 ] 13125 2.832% 2.832% # ( 1, 2 ] 231600 49.966% 52.797% ########## ( 2, 3 ] 107128 23.112% 75.909% ##### ( 3, 4 ] 46015 9.927% 85.837% ## ( 4, 6 ] 46790 10.095% 95.931% ## ( 6, 10 ] 13432 2.898% 98.829% # ( 10, 15 ] 2366 0.510% 99.339% ( 15, 22 ] 348 0.075% 99.414% ( 22, 34 ] 63 0.014% 99.428% ( 34, 51 ] 44 0.009% 99.438% ( 51, 76 ] 26 0.006% 99.443% ( 76, 110 ] 38 0.008% 99.451% ( 110, 170 ] 113 0.024% 99.476% ( 170, 250 ] 109 0.024% 99.499% ( 250, 380 ] 204 0.044% 99.543% ( 380, 580 ] 350 0.076% 99.619% ( 580, 870 ] 533 0.115% 99.734% ( 870, 1300 ] 560 0.121% 99.855% ( 1300, 1900 ] 386 0.083% 99.938% ( 1900, 2900 ] 337 0.073% 100.011% ( 2900, 4400 ] 343 0.074% 100.085% ( 4400, 6600 ] 109 0.024% 100.108% ( 6600, 9900 ] 11 0.002% 100.110% ** Level 3 read latency histogram (micros): Count: 1467638 Average: 11.9680 StdDev: 152.63 Min: 1 Median: 1.9159 Max: 18986 Percentiles: P50: 1.92 P75: 2.88 P99: 10.85 P99.9: 871.29 P99.99: 1188.24 ------------------------------------------------------ [ 0, 1 ] 32918 2.243% 2.243% ( 1, 2 ] 765220 52.140% 54.382% ########## ( 2, 3 ] 342659 23.348% 77.730% ##### ( 3, 4 ] 138915 9.465% 87.195% ## ( 4, 6 ] 135089 9.205% 96.400% ## ( 6, 10 ] 37192 2.534% 98.934% # ( 10, 15 ] 5693 0.388% 99.322% ( 15, 22 ] 756 0.052% 99.373% ( 22, 34 ] 51 0.003% 99.377% ( 34, 51 ] 72 0.005% 99.382% ( 51, 76 ] 81 0.006% 99.387% ( 76, 110 ] 617 0.042% 99.429% ( 110, 170 ] 1855 0.126% 99.556% ( 170, 250 ] 1076 0.073% 99.629% ( 250, 380 ] 907 0.062% 99.691% ( 380, 580 ] 1344 0.092% 99.782% ( 580, 870 ] 1720 0.117% 99.900% ( 870, 1300 ] 1792 0.122% 100.022% ( 1300, 1900 ] 1201 0.082% 100.104% ( 1900, 2900 ] 920 0.063% 100.166% ( 2900, 4400 ] 846 0.058% 100.224% ( 4400, 6600 ] 243 0.017% 100.240% ( 6600, 9900 ] 45 0.003% 100.244% ( 9900, 14000 ] 7 0.000% 100.244% ( 14000, 22000 ] 9 0.001% 100.245% ** Level 4 read latency histogram (micros): Count: 916494 Average: 26.6340 StdDev: 204.78 Min: 1 Median: 1.9544 Max: 21134 Percentiles: P50: 1.95 P75: 3.09 P99: 509.82 P99.9: 1958.26 P99.99: 2793.97 ------------------------------------------------------ [ 0, 1 ] 20578 2.245% 2.245% ( 1, 2 ] 458560 50.034% 52.279% ########## ( 2, 3 ] 200952 21.926% 74.206% #### ( 3, 4 ] 84785 9.251% 83.457% ## ( 4, 6 ] 80000 8.729% 92.186% ## ( 6, 10 ] 21224 2.316% 94.501% ( 10, 15 ] 3076 0.336% 94.837% ( 15, 22 ] 413 0.045% 94.882% ( 22, 34 ] 31 0.003% 94.885% ( 34, 51 ] 59 0.006% 94.892% ( 51, 76 ] 64 0.007% 94.899% ( 76, 110 ] 5113 0.558% 95.457% ( 110, 170 ] 16895 1.843% 97.300% ( 170, 250 ] 8355 0.912% 98.212% ( 250, 380 ] 4432 0.484% 98.695% ( 380, 580 ] 4303 0.470% 99.165% ( 580, 870 ] 3037 0.331% 99.496% ( 870, 1300 ] 2303 0.251% 99.748% ( 1300, 1900 ] 1341 0.146% 99.894% ( 1900, 2900 ] 987 0.108% 100.002% ( 2900, 4400 ] 843 0.092% 100.094% ( 4400, 6600 ] 286 0.031% 100.125% ( 6600, 9900 ] 41 0.004% 100.129% ( 9900, 14000 ] 5 0.001% 100.130% ( 14000, 22000 ] 9 0.001% 100.131% ** Level 5 read latency histogram (micros): Count: 247034 Average: 340.6555 StdDev: 588.18 Min: 73 Median: 167.5827 Max: 22050 Percentiles: P50: 167.58 P75: 312.01 P99: 3485.49 P99.9: 6331.03 P99.99: 9517.94 ------------------------------------------------------ ( 51, 76 ] 30 0.012% 0.012% ( 76, 110 ] 30107 12.187% 12.200% ## ( 110, 170 ] 97300 39.387% 51.587% ######## ( 170, 250 ] 46182 18.695% 70.281% #### ( 250, 380 ] 24439 9.893% 80.174% ## ( 380, 580 ] 21309 8.626% 88.800% ## ( 580, 870 ] 11758 4.760% 93.560% # ( 870, 1300 ] 6593 2.669% 96.229% # ( 1300, 1900 ] 3458 1.400% 97.629% ( 1900, 2900 ] 2470 1.000% 98.629% ( 2900, 4400 ] 2351 0.952% 99.580% ( 4400, 6600 ] 900 0.364% 99.945% ( 6600, 9900 ] 127 0.051% 99.996% ( 9900, 14000 ] 9 0.004% 100.000% ( 14000, 22000 ] 11 0.004% 100.004% ( 22000, 33000 ] 1 0.000% 100.004% ** DB Stats ** Uptime(secs): 31.5 total, 5.4 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 187.59 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.403 H:M:S, 29.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 181.08 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.851 H:M:S, 34.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 259.176 micros/op 15427 ops/sec 60.088 seconds 926996 operations; 14.9 MB/s (228999 of 228999 found) Microseconds per read: Count: 926996 Average: 259.1786 StdDev: 718.21 Min: 2 Median: 179.2693 Max: 70108 Percentiles: P50: 179.27 P75: 238.94 P99: 1420.73 P99.9: 11774.32 P99.99: 29470.28 ------------------------------------------------------ ( 1, 2 ] 5 0.001% 0.001% ( 2, 3 ] 51 0.006% 0.006% ( 3, 4 ] 73 0.008% 0.014% ( 4, 6 ] 74 0.008% 0.022% ( 6, 10 ] 326 0.035% 0.057% ( 10, 15 ] 2254 0.243% 0.300% ( 15, 22 ] 3474 0.375% 0.675% ( 22, 34 ] 9552 1.030% 1.705% ( 34, 51 ] 3257 0.351% 2.057% ( 51, 76 ] 320 0.035% 2.091% ( 76, 110 ] 15089 1.628% 3.719% ( 110, 170 ] 393025 42.398% 46.117% ######## ( 170, 250 ] 310686 33.515% 79.632% ####### ( 250, 380 ] 102378 11.044% 90.676% ## ( 380, 580 ] 46241 4.988% 95.664% # ( 580, 870 ] 20941 2.259% 97.923% ( 870, 1300 ] 9184 0.991% 98.914% ( 1300, 1900 ] 3956 0.427% 99.341% ( 1900, 2900 ] 2285 0.246% 99.587% ( 2900, 4400 ] 1826 0.197% 99.784% ( 4400, 6600 ] 613 0.066% 99.850% ( 6600, 9900 ] 251 0.027% 99.878% ( 9900, 14000 ] 455 0.049% 99.927% ( 14000, 22000 ] 480 0.052% 99.978% ( 22000, 33000 ] 158 0.017% 99.995% ( 33000, 50000 ] 33 0.004% 99.999% ( 50000, 75000 ] 9 0.001% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 15079964 rocksdb.bloom.filter.full.positive COUNT : 1073769 rocksdb.bloom.filter.full.true.positive COUNT : 926510 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 486 rocksdb.memtable.miss COUNT : 926510 rocksdb.l0.hit COUNT : 38839 rocksdb.l1.hit COUNT : 9766 rocksdb.l2andup.hit COUNT : 877905 rocksdb.compaction.key.drop.new COUNT : 212792 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 6233119 rocksdb.number.keys.read COUNT : 926996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 6432578808 rocksdb.bytes.read COUNT : 926996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1689 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 36436588 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 24469748558 rocksdb.compact.write.bytes COUNT : 24246890465 rocksdb.flush.write.bytes COUNT : 6407669158 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 27 rocksdb.number.superversion_acquires COUNT : 1219 rocksdb.number.superversion_releases COUNT : 1169 rocksdb.number.superversion_cleanups COUNT : 304 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 310 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 6355849216 rocksdb.memtable.garbage.bytes.at.flush COUNT : 2336768 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 28478847200 rocksdb.non.last.level.read.count COUNT : 6737048 rocksdb.block.checksum.compute.count COUNT : 7010292 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 177.588633 P95 : 552.670164 P99 : 1419.308742 P100 : 70107.000000 COUNT : 926996 SUM : 239356853 rocksdb.db.write.micros P50 : 1.555557 P95 : 4.388445 P99 : 9.009961 P100 : 897750.000000 COUNT : 6233119 SUM : 52041064 rocksdb.compaction.times.micros P50 : 1360000.000000 P95 : 3824000.000000 P99 : 10052000.000000 P100 : 11566813.000000 COUNT : 94 SUM : 158850368 rocksdb.compaction.times.cpu_micros P50 : 860000.000000 P95 : 1855454.545455 P99 : 6698000.000000 P100 : 7585108.000000 COUNT : 94 SUM : 97759533 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 820.638298 P95 : 54166.666667 P99 : 178000.000000 P100 : 232289.000000 COUNT : 90 SUM : 818611 rocksdb.compaction.outfile.sync.micros P50 : 28217.391304 P95 : 249529.411765 P99 : 361800.000000 P100 : 546343.000000 COUNT : 282 SUM : 17965316 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 204.285714 P95 : 3670.000000 P99 : 5089.333333 P100 : 6237.000000 COUNT : 206 SUM : 165749 rocksdb.table.open.io.micros P50 : 8508.953722 P95 : 19836.097561 P99 : 26593.448276 P100 : 30822.000000 COUNT : 1689 SUM : 14618686 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.885625 P95 : 9.885956 P99 : 163.277257 P100 : 29754.000000 COUNT : 5925135 SUM : 84935884 rocksdb.read.block.get.micros P50 : 141.855654 P95 : 455.805181 P99 : 1297.582422 P100 : 30251.000000 COUNT : 1083482 SUM : 204481110 rocksdb.write.raw.block.micros P50 : 0.527170 P95 : 1.046978 P99 : 3.616775 P100 : 41314.000000 COUNT : 7424589 SUM : 20918621 rocksdb.numfiles.in.singlecompaction P50 : 2.724490 P95 : 5.025000 P99 : 9.746667 P100 : 23.000000 COUNT : 119 SUM : 409 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.568392 P95 : 1228.923804 P99 : 2624.807350 P100 : 897585.000000 COUNT : 187200 SUM : 36452658 rocksdb.sst.read.micros P50 : 1.929419 P95 : 164.288835 P99 : 501.106061 P100 : 22050.000000 COUNT : 6737107 SUM : 271402397 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 926996 SUM : 926996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 6233119 SUM : 6432578808 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 220000.000000 P95 : 839285.714286 P99 : 1123500.000000 P100 : 1146934.000000 COUNT : 90 SUM : 26499392 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 310 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0