[2024-12-16 08:37:21.642468] Starting SPDK v25.01-pre git sha1 e01cb43b8 / DPDK 22.11.4 initialization... [2024-12-16 08:37:21.642762] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid195551 ] Set seed to 1734334641635797 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-16 08:37:21.749539] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-16 08:37:21.772933] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-16 08:37:21.772977] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 195551' to capture a snapshot of events at runtime. [2024-12-16 08:37:21.772987] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-16 08:37:21.772997] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-16 08:37:21.773005] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid195551 for offline analysis/debug. [2024-12-16 08:37:21.773562] reactor.c: 995:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Dec 16 08:37:26 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/16-08:37:33 ... thread 0: (1000000,1000000) ops and (135821.9,135821.9) ops/second in (7.362583,7.362583) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 6/4 818.32 MB 18.9 0.4 0.0 0.4 1.2 0.8 0.0 1.5 38.4 121.6 9.84 3.94 10 0.984 383K 533 0.0 0.0 L1 3/3 248.14 MB 0.0 0.6 0.5 0.1 0.6 0.5 0.0 1.2 114.6 114.3 5.72 3.12 1 5.724 666K 1513 0.0 0.0 L2 8/5 701.77 MB 1.5 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 41/13 2.91 GB 1.9 2.1 1.4 0.7 2.1 1.4 0.0 1.5 98.5 97.8 22.34 10.75 13 1.718 1642K 552 0.0 0.0 L4 208/9 11.91 GB 1.2 3.4 2.4 1.0 3.4 2.4 0.0 1.4 105.1 104.5 32.98 16.67 21 1.571 1030K 0 0.0 0.0 L5 1088/0 89.38 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1354/34 105.94 GB 0.0 6.5 4.4 2.2 7.3 5.1 0.4 9.1 94.5 105.5 70.88 34.48 45 1.575 3722K 2598 0.0 0.0 Int 0/0 0.00 KB 0.0 6.5 4.4 2.2 7.3 5.1 0.4 9.1 94.5 105.5 70.88 34.48 45 1.575 3722K 2598 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.5 4.4 2.2 6.5 4.3 0.0 0.0 104.3 103.7 64.23 32.08 36 1.784 3722K 2598 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 123.1 6.65 2.40 9 0.739 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.5 total, 7.4 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.30 GB write, 992.08 MB/s write, 6.54 GB read, 888.58 MB/s read, 70.9 seconds Interval compaction: 7.30 GB write, 1015.74 MB/s write, 6.54 GB read, 909.77 MB/s read, 70.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 41, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 41, interval: 41 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 249683 Average: 10.8760 StdDev: 155.82 Min: 1 Median: 1.9153 Max: 16963 Percentiles: P50: 1.92 P75: 2.93 P99: 10.04 P99.9: 2039.16 P99.99: 4107.70 ------------------------------------------------------ [ 0, 1 ] 7887 3.159% 3.159% # ( 1, 2 ] 127783 51.178% 54.337% ########## ( 2, 3 ] 55441 22.205% 76.541% #### ( 3, 4 ] 25007 10.015% 86.557% ## ( 4, 6 ] 24943 9.990% 96.547% ## ( 6, 10 ] 6118 2.450% 98.997% ( 10, 15 ] 1005 0.403% 99.400% ( 15, 22 ] 134 0.054% 99.453% ( 22, 34 ] 14 0.006% 99.459% ( 34, 51 ] 14 0.006% 99.465% ( 51, 76 ] 6 0.002% 99.467% ( 76, 110 ] 7 0.003% 99.470% ( 110, 170 ] 24 0.010% 99.479% ( 170, 250 ] 47 0.019% 99.498% ( 250, 380 ] 94 0.038% 99.536% ( 380, 580 ] 211 0.085% 99.620% ( 580, 870 ] 255 0.102% 99.722% ( 870, 1300 ] 260 0.104% 99.827% ( 1300, 1900 ] 163 0.065% 99.892% ( 1900, 2900 ] 146 0.058% 99.950% ( 2900, 4400 ] 123 0.049% 100.000% ( 4400, 6600 ] 36 0.014% 100.014% ( 6600, 9900 ] 9 0.004% 100.018% ( 9900, 14000 ] 2 0.001% 100.018% ( 14000, 22000 ] 4 0.002% 100.020% ** Level 1 read latency histogram (micros): Count: 29477 Average: 18.7508 StdDev: 184.10 Min: 1 Median: 1.9964 Max: 6119 Percentiles: P50: 2.00 P75: 3.05 P99: 458.00 P99.9: 3252.64 P99.99: 5303.01 ------------------------------------------------------ [ 0, 1 ] 547 1.856% 1.856% ( 1, 2 ] 14243 48.319% 50.175% ########## ( 2, 3 ] 7161 24.294% 74.468% ##### ( 3, 4 ] 2992 10.150% 84.619% ## ( 4, 6 ] 3157 10.710% 95.329% ## ( 6, 10 ] 835 2.833% 98.161% # ( 10, 15 ] 157 0.533% 98.694% ( 15, 22 ] 25 0.085% 98.779% ( 22, 34 ] 2 0.007% 98.785% ( 34, 51 ] 1 0.003% 98.789% ( 51, 76 ] 2 0.007% 98.796% ( 110, 170 ] 6 0.020% 98.816% ( 170, 250 ] 11 0.037% 98.853% ( 250, 380 ] 21 0.071% 98.925% ( 380, 580 ] 57 0.193% 99.118% ( 580, 870 ] 70 0.237% 99.355% ( 870, 1300 ] 61 0.207% 99.562% ( 1300, 1900 ] 61 0.207% 99.769% ( 1900, 2900 ] 31 0.105% 99.874% ( 2900, 4400 ] 32 0.109% 99.983% ( 4400, 6600 ] 5 0.017% 100.000% ** Level 2 read latency histogram (micros): Count: 410612 Average: 13.2662 StdDev: 184.32 Min: 1 Median: 2.0202 Max: 25494 Percentiles: P50: 2.02 P75: 3.06 P99: 12.06 P99.9: 1256.00 P99.99: 1838.57 ------------------------------------------------------ [ 0, 1 ] 10189 2.481% 2.481% ( 1, 2 ] 193054 47.016% 49.498% ######### ( 2, 3 ] 101962 24.832% 74.329% ##### ( 3, 4 ] 43287 10.542% 84.871% ## ( 4, 6 ] 43847 10.678% 95.550% ## ( 6, 10 ] 13059 3.180% 98.730% # ( 10, 15 ] 2689 0.655% 99.385% ( 15, 22 ] 447 0.109% 99.494% ( 22, 34 ] 92 0.022% 99.516% ( 34, 51 ] 43 0.010% 99.527% ( 51, 76 ] 35 0.009% 99.535% ( 76, 110 ] 17 0.004% 99.539% ( 110, 170 ] 50 0.012% 99.552% ( 170, 250 ] 101 0.025% 99.576% ( 250, 380 ] 189 0.046% 99.622% ( 380, 580 ] 337 0.082% 99.704% ( 580, 870 ] 413 0.101% 99.805% ( 870, 1300 ] 436 0.106% 99.911% ( 1300, 1900 ] 362 0.088% 99.999% ( 1900, 2900 ] 303 0.074% 100.073% ( 2900, 4400 ] 302 0.074% 100.147% ( 4400, 6600 ] 91 0.022% 100.169% ( 6600, 9900 ] 28 0.007% 100.176% ( 9900, 14000 ] 8 0.002% 100.178% ( 14000, 22000 ] 4 0.001% 100.179% ( 22000, 33000 ] 1 0.000% 100.179% ** Level 3 read latency histogram (micros): Count: 798426 Average: 14.4956 StdDev: 190.97 Min: 1 Median: 1.9900 Max: 24262 Percentiles: P50: 1.99 P75: 2.97 P99: 11.06 P99.9: 1540.29 P99.99: 2279.59 ------------------------------------------------------ [ 0, 1 ] 13453 1.685% 1.685% ( 1, 2 ] 389663 48.804% 50.489% ########## ( 2, 3 ] 202207 25.326% 75.815% ##### ( 3, 4 ] 81620 10.223% 86.037% ## ( 4, 6 ] 81852 10.252% 96.289% ## ( 6, 10 ] 20821 2.608% 98.897% # ( 10, 15 ] 3921 0.491% 99.388% ( 15, 22 ] 539 0.068% 99.455% ( 22, 34 ] 58 0.007% 99.462% ( 34, 51 ] 52 0.007% 99.469% ( 51, 76 ] 52 0.007% 99.475% ( 76, 110 ] 29 0.004% 99.479% ( 110, 170 ] 83 0.010% 99.489% ( 170, 250 ] 158 0.020% 99.509% ( 250, 380 ] 303 0.038% 99.547% ( 380, 580 ] 594 0.074% 99.622% ( 580, 870 ] 891 0.112% 99.733% ( 870, 1300 ] 1030 0.129% 99.862% ( 1300, 1900 ] 763 0.096% 99.958% ( 1900, 2900 ] 688 0.086% 100.044% ( 2900, 4400 ] 685 0.086% 100.130% ( 4400, 6600 ] 234 0.029% 100.159% ( 6600, 9900 ] 50 0.006% 100.165% ( 9900, 14000 ] 4 0.001% 100.166% ( 14000, 22000 ] 7 0.001% 100.167% ( 22000, 33000 ] 2 0.000% 100.167% ** Level 4 read latency histogram (micros): Count: 272620 Average: 27.2156 StdDev: 258.43 Min: 1 Median: 2.0023 Max: 25985 Percentiles: P50: 2.00 P75: 3.00 P99: 745.54 P99.9: 3262.55 P99.99: 4106.67 ------------------------------------------------------ [ 0, 1 ] 4514 1.656% 1.656% ( 1, 2 ] 131640 48.287% 49.943% ########## ( 2, 3 ] 68315 25.059% 75.001% ##### ( 3, 4 ] 28719 10.534% 85.536% ## ( 4, 6 ] 27404 10.052% 95.588% ## ( 6, 10 ] 6456 2.368% 97.956% ( 10, 15 ] 1075 0.394% 98.350% ( 15, 22 ] 166 0.061% 98.411% ( 22, 34 ] 16 0.006% 98.417% ( 34, 51 ] 13 0.005% 98.422% ( 51, 76 ] 15 0.006% 98.427% ( 76, 110 ] 12 0.004% 98.432% ( 110, 170 ] 71 0.026% 98.458% ( 170, 250 ] 158 0.058% 98.516% ( 250, 380 ] 262 0.096% 98.612% ( 380, 580 ] 592 0.217% 98.829% ( 580, 870 ] 816 0.299% 99.128% ( 870, 1300 ] 886 0.325% 99.453% ( 1300, 1900 ] 619 0.227% 99.681% ( 1900, 2900 ] 493 0.181% 99.861% ( 2900, 4400 ] 436 0.160% 100.021% ( 4400, 6600 ] 146 0.054% 100.075% ( 6600, 9900 ] 20 0.007% 100.082% ( 9900, 14000 ] 2 0.001% 100.083% ( 14000, 22000 ] 4 0.001% 100.084% ( 22000, 33000 ] 1 0.000% 100.085% ** Level 5 read latency histogram (micros): Count: 11660 Average: 1797.8846 StdDev: 1672.69 Min: 80 Median: 1209.0345 Max: 28742 Percentiles: P50: 1209.03 P75: 2551.93 P99: 7580.29 P99.9: 16672.00 P99.99: 28742.00 ------------------------------------------------------ ( 76, 110 ] 9 0.077% 0.077% ( 110, 170 ] 79 0.678% 0.755% ( 170, 250 ] 223 1.913% 2.667% ( 250, 380 ] 639 5.480% 8.148% # ( 380, 580 ] 1283 11.003% 19.151% ## ( 580, 870 ] 1931 16.561% 35.712% ### ( 870, 1300 ] 2113 18.122% 53.834% #### ( 1300, 1900 ] 1539 13.199% 67.033% ### ( 1900, 2900 ] 1425 12.221% 79.254% ## ( 2900, 4400 ] 1561 13.388% 92.642% ### ( 4400, 6600 ] 701 6.012% 98.654% # ( 6600, 9900 ] 136 1.166% 99.820% ( 9900, 14000 ] 6 0.051% 99.871% ( 14000, 22000 ] 10 0.086% 99.957% ( 22000, 33000 ] 5 0.043% 100.000% ** DB Stats ** Uptime(secs): 7.5 total, 7.4 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 130.52 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.358 H:M:S, 44.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 133.62 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.358 H:M:S, 45.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/12/16-08:37:40 ... thread 0: (1000000,2000000) ops and (151021.4,143019.0) ops/second in (6.621576,13.984159) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/8 1.17 GB 56.6 1.1 0.0 1.1 2.9 1.8 0.0 1.6 54.8 142.9 20.70 9.86 25 0.828 1151K 1584 0.0 0.0 L1 4/0 234.09 MB 23.4 1.5 1.2 0.3 1.5 1.2 0.0 1.3 120.0 119.6 12.78 7.23 2 6.391 1557K 4357 0.0 0.0 L2 8/6 654.75 MB 1.8 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 20/10 1.18 GB 0.6 3.6 2.0 1.6 3.6 2.0 0.0 1.7 109.4 108.3 33.88 17.72 20 1.694 2540K 642 0.0 0.0 L4 180/5 9.60 GB 1.0 7.8 4.5 3.3 7.7 4.4 0.3 1.7 112.4 110.9 70.95 37.41 49 1.448 3135K 1720 0.0 0.0 L5 1144/0 94.00 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1368/29 106.81 GB 0.0 14.0 7.7 6.3 15.7 9.3 6.0 8.8 103.7 115.9 138.31 72.23 96 1.441 8384K 8303 0.0 0.0 Int 0/0 0.00 KB 0.0 14.0 7.7 6.3 15.7 9.3 6.0 8.8 103.7 115.9 138.31 72.23 96 1.441 8384K 8303 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.0 7.7 6.3 13.9 7.5 0.0 0.0 114.7 113.5 125.07 66.94 74 1.690 8384K 8303 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 138.0 13.24 5.29 22 0.602 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.2 total, 14.0 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 15.65 GB write, 1131.65 MB/s write, 14.01 GB read, 1013.25 MB/s read, 138.3 seconds Interval compaction: 15.65 GB write, 1145.86 MB/s write, 14.01 GB read, 1025.97 MB/s read, 138.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 62, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 62, interval: 62 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 641609 Average: 10.6514 StdDev: 139.34 Min: 1 Median: 1.8632 Max: 16963 Percentiles: P50: 1.86 P75: 2.82 P99: 10.32 P99.9: 1584.09 P99.99: 3281.74 ------------------------------------------------------ [ 0, 1 ] 24155 3.765% 3.765% # ( 1, 2 ] 343676 53.565% 57.329% ########### ( 2, 3 ] 138468 21.581% 78.911% #### ( 3, 4 ] 58927 9.184% 88.095% ## ( 4, 6 ] 56578 8.818% 96.913% ## ( 6, 10 ] 13255 2.066% 98.979% ( 10, 15 ] 2141 0.334% 99.313% ( 15, 22 ] 254 0.040% 99.352% ( 22, 34 ] 21 0.003% 99.356% ( 34, 51 ] 45 0.007% 99.363% ( 51, 76 ] 27 0.004% 99.367% ( 76, 110 ] 30 0.005% 99.372% ( 110, 170 ] 119 0.019% 99.390% ( 170, 250 ] 259 0.040% 99.430% ( 250, 380 ] 423 0.066% 99.496% ( 380, 580 ] 722 0.113% 99.609% ( 580, 870 ] 893 0.139% 99.748% ( 870, 1300 ] 786 0.123% 99.871% ( 1300, 1900 ] 400 0.062% 99.933% ( 1900, 2900 ] 291 0.045% 99.978% ( 2900, 4400 ] 298 0.046% 100.025% ( 4400, 6600 ] 83 0.013% 100.038% ( 6600, 9900 ] 18 0.003% 100.041% ( 9900, 14000 ] 2 0.000% 100.041% ( 14000, 22000 ] 4 0.001% 100.041% ** Level 1 read latency histogram (micros): Count: 187887 Average: 12.7359 StdDev: 152.85 Min: 1 Median: 1.9992 Max: 9489 Percentiles: P50: 2.00 P75: 2.99 P99: 13.21 P99.9: 2310.11 P99.99: 4170.87 ------------------------------------------------------ [ 0, 1 ] 2802 1.491% 1.491% ( 1, 2 ] 91213 48.547% 50.038% ########## ( 2, 3 ] 47166 25.103% 75.141% ##### ( 3, 4 ] 19164 10.200% 85.341% ## ( 4, 6 ] 19944 10.615% 95.956% ## ( 6, 10 ] 5176 2.755% 98.711% # ( 10, 15 ] 849 0.452% 99.163% ( 15, 22 ] 107 0.057% 99.220% ( 22, 34 ] 10 0.005% 99.225% ( 34, 51 ] 15 0.008% 99.233% ( 51, 76 ] 46 0.024% 99.258% ( 76, 110 ] 12 0.006% 99.264% ( 110, 170 ] 33 0.018% 99.281% ( 170, 250 ] 53 0.028% 99.310% ( 250, 380 ] 114 0.061% 99.370% ( 380, 580 ] 228 0.121% 99.492% ( 580, 870 ] 310 0.165% 99.657% ( 870, 1300 ] 247 0.131% 99.788% ( 1300, 1900 ] 167 0.089% 99.877% ( 1900, 2900 ] 110 0.059% 99.936% ( 2900, 4400 ] 123 0.065% 100.001% ( 4400, 6600 ] 33 0.018% 100.019% ( 6600, 9900 ] 6 0.003% 100.022% ** Level 2 read latency histogram (micros): Count: 418058 Average: 13.1091 StdDev: 182.79 Min: 1 Median: 2.0170 Max: 25494 Percentiles: P50: 2.02 P75: 3.06 P99: 11.99 P99.9: 1252.71 P99.99: 1840.98 ------------------------------------------------------ [ 0, 1 ] 10348 2.475% 2.475% ( 1, 2 ] 196921 47.104% 49.579% ######### ( 2, 3 ] 103665 24.797% 74.376% ##### ( 3, 4 ] 44023 10.530% 84.906% ## ( 4, 6 ] 44596 10.667% 95.574% ## ( 6, 10 ] 13247 3.169% 98.742% # ( 10, 15 ] 2712 0.649% 99.391% ( 15, 22 ] 455 0.109% 99.500% ( 22, 34 ] 92 0.022% 99.522% ( 34, 51 ] 43 0.010% 99.532% ( 51, 76 ] 36 0.009% 99.541% ( 76, 110 ] 18 0.004% 99.545% ( 110, 170 ] 50 0.012% 99.557% ( 170, 250 ] 101 0.024% 99.581% ( 250, 380 ] 190 0.045% 99.627% ( 380, 580 ] 341 0.082% 99.708% ( 580, 870 ] 413 0.099% 99.807% ( 870, 1300 ] 437 0.105% 99.911% ( 1300, 1900 ] 364 0.087% 99.999% ( 1900, 2900 ] 305 0.073% 100.072% ( 2900, 4400 ] 302 0.072% 100.144% ( 4400, 6600 ] 91 0.022% 100.166% ( 6600, 9900 ] 28 0.007% 100.172% ( 9900, 14000 ] 8 0.002% 100.174% ( 14000, 22000 ] 4 0.001% 100.175% ( 22000, 33000 ] 1 0.000% 100.175% ** Level 3 read latency histogram (micros): Count: 1503876 Average: 11.7734 StdDev: 162.86 Min: 1 Median: 1.9824 Max: 24262 Percentiles: P50: 1.98 P75: 2.96 P99: 9.87 P99.9: 1189.67 P99.99: 1847.33 ------------------------------------------------------ [ 0, 1 ] 23824 1.584% 1.584% ( 1, 2 ] 741174 49.284% 50.868% ########## ( 2, 3 ] 377847 25.125% 75.993% ##### ( 3, 4 ] 153331 10.196% 86.189% ## ( 4, 6 ] 155513 10.341% 96.530% ## ( 6, 10 ] 38369 2.551% 99.081% # ( 10, 15 ] 6492 0.432% 99.513% ( 15, 22 ] 842 0.056% 99.569% ( 22, 34 ] 87 0.006% 99.575% ( 34, 51 ] 112 0.007% 99.582% ( 51, 76 ] 151 0.010% 99.592% ( 76, 110 ] 62 0.004% 99.596% ( 110, 170 ] 137 0.009% 99.605% ( 170, 250 ] 269 0.018% 99.623% ( 250, 380 ] 499 0.033% 99.656% ( 380, 580 ] 993 0.066% 99.722% ( 580, 870 ] 1501 0.100% 99.822% ( 870, 1300 ] 1574 0.105% 99.927% ( 1300, 1900 ] 1041 0.069% 99.996% ( 1900, 2900 ] 915 0.061% 100.057% ( 2900, 4400 ] 1016 0.068% 100.125% ( 4400, 6600 ] 321 0.021% 100.146% ( 6600, 9900 ] 70 0.005% 100.151% ( 9900, 14000 ] 5 0.000% 100.151% ( 14000, 22000 ] 7 0.000% 100.151% ( 22000, 33000 ] 2 0.000% 100.151% ** Level 4 read latency histogram (micros): Count: 909542 Average: 19.3890 StdDev: 202.75 Min: 1 Median: 2.0046 Max: 25985 Percentiles: P50: 2.00 P75: 3.01 P99: 301.50 P99.9: 2082.38 P99.99: 2922.51 ------------------------------------------------------ [ 0, 1 ] 13702 1.506% 1.506% ( 1, 2 ] 440019 48.378% 49.885% ########## ( 2, 3 ] 227680 25.032% 74.917% ##### ( 3, 4 ] 94795 10.422% 85.339% ## ( 4, 6 ] 95656 10.517% 95.856% ## ( 6, 10 ] 23156 2.546% 98.402% # ( 10, 15 ] 3740 0.411% 98.813% ( 15, 22 ] 500 0.055% 98.868% ( 22, 34 ] 44 0.005% 98.873% ( 34, 51 ] 67 0.007% 98.880% ( 51, 76 ] 108 0.012% 98.892% ( 76, 110 ] 47 0.005% 98.897% ( 110, 170 ] 178 0.020% 98.917% ( 170, 250 ] 442 0.049% 98.966% ( 250, 380 ] 789 0.087% 99.052% ( 380, 580 ] 1615 0.178% 99.230% ( 580, 870 ] 2298 0.253% 99.483% ( 870, 1300 ] 2245 0.247% 99.729% ( 1300, 1900 ] 1372 0.151% 99.880% ( 1900, 2900 ] 984 0.108% 99.988% ( 2900, 4400 ] 936 0.103% 100.091% ( 4400, 6600 ] 303 0.033% 100.125% ( 6600, 9900 ] 45 0.005% 100.130% ( 9900, 14000 ] 2 0.000% 100.130% ( 14000, 22000 ] 4 0.000% 100.130% ( 22000, 33000 ] 1 0.000% 100.130% ** Level 5 read latency histogram (micros): Count: 29958 Average: 1363.8149 StdDev: 1378.22 Min: 80 Median: 882.1720 Max: 28742 Percentiles: P50: 882.17 P75: 1667.25 P99: 6414.73 P99.9: 9761.78 P99.99: 26409.24 ------------------------------------------------------ ( 76, 110 ] 64 0.214% 0.214% ( 110, 170 ] 482 1.609% 1.823% ( 170, 250 ] 1259 4.203% 6.025% # ( 250, 380 ] 2437 8.135% 14.160% ## ( 380, 580 ] 4454 14.867% 29.027% ### ( 580, 870 ] 6124 20.442% 49.469% #### ( 870, 1300 ] 5617 18.750% 68.219% #### ( 1300, 1900 ] 3319 11.079% 79.298% ## ( 1900, 2900 ] 2347 7.834% 87.132% ## ( 2900, 4400 ] 2603 8.689% 95.821% ## ( 4400, 6600 ] 1040 3.472% 99.292% # ( 6600, 9900 ] 190 0.634% 99.927% ( 9900, 14000 ] 7 0.023% 99.950% ( 14000, 22000 ] 10 0.033% 99.983% ( 22000, 33000 ] 5 0.017% 100.000% ** DB Stats ** Uptime(secs): 14.2 total, 6.6 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 138.99 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.039 H:M:S, 42.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 148.63 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.681 H:M:S, 40.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 9 num-running-flushes: 1 2024/12/16-08:37:44 ... thread 0: (1000000,3000000) ops and (219529.3,161817.9) ops/second in (4.555200,18.539359) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/10 2.15 GB 50.4 1.9 0.0 1.9 4.7 2.8 0.0 1.7 68.4 167.6 28.54 15.91 42 0.680 1982K 2404 0.0 0.0 L1 4/4 234.09 MB 0.0 1.5 1.2 0.3 1.5 1.2 0.0 1.3 120.0 119.6 12.78 7.23 2 6.391 1557K 4357 0.0 0.0 L3 22/3 1.13 GB 0.9 4.9 2.7 2.2 4.9 2.6 0.0 1.8 121.0 120.0 41.71 23.91 27 1.545 3770K 4982 0.0 0.0 L4 190/14 10.23 GB 1.0 9.4 5.1 4.3 9.3 4.9 0.4 1.8 119.6 117.6 80.68 44.79 56 1.441 3774K 1720 0.0 0.0 L5 1144/0 94.00 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1374/31 107.74 GB 0.0 17.8 8.9 8.8 20.3 11.5 5.1 7.3 111.0 127.1 163.72 91.85 127 1.289 11M 13K 0.0 0.0 Int 0/0 0.00 KB 0.0 17.8 8.9 8.8 20.3 11.5 5.1 7.3 111.0 127.1 163.72 91.85 127 1.289 11M 13K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 17.8 8.9 8.8 17.5 8.7 0.0 0.0 124.4 123.0 146.07 83.80 91 1.605 11M 13K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 160.6 17.65 8.05 36 0.490 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 18.7 total, 18.5 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 20.32 GB write, 1111.55 MB/s write, 17.75 GB read, 971.21 MB/s read, 163.7 seconds Interval compaction: 20.32 GB write, 1122.08 MB/s write, 17.75 GB read, 980.41 MB/s read, 163.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 68, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 68, interval: 68 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 976359 Average: 9.1784 StdDev: 120.81 Min: 1 Median: 1.8387 Max: 16963 Percentiles: P50: 1.84 P75: 2.75 P99: 9.85 P99.9: 1276.56 P99.99: 2906.00 ------------------------------------------------------ [ 0, 1 ] 39785 4.075% 4.075% # ( 1, 2 ] 534656 54.760% 58.835% ########### ( 2, 3 ] 209476 21.455% 80.290% #### ( 3, 4 ] 85831 8.791% 89.081% ## ( 4, 6 ] 80195 8.214% 97.294% ## ( 6, 10 ] 17319 1.774% 99.068% ( 10, 15 ] 2854 0.292% 99.361% ( 15, 22 ] 340 0.035% 99.395% ( 22, 34 ] 23 0.002% 99.398% ( 34, 51 ] 70 0.007% 99.405% ( 51, 76 ] 47 0.005% 99.410% ( 76, 110 ] 65 0.007% 99.416% ( 110, 170 ] 257 0.026% 99.443% ( 170, 250 ] 476 0.049% 99.491% ( 250, 380 ] 703 0.072% 99.563% ( 380, 580 ] 1120 0.115% 99.678% ( 580, 870 ] 1241 0.127% 99.805% ( 870, 1300 ] 979 0.100% 99.906% ( 1300, 1900 ] 471 0.048% 99.954% ( 1900, 2900 ] 353 0.036% 99.990% ( 2900, 4400 ] 341 0.035% 100.025% ( 4400, 6600 ] 90 0.009% 100.034% ( 6600, 9900 ] 18 0.002% 100.036% ( 9900, 14000 ] 2 0.000% 100.036% ( 14000, 22000 ] 4 0.000% 100.037% ** Level 1 read latency histogram (micros): Count: 355913 Average: 9.6909 StdDev: 125.24 Min: 1 Median: 1.9787 Max: 9489 Percentiles: P50: 1.98 P75: 2.94 P99: 10.00 P99.9: 1312.38 P99.99: 2777.84 ------------------------------------------------------ [ 0, 1 ] 4560 1.281% 1.281% ( 1, 2 ] 177177 49.781% 51.062% ########## ( 2, 3 ] 90482 25.423% 76.485% ##### ( 3, 4 ] 35428 9.954% 86.439% ## ( 4, 6 ] 35790 10.056% 96.495% ## ( 6, 10 ] 8928 2.508% 99.003% # ( 10, 15 ] 1310 0.368% 99.371% ( 15, 22 ] 159 0.045% 99.416% ( 22, 34 ] 15 0.004% 99.420% ( 34, 51 ] 48 0.013% 99.434% ( 51, 76 ] 72 0.020% 99.454% ( 76, 110 ] 24 0.007% 99.461% ( 110, 170 ] 105 0.030% 99.490% ( 170, 250 ] 139 0.039% 99.529% ( 250, 380 ] 228 0.064% 99.593% ( 380, 580 ] 353 0.099% 99.692% ( 580, 870 ] 408 0.115% 99.807% ( 870, 1300 ] 328 0.092% 99.899% ( 1300, 1900 ] 198 0.056% 99.955% ( 1900, 2900 ] 144 0.040% 99.995% ( 2900, 4400 ] 153 0.043% 100.038% ( 4400, 6600 ] 38 0.011% 100.049% ( 6600, 9900 ] 10 0.003% 100.052% ** Level 2 read latency histogram (micros): Count: 418058 Average: 13.1091 StdDev: 182.79 Min: 1 Median: 2.0170 Max: 25494 Percentiles: P50: 2.02 P75: 3.06 P99: 11.99 P99.9: 1252.71 P99.99: 1840.98 ------------------------------------------------------ [ 0, 1 ] 10348 2.475% 2.475% ( 1, 2 ] 196921 47.104% 49.579% ######### ( 2, 3 ] 103665 24.797% 74.376% ##### ( 3, 4 ] 44023 10.530% 84.906% ## ( 4, 6 ] 44596 10.667% 95.574% ## ( 6, 10 ] 13247 3.169% 98.742% # ( 10, 15 ] 2712 0.649% 99.391% ( 15, 22 ] 455 0.109% 99.500% ( 22, 34 ] 92 0.022% 99.522% ( 34, 51 ] 43 0.010% 99.532% ( 51, 76 ] 36 0.009% 99.541% ( 76, 110 ] 18 0.004% 99.545% ( 110, 170 ] 50 0.012% 99.557% ( 170, 250 ] 101 0.024% 99.581% ( 250, 380 ] 190 0.045% 99.627% ( 380, 580 ] 341 0.082% 99.708% ( 580, 870 ] 413 0.099% 99.807% ( 870, 1300 ] 437 0.105% 99.911% ( 1300, 1900 ] 364 0.087% 99.999% ( 1900, 2900 ] 305 0.073% 100.072% ( 2900, 4400 ] 302 0.072% 100.144% ( 4400, 6600 ] 91 0.022% 100.166% ( 6600, 9900 ] 28 0.007% 100.172% ( 9900, 14000 ] 8 0.002% 100.174% ( 14000, 22000 ] 4 0.001% 100.175% ( 22000, 33000 ] 1 0.000% 100.175% ** Level 3 read latency histogram (micros): Count: 1843696 Average: 10.6007 StdDev: 150.09 Min: 0 Median: 1.9709 Max: 24262 Percentiles: P50: 1.97 P75: 2.94 P99: 9.74 P99.9: 1085.83 P99.99: 1740.53 ------------------------------------------------------ [ 0, 1 ] 28750 1.559% 1.559% ( 1, 2 ] 919856 49.892% 51.451% ########## ( 2, 3 ] 462189 25.069% 76.520% ##### ( 3, 4 ] 185839 10.080% 86.600% ## ( 4, 6 ] 187317 10.160% 96.759% ## ( 6, 10 ] 44215 2.398% 99.158% ( 10, 15 ] 7321 0.397% 99.555% ( 15, 22 ] 932 0.051% 99.605% ( 22, 34 ] 96 0.005% 99.611% ( 34, 51 ] 151 0.008% 99.619% ( 51, 76 ] 187 0.010% 99.629% ( 76, 110 ] 77 0.004% 99.633% ( 110, 170 ] 187 0.010% 99.643% ( 170, 250 ] 360 0.020% 99.663% ( 250, 380 ] 638 0.035% 99.697% ( 380, 580 ] 1182 0.064% 99.761% ( 580, 870 ] 1698 0.092% 99.854% ( 870, 1300 ] 1710 0.093% 99.946% ( 1300, 1900 ] 1100 0.060% 100.006% ( 1900, 2900 ] 976 0.053% 100.059% ( 2900, 4400 ] 1058 0.057% 100.116% ( 4400, 6600 ] 323 0.018% 100.134% ( 6600, 9900 ] 72 0.004% 100.138% ( 9900, 14000 ] 6 0.000% 100.138% ( 14000, 22000 ] 7 0.000% 100.138% ( 22000, 33000 ] 2 0.000% 100.138% ** Level 4 read latency histogram (micros): Count: 1313771 Average: 17.3632 StdDev: 183.53 Min: 1 Median: 1.9791 Max: 25985 Percentiles: P50: 1.98 P75: 2.97 P99: 242.46 P99.9: 1818.35 P99.99: 2687.74 ------------------------------------------------------ [ 0, 1 ] 19759 1.504% 1.504% ( 1, 2 ] 650708 49.530% 51.034% ########## ( 2, 3 ] 326099 24.822% 75.855% ##### ( 3, 4 ] 132602 10.093% 85.949% ## ( 4, 6 ] 131008 9.972% 95.921% ## ( 6, 10 ] 32789 2.496% 98.416% ( 10, 15 ] 4967 0.378% 98.794% ( 15, 22 ] 627 0.048% 98.842% ( 22, 34 ] 64 0.005% 98.847% ( 34, 51 ] 155 0.012% 98.859% ( 51, 76 ] 173 0.013% 98.872% ( 76, 110 ] 153 0.012% 98.884% ( 110, 170 ] 592 0.045% 98.929% ( 170, 250 ] 1037 0.079% 99.008% ( 250, 380 ] 1622 0.123% 99.131% ( 380, 580 ] 2648 0.202% 99.333% ( 580, 870 ] 3230 0.246% 99.578% ( 870, 1300 ] 2831 0.215% 99.794% ( 1300, 1900 ] 1615 0.123% 99.917% ( 1900, 2900 ] 1222 0.093% 100.010% ( 2900, 4400 ] 1129 0.086% 100.096% ( 4400, 6600 ] 334 0.025% 100.121% ( 6600, 9900 ] 50 0.004% 100.125% ( 9900, 14000 ] 2 0.000% 100.125% ( 14000, 22000 ] 4 0.000% 100.126% ( 22000, 33000 ] 1 0.000% 100.126% ** Level 5 read latency histogram (micros): Count: 48876 Average: 1114.1794 StdDev: 1222.75 Min: 80 Median: 717.5276 Max: 28742 Percentiles: P50: 717.53 P75: 1268.21 P99: 6069.52 P99.9: 9489.55 P99.99: 21289.92 ------------------------------------------------------ ( 76, 110 ] 324 0.663% 0.663% ( 110, 170 ] 1983 4.057% 4.720% # ( 170, 250 ] 3663 7.494% 12.215% # ( 250, 380 ] 5382 11.012% 23.226% ## ( 380, 580 ] 8448 17.285% 40.511% ### ( 580, 870 ] 9780 20.010% 60.521% #### ( 870, 1300 ] 7642 15.635% 76.156% ### ( 1300, 1900 ] 4103 8.395% 84.551% ## ( 1900, 2900 ] 3061 6.263% 90.813% # ( 2900, 4400 ] 3168 6.482% 97.295% # ( 4400, 6600 ] 1098 2.247% 99.542% ( 6600, 9900 ] 200 0.409% 99.951% ( 9900, 14000 ] 10 0.020% 99.971% ( 14000, 22000 ] 10 0.020% 99.992% ( 22000, 33000 ] 5 0.010% 100.002% ** DB Stats ** Uptime(secs): 18.7 total, 4.6 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 157.75 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.598 H:M:S, 35.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 216.06 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.559 H:M:S, 12.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 5 num-running-flushes: 1 2024/12/16-08:37:48 ... thread 0: (1000000,4000000) ops and (278552.1,180755.4) ops/second in (3.589993,22.129352) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/10 3.13 GB 151.1 2.9 0.0 2.9 6.6 3.7 0.0 1.8 82.3 189.1 35.96 22.79 62 0.580 3006K 3379 0.0 0.0 L1 4/4 234.09 MB 0.0 1.5 1.2 0.3 1.5 1.2 0.0 1.3 120.0 119.6 12.78 7.23 2 6.391 1557K 4357 0.0 0.0 L3 19/0 939.66 MB 0.9 4.9 2.7 2.2 4.9 2.6 0.0 1.8 121.0 120.0 41.71 23.91 27 1.545 3770K 4982 0.0 0.0 L4 183/0 9.76 GB 1.0 10.4 5.3 5.1 10.2 5.1 0.4 1.9 123.2 121.0 86.49 49.34 59 1.466 4536K 7064 0.0 0.0 L5 1152/0 94.65 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1376/14 108.69 GB 0.0 19.7 9.2 10.6 23.2 12.7 5.7 6.2 114.1 134.5 176.94 103.27 150 1.180 12M 19K 0.0 0.0 Int 0/0 0.00 KB 0.0 19.7 9.2 10.6 23.2 12.7 5.7 6.2 114.1 134.5 176.94 103.27 150 1.180 12M 19K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 19.7 9.2 10.6 19.5 8.9 0.0 0.0 129.6 128.0 155.87 92.26 98 1.591 12M 19K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 182.4 21.06 11.02 52 0.405 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 22.3 total, 22.1 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 23.24 GB write, 1066.78 MB/s write, 19.72 GB read, 905.26 MB/s read, 176.9 seconds Interval compaction: 23.24 GB write, 1075.24 MB/s write, 19.72 GB read, 912.45 MB/s read, 176.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 68, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 68, interval: 68 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1275886 Average: 10.1524 StdDev: 109.58 Min: 1 Median: 1.7558 Max: 16963 Percentiles: P50: 1.76 P75: 2.62 P99: 165.75 P99.9: 1082.78 P99.99: 2285.75 ------------------------------------------------------ [ 0, 1 ] 105554 8.273% 8.273% ## ( 1, 2 ] 704396 55.208% 63.481% ########### ( 2, 3 ] 238602 18.701% 82.182% #### ( 3, 4 ] 96439 7.559% 89.741% ## ( 4, 6 ] 86617 6.789% 96.530% # ( 6, 10 ] 18574 1.456% 97.985% ( 10, 15 ] 3166 0.248% 98.234% ( 15, 22 ] 396 0.031% 98.265% ( 22, 34 ] 40 0.003% 98.268% ( 34, 51 ] 74 0.006% 98.274% ( 51, 76 ] 53 0.004% 98.278% ( 76, 110 ] 1820 0.143% 98.420% ( 110, 170 ] 7960 0.624% 99.044% ( 170, 250 ] 4292 0.336% 99.381% ( 250, 380 ] 2170 0.170% 99.551% ( 380, 580 ] 2299 0.180% 99.731% ( 580, 870 ] 1643 0.129% 99.860% ( 870, 1300 ] 1041 0.082% 99.941% ( 1300, 1900 ] 482 0.038% 99.979% ( 1900, 2900 ] 364 0.029% 100.008% ( 2900, 4400 ] 346 0.027% 100.035% ( 4400, 6600 ] 90 0.007% 100.042% ( 6600, 9900 ] 18 0.001% 100.043% ( 9900, 14000 ] 2 0.000% 100.043% ( 14000, 22000 ] 4 0.000% 100.044% ** Level 1 read latency histogram (micros): Count: 369365 Average: 9.6826 StdDev: 123.18 Min: 1 Median: 1.9960 Max: 9489 Percentiles: P50: 2.00 P75: 2.99 P99: 11.57 P99.9: 1287.80 P99.99: 2768.50 ------------------------------------------------------ [ 0, 1 ] 4787 1.296% 1.296% ( 1, 2 ] 180614 48.899% 50.195% ########## ( 2, 3 ] 93009 25.181% 75.375% ##### ( 3, 4 ] 37600 10.180% 85.555% ## ( 4, 6 ] 39487 10.691% 96.245% ## ( 6, 10 ] 9715 2.630% 98.876% # ( 10, 15 ] 1459 0.395% 99.271% ( 15, 22 ] 246 0.067% 99.337% ( 22, 34 ] 29 0.008% 99.345% ( 34, 51 ] 48 0.013% 99.358% ( 51, 76 ] 72 0.019% 99.378% ( 76, 110 ] 60 0.016% 99.394% ( 110, 170 ] 271 0.073% 99.467% ( 170, 250 ] 212 0.057% 99.525% ( 250, 380 ] 272 0.074% 99.598% ( 380, 580 ] 376 0.102% 99.700% ( 580, 870 ] 418 0.113% 99.813% ( 870, 1300 ] 330 0.089% 99.903% ( 1300, 1900 ] 198 0.054% 99.956% ( 1900, 2900 ] 144 0.039% 99.995% ( 2900, 4400 ] 153 0.041% 100.037% ( 4400, 6600 ] 38 0.010% 100.047% ( 6600, 9900 ] 10 0.003% 100.050% ** Level 2 read latency histogram (micros): Count: 418058 Average: 13.1091 StdDev: 182.79 Min: 1 Median: 2.0170 Max: 25494 Percentiles: P50: 2.02 P75: 3.06 P99: 11.99 P99.9: 1252.71 P99.99: 1840.98 ------------------------------------------------------ [ 0, 1 ] 10348 2.475% 2.475% ( 1, 2 ] 196921 47.104% 49.579% ######### ( 2, 3 ] 103665 24.797% 74.376% ##### ( 3, 4 ] 44023 10.530% 84.906% ## ( 4, 6 ] 44596 10.667% 95.574% ## ( 6, 10 ] 13247 3.169% 98.742% # ( 10, 15 ] 2712 0.649% 99.391% ( 15, 22 ] 455 0.109% 99.500% ( 22, 34 ] 92 0.022% 99.522% ( 34, 51 ] 43 0.010% 99.532% ( 51, 76 ] 36 0.009% 99.541% ( 76, 110 ] 18 0.004% 99.545% ( 110, 170 ] 50 0.012% 99.557% ( 170, 250 ] 101 0.024% 99.581% ( 250, 380 ] 190 0.045% 99.627% ( 380, 580 ] 341 0.082% 99.708% ( 580, 870 ] 413 0.099% 99.807% ( 870, 1300 ] 437 0.105% 99.911% ( 1300, 1900 ] 364 0.087% 99.999% ( 1900, 2900 ] 305 0.073% 100.072% ( 2900, 4400 ] 302 0.072% 100.144% ( 4400, 6600 ] 91 0.022% 100.166% ( 6600, 9900 ] 28 0.007% 100.172% ( 9900, 14000 ] 8 0.002% 100.174% ( 14000, 22000 ] 4 0.001% 100.175% ( 22000, 33000 ] 1 0.000% 100.175% ** Level 3 read latency histogram (micros): Count: 1857892 Average: 10.6487 StdDev: 149.65 Min: 0 Median: 1.9693 Max: 24262 Percentiles: P50: 1.97 P75: 2.94 P99: 9.82 P99.9: 1084.28 P99.99: 1741.13 ------------------------------------------------------ [ 0, 1 ] 29303 1.577% 1.577% ( 1, 2 ] 928104 49.955% 51.532% ########## ( 2, 3 ] 465037 25.030% 76.562% ##### ( 3, 4 ] 186647 10.046% 86.608% ## ( 4, 6 ] 187941 10.116% 96.724% ## ( 6, 10 ] 44318 2.385% 99.110% ( 10, 15 ] 7338 0.395% 99.505% ( 15, 22 ] 935 0.050% 99.555% ( 22, 34 ] 96 0.005% 99.560% ( 34, 51 ] 151 0.008% 99.568% ( 51, 76 ] 187 0.010% 99.578% ( 76, 110 ] 176 0.009% 99.588% ( 110, 170 ] 683 0.037% 99.625% ( 170, 250 ] 587 0.032% 99.656% ( 250, 380 ] 718 0.039% 99.695% ( 380, 580 ] 1235 0.066% 99.761% ( 580, 870 ] 1722 0.093% 99.854% ( 870, 1300 ] 1718 0.092% 99.946% ( 1300, 1900 ] 1102 0.059% 100.006% ( 1900, 2900 ] 978 0.053% 100.058% ( 2900, 4400 ] 1058 0.057% 100.115% ( 4400, 6600 ] 323 0.017% 100.133% ( 6600, 9900 ] 72 0.004% 100.137% ( 9900, 14000 ] 6 0.000% 100.137% ( 14000, 22000 ] 7 0.000% 100.137% ( 22000, 33000 ] 2 0.000% 100.137% ** Level 4 read latency histogram (micros): Count: 1335689 Average: 18.2153 StdDev: 182.96 Min: 1 Median: 1.9821 Max: 25985 Percentiles: P50: 1.98 P75: 2.98 P99: 312.02 P99.9: 1811.65 P99.99: 2683.28 ------------------------------------------------------ [ 0, 1 ] 20284 1.519% 1.519% ( 1, 2 ] 659366 49.365% 50.884% ########## ( 2, 3 ] 329277 24.652% 75.536% ##### ( 3, 4 ] 133752 10.014% 85.550% ## ( 4, 6 ] 132109 9.891% 95.440% ## ( 6, 10 ] 33147 2.482% 97.922% ( 10, 15 ] 4997 0.374% 98.296% ( 15, 22 ] 632 0.047% 98.344% ( 22, 34 ] 64 0.005% 98.348% ( 34, 51 ] 155 0.012% 98.360% ( 51, 76 ] 173 0.013% 98.373% ( 76, 110 ] 900 0.067% 98.440% ( 110, 170 ] 3801 0.285% 98.725% ( 170, 250 ] 2586 0.194% 98.918% ( 250, 380 ] 2283 0.171% 99.089% ( 380, 580 ] 3155 0.236% 99.326% ( 580, 870 ] 3420 0.256% 99.582% ( 870, 1300 ] 2870 0.215% 99.797% ( 1300, 1900 ] 1621 0.121% 99.918% ( 1900, 2900 ] 1230 0.092% 100.010% ( 2900, 4400 ] 1130 0.085% 100.095% ( 4400, 6600 ] 334 0.025% 100.120% ( 6600, 9900 ] 50 0.004% 100.123% ( 9900, 14000 ] 2 0.000% 100.123% ( 14000, 22000 ] 4 0.000% 100.124% ( 22000, 33000 ] 1 0.000% 100.124% ** Level 5 read latency histogram (micros): Count: 90460 Average: 697.9419 StdDev: 1011.50 Min: 76 Median: 331.3380 Max: 28742 Percentiles: P50: 331.34 P75: 787.51 P99: 5232.31 P99.9: 8770.41 P99.99: 16363.20 ------------------------------------------------------ ( 51, 76 ] 1 0.001% 0.001% ( 76, 110 ] 4862 5.375% 5.376% # ( 110, 170 ] 21707 23.996% 29.372% ##### ( 170, 250 ] 12885 14.244% 43.616% ### ( 250, 380 ] 9230 10.203% 53.819% ## ( 380, 580 ] 11425 12.630% 66.449% ### ( 580, 870 ] 10810 11.950% 78.399% ## ( 870, 1300 ] 7817 8.641% 87.041% ## ( 1300, 1900 ] 4131 4.567% 91.607% # ( 1900, 2900 ] 3099 3.426% 95.033% # ( 2900, 4400 ] 3173 3.508% 98.541% # ( 4400, 6600 ] 1098 1.214% 99.755% ( 6600, 9900 ] 200 0.221% 99.976% ( 9900, 14000 ] 10 0.011% 99.987% ( 14000, 22000 ] 10 0.011% 99.998% ( 22000, 33000 ] 5 0.006% 100.003% ** DB Stats ** Uptime(secs): 22.3 total, 3.6 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 176.48 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.598 H:M:S, 29.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 274.15 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/16-08:37:53 ... thread 0: (1000000,5000000) ops and (201024.3,184475.5) ops/second in (4.974522,27.103874) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/6 4.12 GB 276.9 4.1 0.0 4.1 8.9 4.7 0.0 1.9 95.2 204.5 44.33 30.61 83 0.534 4285K 4603 0.0 0.0 L1 4/4 234.09 MB 0.0 1.5 1.2 0.3 1.5 1.2 0.0 1.3 120.0 119.6 12.78 7.23 2 6.391 1557K 4357 0.0 0.0 L3 19/0 939.66 MB 0.9 4.9 2.7 2.2 4.9 2.6 0.0 1.8 121.0 120.0 41.71 23.91 27 1.545 3770K 4982 0.0 0.0 L4 183/0 9.76 GB 1.0 10.4 5.3 5.1 10.2 5.1 0.4 1.9 123.2 121.0 86.49 49.34 59 1.466 4536K 7064 0.0 0.0 L5 1152/0 94.65 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1377/10 109.68 GB 0.0 21.0 9.2 11.8 25.5 13.7 5.7 5.4 115.8 140.6 185.31 111.09 171 1.084 14M 21K 0.0 0.0 Int 0/0 0.00 KB 0.0 21.0 9.2 11.8 25.5 13.7 5.7 5.4 115.8 140.6 185.31 111.09 171 1.084 14M 21K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 21.0 9.2 11.8 20.7 8.9 0.0 0.0 133.4 131.9 160.88 97.16 103 1.562 14M 21K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 198.5 24.43 13.93 68 0.359 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 27.3 total, 27.1 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 25.45 GB write, 955.34 MB/s write, 20.95 GB read, 786.38 MB/s read, 185.3 seconds Interval compaction: 25.45 GB write, 961.53 MB/s write, 20.95 GB read, 791.48 MB/s read, 185.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 6, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 6, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 68, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 6, total-stops: 68, interval: 74 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1587196 Average: 11.9792 StdDev: 103.17 Min: 1 Median: 1.7006 Max: 16963 Percentiles: P50: 1.70 P75: 2.53 P99: 215.72 P99.9: 926.85 P99.99: 1851.04 ------------------------------------------------------ [ 0, 1 ] 186339 11.740% 11.740% ## ( 1, 2 ] 866813 54.613% 66.353% ########### ( 2, 3 ] 259516 16.351% 82.704% ### ( 3, 4 ] 105356 6.638% 89.341% # ( 4, 6 ] 93640 5.900% 95.241% # ( 6, 10 ] 19788 1.247% 96.488% ( 10, 15 ] 3472 0.219% 96.707% ( 15, 22 ] 446 0.028% 96.735% ( 22, 34 ] 47 0.003% 96.738% ( 34, 51 ] 75 0.005% 96.742% ( 51, 76 ] 66 0.004% 96.747% ( 76, 110 ] 5839 0.368% 97.114% ( 110, 170 ] 24152 1.522% 98.636% ( 170, 250 ] 10106 0.637% 99.273% ( 250, 380 ] 4032 0.254% 99.527% ( 380, 580 ] 3762 0.237% 99.764% ( 580, 870 ] 2012 0.127% 99.891% ( 870, 1300 ] 1118 0.070% 99.961% ( 1300, 1900 ] 499 0.031% 99.993% ( 1900, 2900 ] 379 0.024% 100.016% ( 2900, 4400 ] 363 0.023% 100.039% ( 4400, 6600 ] 90 0.006% 100.045% ( 6600, 9900 ] 18 0.001% 100.046% ( 9900, 14000 ] 2 0.000% 100.046% ( 14000, 22000 ] 4 0.000% 100.046% ** Level 1 read latency histogram (micros): Count: 370368 Average: 9.9117 StdDev: 123.29 Min: 1 Median: 1.9985 Max: 9489 Percentiles: P50: 2.00 P75: 2.99 P99: 13.30 P99.9: 1287.87 P99.99: 2768.71 ------------------------------------------------------ [ 0, 1 ] 4789 1.293% 1.293% ( 1, 2 ] 180664 48.780% 50.073% ########## ( 2, 3 ] 93079 25.131% 75.204% ##### ( 3, 4 ] 37681 10.174% 85.378% ## ( 4, 6 ] 39724 10.726% 96.104% ## ( 6, 10 ] 9757 2.634% 98.738% # ( 10, 15 ] 1471 0.397% 99.135% ( 15, 22 ] 253 0.068% 99.203% ( 22, 34 ] 29 0.008% 99.211% ( 34, 51 ] 48 0.013% 99.224% ( 51, 76 ] 72 0.019% 99.244% ( 76, 110 ] 129 0.035% 99.279% ( 110, 170 ] 560 0.151% 99.430% ( 170, 250 ] 300 0.081% 99.511% ( 250, 380 ] 293 0.079% 99.590% ( 380, 580 ] 400 0.108% 99.698% ( 580, 870 ] 426 0.115% 99.813% ( 870, 1300 ] 332 0.090% 99.903% ( 1300, 1900 ] 198 0.053% 99.956% ( 1900, 2900 ] 145 0.039% 99.995% ( 2900, 4400 ] 153 0.041% 100.036% ( 4400, 6600 ] 38 0.010% 100.047% ( 6600, 9900 ] 10 0.003% 100.049% ** Level 2 read latency histogram (micros): Count: 418058 Average: 13.1091 StdDev: 182.79 Min: 1 Median: 2.0170 Max: 25494 Percentiles: P50: 2.02 P75: 3.06 P99: 11.99 P99.9: 1252.71 P99.99: 1840.98 ------------------------------------------------------ [ 0, 1 ] 10348 2.475% 2.475% ( 1, 2 ] 196921 47.104% 49.579% ######### ( 2, 3 ] 103665 24.797% 74.376% ##### ( 3, 4 ] 44023 10.530% 84.906% ## ( 4, 6 ] 44596 10.667% 95.574% ## ( 6, 10 ] 13247 3.169% 98.742% # ( 10, 15 ] 2712 0.649% 99.391% ( 15, 22 ] 455 0.109% 99.500% ( 22, 34 ] 92 0.022% 99.522% ( 34, 51 ] 43 0.010% 99.532% ( 51, 76 ] 36 0.009% 99.541% ( 76, 110 ] 18 0.004% 99.545% ( 110, 170 ] 50 0.012% 99.557% ( 170, 250 ] 101 0.024% 99.581% ( 250, 380 ] 190 0.045% 99.627% ( 380, 580 ] 341 0.082% 99.708% ( 580, 870 ] 413 0.099% 99.807% ( 870, 1300 ] 437 0.105% 99.911% ( 1300, 1900 ] 364 0.087% 99.999% ( 1900, 2900 ] 305 0.073% 100.072% ( 2900, 4400 ] 302 0.072% 100.144% ( 4400, 6600 ] 91 0.022% 100.166% ( 6600, 9900 ] 28 0.007% 100.172% ( 9900, 14000 ] 8 0.002% 100.174% ( 14000, 22000 ] 4 0.001% 100.175% ( 22000, 33000 ] 1 0.000% 100.175% ** Level 3 read latency histogram (micros): Count: 1859456 Average: 10.7900 StdDev: 149.70 Min: 0 Median: 1.9702 Max: 24262 Percentiles: P50: 1.97 P75: 2.94 P99: 9.96 P99.9: 1084.26 P99.99: 1741.59 ------------------------------------------------------ [ 0, 1 ] 29303 1.576% 1.576% ( 1, 2 ] 928104 49.913% 51.489% ########## ( 2, 3 ] 465037 25.009% 76.498% ##### ( 3, 4 ] 186647 10.038% 86.536% ## ( 4, 6 ] 187941 10.107% 96.643% ## ( 6, 10 ] 44318 2.383% 99.026% ( 10, 15 ] 7338 0.395% 99.421% ( 15, 22 ] 935 0.050% 99.471% ( 22, 34 ] 96 0.005% 99.476% ( 34, 51 ] 151 0.008% 99.484% ( 51, 76 ] 187 0.010% 99.495% ( 76, 110 ] 365 0.020% 99.514% ( 110, 170 ] 1571 0.084% 99.599% ( 170, 250 ] 891 0.048% 99.647% ( 250, 380 ] 811 0.044% 99.690% ( 380, 580 ] 1304 0.070% 99.760% ( 580, 870 ] 1741 0.094% 99.854% ( 870, 1300 ] 1719 0.092% 99.946% ( 1300, 1900 ] 1102 0.059% 100.006% ( 1900, 2900 ] 979 0.053% 100.058% ( 2900, 4400 ] 1058 0.057% 100.115% ( 4400, 6600 ] 323 0.017% 100.133% ( 6600, 9900 ] 72 0.004% 100.136% ( 9900, 14000 ] 6 0.000% 100.137% ( 14000, 22000 ] 7 0.000% 100.137% ( 22000, 33000 ] 2 0.000% 100.137% ** Level 4 read latency histogram (micros): Count: 1347251 Average: 19.5514 StdDev: 183.09 Min: 1 Median: 1.9908 Max: 25985 Percentiles: P50: 1.99 P75: 3.01 P99: 352.87 P99.9: 1808.93 P99.99: 2683.50 ------------------------------------------------------ [ 0, 1 ] 20284 1.506% 1.506% ( 1, 2 ] 659378 48.942% 50.448% ########## ( 2, 3 ] 329311 24.443% 74.891% ##### ( 3, 4 ] 133879 9.937% 84.828% ## ( 4, 6 ] 132386 9.826% 94.655% ## ( 6, 10 ] 33175 2.462% 97.117% ( 10, 15 ] 5008 0.372% 97.489% ( 15, 22 ] 632 0.047% 97.536% ( 22, 34 ] 64 0.005% 97.541% ( 34, 51 ] 155 0.012% 97.552% ( 51, 76 ] 173 0.013% 97.565% ( 76, 110 ] 2342 0.174% 97.739% ( 110, 170 ] 9955 0.739% 98.478% ( 170, 250 ] 4676 0.347% 98.825% ( 250, 380 ] 2983 0.221% 99.046% ( 380, 580 ] 3682 0.273% 99.320% ( 580, 870 ] 3550 0.263% 99.583% ( 870, 1300 ] 2889 0.214% 99.797% ( 1300, 1900 ] 1629 0.121% 99.918% ( 1900, 2900 ] 1232 0.091% 100.010% ( 2900, 4400 ] 1131 0.084% 100.094% ( 4400, 6600 ] 334 0.025% 100.119% ( 6600, 9900 ] 50 0.004% 100.122% ( 9900, 14000 ] 2 0.000% 100.122% ( 14000, 22000 ] 4 0.000% 100.123% ( 22000, 33000 ] 1 0.000% 100.123% ** Level 5 read latency histogram (micros): Count: 154303 Average: 484.1446 StdDev: 819.20 Min: 76 Median: 190.6574 Max: 28742 Percentiles: P50: 190.66 P75: 480.61 P99: 4294.36 P99.9: 7700.50 P99.99: 12183.58 ------------------------------------------------------ ( 51, 76 ] 1 0.001% 0.001% ( 76, 110 ] 13779 8.930% 8.930% ## ( 110, 170 ] 56993 36.936% 45.866% ####### ( 170, 250 ] 24702 16.009% 61.875% ### ( 250, 380 ] 12997 8.423% 70.298% ## ( 380, 580 ] 14423 9.347% 79.645% ## ( 580, 870 ] 11675 7.566% 87.212% ## ( 870, 1300 ] 7940 5.146% 92.357% # ( 1300, 1900 ] 4178 2.708% 95.065% # ( 1900, 2900 ] 3115 2.019% 97.084% ( 2900, 4400 ] 3181 2.062% 99.145% ( 4400, 6600 ] 1098 0.712% 99.857% ( 6600, 9900 ] 200 0.130% 99.986% ( 9900, 14000 ] 10 0.006% 99.993% ( 14000, 22000 ] 10 0.006% 99.999% ( 22000, 33000 ] 5 0.003% 100.003% ** DB Stats ** Uptime(secs): 27.3 total, 5.0 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 180.38 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.880 H:M:S, 28.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 197.85 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.282 H:M:S, 25.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/16-08:38:03 ... thread 0: (1000000,6000000) ops and (100711.9,162016.8) ops/second in (9.929318,37.033192) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/10 5.03 GB 345.1 6.4 0.0 6.4 12.0 5.6 0.0 2.1 116.5 219.2 56.25 41.99 102 0.551 6650K 15K 0.0 0.0 L1 4/4 234.09 MB 0.0 1.5 1.2 0.3 1.5 1.2 0.0 1.3 120.0 119.6 12.78 7.23 2 6.391 1557K 4357 0.0 0.0 L3 19/0 939.66 MB 0.9 4.9 2.7 2.2 4.9 2.6 0.0 1.8 121.0 120.0 41.71 23.91 27 1.545 3770K 4982 0.0 0.0 L4 183/0 9.76 GB 1.0 10.4 5.3 5.1 10.2 5.1 0.4 1.9 123.2 121.0 86.49 49.34 59 1.466 4536K 7064 0.0 0.0 L5 1152/0 94.65 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1377/14 110.59 GB 0.0 23.2 9.2 14.1 28.6 14.6 5.7 5.1 120.6 148.7 197.23 122.48 190 1.038 16M 32K 0.0 0.0 Int 0/0 0.00 KB 0.0 23.2 9.2 14.1 28.6 14.6 5.7 5.1 120.6 148.7 197.23 122.48 190 1.038 16M 32K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 23.2 9.2 14.1 23.0 8.9 0.0 0.0 140.2 138.7 169.68 105.82 107 1.586 16M 32K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 210.4 27.55 16.65 83 0.332 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 37.2 total, 37.0 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 28.64 GB write, 788.12 MB/s write, 23.23 GB read, 639.15 MB/s read, 197.2 seconds Interval compaction: 28.64 GB write, 791.85 MB/s write, 23.23 GB read, 642.18 MB/s read, 197.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 14, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 14, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 68, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 14, total-stops: 68, interval: 82 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2310688 Average: 13.9141 StdDev: 92.32 Min: 0 Median: 1.6151 Max: 16963 Percentiles: P50: 1.62 P75: 2.33 P99: 231.56 P99.9: 769.60 P99.99: 1281.67 ------------------------------------------------------ [ 0, 1 ] 393196 17.016% 17.016% ### ( 1, 2 ] 1239137 53.626% 70.643% ########### ( 2, 3 ] 301346 13.041% 83.684% ### ( 3, 4 ] 123407 5.341% 89.025% # ( 4, 6 ] 106540 4.611% 93.636% # ( 6, 10 ] 21789 0.943% 94.579% ( 10, 15 ] 3905 0.169% 94.748% ( 15, 22 ] 505 0.022% 94.769% ( 22, 34 ] 57 0.002% 94.772% ( 34, 51 ] 77 0.003% 94.775% ( 51, 76 ] 71 0.003% 94.778% ( 76, 110 ] 17021 0.737% 95.515% ( 110, 170 ] 63614 2.753% 98.268% # ( 170, 250 ] 21986 0.951% 99.219% ( 250, 380 ] 7586 0.328% 99.548% ( 380, 580 ] 6380 0.276% 99.824% ( 580, 870 ] 2694 0.117% 99.940% ( 870, 1300 ] 1198 0.052% 99.992% ( 1300, 1900 ] 507 0.022% 100.014% ( 1900, 2900 ] 404 0.017% 100.032% ( 2900, 4400 ] 379 0.016% 100.048% ( 4400, 6600 ] 91 0.004% 100.052% ( 6600, 9900 ] 18 0.001% 100.053% ( 9900, 14000 ] 2 0.000% 100.053% ( 14000, 22000 ] 4 0.000% 100.053% ** Level 1 read latency histogram (micros): Count: 371390 Average: 10.3622 StdDev: 123.55 Min: 1 Median: 2.0026 Max: 9489 Percentiles: P50: 2.00 P75: 3.00 P99: 89.44 P99.9: 1286.66 P99.99: 2768.01 ------------------------------------------------------ [ 0, 1 ] 4789 1.289% 1.289% ( 1, 2 ] 180664 48.645% 49.935% ########## ( 2, 3 ] 93079 25.062% 74.997% ##### ( 3, 4 ] 37681 10.146% 85.143% ## ( 4, 6 ] 39724 10.696% 95.839% ## ( 6, 10 ] 9757 2.627% 98.466% # ( 10, 15 ] 1471 0.396% 98.862% ( 15, 22 ] 253 0.068% 98.931% ( 22, 34 ] 29 0.008% 98.938% ( 34, 51 ] 48 0.013% 98.951% ( 51, 76 ] 72 0.019% 98.971% ( 76, 110 ] 276 0.074% 99.045% ( 110, 170 ] 1143 0.308% 99.353% ( 170, 250 ] 482 0.130% 99.482% ( 250, 380 ] 349 0.094% 99.576% ( 380, 580 ] 442 0.119% 99.695% ( 580, 870 ] 435 0.117% 99.813% ( 870, 1300 ] 335 0.090% 99.903% ( 1300, 1900 ] 198 0.053% 99.956% ( 1900, 2900 ] 145 0.039% 99.995% ( 2900, 4400 ] 153 0.041% 100.036% ( 4400, 6600 ] 38 0.010% 100.047% ( 6600, 9900 ] 10 0.003% 100.049% ** Level 2 read latency histogram (micros): Count: 418058 Average: 13.1091 StdDev: 182.79 Min: 1 Median: 2.0170 Max: 25494 Percentiles: P50: 2.02 P75: 3.06 P99: 11.99 P99.9: 1252.71 P99.99: 1840.98 ------------------------------------------------------ [ 0, 1 ] 10348 2.475% 2.475% ( 1, 2 ] 196921 47.104% 49.579% ######### ( 2, 3 ] 103665 24.797% 74.376% ##### ( 3, 4 ] 44023 10.530% 84.906% ## ( 4, 6 ] 44596 10.667% 95.574% ## ( 6, 10 ] 13247 3.169% 98.742% # ( 10, 15 ] 2712 0.649% 99.391% ( 15, 22 ] 455 0.109% 99.500% ( 22, 34 ] 92 0.022% 99.522% ( 34, 51 ] 43 0.010% 99.532% ( 51, 76 ] 36 0.009% 99.541% ( 76, 110 ] 18 0.004% 99.545% ( 110, 170 ] 50 0.012% 99.557% ( 170, 250 ] 101 0.024% 99.581% ( 250, 380 ] 190 0.045% 99.627% ( 380, 580 ] 341 0.082% 99.708% ( 580, 870 ] 413 0.099% 99.807% ( 870, 1300 ] 437 0.105% 99.911% ( 1300, 1900 ] 364 0.087% 99.999% ( 1900, 2900 ] 305 0.073% 100.072% ( 2900, 4400 ] 302 0.072% 100.144% ( 4400, 6600 ] 91 0.022% 100.166% ( 6600, 9900 ] 28 0.007% 100.172% ( 9900, 14000 ] 8 0.002% 100.174% ( 14000, 22000 ] 4 0.001% 100.175% ( 22000, 33000 ] 1 0.000% 100.175% ** Level 3 read latency histogram (micros): Count: 1862773 Average: 11.0737 StdDev: 149.77 Min: 0 Median: 1.9720 Max: 24262 Percentiles: P50: 1.97 P75: 2.95 P99: 11.90 P99.9: 1083.81 P99.99: 1741.55 ------------------------------------------------------ [ 0, 1 ] 29303 1.573% 1.573% ( 1, 2 ] 928104 49.824% 51.397% ########## ( 2, 3 ] 465037 24.965% 76.362% ##### ( 3, 4 ] 186647 10.020% 86.381% ## ( 4, 6 ] 187941 10.089% 96.471% ## ( 6, 10 ] 44318 2.379% 98.850% ( 10, 15 ] 7338 0.394% 99.244% ( 15, 22 ] 935 0.050% 99.294% ( 22, 34 ] 96 0.005% 99.299% ( 34, 51 ] 151 0.008% 99.307% ( 51, 76 ] 187 0.010% 99.317% ( 76, 110 ] 825 0.044% 99.362% ( 110, 170 ] 3472 0.186% 99.548% ( 170, 250 ] 1532 0.082% 99.630% ( 250, 380 ] 970 0.052% 99.682% ( 380, 580 ] 1431 0.077% 99.759% ( 580, 870 ] 1768 0.095% 99.854% ( 870, 1300 ] 1720 0.092% 99.946% ( 1300, 1900 ] 1103 0.059% 100.006% ( 1900, 2900 ] 979 0.053% 100.058% ( 2900, 4400 ] 1058 0.057% 100.115% ( 4400, 6600 ] 323 0.017% 100.132% ( 6600, 9900 ] 72 0.004% 100.136% ( 9900, 14000 ] 6 0.000% 100.137% ( 14000, 22000 ] 7 0.000% 100.137% ( 22000, 33000 ] 2 0.000% 100.137% ** Level 4 read latency histogram (micros): Count: 1371546 Average: 22.2166 StdDev: 183.06 Min: 1 Median: 2.0186 Max: 25985 Percentiles: P50: 2.02 P75: 3.15 P99: 392.80 P99.9: 1801.51 P99.99: 2682.06 ------------------------------------------------------ [ 0, 1 ] 20284 1.479% 1.479% ( 1, 2 ] 659380 48.076% 49.555% ########## ( 2, 3 ] 329313 24.010% 73.565% ##### ( 3, 4 ] 133960 9.767% 83.332% ## ( 4, 6 ] 132493 9.660% 92.992% ## ( 6, 10 ] 33183 2.419% 95.412% ( 10, 15 ] 5012 0.365% 95.777% ( 15, 22 ] 632 0.046% 95.823% ( 22, 34 ] 64 0.005% 95.828% ( 34, 51 ] 155 0.011% 95.839% ( 51, 76 ] 175 0.013% 95.852% ( 76, 110 ] 5834 0.425% 96.277% ( 110, 170 ] 23873 1.741% 98.018% ( 170, 250 ] 8986 0.655% 98.673% ( 250, 380 ] 4193 0.306% 98.979% ( 380, 580 ] 4585 0.334% 99.313% ( 580, 870 ] 3773 0.275% 99.588% ( 870, 1300 ] 2912 0.212% 99.800% ( 1300, 1900 ] 1636 0.119% 99.920% ( 1900, 2900 ] 1235 0.090% 100.010% ( 2900, 4400 ] 1131 0.082% 100.092% ( 4400, 6600 ] 334 0.024% 100.116% ( 6600, 9900 ] 50 0.004% 100.120% ( 9900, 14000 ] 2 0.000% 100.120% ( 14000, 22000 ] 4 0.000% 100.121% ( 22000, 33000 ] 1 0.000% 100.121% ** Level 5 read latency histogram (micros): Count: 287014 Average: 339.3444 StdDev: 624.59 Min: 76 Median: 159.3728 Max: 28742 Percentiles: P50: 159.37 P75: 250.93 P99: 3667.84 P99.9: 6461.72 P99.99: 9723.43 ------------------------------------------------------ ( 51, 76 ] 2 0.001% 0.001% ( 76, 110 ] 33574 11.698% 11.698% ## ( 110, 170 ] 133593 46.546% 58.244% ######### ( 170, 250 ] 47952 16.707% 74.951% ### ( 250, 380 ] 19552 6.812% 81.764% # ( 380, 580 ] 19533 6.806% 88.569% # ( 580, 870 ] 12908 4.497% 93.067% # ( 870, 1300 ] 8063 2.809% 95.876% # ( 1300, 1900 ] 4196 1.462% 97.338% ( 1900, 2900 ] 3142 1.095% 98.432% ( 2900, 4400 ] 3184 1.109% 99.542% ( 4400, 6600 ] 1098 0.383% 99.924% ( 6600, 9900 ] 200 0.070% 99.994% ( 9900, 14000 ] 10 0.003% 99.998% ( 14000, 22000 ] 10 0.003% 100.001% ( 22000, 33000 ] 5 0.002% 100.003% ** DB Stats ** Uptime(secs): 37.2 total, 9.9 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 158.69 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:13.883 H:M:S, 37.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 99.12 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:6.002 H:M:S, 60.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 298.589 micros/op 13380 ops/sec 60.164 seconds 804996 operations; 13.0 MB/s (200999 of 200999 found) Microseconds per read: Count: 804996 Average: 298.5938 StdDev: 860.68 Min: 2 Median: 199.4604 Max: 102601 Percentiles: P50: 199.46 P75: 249.83 P99: 1929.15 P99.9: 14058.03 P99.99: 31707.52 ------------------------------------------------------ ( 1, 2 ] 1 0.000% 0.000% ( 2, 3 ] 1 0.000% 0.000% ( 3, 4 ] 7 0.001% 0.001% ( 4, 6 ] 48 0.006% 0.007% ( 6, 10 ] 211 0.026% 0.033% ( 10, 15 ] 150 0.019% 0.052% ( 15, 22 ] 490 0.061% 0.113% ( 22, 34 ] 8406 1.044% 1.157% ( 34, 51 ] 3088 0.384% 1.541% ( 51, 76 ] 416 0.052% 1.592% ( 76, 110 ] 444 0.055% 1.647% ( 110, 170 ] 271522 33.730% 35.377% ####### ( 170, 250 ] 319653 39.709% 75.086% ######## ( 250, 380 ] 105157 13.063% 88.149% ### ( 380, 580 ] 48656 6.044% 94.193% # ( 580, 870 ] 22588 2.806% 96.999% # ( 870, 1300 ] 11045 1.372% 98.371% ( 1300, 1900 ] 4972 0.618% 98.989% ( 1900, 2900 ] 3123 0.388% 99.377% ( 2900, 4400 ] 2659 0.330% 99.707% ( 4400, 6600 ] 859 0.107% 99.814% ( 6600, 9900 ] 325 0.040% 99.854% ( 9900, 14000 ] 366 0.045% 99.900% ( 14000, 22000 ] 552 0.069% 99.968% ( 22000, 33000 ] 200 0.025% 99.993% ( 33000, 50000 ] 40 0.005% 99.998% ( 50000, 75000 ] 12 0.001% 99.999% ( 75000, 110000 ] 5 0.001% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 14069010 rocksdb.bloom.filter.full.positive COUNT : 941635 rocksdb.bloom.filter.full.true.positive COUNT : 804584 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 412 rocksdb.memtable.miss COUNT : 804584 rocksdb.l0.hit COUNT : 37444 rocksdb.l1.hit COUNT : 1822 rocksdb.l2andup.hit COUNT : 765318 rocksdb.compaction.key.drop.new COUNT : 271681 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 6315426 rocksdb.number.keys.read COUNT : 804996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 6517519632 rocksdb.bytes.read COUNT : 804996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1745 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 34902077 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 28275262291 rocksdb.compact.write.bytes COUNT : 28000047092 rocksdb.flush.write.bytes COUNT : 6473697248 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 18 rocksdb.number.superversion_acquires COUNT : 1308 rocksdb.number.superversion_releases COUNT : 1254 rocksdb.number.superversion_cleanups COUNT : 324 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 259 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 6421413888 rocksdb.memtable.garbage.bytes.at.flush COUNT : 2420736 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 32350275210 rocksdb.non.last.level.read.count COUNT : 7675587 rocksdb.block.checksum.compute.count COUNT : 7801701 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 198.099359 P95 : 661.598558 P99 : 1927.603465 P100 : 102598.000000 COUNT : 804996 SUM : 239323556 rocksdb.db.write.micros P50 : 1.554044 P95 : 4.674681 P99 : 9.621976 P100 : 899496.000000 COUNT : 6315426 SUM : 51204944 rocksdb.compaction.times.micros P50 : 1396000.000000 P95 : 3133333.333333 P99 : 6390000.000000 P100 : 7058999.000000 COUNT : 110 SUM : 175605177 rocksdb.compaction.times.cpu_micros P50 : 907441.860465 P95 : 1825000.000000 P99 : 4253333.333333 P100 : 5802934.000000 COUNT : 110 SUM : 111683742 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 858.552632 P95 : 42350.000000 P99 : 264300.000000 P100 : 308840.000000 COUNT : 89 SUM : 1029050 rocksdb.compaction.outfile.sync.micros P50 : 26125.000000 P95 : 216823.529412 P99 : 357087.500000 P100 : 1130515.000000 COUNT : 341 SUM : 20774733 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 270.526316 P95 : 3617.857143 P99 : 5098.000000 P100 : 5098.000000 COUNT : 226 SUM : 193261 rocksdb.table.open.io.micros P50 : 8985.695876 P95 : 20234.042553 P99 : 25954.761905 P100 : 72845.000000 COUNT : 1745 SUM : 15869720 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.911621 P95 : 9.904467 P99 : 179.040159 P100 : 25497.000000 COUNT : 6848482 SUM : 99465499 rocksdb.read.block.get.micros P50 : 146.084803 P95 : 534.893819 P99 : 1806.122056 P100 : 28746.000000 COUNT : 951501 SUM : 201133175 rocksdb.write.raw.block.micros P50 : 0.535218 P95 : 1.362771 P99 : 3.874467 P100 : 43820.000000 COUNT : 8349446 SUM : 25232871 rocksdb.numfiles.in.singlecompaction P50 : 2.775510 P95 : 5.333333 P99 : 8.600000 P100 : 10.000000 COUNT : 140 SUM : 467 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.572520 P95 : 1220.672651 P99 : 2598.205546 P100 : 899318.000000 COUNT : 168400 SUM : 34922656 rocksdb.sst.read.micros P50 : 1.955710 P95 : 160.999104 P99 : 500.355793 P100 : 28742.000000 COUNT : 7675641 SUM : 281916019 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 804996 SUM : 804996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 6315426 SUM : 6517519632 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 228360.655738 P95 : 897400.000000 P99 : 1139480.000000 P100 : 1164485.000000 COUNT : 89 SUM : 28790773 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 374 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0