[2024-07-15 15:53:49.050963] Starting SPDK v24.09-pre git sha1 a95bbf233 / DPDK 24.03.0 initialization... [2024-07-15 15:53:49.051245] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2612602 ] Set seed to 1721051629044438 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-15 15:53:49.164553] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-15 15:53:49.377257] app.c: 603:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-15 15:53:49.377308] app.c: 604:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2612602' to capture a snapshot of events at runtime. [2024-07-15 15:53:49.377318] app.c: 609:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-15 15:53:49.377328] app.c: 610:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-15 15:53:49.377335] app.c: 611:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2612602 for offline analysis/debug. [2024-07-15 15:53:49.377430] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Jul 15 15:53:55 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/07/15-15:53:58 ... thread 0: (1000000,1000000) ops and (345954.6,345954.6) ops/second in (2.890553,2.890553) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/6 628.96 MB 25.1 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 360.2 2.44 2.20 12 0.204 0 0 0.0 0.0 L2 3/2 250.42 MB 0.1 0.4 0.2 0.2 0.4 0.2 0.1 2.0 258.5 257.8 1.46 1.41 2 0.729 382K 1031 0.0 0.0 L3 10/0 853.84 MB 0.9 0.0 0.0 0.0 0.0 0.0 0.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.54 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 108/0 8.86 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 246/8 20.09 GB 0.0 0.4 0.2 0.2 1.2 1.0 0.2 1.4 96.6 321.9 3.90 3.61 14 0.279 382K 1031 0.0 0.0 Int 0/0 0.00 KB 0.0 0.4 0.2 0.2 1.2 1.0 0.1 1.4 96.6 321.9 3.90 3.61 14 0.279 382K 1031 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 0.0 0.0 258.5 257.8 1.46 1.41 2 0.729 382K 1031 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 360.2 2.44 2.20 12 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.0 total, 2.9 interval Flush(GB): cumulative 0.860, interval 0.860 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.23 GB write, 418.41 MB/s write, 0.37 GB read, 125.57 MB/s read, 3.9 seconds Interval compaction: 1.23 GB write, 434.34 MB/s write, 0.37 GB read, 130.35 MB/s read, 3.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 3, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 3, interval: 3 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 112938 Average: 2.0077 StdDev: 21.10 Min: 0 Median: 1.3088 Max: 3509 Percentiles: P50: 1.31 P75: 1.71 P99: 4.09 P99.9: 9.39 P99.99: 427.08 ------------------------------------------------------ [ 0, 1 ] 34976 30.969% 30.969% ###### ( 1, 2 ] 69611 61.636% 92.606% ############ ( 2, 3 ] 5433 4.811% 97.416% # ( 3, 4 ] 1752 1.551% 98.968% ( 4, 6 ] 818 0.724% 99.692% ( 6, 10 ] 277 0.245% 99.937% ( 10, 15 ] 29 0.026% 99.963% ( 15, 22 ] 13 0.012% 99.974% ( 22, 34 ] 2 0.002% 99.976% ( 34, 51 ] 3 0.003% 99.979% ( 110, 170 ] 8 0.007% 99.986% ( 170, 250 ] 2 0.002% 99.988% ( 250, 380 ] 2 0.002% 99.989% ( 380, 580 ] 3 0.003% 99.992% ( 580, 870 ] 3 0.003% 99.995% ( 870, 1300 ] 1 0.001% 99.996% ( 1900, 2900 ] 1 0.001% 99.996% ( 2900, 4400 ] 4 0.004% 100.000% ** Level 2 read latency histogram (micros): Count: 60904 Average: 2.1432 StdDev: 21.84 Min: 0 Median: 1.3510 Max: 3229 Percentiles: P50: 1.35 P75: 1.75 P99: 5.26 P99.9: 11.83 P99.99: 711.88 ------------------------------------------------------ [ 0, 1 ] 17034 27.969% 27.969% ###### ( 1, 2 ] 38232 62.774% 90.743% ############# ( 2, 3 ] 3417 5.610% 96.353% # ( 3, 4 ] 1177 1.933% 98.286% ( 4, 6 ] 691 1.135% 99.420% ( 6, 10 ] 273 0.448% 99.869% ( 10, 15 ] 55 0.090% 99.959% ( 15, 22 ] 6 0.010% 99.969% ( 22, 34 ] 3 0.005% 99.974% ( 34, 51 ] 1 0.002% 99.975% ( 51, 76 ] 1 0.002% 99.977% ( 250, 380 ] 6 0.010% 99.987% ( 380, 580 ] 2 0.003% 99.990% ( 580, 870 ] 2 0.003% 99.993% ( 870, 1300 ] 1 0.002% 99.995% ( 1300, 1900 ] 1 0.002% 99.997% ( 1900, 2900 ] 2 0.003% 100.000% ( 2900, 4400 ] 1 0.002% 100.002% ** Level 3 read latency histogram (micros): Count: 40 Average: 2984.0500 StdDev: 1974.86 Min: 90 Median: 2700.0000 Max: 7880 Percentiles: P50: 2700.00 P75: 4620.00 P99: 7880.00 P99.9: 7880.00 P99.99: 7880.00 ------------------------------------------------------ ( 76, 110 ] 1 2.500% 2.500% # ( 110, 170 ] 3 7.500% 10.000% ## ( 170, 250 ] 2 5.000% 15.000% # ( 870, 1300 ] 2 5.000% 20.000% # ( 1300, 1900 ] 4 10.000% 30.000% ## ( 1900, 2900 ] 10 25.000% 55.000% ##### ( 2900, 4400 ] 7 17.500% 72.500% #### ( 4400, 6600 ] 10 25.000% 97.500% ##### ( 6600, 9900 ] 1 2.500% 100.000% # ** Level 4 read latency histogram (micros): Count: 30 Average: 3144.1333 StdDev: 2001.72 Min: 147 Median: 2757.1429 Max: 7133 Percentiles: P50: 2757.14 P75: 4620.00 P99: 7133.00 P99.9: 7133.00 P99.99: 7133.00 ------------------------------------------------------ ( 110, 170 ] 1 3.333% 3.333% # ( 250, 380 ] 1 3.333% 6.667% # ( 380, 580 ] 1 3.333% 10.000% # ( 580, 870 ] 1 3.333% 13.333% # ( 870, 1300 ] 1 3.333% 16.667% # ( 1300, 1900 ] 4 13.333% 30.000% ### ( 1900, 2900 ] 7 23.333% 53.333% ##### ( 2900, 4400 ] 6 20.000% 73.333% #### ( 4400, 6600 ] 5 16.667% 90.000% ### ( 6600, 9900 ] 3 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 3.0 total, 2.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 327.72 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.371 H:M:S, 12.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 340.12 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.371 H:M:S, 12.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-15:54:01 ... thread 0: (1000000,2000000) ops and (320791.4,332898.1) ops/second in (3.117291,6.007844) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/10 1.23 GB 6.3 0.6 0.0 0.6 2.4 1.8 0.0 1.3 74.2 321.2 7.63 6.88 28 0.272 574K 3052 0.0 0.0 L1 0/0 0.00 KB 0.0 0.4 0.4 0.0 0.4 0.4 0.2 1.0 243.0 241.0 1.55 1.51 1 1.553 383K 3060 0.0 0.0 L2 1/0 59.65 MB 0.6 0.4 0.2 0.2 0.4 0.2 0.4 1.8 248.2 247.5 1.78 1.70 3 0.594 448K 1236 0.0 0.0 L3 17/0 981.01 MB 1.0 0.8 0.4 0.4 0.8 0.4 0.3 1.9 220.5 218.6 3.82 3.49 5 0.763 854K 7107 0.0 0.0 L4 122/0 9.65 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.3 1.0 183.6 183.6 0.63 0.50 1 0.631 117K 0 0.0 0.0 L5 112/0 9.17 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 263/10 21.06 GB 0.0 2.3 1.1 1.1 4.1 3.0 1.5 2.2 152.1 273.6 15.41 14.08 38 0.405 2379K 14K 0.0 0.0 Int 0/0 0.00 KB 0.0 2.3 1.1 1.1 4.1 3.0 1.4 2.2 152.1 273.6 15.41 14.08 38 0.405 2379K 14K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.3 1.1 1.1 2.3 1.1 0.0 0.0 234.2 232.8 10.00 9.32 12 0.834 2379K 14K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 349.1 5.40 4.76 26 0.208 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.1 total, 6.0 interval Flush(GB): cumulative 1.842, interval 1.842 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 4.12 GB write, 688.68 MB/s write, 2.29 GB read, 382.78 MB/s read, 15.4 seconds Interval compaction: 4.12 GB write, 701.30 MB/s write, 2.29 GB read, 389.80 MB/s read, 15.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 10, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 10, interval: 10 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 381742 Average: 2.3389 StdDev: 14.62 Min: 0 Median: 1.5492 Max: 3560 Percentiles: P50: 1.55 P75: 1.95 P99: 5.86 P99.9: 11.60 P99.99: 16.48 ------------------------------------------------------ [ 0, 1 ] 59494 15.585% 15.585% ### ( 1, 2 ] 239210 62.663% 78.248% ############# ( 2, 3 ] 51642 13.528% 91.776% ### ( 3, 4 ] 15571 4.079% 95.855% # ( 4, 6 ] 12889 3.376% 99.231% # ( 6, 10 ] 2399 0.628% 99.859% ( 10, 15 ] 488 0.128% 99.987% ( 15, 22 ] 56 0.015% 100.002% ( 22, 34 ] 6 0.002% 100.003% ( 34, 51 ] 5 0.001% 100.005% ( 51, 76 ] 1 0.000% 100.005% ( 76, 110 ] 1 0.000% 100.005% ( 110, 170 ] 14 0.004% 100.009% ( 170, 250 ] 10 0.003% 100.012% ( 250, 380 ] 7 0.002% 100.013% ( 380, 580 ] 6 0.002% 100.015% ( 580, 870 ] 9 0.002% 100.017% ( 870, 1300 ] 5 0.001% 100.019% ( 1300, 1900 ] 2 0.001% 100.019% ( 1900, 2900 ] 2 0.001% 100.020% ( 2900, 4400 ] 5 0.001% 100.021% ** Level 1 read latency histogram (micros): Count: 62376 Average: 3.0055 StdDev: 12.18 Min: 1 Median: 1.8010 Max: 966 Percentiles: P50: 1.80 P75: 2.61 P99: 8.74 P99.9: 84.80 P99.99: 499.06 ------------------------------------------------------ [ 0, 1 ] 2786 4.466% 4.466% # ( 1, 2 ] 35457 56.844% 61.310% ########### ( 2, 3 ] 13923 22.321% 83.632% #### ( 3, 4 ] 4395 7.046% 90.678% # ( 4, 6 ] 4485 7.190% 97.868% # ( 6, 10 ] 1032 1.654% 99.522% ( 10, 15 ] 189 0.303% 99.825% ( 15, 22 ] 39 0.063% 99.888% ( 22, 34 ] 2 0.003% 99.891% ( 34, 51 ] 1 0.002% 99.893% ( 51, 76 ] 1 0.002% 99.894% ( 76, 110 ] 14 0.022% 99.917% ( 110, 170 ] 25 0.040% 99.957% ( 170, 250 ] 9 0.014% 99.971% ( 250, 380 ] 7 0.011% 99.982% ( 380, 580 ] 8 0.013% 99.995% ( 580, 870 ] 4 0.006% 100.002% ( 870, 1300 ] 5 0.008% 100.010% ** Level 2 read latency histogram (micros): Count: 137684 Average: 2.9732 StdDev: 33.48 Min: 0 Median: 1.5696 Max: 4302 Percentiles: P50: 1.57 P75: 1.98 P99: 8.08 P99.9: 132.90 P99.99: 1546.32 ------------------------------------------------------ [ 0, 1 ] 20895 15.176% 15.176% ### ( 1, 2 ] 84183 61.142% 76.318% ############ ( 2, 3 ] 18451 13.401% 89.719% ### ( 3, 4 ] 6369 4.626% 94.345% # ( 4, 6 ] 5592 4.061% 98.406% # ( 6, 10 ] 1572 1.142% 99.548% ( 10, 15 ] 383 0.278% 99.826% ( 15, 22 ] 57 0.041% 99.868% ( 22, 34 ] 9 0.007% 99.874% ( 34, 51 ] 5 0.004% 99.878% ( 51, 76 ] 1 0.001% 99.879% ( 76, 110 ] 11 0.008% 99.887% ( 110, 170 ] 48 0.035% 99.922% ( 170, 250 ] 29 0.021% 99.943% ( 250, 380 ] 32 0.023% 99.966% ( 380, 580 ] 18 0.013% 99.979% ( 580, 870 ] 10 0.007% 99.986% ( 870, 1300 ] 4 0.003% 99.989% ( 1300, 1900 ] 3 0.002% 99.991% ( 1900, 2900 ] 10 0.007% 99.999% ( 2900, 4400 ] 5 0.004% 100.002% ** Level 3 read latency histogram (micros): Count: 91561 Average: 4.1130 StdDev: 78.64 Min: 0 Median: 1.7303 Max: 7880 Percentiles: P50: 1.73 P75: 2.44 P99: 7.75 P99.9: 16.36 P99.99: 2784.39 ------------------------------------------------------ [ 0, 1 ] 5426 5.926% 5.926% # ( 1, 2 ] 55258 60.351% 66.277% ############ ( 2, 3 ] 18092 19.760% 86.037% #### ( 3, 4 ] 5677 6.200% 92.237% # ( 4, 6 ] 5713 6.240% 98.476% # ( 6, 10 ] 1093 1.194% 99.670% ( 10, 15 ] 205 0.224% 99.894% ( 15, 22 ] 28 0.031% 99.925% ( 22, 34 ] 7 0.008% 99.932% ( 34, 51 ] 1 0.001% 99.933% ( 51, 76 ] 4 0.004% 99.938% ( 76, 110 ] 4 0.004% 99.942% ( 110, 170 ] 8 0.009% 99.951% ( 170, 250 ] 6 0.007% 99.957% ( 250, 380 ] 6 0.007% 99.964% ( 380, 580 ] 2 0.002% 99.966% ( 580, 870 ] 5 0.005% 99.972% ( 870, 1300 ] 3 0.003% 99.975% ( 1300, 1900 ] 5 0.005% 99.980% ( 1900, 2900 ] 10 0.011% 99.991% ( 2900, 4400 ] 10 0.011% 100.002% ( 4400, 6600 ] 10 0.011% 100.013% ( 6600, 9900 ] 1 0.001% 100.014% ** Level 4 read latency histogram (micros): Count: 60 Average: 1587.8667 StdDev: 2105.00 Min: 1 Median: 170.0000 Max: 7133 Percentiles: P50: 170.00 P75: 2757.14 P99: 7133.00 P99.9: 7133.00 P99.99: 7133.00 ------------------------------------------------------ [ 0, 1 ] 8 13.333% 13.333% ### ( 1, 2 ] 7 11.667% 25.000% ## ( 2, 3 ] 2 3.333% 28.333% # ( 6, 10 ] 2 3.333% 31.667% # ( 10, 15 ] 2 3.333% 35.000% # ( 15, 22 ] 2 3.333% 38.333% # ( 22, 34 ] 4 6.667% 45.000% # ( 51, 76 ] 2 3.333% 48.333% # ( 110, 170 ] 1 1.667% 50.000% ( 250, 380 ] 1 1.667% 51.667% ( 380, 580 ] 1 1.667% 53.333% ( 580, 870 ] 2 3.333% 56.667% # ( 870, 1300 ] 1 1.667% 58.333% ( 1300, 1900 ] 4 6.667% 65.000% # ( 1900, 2900 ] 7 11.667% 76.667% ## ( 2900, 4400 ] 6 10.000% 86.667% ## ( 4400, 6600 ] 5 8.333% 95.000% ## ( 6600, 9900 ] 3 5.000% 100.000% # ** Level 5 read latency histogram (micros): Count: 20 Average: 623.9000 StdDev: 730.57 Min: 130 Median: 160.0000 Max: 2473 Percentiles: P50: 160.00 P75: 870.00 P99: 2473.00 P99.9: 2473.00 P99.99: 2473.00 ------------------------------------------------------ ( 110, 170 ] 12 60.000% 60.000% ############ ( 380, 580 ] 2 10.000% 70.000% ## ( 580, 870 ] 1 5.000% 75.000% # ( 1300, 1900 ] 3 15.000% 90.000% ### ( 1900, 2900 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 6.1 total, 3.1 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 321.61 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.899 H:M:S, 14.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 315.71 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.528 H:M:S, 16.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-15:54:04 ... thread 0: (1000000,3000000) ops and (336309.1,334027.4) ops/second in (2.973455,8.981299) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/4 1.28 GB 100.1 1.4 0.0 1.4 4.2 2.8 0.0 1.5 106.3 313.8 13.60 12.49 44 0.309 1468K 8029 0.0 0.0 L1 9/0 862.60 MB 86.3 1.2 1.2 0.0 1.2 1.2 0.2 1.0 235.7 232.4 5.32 5.10 2 2.661 1274K 17K 0.0 0.0 L2 1/0 59.65 MB 0.6 0.4 0.2 0.2 0.4 0.2 0.4 1.8 248.2 247.5 1.78 1.70 3 0.594 448K 1236 0.0 0.0 L3 17/0 981.01 MB 1.0 0.8 0.4 0.4 0.8 0.4 0.3 1.9 220.5 218.6 3.82 3.49 5 0.763 854K 7107 0.0 0.0 L4 122/0 9.65 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.3 1.0 183.6 183.6 0.63 0.50 1 0.631 117K 0 0.0 0.0 L5 112/0 9.17 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 270/4 21.96 GB 0.0 4.0 2.0 2.0 6.7 4.7 1.5 2.4 163.0 274.2 25.15 23.28 55 0.457 4164K 34K 0.0 0.0 Int 0/0 0.00 KB 0.0 4.0 2.0 2.0 6.7 4.7 1.4 2.4 163.0 274.2 25.15 23.28 55 0.457 4164K 34K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.0 2.0 2.0 4.0 2.0 0.0 0.0 240.4 238.4 17.06 16.14 16 1.066 4164K 34K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 349.5 8.10 7.14 39 0.208 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.1 total, 9.0 interval Flush(GB): cumulative 2.764, interval 2.764 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.73 GB write, 758.34 MB/s write, 4.00 GB read, 450.91 MB/s read, 25.2 seconds Interval compaction: 6.73 GB write, 767.64 MB/s write, 4.00 GB read, 456.44 MB/s read, 25.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 14, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 14, interval: 14 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 781012 Average: 2.4047 StdDev: 14.39 Min: 0 Median: 1.5672 Max: 3560 Percentiles: P50: 1.57 P75: 1.95 P99: 5.93 P99.9: 12.25 P99.99: 116.32 ------------------------------------------------------ [ 0, 1 ] 102612 13.138% 13.138% ### ( 1, 2 ] 507527 64.983% 78.122% ############# ( 2, 3 ] 111647 14.295% 92.417% ### ( 3, 4 ] 28514 3.651% 96.068% # ( 4, 6 ] 23751 3.041% 99.109% # ( 6, 10 ] 5753 0.737% 99.845% ( 10, 15 ] 950 0.122% 99.967% ( 15, 22 ] 84 0.011% 99.978% ( 22, 34 ] 8 0.001% 99.979% ( 34, 51 ] 6 0.001% 99.980% ( 51, 76 ] 2 0.000% 99.980% ( 76, 110 ] 69 0.009% 99.989% ( 110, 170 ] 113 0.014% 100.003% ( 170, 250 ] 52 0.007% 100.010% ( 250, 380 ] 36 0.005% 100.014% ( 380, 580 ] 39 0.005% 100.019% ( 580, 870 ] 18 0.002% 100.022% ( 870, 1300 ] 13 0.002% 100.023% ( 1300, 1900 ] 6 0.001% 100.024% ( 1900, 2900 ] 2 0.000% 100.024% ( 2900, 4400 ] 9 0.001% 100.025% ** Level 1 read latency histogram (micros): Count: 62421 Average: 3.2654 StdDev: 22.78 Min: 1 Median: 1.8012 Max: 2798 Percentiles: P50: 1.80 P75: 2.61 P99: 8.83 P99.9: 133.96 P99.99: 902.59 ------------------------------------------------------ [ 0, 1 ] 2794 4.476% 4.476% # ( 1, 2 ] 35466 56.817% 61.293% ########### ( 2, 3 ] 13926 22.310% 83.603% #### ( 3, 4 ] 4396 7.043% 90.646% # ( 4, 6 ] 4485 7.185% 97.831% # ( 6, 10 ] 1033 1.655% 99.486% ( 10, 15 ] 190 0.304% 99.790% ( 15, 22 ] 39 0.062% 99.853% ( 22, 34 ] 2 0.003% 99.856% ( 34, 51 ] 1 0.002% 99.857% ( 51, 76 ] 1 0.002% 99.859% ( 76, 110 ] 14 0.022% 99.881% ( 110, 170 ] 29 0.046% 99.928% ( 170, 250 ] 11 0.018% 99.946% ( 250, 380 ] 11 0.018% 99.963% ( 380, 580 ] 11 0.018% 99.981% ( 580, 870 ] 5 0.008% 99.989% ( 870, 1300 ] 10 0.016% 100.005% ( 1300, 1900 ] 1 0.002% 100.006% ( 1900, 2900 ] 2 0.003% 100.010% ** Level 2 read latency histogram (micros): Count: 137684 Average: 2.9732 StdDev: 33.48 Min: 0 Median: 1.5696 Max: 4302 Percentiles: P50: 1.57 P75: 1.98 P99: 8.08 P99.9: 132.90 P99.99: 1546.32 ------------------------------------------------------ [ 0, 1 ] 20895 15.176% 15.176% ### ( 1, 2 ] 84183 61.142% 76.318% ############ ( 2, 3 ] 18451 13.401% 89.719% ### ( 3, 4 ] 6369 4.626% 94.345% # ( 4, 6 ] 5592 4.061% 98.406% # ( 6, 10 ] 1572 1.142% 99.548% ( 10, 15 ] 383 0.278% 99.826% ( 15, 22 ] 57 0.041% 99.868% ( 22, 34 ] 9 0.007% 99.874% ( 34, 51 ] 5 0.004% 99.878% ( 51, 76 ] 1 0.001% 99.879% ( 76, 110 ] 11 0.008% 99.887% ( 110, 170 ] 48 0.035% 99.922% ( 170, 250 ] 29 0.021% 99.943% ( 250, 380 ] 32 0.023% 99.966% ( 380, 580 ] 18 0.013% 99.979% ( 580, 870 ] 10 0.007% 99.986% ( 870, 1300 ] 4 0.003% 99.989% ( 1300, 1900 ] 3 0.002% 99.991% ( 1900, 2900 ] 10 0.007% 99.999% ( 2900, 4400 ] 5 0.004% 100.002% ** Level 3 read latency histogram (micros): Count: 91561 Average: 4.1130 StdDev: 78.64 Min: 0 Median: 1.7303 Max: 7880 Percentiles: P50: 1.73 P75: 2.44 P99: 7.75 P99.9: 16.36 P99.99: 2784.39 ------------------------------------------------------ [ 0, 1 ] 5426 5.926% 5.926% # ( 1, 2 ] 55258 60.351% 66.277% ############ ( 2, 3 ] 18092 19.760% 86.037% #### ( 3, 4 ] 5677 6.200% 92.237% # ( 4, 6 ] 5713 6.240% 98.476% # ( 6, 10 ] 1093 1.194% 99.670% ( 10, 15 ] 205 0.224% 99.894% ( 15, 22 ] 28 0.031% 99.925% ( 22, 34 ] 7 0.008% 99.932% ( 34, 51 ] 1 0.001% 99.933% ( 51, 76 ] 4 0.004% 99.938% ( 76, 110 ] 4 0.004% 99.942% ( 110, 170 ] 8 0.009% 99.951% ( 170, 250 ] 6 0.007% 99.957% ( 250, 380 ] 6 0.007% 99.964% ( 380, 580 ] 2 0.002% 99.966% ( 580, 870 ] 5 0.005% 99.972% ( 870, 1300 ] 3 0.003% 99.975% ( 1300, 1900 ] 5 0.005% 99.980% ( 1900, 2900 ] 10 0.011% 99.991% ( 2900, 4400 ] 10 0.011% 100.002% ( 4400, 6600 ] 10 0.011% 100.013% ( 6600, 9900 ] 1 0.001% 100.014% ** Level 4 read latency histogram (micros): Count: 60 Average: 1587.8667 StdDev: 2105.00 Min: 1 Median: 170.0000 Max: 7133 Percentiles: P50: 170.00 P75: 2757.14 P99: 7133.00 P99.9: 7133.00 P99.99: 7133.00 ------------------------------------------------------ [ 0, 1 ] 8 13.333% 13.333% ### ( 1, 2 ] 7 11.667% 25.000% ## ( 2, 3 ] 2 3.333% 28.333% # ( 6, 10 ] 2 3.333% 31.667% # ( 10, 15 ] 2 3.333% 35.000% # ( 15, 22 ] 2 3.333% 38.333% # ( 22, 34 ] 4 6.667% 45.000% # ( 51, 76 ] 2 3.333% 48.333% # ( 110, 170 ] 1 1.667% 50.000% ( 250, 380 ] 1 1.667% 51.667% ( 380, 580 ] 1 1.667% 53.333% ( 580, 870 ] 2 3.333% 56.667% # ( 870, 1300 ] 1 1.667% 58.333% ( 1300, 1900 ] 4 6.667% 65.000% # ( 1900, 2900 ] 7 11.667% 76.667% ## ( 2900, 4400 ] 6 10.000% 86.667% ## ( 4400, 6600 ] 5 8.333% 95.000% ## ( 6600, 9900 ] 3 5.000% 100.000% # ** Level 5 read latency histogram (micros): Count: 20 Average: 623.9000 StdDev: 730.57 Min: 130 Median: 160.0000 Max: 2473 Percentiles: P50: 160.00 P75: 870.00 P99: 2473.00 P99.9: 2473.00 P99.99: 2473.00 ------------------------------------------------------ ( 110, 170 ] 12 60.000% 60.000% ############ ( 380, 580 ] 2 10.000% 70.000% ## ( 580, 870 ] 1 5.000% 75.000% # ( 1300, 1900 ] 3 15.000% 90.000% ### ( 1900, 2900 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 9.1 total, 3.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 324.67 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.320 H:M:S, 14.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 330.99 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.421 H:M:S, 14.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/15-15:54:07 ... thread 0: (1000000,4000000) ops and (341236.8,335801.1) ops/second in (2.930516,11.911815) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/4 2.20 GB 194.0 2.3 0.0 2.3 5.9 3.7 0.0 1.6 119.0 311.3 19.55 18.08 60 0.326 2362K 12K 0.0 0.0 L1 9/0 862.60 MB 86.3 1.2 1.2 0.0 1.2 1.2 0.2 1.0 235.7 232.4 5.32 5.10 2 2.661 1274K 17K 0.0 0.0 L2 1/0 59.65 MB 0.6 0.4 0.2 0.2 0.4 0.2 0.4 1.8 248.2 247.5 1.78 1.70 3 0.594 448K 1236 0.0 0.0 L3 17/0 981.01 MB 1.0 0.8 0.4 0.4 0.8 0.4 0.3 1.9 220.5 218.6 3.82 3.49 5 0.763 854K 7107 0.0 0.0 L4 122/0 9.65 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.3 1.0 183.6 183.6 0.63 0.50 1 0.631 117K 0 0.0 0.0 L5 112/0 9.17 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 274/4 22.87 GB 0.0 4.9 2.0 2.9 8.5 5.6 1.5 2.3 160.1 280.2 31.10 28.87 71 0.438 5058K 39K 0.0 0.0 Int 0/0 0.00 KB 0.0 4.9 2.0 2.9 8.5 5.6 1.4 2.3 160.1 280.2 31.10 28.87 71 0.438 5058K 39K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.9 2.0 2.9 4.8 2.0 0.0 0.0 244.3 242.4 20.39 19.41 19 1.073 5058K 39K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 352.0 10.72 9.47 52 0.206 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.0 total, 11.9 interval Flush(GB): cumulative 3.685, interval 3.685 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.51 GB write, 724.77 MB/s write, 4.86 GB read, 414.23 MB/s read, 31.1 seconds Interval compaction: 8.51 GB write, 731.47 MB/s write, 4.86 GB read, 418.06 MB/s read, 31.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 19, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 19, interval: 19 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1022115 Average: 2.3108 StdDev: 13.91 Min: 0 Median: 1.5241 Max: 3560 Percentiles: P50: 1.52 P75: 1.91 P99: 5.81 P99.9: 11.74 P99.99: 106.45 ------------------------------------------------------ [ 0, 1 ] 161748 15.825% 15.825% ### ( 1, 2 ] 666445 65.203% 81.027% ############# ( 2, 3 ] 127095 12.435% 93.462% ## ( 3, 4 ] 32894 3.218% 96.680% # ( 4, 6 ] 26232 2.566% 99.247% # ( 6, 10 ] 6289 0.615% 99.862% ( 10, 15 ] 1122 0.110% 99.972% ( 15, 22 ] 107 0.010% 99.982% ( 22, 34 ] 9 0.001% 99.983% ( 34, 51 ] 7 0.001% 99.984% ( 51, 76 ] 3 0.000% 99.984% ( 76, 110 ] 69 0.007% 99.991% ( 110, 170 ] 121 0.012% 100.003% ( 170, 250 ] 55 0.005% 100.008% ( 250, 380 ] 40 0.004% 100.012% ( 380, 580 ] 43 0.004% 100.016% ( 580, 870 ] 20 0.002% 100.018% ( 870, 1300 ] 19 0.002% 100.020% ( 1300, 1900 ] 9 0.001% 100.021% ( 1900, 2900 ] 2 0.000% 100.021% ( 2900, 4400 ] 11 0.001% 100.022% ** Level 1 read latency histogram (micros): Count: 62421 Average: 3.2654 StdDev: 22.78 Min: 1 Median: 1.8012 Max: 2798 Percentiles: P50: 1.80 P75: 2.61 P99: 8.83 P99.9: 133.96 P99.99: 902.59 ------------------------------------------------------ [ 0, 1 ] 2794 4.476% 4.476% # ( 1, 2 ] 35466 56.817% 61.293% ########### ( 2, 3 ] 13926 22.310% 83.603% #### ( 3, 4 ] 4396 7.043% 90.646% # ( 4, 6 ] 4485 7.185% 97.831% # ( 6, 10 ] 1033 1.655% 99.486% ( 10, 15 ] 190 0.304% 99.790% ( 15, 22 ] 39 0.062% 99.853% ( 22, 34 ] 2 0.003% 99.856% ( 34, 51 ] 1 0.002% 99.857% ( 51, 76 ] 1 0.002% 99.859% ( 76, 110 ] 14 0.022% 99.881% ( 110, 170 ] 29 0.046% 99.928% ( 170, 250 ] 11 0.018% 99.946% ( 250, 380 ] 11 0.018% 99.963% ( 380, 580 ] 11 0.018% 99.981% ( 580, 870 ] 5 0.008% 99.989% ( 870, 1300 ] 10 0.016% 100.005% ( 1300, 1900 ] 1 0.002% 100.006% ( 1900, 2900 ] 2 0.003% 100.010% ** Level 2 read latency histogram (micros): Count: 137684 Average: 2.9732 StdDev: 33.48 Min: 0 Median: 1.5696 Max: 4302 Percentiles: P50: 1.57 P75: 1.98 P99: 8.08 P99.9: 132.90 P99.99: 1546.32 ------------------------------------------------------ [ 0, 1 ] 20895 15.176% 15.176% ### ( 1, 2 ] 84183 61.142% 76.318% ############ ( 2, 3 ] 18451 13.401% 89.719% ### ( 3, 4 ] 6369 4.626% 94.345% # ( 4, 6 ] 5592 4.061% 98.406% # ( 6, 10 ] 1572 1.142% 99.548% ( 10, 15 ] 383 0.278% 99.826% ( 15, 22 ] 57 0.041% 99.868% ( 22, 34 ] 9 0.007% 99.874% ( 34, 51 ] 5 0.004% 99.878% ( 51, 76 ] 1 0.001% 99.879% ( 76, 110 ] 11 0.008% 99.887% ( 110, 170 ] 48 0.035% 99.922% ( 170, 250 ] 29 0.021% 99.943% ( 250, 380 ] 32 0.023% 99.966% ( 380, 580 ] 18 0.013% 99.979% ( 580, 870 ] 10 0.007% 99.986% ( 870, 1300 ] 4 0.003% 99.989% ( 1300, 1900 ] 3 0.002% 99.991% ( 1900, 2900 ] 10 0.007% 99.999% ( 2900, 4400 ] 5 0.004% 100.002% ** Level 3 read latency histogram (micros): Count: 91561 Average: 4.1130 StdDev: 78.64 Min: 0 Median: 1.7303 Max: 7880 Percentiles: P50: 1.73 P75: 2.44 P99: 7.75 P99.9: 16.36 P99.99: 2784.39 ------------------------------------------------------ [ 0, 1 ] 5426 5.926% 5.926% # ( 1, 2 ] 55258 60.351% 66.277% ############ ( 2, 3 ] 18092 19.760% 86.037% #### ( 3, 4 ] 5677 6.200% 92.237% # ( 4, 6 ] 5713 6.240% 98.476% # ( 6, 10 ] 1093 1.194% 99.670% ( 10, 15 ] 205 0.224% 99.894% ( 15, 22 ] 28 0.031% 99.925% ( 22, 34 ] 7 0.008% 99.932% ( 34, 51 ] 1 0.001% 99.933% ( 51, 76 ] 4 0.004% 99.938% ( 76, 110 ] 4 0.004% 99.942% ( 110, 170 ] 8 0.009% 99.951% ( 170, 250 ] 6 0.007% 99.957% ( 250, 380 ] 6 0.007% 99.964% ( 380, 580 ] 2 0.002% 99.966% ( 580, 870 ] 5 0.005% 99.972% ( 870, 1300 ] 3 0.003% 99.975% ( 1300, 1900 ] 5 0.005% 99.980% ( 1900, 2900 ] 10 0.011% 99.991% ( 2900, 4400 ] 10 0.011% 100.002% ( 4400, 6600 ] 10 0.011% 100.013% ( 6600, 9900 ] 1 0.001% 100.014% ** Level 4 read latency histogram (micros): Count: 60 Average: 1587.8667 StdDev: 2105.00 Min: 1 Median: 170.0000 Max: 7133 Percentiles: P50: 170.00 P75: 2757.14 P99: 7133.00 P99.9: 7133.00 P99.99: 7133.00 ------------------------------------------------------ [ 0, 1 ] 8 13.333% 13.333% ### ( 1, 2 ] 7 11.667% 25.000% ## ( 2, 3 ] 2 3.333% 28.333% # ( 6, 10 ] 2 3.333% 31.667% # ( 10, 15 ] 2 3.333% 35.000% # ( 15, 22 ] 2 3.333% 38.333% # ( 22, 34 ] 4 6.667% 45.000% # ( 51, 76 ] 2 3.333% 48.333% # ( 110, 170 ] 1 1.667% 50.000% ( 250, 380 ] 1 1.667% 51.667% ( 380, 580 ] 1 1.667% 53.333% ( 580, 870 ] 2 3.333% 56.667% # ( 870, 1300 ] 1 1.667% 58.333% ( 1300, 1900 ] 4 6.667% 65.000% # ( 1900, 2900 ] 7 11.667% 76.667% ## ( 2900, 4400 ] 6 10.000% 86.667% ## ( 4400, 6600 ] 5 8.333% 95.000% ## ( 6600, 9900 ] 3 5.000% 100.000% # ** Level 5 read latency histogram (micros): Count: 20 Average: 623.9000 StdDev: 730.57 Min: 130 Median: 160.0000 Max: 2473 Percentiles: P50: 160.00 P75: 870.00 P99: 2473.00 P99.9: 2473.00 P99.99: 2473.00 ------------------------------------------------------ ( 110, 170 ] 12 60.000% 60.000% ############ ( 380, 580 ] 2 10.000% 70.000% ## ( 580, 870 ] 1 5.000% 75.000% # ( 1300, 1900 ] 3 15.000% 90.000% ### ( 1900, 2900 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 12.0 total, 2.9 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 327.40 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.722 H:M:S, 14.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 335.84 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.402 H:M:S, 13.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/15-15:54:10 ... thread 0: (1000000,5000000) ops and (346081.5,337808.0) ops/second in (2.889493,14.801308) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/4 3.18 GB 293.9 3.4 0.0 3.4 8.0 4.6 0.0 1.7 130.4 309.9 26.52 24.70 78 0.340 3512K 19K 0.0 0.0 L1 9/0 862.60 MB 86.3 1.2 1.2 0.0 1.2 1.2 0.2 1.0 235.7 232.4 5.32 5.10 2 2.661 1274K 17K 0.0 0.0 L2 1/0 59.65 MB 0.6 0.4 0.2 0.2 0.4 0.2 0.4 1.8 248.2 247.5 1.78 1.70 3 0.594 448K 1236 0.0 0.0 L3 17/0 981.01 MB 1.0 0.8 0.4 0.4 0.8 0.4 0.3 1.9 220.5 218.6 3.82 3.49 5 0.763 854K 7107 0.0 0.0 L4 122/0 9.65 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.3 1.0 183.6 183.6 0.63 0.50 1 0.631 117K 0 0.0 0.0 L5 112/0 9.17 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 276/4 23.85 GB 0.0 6.0 2.0 4.0 10.6 6.6 1.5 2.3 160.6 284.9 38.07 35.49 89 0.428 6207K 45K 0.0 0.0 Int 0/0 0.00 KB 0.0 6.0 2.0 4.0 10.6 6.6 1.4 2.3 160.6 284.9 38.07 35.49 89 0.428 6207K 45K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.0 2.0 4.0 5.9 2.0 0.0 0.0 248.1 246.3 24.64 23.59 23 1.071 6207K 45K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 355.8 13.43 11.90 66 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.9 total, 14.8 interval Flush(GB): cumulative 4.667, interval 4.667 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.59 GB write, 727.31 MB/s write, 5.97 GB read, 409.88 MB/s read, 38.1 seconds Interval compaction: 10.59 GB write, 732.72 MB/s write, 5.97 GB read, 412.93 MB/s read, 38.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 24, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 24, interval: 24 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1277673 Average: 2.2575 StdDev: 13.11 Min: 0 Median: 1.5029 Max: 3560 Percentiles: P50: 1.50 P75: 1.88 P99: 5.72 P99.9: 11.52 P99.99: 93.36 ------------------------------------------------------ [ 0, 1 ] 217690 17.038% 17.038% ### ( 1, 2 ] 837407 65.542% 82.580% ############# ( 2, 3 ] 146727 11.484% 94.064% ## ( 3, 4 ] 38114 2.983% 97.047% # ( 4, 6 ] 29061 2.275% 99.321% ( 6, 10 ] 6978 0.546% 99.867% ( 10, 15 ] 1377 0.108% 99.975% ( 15, 22 ] 132 0.010% 99.985% ( 22, 34 ] 12 0.001% 99.986% ( 34, 51 ] 7 0.001% 99.987% ( 51, 76 ] 6 0.000% 99.987% ( 76, 110 ] 69 0.005% 99.993% ( 110, 170 ] 128 0.010% 100.003% ( 170, 250 ] 60 0.005% 100.007% ( 250, 380 ] 42 0.003% 100.011% ( 380, 580 ] 52 0.004% 100.015% ( 580, 870 ] 25 0.002% 100.017% ( 870, 1300 ] 24 0.002% 100.019% ( 1300, 1900 ] 10 0.001% 100.019% ( 1900, 2900 ] 4 0.000% 100.020% ( 2900, 4400 ] 11 0.001% 100.021% ** Level 1 read latency histogram (micros): Count: 62421 Average: 3.2654 StdDev: 22.78 Min: 1 Median: 1.8012 Max: 2798 Percentiles: P50: 1.80 P75: 2.61 P99: 8.83 P99.9: 133.96 P99.99: 902.59 ------------------------------------------------------ [ 0, 1 ] 2794 4.476% 4.476% # ( 1, 2 ] 35466 56.817% 61.293% ########### ( 2, 3 ] 13926 22.310% 83.603% #### ( 3, 4 ] 4396 7.043% 90.646% # ( 4, 6 ] 4485 7.185% 97.831% # ( 6, 10 ] 1033 1.655% 99.486% ( 10, 15 ] 190 0.304% 99.790% ( 15, 22 ] 39 0.062% 99.853% ( 22, 34 ] 2 0.003% 99.856% ( 34, 51 ] 1 0.002% 99.857% ( 51, 76 ] 1 0.002% 99.859% ( 76, 110 ] 14 0.022% 99.881% ( 110, 170 ] 29 0.046% 99.928% ( 170, 250 ] 11 0.018% 99.946% ( 250, 380 ] 11 0.018% 99.963% ( 380, 580 ] 11 0.018% 99.981% ( 580, 870 ] 5 0.008% 99.989% ( 870, 1300 ] 10 0.016% 100.005% ( 1300, 1900 ] 1 0.002% 100.006% ( 1900, 2900 ] 2 0.003% 100.010% ** Level 2 read latency histogram (micros): Count: 137684 Average: 2.9732 StdDev: 33.48 Min: 0 Median: 1.5696 Max: 4302 Percentiles: P50: 1.57 P75: 1.98 P99: 8.08 P99.9: 132.90 P99.99: 1546.32 ------------------------------------------------------ [ 0, 1 ] 20895 15.176% 15.176% ### ( 1, 2 ] 84183 61.142% 76.318% ############ ( 2, 3 ] 18451 13.401% 89.719% ### ( 3, 4 ] 6369 4.626% 94.345% # ( 4, 6 ] 5592 4.061% 98.406% # ( 6, 10 ] 1572 1.142% 99.548% ( 10, 15 ] 383 0.278% 99.826% ( 15, 22 ] 57 0.041% 99.868% ( 22, 34 ] 9 0.007% 99.874% ( 34, 51 ] 5 0.004% 99.878% ( 51, 76 ] 1 0.001% 99.879% ( 76, 110 ] 11 0.008% 99.887% ( 110, 170 ] 48 0.035% 99.922% ( 170, 250 ] 29 0.021% 99.943% ( 250, 380 ] 32 0.023% 99.966% ( 380, 580 ] 18 0.013% 99.979% ( 580, 870 ] 10 0.007% 99.986% ( 870, 1300 ] 4 0.003% 99.989% ( 1300, 1900 ] 3 0.002% 99.991% ( 1900, 2900 ] 10 0.007% 99.999% ( 2900, 4400 ] 5 0.004% 100.002% ** Level 3 read latency histogram (micros): Count: 91561 Average: 4.1130 StdDev: 78.64 Min: 0 Median: 1.7303 Max: 7880 Percentiles: P50: 1.73 P75: 2.44 P99: 7.75 P99.9: 16.36 P99.99: 2784.39 ------------------------------------------------------ [ 0, 1 ] 5426 5.926% 5.926% # ( 1, 2 ] 55258 60.351% 66.277% ############ ( 2, 3 ] 18092 19.760% 86.037% #### ( 3, 4 ] 5677 6.200% 92.237% # ( 4, 6 ] 5713 6.240% 98.476% # ( 6, 10 ] 1093 1.194% 99.670% ( 10, 15 ] 205 0.224% 99.894% ( 15, 22 ] 28 0.031% 99.925% ( 22, 34 ] 7 0.008% 99.932% ( 34, 51 ] 1 0.001% 99.933% ( 51, 76 ] 4 0.004% 99.938% ( 76, 110 ] 4 0.004% 99.942% ( 110, 170 ] 8 0.009% 99.951% ( 170, 250 ] 6 0.007% 99.957% ( 250, 380 ] 6 0.007% 99.964% ( 380, 580 ] 2 0.002% 99.966% ( 580, 870 ] 5 0.005% 99.972% ( 870, 1300 ] 3 0.003% 99.975% ( 1300, 1900 ] 5 0.005% 99.980% ( 1900, 2900 ] 10 0.011% 99.991% ( 2900, 4400 ] 10 0.011% 100.002% ( 4400, 6600 ] 10 0.011% 100.013% ( 6600, 9900 ] 1 0.001% 100.014% ** Level 4 read latency histogram (micros): Count: 60 Average: 1587.8667 StdDev: 2105.00 Min: 1 Median: 170.0000 Max: 7133 Percentiles: P50: 170.00 P75: 2757.14 P99: 7133.00 P99.9: 7133.00 P99.99: 7133.00 ------------------------------------------------------ [ 0, 1 ] 8 13.333% 13.333% ### ( 1, 2 ] 7 11.667% 25.000% ## ( 2, 3 ] 2 3.333% 28.333% # ( 6, 10 ] 2 3.333% 31.667% # ( 10, 15 ] 2 3.333% 35.000% # ( 15, 22 ] 2 3.333% 38.333% # ( 22, 34 ] 4 6.667% 45.000% # ( 51, 76 ] 2 3.333% 48.333% # ( 110, 170 ] 1 1.667% 50.000% ( 250, 380 ] 1 1.667% 51.667% ( 380, 580 ] 1 1.667% 53.333% ( 580, 870 ] 2 3.333% 56.667% # ( 870, 1300 ] 1 1.667% 58.333% ( 1300, 1900 ] 4 6.667% 65.000% # ( 1900, 2900 ] 7 11.667% 76.667% ## ( 2900, 4400 ] 6 10.000% 86.667% ## ( 4400, 6600 ] 5 8.333% 95.000% ## ( 6600, 9900 ] 3 5.000% 100.000% # ** Level 5 read latency histogram (micros): Count: 20 Average: 623.9000 StdDev: 730.57 Min: 130 Median: 160.0000 Max: 2473 Percentiles: P50: 160.00 P75: 870.00 P99: 2473.00 P99.9: 2473.00 P99.99: 2473.00 ------------------------------------------------------ ( 110, 170 ] 12 60.000% 60.000% ############ ( 380, 580 ] 2 10.000% 70.000% ## ( 580, 870 ] 1 5.000% 75.000% # ( 1300, 1900 ] 3 15.000% 90.000% ### ( 1900, 2900 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 14.9 total, 2.9 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 329.96 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.085 H:M:S, 14.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 340.62 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.363 H:M:S, 12.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/15-15:54:13 ... thread 0: (1000000,6000000) ops and (292925.7,329396.3) ops/second in (3.413835,18.215143) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 21/21 4.22 GB 0.0 4.2 0.0 4.2 9.9 5.7 0.0 1.7 131.6 308.2 32.98 30.70 96 0.344 4406K 23K 0.0 0.0 L1 9/9 862.60 MB 0.0 1.2 1.2 0.0 1.2 1.2 0.2 1.0 235.7 232.4 5.32 5.10 2 2.661 1274K 17K 0.0 0.0 L2 1/0 59.65 MB 0.6 0.4 0.2 0.2 0.4 0.2 0.4 1.8 248.2 247.5 1.78 1.70 3 0.594 448K 1236 0.0 0.0 L3 17/0 981.01 MB 1.0 0.8 0.4 0.4 0.8 0.4 0.3 1.9 220.5 218.6 3.82 3.49 5 0.763 854K 7107 0.0 0.0 L4 122/0 9.65 GB 1.0 0.1 0.1 0.0 0.1 0.1 0.3 1.0 183.6 183.6 0.63 0.50 1 0.631 117K 0 0.0 0.0 L5 112/0 9.17 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 282/30 24.89 GB 0.0 6.8 2.0 4.8 12.5 7.7 1.5 2.2 157.0 287.2 44.53 41.49 107 0.416 7101K 50K 0.0 0.0 Int 0/0 0.00 KB 0.0 6.8 2.0 4.8 12.5 7.7 1.4 2.2 157.0 287.2 44.53 41.49 107 0.416 7101K 50K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.8 2.0 4.8 6.8 2.0 0.0 0.0 249.0 247.2 28.08 26.90 26 1.080 7101K 50K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 355.6 16.45 14.59 81 0.203 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 18.3 total, 18.2 interval Flush(GB): cumulative 5.711, interval 5.711 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.49 GB write, 697.94 MB/s write, 6.83 GB read, 381.57 MB/s read, 44.5 seconds Interval compaction: 12.49 GB write, 702.16 MB/s write, 6.83 GB read, 383.87 MB/s read, 44.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 27, interval: 31 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1539611 Average: 2.2931 StdDev: 14.17 Min: 0 Median: 1.4890 Max: 4079 Percentiles: P50: 1.49 P75: 1.87 P99: 5.70 P99.9: 13.14 P99.99: 160.66 ------------------------------------------------------ [ 0, 1 ] 276522 17.961% 17.961% #### ( 1, 2 ] 1008747 65.520% 83.480% ############# ( 2, 3 ] 167485 10.878% 94.359% ## ( 3, 4 ] 43999 2.858% 97.216% # ( 4, 6 ] 32274 2.096% 99.313% ( 6, 10 ] 8046 0.523% 99.835% ( 10, 15 ] 1594 0.104% 99.939% ( 15, 22 ] 175 0.011% 99.950% ( 22, 34 ] 15 0.001% 99.951% ( 34, 51 ] 9 0.001% 99.952% ( 51, 76 ] 8 0.001% 99.952% ( 76, 110 ] 173 0.011% 99.963% ( 110, 170 ] 488 0.032% 99.995% ( 170, 250 ] 114 0.007% 100.002% ( 250, 380 ] 82 0.005% 100.008% ( 380, 580 ] 82 0.005% 100.013% ( 580, 870 ] 36 0.002% 100.015% ( 870, 1300 ] 34 0.002% 100.018% ( 1300, 1900 ] 11 0.001% 100.018% ( 1900, 2900 ] 6 0.000% 100.019% ( 2900, 4400 ] 14 0.001% 100.020% ** Level 1 read latency histogram (micros): Count: 71384 Average: 3.2663 StdDev: 22.11 Min: 1 Median: 1.7476 Max: 2798 Percentiles: P50: 1.75 P75: 2.52 P99: 8.89 P99.9: 161.62 P99.99: 861.97 ------------------------------------------------------ [ 0, 1 ] 4819 6.751% 6.751% # ( 1, 2 ] 41295 57.849% 64.600% ############ ( 2, 3 ] 14411 20.188% 84.788% #### ( 3, 4 ] 4641 6.501% 91.289% # ( 4, 6 ] 4670 6.542% 97.831% # ( 6, 10 ] 1154 1.617% 99.448% ( 10, 15 ] 197 0.276% 99.724% ( 15, 22 ] 41 0.057% 99.781% ( 22, 34 ] 2 0.003% 99.784% ( 34, 51 ] 1 0.001% 99.786% ( 51, 76 ] 1 0.001% 99.787% ( 76, 110 ] 29 0.041% 99.828% ( 110, 170 ] 60 0.084% 99.912% ( 170, 250 ] 19 0.027% 99.938% ( 250, 380 ] 15 0.021% 99.959% ( 380, 580 ] 17 0.024% 99.983% ( 580, 870 ] 5 0.007% 99.990% ( 870, 1300 ] 10 0.014% 100.004% ( 1300, 1900 ] 1 0.001% 100.006% ( 1900, 2900 ] 2 0.003% 100.008% ** Level 2 read latency histogram (micros): Count: 137684 Average: 2.9732 StdDev: 33.48 Min: 0 Median: 1.5696 Max: 4302 Percentiles: P50: 1.57 P75: 1.98 P99: 8.08 P99.9: 132.90 P99.99: 1546.32 ------------------------------------------------------ [ 0, 1 ] 20895 15.176% 15.176% ### ( 1, 2 ] 84183 61.142% 76.318% ############ ( 2, 3 ] 18451 13.401% 89.719% ### ( 3, 4 ] 6369 4.626% 94.345% # ( 4, 6 ] 5592 4.061% 98.406% # ( 6, 10 ] 1572 1.142% 99.548% ( 10, 15 ] 383 0.278% 99.826% ( 15, 22 ] 57 0.041% 99.868% ( 22, 34 ] 9 0.007% 99.874% ( 34, 51 ] 5 0.004% 99.878% ( 51, 76 ] 1 0.001% 99.879% ( 76, 110 ] 11 0.008% 99.887% ( 110, 170 ] 48 0.035% 99.922% ( 170, 250 ] 29 0.021% 99.943% ( 250, 380 ] 32 0.023% 99.966% ( 380, 580 ] 18 0.013% 99.979% ( 580, 870 ] 10 0.007% 99.986% ( 870, 1300 ] 4 0.003% 99.989% ( 1300, 1900 ] 3 0.002% 99.991% ( 1900, 2900 ] 10 0.007% 99.999% ( 2900, 4400 ] 5 0.004% 100.002% ** Level 3 read latency histogram (micros): Count: 91561 Average: 4.1130 StdDev: 78.64 Min: 0 Median: 1.7303 Max: 7880 Percentiles: P50: 1.73 P75: 2.44 P99: 7.75 P99.9: 16.36 P99.99: 2784.39 ------------------------------------------------------ [ 0, 1 ] 5426 5.926% 5.926% # ( 1, 2 ] 55258 60.351% 66.277% ############ ( 2, 3 ] 18092 19.760% 86.037% #### ( 3, 4 ] 5677 6.200% 92.237% # ( 4, 6 ] 5713 6.240% 98.476% # ( 6, 10 ] 1093 1.194% 99.670% ( 10, 15 ] 205 0.224% 99.894% ( 15, 22 ] 28 0.031% 99.925% ( 22, 34 ] 7 0.008% 99.932% ( 34, 51 ] 1 0.001% 99.933% ( 51, 76 ] 4 0.004% 99.938% ( 76, 110 ] 4 0.004% 99.942% ( 110, 170 ] 8 0.009% 99.951% ( 170, 250 ] 6 0.007% 99.957% ( 250, 380 ] 6 0.007% 99.964% ( 380, 580 ] 2 0.002% 99.966% ( 580, 870 ] 5 0.005% 99.972% ( 870, 1300 ] 3 0.003% 99.975% ( 1300, 1900 ] 5 0.005% 99.980% ( 1900, 2900 ] 10 0.011% 99.991% ( 2900, 4400 ] 10 0.011% 100.002% ( 4400, 6600 ] 10 0.011% 100.013% ( 6600, 9900 ] 1 0.001% 100.014% ** Level 4 read latency histogram (micros): Count: 60 Average: 1587.8667 StdDev: 2105.00 Min: 1 Median: 170.0000 Max: 7133 Percentiles: P50: 170.00 P75: 2757.14 P99: 7133.00 P99.9: 7133.00 P99.99: 7133.00 ------------------------------------------------------ [ 0, 1 ] 8 13.333% 13.333% ### ( 1, 2 ] 7 11.667% 25.000% ## ( 2, 3 ] 2 3.333% 28.333% # ( 6, 10 ] 2 3.333% 31.667% # ( 10, 15 ] 2 3.333% 35.000% # ( 15, 22 ] 2 3.333% 38.333% # ( 22, 34 ] 4 6.667% 45.000% # ( 51, 76 ] 2 3.333% 48.333% # ( 110, 170 ] 1 1.667% 50.000% ( 250, 380 ] 1 1.667% 51.667% ( 380, 580 ] 1 1.667% 53.333% ( 580, 870 ] 2 3.333% 56.667% # ( 870, 1300 ] 1 1.667% 58.333% ( 1300, 1900 ] 4 6.667% 65.000% # ( 1900, 2900 ] 7 11.667% 76.667% ## ( 2900, 4400 ] 6 10.000% 86.667% ## ( 4400, 6600 ] 5 8.333% 95.000% ## ( 6600, 9900 ] 3 5.000% 100.000% # ** Level 5 read latency histogram (micros): Count: 20 Average: 623.9000 StdDev: 730.57 Min: 130 Median: 160.0000 Max: 2473 Percentiles: P50: 160.00 P75: 870.00 P99: 2473.00 P99.9: 2473.00 P99.99: 2473.00 ------------------------------------------------------ ( 110, 170 ] 12 60.000% 60.000% ############ ( 380, 580 ] 2 10.000% 70.000% ## ( 580, 870 ] 1 5.000% 75.000% # ( 1300, 1900 ] 3 15.000% 90.000% ### ( 1900, 2900 ] 2 10.000% 100.000% ## ** DB Stats ** Uptime(secs): 18.3 total, 3.4 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 322.20 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.973 H:M:S, 16.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 288.29 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.888 H:M:S, 26.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 3.250 micros/op 307644 ops/sec 20.069 seconds 6173999 operations; 298.1 MB/s Microseconds per write: Count: 6173999 Average: 3.2505 StdDev: 277.22 Min: 0 Median: 1.6772 Max: 244571 Percentiles: P50: 1.68 P75: 2.53 P99: 5.82 P99.9: 86.62 P99.99: 1237.02 ------------------------------------------------------ [ 0, 1 ] 979536 15.866% 15.866% ### ( 1, 2 ] 3112249 50.409% 66.274% ########## ( 2, 3 ] 1015856 16.454% 82.728% ### ( 3, 4 ] 793314 12.849% 95.578% ### ( 4, 6 ] 232207 3.761% 99.339% # ( 6, 10 ] 23344 0.378% 99.717% ( 10, 15 ] 5786 0.094% 99.810% ( 15, 22 ] 3124 0.051% 99.861% ( 22, 34 ] 902 0.015% 99.876% ( 34, 51 ] 59 0.001% 99.877% ( 51, 76 ] 6 0.000% 99.877% ( 76, 110 ] 4615 0.075% 99.951% ( 110, 170 ] 1490 0.024% 99.976% ( 170, 250 ] 145 0.002% 99.978% ( 250, 380 ] 6 0.000% 99.978% ( 580, 870 ] 6 0.000% 99.978% ( 870, 1300 ] 863 0.014% 99.992% ( 1300, 1900 ] 2 0.000% 99.992% ( 1900, 2900 ] 467 0.008% 100.000% ( 2900, 4400 ] 1 0.000% 100.000% ( 4400, 6600 ] 2 0.000% 100.000% ( 6600, 9900 ] 1 0.000% 100.000% ( 9900, 14000 ] 1 0.000% 100.000% ( 14000, 22000 ] 3 0.000% 100.000% ( 22000, 33000 ] 1 0.000% 100.000% ( 50000, 75000 ] 1 0.000% 100.000% ( 170000, 250000 ] 12 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 123645 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 6173999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 6371566968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 169 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 4381885 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 8290859879 rocksdb.compact.write.bytes COUNT : 8164520232 rocksdb.flush.write.bytes COUNT : 6330516846 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 14 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 98 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 6290422784 rocksdb.memtable.garbage.bytes.at.flush COUNT : 12509184 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 8339333533 rocksdb.non.last.level.read.count COUNT : 2009349 rocksdb.block.checksum.compute.count COUNT : 2009025 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.289055 P95 : 3.612633 P99 : 5.401451 P100 : 244568.000000 COUNT : 6173999 SUM : 17365383 rocksdb.compaction.times.micros P50 : 1019375.000000 P95 : 1838750.000000 P99 : 3769534.000000 P100 : 3769534.000000 COUNT : 27 SUM : 29009883 rocksdb.compaction.times.cpu_micros P50 : 994210.526316 P95 : 1655000.000000 P99 : 3592254.000000 P100 : 3592254.000000 COUNT : 27 SUM : 27808845 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 725.000000 P95 : 1268.466667 P99 : 1412.000000 P100 : 1412.000000 COUNT : 84 SUM : 64951 rocksdb.compaction.outfile.sync.micros P50 : 825.869565 P95 : 1832.500000 P99 : 10320.000000 P100 : 10320.000000 COUNT : 69 SUM : 68538 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 85.714286 P95 : 312.833333 P99 : 806.200000 P100 : 2564.000000 COUNT : 122 SUM : 16419 rocksdb.table.open.io.micros P50 : 1357.692308 P95 : 16366.666667 P99 : 20873.333333 P100 : 21556.000000 COUNT : 169 SUM : 486178 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.532734 P95 : 5.978792 P99 : 9.682381 P100 : 15935.000000 COUNT : 2008633 SUM : 8874212 rocksdb.read.block.get.micros P50 : 485.263158 P95 : 4416.923077 P99 : 7111.500000 P100 : 7939.000000 COUNT : 338 SUM : 359452 rocksdb.write.raw.block.micros P50 : 0.510679 P95 : 0.970290 P99 : 2.735089 P100 : 5476.000000 COUNT : 3511036 SUM : 6578047 rocksdb.numfiles.in.singlecompaction P50 : 3.210526 P95 : 6.400000 P99 : 26.000000 P100 : 26.000000 COUNT : 38 SUM : 146 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.555873 P95 : 1207.547509 P99 : 2652.553648 P100 : 244374.000000 COUNT : 13431 SUM : 4382333 rocksdb.sst.read.micros P50 : 1.509128 P95 : 3.504110 P99 : 5.895989 P100 : 7880.000000 COUNT : 2009548 SUM : 5027853 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 6173999 SUM : 6371566968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 202340.425532 P95 : 341000.000000 P99 : 372200.000000 P100 : 374507.000000 COUNT : 84 SUM : 16995143 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 101 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0