[2024-07-14 06:25:50.675381] Starting SPDK v24.09-pre git sha1 719d03c6a / DPDK 24.03.0 initialization... [2024-07-14 06:25:50.675529] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3596063 ] Set seed to 1720931150672058 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-14 06:25:50.778951] app.c: 908:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-14 06:25:50.990468] app.c: 603:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-14 06:25:50.990523] app.c: 604:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3596063' to capture a snapshot of events at runtime. [2024-07-14 06:25:50.990534] app.c: 609:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-14 06:25:50.990544] app.c: 610:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-14 06:25:50.990552] app.c: 611:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3596063 for offline analysis/debug. [2024-07-14 06:25:50.990651] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sun Jul 14 06:26:00 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/07/14-06:26:13 ... thread 0: (1000000,1000000) ops and (72897.9,72897.9) ops/second in (13.717822,13.717822) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/9 881.88 MB 18.9 0.0 0.0 0.0 0.9 0.9 0.0 1.0 0.0 159.6 5.52 2.44 11 0.502 0 0 0.0 0.0 L1 4/4 182.94 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.0 1.9 122.9 122.8 2.01 1.20 1 2.015 251K 216 0.0 0.0 L2 3/2 240.06 MB 0.6 5.1 1.0 4.2 5.1 0.9 0.0 5.2 109.6 108.5 47.96 25.05 17 2.821 961K 0 0.0 0.0 L3 18/6 1.21 GB 0.9 8.0 5.3 2.6 7.9 5.3 0.1 1.5 120.4 120.2 67.70 38.73 47 1.440 6675K 3071 0.0 0.0 L4 149/0 9.64 GB 1.0 0.0 0.0 0.0 0.0 0.0 5.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1108/0 91.23 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1293/21 103.35 GB 0.0 13.3 6.4 6.9 14.1 7.2 10.4 16.4 110.8 117.5 123.20 67.41 76 1.621 7888K 3287 0.0 0.0 Int 0/0 0.00 KB 0.0 13.3 6.4 6.9 14.1 7.2 10.4 16.4 110.8 117.5 123.20 67.41 76 1.621 7888K 3287 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.3 6.4 6.9 13.3 6.4 0.0 0.0 116.0 115.5 117.68 64.98 65 1.810 7888K 3287 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 159.6 5.52 2.44 11 0.502 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 13.9 total, 13.7 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.13 GB write, 1041.22 MB/s write, 13.33 GB read, 982.13 MB/s read, 123.2 seconds Interval compaction: 14.13 GB write, 1054.98 MB/s write, 13.33 GB read, 995.11 MB/s read, 123.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 37, pending-compaction-bytes-stops: 0, total-delays: 37, total-stops: 27, interval: 63 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 57852 Average: 85.7105 StdDev: 423.80 Min: 1 Median: 2.1435 Max: 30073 Percentiles: P50: 2.14 P75: 3.45 P99: 2061.57 P99.9: 4523.67 P99.99: 6569.15 ------------------------------------------------------ [ 0, 1 ] 1671 2.888% 2.888% # ( 1, 2 ] 25234 43.618% 46.507% ######### ( 2, 3 ] 14095 24.364% 70.870% ##### ( 3, 4 ] 5370 9.282% 80.153% ## ( 4, 6 ] 5394 9.324% 89.477% ## ( 6, 10 ] 1313 2.270% 91.746% ( 10, 15 ] 269 0.465% 92.211% ( 15, 22 ] 57 0.099% 92.310% ( 22, 34 ] 5 0.009% 92.318% ( 34, 51 ] 5 0.009% 92.327% ( 51, 76 ] 1 0.002% 92.329% ( 76, 110 ] 17 0.029% 92.358% ( 110, 170 ] 165 0.285% 92.643% ( 170, 250 ] 313 0.541% 93.184% ( 250, 380 ] 493 0.852% 94.037% ( 380, 580 ] 686 1.186% 95.222% ( 580, 870 ] 847 1.464% 96.686% ( 870, 1300 ] 793 1.371% 98.057% ( 1300, 1900 ] 494 0.854% 98.911% ( 1900, 2900 ] 331 0.572% 99.483% ( 2900, 4400 ] 240 0.415% 99.898% ( 4400, 6600 ] 56 0.097% 99.995% ( 6600, 9900 ] 3 0.005% 100.000% ( 9900, 14000 ] 2 0.003% 100.003% ( 14000, 22000 ] 2 0.003% 100.007% ( 22000, 33000 ] 1 0.002% 100.009% ** Level 1 read latency histogram (micros): Count: 226722 Average: 12.5985 StdDev: 139.93 Min: 1 Median: 2.2519 Max: 7479 Percentiles: P50: 2.25 P75: 3.35 P99: 13.38 P99.9: 1870.24 P99.99: 2809.80 ------------------------------------------------------ [ 0, 1 ] 2896 1.277% 1.277% ( 1, 2 ] 94226 41.560% 42.837% ######## ( 2, 3 ] 64473 28.437% 71.275% ###### ( 3, 4 ] 24076 10.619% 81.894% ## ( 4, 6 ] 29920 13.197% 95.090% ### ( 6, 10 ] 7871 3.472% 98.562% # ( 10, 15 ] 1470 0.648% 99.210% ( 15, 22 ] 238 0.105% 99.315% ( 22, 34 ] 21 0.009% 99.325% ( 34, 51 ] 11 0.005% 99.330% ( 51, 76 ] 13 0.006% 99.335% ( 76, 110 ] 7 0.003% 99.338% ( 110, 170 ] 30 0.013% 99.352% ( 170, 250 ] 60 0.026% 99.378% ( 250, 380 ] 108 0.048% 99.426% ( 380, 580 ] 185 0.082% 99.507% ( 580, 870 ] 264 0.116% 99.624% ( 870, 1300 ] 325 0.143% 99.767% ( 1300, 1900 ] 317 0.140% 99.907% ( 1900, 2900 ] 207 0.091% 99.998% ( 2900, 4400 ] 93 0.041% 100.039% ( 4400, 6600 ] 28 0.012% 100.052% ( 6600, 9900 ] 1 0.000% 100.052% ** Level 2 read latency histogram (micros): Count: 2164314 Average: 14.0400 StdDev: 172.93 Min: 0 Median: 2.2744 Max: 21459 Percentiles: P50: 2.27 P75: 3.26 P99: 10.85 P99.9: 940.91 P99.99: 1294.48 ------------------------------------------------------ [ 0, 1 ] 14666 0.678% 0.678% ( 1, 2 ] 886608 40.965% 41.642% ######## ( 2, 3 ] 659108 30.453% 72.096% ###### ( 3, 4 ] 238448 11.017% 83.113% ## ( 4, 6 ] 275157 12.713% 95.827% ### ( 6, 10 ] 66153 3.057% 98.883% # ( 10, 15 ] 14965 0.691% 99.575% ( 15, 22 ] 1981 0.092% 99.666% ( 22, 34 ] 142 0.007% 99.673% ( 34, 51 ] 174 0.008% 99.681% ( 51, 76 ] 149 0.007% 99.688% ( 76, 110 ] 45 0.002% 99.690% ( 110, 170 ] 137 0.006% 99.696% ( 170, 250 ] 303 0.014% 99.710% ( 250, 380 ] 602 0.028% 99.738% ( 380, 580 ] 1199 0.055% 99.793% ( 580, 870 ] 1922 0.089% 99.882% ( 870, 1300 ] 2369 0.109% 99.991% ( 1300, 1900 ] 2208 0.102% 100.093% ( 1900, 2900 ] 2432 0.112% 100.206% ( 2900, 4400 ] 1918 0.089% 100.294% ( 4400, 6600 ] 472 0.022% 100.316% ( 6600, 9900 ] 50 0.002% 100.319% ( 9900, 14000 ] 10 0.000% 100.319% ( 14000, 22000 ] 5 0.000% 100.319% ** Level 3 read latency histogram (micros): Count: 743575 Average: 17.1057 StdDev: 188.16 Min: 0 Median: 2.2256 Max: 19839 Percentiles: P50: 2.23 P75: 3.15 P99: 14.09 P99.9: 2052.91 P99.99: 2766.36 ------------------------------------------------------ [ 0, 1 ] 6656 0.895% 0.895% ( 1, 2 ] 314571 42.305% 43.200% ######## ( 2, 3 ] 224154 30.145% 73.346% ###### ( 3, 4 ] 79960 10.753% 84.099% ## ( 4, 6 ] 87805 11.808% 95.908% ## ( 6, 10 ] 19544 2.628% 98.536% # ( 10, 15 ] 4218 0.567% 99.103% ( 15, 22 ] 550 0.074% 99.177% ( 22, 34 ] 43 0.006% 99.183% ( 34, 51 ] 75 0.010% 99.193% ( 51, 76 ] 56 0.008% 99.201% ( 76, 110 ] 28 0.004% 99.205% ( 110, 170 ] 85 0.011% 99.216% ( 170, 250 ] 253 0.034% 99.250% ( 250, 380 ] 374 0.050% 99.300% ( 380, 580 ] 721 0.097% 99.397% ( 580, 870 ] 1139 0.153% 99.550% ( 870, 1300 ] 1349 0.181% 99.732% ( 1300, 1900 ] 1108 0.149% 99.881% ( 1900, 2900 ] 938 0.126% 100.007% ( 2900, 4400 ] 761 0.102% 100.109% ( 4400, 6600 ] 157 0.021% 100.130% ( 6600, 9900 ] 14 0.002% 100.132% ( 9900, 14000 ] 3 0.000% 100.133% ( 14000, 22000 ] 4 0.001% 100.133% ** Level 4 read latency histogram (micros): Count: 3156 Average: 1472.1759 StdDev: 1376.70 Min: 86 Median: 996.1538 Max: 9719 Percentiles: P50: 996.15 P75: 1889.53 P99: 6482.67 P99.9: 9483.41 P99.99: 9719.00 ------------------------------------------------------ ( 76, 110 ] 5 0.158% 0.158% ( 110, 170 ] 84 2.662% 2.820% # ( 170, 250 ] 160 5.070% 7.890% # ( 250, 380 ] 239 7.573% 15.463% ## ( 380, 580 ] 363 11.502% 26.965% ## ( 580, 870 ] 563 17.839% 44.804% #### ( 870, 1300 ] 559 17.712% 62.516% #### ( 1300, 1900 ] 401 12.706% 75.222% ### ( 1900, 2900 ] 316 10.013% 85.234% ## ( 2900, 4400 ] 318 10.076% 95.311% ## ( 4400, 6600 ] 123 3.897% 99.208% # ( 6600, 9900 ] 25 0.792% 100.000% ** Level 5 read latency histogram (micros): Count: 27635 Average: 1497.8540 StdDev: 1424.49 Min: 77 Median: 1025.9952 Max: 32901 Percentiles: P50: 1026.00 P75: 1978.86 P99: 6463.69 P99.9: 9766.69 P99.99: 19984.57 ------------------------------------------------------ ( 76, 110 ] 91 0.329% 0.329% ( 110, 170 ] 722 2.613% 2.942% # ( 170, 250 ] 1250 4.523% 7.465% # ( 250, 380 ] 1950 7.056% 14.521% # ( 380, 580 ] 3300 11.941% 26.463% ## ( 580, 870 ] 4753 17.199% 43.662% ### ( 870, 1300 ] 4828 17.471% 61.133% ### ( 1300, 1900 ] 3597 13.016% 74.149% ### ( 1900, 2900 ] 2983 10.794% 84.943% ## ( 2900, 4400 ] 2865 10.367% 95.310% ## ( 4400, 6600 ] 1087 3.933% 99.244% # ( 6600, 9900 ] 189 0.684% 99.928% ( 9900, 14000 ] 12 0.043% 99.971% ( 14000, 22000 ] 7 0.025% 99.996% ( 22000, 33000 ] 3 0.011% 100.007% ** DB Stats ** Uptime(secs): 13.9 total, 13.7 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 70.80 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.854 H:M:S, 70.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 71.73 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:9.854 H:M:S, 71.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 5 num-running-flushes: 1 2024/07/14-06:26:17 ... thread 0: (1000000,2000000) ops and (301497.6,117408.1) ops/second in (3.316776,17.034598) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/9 1.78 GB 119.6 1.0 0.0 1.0 2.9 1.8 0.0 1.6 81.6 225.6 13.12 9.36 31 0.423 1087K 1126 0.0 0.0 L1 4/4 182.94 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.0 1.9 122.9 122.8 2.01 1.20 1 2.015 251K 216 0.0 0.0 L2 1/0 55.67 MB 0.6 5.1 1.0 4.2 5.1 0.9 0.0 5.2 109.6 108.5 47.96 25.05 17 2.821 961K 0 0.0 0.0 L3 11/0 896.94 MB 0.9 8.3 5.5 2.8 8.3 5.5 0.1 1.5 122.0 121.8 69.86 40.45 49 1.426 7052K 3121 0.0 0.0 L4 149/0 9.47 GB 1.0 0.1 0.1 0.0 0.1 0.1 5.5 1.0 210.9 210.9 0.62 0.60 1 0.620 132K 0 0.0 0.0 L5 1116/0 91.91 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1295/13 104.27 GB 0.0 14.9 6.7 8.1 16.7 8.5 11.5 9.0 114.0 127.7 133.58 76.65 99 1.349 9486K 4463 0.0 0.0 Int 0/0 0.00 KB 0.0 14.9 6.7 8.1 16.7 8.5 11.5 9.0 114.0 127.7 133.58 76.65 99 1.349 9486K 4463 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.9 6.7 8.1 14.8 6.7 0.0 0.0 122.1 121.6 124.69 71.37 72 1.732 9486K 4463 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 212.7 8.89 5.28 27 0.329 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 17.2 total, 17.0 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.65 GB write, 990.51 MB/s write, 14.87 GB read, 884.33 MB/s read, 133.6 seconds Interval compaction: 16.65 GB write, 1001.05 MB/s write, 14.87 GB read, 893.75 MB/s read, 133.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 37, pending-compaction-bytes-stops: 0, total-delays: 37, total-stops: 27, interval: 63 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 346064 Average: 26.2597 StdDev: 185.14 Min: 0 Median: 1.6033 Max: 30073 Percentiles: P50: 1.60 P75: 2.15 P99: 557.19 P99.9: 2332.56 P99.99: 3615.81 ------------------------------------------------------ [ 0, 1 ] 52274 15.105% 15.105% ### ( 1, 2 ] 200170 57.842% 72.947% ############ ( 2, 3 ] 46319 13.385% 86.332% ### ( 3, 4 ] 13461 3.890% 90.221% # ( 4, 6 ] 11063 3.197% 93.418% # ( 6, 10 ] 2577 0.745% 94.163% ( 10, 15 ] 690 0.199% 94.362% ( 15, 22 ] 115 0.033% 94.396% ( 22, 34 ] 16 0.005% 94.400% ( 34, 51 ] 8 0.002% 94.402% ( 51, 76 ] 2 0.001% 94.403% ( 76, 110 ] 542 0.157% 94.560% ( 110, 170 ] 5980 1.728% 96.288% ( 170, 250 ] 5229 1.511% 97.799% ( 250, 380 ] 2451 0.708% 98.507% ( 380, 580 ] 1926 0.557% 99.063% ( 580, 870 ] 1342 0.388% 99.451% ( 870, 1300 ] 897 0.259% 99.710% ( 1300, 1900 ] 508 0.147% 99.857% ( 1900, 2900 ] 342 0.099% 99.956% ( 2900, 4400 ] 246 0.071% 100.027% ( 4400, 6600 ] 56 0.016% 100.043% ( 6600, 9900 ] 3 0.001% 100.044% ( 9900, 14000 ] 2 0.001% 100.045% ( 14000, 22000 ] 2 0.001% 100.045% ( 22000, 33000 ] 1 0.000% 100.046% ** Level 1 read latency histogram (micros): Count: 232506 Average: 12.8833 StdDev: 138.68 Min: 1 Median: 2.2766 Max: 7479 Percentiles: P50: 2.28 P75: 3.42 P99: 16.09 P99.9: 1859.29 P99.99: 2807.00 ------------------------------------------------------ [ 0, 1 ] 2920 1.256% 1.256% ( 1, 2 ] 95148 40.923% 42.179% ######## ( 2, 3 ] 65743 28.276% 70.455% ###### ( 3, 4 ] 25260 10.864% 81.319% ## ( 4, 6 ] 31314 13.468% 94.787% ### ( 6, 10 ] 8171 3.514% 98.301% # ( 10, 15 ] 1581 0.680% 98.981% ( 15, 22 ] 281 0.121% 99.102% ( 22, 34 ] 34 0.015% 99.117% ( 34, 51 ] 12 0.005% 99.122% ( 51, 76 ] 14 0.006% 99.128% ( 76, 110 ] 20 0.009% 99.136% ( 110, 170 ] 239 0.103% 99.239% ( 170, 250 ] 243 0.105% 99.344% ( 250, 380 ] 161 0.069% 99.413% ( 380, 580 ] 236 0.102% 99.514% ( 580, 870 ] 274 0.118% 99.632% ( 870, 1300 ] 327 0.141% 99.773% ( 1300, 1900 ] 317 0.136% 99.909% ( 1900, 2900 ] 207 0.089% 99.998% ( 2900, 4400 ] 93 0.040% 100.038% ( 4400, 6600 ] 28 0.012% 100.050% ( 6600, 9900 ] 1 0.000% 100.051% ** Level 2 read latency histogram (micros): Count: 2165953 Average: 14.0376 StdDev: 172.87 Min: 0 Median: 2.2741 Max: 21459 Percentiles: P50: 2.27 P75: 3.26 P99: 10.86 P99.9: 940.77 P99.99: 1294.45 ------------------------------------------------------ [ 0, 1 ] 14671 0.677% 0.677% ( 1, 2 ] 887489 40.975% 41.652% ######## ( 2, 3 ] 659563 30.451% 72.103% ###### ( 3, 4 ] 238545 11.013% 83.117% ## ( 4, 6 ] 275261 12.709% 95.825% ### ( 6, 10 ] 66188 3.056% 98.881% # ( 10, 15 ] 14968 0.691% 99.572% ( 15, 22 ] 1981 0.091% 99.664% ( 22, 34 ] 142 0.007% 99.670% ( 34, 51 ] 174 0.008% 99.678% ( 51, 76 ] 149 0.007% 99.685% ( 76, 110 ] 49 0.002% 99.687% ( 110, 170 ] 164 0.008% 99.695% ( 170, 250 ] 318 0.015% 99.710% ( 250, 380 ] 608 0.028% 99.738% ( 380, 580 ] 1204 0.056% 99.793% ( 580, 870 ] 1923 0.089% 99.882% ( 870, 1300 ] 2370 0.109% 99.991% ( 1300, 1900 ] 2208 0.102% 100.093% ( 1900, 2900 ] 2432 0.112% 100.206% ( 2900, 4400 ] 1918 0.089% 100.294% ( 4400, 6600 ] 472 0.022% 100.316% ( 6600, 9900 ] 50 0.002% 100.318% ( 9900, 14000 ] 10 0.000% 100.319% ( 14000, 22000 ] 5 0.000% 100.319% ** Level 3 read latency histogram (micros): Count: 773367 Average: 17.2139 StdDev: 185.05 Min: 0 Median: 2.2123 Max: 19839 Percentiles: P50: 2.21 P75: 3.13 P99: 132.97 P99.9: 2016.76 P99.99: 2758.00 ------------------------------------------------------ [ 0, 1 ] 7129 0.922% 0.922% ( 1, 2 ] 330287 42.708% 43.629% ######### ( 2, 3 ] 232013 30.000% 73.630% ###### ( 3, 4 ] 81480 10.536% 84.166% ## ( 4, 6 ] 89452 11.567% 95.732% ## ( 6, 10 ] 19785 2.558% 98.290% # ( 10, 15 ] 4276 0.553% 98.843% ( 15, 22 ] 554 0.072% 98.915% ( 22, 34 ] 43 0.006% 98.921% ( 34, 51 ] 75 0.010% 98.930% ( 51, 76 ] 56 0.007% 98.938% ( 76, 110 ] 107 0.014% 98.951% ( 110, 170 ] 983 0.127% 99.078% ( 170, 250 ] 1033 0.134% 99.212% ( 250, 380 ] 638 0.082% 99.295% ( 380, 580 ] 887 0.115% 99.409% ( 580, 870 ] 1216 0.157% 99.566% ( 870, 1300 ] 1362 0.176% 99.743% ( 1300, 1900 ] 1108 0.143% 99.886% ( 1900, 2900 ] 939 0.121% 100.007% ( 2900, 4400 ] 761 0.098% 100.106% ( 4400, 6600 ] 157 0.020% 100.126% ( 6600, 9900 ] 14 0.002% 100.128% ( 9900, 14000 ] 3 0.000% 100.128% ( 14000, 22000 ] 4 0.001% 100.129% ** Level 4 read latency histogram (micros): Count: 7483 Average: 752.3051 StdDev: 1090.98 Min: 1 Median: 288.8996 Max: 9719 Percentiles: P50: 288.90 P75: 825.69 P99: 5708.73 P99.9: 8912.24 P99.99: 9719.00 ------------------------------------------------------ [ 0, 1 ] 2 0.027% 0.027% ( 1, 2 ] 6 0.080% 0.107% ( 2, 3 ] 24 0.321% 0.428% ( 3, 4 ] 12 0.160% 0.588% ( 4, 6 ] 23 0.307% 0.895% ( 6, 10 ] 3 0.040% 0.935% ( 10, 15 ] 2 0.027% 0.962% ( 76, 110 ] 160 2.138% 3.100% ( 110, 170 ] 1734 23.173% 26.273% ##### ( 170, 250 ] 1543 20.620% 46.893% #### ( 250, 380 ] 777 10.384% 57.276% ## ( 380, 580 ] 751 10.036% 67.313% ## ( 580, 870 ] 679 9.074% 76.386% ## ( 870, 1300 ] 580 7.751% 84.137% ## ( 1300, 1900 ] 403 5.386% 89.523% # ( 1900, 2900 ] 318 4.250% 93.773% # ( 2900, 4400 ] 318 4.250% 98.022% # ( 4400, 6600 ] 123 1.644% 99.666% ( 6600, 9900 ] 25 0.334% 100.000% ** Level 5 read latency histogram (micros): Count: 64705 Average: 772.1603 StdDev: 1128.17 Min: 77 Median: 295.6224 Max: 32901 Percentiles: P50: 295.62 P75: 851.75 P99: 5707.35 P99.9: 9067.06 P99.99: 13497.58 ------------------------------------------------------ ( 76, 110 ] 1460 2.256% 2.256% ( 110, 170 ] 15359 23.737% 25.993% ##### ( 170, 250 ] 13278 20.521% 46.514% #### ( 250, 380 ] 6427 9.933% 56.447% ## ( 380, 580 ] 6477 10.010% 66.457% ## ( 580, 870 ] 5899 9.117% 75.574% ## ( 870, 1300 ] 5026 7.768% 83.341% ## ( 1300, 1900 ] 3615 5.587% 88.928% # ( 1900, 2900 ] 3005 4.644% 93.572% # ( 2900, 4400 ] 2866 4.429% 98.002% # ( 4400, 6600 ] 1087 1.680% 99.682% ( 6600, 9900 ] 189 0.292% 99.974% ( 9900, 14000 ] 12 0.019% 99.992% ( 14000, 22000 ] 7 0.011% 100.003% ( 22000, 33000 ] 3 0.005% 100.008% ** DB Stats ** Uptime(secs): 17.2 total, 3.3 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 114.32 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.854 H:M:S, 57.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 296.74 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/14-06:26:20 ... thread 0: (1000000,3000000) ops and (272018.5,144851.8) ops/second in (3.676221,20.710819) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/9 2.77 GB 220.3 2.0 0.0 2.0 4.8 2.8 0.0 1.7 103.9 245.5 20.00 15.69 50 0.400 2110K 2118 0.0 0.0 L1 4/4 182.94 MB 0.0 0.2 0.1 0.1 0.2 0.1 0.0 1.9 122.9 122.8 2.01 1.20 1 2.015 251K 216 0.0 0.0 L2 1/0 55.67 MB 0.6 5.1 1.0 4.2 5.1 0.9 0.0 5.2 109.6 108.5 47.96 25.05 17 2.821 961K 0 0.0 0.0 L3 11/0 896.94 MB 0.9 8.3 5.5 2.8 8.3 5.5 0.1 1.5 122.0 121.8 69.86 40.45 49 1.426 7052K 3121 0.0 0.0 L4 149/0 9.47 GB 1.0 0.1 0.1 0.0 0.1 0.1 5.5 1.0 210.9 210.9 0.62 0.60 1 0.620 132K 0 0.0 0.0 L5 1116/0 91.91 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1299/13 105.26 GB 0.0 15.9 6.7 9.1 18.6 9.4 11.5 6.7 115.6 135.3 140.47 82.99 118 1.190 10M 5455 0.0 0.0 Int 0/0 0.00 KB 0.0 15.9 6.7 9.1 18.6 9.4 11.5 6.7 115.6 135.3 140.47 82.99 118 1.190 10M 5455 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 15.9 6.7 9.1 15.8 6.7 0.0 0.0 126.3 125.8 128.57 75.16 76 1.692 10M 5455 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 238.3 11.90 7.82 42 0.283 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.9 total, 20.7 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 18.56 GB write, 909.65 MB/s write, 15.85 GB read, 776.98 MB/s read, 140.5 seconds Interval compaction: 18.56 GB write, 917.62 MB/s write, 15.85 GB read, 783.78 MB/s read, 140.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 2, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 2, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 37, pending-compaction-bytes-stops: 0, total-delays: 39, total-stops: 27, interval: 65 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 682032 Average: 18.1494 StdDev: 137.16 Min: 0 Median: 1.5783 Max: 30073 Percentiles: P50: 1.58 P75: 2.00 P99: 361.61 P99.9: 1457.24 P99.99: 2569.90 ------------------------------------------------------ [ 0, 1 ] 106659 15.638% 15.638% ### ( 1, 2 ] 405226 59.415% 75.053% ############ ( 2, 3 ] 90856 13.321% 88.374% ### ( 3, 4 ] 23706 3.476% 91.850% # ( 4, 6 ] 18952 2.779% 94.629% # ( 6, 10 ] 4559 0.668% 95.297% ( 10, 15 ] 1267 0.186% 95.483% ( 15, 22 ] 206 0.030% 95.513% ( 22, 34 ] 35 0.005% 95.518% ( 34, 51 ] 10 0.001% 95.520% ( 51, 76 ] 4 0.001% 95.520% ( 76, 110 ] 1165 0.171% 95.691% ( 110, 170 ] 10590 1.553% 97.244% ( 170, 250 ] 8747 1.282% 98.526% ( 250, 380 ] 3764 0.552% 99.078% ( 380, 580 ] 2883 0.423% 99.501% ( 580, 870 ] 1637 0.240% 99.741% ( 870, 1300 ] 951 0.139% 99.881% ( 1300, 1900 ] 515 0.076% 99.956% ( 1900, 2900 ] 349 0.051% 100.007% ( 2900, 4400 ] 253 0.037% 100.044% ( 4400, 6600 ] 57 0.008% 100.053% ( 6600, 9900 ] 3 0.000% 100.053% ( 9900, 14000 ] 2 0.000% 100.053% ( 14000, 22000 ] 2 0.000% 100.054% ( 22000, 33000 ] 1 0.000% 100.054% ** Level 1 read latency histogram (micros): Count: 267016 Average: 12.2262 StdDev: 130.32 Min: 1 Median: 2.2543 Max: 7479 Percentiles: P50: 2.25 P75: 3.40 P99: 113.52 P99.9: 1795.87 P99.99: 2795.16 ------------------------------------------------------ [ 0, 1 ] 4103 1.537% 1.537% ( 1, 2 ] 110485 41.378% 42.914% ######## ( 2, 3 ] 74395 27.862% 70.776% ###### ( 3, 4 ] 28054 10.506% 81.282% ## ( 4, 6 ] 35712 13.374% 94.657% ### ( 6, 10 ] 9222 3.454% 98.111% # ( 10, 15 ] 1830 0.685% 98.796% ( 15, 22 ] 382 0.143% 98.939% ( 22, 34 ] 44 0.016% 98.955% ( 34, 51 ] 12 0.004% 98.960% ( 51, 76 ] 14 0.005% 98.965% ( 76, 110 ] 60 0.022% 98.988% ( 110, 170 ] 559 0.209% 99.197% ( 170, 250 ] 460 0.172% 99.369% ( 250, 380 ] 245 0.092% 99.461% ( 380, 580 ] 285 0.107% 99.568% ( 580, 870 ] 293 0.110% 99.678% ( 870, 1300 ] 332 0.124% 99.802% ( 1300, 1900 ] 317 0.119% 99.921% ( 1900, 2900 ] 207 0.078% 99.998% ( 2900, 4400 ] 94 0.035% 100.033% ( 4400, 6600 ] 28 0.010% 100.044% ( 6600, 9900 ] 1 0.000% 100.044% ** Level 2 read latency histogram (micros): Count: 2166020 Average: 14.0438 StdDev: 172.87 Min: 0 Median: 2.2742 Max: 21459 Percentiles: P50: 2.27 P75: 3.26 P99: 10.88 P99.9: 940.76 P99.99: 1294.45 ------------------------------------------------------ [ 0, 1 ] 14671 0.677% 0.677% ( 1, 2 ] 887489 40.973% 41.651% ######## ( 2, 3 ] 659563 30.450% 72.101% ###### ( 3, 4 ] 238545 11.013% 83.114% ## ( 4, 6 ] 275261 12.708% 95.822% ### ( 6, 10 ] 66188 3.056% 98.878% # ( 10, 15 ] 14968 0.691% 99.569% ( 15, 22 ] 1981 0.091% 99.660% ( 22, 34 ] 142 0.007% 99.667% ( 34, 51 ] 174 0.008% 99.675% ( 51, 76 ] 149 0.007% 99.682% ( 76, 110 ] 52 0.002% 99.684% ( 110, 170 ] 198 0.009% 99.693% ( 170, 250 ] 335 0.015% 99.709% ( 250, 380 ] 612 0.028% 99.737% ( 380, 580 ] 1212 0.056% 99.793% ( 580, 870 ] 1924 0.089% 99.882% ( 870, 1300 ] 2370 0.109% 99.991% ( 1300, 1900 ] 2208 0.102% 100.093% ( 1900, 2900 ] 2432 0.112% 100.206% ( 2900, 4400 ] 1918 0.089% 100.294% ( 4400, 6600 ] 472 0.022% 100.316% ( 6600, 9900 ] 50 0.002% 100.318% ( 9900, 14000 ] 10 0.000% 100.319% ( 14000, 22000 ] 5 0.000% 100.319% ** Level 3 read latency histogram (micros): Count: 776946 Average: 18.0590 StdDev: 185.45 Min: 0 Median: 2.2198 Max: 19839 Percentiles: P50: 2.22 P75: 3.16 P99: 212.01 P99.9: 2016.01 P99.99: 2759.90 ------------------------------------------------------ [ 0, 1 ] 7129 0.918% 0.918% ( 1, 2 ] 330306 42.513% 43.431% ######### ( 2, 3 ] 232184 29.884% 73.315% ###### ( 3, 4 ] 81565 10.498% 83.813% ## ( 4, 6 ] 89637 11.537% 95.350% ## ( 6, 10 ] 19806 2.549% 97.900% # ( 10, 15 ] 4279 0.551% 98.450% ( 15, 22 ] 556 0.072% 98.522% ( 22, 34 ] 43 0.006% 98.527% ( 34, 51 ] 75 0.010% 98.537% ( 51, 76 ] 56 0.007% 98.544% ( 76, 110 ] 233 0.030% 98.574% ( 110, 170 ] 2272 0.292% 98.867% ( 170, 250 ] 1972 0.254% 99.121% ( 250, 380 ] 992 0.128% 99.248% ( 380, 580 ] 1152 0.148% 99.396% ( 580, 870 ] 1310 0.169% 99.565% ( 870, 1300 ] 1383 0.178% 99.743% ( 1300, 1900 ] 1110 0.143% 99.886% ( 1900, 2900 ] 940 0.121% 100.007% ( 2900, 4400 ] 763 0.098% 100.105% ( 4400, 6600 ] 157 0.020% 100.125% ( 6600, 9900 ] 14 0.002% 100.127% ( 9900, 14000 ] 3 0.000% 100.128% ( 14000, 22000 ] 4 0.001% 100.128% ** Level 4 read latency histogram (micros): Count: 12183 Average: 549.4551 StdDev: 899.07 Min: 1 Median: 223.1939 Max: 9719 Percentiles: P50: 223.19 P75: 505.35 P99: 4868.08 P99.9: 8291.84 P99.99: 9719.00 ------------------------------------------------------ [ 0, 1 ] 2 0.016% 0.016% ( 1, 2 ] 9 0.074% 0.090% ( 2, 3 ] 56 0.460% 0.550% ( 3, 4 ] 27 0.222% 0.772% ( 4, 6 ] 53 0.435% 1.207% ( 6, 10 ] 9 0.074% 1.280% ( 10, 15 ] 5 0.041% 1.322% ( 76, 110 ] 385 3.160% 4.482% # ( 110, 170 ] 3574 29.336% 33.818% ###### ( 170, 250 ] 2965 24.337% 58.155% ##### ( 250, 380 ] 1344 11.032% 69.187% ## ( 380, 580 ] 1130 9.275% 78.462% ## ( 580, 870 ] 817 6.706% 85.168% # ( 870, 1300 ] 610 5.007% 90.175% # ( 1300, 1900 ] 408 3.349% 93.524% # ( 1900, 2900 ] 322 2.643% 96.167% # ( 2900, 4400 ] 319 2.618% 98.785% # ( 4400, 6600 ] 123 1.010% 99.795% ( 6600, 9900 ] 25 0.205% 100.000% ** Level 5 read latency histogram (micros): Count: 105645 Average: 562.6396 StdDev: 927.45 Min: 77 Median: 224.2030 Max: 32901 Percentiles: P50: 224.20 P75: 517.97 P99: 4878.76 P99.9: 8352.23 P99.99: 12098.80 ------------------------------------------------------ ( 76, 110 ] 3426 3.243% 3.243% # ( 110, 170 ] 32017 30.306% 33.549% ###### ( 170, 250 ] 25651 24.280% 57.830% ##### ( 250, 380 ] 11368 10.761% 68.590% ## ( 380, 580 ] 9816 9.291% 77.882% ## ( 580, 870 ] 7231 6.845% 84.726% # ( 870, 1300 ] 5273 4.991% 89.717% # ( 1300, 1900 ] 3652 3.457% 93.174% # ( 1900, 2900 ] 3040 2.878% 96.052% # ( 2900, 4400 ] 2878 2.724% 98.776% # ( 4400, 6600 ] 1087 1.029% 99.805% ( 6600, 9900 ] 189 0.179% 99.984% ( 9900, 14000 ] 12 0.011% 99.995% ( 14000, 22000 ] 7 0.007% 100.002% ( 22000, 33000 ] 3 0.003% 100.005% ** DB Stats ** Uptime(secs): 20.9 total, 3.7 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 141.31 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:10.134 H:M:S, 48.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 267.72 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.280 H:M:S, 7.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/14-06:26:25 ... thread 0: (1000000,4000000) ops and (196472.8,155035.3) ops/second in (5.089764,25.800583) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/13 3.44 GB 69.2 3.0 0.0 3.0 6.8 3.8 0.0 1.8 113.3 256.6 27.24 22.33 71 0.384 3134K 3075 0.0 0.0 L1 10/10 559.55 MB 0.0 0.8 0.5 0.3 0.8 0.5 0.0 1.6 55.9 55.7 14.48 4.99 2 7.238 821K 1507 0.0 0.0 L2 1/0 55.67 MB 0.6 5.1 1.0 4.2 5.1 0.9 0.0 5.2 109.6 108.5 47.96 25.05 17 2.821 961K 0 0.0 0.0 L3 11/0 896.94 MB 0.9 8.3 5.5 2.8 8.3 5.5 0.1 1.5 122.0 121.8 69.86 40.45 49 1.426 7052K 3121 0.0 0.0 L4 149/0 9.47 GB 1.0 0.1 0.1 0.0 0.1 0.1 5.5 1.0 210.9 210.9 0.62 0.60 1 0.620 132K 0 0.0 0.0 L5 1116/0 91.91 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1305/23 106.30 GB 0.0 17.4 7.1 10.3 21.1 10.8 11.5 5.5 111.2 135.1 160.16 93.41 140 1.144 12M 7703 0.0 0.0 Int 0/0 0.00 KB 0.0 17.4 7.1 10.3 21.1 10.8 11.5 5.5 111.2 135.1 160.16 93.41 140 1.144 12M 7703 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 17.4 7.1 10.3 17.3 7.0 0.0 0.0 122.9 122.4 144.89 82.74 81 1.789 12M 7703 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 255.8 15.27 10.67 59 0.259 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 26.0 total, 25.8 interval Flush(GB): cumulative 3.814, interval 3.814 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 21.14 GB write, 832.97 MB/s write, 17.39 GB read, 685.16 MB/s read, 160.2 seconds Interval compaction: 21.14 GB write, 838.83 MB/s write, 17.39 GB read, 689.98 MB/s read, 160.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 11, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 11, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 37, pending-compaction-bytes-stops: 0, total-delays: 48, total-stops: 27, interval: 74 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 987934 Average: 17.1559 StdDev: 119.24 Min: 0 Median: 1.5812 Max: 30073 Percentiles: P50: 1.58 P75: 2.07 P99: 329.17 P99.9: 1154.00 P99.99: 1961.51 ------------------------------------------------------ [ 0, 1 ] 163234 16.523% 16.523% ### ( 1, 2 ] 569066 57.602% 74.124% ############ ( 2, 3 ] 127412 12.897% 87.021% ### ( 3, 4 ] 42022 4.254% 91.275% # ( 4, 6 ] 28657 2.901% 94.175% # ( 6, 10 ] 6500 0.658% 94.833% ( 10, 15 ] 1833 0.186% 95.019% ( 15, 22 ] 293 0.030% 95.049% ( 22, 34 ] 50 0.005% 95.054% ( 34, 51 ] 10 0.001% 95.055% ( 51, 76 ] 5 0.001% 95.055% ( 76, 110 ] 1956 0.198% 95.253% ( 110, 170 ] 18857 1.909% 97.162% ( 170, 250 ] 14826 1.501% 98.663% ( 250, 380 ] 5474 0.554% 99.217% ( 380, 580 ] 4067 0.412% 99.628% ( 580, 870 ] 2017 0.204% 99.832% ( 870, 1300 ] 1010 0.102% 99.935% ( 1300, 1900 ] 524 0.053% 99.988% ( 1900, 2900 ] 361 0.037% 100.024% ( 2900, 4400 ] 261 0.026% 100.051% ( 4400, 6600 ] 58 0.006% 100.057% ( 6600, 9900 ] 3 0.000% 100.057% ( 9900, 14000 ] 2 0.000% 100.057% ( 14000, 22000 ] 2 0.000% 100.057% ( 22000, 33000 ] 1 0.000% 100.057% ** Level 1 read latency histogram (micros): Count: 281167 Average: 12.2136 StdDev: 127.58 Min: 1 Median: 2.2880 Max: 7479 Percentiles: P50: 2.29 P75: 3.48 P99: 138.05 P99.9: 1770.31 P99.99: 2788.33 ------------------------------------------------------ [ 0, 1 ] 4371 1.555% 1.555% ( 1, 2 ] 113808 40.477% 42.032% ######## ( 2, 3 ] 77797 27.669% 69.701% ###### ( 3, 4 ] 30982 11.019% 80.720% ## ( 4, 6 ] 38580 13.721% 94.441% ### ( 6, 10 ] 9828 3.495% 97.937% # ( 10, 15 ] 1980 0.704% 98.641% ( 15, 22 ] 476 0.169% 98.810% ( 22, 34 ] 53 0.019% 98.829% ( 34, 51 ] 13 0.005% 98.834% ( 51, 76 ] 14 0.005% 98.839% ( 76, 110 ] 91 0.032% 98.871% ( 110, 170 ] 775 0.276% 99.147% ( 170, 250 ] 596 0.212% 99.359% ( 250, 380 ] 300 0.107% 99.465% ( 380, 580 ] 321 0.114% 99.580% ( 580, 870 ] 316 0.112% 99.692% ( 870, 1300 ] 334 0.119% 99.811% ( 1300, 1900 ] 320 0.114% 99.925% ( 1900, 2900 ] 207 0.074% 99.998% ( 2900, 4400 ] 94 0.033% 100.032% ( 4400, 6600 ] 28 0.010% 100.042% ( 6600, 9900 ] 1 0.000% 100.042% ** Level 2 read latency histogram (micros): Count: 2166115 Average: 14.0515 StdDev: 172.87 Min: 0 Median: 2.2743 Max: 21459 Percentiles: P50: 2.27 P75: 3.26 P99: 10.91 P99.9: 940.74 P99.99: 1294.45 ------------------------------------------------------ [ 0, 1 ] 14671 0.677% 0.677% ( 1, 2 ] 887489 40.971% 41.649% ######## ( 2, 3 ] 659563 30.449% 72.098% ###### ( 3, 4 ] 238545 11.013% 83.110% ## ( 4, 6 ] 275261 12.708% 95.818% ### ( 6, 10 ] 66188 3.056% 98.874% # ( 10, 15 ] 14968 0.691% 99.565% ( 15, 22 ] 1981 0.091% 99.656% ( 22, 34 ] 142 0.007% 99.663% ( 34, 51 ] 174 0.008% 99.671% ( 51, 76 ] 149 0.007% 99.678% ( 76, 110 ] 57 0.003% 99.680% ( 110, 170 ] 244 0.011% 99.691% ( 170, 250 ] 367 0.017% 99.708% ( 250, 380 ] 619 0.029% 99.737% ( 380, 580 ] 1215 0.056% 99.793% ( 580, 870 ] 1926 0.089% 99.882% ( 870, 1300 ] 2370 0.109% 99.991% ( 1300, 1900 ] 2208 0.102% 100.093% ( 1900, 2900 ] 2432 0.112% 100.206% ( 2900, 4400 ] 1918 0.089% 100.294% ( 4400, 6600 ] 472 0.022% 100.316% ( 6600, 9900 ] 50 0.002% 100.318% ( 9900, 14000 ] 10 0.000% 100.319% ( 14000, 22000 ] 5 0.000% 100.319% ** Level 3 read latency histogram (micros): Count: 782898 Average: 19.4437 StdDev: 185.81 Min: 0 Median: 2.2326 Max: 19839 Percentiles: P50: 2.23 P75: 3.21 P99: 248.77 P99.9: 2013.46 P99.99: 2759.86 ------------------------------------------------------ [ 0, 1 ] 7129 0.911% 0.911% ( 1, 2 ] 330310 42.191% 43.101% ######## ( 2, 3 ] 232203 29.659% 72.761% ###### ( 3, 4 ] 81581 10.420% 83.181% ## ( 4, 6 ] 89669 11.453% 94.635% ## ( 6, 10 ] 19809 2.530% 97.165% # ( 10, 15 ] 4279 0.547% 97.711% ( 15, 22 ] 556 0.071% 97.782% ( 22, 34 ] 43 0.005% 97.788% ( 34, 51 ] 75 0.010% 97.797% ( 51, 76 ] 56 0.007% 97.805% ( 76, 110 ] 602 0.077% 97.881% ( 110, 170 ] 5094 0.651% 98.532% ( 170, 250 ] 3720 0.475% 99.007% ( 250, 380 ] 1476 0.189% 99.196% ( 380, 580 ] 1471 0.188% 99.384% ( 580, 870 ] 1422 0.182% 99.565% ( 870, 1300 ] 1400 0.179% 99.744% ( 1300, 1900 ] 1113 0.142% 99.886% ( 1900, 2900 ] 944 0.121% 100.007% ( 2900, 4400 ] 763 0.097% 100.104% ( 4400, 6600 ] 157 0.020% 100.124% ( 6600, 9900 ] 14 0.002% 100.126% ( 9900, 14000 ] 3 0.000% 100.127% ( 14000, 22000 ] 4 0.001% 100.127% ** Level 4 read latency histogram (micros): Count: 19587 Average: 416.2853 StdDev: 733.55 Min: 1 Median: 194.2015 Max: 9719 Percentiles: P50: 194.20 P75: 338.92 P99: 4176.31 P99.9: 7314.52 P99.99: 9641.45 ------------------------------------------------------ [ 0, 1 ] 2 0.010% 0.010% ( 1, 2 ] 15 0.077% 0.087% ( 2, 3 ] 71 0.362% 0.449% ( 3, 4 ] 38 0.194% 0.643% ( 4, 6 ] 74 0.378% 1.021% ( 6, 10 ] 12 0.061% 1.082% ( 10, 15 ] 5 0.026% 1.108% ( 76, 110 ] 864 4.411% 5.519% # ( 110, 170 ] 7163 36.570% 42.089% ####### ( 170, 250 ] 5122 26.150% 68.239% ##### ( 250, 380 ] 1936 9.884% 78.123% ## ( 380, 580 ] 1511 7.714% 85.838% ## ( 580, 870 ] 944 4.820% 90.657% # ( 870, 1300 ] 630 3.216% 93.873% # ( 1300, 1900 ] 409 2.088% 95.962% ( 1900, 2900 ] 322 1.644% 97.606% ( 2900, 4400 ] 321 1.639% 99.244% ( 4400, 6600 ] 123 0.628% 99.872% ( 6600, 9900 ] 25 0.128% 100.000% ** Level 5 read latency histogram (micros): Count: 170676 Average: 424.9750 StdDev: 755.00 Min: 77 Median: 195.1724 Max: 32901 Percentiles: P50: 195.17 P75: 345.06 P99: 4184.06 P99.9: 7181.85 P99.99: 9863.90 ------------------------------------------------------ ( 76, 110 ] 7429 4.353% 4.353% # ( 110, 170 ] 63938 37.462% 41.814% ####### ( 170, 250 ] 44401 26.015% 67.829% ##### ( 250, 380 ] 16737 9.806% 77.635% ## ( 380, 580 ] 13295 7.790% 85.425% ## ( 580, 870 ] 8484 4.971% 90.396% # ( 870, 1300 ] 5466 3.203% 93.598% # ( 1300, 1900 ] 3683 2.158% 95.756% ( 1900, 2900 ] 3064 1.795% 97.552% ( 2900, 4400 ] 2888 1.692% 99.244% ( 4400, 6600 ] 1087 0.637% 99.880% ( 6600, 9900 ] 189 0.111% 99.991% ( 9900, 14000 ] 12 0.007% 99.998% ( 14000, 22000 ] 7 0.004% 100.002% ( 22000, 33000 ] 3 0.002% 100.004% ** DB Stats ** Uptime(secs): 26.0 total, 5.1 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 151.51 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:11.836 H:M:S, 45.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 193.37 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.702 H:M:S, 33.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 2024/07/14-06:26:33 ... thread 0: (1000000,5000000) ops and (131826.7,149762.0) ops/second in (7.585716,33.386299) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/17 4.36 GB 81.6 4.8 0.0 4.8 9.5 4.7 0.0 2.0 132.6 263.4 37.05 31.56 91 0.407 4988K 6481 0.0 0.0 L1 10/10 559.55 MB 0.0 0.8 0.5 0.3 0.8 0.5 0.0 1.6 55.9 55.7 14.48 4.99 2 7.238 821K 1507 0.0 0.0 L2 1/0 55.67 MB 0.6 5.1 1.0 4.2 5.1 0.9 0.0 5.2 109.6 108.5 47.96 25.05 17 2.821 961K 0 0.0 0.0 L3 11/0 896.94 MB 0.9 8.3 5.5 2.8 8.3 5.5 0.1 1.5 122.0 121.8 69.86 40.45 49 1.426 7052K 3121 0.0 0.0 L4 149/0 9.47 GB 1.0 0.1 0.1 0.0 0.1 0.1 5.5 1.0 210.9 210.9 0.62 0.60 1 0.620 132K 0 0.0 0.0 L5 1116/0 91.91 GB 0.9 0.0 0.0 0.0 0.0 0.0 5.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1305/27 107.22 GB 0.0 19.2 7.1 12.1 23.8 11.8 11.5 5.0 115.5 143.6 169.97 102.64 160 1.062 13M 11K 0.0 0.0 Int 0/0 0.00 KB 0.0 19.2 7.1 12.1 23.8 11.8 11.5 5.0 115.5 143.6 169.97 102.64 160 1.062 13M 11K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 19.2 7.1 12.1 19.1 7.0 0.0 0.0 129.4 128.9 151.74 89.49 86 1.764 13M 11K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 266.1 18.23 13.16 74 0.246 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 33.6 total, 33.4 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 23.84 GB write, 727.19 MB/s write, 19.17 GB read, 584.74 MB/s read, 170.0 seconds Interval compaction: 23.84 GB write, 731.15 MB/s write, 19.17 GB read, 587.92 MB/s read, 170.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 22, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 22, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 37, pending-compaction-bytes-stops: 0, total-delays: 59, total-stops: 27, interval: 85 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1549690 Average: 16.6944 StdDev: 102.29 Min: 0 Median: 1.5714 Max: 30073 Percentiles: P50: 1.57 P75: 2.04 P99: 294.91 P99.9: 846.27 P99.99: 1392.00 ------------------------------------------------------ [ 0, 1 ] 268406 17.320% 17.320% ### ( 1, 2 ] 886378 57.197% 74.517% ########### ( 2, 3 ] 183757 11.858% 86.375% ## ( 3, 4 ] 67530 4.358% 90.732% # ( 4, 6 ] 44580 2.877% 93.609% # ( 6, 10 ] 8674 0.560% 94.169% ( 10, 15 ] 2539 0.164% 94.333% ( 15, 22 ] 372 0.024% 94.357% ( 22, 34 ] 53 0.003% 94.360% ( 34, 51 ] 14 0.001% 94.361% ( 51, 76 ] 7 0.000% 94.361% ( 76, 110 ] 3761 0.243% 94.604% ( 110, 170 ] 37717 2.434% 97.038% ( 170, 250 ] 27507 1.775% 98.813% ( 250, 380 ] 8394 0.542% 99.355% ( 380, 580 ] 6022 0.389% 99.743% ( 580, 870 ] 2648 0.171% 99.914% ( 870, 1300 ] 1096 0.071% 99.985% ( 1300, 1900 ] 535 0.035% 100.019% ( 1900, 2900 ] 389 0.025% 100.044% ( 2900, 4400 ] 270 0.017% 100.062% ( 4400, 6600 ] 59 0.004% 100.066% ( 6600, 9900 ] 3 0.000% 100.066% ( 9900, 14000 ] 2 0.000% 100.066% ( 14000, 22000 ] 2 0.000% 100.066% ( 22000, 33000 ] 1 0.000% 100.066% ** Level 1 read latency histogram (micros): Count: 292815 Average: 12.3866 StdDev: 125.64 Min: 1 Median: 2.3371 Max: 7479 Percentiles: P50: 2.34 P75: 3.61 P99: 157.49 P99.9: 1750.35 P99.99: 2787.53 ------------------------------------------------------ [ 0, 1 ] 4449 1.519% 1.519% ( 1, 2 ] 115108 39.311% 40.830% ######## ( 2, 3 ] 79651 27.202% 68.032% ##### ( 3, 4 ] 33677 11.501% 79.533% ## ( 4, 6 ] 42668 14.572% 94.105% ### ( 6, 10 ] 10463 3.573% 97.678% # ( 10, 15 ] 2136 0.729% 98.408% ( 15, 22 ] 601 0.205% 98.613% ( 22, 34 ] 57 0.019% 98.632% ( 34, 51 ] 13 0.004% 98.637% ( 51, 76 ] 14 0.005% 98.641% ( 76, 110 ] 138 0.047% 98.689% ( 110, 170 ] 1152 0.393% 99.082% ( 170, 250 ] 768 0.262% 99.344% ( 250, 380 ] 364 0.124% 99.469% ( 380, 580 ] 362 0.124% 99.592% ( 580, 870 ] 324 0.111% 99.703% ( 870, 1300 ] 337 0.115% 99.818% ( 1300, 1900 ] 320 0.109% 99.927% ( 1900, 2900 ] 207 0.071% 99.998% ( 2900, 4400 ] 95 0.032% 100.030% ( 4400, 6600 ] 28 0.010% 100.040% ( 6600, 9900 ] 1 0.000% 100.040% ** Level 2 read latency histogram (micros): Count: 2166279 Average: 14.0648 StdDev: 172.87 Min: 0 Median: 2.2744 Max: 21459 Percentiles: P50: 2.27 P75: 3.26 P99: 10.97 P99.9: 940.71 P99.99: 1294.44 ------------------------------------------------------ [ 0, 1 ] 14671 0.677% 0.677% ( 1, 2 ] 887489 40.968% 41.646% ######## ( 2, 3 ] 659563 30.447% 72.092% ###### ( 3, 4 ] 238545 11.012% 83.104% ## ( 4, 6 ] 275261 12.707% 95.811% ### ( 6, 10 ] 66188 3.055% 98.866% # ( 10, 15 ] 14968 0.691% 99.557% ( 15, 22 ] 1981 0.091% 99.649% ( 22, 34 ] 142 0.007% 99.655% ( 34, 51 ] 174 0.008% 99.663% ( 51, 76 ] 149 0.007% 99.670% ( 76, 110 ] 68 0.003% 99.673% ( 110, 170 ] 332 0.015% 99.688% ( 170, 250 ] 406 0.019% 99.707% ( 250, 380 ] 634 0.029% 99.737% ( 380, 580 ] 1225 0.057% 99.793% ( 580, 870 ] 1927 0.089% 99.882% ( 870, 1300 ] 2370 0.109% 99.991% ( 1300, 1900 ] 2208 0.102% 100.093% ( 1900, 2900 ] 2432 0.112% 100.206% ( 2900, 4400 ] 1918 0.089% 100.294% ( 4400, 6600 ] 472 0.022% 100.316% ( 6600, 9900 ] 50 0.002% 100.318% ( 9900, 14000 ] 10 0.000% 100.319% ( 14000, 22000 ] 5 0.000% 100.319% ** Level 3 read latency histogram (micros): Count: 791559 Average: 21.3314 StdDev: 186.06 Min: 0 Median: 2.2512 Max: 19839 Percentiles: P50: 2.25 P75: 3.29 P99: 311.66 P99.9: 2006.17 P99.99: 2759.24 ------------------------------------------------------ [ 0, 1 ] 7129 0.901% 0.901% ( 1, 2 ] 330310 41.729% 42.630% ######## ( 2, 3 ] 232203 29.335% 71.965% ###### ( 3, 4 ] 81581 10.306% 82.271% ## ( 4, 6 ] 89669 11.328% 93.599% ## ( 6, 10 ] 19809 2.503% 96.102% # ( 10, 15 ] 4279 0.541% 96.642% ( 15, 22 ] 556 0.070% 96.712% ( 22, 34 ] 43 0.005% 96.718% ( 34, 51 ] 75 0.009% 96.727% ( 51, 76 ] 56 0.007% 96.734% ( 76, 110 ] 1104 0.139% 96.874% ( 110, 170 ] 9635 1.217% 98.091% ( 170, 250 ] 6232 0.787% 98.878% ( 250, 380 ] 2029 0.256% 99.135% ( 380, 580 ] 1881 0.238% 99.372% ( 580, 870 ] 1548 0.196% 99.568% ( 870, 1300 ] 1412 0.178% 99.746% ( 1300, 1900 ] 1116 0.141% 99.887% ( 1900, 2900 ] 946 0.120% 100.007% ( 2900, 4400 ] 763 0.096% 100.103% ( 4400, 6600 ] 157 0.020% 100.123% ( 6600, 9900 ] 14 0.002% 100.125% ( 9900, 14000 ] 3 0.000% 100.125% ( 14000, 22000 ] 4 0.001% 100.126% ** Level 4 read latency histogram (micros): Count: 30942 Average: 333.8322 StdDev: 598.12 Min: 1 Median: 176.0234 Max: 9719 Percentiles: P50: 176.02 P75: 249.28 P99: 3648.04 P99.9: 6493.72 P99.99: 9491.57 ------------------------------------------------------ [ 0, 1 ] 2 0.006% 0.006% ( 1, 2 ] 15 0.048% 0.055% ( 2, 3 ] 71 0.229% 0.284% ( 3, 4 ] 38 0.123% 0.407% ( 4, 6 ] 74 0.239% 0.646% ( 6, 10 ] 12 0.039% 0.685% ( 10, 15 ] 5 0.016% 0.701% ( 76, 110 ] 1499 4.845% 5.546% # ( 110, 170 ] 13119 42.399% 47.945% ######## ( 170, 250 ] 8447 27.299% 75.244% ##### ( 250, 380 ] 2682 8.668% 83.912% ## ( 380, 580 ] 2018 6.522% 90.434% # ( 580, 870 ] 1094 3.536% 93.969% # ( 870, 1300 ] 658 2.127% 96.096% ( 1300, 1900 ] 411 1.328% 97.424% ( 1900, 2900 ] 327 1.057% 98.481% ( 2900, 4400 ] 322 1.041% 99.522% ( 4400, 6600 ] 123 0.398% 99.919% ( 6600, 9900 ] 25 0.081% 100.000% ** Level 5 read latency histogram (micros): Count: 269004 Average: 339.4143 StdDev: 615.67 Min: 77 Median: 176.6658 Max: 32901 Percentiles: P50: 176.67 P75: 250.98 P99: 3672.31 P99.9: 6464.39 P99.99: 9657.29 ------------------------------------------------------ ( 76, 110 ] 13219 4.914% 4.914% # ( 110, 170 ] 115186 42.819% 47.733% ######### ( 170, 250 ] 73173 27.201% 74.935% ##### ( 250, 380 ] 23204 8.626% 83.561% ## ( 380, 580 ] 17766 6.604% 90.165% # ( 580, 870 ] 9814 3.648% 93.813% # ( 870, 1300 ] 5663 2.105% 95.919% ( 1300, 1900 ] 3712 1.380% 97.299% ( 1900, 2900 ] 3090 1.149% 98.447% ( 2900, 4400 ] 2888 1.074% 99.521% ( 4400, 6600 ] 1087 0.404% 99.925% ( 6600, 9900 ] 189 0.070% 99.995% ( 9900, 14000 ] 12 0.004% 100.000% ( 14000, 22000 ] 7 0.003% 100.002% ( 22000, 33000 ] 3 0.001% 100.003% ** DB Stats ** Uptime(secs): 33.6 total, 7.6 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 146.59 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:15.828 H:M:S, 47.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 129.74 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.992 H:M:S, 52.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 286.984 micros/op 13919 ops/sec 60.204 seconds 837996 operations; 13.5 MB/s (211999 of 211999 found) Microseconds per read: Count: 837996 Average: 286.9870 StdDev: 797.22 Min: 2 Median: 205.3234 Max: 53817 Percentiles: P50: 205.32 P75: 248.25 P99: 1707.09 P99.9: 14224.36 P99.99: 31039.31 ------------------------------------------------------ ( 1, 2 ] 9 0.001% 0.001% ( 2, 3 ] 28 0.003% 0.004% ( 3, 4 ] 45 0.005% 0.010% ( 4, 6 ] 81 0.010% 0.019% ( 6, 10 ] 285 0.034% 0.053% ( 10, 15 ] 3433 0.410% 0.463% ( 15, 22 ] 3771 0.450% 0.913% ( 22, 34 ] 9970 1.190% 2.103% ( 34, 51 ] 2990 0.357% 2.460% ( 51, 76 ] 413 0.049% 2.509% ( 76, 110 ] 2208 0.263% 2.772% ( 110, 170 ] 223357 26.654% 29.426% ##### ( 170, 250 ] 390467 46.595% 76.021% ######### ( 250, 380 ] 109003 13.008% 89.029% ### ( 380, 580 ] 53313 6.362% 95.391% # ( 580, 870 ] 19559 2.334% 97.725% ( 870, 1300 ] 8080 0.964% 98.689% ( 1300, 1900 ] 3838 0.458% 99.147% ( 1900, 2900 ] 2791 0.333% 99.480% ( 2900, 4400 ] 2385 0.285% 99.765% ( 4400, 6600 ] 659 0.079% 99.844% ( 6600, 9900 ] 207 0.025% 99.868% ( 9900, 14000 ] 251 0.030% 99.898% ( 14000, 22000 ] 535 0.064% 99.962% ( 22000, 33000 ] 285 0.034% 99.996% ( 33000, 50000 ] 29 0.003% 100.000% ( 50000, 75000 ] 4 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 16200691 rocksdb.bloom.filter.full.positive COUNT : 995477 rocksdb.bloom.filter.full.true.positive COUNT : 837612 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 384 rocksdb.memtable.miss COUNT : 837612 rocksdb.l0.hit COUNT : 34243 rocksdb.l1.hit COUNT : 4001 rocksdb.l2andup.hit COUNT : 799368 rocksdb.compaction.key.drop.new COUNT : 89212 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 5420707 rocksdb.number.keys.read COUNT : 837996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 5594169624 rocksdb.bytes.read COUNT : 837996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1621 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 40729801 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 22877319604 rocksdb.compact.write.bytes COUNT : 22779754093 rocksdb.flush.write.bytes COUNT : 5548998817 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 33 rocksdb.number.superversion_acquires COUNT : 1233 rocksdb.number.superversion_releases COUNT : 1193 rocksdb.number.superversion_cleanups COUNT : 303 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 279 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 5504050176 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1969152 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 26085551418 rocksdb.non.last.level.read.count COUNT : 6162946 rocksdb.block.checksum.compute.count COUNT : 6546680 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 204.460174 P95 : 567.123791 P99 : 1706.160104 P100 : 53814.000000 COUNT : 837996 SUM : 239728976 rocksdb.db.write.micros P50 : 1.489216 P95 : 3.943728 P99 : 8.960510 P100 : 594070.000000 COUNT : 5420707 SUM : 53521396 rocksdb.compaction.times.micros P50 : 1329629.629630 P95 : 4020000.000000 P99 : 10304000.000000 P100 : 12461346.000000 COUNT : 88 SUM : 155740797 rocksdb.compaction.times.cpu_micros P50 : 976875.000000 P95 : 1885263.157895 P99 : 4564000.000000 P100 : 4697017.000000 COUNT : 88 SUM : 93436605 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 843.289474 P95 : 21600.000000 P99 : 81650.000000 P100 : 83536.000000 COUNT : 81 SUM : 267504 rocksdb.compaction.outfile.sync.micros P50 : 7129.245283 P95 : 102750.000000 P99 : 205951.000000 P100 : 205951.000000 COUNT : 309 SUM : 5936357 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 238.444444 P95 : 2558.333333 P99 : 4355.000000 P100 : 8635.000000 COUNT : 221 SUM : 140772 rocksdb.table.open.io.micros P50 : 10040.482234 P95 : 21157.948718 P99 : 28710.000000 P100 : 38054.000000 COUNT : 1621 SUM : 16090884 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.173313 P95 : 12.463627 P99 : 231.117459 P100 : 30077.000000 COUNT : 5539097 SUM : 104163781 rocksdb.read.block.get.micros P50 : 154.855941 P95 : 451.156046 P99 : 1612.894556 P100 : 32906.000000 COUNT : 1005936 SUM : 206003802 rocksdb.write.raw.block.micros P50 : 0.537922 P95 : 1.417934 P99 : 3.905482 P100 : 38731.000000 COUNT : 6861640 SUM : 17997197 rocksdb.numfiles.in.singlecompaction P50 : 3.030769 P95 : 4.000000 P99 : 13.000000 P100 : 23.000000 COUNT : 140 SUM : 510 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.561846 P95 : 1260.435624 P99 : 2702.284385 P100 : 593888.000000 COUNT : 237691 SUM : 40746198 rocksdb.sst.read.micros P50 : 2.277870 P95 : 203.308088 P99 : 542.690892 P100 : 32901.000000 COUNT : 6162958 SUM : 288892814 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 837996 SUM : 837996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 5420707 SUM : 5594169624 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 215633.802817 P95 : 566833.333333 P99 : 924600.000000 P100 : 926266.000000 COUNT : 81 SUM : 19681874 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 318 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0