[2024-12-16 09:26:44.526956] Starting SPDK v25.01-pre git sha1 e01cb43b8 / DPDK 23.11.0 initialization... [2024-12-16 09:26:44.527224] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid448134 ] Set seed to 1734337604520200 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-16 09:26:44.659730] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-16 09:26:44.683109] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-16 09:26:44.683158] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 448134' to capture a snapshot of events at runtime. [2024-12-16 09:26:44.683172] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-16 09:26:44.683185] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-16 09:26:44.683196] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid448134 for offline analysis/debug. [2024-12-16 09:26:44.683791] reactor.c: 995:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Dec 16 09:26:48 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/16-09:26:56 ... thread 0: (1000000,1000000) ops and (128431.6,128431.6) ops/second in (7.786248,7.786248) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/11 2.24 GB 44.0 0.4 0.0 0.4 1.2 0.8 0.0 1.5 38.4 121.6 9.84 4.09 10 0.984 383K 514 0.0 0.0 L1 11/11 1.27 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/1 128.78 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 20/6 1.35 GB 1.0 2.6 1.5 1.2 2.6 1.5 0.0 1.8 105.5 104.8 25.55 13.01 12 2.129 839K 1410 0.0 0.0 L4 163/7 10.10 GB 1.0 3.9 1.9 2.0 3.8 1.8 0.1 2.0 110.0 107.5 36.61 19.69 28 1.308 1582K 0 0.0 0.0 L5 1120/0 92.15 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1328/36 107.24 GB 0.0 6.9 3.4 3.6 7.6 4.1 1.3 9.5 98.6 108.5 72.00 36.79 50 1.440 2806K 1924 0.0 0.0 Int 0/0 0.00 KB 0.0 6.9 3.4 3.6 7.6 4.1 1.3 9.5 98.6 108.5 72.00 36.79 50 1.440 2806K 1924 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.9 3.4 3.6 6.8 3.3 0.0 0.0 109.2 107.6 65.01 34.43 41 1.586 2806K 1924 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 117.2 6.99 2.36 9 0.776 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.0 total, 7.8 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.63 GB write, 978.24 MB/s write, 6.93 GB read, 889.03 MB/s read, 72.0 seconds Interval compaction: 7.63 GB write, 1002.92 MB/s write, 6.93 GB read, 911.46 MB/s read, 72.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 31, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 31, interval: 31 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 195919 Average: 30.2121 StdDev: 351.13 Min: 1 Median: 1.9567 Max: 79990 Percentiles: P50: 1.96 P75: 3.00 P99: 924.12 P99.9: 3943.02 P99.99: 6352.10 ------------------------------------------------------ [ 0, 1 ] 6425 3.279% 3.279% # ( 1, 2 ] 95678 48.835% 52.115% ########## ( 2, 3 ] 44859 22.897% 75.012% ##### ( 3, 4 ] 19146 9.772% 84.784% ## ( 4, 6 ] 19973 10.195% 94.979% ## ( 6, 10 ] 5021 2.563% 97.541% # ( 10, 15 ] 910 0.464% 98.006% ( 15, 22 ] 105 0.054% 98.059% ( 22, 34 ] 13 0.007% 98.066% ( 34, 51 ] 10 0.005% 98.071% ( 51, 76 ] 16 0.008% 98.079% ( 76, 110 ] 20 0.010% 98.090% ( 110, 170 ] 49 0.025% 98.115% ( 170, 250 ] 102 0.052% 98.167% ( 250, 380 ] 259 0.132% 98.299% ( 380, 580 ] 520 0.265% 98.564% ( 580, 870 ] 753 0.384% 98.949% ( 870, 1300 ] 801 0.409% 99.357% ( 1300, 1900 ] 513 0.262% 99.619% ( 1900, 2900 ] 338 0.173% 99.792% ( 2900, 4400 ] 305 0.156% 99.947% ( 4400, 6600 ] 94 0.048% 99.995% ( 6600, 9900 ] 14 0.007% 100.003% ( 9900, 14000 ] 3 0.002% 100.004% ( 14000, 22000 ] 2 0.001% 100.005% ( 22000, 33000 ] 4 0.002% 100.007% ( 33000, 50000 ] 2 0.001% 100.008% ( 75000, 110000 ] 1 0.001% 100.009% ** Level 1 read latency histogram (micros): Count: 29286 Average: 23.7957 StdDev: 406.35 Min: 1 Median: 2.3337 Max: 33727 Percentiles: P50: 2.33 P75: 3.64 P99: 20.42 P99.9: 4588.54 P99.99: 19523.73 ------------------------------------------------------ [ 0, 1 ] 312 1.065% 1.065% ( 1, 2 ] 11789 40.255% 41.320% ######## ( 2, 3 ] 7617 26.009% 67.329% ##### ( 3, 4 ] 3500 11.951% 79.280% ## ( 4, 6 ] 4180 14.273% 93.553% ### ( 6, 10 ] 1321 4.511% 98.064% # ( 10, 15 ] 237 0.809% 98.873% ( 15, 22 ] 48 0.164% 99.037% ( 22, 34 ] 5 0.017% 99.054% ( 51, 76 ] 1 0.003% 99.058% ( 76, 110 ] 2 0.007% 99.064% ( 110, 170 ] 5 0.017% 99.081% ( 170, 250 ] 13 0.044% 99.126% ( 250, 380 ] 12 0.041% 99.167% ( 380, 580 ] 29 0.099% 99.266% ( 580, 870 ] 47 0.160% 99.426% ( 870, 1300 ] 48 0.164% 99.590% ( 1300, 1900 ] 32 0.109% 99.700% ( 1900, 2900 ] 31 0.106% 99.805% ( 2900, 4400 ] 26 0.089% 99.894% ( 4400, 6600 ] 20 0.068% 99.962% ( 6600, 9900 ] 4 0.014% 99.976% ( 9900, 14000 ] 2 0.007% 99.983% ( 14000, 22000 ] 3 0.010% 99.993% ( 22000, 33000 ] 1 0.003% 99.997% ( 33000, 50000 ] 1 0.003% 100.000% ** Level 2 read latency histogram (micros): Count: 169648 Average: 14.1406 StdDev: 172.24 Min: 1 Median: 2.0473 Max: 7844 Percentiles: P50: 2.05 P75: 3.11 P99: 12.59 P99.9: 2219.22 P99.99: 3304.95 ------------------------------------------------------ [ 0, 1 ] 2445 1.441% 1.441% ( 1, 2 ] 80372 47.376% 48.817% ######### ( 2, 3 ] 42392 24.988% 73.805% ##### ( 3, 4 ] 17861 10.528% 84.333% ## ( 4, 6 ] 19193 11.313% 95.647% ## ( 6, 10 ] 5143 3.032% 98.678% # ( 10, 15 ] 1054 0.621% 99.300% ( 15, 22 ] 157 0.093% 99.392% ( 22, 34 ] 21 0.012% 99.405% ( 34, 51 ] 11 0.006% 99.411% ( 51, 76 ] 24 0.014% 99.425% ( 76, 110 ] 5 0.003% 99.428% ( 110, 170 ] 17 0.010% 99.438% ( 170, 250 ] 18 0.011% 99.449% ( 250, 380 ] 51 0.030% 99.479% ( 380, 580 ] 102 0.060% 99.539% ( 580, 870 ] 178 0.105% 99.644% ( 870, 1300 ] 209 0.123% 99.767% ( 1300, 1900 ] 173 0.102% 99.869% ( 1900, 2900 ] 164 0.097% 99.966% ( 2900, 4400 ] 152 0.090% 100.055% ( 4400, 6600 ] 44 0.026% 100.081% ( 6600, 9900 ] 5 0.003% 100.084% ** Level 3 read latency histogram (micros): Count: 684567 Average: 9.3861 StdDev: 169.14 Min: 1 Median: 1.9902 Max: 27978 Percentiles: P50: 1.99 P75: 2.99 P99: 9.62 P99.9: 779.92 P99.99: 1233.61 ------------------------------------------------------ [ 0, 1 ] 12461 1.820% 1.820% ( 1, 2 ] 333081 48.656% 50.476% ########## ( 2, 3 ] 168948 24.680% 75.156% ##### ( 3, 4 ] 70842 10.348% 85.504% ## ( 4, 6 ] 77341 11.298% 96.802% ## ( 6, 10 ] 16616 2.427% 99.229% ( 10, 15 ] 3144 0.459% 99.688% ( 15, 22 ] 422 0.062% 99.750% ( 22, 34 ] 53 0.008% 99.758% ( 34, 51 ] 43 0.006% 99.764% ( 51, 76 ] 54 0.008% 99.772% ( 76, 110 ] 17 0.002% 99.774% ( 110, 170 ] 37 0.005% 99.780% ( 170, 250 ] 90 0.013% 99.793% ( 250, 380 ] 130 0.019% 99.812% ( 380, 580 ] 287 0.042% 99.854% ( 580, 870 ] 459 0.067% 99.921% ( 870, 1300 ] 560 0.082% 100.003% ( 1300, 1900 ] 365 0.053% 100.056% ( 1900, 2900 ] 245 0.036% 100.092% ( 2900, 4400 ] 259 0.038% 100.130% ( 4400, 6600 ] 102 0.015% 100.144% ( 6600, 9900 ] 23 0.003% 100.148% ( 9900, 14000 ] 4 0.001% 100.148% ( 14000, 22000 ] 17 0.002% 100.151% ( 22000, 33000 ] 5 0.001% 100.152% ** Level 4 read latency histogram (micros): Count: 555459 Average: 19.7007 StdDev: 288.99 Min: 1 Median: 1.9879 Max: 52223 Percentiles: P50: 1.99 P75: 3.06 P99: 14.66 P99.9: 2338.68 P99.99: 3371.35 ------------------------------------------------------ [ 0, 1 ] 11262 2.028% 2.028% ( 1, 2 ] 269721 48.558% 50.586% ########## ( 2, 3 ] 132060 23.775% 74.361% ##### ( 3, 4 ] 56357 10.146% 84.507% ## ( 4, 6 ] 61039 10.989% 95.496% ## ( 6, 10 ] 16835 3.031% 98.526% # ( 10, 15 ] 2824 0.508% 99.035% ( 15, 22 ] 402 0.072% 99.107% ( 22, 34 ] 50 0.009% 99.116% ( 34, 51 ] 40 0.007% 99.123% ( 51, 76 ] 57 0.010% 99.134% ( 76, 110 ] 25 0.005% 99.138% ( 110, 170 ] 74 0.013% 99.152% ( 170, 250 ] 168 0.030% 99.182% ( 250, 380 ] 290 0.052% 99.234% ( 380, 580 ] 619 0.111% 99.345% ( 580, 870 ] 991 0.178% 99.524% ( 870, 1300 ] 1084 0.195% 99.719% ( 1300, 1900 ] 760 0.137% 99.856% ( 1900, 2900 ] 562 0.101% 99.957% ( 2900, 4400 ] 587 0.106% 100.063% ( 4400, 6600 ] 206 0.037% 100.100% ( 6600, 9900 ] 50 0.009% 100.109% ( 9900, 14000 ] 7 0.001% 100.110% ( 14000, 22000 ] 18 0.003% 100.113% ( 22000, 33000 ] 16 0.003% 100.116% ( 33000, 50000 ] 3 0.001% 100.117% ( 50000, 75000 ] 1 0.000% 100.117% ** Level 5 read latency histogram (micros): Count: 12523 Average: 1805.0981 StdDev: 2241.85 Min: 82 Median: 1173.9369 Max: 66420 Percentiles: P50: 1173.94 P75: 2415.16 P99: 7990.21 P99.9: 31711.31 P99.99: 59346.25 ------------------------------------------------------ ( 76, 110 ] 10 0.080% 0.080% ( 110, 170 ] 92 0.735% 0.815% ( 170, 250 ] 248 1.980% 2.795% ( 250, 380 ] 650 5.190% 7.985% # ( 380, 580 ] 1372 10.956% 18.941% ## ( 580, 870 ] 2244 17.919% 36.860% #### ( 870, 1300 ] 2328 18.590% 55.450% #### ( 1300, 1900 ] 1726 13.783% 69.233% ### ( 1900, 2900 ] 1402 11.195% 80.428% ## ( 2900, 4400 ] 1529 12.210% 92.638% ## ( 4400, 6600 ] 734 5.861% 98.499% # ( 6600, 9900 ] 149 1.190% 99.689% ( 9900, 14000 ] 10 0.080% 99.768% ( 14000, 22000 ] 5 0.040% 99.808% ( 22000, 33000 ] 13 0.104% 99.912% ( 33000, 50000 ] 9 0.072% 99.984% ( 50000, 75000 ] 2 0.016% 100.000% ** DB Stats ** Uptime(secs): 8.0 total, 7.8 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 123.26 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.809 H:M:S, 47.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 126.35 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.809 H:M:S, 48.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 7 num-running-flushes: 1 2024/12/16-09:27:00 ... thread 0: (1000000,2000000) ops and (260200.2,171977.3) ops/second in (3.843194,11.629442) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/11 3.22 GB 157.3 1.5 0.0 1.5 3.4 1.8 0.0 1.8 84.2 185.0 18.71 11.87 30 0.624 1599K 1862 0.0 0.0 L1 11/11 1.27 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 14/0 961.53 MB 1.0 2.9 1.6 1.3 2.9 1.6 0.0 1.8 109.2 108.5 27.08 14.24 13 2.083 975K 1410 0.0 0.0 L4 161/0 9.63 GB 1.0 4.9 2.4 2.5 4.8 2.3 0.1 2.0 118.0 115.4 42.39 24.15 35 1.211 1984K 0 0.0 0.0 L5 1132/0 93.14 GB 1.0 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1334/22 108.20 GB 0.0 9.3 4.0 5.3 11.0 5.7 2.4 6.0 108.1 128.1 88.19 50.26 78 1.131 4559K 3272 0.0 0.0 Int 0/0 0.00 KB 0.0 9.3 4.0 5.3 11.0 5.7 2.4 6.0 108.1 128.1 88.19 50.26 78 1.131 4559K 3272 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.3 4.0 5.3 9.2 3.9 0.0 0.0 123.1 121.4 77.46 44.78 53 1.462 4559K 3272 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 176.2 10.73 5.48 25 0.429 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 11.8 total, 11.6 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.03 GB write, 954.92 MB/s write, 9.31 GB read, 806.20 MB/s read, 88.2 seconds Interval compaction: 11.03 GB write, 971.05 MB/s write, 9.31 GB read, 819.81 MB/s read, 88.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 36, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 36, interval: 36 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 521307 Average: 20.8265 StdDev: 226.96 Min: 0 Median: 1.7120 Max: 79990 Percentiles: P50: 1.71 P75: 2.55 P99: 464.41 P99.9: 2125.68 P99.99: 3656.28 ------------------------------------------------------ [ 0, 1 ] 56349 10.809% 10.809% ## ( 1, 2 ] 286940 55.042% 65.852% ########### ( 2, 3 ] 86611 16.614% 82.466% ### ( 3, 4 ] 33358 6.399% 88.865% # ( 4, 6 ] 30941 5.935% 94.800% # ( 6, 10 ] 7169 1.375% 96.175% ( 10, 15 ] 1097 0.210% 96.386% ( 15, 22 ] 141 0.027% 96.413% ( 22, 34 ] 17 0.003% 96.416% ( 34, 51 ] 15 0.003% 96.419% ( 51, 76 ] 19 0.004% 96.422% ( 76, 110 ] 1184 0.227% 96.650% ( 110, 170 ] 5512 1.057% 97.707% ( 170, 250 ] 3501 0.672% 98.378% ( 250, 380 ] 2252 0.432% 98.810% ( 380, 580 ] 2343 0.449% 99.260% ( 580, 870 ] 1635 0.314% 99.574% ( 870, 1300 ] 1045 0.200% 99.774% ( 1300, 1900 ] 569 0.109% 99.883% ( 1900, 2900 ] 393 0.075% 99.959% ( 2900, 4400 ] 327 0.063% 100.021% ( 4400, 6600 ] 96 0.018% 100.040% ( 6600, 9900 ] 19 0.004% 100.043% ( 9900, 14000 ] 4 0.001% 100.044% ( 14000, 22000 ] 2 0.000% 100.045% ( 22000, 33000 ] 4 0.001% 100.045% ( 33000, 50000 ] 2 0.000% 100.046% ( 75000, 110000 ] 1 0.000% 100.046% ** Level 1 read latency histogram (micros): Count: 52273 Average: 19.0895 StdDev: 307.98 Min: 1 Median: 2.9897 Max: 33727 Percentiles: P50: 2.99 P75: 4.82 P99: 237.85 P99.9: 3230.40 P99.99: 13534.03 ------------------------------------------------------ [ 0, 1 ] 378 0.723% 0.723% ( 1, 2 ] 14338 27.429% 28.152% ##### ( 2, 3 ] 11539 22.074% 50.227% #### ( 3, 4 ] 7726 14.780% 65.007% ### ( 4, 6 ] 12735 24.362% 89.369% ##### ( 6, 10 ] 3637 6.958% 96.327% # ( 10, 15 ] 604 1.155% 97.482% ( 15, 22 ] 262 0.501% 97.984% ( 22, 34 ] 32 0.061% 98.045% ( 34, 51 ] 4 0.008% 98.053% ( 51, 76 ] 5 0.010% 98.062% ( 76, 110 ] 69 0.132% 98.194% ( 110, 170 ] 273 0.522% 98.716% ( 170, 250 ] 176 0.337% 99.053% ( 250, 380 ] 122 0.233% 99.286% ( 380, 580 ] 108 0.207% 99.493% ( 580, 870 ] 87 0.166% 99.659% ( 870, 1300 ] 56 0.107% 99.767% ( 1300, 1900 ] 32 0.061% 99.828% ( 1900, 2900 ] 33 0.063% 99.891% ( 2900, 4400 ] 26 0.050% 99.941% ( 4400, 6600 ] 21 0.040% 99.981% ( 6600, 9900 ] 4 0.008% 99.989% ( 9900, 14000 ] 2 0.004% 99.992% ( 14000, 22000 ] 3 0.006% 99.998% ( 22000, 33000 ] 1 0.002% 100.000% ( 33000, 50000 ] 1 0.002% 100.002% ** Level 2 read latency histogram (micros): Count: 196000 Average: 12.6538 StdDev: 160.57 Min: 1 Median: 1.9962 Max: 7844 Percentiles: P50: 2.00 P75: 3.00 P99: 11.66 P99.9: 2064.63 P99.99: 3286.27 ------------------------------------------------------ [ 0, 1 ] 3081 1.572% 1.572% ( 1, 2 ] 95285 48.615% 50.187% ########## ( 2, 3 ] 48593 24.792% 74.979% ##### ( 3, 4 ] 19806 10.105% 85.084% ## ( 4, 6 ] 21113 10.772% 95.856% ## ( 6, 10 ] 5790 2.954% 98.810% # ( 10, 15 ] 1119 0.571% 99.381% ( 15, 22 ] 166 0.085% 99.466% ( 22, 34 ] 21 0.011% 99.477% ( 34, 51 ] 14 0.007% 99.484% ( 51, 76 ] 24 0.012% 99.496% ( 76, 110 ] 5 0.003% 99.498% ( 110, 170 ] 18 0.009% 99.508% ( 170, 250 ] 22 0.011% 99.519% ( 250, 380 ] 54 0.028% 99.546% ( 380, 580 ] 104 0.053% 99.599% ( 580, 870 ] 180 0.092% 99.691% ( 870, 1300 ] 209 0.107% 99.798% ( 1300, 1900 ] 173 0.088% 99.886% ( 1900, 2900 ] 164 0.084% 99.970% ( 2900, 4400 ] 153 0.078% 100.048% ( 4400, 6600 ] 44 0.022% 100.070% ( 6600, 9900 ] 5 0.003% 100.073% ** Level 3 read latency histogram (micros): Count: 804482 Average: 8.6683 StdDev: 158.28 Min: 1 Median: 1.9556 Max: 27978 Percentiles: P50: 1.96 P75: 2.94 P99: 9.58 P99.9: 717.52 P99.99: 1210.70 ------------------------------------------------------ [ 0, 1 ] 15594 1.938% 1.938% ( 1, 2 ] 404611 50.295% 52.233% ########## ( 2, 3 ] 195160 24.259% 76.492% ##### ( 3, 4 ] 79764 9.915% 86.407% ## ( 4, 6 ] 84715 10.530% 96.937% ## ( 6, 10 ] 18518 2.302% 99.239% ( 10, 15 ] 3279 0.408% 99.647% ( 15, 22 ] 441 0.055% 99.702% ( 22, 34 ] 54 0.007% 99.708% ( 34, 51 ] 45 0.006% 99.714% ( 51, 76 ] 58 0.007% 99.721% ( 76, 110 ] 80 0.010% 99.731% ( 110, 170 ] 304 0.038% 99.769% ( 170, 250 ] 242 0.030% 99.799% ( 250, 380 ] 210 0.026% 99.825% ( 380, 580 ] 364 0.045% 99.870% ( 580, 870 ] 503 0.063% 99.933% ( 870, 1300 ] 580 0.072% 100.005% ( 1300, 1900 ] 367 0.046% 100.051% ( 1900, 2900 ] 247 0.031% 100.081% ( 2900, 4400 ] 259 0.032% 100.113% ( 4400, 6600 ] 102 0.013% 100.126% ( 6600, 9900 ] 23 0.003% 100.129% ( 9900, 14000 ] 6 0.001% 100.130% ( 14000, 22000 ] 18 0.002% 100.132% ( 22000, 33000 ] 5 0.001% 100.133% ** Level 4 read latency histogram (micros): Count: 640273 Average: 19.4681 StdDev: 271.60 Min: 1 Median: 1.9748 Max: 52223 Percentiles: P50: 1.97 P75: 3.03 P99: 224.12 P99.9: 2142.95 P99.99: 3261.74 ------------------------------------------------------ [ 0, 1 ] 13286 2.075% 2.075% ( 1, 2 ] 314778 49.163% 51.238% ########## ( 2, 3 ] 150328 23.479% 74.717% ##### ( 3, 4 ] 62834 9.814% 84.531% ## ( 4, 6 ] 67195 10.495% 95.025% ## ( 6, 10 ] 18751 2.929% 97.954% # ( 10, 15 ] 3020 0.472% 98.426% ( 15, 22 ] 436 0.068% 98.494% ( 22, 34 ] 56 0.009% 98.502% ( 34, 51 ] 40 0.006% 98.509% ( 51, 76 ] 59 0.009% 98.518% ( 76, 110 ] 451 0.070% 98.588% ( 110, 170 ] 1861 0.291% 98.879% ( 170, 250 ] 1146 0.179% 99.058% ( 250, 380 ] 894 0.140% 99.198% ( 380, 580 ] 1142 0.178% 99.376% ( 580, 870 ] 1270 0.198% 99.574% ( 870, 1300 ] 1173 0.183% 99.757% ( 1300, 1900 ] 774 0.121% 99.878% ( 1900, 2900 ] 571 0.089% 99.968% ( 2900, 4400 ] 597 0.093% 100.061% ( 4400, 6600 ] 206 0.032% 100.093% ( 6600, 9900 ] 50 0.008% 100.101% ( 9900, 14000 ] 8 0.001% 100.102% ( 14000, 22000 ] 18 0.003% 100.105% ( 22000, 33000 ] 16 0.002% 100.107% ( 33000, 50000 ] 3 0.000% 100.108% ( 50000, 75000 ] 1 0.000% 100.108% ** Level 5 read latency histogram (micros): Count: 49377 Average: 657.6884 StdDev: 1338.26 Min: 74 Median: 239.2070 Max: 66420 Percentiles: P50: 239.21 P75: 629.30 P99: 5700.96 P99.9: 9738.78 P99.99: 40673.23 ------------------------------------------------------ ( 51, 76 ] 2 0.004% 0.004% ( 76, 110 ] 3347 6.778% 6.783% # ( 110, 170 ] 14271 28.902% 35.685% ###### ( 170, 250 ] 8173 16.552% 52.237% ### ( 250, 380 ] 5229 10.590% 62.827% ## ( 380, 580 ] 5279 10.691% 73.518% ## ( 580, 870 ] 4316 8.741% 82.259% ## ( 870, 1300 ] 2917 5.908% 88.167% # ( 1300, 1900 ] 1861 3.769% 91.936% # ( 1900, 2900 ] 1482 3.001% 94.937% # ( 2900, 4400 ] 1573 3.186% 98.123% # ( 4400, 6600 ] 736 1.491% 99.613% ( 6600, 9900 ] 151 0.306% 99.919% ( 9900, 14000 ] 13 0.026% 99.945% ( 14000, 22000 ] 7 0.014% 99.959% ( 22000, 33000 ] 13 0.026% 99.986% ( 33000, 50000 ] 9 0.018% 100.004% ( 50000, 75000 ] 2 0.004% 100.008% ** DB Stats ** Uptime(secs): 11.8 total, 3.8 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 166.42 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.031 H:M:S, 34.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 256.10 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.222 H:M:S, 5.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/16-09:27:08 ... thread 0: (1000000,3000000) ops and (133116.1,156726.0) ops/second in (7.512242,19.141684) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/14 4.27 GB 132.1 2.8 0.0 2.8 5.6 2.8 0.0 2.0 105.7 213.5 26.83 19.49 51 0.526 2878K 3126 0.0 0.0 L1 11/11 1.27 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 14/0 961.53 MB 1.0 2.9 1.6 1.3 2.9 1.6 0.0 1.8 109.2 108.5 27.08 14.24 13 2.083 975K 1410 0.0 0.0 L4 161/0 9.63 GB 1.0 4.9 2.4 2.5 4.8 2.3 0.1 2.0 118.0 115.4 42.39 24.15 35 1.211 1984K 0 0.0 0.0 L5 1132/0 93.14 GB 1.0 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1336/25 109.24 GB 0.0 10.5 4.0 6.6 13.2 6.7 2.4 4.7 112.1 140.8 96.30 57.87 99 0.973 5838K 4536 0.0 0.0 Int 0/0 0.00 KB 0.0 10.5 4.0 6.6 13.2 6.7 2.4 4.7 112.1 140.8 96.30 57.87 99 0.973 5838K 4536 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.5 4.0 6.6 10.4 3.8 0.0 0.0 131.2 129.6 82.30 49.54 58 1.419 5838K 4536 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 207.0 14.00 8.33 41 0.342 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 19.3 total, 19.1 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.24 GB write, 701.20 MB/s write, 10.54 GB read, 558.19 MB/s read, 96.3 seconds Interval compaction: 13.24 GB write, 708.40 MB/s write, 10.54 GB read, 563.92 MB/s read, 96.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 8, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 8, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 36, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 8, total-stops: 36, interval: 44 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1115684 Average: 17.2332 StdDev: 160.80 Min: 0 Median: 1.5778 Max: 79990 Percentiles: P50: 1.58 P75: 2.13 P99: 312.07 P99.9: 1179.80 P99.99: 2156.52 ------------------------------------------------------ [ 0, 1 ] 200477 17.969% 17.969% #### ( 1, 2 ] 618503 55.437% 73.406% ########### ( 2, 3 ] 135784 12.170% 85.577% ## ( 3, 4 ] 49004 4.392% 89.969% # ( 4, 6 ] 43823 3.928% 93.897% # ( 6, 10 ] 9219 0.826% 94.723% ( 10, 15 ] 1561 0.140% 94.863% ( 15, 22 ] 190 0.017% 94.880% ( 22, 34 ] 17 0.002% 94.882% ( 34, 51 ] 17 0.002% 94.883% ( 51, 76 ] 22 0.002% 94.885% ( 76, 110 ] 6573 0.589% 95.474% ( 110, 170 ] 25909 2.322% 97.796% ( 170, 250 ] 11061 0.991% 98.788% ( 250, 380 ] 4958 0.444% 99.232% ( 380, 580 ] 4362 0.391% 99.623% ( 580, 870 ] 2285 0.205% 99.828% ( 870, 1300 ] 1115 0.100% 99.928% ( 1300, 1900 ] 587 0.053% 99.981% ( 1900, 2900 ] 411 0.037% 100.017% ( 2900, 4400 ] 338 0.030% 100.048% ( 4400, 6600 ] 96 0.009% 100.056% ( 6600, 9900 ] 19 0.002% 100.058% ( 9900, 14000 ] 4 0.000% 100.058% ( 14000, 22000 ] 2 0.000% 100.059% ( 22000, 33000 ] 4 0.000% 100.059% ( 33000, 50000 ] 2 0.000% 100.059% ( 75000, 110000 ] 1 0.000% 100.059% ** Level 1 read latency histogram (micros): Count: 107475 Average: 15.0083 StdDev: 216.43 Min: 1 Median: 4.1992 Max: 33727 Percentiles: P50: 4.20 P75: 5.43 P99: 197.48 P99.9: 1591.09 P99.99: 6808.31 ------------------------------------------------------ [ 0, 1 ] 415 0.386% 0.386% ( 1, 2 ] 15073 14.025% 14.411% ### ( 2, 3 ] 13733 12.778% 27.189% ### ( 3, 4 ] 20183 18.779% 45.968% #### ( 4, 6 ] 43500 40.475% 86.442% ######## ( 6, 10 ] 9093 8.461% 94.903% ## ( 10, 15 ] 1488 1.385% 96.288% ( 15, 22 ] 1035 0.963% 97.251% ( 22, 34 ] 81 0.075% 97.326% ( 34, 51 ] 4 0.004% 97.330% ( 51, 76 ] 5 0.005% 97.334% ( 76, 110 ] 363 0.338% 97.672% ( 110, 170 ] 1250 1.163% 98.835% ( 170, 250 ] 516 0.480% 99.315% ( 250, 380 ] 253 0.235% 99.551% ( 380, 580 ] 196 0.182% 99.733% ( 580, 870 ] 103 0.096% 99.829% ( 870, 1300 ] 61 0.057% 99.886% ( 1300, 1900 ] 32 0.030% 99.915% ( 1900, 2900 ] 33 0.031% 99.946% ( 2900, 4400 ] 26 0.024% 99.970% ( 4400, 6600 ] 21 0.020% 99.990% ( 6600, 9900 ] 4 0.004% 99.993% ( 9900, 14000 ] 2 0.002% 99.995% ( 14000, 22000 ] 3 0.003% 99.998% ( 22000, 33000 ] 1 0.001% 99.999% ( 33000, 50000 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 196000 Average: 12.6538 StdDev: 160.57 Min: 1 Median: 1.9962 Max: 7844 Percentiles: P50: 2.00 P75: 3.00 P99: 11.66 P99.9: 2064.63 P99.99: 3286.27 ------------------------------------------------------ [ 0, 1 ] 3081 1.572% 1.572% ( 1, 2 ] 95285 48.615% 50.187% ########## ( 2, 3 ] 48593 24.792% 74.979% ##### ( 3, 4 ] 19806 10.105% 85.084% ## ( 4, 6 ] 21113 10.772% 95.856% ## ( 6, 10 ] 5790 2.954% 98.810% # ( 10, 15 ] 1119 0.571% 99.381% ( 15, 22 ] 166 0.085% 99.466% ( 22, 34 ] 21 0.011% 99.477% ( 34, 51 ] 14 0.007% 99.484% ( 51, 76 ] 24 0.012% 99.496% ( 76, 110 ] 5 0.003% 99.498% ( 110, 170 ] 18 0.009% 99.508% ( 170, 250 ] 22 0.011% 99.519% ( 250, 380 ] 54 0.028% 99.546% ( 380, 580 ] 104 0.053% 99.599% ( 580, 870 ] 180 0.092% 99.691% ( 870, 1300 ] 209 0.107% 99.798% ( 1300, 1900 ] 173 0.088% 99.886% ( 1900, 2900 ] 164 0.084% 99.970% ( 2900, 4400 ] 153 0.078% 100.048% ( 4400, 6600 ] 44 0.022% 100.070% ( 6600, 9900 ] 5 0.003% 100.073% ** Level 3 read latency histogram (micros): Count: 806101 Average: 8.9999 StdDev: 158.39 Min: 1 Median: 1.9576 Max: 27978 Percentiles: P50: 1.96 P75: 2.94 P99: 9.91 P99.9: 726.59 P99.99: 1211.19 ------------------------------------------------------ [ 0, 1 ] 15594 1.934% 1.934% ( 1, 2 ] 404612 50.194% 52.128% ########## ( 2, 3 ] 195161 24.210% 76.339% ##### ( 3, 4 ] 79798 9.899% 86.238% ## ( 4, 6 ] 84765 10.515% 96.753% ## ( 6, 10 ] 18525 2.298% 99.051% ( 10, 15 ] 3280 0.407% 99.458% ( 15, 22 ] 441 0.055% 99.513% ( 22, 34 ] 54 0.007% 99.520% ( 34, 51 ] 45 0.006% 99.525% ( 51, 76 ] 58 0.007% 99.533% ( 76, 110 ] 288 0.036% 99.568% ( 110, 170 ] 1141 0.142% 99.710% ( 170, 250 ] 522 0.065% 99.775% ( 250, 380 ] 301 0.037% 99.812% ( 380, 580 ] 442 0.055% 99.867% ( 580, 870 ] 530 0.066% 99.933% ( 870, 1300 ] 584 0.072% 100.005% ( 1300, 1900 ] 367 0.046% 100.050% ( 1900, 2900 ] 247 0.031% 100.081% ( 2900, 4400 ] 259 0.032% 100.113% ( 4400, 6600 ] 102 0.013% 100.126% ( 6600, 9900 ] 23 0.003% 100.129% ( 9900, 14000 ] 6 0.001% 100.130% ( 14000, 22000 ] 18 0.002% 100.132% ( 22000, 33000 ] 5 0.001% 100.132% ** Level 4 read latency histogram (micros): Count: 654659 Average: 22.7252 StdDev: 270.19 Min: 1 Median: 1.9976 Max: 52223 Percentiles: P50: 2.00 P75: 3.20 P99: 341.90 P99.9: 2129.36 P99.99: 3261.94 ------------------------------------------------------ [ 0, 1 ] 13286 2.029% 2.029% ( 1, 2 ] 314791 48.085% 50.114% ########## ( 2, 3 ] 150396 22.973% 73.087% ##### ( 3, 4 ] 63204 9.654% 82.742% ## ( 4, 6 ] 67854 10.365% 93.107% ## ( 6, 10 ] 18805 2.872% 95.979% # ( 10, 15 ] 3036 0.464% 96.443% ( 15, 22 ] 438 0.067% 96.510% ( 22, 34 ] 56 0.009% 96.518% ( 34, 51 ] 40 0.006% 96.524% ( 51, 76 ] 60 0.009% 96.534% ( 76, 110 ] 2445 0.373% 96.907% ( 110, 170 ] 9017 1.377% 98.284% ( 170, 250 ] 3453 0.527% 98.812% ( 250, 380 ] 1742 0.266% 99.078% ( 380, 580 ] 1806 0.276% 99.354% ( 580, 870 ] 1467 0.224% 99.578% ( 870, 1300 ] 1198 0.183% 99.761% ( 1300, 1900 ] 778 0.119% 99.880% ( 1900, 2900 ] 577 0.088% 99.968% ( 2900, 4400 ] 599 0.091% 100.059% ( 4400, 6600 ] 206 0.031% 100.091% ( 6600, 9900 ] 50 0.008% 100.099% ( 9900, 14000 ] 8 0.001% 100.100% ( 14000, 22000 ] 18 0.003% 100.102% ( 22000, 33000 ] 16 0.002% 100.105% ( 33000, 50000 ] 3 0.000% 100.105% ( 50000, 75000 ] 1 0.000% 100.106% ** Level 5 read latency histogram (micros): Count: 148805 Average: 339.3589 StdDev: 809.00 Min: 74 Median: 159.4108 Max: 66420 Percentiles: P50: 159.41 P75: 247.31 P99: 3865.05 P99.9: 7500.29 P99.99: 25485.73 ------------------------------------------------------ ( 51, 76 ] 4 0.003% 0.003% ( 76, 110 ] 18085 12.153% 12.156% ## ( 110, 170 ] 68382 45.954% 58.110% ######### ( 170, 250 ] 26006 17.477% 75.587% ### ( 250, 380 ] 11411 7.668% 83.255% ## ( 380, 580 ] 10150 6.821% 90.076% # ( 580, 870 ] 5740 3.857% 93.934% # ( 870, 1300 ] 3122 2.098% 96.032% ( 1300, 1900 ] 1889 1.269% 97.301% ( 1900, 2900 ] 1514 1.017% 98.319% ( 2900, 4400 ] 1576 1.059% 99.378% ( 4400, 6600 ] 736 0.495% 99.872% ( 6600, 9900 ] 151 0.101% 99.974% ( 9900, 14000 ] 13 0.009% 99.983% ( 14000, 22000 ] 7 0.005% 99.987% ( 22000, 33000 ] 13 0.009% 99.996% ( 33000, 50000 ] 9 0.006% 100.002% ( 50000, 75000 ] 2 0.001% 100.003% ** DB Stats ** Uptime(secs): 19.3 total, 7.5 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 152.66 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.698 H:M:S, 39.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 131.01 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.667 H:M:S, 48.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 234.105 micros/op 17082 ops/sec 60.179 seconds 1027996 operations; 16.6 MB/s (256999 of 256999 found) Microseconds per read: Count: 1027996 Average: 234.1079 StdDev: 787.73 Min: 3 Median: 182.9323 Max: 253527 Percentiles: P50: 182.93 P75: 233.26 P99: 850.14 P99.9: 10807.51 P99.99: 30637.92 ------------------------------------------------------ ( 2, 3 ] 2 0.000% 0.000% ( 3, 4 ] 4 0.000% 0.001% ( 4, 6 ] 93 0.009% 0.010% ( 6, 10 ] 262 0.025% 0.035% ( 10, 15 ] 86 0.008% 0.043% ( 15, 22 ] 437 0.043% 0.086% ( 22, 34 ] 17113 1.665% 1.751% ( 34, 51 ] 3746 0.364% 2.115% ( 51, 76 ] 460 0.045% 2.160% ( 76, 110 ] 428 0.042% 2.201% ( 110, 170 ] 425327 41.374% 43.576% ######## ( 170, 250 ] 408528 39.740% 83.316% ######## ( 250, 380 ] 112690 10.962% 94.278% ## ( 380, 580 ] 37334 3.632% 97.910% # ( 580, 870 ] 12030 1.170% 99.080% ( 870, 1300 ] 4288 0.417% 99.497% ( 1300, 1900 ] 1689 0.164% 99.662% ( 1900, 2900 ] 1047 0.102% 99.763% ( 2900, 4400 ] 790 0.077% 99.840% ( 4400, 6600 ] 344 0.033% 99.874% ( 6600, 9900 ] 187 0.018% 99.892% ( 9900, 14000 ] 375 0.036% 99.928% ( 14000, 22000 ] 484 0.047% 99.975% ( 22000, 33000 ] 190 0.018% 99.994% ( 33000, 50000 ] 51 0.005% 99.999% ( 50000, 75000 ] 8 0.001% 100.000% ( 170000, 250000 ] 2 0.000% 100.000% ( 250000, 380000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 16335673 rocksdb.bloom.filter.full.positive COUNT : 1187541 rocksdb.bloom.filter.full.true.positive COUNT : 1027567 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 429 rocksdb.memtable.miss COUNT : 1027567 rocksdb.l0.hit COUNT : 45897 rocksdb.l1.hit COUNT : 12799 rocksdb.l2andup.hit COUNT : 968871 rocksdb.compaction.key.drop.new COUNT : 164605 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3338581 rocksdb.number.keys.read COUNT : 1027996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3445415592 rocksdb.bytes.read COUNT : 1027996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1507 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 46039506 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 15411046473 rocksdb.compact.write.bytes COUNT : 15241657518 rocksdb.flush.write.bytes COUNT : 3435034439 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 32 rocksdb.number.superversion_acquires COUNT : 681 rocksdb.number.superversion_releases COUNT : 650 rocksdb.number.superversion_cleanups COUNT : 168 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3407264768 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1256448 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 19286049953 rocksdb.non.last.level.read.count COUNT : 4515280 rocksdb.block.checksum.compute.count COUNT : 4932077 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 181.242945 P95 : 417.233858 P99 : 849.343766 P100 : 253525.000000 COUNT : 1027996 SUM : 239444567 rocksdb.db.write.micros P50 : 1.617272 P95 : 5.676520 P99 : 30.955136 P100 : 1051642.000000 COUNT : 3338581 SUM : 55413664 rocksdb.compaction.times.micros P50 : 1200000.000000 P95 : 3366666.666667 P99 : 5180000.000000 P100 : 5894613.000000 COUNT : 60 SUM : 89125722 rocksdb.compaction.times.cpu_micros P50 : 808823.529412 P95 : 1783333.333333 P99 : 5180000.000000 P100 : 5797543.000000 COUNT : 60 SUM : 56258487 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 848.250000 P95 : 66250.000000 P99 : 446985.000000 P100 : 446985.000000 COUNT : 47 SUM : 838484 rocksdb.compaction.outfile.sync.micros P50 : 43321.428571 P95 : 227000.000000 P99 : 331250.000000 P100 : 402663.000000 COUNT : 175 SUM : 11720594 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 198.000000 P95 : 6018.571429 P99 : 9713.000000 P100 : 51274.000000 COUNT : 117 SUM : 187654 rocksdb.table.open.io.micros P50 : 9805.585831 P95 : 20997.163121 P99 : 29835.142857 P100 : 212112.000000 COUNT : 1507 SUM : 15627446 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.973908 P95 : 124.666555 P99 : 218.121869 P100 : 52228.000000 COUNT : 3732605 SUM : 77331528 rocksdb.read.block.get.micros P50 : 140.528858 P95 : 254.571824 P99 : 790.969593 P100 : 80035.000000 COUNT : 1197993 SUM : 197062975 rocksdb.write.raw.block.micros P50 : 0.561461 P95 : 1.733322 P99 : 4.646205 P100 : 32353.000000 COUNT : 4523904 SUM : 14217386 rocksdb.numfiles.in.singlecompaction P50 : 2.300000 P95 : 3.980000 P99 : 17.100000 P100 : 18.000000 COUNT : 70 SUM : 222 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.577431 P95 : 1275.386918 P99 : 2678.114079 P100 : 1047535.000000 COUNT : 221954 SUM : 46066932 rocksdb.sst.read.micros P50 : 2.672477 P95 : 169.978873 P99 : 449.763786 P100 : 79990.000000 COUNT : 4515370 SUM : 256905124 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1027996 SUM : 1027996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3338581 SUM : 3445415592 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 223714.285714 P95 : 1000250.000000 P99 : 1147680.000000 P100 : 1147680.000000 COUNT : 47 SUM : 15378921 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 180 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0