[2024-12-13 14:43:23.496819] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 23.11.0 initialization... [2024-12-13 14:43:23.497083] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid971176 ] Set seed to 1734097403490113 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-13 14:43:23.609649] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-13 14:43:23.649930] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-13 14:43:23.649978] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 971176' to capture a snapshot of events at runtime. [2024-12-13 14:43:23.649990] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid971176 for offline analysis/debug. [2024-12-13 14:43:23.650019] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Fri Dec 13 14:43:27 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/13-14:43:37 ... thread 0: (1000000,1000000) ops and (103241.2,103241.2) ops/second in (9.686056,9.686056) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/11 1.86 GB 50.3 0.3 0.0 0.3 1.1 0.8 0.0 1.4 31.3 112.8 10.05 4.05 11 0.914 319K 385 0.0 0.0 L1 9/9 1.02 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 20/5 1.58 GB 1.0 1.7 1.0 0.7 1.7 1.0 0.1 1.7 87.4 86.3 20.24 9.31 9 2.249 471K 0 0.0 0.0 L4 179/20 10.33 GB 1.0 5.7 3.2 2.5 5.6 3.1 0.0 1.7 109.0 107.3 53.63 27.33 32 1.676 2226K 5352 0.0 0.0 L5 1116/0 91.97 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1338/45 106.75 GB 0.0 7.7 4.3 3.5 8.4 4.9 2.4 10.5 94.5 102.9 83.92 40.69 52 1.614 3017K 5737 0.0 0.0 Int 0/0 0.00 KB 0.0 7.7 4.3 3.5 8.4 4.9 2.4 10.5 94.5 102.9 83.92 40.69 52 1.614 3017K 5737 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 7.7 4.3 3.5 7.6 4.1 0.0 0.0 103.9 102.4 76.32 38.04 42 1.817 3017K 5737 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 107.6 7.61 2.65 10 0.761 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.9 total, 9.7 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.43 GB write, 873.00 MB/s write, 7.74 GB read, 801.74 MB/s read, 83.9 seconds Interval compaction: 8.43 GB write, 891.21 MB/s write, 7.74 GB read, 818.46 MB/s read, 83.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 18, pending-compaction-bytes-delays: 19, pending-compaction-bytes-stops: 0, total-delays: 19, total-stops: 18, interval: 37 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 328097 Average: 21.2530 StdDev: 298.58 Min: 1 Median: 1.8450 Max: 48818 Percentiles: P50: 1.84 P75: 2.95 P99: 564.09 P99.9: 2896.28 P99.99: 4905.14 ------------------------------------------------------ [ 0, 1 ] 21358 6.510% 6.510% # ( 1, 2 ] 168868 51.469% 57.979% ########## ( 2, 3 ] 58782 17.916% 75.895% #### ( 3, 4 ] 30499 9.296% 85.190% ## ( 4, 6 ] 33431 10.189% 95.380% ## ( 6, 10 ] 8654 2.638% 98.017% # ( 10, 15 ] 1674 0.510% 98.528% ( 15, 22 ] 198 0.060% 98.588% ( 22, 34 ] 29 0.009% 98.597% ( 34, 51 ] 17 0.005% 98.602% ( 51, 76 ] 15 0.005% 98.607% ( 76, 110 ] 11 0.003% 98.610% ( 110, 170 ] 59 0.018% 98.628% ( 170, 250 ] 150 0.046% 98.674% ( 250, 380 ] 355 0.108% 98.782% ( 380, 580 ] 779 0.237% 99.019% ( 580, 870 ] 1078 0.329% 99.348% ( 870, 1300 ] 1004 0.306% 99.654% ( 1300, 1900 ] 515 0.157% 99.811% ( 1900, 2900 ] 295 0.090% 99.901% ( 2900, 4400 ] 271 0.083% 99.983% ( 4400, 6600 ] 101 0.031% 100.014% ( 6600, 9900 ] 9 0.003% 100.017% ( 9900, 14000 ] 3 0.001% 100.018% ( 14000, 22000 ] 7 0.002% 100.020% ( 22000, 33000 ] 11 0.003% 100.023% ( 33000, 50000 ] 4 0.001% 100.024% ** Level 1 read latency histogram (micros): Count: 247 Average: 1368.4980 StdDev: 1276.80 Min: 101 Median: 903.3621 Max: 6889 Percentiles: P50: 903.36 P75: 1525.00 P99: 6889.00 P99.9: 6889.00 P99.99: 6889.00 ------------------------------------------------------ ( 76, 110 ] 2 0.810% 0.810% ( 110, 170 ] 2 0.810% 1.619% ( 170, 250 ] 3 1.215% 2.834% ( 250, 380 ] 11 4.453% 7.287% # ( 380, 580 ] 39 15.789% 23.077% ### ( 580, 870 ] 62 25.101% 48.178% ##### ( 870, 1300 ] 58 23.482% 71.660% ##### ( 1300, 1900 ] 22 8.907% 80.567% ## ( 1900, 2900 ] 16 6.478% 87.045% # ( 2900, 4400 ] 21 8.502% 95.547% ## ( 4400, 6600 ] 8 3.239% 98.785% # ( 6600, 9900 ] 3 1.215% 100.000% ** Level 2 read latency histogram (micros): Count: 65446 Average: 10.2176 StdDev: 134.51 Min: 1 Median: 1.9394 Max: 7133 Percentiles: P50: 1.94 P75: 3.19 P99: 11.79 P99.9: 2371.33 P99.99: 5244.65 ------------------------------------------------------ [ 0, 1 ] 2185 3.339% 3.339% # ( 1, 2 ] 32508 49.671% 53.010% ########## ( 2, 3 ] 13105 20.024% 73.034% #### ( 3, 4 ] 6907 10.554% 83.588% ## ( 4, 6 ] 7993 12.213% 95.801% ## ( 6, 10 ] 1946 2.973% 98.775% # ( 10, 15 ] 411 0.628% 99.403% ( 15, 22 ] 47 0.072% 99.474% ( 22, 34 ] 3 0.005% 99.479% ( 34, 51 ] 6 0.009% 99.488% ( 51, 76 ] 8 0.012% 99.500% ( 76, 110 ] 5 0.008% 99.508% ( 110, 170 ] 2 0.003% 99.511% ( 170, 250 ] 9 0.014% 99.525% ( 250, 380 ] 19 0.029% 99.554% ( 380, 580 ] 46 0.070% 99.624% ( 580, 870 ] 68 0.104% 99.728% ( 870, 1300 ] 64 0.098% 99.826% ( 1300, 1900 ] 33 0.050% 99.876% ( 1900, 2900 ] 33 0.050% 99.927% ( 2900, 4400 ] 38 0.058% 99.985% ( 4400, 6600 ] 9 0.014% 99.998% ( 6600, 9900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 916444 Average: 9.8323 StdDev: 176.38 Min: 1 Median: 1.8942 Max: 44625 Percentiles: P50: 1.89 P75: 3.05 P99: 9.92 P99.9: 746.05 P99.99: 1121.62 ------------------------------------------------------ [ 0, 1 ] 41874 4.569% 4.569% # ( 1, 2 ] 465589 50.804% 55.373% ########## ( 2, 3 ] 175255 19.123% 74.496% #### ( 3, 4 ] 94368 10.297% 84.794% ## ( 4, 6 ] 104868 11.443% 96.237% ## ( 6, 10 ] 25826 2.818% 99.055% # ( 10, 15 ] 5067 0.553% 99.608% ( 15, 22 ] 643 0.070% 99.678% ( 22, 34 ] 82 0.009% 99.687% ( 34, 51 ] 48 0.005% 99.692% ( 51, 76 ] 80 0.009% 99.701% ( 76, 110 ] 26 0.003% 99.703% ( 110, 170 ] 98 0.011% 99.714% ( 170, 250 ] 159 0.017% 99.731% ( 250, 380 ] 382 0.042% 99.773% ( 380, 580 ] 674 0.074% 99.847% ( 580, 870 ] 855 0.093% 99.940% ( 870, 1300 ] 785 0.086% 100.026% ( 1300, 1900 ] 441 0.048% 100.074% ( 1900, 2900 ] 359 0.039% 100.113% ( 2900, 4400 ] 428 0.047% 100.160% ( 4400, 6600 ] 129 0.014% 100.174% ( 6600, 9900 ] 18 0.002% 100.176% ( 9900, 14000 ] 1 0.000% 100.176% ( 14000, 22000 ] 8 0.001% 100.177% ( 22000, 33000 ] 8 0.001% 100.178% ( 33000, 50000 ] 5 0.001% 100.178% ** Level 4 read latency histogram (micros): Count: 794211 Average: 18.7335 StdDev: 265.42 Min: 0 Median: 1.9007 Max: 65549 Percentiles: P50: 1.90 P75: 3.09 P99: 31.29 P99.9: 1725.44 P99.99: 2398.75 ------------------------------------------------------ [ 0, 1 ] 33851 4.262% 4.262% # ( 1, 2 ] 403292 50.779% 55.041% ########## ( 2, 3 ] 151423 19.066% 74.107% #### ( 3, 4 ] 81712 10.288% 84.395% ## ( 4, 6 ] 89271 11.240% 95.636% ## ( 6, 10 ] 21871 2.754% 98.389% # ( 10, 15 ] 4306 0.542% 98.932% ( 15, 22 ] 499 0.063% 98.994% ( 22, 34 ] 58 0.007% 99.002% ( 34, 51 ] 52 0.007% 99.008% ( 51, 76 ] 74 0.009% 99.018% ( 76, 110 ] 44 0.006% 99.023% ( 110, 170 ] 124 0.016% 99.039% ( 170, 250 ] 311 0.039% 99.078% ( 250, 380 ] 661 0.083% 99.161% ( 380, 580 ] 1226 0.154% 99.316% ( 580, 870 ] 1987 0.250% 99.566% ( 870, 1300 ] 1812 0.228% 99.794% ( 1300, 1900 ] 1190 0.150% 99.944% ( 1900, 2900 ] 739 0.093% 100.037% ( 2900, 4400 ] 702 0.088% 100.125% ( 4400, 6600 ] 216 0.027% 100.152% ( 6600, 9900 ] 37 0.005% 100.157% ( 9900, 14000 ] 5 0.001% 100.158% ( 14000, 22000 ] 10 0.001% 100.159% ( 22000, 33000 ] 15 0.002% 100.161% ( 33000, 50000 ] 4 0.001% 100.161% ( 50000, 75000 ] 2 0.000% 100.162% ** Level 5 read latency histogram (micros): Count: 19090 Average: 1442.2006 StdDev: 2045.80 Min: 81 Median: 936.3852 Max: 93424 Percentiles: P50: 936.39 P75: 1727.24 P99: 6355.15 P99.9: 32670.00 P99.99: 76592.50 ------------------------------------------------------ ( 76, 110 ] 24 0.126% 0.126% ( 110, 170 ] 193 1.011% 1.137% ( 170, 250 ] 568 2.975% 4.112% # ( 250, 380 ] 1305 6.836% 10.948% # ( 380, 580 ] 2875 15.060% 26.008% ### ( 580, 870 ] 4006 20.985% 46.993% #### ( 870, 1300 ] 3718 19.476% 66.469% #### ( 1300, 1900 ] 2287 11.980% 78.449% ## ( 1900, 2900 ] 1630 8.539% 86.988% ## ( 2900, 4400 ] 1703 8.921% 95.909% ## ( 4400, 6600 ] 664 3.478% 99.387% # ( 6600, 9900 ] 82 0.430% 99.817% ( 9900, 14000 ] 6 0.031% 99.848% ( 14000, 22000 ] 7 0.037% 99.885% ( 22000, 33000 ] 3 0.016% 99.900% ( 33000, 50000 ] 15 0.079% 99.979% ( 50000, 75000 ] 2 0.010% 99.990% ( 75000, 110000 ] 2 0.010% 100.000% ** DB Stats ** Uptime(secs): 9.9 total, 9.7 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 99.51 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.596 H:M:S, 56.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 101.58 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.596 H:M:S, 57.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 7 num-running-flushes: 1 2024/12/13-14:43:41 ... thread 0: (1000000,2000000) ops and (266482.4,148824.5) ops/second in (3.752593,13.438649) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/11 2.90 GB 163.6 1.4 0.0 1.4 3.3 1.8 0.0 1.8 79.8 183.8 18.16 11.38 31 0.586 1471K 1649 0.0 0.0 L1 9/9 1.02 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 15/0 982.47 MB 1.0 1.7 1.0 0.7 1.7 1.0 0.1 1.7 87.4 86.3 20.24 9.31 9 2.249 471K 0 0.0 0.0 L4 167/0 9.57 GB 1.0 7.1 3.8 3.3 7.0 3.7 0.0 1.8 113.9 112.0 63.91 33.73 37 1.727 2766K 8411 0.0 0.0 L5 1132/0 93.32 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1341/20 107.77 GB 0.0 10.2 4.9 5.4 12.0 6.6 3.8 6.5 102.6 119.7 102.31 54.42 77 1.329 4708K 10K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.2 4.9 5.4 12.0 6.6 3.8 6.5 102.6 119.7 102.31 54.42 77 1.329 4708K 10K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.2 4.9 5.4 10.1 4.7 0.0 0.0 115.1 113.6 91.15 48.73 51 1.787 4708K 10K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 169.3 11.16 5.68 26 0.429 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 13.6 total, 13.4 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.96 GB write, 897.54 MB/s write, 10.25 GB read, 769.28 MB/s read, 102.3 seconds Interval compaction: 11.96 GB write, 911.04 MB/s write, 10.25 GB read, 780.84 MB/s read, 102.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 1, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 1, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 22, pending-compaction-bytes-delays: 19, pending-compaction-bytes-stops: 0, total-delays: 20, total-stops: 22, interval: 42 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 650636 Average: 16.4186 StdDev: 216.51 Min: 0 Median: 1.6737 Max: 48818 Percentiles: P50: 1.67 P75: 2.53 P99: 322.17 P99.9: 1676.79 P99.99: 3306.80 ------------------------------------------------------ [ 0, 1 ] 90221 13.867% 13.867% ### ( 1, 2 ] 348960 53.634% 67.500% ########### ( 2, 3 ] 92638 14.238% 81.738% ### ( 3, 4 ] 43917 6.750% 88.488% # ( 4, 6 ] 43984 6.760% 95.248% # ( 6, 10 ] 11222 1.725% 96.973% ( 10, 15 ] 2213 0.340% 97.313% ( 15, 22 ] 308 0.047% 97.361% ( 22, 34 ] 41 0.006% 97.367% ( 34, 51 ] 20 0.003% 97.370% ( 51, 76 ] 16 0.002% 97.372% ( 76, 110 ] 1241 0.191% 97.563% ( 110, 170 ] 5198 0.799% 98.362% ( 170, 250 ] 3123 0.480% 98.842% ( 250, 380 ] 1851 0.284% 99.127% ( 380, 580 ] 1975 0.304% 99.430% ( 580, 870 ] 1598 0.246% 99.676% ( 870, 1300 ] 1119 0.172% 99.848% ( 1300, 1900 ] 542 0.083% 99.931% ( 1900, 2900 ] 308 0.047% 99.978% ( 2900, 4400 ] 280 0.043% 100.021% ( 4400, 6600 ] 101 0.016% 100.037% ( 6600, 9900 ] 9 0.001% 100.038% ( 9900, 14000 ] 3 0.000% 100.039% ( 14000, 22000 ] 7 0.001% 100.040% ( 22000, 33000 ] 11 0.002% 100.041% ( 33000, 50000 ] 4 0.001% 100.042% ** Level 1 read latency histogram (micros): Count: 45352 Average: 14.3791 StdDev: 141.87 Min: 1 Median: 2.8725 Max: 6889 Percentiles: P50: 2.87 P75: 4.33 P99: 210.57 P99.9: 1996.94 P99.99: 5627.82 ------------------------------------------------------ [ 0, 1 ] 827 1.824% 1.824% ( 1, 2 ] 11901 26.241% 28.065% ##### ( 2, 3 ] 11402 25.141% 53.206% ##### ( 3, 4 ] 8482 18.703% 71.909% #### ( 4, 6 ] 8514 18.773% 90.682% #### ( 6, 10 ] 2548 5.618% 96.300% # ( 10, 15 ] 464 1.023% 97.323% ( 15, 22 ] 302 0.666% 97.989% ( 22, 34 ] 18 0.040% 98.029% ( 34, 51 ] 1 0.002% 98.031% ( 51, 76 ] 2 0.004% 98.035% ( 76, 110 ] 75 0.165% 98.201% ( 110, 170 ] 293 0.646% 98.847% ( 170, 250 ] 137 0.302% 99.149% ( 250, 380 ] 77 0.170% 99.319% ( 380, 580 ] 95 0.209% 99.528% ( 580, 870 ] 78 0.172% 99.700% ( 870, 1300 ] 66 0.146% 99.846% ( 1300, 1900 ] 23 0.051% 99.896% ( 1900, 2900 ] 17 0.037% 99.934% ( 2900, 4400 ] 21 0.046% 99.980% ( 4400, 6600 ] 8 0.018% 99.998% ( 6600, 9900 ] 3 0.007% 100.004% ** Level 2 read latency histogram (micros): Count: 65446 Average: 10.2176 StdDev: 134.51 Min: 1 Median: 1.9394 Max: 7133 Percentiles: P50: 1.94 P75: 3.19 P99: 11.79 P99.9: 2371.33 P99.99: 5244.65 ------------------------------------------------------ [ 0, 1 ] 2185 3.339% 3.339% # ( 1, 2 ] 32508 49.671% 53.010% ########## ( 2, 3 ] 13105 20.024% 73.034% #### ( 3, 4 ] 6907 10.554% 83.588% ## ( 4, 6 ] 7993 12.213% 95.801% ## ( 6, 10 ] 1946 2.973% 98.775% # ( 10, 15 ] 411 0.628% 99.403% ( 15, 22 ] 47 0.072% 99.474% ( 22, 34 ] 3 0.005% 99.479% ( 34, 51 ] 6 0.009% 99.488% ( 51, 76 ] 8 0.012% 99.500% ( 76, 110 ] 5 0.008% 99.508% ( 110, 170 ] 2 0.003% 99.511% ( 170, 250 ] 9 0.014% 99.525% ( 250, 380 ] 19 0.029% 99.554% ( 380, 580 ] 46 0.070% 99.624% ( 580, 870 ] 68 0.104% 99.728% ( 870, 1300 ] 64 0.098% 99.826% ( 1300, 1900 ] 33 0.050% 99.876% ( 1900, 2900 ] 33 0.050% 99.927% ( 2900, 4400 ] 38 0.058% 99.985% ( 4400, 6600 ] 9 0.014% 99.998% ( 6600, 9900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 962425 Average: 9.6397 StdDev: 172.36 Min: 1 Median: 1.8842 Max: 44625 Percentiles: P50: 1.88 P75: 3.00 P99: 9.97 P99.9: 734.13 P99.99: 1118.38 ------------------------------------------------------ [ 0, 1 ] 45052 4.681% 4.681% # ( 1, 2 ] 493278 51.254% 55.935% ########## ( 2, 3 ] 183041 19.019% 74.953% #### ( 3, 4 ] 97349 10.115% 85.068% ## ( 4, 6 ] 107776 11.198% 96.267% ## ( 6, 10 ] 26529 2.756% 99.023% # ( 10, 15 ] 5196 0.540% 99.563% ( 15, 22 ] 654 0.068% 99.631% ( 22, 34 ] 82 0.009% 99.640% ( 34, 51 ] 48 0.005% 99.645% ( 51, 76 ] 80 0.008% 99.653% ( 76, 110 ] 77 0.008% 99.661% ( 110, 170 ] 352 0.037% 99.698% ( 170, 250 ] 299 0.031% 99.729% ( 250, 380 ] 453 0.047% 99.776% ( 380, 580 ] 731 0.076% 99.852% ( 580, 870 ] 876 0.091% 99.943% ( 870, 1300 ] 789 0.082% 100.025% ( 1300, 1900 ] 442 0.046% 100.071% ( 1900, 2900 ] 363 0.038% 100.108% ( 2900, 4400 ] 429 0.045% 100.153% ( 4400, 6600 ] 129 0.013% 100.166% ( 6600, 9900 ] 18 0.002% 100.168% ( 9900, 14000 ] 1 0.000% 100.168% ( 14000, 22000 ] 8 0.001% 100.169% ( 22000, 33000 ] 8 0.001% 100.170% ( 33000, 50000 ] 5 0.001% 100.170% ** Level 4 read latency histogram (micros): Count: 854954 Average: 19.3087 StdDev: 257.69 Min: 0 Median: 1.8924 Max: 65549 Percentiles: P50: 1.89 P75: 3.07 P99: 279.48 P99.9: 1702.85 P99.99: 2394.72 ------------------------------------------------------ [ 0, 1 ] 37941 4.438% 4.438% # ( 1, 2 ] 436493 51.055% 55.492% ########## ( 2, 3 ] 160522 18.776% 74.268% #### ( 3, 4 ] 85276 9.974% 84.242% ## ( 4, 6 ] 92729 10.846% 95.088% ## ( 6, 10 ] 23030 2.694% 97.782% # ( 10, 15 ] 4493 0.526% 98.308% ( 15, 22 ] 514 0.060% 98.368% ( 22, 34 ] 61 0.007% 98.375% ( 34, 51 ] 54 0.006% 98.381% ( 51, 76 ] 76 0.009% 98.390% ( 76, 110 ] 719 0.084% 98.474% ( 110, 170 ] 2601 0.304% 98.778% ( 170, 250 ] 1607 0.188% 98.966% ( 250, 380 ] 1272 0.149% 99.115% ( 380, 580 ] 1783 0.209% 99.324% ( 580, 870 ] 2235 0.261% 99.585% ( 870, 1300 ] 1886 0.221% 99.806% ( 1300, 1900 ] 1202 0.141% 99.946% ( 1900, 2900 ] 757 0.089% 100.035% ( 2900, 4400 ] 711 0.083% 100.118% ( 4400, 6600 ] 219 0.026% 100.144% ( 6600, 9900 ] 37 0.004% 100.148% ( 9900, 14000 ] 5 0.001% 100.148% ( 14000, 22000 ] 10 0.001% 100.150% ( 22000, 33000 ] 15 0.002% 100.151% ( 33000, 50000 ] 4 0.000% 100.152% ( 50000, 75000 ] 2 0.000% 100.152% ** Level 5 read latency histogram (micros): Count: 59965 Average: 617.6279 StdDev: 1295.75 Min: 75 Median: 237.0746 Max: 93424 Percentiles: P50: 237.07 P75: 642.73 P99: 4996.65 P99.9: 8788.14 P99.99: 44337.30 ------------------------------------------------------ ( 51, 76 ] 2 0.003% 0.003% ( 76, 110 ] 4816 8.031% 8.035% ## ( 110, 170 ] 17326 28.894% 36.928% ###### ( 170, 250 ] 9349 15.591% 52.519% ### ( 250, 380 ] 5827 9.717% 62.236% ## ( 380, 580 ] 6465 10.781% 73.018% ## ( 580, 870 ] 5496 9.165% 82.183% ## ( 870, 1300 ] 4120 6.871% 89.054% # ( 1300, 1900 ] 2392 3.989% 93.043% # ( 1900, 2900 ] 1670 2.785% 95.828% # ( 2900, 4400 ] 1722 2.872% 98.699% # ( 4400, 6600 ] 665 1.109% 99.808% ( 6600, 9900 ] 83 0.138% 99.947% ( 9900, 14000 ] 6 0.010% 99.957% ( 14000, 22000 ] 7 0.012% 99.968% ( 22000, 33000 ] 3 0.005% 99.973% ( 33000, 50000 ] 15 0.025% 99.998% ( 50000, 75000 ] 2 0.003% 100.002% ( 75000, 110000 ] 2 0.003% 100.005% ** DB Stats ** Uptime(secs): 13.6 total, 3.8 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 144.28 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.903 H:M:S, 43.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 262.27 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.307 H:M:S, 8.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/13-14:43:48 ... thread 0: (1000000,3000000) ops and (136458.5,144460.7) ops/second in (7.328238,20.766887) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/13 3.82 GB 132.1 2.6 0.0 2.6 5.5 2.8 0.0 1.9 104.2 215.5 26.01 18.72 52 0.500 2750K 2857 0.0 0.0 L1 9/9 1.02 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 15/0 982.47 MB 1.0 1.7 1.0 0.7 1.7 1.0 0.1 1.7 87.4 86.3 20.24 9.31 9 2.249 471K 0 0.0 0.0 L4 167/0 9.57 GB 1.0 7.1 3.8 3.3 7.0 3.7 0.0 1.8 113.9 112.0 63.91 33.73 37 1.727 2766K 8411 0.0 0.0 L5 1132/0 93.32 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1341/22 108.69 GB 0.0 11.5 4.9 6.6 14.2 7.6 3.8 5.0 106.7 131.7 110.16 61.75 98 1.124 5988K 11K 0.0 0.0 Int 0/0 0.00 KB 0.0 11.5 4.9 6.6 14.2 7.6 3.8 5.0 106.7 131.7 110.16 61.75 98 1.124 5988K 11K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.5 4.9 6.6 11.3 4.7 0.0 0.0 122.5 121.0 95.96 53.44 56 1.714 5988K 11K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 204.0 14.20 8.31 42 0.338 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 21.0 total, 20.8 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.17 GB write, 691.99 MB/s write, 11.48 GB read, 560.54 MB/s read, 110.2 seconds Interval compaction: 14.17 GB write, 698.73 MB/s write, 11.48 GB read, 566.00 MB/s read, 110.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 12, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 12, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 22, pending-compaction-bytes-delays: 19, pending-compaction-bytes-stops: 0, total-delays: 31, total-stops: 22, interval: 53 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1180443 Average: 15.2962 StdDev: 165.39 Min: 0 Median: 1.5633 Max: 48818 Percentiles: P50: 1.56 P75: 2.14 P99: 266.37 P99.9: 1116.32 P99.99: 1892.36 ------------------------------------------------------ [ 0, 1 ] 233559 19.786% 19.786% #### ( 1, 2 ] 633167 53.638% 73.424% ########### ( 2, 3 ] 132629 11.236% 84.659% ## ( 3, 4 ] 59464 5.037% 89.697% # ( 4, 6 ] 54759 4.639% 94.336% # ( 6, 10 ] 13195 1.118% 95.453% ( 10, 15 ] 2728 0.231% 95.685% ( 15, 22 ] 365 0.031% 95.715% ( 22, 34 ] 42 0.004% 95.719% ( 34, 51 ] 21 0.002% 95.721% ( 51, 76 ] 30 0.003% 95.723% ( 76, 110 ] 6370 0.540% 96.263% ( 110, 170 ] 22119 1.874% 98.137% ( 170, 250 ] 9636 0.816% 98.953% ( 250, 380 ] 4412 0.374% 99.327% ( 380, 580 ] 3927 0.333% 99.659% ( 580, 870 ] 2148 0.182% 99.841% ( 870, 1300 ] 1209 0.102% 99.944% ( 1300, 1900 ] 553 0.047% 99.991% ( 1900, 2900 ] 322 0.027% 100.018% ( 2900, 4400 ] 289 0.024% 100.042% ( 4400, 6600 ] 102 0.009% 100.051% ( 6600, 9900 ] 9 0.001% 100.052% ( 9900, 14000 ] 3 0.000% 100.052% ( 14000, 22000 ] 7 0.001% 100.053% ( 22000, 33000 ] 11 0.001% 100.054% ( 33000, 50000 ] 4 0.000% 100.054% ** Level 1 read latency histogram (micros): Count: 184032 Average: 7.7168 StdDev: 73.25 Min: 1 Median: 2.6393 Max: 6889 Percentiles: P50: 2.64 P75: 3.68 P99: 129.12 P99.9: 732.53 P99.99: 3442.63 ------------------------------------------------------ [ 0, 1 ] 2542 1.381% 1.381% ( 1, 2 ] 50416 27.395% 28.777% ##### ( 2, 3 ] 61096 33.199% 61.975% ####### ( 3, 4 ] 35038 19.039% 81.014% #### ( 4, 6 ] 22617 12.290% 93.304% ## ( 6, 10 ] 7102 3.859% 97.163% # ( 10, 15 ] 1886 1.025% 98.188% ( 15, 22 ] 722 0.392% 98.580% ( 22, 34 ] 59 0.032% 98.612% ( 34, 51 ] 2 0.001% 98.613% ( 51, 76 ] 2 0.001% 98.614% ( 76, 110 ] 333 0.181% 98.795% ( 110, 170 ] 1182 0.642% 99.438% ( 170, 250 ] 445 0.242% 99.679% ( 250, 380 ] 190 0.103% 99.783% ( 380, 580 ] 166 0.090% 99.873% ( 580, 870 ] 95 0.052% 99.924% ( 870, 1300 ] 73 0.040% 99.964% ( 1300, 1900 ] 23 0.012% 99.977% ( 1900, 2900 ] 17 0.009% 99.986% ( 2900, 4400 ] 21 0.011% 99.997% ( 4400, 6600 ] 8 0.004% 100.002% ( 6600, 9900 ] 3 0.002% 100.003% ** Level 2 read latency histogram (micros): Count: 65446 Average: 10.2176 StdDev: 134.51 Min: 1 Median: 1.9394 Max: 7133 Percentiles: P50: 1.94 P75: 3.19 P99: 11.79 P99.9: 2371.33 P99.99: 5244.65 ------------------------------------------------------ [ 0, 1 ] 2185 3.339% 3.339% # ( 1, 2 ] 32508 49.671% 53.010% ########## ( 2, 3 ] 13105 20.024% 73.034% #### ( 3, 4 ] 6907 10.554% 83.588% ## ( 4, 6 ] 7993 12.213% 95.801% ## ( 6, 10 ] 1946 2.973% 98.775% # ( 10, 15 ] 411 0.628% 99.403% ( 15, 22 ] 47 0.072% 99.474% ( 22, 34 ] 3 0.005% 99.479% ( 34, 51 ] 6 0.009% 99.488% ( 51, 76 ] 8 0.012% 99.500% ( 76, 110 ] 5 0.008% 99.508% ( 110, 170 ] 2 0.003% 99.511% ( 170, 250 ] 9 0.014% 99.525% ( 250, 380 ] 19 0.029% 99.554% ( 380, 580 ] 46 0.070% 99.624% ( 580, 870 ] 68 0.104% 99.728% ( 870, 1300 ] 64 0.098% 99.826% ( 1300, 1900 ] 33 0.050% 99.876% ( 1900, 2900 ] 33 0.050% 99.927% ( 2900, 4400 ] 38 0.058% 99.985% ( 4400, 6600 ] 9 0.014% 99.998% ( 6600, 9900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 964000 Average: 9.9173 StdDev: 172.41 Min: 1 Median: 1.8858 Max: 44625 Percentiles: P50: 1.89 P75: 3.02 P99: 11.28 P99.9: 737.42 P99.99: 1118.53 ------------------------------------------------------ [ 0, 1 ] 45052 4.673% 4.673% # ( 1, 2 ] 493278 51.170% 55.843% ########## ( 2, 3 ] 183043 18.988% 74.831% #### ( 3, 4 ] 97349 10.098% 84.930% ## ( 4, 6 ] 107778 11.180% 96.110% ## ( 6, 10 ] 26529 2.752% 98.862% # ( 10, 15 ] 5196 0.539% 99.401% ( 15, 22 ] 654 0.068% 99.469% ( 22, 34 ] 82 0.009% 99.477% ( 34, 51 ] 48 0.005% 99.482% ( 51, 76 ] 80 0.008% 99.491% ( 76, 110 ] 337 0.035% 99.526% ( 110, 170 ] 1176 0.122% 99.648% ( 170, 250 ] 587 0.061% 99.708% ( 250, 380 ] 538 0.056% 99.764% ( 380, 580 ] 821 0.085% 99.849% ( 580, 870 ] 899 0.093% 99.943% ( 870, 1300 ] 790 0.082% 100.025% ( 1300, 1900 ] 442 0.046% 100.070% ( 1900, 2900 ] 363 0.038% 100.108% ( 2900, 4400 ] 429 0.045% 100.153% ( 4400, 6600 ] 129 0.013% 100.166% ( 6600, 9900 ] 18 0.002% 100.168% ( 9900, 14000 ] 1 0.000% 100.168% ( 14000, 22000 ] 8 0.001% 100.169% ( 22000, 33000 ] 8 0.001% 100.170% ( 33000, 50000 ] 5 0.001% 100.170% ** Level 4 read latency histogram (micros): Count: 871423 Average: 22.1777 StdDev: 256.64 Min: 0 Median: 1.9111 Max: 65549 Percentiles: P50: 1.91 P75: 3.21 P99: 374.56 P99.9: 1697.31 P99.99: 2395.20 ------------------------------------------------------ [ 0, 1 ] 37941 4.354% 4.354% # ( 1, 2 ] 436566 50.098% 54.452% ########## ( 2, 3 ] 160821 18.455% 72.907% #### ( 3, 4 ] 85471 9.808% 82.715% ## ( 4, 6 ] 92969 10.669% 93.384% ## ( 6, 10 ] 23082 2.649% 96.033% # ( 10, 15 ] 4501 0.517% 96.549% ( 15, 22 ] 514 0.059% 96.608% ( 22, 34 ] 61 0.007% 96.615% ( 34, 51 ] 54 0.006% 96.621% ( 51, 76 ] 77 0.009% 96.630% ( 76, 110 ] 3327 0.382% 97.012% ( 110, 170 ] 10819 1.242% 98.253% ( 170, 250 ] 4304 0.494% 98.747% ( 250, 380 ] 2298 0.264% 99.011% ( 380, 580 ] 2575 0.295% 99.307% ( 580, 870 ] 2452 0.281% 99.588% ( 870, 1300 ] 1919 0.220% 99.808% ( 1300, 1900 ] 1209 0.139% 99.947% ( 1900, 2900 ] 759 0.087% 100.034% ( 2900, 4400 ] 712 0.082% 100.116% ( 4400, 6600 ] 219 0.025% 100.141% ( 6600, 9900 ] 37 0.004% 100.145% ( 9900, 14000 ] 5 0.001% 100.146% ( 14000, 22000 ] 10 0.001% 100.147% ( 22000, 33000 ] 15 0.002% 100.148% ( 33000, 50000 ] 4 0.000% 100.149% ( 50000, 75000 ] 2 0.000% 100.149% ** Level 5 read latency histogram (micros): Count: 160266 Average: 343.8545 StdDev: 825.87 Min: 72 Median: 160.0093 Max: 93424 Percentiles: P50: 160.01 P75: 268.58 P99: 3682.03 P99.9: 6440.32 P99.99: 21969.60 ------------------------------------------------------ ( 51, 76 ] 11 0.007% 0.007% ( 76, 110 ] 22608 14.107% 14.113% ### ( 110, 170 ] 69004 43.056% 57.169% ######### ( 170, 250 ] 26795 16.719% 73.888% ### ( 250, 380 ] 12465 7.778% 81.666% ## ( 380, 580 ] 11563 7.215% 88.881% # ( 580, 870 ] 6888 4.298% 93.179% # ( 870, 1300 ] 4320 2.696% 95.874% # ( 1300, 1900 ] 2416 1.507% 97.382% ( 1900, 2900 ] 1694 1.057% 98.439% ( 2900, 4400 ] 1725 1.076% 99.515% ( 4400, 6600 ] 665 0.415% 99.930% ( 6600, 9900 ] 83 0.052% 99.982% ( 9900, 14000 ] 6 0.004% 99.986% ( 14000, 22000 ] 7 0.004% 99.990% ( 22000, 33000 ] 3 0.002% 99.992% ( 33000, 50000 ] 15 0.009% 100.001% ( 50000, 75000 ] 2 0.001% 100.002% ( 75000, 110000 ] 2 0.001% 100.004% ** DB Stats ** Uptime(secs): 21.0 total, 7.3 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 140.79 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.652 H:M:S, 46.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 134.30 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.749 H:M:S, 51.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 231.326 micros/op 17279 ops/sec 60.128 seconds 1038996 operations; 16.7 MB/s (249999 of 249999 found) Microseconds per read: Count: 1038996 Average: 231.3278 StdDev: 839.49 Min: 2 Median: 168.5198 Max: 300673 Percentiles: P50: 168.52 P75: 227.84 P99: 1067.81 P99.9: 9836.17 P99.99: 27972.94 ------------------------------------------------------ ( 1, 2 ] 11 0.001% 0.001% ( 2, 3 ] 44 0.004% 0.005% ( 3, 4 ] 46 0.004% 0.010% ( 4, 6 ] 82 0.008% 0.018% ( 6, 10 ] 625 0.060% 0.078% ( 10, 15 ] 3826 0.368% 0.446% ( 15, 22 ] 3633 0.350% 0.796% ( 22, 34 ] 10855 1.045% 1.840% ( 34, 51 ] 2768 0.266% 2.107% ( 51, 76 ] 322 0.031% 2.138% ( 76, 110 ] 24542 2.362% 4.500% ( 110, 170 ] 484702 46.651% 51.151% ######### ( 170, 250 ] 342726 32.986% 84.137% ####### ( 250, 380 ] 98215 9.453% 93.590% ## ( 380, 580 ] 38070 3.664% 97.254% # ( 580, 870 ] 15297 1.472% 98.726% ( 870, 1300 ] 6178 0.595% 99.321% ( 1300, 1900 ] 2543 0.245% 99.566% ( 1900, 2900 ] 1544 0.149% 99.714% ( 2900, 4400 ] 1187 0.114% 99.829% ( 4400, 6600 ] 437 0.042% 99.871% ( 6600, 9900 ] 310 0.030% 99.901% ( 9900, 14000 ] 485 0.047% 99.947% ( 14000, 22000 ] 386 0.037% 99.984% ( 22000, 33000 ] 107 0.010% 99.995% ( 33000, 50000 ] 34 0.003% 99.998% ( 50000, 75000 ] 15 0.001% 99.999% ( 110000, 170000 ] 3 0.000% 100.000% ( 250000, 380000 ] 3 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 16286105 rocksdb.bloom.filter.full.positive COUNT : 1197947 rocksdb.bloom.filter.full.true.positive COUNT : 1038563 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 433 rocksdb.memtable.miss COUNT : 1038563 rocksdb.l0.hit COUNT : 38474 rocksdb.l1.hit COUNT : 18623 rocksdb.l2andup.hit COUNT : 981466 rocksdb.compaction.key.drop.new COUNT : 196921 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3978742 rocksdb.number.keys.read COUNT : 1038996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 4106061744 rocksdb.bytes.read COUNT : 1038996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1581 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 44530962 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 18402814783 rocksdb.compact.write.bytes COUNT : 18195231963 rocksdb.flush.write.bytes COUNT : 4095634258 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 24 rocksdb.number.superversion_acquires COUNT : 777 rocksdb.number.superversion_releases COUNT : 744 rocksdb.number.superversion_cleanups COUNT : 192 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 226 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 4062491648 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1476608 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 22319617998 rocksdb.non.last.level.read.count COUNT : 5249138 rocksdb.block.checksum.compute.count COUNT : 5665463 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 167.673152 P95 : 455.596816 P99 : 1066.626129 P100 : 300669.000000 COUNT : 1038996 SUM : 239476736 rocksdb.db.write.micros P50 : 1.524954 P95 : 5.044067 P99 : 14.044819 P100 : 829186.000000 COUNT : 3978742 SUM : 54855333 rocksdb.compaction.times.micros P50 : 1661363.636364 P95 : 2850000.000000 P99 : 25138427.000000 P100 : 25138427.000000 COUNT : 61 SUM : 129533434 rocksdb.compaction.times.cpu_micros P50 : 997307.692308 P95 : 1843461.538462 P99 : 11438000.000000 P100 : 12736788.000000 COUNT : 61 SUM : 74434444 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 790.909091 P95 : 132000.000000 P99 : 283984.000000 P100 : 283984.000000 COUNT : 58 SUM : 998125 rocksdb.compaction.outfile.sync.micros P50 : 80625.000000 P95 : 341590.909091 P99 : 514357.142857 P100 : 552048.000000 COUNT : 205 SUM : 20859634 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 168.285714 P95 : 3864.285714 P99 : 5112.000000 P100 : 5112.000000 COUNT : 130 SUM : 121655 rocksdb.table.open.io.micros P50 : 8392.788462 P95 : 19413.664596 P99 : 28478.421053 P100 : 146095.000000 COUNT : 1581 SUM : 14440374 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.907287 P95 : 87.100221 P99 : 189.227928 P100 : 48823.000000 COUNT : 4455887 SUM : 78219406 rocksdb.read.block.get.micros P50 : 138.901950 P95 : 332.682955 P99 : 956.112611 P100 : 93553.000000 COUNT : 1207979 SUM : 203860076 rocksdb.write.raw.block.micros P50 : 0.530584 P95 : 1.200216 P99 : 3.847793 P100 : 66810.000000 COUNT : 5399166 SUM : 15657092 rocksdb.numfiles.in.singlecompaction P50 : 3.229730 P95 : 5.500000 P99 : 35.000000 P100 : 35.000000 COUNT : 75 SUM : 310 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.574284 P95 : 1928.412144 P99 : 2714.351603 P100 : 829010.000000 COUNT : 219395 SUM : 44554456 rocksdb.sst.read.micros P50 : 2.270439 P95 : 169.183064 P99 : 481.630694 P100 : 93424.000000 COUNT : 5249148 SUM : 264054164 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1038996 SUM : 1038996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3978742 SUM : 4106061744 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 219361.702128 P95 : 984666.666667 P99 : 1202168.000000 P100 : 1202168.000000 COUNT : 58 SUM : 17450938 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 236 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0