[2024-11-18 16:43:48.716774] Starting SPDK v24.09.1-pre git sha1 b18e1bd62 / DPDK 22.11.4 initialization... [2024-11-18 16:43:48.716916] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1219067 ] Set seed to 1731944628713330 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-18 16:43:48.805779] app.c: 917:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-18 16:43:48.843377] app.c: 610:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-18 16:43:48.843423] app.c: 611:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1219067' to capture a snapshot of events at runtime. [2024-11-18 16:43:48.843433] app.c: 616:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-18 16:43:48.843443] app.c: 617:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-18 16:43:48.843451] app.c: 618:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1219067 for offline analysis/debug. [2024-11-18 16:43:48.843528] reactor.c: 990:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Nov 18 16:43:53 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/18-16:44:00 ... thread 0: (1000000,1000000) ops and (139287.5,139287.5) ops/second in (7.179393,7.179393) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/9 3.14 GB 62.9 0.3 0.0 0.3 1.2 0.9 0.0 1.4 48.6 184.6 6.48 3.80 13 0.499 319K 402 0.0 0.0 L1 7/7 867.32 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 15/0 955.32 MB 1.0 0.1 0.1 0.0 0.1 0.1 0.0 1.2 148.0 147.5 1.03 0.68 1 1.028 130K 0 0.0 0.0 L4 171/0 9.57 GB 1.0 4.9 2.3 2.6 4.8 2.2 0.0 2.1 109.6 107.7 45.37 23.03 26 1.745 665K 3020 0.0 0.0 L5 1132/0 93.14 GB 1.0 0.0 0.0 0.0 0.0 0.0 2.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1339/16 107.62 GB 0.0 5.3 2.4 2.9 6.1 3.2 2.6 7.1 102.8 117.9 52.88 27.51 40 1.322 1116K 3422 0.0 0.0 Int 0/0 0.00 KB 0.0 5.3 2.4 2.9 6.1 3.2 2.6 7.1 102.8 117.9 52.88 27.51 40 1.322 1116K 3422 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.3 2.4 2.9 5.2 2.3 0.0 0.0 113.1 111.3 48.10 24.95 28 1.718 1116K 3422 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 184.6 4.78 2.56 12 0.398 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.4 total, 7.2 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.09 GB write, 845.30 MB/s write, 5.31 GB read, 737.38 MB/s read, 52.9 seconds Interval compaction: 6.09 GB write, 868.09 MB/s write, 5.31 GB read, 757.26 MB/s read, 52.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 7, pending-compaction-bytes-delays: 15, pending-compaction-bytes-stops: 0, total-delays: 15, total-stops: 7, interval: 22 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 382343 Average: 11.3158 StdDev: 116.29 Min: 1 Median: 1.8336 Max: 13350 Percentiles: P50: 1.83 P75: 2.78 P99: 189.39 P99.9: 1203.73 P99.99: 2362.12 ------------------------------------------------------ [ 0, 1 ] 29108 7.613% 7.613% ## ( 1, 2 ] 194411 50.847% 58.460% ########## ( 2, 3 ] 81382 21.285% 79.745% #### ( 3, 4 ] 28052 7.337% 87.082% # ( 4, 6 ] 31437 8.222% 95.304% ## ( 6, 10 ] 9365 2.449% 97.754% ( 10, 15 ] 1447 0.378% 98.132% ( 15, 22 ] 187 0.049% 98.181% ( 22, 34 ] 28 0.007% 98.189% ( 34, 51 ] 11 0.003% 98.191% ( 51, 76 ] 17 0.004% 98.196% ( 76, 110 ] 660 0.173% 98.368% ( 110, 170 ] 2122 0.555% 98.923% ( 170, 250 ] 1215 0.318% 99.241% ( 250, 380 ] 800 0.209% 99.450% ( 380, 580 ] 829 0.217% 99.667% ( 580, 870 ] 575 0.150% 99.818% ( 870, 1300 ] 408 0.107% 99.924% ( 1300, 1900 ] 195 0.051% 99.975% ( 1900, 2900 ] 125 0.033% 100.008% ( 2900, 4400 ] 108 0.028% 100.036% ( 4400, 6600 ] 29 0.008% 100.044% ( 6600, 9900 ] 8 0.002% 100.046% ( 9900, 14000 ] 2 0.001% 100.047% ** Level 1 read latency histogram (micros): Count: 252 Average: 817.1667 StdDev: 1834.22 Min: 84 Median: 244.8936 Max: 13473 Percentiles: P50: 244.89 P75: 688.75 P99: 11933.60 P99.9: 13473.00 P99.99: 13473.00 ------------------------------------------------------ ( 76, 110 ] 12 4.762% 4.762% # ( 110, 170 ] 70 27.778% 32.540% ###### ( 170, 250 ] 47 18.651% 51.190% #### ( 250, 380 ] 31 12.302% 63.492% ## ( 380, 580 ] 23 9.127% 72.619% ## ( 580, 870 ] 16 6.349% 78.968% # ( 870, 1300 ] 22 8.730% 87.698% ## ( 1300, 1900 ] 12 4.762% 92.460% # ( 1900, 2900 ] 6 2.381% 94.841% ( 2900, 4400 ] 6 2.381% 97.222% ( 4400, 6600 ] 1 0.397% 97.619% ( 6600, 9900 ] 1 0.397% 98.016% ( 9900, 14000 ] 5 1.984% 100.000% ** Level 2 read latency histogram (micros): Count: 32712 Average: 8.2273 StdDev: 114.34 Min: 1 Median: 1.9870 Max: 12487 Percentiles: P50: 1.99 P75: 2.90 P99: 9.98 P99.9: 1261.07 P99.99: 4934.45 ------------------------------------------------------ [ 0, 1 ] 372 1.137% 1.137% ( 1, 2 ] 16194 49.505% 50.642% ########## ( 2, 3 ] 8857 27.076% 77.718% ##### ( 3, 4 ] 2916 8.914% 86.632% ## ( 4, 6 ] 3160 9.660% 96.292% ## ( 6, 10 ] 891 2.724% 99.016% # ( 10, 15 ] 142 0.434% 99.450% ( 15, 22 ] 12 0.037% 99.486% ( 22, 34 ] 2 0.006% 99.493% ( 51, 76 ] 1 0.003% 99.496% ( 110, 170 ] 2 0.006% 99.502% ( 170, 250 ] 4 0.012% 99.514% ( 250, 380 ] 18 0.055% 99.569% ( 380, 580 ] 35 0.107% 99.676% ( 580, 870 ] 36 0.110% 99.786% ( 870, 1300 ] 41 0.125% 99.911% ( 1300, 1900 ] 13 0.040% 99.951% ( 1900, 2900 ] 7 0.021% 99.972% ( 2900, 4400 ] 5 0.015% 99.988% ( 4400, 6600 ] 3 0.009% 99.997% ( 9900, 14000 ] 1 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 411749 Average: 11.0676 StdDev: 136.45 Min: 1 Median: 1.9822 Max: 10397 Percentiles: P50: 1.98 P75: 3.05 P99: 10.90 P99.9: 1282.44 P99.99: 1800.94 ------------------------------------------------------ [ 0, 1 ] 10707 2.600% 2.600% # ( 1, 2 ] 198703 48.258% 50.859% ########## ( 2, 3 ] 97193 23.605% 74.464% ##### ( 3, 4 ] 42567 10.338% 84.802% ## ( 4, 6 ] 45437 11.035% 95.837% ## ( 6, 10 ] 12647 3.072% 98.908% # ( 10, 15 ] 2096 0.509% 99.417% ( 15, 22 ] 316 0.077% 99.494% ( 22, 34 ] 40 0.010% 99.504% ( 34, 51 ] 20 0.005% 99.509% ( 51, 76 ] 26 0.006% 99.515% ( 76, 110 ] 41 0.010% 99.525% ( 110, 170 ] 137 0.033% 99.558% ( 170, 250 ] 111 0.027% 99.585% ( 250, 380 ] 157 0.038% 99.623% ( 380, 580 ] 230 0.056% 99.679% ( 580, 870 ] 422 0.102% 99.782% ( 870, 1300 ] 508 0.123% 99.905% ( 1300, 1900 ] 419 0.102% 100.007% ( 1900, 2900 ] 269 0.065% 100.072% ( 2900, 4400 ] 171 0.042% 100.114% ( 4400, 6600 ] 60 0.015% 100.128% ( 6600, 9900 ] 8 0.002% 100.130% ( 9900, 14000 ] 1 0.000% 100.130% ** Level 4 read latency histogram (micros): Count: 612925 Average: 17.8374 StdDev: 180.79 Min: 1 Median: 1.9983 Max: 12238 Percentiles: P50: 2.00 P75: 3.11 P99: 158.86 P99.9: 1665.58 P99.99: 2101.04 ------------------------------------------------------ [ 0, 1 ] 14614 2.384% 2.384% ( 1, 2 ] 292357 47.699% 50.083% ########## ( 2, 3 ] 145889 23.802% 73.885% ##### ( 3, 4 ] 63358 10.337% 84.222% ## ( 4, 6 ] 67425 11.001% 95.223% ## ( 6, 10 ] 18312 2.988% 98.210% # ( 10, 15 ] 3266 0.533% 98.743% ( 15, 22 ] 479 0.078% 98.821% ( 22, 34 ] 70 0.011% 98.833% ( 34, 51 ] 43 0.007% 98.840% ( 51, 76 ] 34 0.006% 98.845% ( 76, 110 ] 259 0.042% 98.887% ( 110, 170 ] 847 0.138% 99.026% ( 170, 250 ] 652 0.106% 99.132% ( 250, 380 ] 595 0.097% 99.229% ( 380, 580 ] 871 0.142% 99.371% ( 580, 870 ] 1182 0.193% 99.564% ( 870, 1300 ] 1412 0.230% 99.794% ( 1300, 1900 ] 1062 0.173% 99.968% ( 1900, 2900 ] 680 0.111% 100.079% ( 2900, 4400 ] 482 0.079% 100.157% ( 4400, 6600 ] 160 0.026% 100.183% ( 6600, 9900 ] 21 0.003% 100.187% ( 9900, 14000 ] 1 0.000% 100.187% ** Level 5 read latency histogram (micros): Count: 26322 Average: 759.8210 StdDev: 1128.47 Min: 77 Median: 292.3841 Max: 15211 Percentiles: P50: 292.38 P75: 809.63 P99: 5833.83 P99.9: 9012.50 P99.99: 11301.99 ------------------------------------------------------ ( 76, 110 ] 1714 6.512% 6.512% # ( 110, 170 ] 6485 24.637% 31.149% ##### ( 170, 250 ] 4069 15.459% 46.607% ### ( 250, 380 ] 2739 10.406% 57.013% ## ( 380, 580 ] 2865 10.884% 67.898% ## ( 580, 870 ] 2361 8.970% 76.867% ## ( 870, 1300 ] 1896 7.203% 84.070% # ( 1300, 1900 ] 1314 4.992% 89.062% # ( 1900, 2900 ] 1142 4.339% 93.401% # ( 2900, 4400 ] 1144 4.346% 97.747% # ( 4400, 6600 ] 506 1.922% 99.669% ( 6600, 9900 ] 83 0.315% 99.985% ( 9900, 14000 ] 4 0.015% 100.000% ( 14000, 22000 ] 1 0.004% 100.004% ** DB Stats ** Uptime(secs): 7.4 total, 7.2 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 133.45 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.357 H:M:S, 45.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 137.03 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.357 H:M:S, 46.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/18-16:44:03 ... thread 0: (1000000,2000000) ops and (290912.0,188379.6) ops/second in (3.437466,10.616859) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/9 4.12 GB 163.6 1.3 0.0 1.3 3.1 1.8 0.0 1.7 98.6 239.4 13.42 10.23 33 0.407 1343K 1421 0.0 0.0 L1 7/7 867.32 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 15/0 955.32 MB 1.0 0.1 0.1 0.0 0.1 0.1 0.0 1.2 148.0 147.5 1.03 0.68 1 1.028 130K 0 0.0 0.0 L4 171/0 9.57 GB 1.0 4.9 2.3 2.6 4.8 2.2 0.0 2.1 109.6 107.7 45.37 23.03 26 1.745 665K 3020 0.0 0.0 L5 1132/0 93.14 GB 1.0 0.0 0.0 0.0 0.0 0.0 2.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1343/16 108.60 GB 0.0 6.3 2.4 3.9 8.1 4.1 2.6 4.4 107.8 137.9 59.81 33.94 60 0.997 2139K 4441 0.0 0.0 Int 0/0 0.00 KB 0.0 6.3 2.4 3.9 8.1 4.1 2.6 4.4 107.8 137.9 59.81 33.94 60 0.997 2139K 4441 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.3 2.4 3.9 6.2 2.3 0.0 0.0 124.1 122.4 51.95 28.72 32 1.624 2139K 4441 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 240.4 7.86 5.22 28 0.281 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 10.8 total, 10.6 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.06 GB write, 762.91 MB/s write, 6.29 GB read, 596.18 MB/s read, 59.8 seconds Interval compaction: 8.06 GB write, 776.82 MB/s write, 6.29 GB read, 607.05 MB/s read, 59.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 7, pending-compaction-bytes-delays: 15, pending-compaction-bytes-stops: 0, total-delays: 15, total-stops: 7, interval: 22 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 760667 Average: 9.8326 StdDev: 88.83 Min: 0 Median: 1.6640 Max: 13350 Percentiles: P50: 1.66 P75: 2.38 P99: 192.54 P99.9: 763.63 P99.99: 1228.91 ------------------------------------------------------ [ 0, 1 ] 100365 13.194% 13.194% ### ( 1, 2 ] 421645 55.431% 68.625% ########### ( 2, 3 ] 127055 16.703% 85.328% ### ( 3, 4 ] 40691 5.349% 90.678% # ( 4, 6 ] 38823 5.104% 95.782% # ( 6, 10 ] 12271 1.613% 97.395% ( 10, 15 ] 1919 0.252% 97.647% ( 15, 22 ] 268 0.035% 97.682% ( 22, 34 ] 40 0.005% 97.688% ( 34, 51 ] 13 0.002% 97.689% ( 51, 76 ] 27 0.004% 97.693% ( 76, 110 ] 1908 0.251% 97.944% ( 110, 170 ] 7014 0.922% 98.866% ( 170, 250 ] 3625 0.477% 99.342% ( 250, 380 ] 1926 0.253% 99.595% ( 380, 580 ] 1780 0.234% 99.829% ( 580, 870 ] 847 0.111% 99.941% ( 870, 1300 ] 448 0.059% 100.000% ( 1300, 1900 ] 207 0.027% 100.027% ( 1900, 2900 ] 142 0.019% 100.046% ( 2900, 4400 ] 110 0.014% 100.060% ( 4400, 6600 ] 29 0.004% 100.064% ( 6600, 9900 ] 8 0.001% 100.065% ( 9900, 14000 ] 2 0.000% 100.065% ** Level 1 read latency histogram (micros): Count: 702 Average: 423.9359 StdDev: 1142.11 Min: 77 Median: 178.8889 Max: 13473 Percentiles: P50: 178.89 P75: 321.80 P99: 4395.00 P99.9: 13424.36 P99.99: 13473.00 ------------------------------------------------------ ( 76, 110 ] 75 10.684% 10.684% ## ( 110, 170 ] 260 37.037% 47.721% ####### ( 170, 250 ] 144 20.513% 68.234% #### ( 250, 380 ] 86 12.251% 80.484% ## ( 380, 580 ] 59 8.405% 88.889% ## ( 580, 870 ] 24 3.419% 92.308% # ( 870, 1300 ] 23 3.276% 95.584% # ( 1300, 1900 ] 12 1.709% 97.293% ( 1900, 2900 ] 6 0.855% 98.148% ( 2900, 4400 ] 6 0.855% 99.003% ( 4400, 6600 ] 1 0.142% 99.145% ( 6600, 9900 ] 1 0.142% 99.288% ( 9900, 14000 ] 5 0.712% 100.000% ** Level 2 read latency histogram (micros): Count: 32712 Average: 8.2273 StdDev: 114.34 Min: 1 Median: 1.9870 Max: 12487 Percentiles: P50: 1.99 P75: 2.90 P99: 9.98 P99.9: 1261.07 P99.99: 4934.45 ------------------------------------------------------ [ 0, 1 ] 372 1.137% 1.137% ( 1, 2 ] 16194 49.505% 50.642% ########## ( 2, 3 ] 8857 27.076% 77.718% ##### ( 3, 4 ] 2916 8.914% 86.632% ## ( 4, 6 ] 3160 9.660% 96.292% ## ( 6, 10 ] 891 2.724% 99.016% # ( 10, 15 ] 142 0.434% 99.450% ( 15, 22 ] 12 0.037% 99.486% ( 22, 34 ] 2 0.006% 99.493% ( 51, 76 ] 1 0.003% 99.496% ( 110, 170 ] 2 0.006% 99.502% ( 170, 250 ] 4 0.012% 99.514% ( 250, 380 ] 18 0.055% 99.569% ( 380, 580 ] 35 0.107% 99.676% ( 580, 870 ] 36 0.110% 99.786% ( 870, 1300 ] 41 0.125% 99.911% ( 1300, 1900 ] 13 0.040% 99.951% ( 1900, 2900 ] 7 0.021% 99.972% ( 2900, 4400 ] 5 0.015% 99.988% ( 4400, 6600 ] 3 0.009% 99.997% ( 9900, 14000 ] 1 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 412372 Average: 11.3797 StdDev: 136.88 Min: 1 Median: 1.9838 Max: 10397 Percentiles: P50: 1.98 P75: 3.06 P99: 12.29 P99.9: 1284.48 P99.99: 1803.95 ------------------------------------------------------ [ 0, 1 ] 10707 2.596% 2.596% # ( 1, 2 ] 198704 48.186% 50.782% ########## ( 2, 3 ] 97208 23.573% 74.355% ##### ( 3, 4 ] 42573 10.324% 84.679% ## ( 4, 6 ] 45446 11.021% 95.700% ## ( 6, 10 ] 12650 3.068% 98.767% # ( 10, 15 ] 2096 0.508% 99.275% ( 15, 22 ] 316 0.077% 99.352% ( 22, 34 ] 40 0.010% 99.362% ( 34, 51 ] 20 0.005% 99.367% ( 51, 76 ] 26 0.006% 99.373% ( 76, 110 ] 109 0.026% 99.399% ( 110, 170 ] 379 0.092% 99.491% ( 170, 250 ] 233 0.057% 99.548% ( 250, 380 ] 241 0.058% 99.606% ( 380, 580 ] 285 0.069% 99.675% ( 580, 870 ] 436 0.106% 99.781% ( 870, 1300 ] 509 0.123% 99.904% ( 1300, 1900 ] 420 0.102% 100.006% ( 1900, 2900 ] 270 0.065% 100.072% ( 2900, 4400 ] 172 0.042% 100.113% ( 4400, 6600 ] 60 0.015% 100.128% ( 6600, 9900 ] 8 0.002% 100.130% ( 9900, 14000 ] 1 0.000% 100.130% ** Level 4 read latency histogram (micros): Count: 619125 Average: 19.5703 StdDev: 181.37 Min: 1 Median: 2.0172 Max: 12238 Percentiles: P50: 2.02 P75: 3.18 P99: 286.81 P99.9: 1664.44 P99.99: 2103.94 ------------------------------------------------------ [ 0, 1 ] 14614 2.360% 2.360% ( 1, 2 ] 292429 47.233% 49.593% ######### ( 2, 3 ] 146116 23.600% 73.193% ##### ( 3, 4 ] 63467 10.251% 83.445% ## ( 4, 6 ] 67564 10.913% 94.357% ## ( 6, 10 ] 18324 2.960% 97.317% # ( 10, 15 ] 3270 0.528% 97.845% ( 15, 22 ] 481 0.078% 97.923% ( 22, 34 ] 70 0.011% 97.934% ( 34, 51 ] 43 0.007% 97.941% ( 51, 76 ] 34 0.005% 97.947% ( 76, 110 ] 1050 0.170% 98.116% ( 110, 170 ] 3317 0.536% 98.652% ( 170, 250 ] 1824 0.295% 98.947% ( 250, 380 ] 1168 0.189% 99.135% ( 380, 580 ] 1333 0.215% 99.351% ( 580, 870 ] 1326 0.214% 99.565% ( 870, 1300 ] 1429 0.231% 99.796% ( 1300, 1900 ] 1065 0.172% 99.968% ( 1900, 2900 ] 682 0.110% 100.078% ( 2900, 4400 ] 483 0.078% 100.156% ( 4400, 6600 ] 160 0.026% 100.182% ( 6600, 9900 ] 21 0.003% 100.185% ( 9900, 14000 ] 1 0.000% 100.185% ** Level 5 read latency histogram (micros): Count: 69064 Average: 420.2841 StdDev: 755.20 Min: 72 Median: 182.2200 Max: 15211 Percentiles: P50: 182.22 P75: 359.72 P99: 4266.73 P99.9: 7154.08 P99.99: 9625.41 ------------------------------------------------------ ( 51, 76 ] 7 0.010% 0.010% ( 76, 110 ] 7309 10.583% 10.593% ## ( 110, 170 ] 25191 36.475% 47.068% ####### ( 170, 250 ] 13257 19.195% 66.263% #### ( 250, 380 ] 7149 10.351% 76.614% ## ( 380, 580 ] 6404 9.273% 85.887% ## ( 580, 870 ] 3459 5.008% 90.895% # ( 870, 1300 ] 2056 2.977% 93.872% # ( 1300, 1900 ] 1327 1.921% 95.794% ( 1900, 2900 ] 1172 1.697% 97.491% ( 2900, 4400 ] 1144 1.656% 99.147% ( 4400, 6600 ] 506 0.733% 99.880% ( 6600, 9900 ] 83 0.120% 100.000% ( 9900, 14000 ] 4 0.006% 100.006% ( 14000, 22000 ] 1 0.001% 100.007% ** DB Stats ** Uptime(secs): 10.8 total, 3.4 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 182.04 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.357 H:M:S, 31.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 286.29 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/18-16:44:31 ... thread 0: (1000000,3000000) ops and (36159.0,78385.2) ops/second in (27.655666,38.272525) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/7 2.82 GB 12.6 4.0 0.0 4.0 6.8 2.8 0.0 2.4 150.9 257.2 27.13 23.21 55 0.493 4156K 12K 0.0 0.0 L1 22/22 2.59 GB 0.0 3.1 2.3 0.8 3.1 2.2 0.0 1.4 94.5 93.4 33.85 18.35 1 33.851 3250K 37K 0.0 0.0 L3 14/0 882.35 MB 0.9 0.8 0.5 0.3 0.8 0.5 0.1 1.6 195.9 194.7 4.20 3.57 4 1.050 811K 4743 0.0 0.0 L4 178/0 9.74 GB 1.0 6.4 2.8 3.6 6.3 2.7 0.0 2.2 124.3 122.1 53.01 29.94 35 1.514 1200K 6699 0.0 0.0 L5 1136/0 93.51 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1359/29 109.51 GB 0.0 14.4 5.6 8.8 17.0 8.3 3.1 6.0 124.4 147.5 118.19 75.07 95 1.244 9418K 61K 0.0 0.0 Int 0/0 0.00 KB 0.0 14.4 5.6 8.8 17.0 8.3 3.1 6.0 124.4 147.5 118.19 75.07 95 1.244 9418K 61K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.4 5.6 8.8 14.2 5.4 0.0 0.0 137.6 136.0 106.89 67.02 51 2.096 9418K 61K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 256.6 11.29 8.05 44 0.257 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 38.5 total, 38.3 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 17.02 GB write, 453.15 MB/s write, 14.36 GB read, 382.23 MB/s read, 118.2 seconds Interval compaction: 17.02 GB write, 455.44 MB/s write, 14.36 GB read, 384.16 MB/s read, 118.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 3, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 3, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 7, pending-compaction-bytes-delays: 37, pending-compaction-bytes-stops: 0, total-delays: 40, total-stops: 7, interval: 47 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1757537 Average: 20.7148 StdDev: 85.71 Min: 0 Median: 1.6116 Max: 13350 Percentiles: P50: 1.61 P75: 2.42 P99: 270.06 P99.9: 713.22 P99.99: 1004.00 ------------------------------------------------------ [ 0, 1 ] 320753 18.250% 18.250% #### ( 1, 2 ] 912312 51.909% 70.159% ########## ( 2, 3 ] 204038 11.609% 81.768% ## ( 3, 4 ] 65288 3.715% 85.483% # ( 4, 6 ] 54621 3.108% 88.591% # ( 6, 10 ] 16720 0.951% 89.542% ( 10, 15 ] 2927 0.167% 89.708% ( 15, 22 ] 418 0.024% 89.732% ( 22, 34 ] 57 0.003% 89.735% ( 34, 51 ] 16 0.001% 89.736% ( 51, 76 ] 77 0.004% 89.741% ( 76, 110 ] 34729 1.976% 91.717% ( 110, 170 ] 100383 5.712% 97.428% # ( 170, 250 ] 26175 1.489% 98.918% ( 250, 380 ] 9387 0.534% 99.452% ( 380, 580 ] 6748 0.384% 99.836% ( 580, 870 ] 2463 0.140% 99.976% ( 870, 1300 ] 803 0.046% 100.022% ( 1300, 1900 ] 299 0.017% 100.039% ( 1900, 2900 ] 234 0.013% 100.052% ( 2900, 4400 ] 178 0.010% 100.062% ( 4400, 6600 ] 31 0.002% 100.064% ( 6600, 9900 ] 9 0.001% 100.064% ( 9900, 14000 ] 2 0.000% 100.064% ** Level 1 read latency histogram (micros): Count: 325861 Average: 7.4908 StdDev: 65.37 Min: 0 Median: 2.3991 Max: 13473 Percentiles: P50: 2.40 P75: 3.93 P99: 147.08 P99.9: 484.39 P99.99: 1168.59 ------------------------------------------------------ [ 0, 1 ] 17281 5.303% 5.303% # ( 1, 2 ] 121505 37.287% 42.591% ####### ( 2, 3 ] 60500 18.566% 61.157% #### ( 3, 4 ] 48519 14.889% 76.046% ### ( 4, 6 ] 60410 18.539% 94.585% #### ( 6, 10 ] 7906 2.426% 97.011% ( 10, 15 ] 2129 0.653% 97.664% ( 15, 22 ] 796 0.244% 97.909% ( 22, 34 ] 28 0.009% 97.917% ( 34, 51 ] 2 0.001% 97.918% ( 51, 76 ] 3 0.001% 97.919% ( 76, 110 ] 1238 0.380% 98.299% ( 110, 170 ] 3698 1.135% 99.434% ( 170, 250 ] 977 0.300% 99.733% ( 250, 380 ] 397 0.122% 99.855% ( 380, 580 ] 280 0.086% 99.941% ( 580, 870 ] 124 0.038% 99.979% ( 870, 1300 ] 51 0.016% 99.995% ( 1300, 1900 ] 20 0.006% 100.001% ( 1900, 2900 ] 11 0.003% 100.004% ( 2900, 4400 ] 10 0.003% 100.007% ( 4400, 6600 ] 1 0.000% 100.008% ( 6600, 9900 ] 1 0.000% 100.008% ( 9900, 14000 ] 5 0.002% 100.010% ** Level 2 read latency histogram (micros): Count: 32712 Average: 8.2273 StdDev: 114.34 Min: 1 Median: 1.9870 Max: 12487 Percentiles: P50: 1.99 P75: 2.90 P99: 9.98 P99.9: 1261.07 P99.99: 4934.45 ------------------------------------------------------ [ 0, 1 ] 372 1.137% 1.137% ( 1, 2 ] 16194 49.505% 50.642% ########## ( 2, 3 ] 8857 27.076% 77.718% ##### ( 3, 4 ] 2916 8.914% 86.632% ## ( 4, 6 ] 3160 9.660% 96.292% ## ( 6, 10 ] 891 2.724% 99.016% # ( 10, 15 ] 142 0.434% 99.450% ( 15, 22 ] 12 0.037% 99.486% ( 22, 34 ] 2 0.006% 99.493% ( 51, 76 ] 1 0.003% 99.496% ( 110, 170 ] 2 0.006% 99.502% ( 170, 250 ] 4 0.012% 99.514% ( 250, 380 ] 18 0.055% 99.569% ( 380, 580 ] 35 0.107% 99.676% ( 580, 870 ] 36 0.110% 99.786% ( 870, 1300 ] 41 0.125% 99.911% ( 1300, 1900 ] 13 0.040% 99.951% ( 1900, 2900 ] 7 0.021% 99.972% ( 2900, 4400 ] 5 0.015% 99.988% ( 4400, 6600 ] 3 0.009% 99.997% ( 9900, 14000 ] 1 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 639043 Average: 10.1558 StdDev: 112.34 Min: 1 Median: 1.8873 Max: 10397 Percentiles: P50: 1.89 P75: 2.89 P99: 132.27 P99.9: 965.79 P99.99: 1521.84 ------------------------------------------------------ [ 0, 1 ] 28024 4.385% 4.385% # ( 1, 2 ] 328520 51.408% 55.793% ########## ( 2, 3 ] 137529 21.521% 77.315% #### ( 3, 4 ] 57344 8.973% 86.288% ## ( 4, 6 ] 59281 9.277% 95.564% ## ( 6, 10 ] 15889 2.486% 98.051% ( 10, 15 ] 2513 0.393% 98.444% ( 15, 22 ] 369 0.058% 98.502% ( 22, 34 ] 43 0.007% 98.509% ( 34, 51 ] 25 0.004% 98.512% ( 51, 76 ] 29 0.005% 98.517% ( 76, 110 ] 1476 0.231% 98.748% ( 110, 170 ] 4339 0.679% 99.427% ( 170, 250 ] 1197 0.187% 99.614% ( 250, 380 ] 629 0.098% 99.713% ( 380, 580 ] 537 0.084% 99.797% ( 580, 870 ] 541 0.085% 99.881% ( 870, 1300 ] 534 0.084% 99.965% ( 1300, 1900 ] 433 0.068% 100.033% ( 1900, 2900 ] 272 0.043% 100.075% ( 2900, 4400 ] 174 0.027% 100.102% ( 4400, 6600 ] 60 0.009% 100.112% ( 6600, 9900 ] 8 0.001% 100.113% ( 9900, 14000 ] 1 0.000% 100.113% ** Level 4 read latency histogram (micros): Count: 944218 Average: 24.7188 StdDev: 154.94 Min: 0 Median: 2.0269 Max: 12238 Percentiles: P50: 2.03 P75: 3.50 P99: 317.12 P99.9: 1265.62 P99.99: 1689.56 ------------------------------------------------------ [ 0, 1 ] 29015 3.073% 3.073% # ( 1, 2 ] 437732 46.359% 49.432% ######### ( 2, 3 ] 199140 21.090% 70.523% #### ( 3, 4 ] 84596 8.959% 79.482% ## ( 4, 6 ] 87829 9.302% 88.784% ## ( 6, 10 ] 24819 2.629% 91.412% # ( 10, 15 ] 4024 0.426% 91.838% ( 15, 22 ] 628 0.067% 91.905% ( 22, 34 ] 81 0.009% 91.914% ( 34, 51 ] 46 0.005% 91.918% ( 51, 76 ] 61 0.006% 91.925% ( 76, 110 ] 14698 1.557% 93.481% ( 110, 170 ] 39613 4.195% 97.677% # ( 170, 250 ] 10462 1.108% 98.785% ( 250, 380 ] 3935 0.417% 99.202% ( 380, 580 ] 3173 0.336% 99.538% ( 580, 870 ] 1981 0.210% 99.747% ( 870, 1300 ] 1566 0.166% 99.913% ( 1300, 1900 ] 1116 0.118% 100.031% ( 1900, 2900 ] 699 0.074% 100.105% ( 2900, 4400 ] 494 0.052% 100.158% ( 4400, 6600 ] 160 0.017% 100.175% ( 6600, 9900 ] 21 0.002% 100.177% ( 9900, 14000 ] 1 0.000% 100.177% ** Level 5 read latency histogram (micros): Count: 500735 Average: 198.3319 StdDev: 314.36 Min: 72 Median: 143.2212 Max: 15211 Percentiles: P50: 143.22 P75: 174.33 P99: 1249.95 P99.9: 4763.50 P99.99: 7591.05 ------------------------------------------------------ ( 51, 76 ] 113 0.023% 0.023% ( 76, 110 ] 99650 19.901% 19.923% #### ( 110, 170 ] 272003 54.321% 74.244% ########### ( 170, 250 ] 69931 13.966% 88.210% ### ( 250, 380 ] 25222 5.037% 93.247% # ( 380, 580 ] 18676 3.730% 96.976% # ( 580, 870 ] 7534 1.505% 98.481% ( 870, 1300 ] 2941 0.587% 99.068% ( 1300, 1900 ] 1532 0.306% 99.374% ( 1900, 2900 ] 1329 0.265% 99.640% ( 2900, 4400 ] 1219 0.243% 99.883% ( 4400, 6600 ] 510 0.102% 99.985% ( 6600, 9900 ] 83 0.017% 100.002% ( 9900, 14000 ] 4 0.001% 100.002% ( 14000, 22000 ] 1 0.000% 100.003% ** DB Stats ** Uptime(secs): 38.5 total, 27.7 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 76.75 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:26.723 H:M:S, 69.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 35.59 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:23.366 H:M:S, 84.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 216.939 micros/op 18430 ops/sec 60.065 seconds 1106996 operations; 17.9 MB/s (282999 of 282999 found) Microseconds per read: Count: 1106996 Average: 216.9415 StdDev: 594.70 Min: 1 Median: 165.5165 Max: 80976 Percentiles: P50: 165.52 P75: 225.97 P99: 805.07 P99.9: 8296.68 P99.99: 26933.11 ------------------------------------------------------ [ 0, 1 ] 1 0.000% 0.000% ( 1, 2 ] 9 0.001% 0.001% ( 2, 3 ] 52 0.005% 0.006% ( 3, 4 ] 66 0.006% 0.012% ( 4, 6 ] 81 0.007% 0.019% ( 6, 10 ] 440 0.040% 0.059% ( 10, 15 ] 4412 0.399% 0.457% ( 15, 22 ] 5090 0.460% 0.917% ( 22, 34 ] 9226 0.833% 1.750% ( 34, 51 ] 2322 0.210% 1.960% ( 51, 76 ] 258 0.023% 1.983% ( 76, 110 ] 34205 3.090% 5.073% # ( 110, 170 ] 537501 48.555% 53.628% ########## ( 170, 250 ] 338144 30.546% 84.174% ###### ( 250, 380 ] 110278 9.962% 94.136% ## ( 380, 580 ] 42361 3.827% 97.963% # ( 580, 870 ] 14792 1.336% 99.299% ( 870, 1300 ] 4017 0.363% 99.662% ( 1300, 1900 ] 1165 0.105% 99.767% ( 1900, 2900 ] 738 0.067% 99.834% ( 2900, 4400 ] 475 0.043% 99.877% ( 4400, 6600 ] 165 0.015% 99.892% ( 6600, 9900 ] 177 0.016% 99.908% ( 9900, 14000 ] 421 0.038% 99.946% ( 14000, 22000 ] 431 0.039% 99.985% ( 22000, 33000 ] 130 0.012% 99.996% ( 33000, 50000 ] 29 0.003% 99.999% ( 50000, 75000 ] 9 0.001% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 14388516 rocksdb.bloom.filter.full.positive COUNT : 1247100 rocksdb.bloom.filter.full.true.positive COUNT : 1106595 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 401 rocksdb.memtable.miss COUNT : 1106595 rocksdb.l0.hit COUNT : 43785 rocksdb.l1.hit COUNT : 19222 rocksdb.l2andup.hit COUNT : 1043588 rocksdb.compaction.key.drop.new COUNT : 177583 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3383476 rocksdb.number.keys.read COUNT : 1106996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3491747232 rocksdb.bytes.read COUNT : 1106996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1554 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 46295813 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 16587213751 rocksdb.compact.write.bytes COUNT : 16399506063 rocksdb.flush.write.bytes COUNT : 3435107042 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 37 rocksdb.number.superversion_acquires COUNT : 664 rocksdb.number.superversion_releases COUNT : 639 rocksdb.number.superversion_cleanups COUNT : 166 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 154 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3407288320 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1224704 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 21429829206 rocksdb.non.last.level.read.count COUNT : 5032309 rocksdb.block.checksum.compute.count COUNT : 5274401 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 164.639487 P95 : 423.509806 P99 : 804.212747 P100 : 80973.000000 COUNT : 1106996 SUM : 239153225 rocksdb.db.write.micros P50 : 1.590219 P95 : 5.568184 P99 : 894.413022 P100 : 434844.000000 COUNT : 3383476 SUM : 55457945 rocksdb.compaction.times.micros P50 : 1225000.000000 P95 : 3390000.000000 P99 : 33850805.000000 P100 : 33850805.000000 COUNT : 53 SUM : 108766853 rocksdb.compaction.times.cpu_micros P50 : 900476.190476 P95 : 1824166.666667 P99 : 17760000.000000 P100 : 18352813.000000 COUNT : 53 SUM : 68864514 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 797.500000 P95 : 16000.000000 P99 : 51872.000000 P100 : 51872.000000 COUNT : 50 SUM : 143369 rocksdb.compaction.outfile.sync.micros P50 : 9350.000000 P95 : 191454.545455 P99 : 244109.090909 P100 : 259306.000000 COUNT : 181 SUM : 8102924 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 161.951220 P95 : 4491.666667 P99 : 6178.333333 P100 : 6317.000000 COUNT : 115 SUM : 90200 rocksdb.table.open.io.micros P50 : 8746.813187 P95 : 19815.929204 P99 : 22316.250000 P100 : 31780.000000 COUNT : 1554 SUM : 13951003 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.845447 P95 : 111.622676 P99 : 214.525669 P100 : 10565.000000 COUNT : 4016224 SUM : 69824752 rocksdb.read.block.get.micros P50 : 139.655007 P95 : 321.384887 P99 : 735.546544 P100 : 15672.000000 COUNT : 1256609 SUM : 204895546 rocksdb.write.raw.block.micros P50 : 0.540016 P95 : 1.456724 P99 : 4.076331 P100 : 14733.000000 COUNT : 4804325 SUM : 11633208 rocksdb.numfiles.in.singlecompaction P50 : 3.148148 P95 : 5.914286 P99 : 26.080000 P100 : 29.000000 COUNT : 66 SUM : 267 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.567782 P95 : 1183.993941 P99 : 2501.545136 P100 : 434639.000000 COUNT : 297896 SUM : 46324231 rocksdb.sst.read.micros P50 : 2.280005 P95 : 180.866180 P99 : 450.499547 P100 : 15211.000000 COUNT : 5032344 SUM : 259449931 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1106996 SUM : 1106996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3383476 SUM : 3491747232 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 216511.627907 P95 : 642500.000000 P99 : 1030000.000000 P100 : 1039803.000000 COUNT : 50 SUM : 12496497 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 195 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0