[2024-12-17 08:45:19.541621] Starting SPDK v24.09.1-pre git sha1 b18e1bd62 / DPDK 23.11.0 initialization... [2024-12-17 08:45:19.541884] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3527396 ] Set seed to 1734421519534968 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-17 08:45:19.648139] app.c: 917:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-17 08:45:19.685692] app.c: 610:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-17 08:45:19.685736] app.c: 611:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3527396' to capture a snapshot of events at runtime. [2024-12-17 08:45:19.685747] app.c: 616:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-17 08:45:19.685757] app.c: 617:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-17 08:45:19.685764] app.c: 618:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3527396 for offline analysis/debug. [2024-12-17 08:45:19.685844] reactor.c: 990:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Tue Dec 17 08:45:23 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/17-08:45:32 ... thread 0: (1000000,1000000) ops and (114915.5,114915.5) ops/second in (8.702049,8.702049) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/7 2.05 GB 50.4 0.3 0.0 0.3 1.2 0.9 0.0 1.4 37.2 141.2 8.47 3.81 11 0.770 319K 365 0.0 0.0 L1 6/6 767.85 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 20/5 1.31 GB 0.9 2.0 1.2 0.8 2.0 1.2 0.0 1.6 110.3 109.1 18.99 9.69 8 2.373 1416K 3276 0.0 0.0 L4 167/11 9.66 GB 1.0 5.1 3.6 1.5 5.1 3.6 0.0 1.4 106.8 106.0 49.08 24.52 32 1.534 1955K 560 0.0 0.0 L5 1120/0 92.03 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1323/29 105.80 GB 0.0 7.5 4.9 2.6 8.3 5.7 3.6 9.6 100.0 110.7 76.54 38.02 51 1.501 3691K 4201 0.0 0.0 Int 0/0 0.00 KB 0.0 7.5 4.9 2.6 8.3 5.7 3.6 9.6 100.0 110.7 76.54 38.02 51 1.501 3691K 4201 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 7.5 4.9 2.6 7.4 4.8 0.0 0.0 109.4 108.6 69.92 35.56 41 1.705 3691K 4201 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 133.3 6.61 2.46 10 0.661 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.9 total, 8.7 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.27 GB write, 952.66 MB/s write, 7.47 GB read, 860.57 MB/s read, 76.5 seconds Interval compaction: 8.27 GB write, 973.38 MB/s write, 7.47 GB read, 879.28 MB/s read, 76.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 25, pending-compaction-bytes-delays: 4, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 25, interval: 28 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 409597 Average: 11.8744 StdDev: 184.20 Min: 1 Median: 2.0682 Max: 58174 Percentiles: P50: 2.07 P75: 3.00 P99: 11.38 P99.9: 1989.32 P99.99: 3774.71 ------------------------------------------------------ [ 0, 1 ] 9577 2.338% 2.338% ( 1, 2 ] 187708 45.827% 48.166% ######### ( 2, 3 ] 110245 26.915% 75.081% ##### ( 3, 4 ] 39550 9.656% 84.737% ## ( 4, 6 ] 46852 11.439% 96.176% ## ( 6, 10 ] 11072 2.703% 98.879% # ( 10, 15 ] 1814 0.443% 99.322% ( 15, 22 ] 248 0.061% 99.382% ( 22, 34 ] 23 0.006% 99.388% ( 34, 51 ] 23 0.006% 99.393% ( 51, 76 ] 24 0.006% 99.399% ( 76, 110 ] 18 0.004% 99.404% ( 110, 170 ] 78 0.019% 99.423% ( 170, 250 ] 157 0.038% 99.461% ( 250, 380 ] 232 0.057% 99.518% ( 380, 580 ] 339 0.083% 99.600% ( 580, 870 ] 450 0.110% 99.710% ( 870, 1300 ] 437 0.107% 99.817% ( 1300, 1900 ] 319 0.078% 99.895% ( 1900, 2900 ] 262 0.064% 99.959% ( 2900, 4400 ] 223 0.054% 100.013% ( 4400, 6600 ] 70 0.017% 100.030% ( 6600, 9900 ] 18 0.004% 100.035% ( 9900, 14000 ] 3 0.001% 100.035% ( 14000, 22000 ] 2 0.000% 100.036% ( 22000, 33000 ] 1 0.000% 100.036% ( 50000, 75000 ] 1 0.000% 100.036% ** Level 1 read latency histogram (micros): Count: 150 Average: 1504.6333 StdDev: 1620.05 Min: 132 Median: 883.8710 Max: 8360 Percentiles: P50: 883.87 P75: 1675.00 P99: 8360.00 P99.9: 8360.00 P99.99: 8360.00 ------------------------------------------------------ ( 110, 170 ] 3 2.000% 2.000% ( 170, 250 ] 5 3.333% 5.333% # ( 250, 380 ] 15 10.000% 15.333% ## ( 380, 580 ] 22 14.667% 30.000% ### ( 580, 870 ] 29 19.333% 49.333% #### ( 870, 1300 ] 31 20.667% 70.000% #### ( 1300, 1900 ] 12 8.000% 78.000% ## ( 1900, 2900 ] 9 6.000% 84.000% # ( 2900, 4400 ] 16 10.667% 94.667% ## ( 4400, 6600 ] 4 2.667% 97.333% # ( 6600, 9900 ] 4 2.667% 100.000% # ** Level 2 read latency histogram (micros): Count: 256520 Average: 9.9542 StdDev: 135.92 Min: 1 Median: 1.9844 Max: 8849 Percentiles: P50: 1.98 P75: 3.01 P99: 10.25 P99.9: 1397.56 P99.99: 2650.82 ------------------------------------------------------ [ 0, 1 ] 7349 2.865% 2.865% # ( 1, 2 ] 122830 47.883% 50.748% ########## ( 2, 3 ] 61829 24.103% 74.851% ##### ( 3, 4 ] 26853 10.468% 85.319% ## ( 4, 6 ] 27510 10.724% 96.044% ## ( 6, 10 ] 7522 2.932% 98.976% # ( 10, 15 ] 1220 0.476% 99.452% ( 15, 22 ] 187 0.073% 99.524% ( 22, 34 ] 47 0.018% 99.543% ( 34, 51 ] 13 0.005% 99.548% ( 51, 76 ] 11 0.004% 99.552% ( 76, 110 ] 10 0.004% 99.556% ( 110, 170 ] 48 0.019% 99.575% ( 170, 250 ] 60 0.023% 99.598% ( 250, 380 ] 127 0.050% 99.648% ( 380, 580 ] 156 0.061% 99.708% ( 580, 870 ] 233 0.091% 99.799% ( 870, 1300 ] 231 0.090% 99.889% ( 1300, 1900 ] 169 0.066% 99.955% ( 1900, 2900 ] 119 0.046% 100.002% ( 2900, 4400 ] 111 0.043% 100.045% ( 4400, 6600 ] 41 0.016% 100.061% ( 6600, 9900 ] 12 0.005% 100.065% ** Level 3 read latency histogram (micros): Count: 1014398 Average: 12.6391 StdDev: 195.96 Min: 0 Median: 2.0132 Max: 55559 Percentiles: P50: 2.01 P75: 3.00 P99: 9.75 P99.9: 1079.66 P99.99: 1572.46 ------------------------------------------------------ [ 0, 1 ] 15602 1.538% 1.538% ( 1, 2 ] 488220 48.129% 49.667% ########## ( 2, 3 ] 256792 25.315% 74.982% ##### ( 3, 4 ] 109807 10.825% 85.807% ## ( 4, 6 ] 109932 10.837% 96.644% ## ( 6, 10 ] 25474 2.511% 99.155% # ( 10, 15 ] 4120 0.406% 99.561% ( 15, 22 ] 557 0.055% 99.616% ( 22, 34 ] 80 0.008% 99.624% ( 34, 51 ] 62 0.006% 99.630% ( 51, 76 ] 52 0.005% 99.635% ( 76, 110 ] 27 0.003% 99.638% ( 110, 170 ] 51 0.005% 99.643% ( 170, 250 ] 131 0.013% 99.656% ( 250, 380 ] 327 0.032% 99.688% ( 380, 580 ] 607 0.060% 99.748% ( 580, 870 ] 1019 0.100% 99.848% ( 870, 1300 ] 1078 0.106% 99.955% ( 1300, 1900 ] 794 0.078% 100.033% ( 1900, 2900 ] 748 0.074% 100.107% ( 2900, 4400 ] 760 0.075% 100.182% ( 4400, 6600 ] 216 0.021% 100.203% ( 6600, 9900 ] 37 0.004% 100.207% ( 9900, 14000 ] 3 0.000% 100.207% ( 14000, 22000 ] 6 0.001% 100.207% ( 22000, 33000 ] 3 0.000% 100.208% ( 33000, 50000 ] 2 0.000% 100.208% ( 50000, 75000 ] 2 0.000% 100.208% ** Level 4 read latency histogram (micros): Count: 420065 Average: 25.1095 StdDev: 278.09 Min: 1 Median: 1.9992 Max: 54619 Percentiles: P50: 2.00 P75: 3.03 P99: 634.82 P99.9: 3056.60 P99.99: 3968.31 ------------------------------------------------------ [ 0, 1 ] 7362 1.753% 1.753% ( 1, 2 ] 202830 48.285% 50.038% ########## ( 2, 3 ] 103604 24.664% 74.702% ##### ( 3, 4 ] 45395 10.807% 85.508% ## ( 4, 6 ] 43279 10.303% 95.811% ## ( 6, 10 ] 9724 2.315% 98.126% ( 10, 15 ] 1423 0.339% 98.465% ( 15, 22 ] 199 0.047% 98.512% ( 22, 34 ] 22 0.005% 98.518% ( 34, 51 ] 30 0.007% 98.525% ( 51, 76 ] 26 0.006% 98.531% ( 76, 110 ] 18 0.004% 98.535% ( 110, 170 ] 111 0.026% 98.562% ( 170, 250 ] 294 0.070% 98.632% ( 250, 380 ] 457 0.109% 98.740% ( 380, 580 ] 862 0.205% 98.946% ( 580, 870 ] 1208 0.288% 99.233% ( 870, 1300 ] 1237 0.294% 99.528% ( 1300, 1900 ] 827 0.197% 99.725% ( 1900, 2900 ] 672 0.160% 99.885% ( 2900, 4400 ] 622 0.148% 100.033% ( 4400, 6600 ] 179 0.043% 100.075% ( 6600, 9900 ] 22 0.005% 100.080% ( 9900, 14000 ] 3 0.001% 100.081% ( 14000, 22000 ] 6 0.001% 100.083% ( 22000, 33000 ] 1 0.000% 100.083% ( 33000, 50000 ] 3 0.001% 100.084% ( 50000, 75000 ] 1 0.000% 100.084% ** Level 5 read latency histogram (micros): Count: 17136 Average: 1474.5940 StdDev: 1573.97 Min: 79 Median: 961.4956 Max: 57960 Percentiles: P50: 961.50 P75: 1889.81 P99: 6444.42 P99.9: 11577.49 P99.99: 35434.40 ------------------------------------------------------ ( 76, 110 ] 66 0.385% 0.385% ( 110, 170 ] 357 2.083% 2.468% ( 170, 250 ] 732 4.272% 6.740% # ( 250, 380 ] 1363 7.954% 14.694% ## ( 380, 580 ] 2303 13.440% 28.134% ### ( 580, 870 ] 3121 18.213% 46.347% #### ( 870, 1300 ] 2942 17.169% 63.515% ### ( 1300, 1900 ] 2002 11.683% 75.198% ## ( 1900, 2900 ] 1666 9.722% 84.921% ## ( 2900, 4400 ] 1764 10.294% 95.215% ## ( 4400, 6600 ] 698 4.073% 99.288% # ( 6600, 9900 ] 102 0.595% 99.883% ( 9900, 14000 ] 7 0.041% 99.924% ( 14000, 22000 ] 5 0.029% 99.953% ( 22000, 33000 ] 6 0.035% 99.988% ( 33000, 50000 ] 2 0.012% 100.000% ( 50000, 75000 ] 1 0.006% 100.006% ** DB Stats ** Uptime(secs): 8.9 total, 8.7 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 110.67 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.864 H:M:S, 54.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 113.06 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.864 H:M:S, 55.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 7 num-running-flushes: 1 2024/12/17-08:45:36 ... thread 0: (1000000,2000000) ops and (285111.5,163807.6) ops/second in (3.507400,12.209449) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/7 2.97 GB 157.3 1.5 0.0 1.5 3.3 1.8 0.0 1.8 89.7 198.0 16.86 11.23 30 0.562 1535K 1730 0.0 0.0 L1 6/6 767.85 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 15/0 877.22 MB 0.9 2.0 1.2 0.8 2.0 1.2 0.0 1.6 110.3 109.1 18.99 9.69 8 2.373 1416K 3276 0.0 0.0 L4 167/0 9.73 GB 1.0 5.9 4.1 1.9 5.9 4.0 0.0 1.4 112.9 111.9 53.98 28.30 37 1.459 2329K 560 0.0 0.0 L5 1124/0 92.40 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1325/13 106.70 GB 0.0 9.5 5.3 4.2 11.2 7.0 4.0 6.3 108.0 127.4 89.82 49.22 75 1.198 5280K 5566 0.0 0.0 Int 0/0 0.00 KB 0.0 9.5 5.3 4.2 11.2 7.0 4.0 6.3 108.0 127.4 89.82 49.22 75 1.198 5280K 5566 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.5 5.3 4.2 9.4 5.2 0.0 0.0 121.5 120.5 79.81 44.03 50 1.596 5280K 5566 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 182.5 10.01 5.20 25 0.400 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.4 total, 12.2 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.18 GB write, 923.15 MB/s write, 9.47 GB read, 782.16 MB/s read, 89.8 seconds Interval compaction: 11.18 GB write, 937.46 MB/s write, 9.47 GB read, 794.29 MB/s read, 89.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 25, pending-compaction-bytes-delays: 4, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 25, interval: 28 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 707339 Average: 10.9941 StdDev: 146.79 Min: 1 Median: 1.8156 Max: 58174 Percentiles: P50: 1.82 P75: 2.73 P99: 161.13 P99.9: 1306.33 P99.99: 2880.82 ------------------------------------------------------ [ 0, 1 ] 56696 8.015% 8.015% ## ( 1, 2 ] 364103 51.475% 59.490% ########## ( 2, 3 ] 150659 21.299% 80.790% #### ( 3, 4 ] 52084 7.363% 88.153% # ( 4, 6 ] 57432 8.119% 96.273% ## ( 6, 10 ] 12993 1.837% 98.110% ( 10, 15 ] 2025 0.286% 98.396% ( 15, 22 ] 275 0.039% 98.435% ( 22, 34 ] 28 0.004% 98.439% ( 34, 51 ] 35 0.005% 98.444% ( 51, 76 ] 30 0.004% 98.448% ( 76, 110 ] 1046 0.148% 98.596% ( 110, 170 ] 3357 0.475% 99.070% ( 170, 250 ] 1976 0.279% 99.350% ( 250, 380 ] 1255 0.177% 99.527% ( 380, 580 ] 1221 0.173% 99.700% ( 580, 870 ] 860 0.122% 99.821% ( 870, 1300 ] 554 0.078% 99.900% ( 1300, 1900 ] 347 0.049% 99.949% ( 1900, 2900 ] 299 0.042% 99.991% ( 2900, 4400 ] 242 0.034% 100.025% ( 4400, 6600 ] 71 0.010% 100.035% ( 6600, 9900 ] 19 0.003% 100.038% ( 9900, 14000 ] 3 0.000% 100.038% ( 14000, 22000 ] 2 0.000% 100.039% ( 22000, 33000 ] 1 0.000% 100.039% ( 50000, 75000 ] 1 0.000% 100.039% ** Level 1 read latency histogram (micros): Count: 68431 Average: 9.2910 StdDev: 113.05 Min: 1 Median: 2.3404 Max: 8360 Percentiles: P50: 2.34 P75: 3.73 P99: 113.51 P99.9: 1177.12 P99.99: 4469.04 ------------------------------------------------------ [ 0, 1 ] 1364 1.993% 1.993% ( 1, 2 ] 26880 39.280% 41.274% ######## ( 2, 3 ] 17542 25.635% 66.908% ##### ( 3, 4 ] 7560 11.048% 77.956% ## ( 4, 6 ] 10586 15.470% 93.425% ### ( 6, 10 ] 3060 4.472% 97.897% # ( 10, 15 ] 450 0.658% 98.555% ( 15, 22 ] 198 0.289% 98.844% ( 22, 34 ] 18 0.026% 98.870% ( 34, 51 ] 5 0.007% 98.878% ( 51, 76 ] 1 0.001% 98.879% ( 76, 110 ] 69 0.101% 98.980% ( 110, 170 ] 234 0.342% 99.322% ( 170, 250 ] 118 0.172% 99.494% ( 250, 380 ] 99 0.145% 99.639% ( 380, 580 ] 97 0.142% 99.781% ( 580, 870 ] 53 0.077% 99.858% ( 870, 1300 ] 40 0.058% 99.917% ( 1300, 1900 ] 16 0.023% 99.940% ( 1900, 2900 ] 17 0.025% 99.965% ( 2900, 4400 ] 17 0.025% 99.990% ( 4400, 6600 ] 5 0.007% 99.997% ( 6600, 9900 ] 4 0.006% 100.003% ** Level 2 read latency histogram (micros): Count: 256520 Average: 9.9542 StdDev: 135.92 Min: 1 Median: 1.9844 Max: 8849 Percentiles: P50: 1.98 P75: 3.01 P99: 10.25 P99.9: 1397.56 P99.99: 2650.82 ------------------------------------------------------ [ 0, 1 ] 7349 2.865% 2.865% # ( 1, 2 ] 122830 47.883% 50.748% ########## ( 2, 3 ] 61829 24.103% 74.851% ##### ( 3, 4 ] 26853 10.468% 85.319% ## ( 4, 6 ] 27510 10.724% 96.044% ## ( 6, 10 ] 7522 2.932% 98.976% # ( 10, 15 ] 1220 0.476% 99.452% ( 15, 22 ] 187 0.073% 99.524% ( 22, 34 ] 47 0.018% 99.543% ( 34, 51 ] 13 0.005% 99.548% ( 51, 76 ] 11 0.004% 99.552% ( 76, 110 ] 10 0.004% 99.556% ( 110, 170 ] 48 0.019% 99.575% ( 170, 250 ] 60 0.023% 99.598% ( 250, 380 ] 127 0.050% 99.648% ( 380, 580 ] 156 0.061% 99.708% ( 580, 870 ] 233 0.091% 99.799% ( 870, 1300 ] 231 0.090% 99.889% ( 1300, 1900 ] 169 0.066% 99.955% ( 1900, 2900 ] 119 0.046% 100.002% ( 2900, 4400 ] 111 0.043% 100.045% ( 4400, 6600 ] 41 0.016% 100.061% ( 6600, 9900 ] 12 0.005% 100.065% ** Level 3 read latency histogram (micros): Count: 1108510 Average: 12.0323 StdDev: 188.43 Min: 0 Median: 1.9943 Max: 55559 Percentiles: P50: 1.99 P75: 2.98 P99: 9.74 P99.9: 1026.98 P99.99: 1520.89 ------------------------------------------------------ [ 0, 1 ] 17808 1.606% 1.606% ( 1, 2 ] 539500 48.669% 50.275% ########## ( 2, 3 ] 278340 25.109% 75.385% ##### ( 3, 4 ] 118379 10.679% 86.064% ## ( 4, 6 ] 118294 10.671% 96.735% ## ( 6, 10 ] 26879 2.425% 99.160% ( 10, 15 ] 4230 0.382% 99.542% ( 15, 22 ] 575 0.052% 99.594% ( 22, 34 ] 86 0.008% 99.601% ( 34, 51 ] 75 0.007% 99.608% ( 51, 76 ] 59 0.005% 99.613% ( 76, 110 ] 84 0.008% 99.621% ( 110, 170 ] 254 0.023% 99.644% ( 170, 250 ] 228 0.021% 99.665% ( 250, 380 ] 417 0.038% 99.702% ( 380, 580 ] 707 0.064% 99.766% ( 580, 870 ] 1082 0.098% 99.864% ( 870, 1300 ] 1108 0.100% 99.963% ( 1300, 1900 ] 799 0.072% 100.036% ( 1900, 2900 ] 751 0.068% 100.103% ( 2900, 4400 ] 766 0.069% 100.172% ( 4400, 6600 ] 217 0.020% 100.192% ( 6600, 9900 ] 38 0.003% 100.195% ( 9900, 14000 ] 4 0.000% 100.196% ( 14000, 22000 ] 6 0.001% 100.196% ( 22000, 33000 ] 3 0.000% 100.197% ( 33000, 50000 ] 2 0.000% 100.197% ( 50000, 75000 ] 2 0.000% 100.197% ** Level 4 read latency histogram (micros): Count: 489975 Average: 24.9995 StdDev: 261.16 Min: 1 Median: 1.9913 Max: 54619 Percentiles: P50: 1.99 P75: 3.04 P99: 567.18 P99.9: 2866.94 P99.99: 3881.23 ------------------------------------------------------ [ 0, 1 ] 9105 1.858% 1.858% ( 1, 2 ] 237961 48.566% 50.424% ########## ( 2, 3 ] 118481 24.181% 74.605% ##### ( 3, 4 ] 51230 10.456% 85.061% ## ( 4, 6 ] 48492 9.897% 94.958% ## ( 6, 10 ] 11061 2.257% 97.215% ( 10, 15 ] 1522 0.311% 97.526% ( 15, 22 ] 213 0.043% 97.569% ( 22, 34 ] 25 0.005% 97.574% ( 34, 51 ] 40 0.008% 97.583% ( 51, 76 ] 29 0.006% 97.588% ( 76, 110 ] 668 0.136% 97.725% ( 110, 170 ] 2313 0.472% 98.197% ( 170, 250 ] 1474 0.301% 98.498% ( 250, 380 ] 1122 0.229% 98.727% ( 380, 580 ] 1431 0.292% 99.019% ( 580, 870 ] 1471 0.300% 99.319% ( 870, 1300 ] 1321 0.270% 99.589% ( 1300, 1900 ] 854 0.174% 99.763% ( 1900, 2900 ] 695 0.142% 99.905% ( 2900, 4400 ] 639 0.130% 100.035% ( 4400, 6600 ] 181 0.037% 100.072% ( 6600, 9900 ] 22 0.004% 100.077% ( 9900, 14000 ] 3 0.001% 100.077% ( 14000, 22000 ] 6 0.001% 100.078% ( 22000, 33000 ] 1 0.000% 100.079% ( 33000, 50000 ] 3 0.001% 100.079% ( 50000, 75000 ] 1 0.000% 100.079% ** Level 5 read latency histogram (micros): Count: 53314 Average: 648.9700 StdDev: 1085.17 Min: 75 Median: 245.1475 Max: 57960 Percentiles: P50: 245.15 P75: 667.71 P99: 5327.84 P99.9: 8738.34 P99.99: 19869.76 ------------------------------------------------------ ( 51, 76 ] 4 0.008% 0.008% ( 76, 110 ] 4112 7.713% 7.720% ## ( 110, 170 ] 14519 27.233% 34.953% ##### ( 170, 250 ] 8540 16.018% 50.972% ### ( 250, 380 ] 5510 10.335% 61.307% ## ( 380, 580 ] 5859 10.990% 72.296% ## ( 580, 870 ] 4766 8.939% 81.236% ## ( 870, 1300 ] 3472 6.512% 87.748% # ( 1300, 1900 ] 2102 3.943% 91.691% # ( 1900, 2900 ] 1755 3.292% 94.983% # ( 2900, 4400 ] 1842 3.455% 98.438% # ( 4400, 6600 ] 711 1.334% 99.771% ( 6600, 9900 ] 106 0.199% 99.970% ( 9900, 14000 ] 7 0.013% 99.983% ( 14000, 22000 ] 5 0.009% 99.992% ( 22000, 33000 ] 6 0.011% 100.004% ( 33000, 50000 ] 2 0.004% 100.008% ( 50000, 75000 ] 1 0.002% 100.009% ** DB Stats ** Uptime(secs): 12.4 total, 3.5 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 158.74 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.864 H:M:S, 39.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 280.61 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/17-08:45:39 ... thread 0: (1000000,3000000) ops and (288473.6,191375.7) ops/second in (3.466522,15.675971) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/7 3.96 GB 258.0 2.5 0.0 2.5 5.2 2.8 0.0 1.9 105.2 223.5 23.95 17.86 50 0.479 2558K 2723 0.0 0.0 L1 6/6 767.85 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 15/0 877.22 MB 0.9 2.0 1.2 0.8 2.0 1.2 0.0 1.6 110.3 109.1 18.99 9.69 8 2.373 1416K 3276 0.0 0.0 L4 167/0 9.73 GB 1.0 5.9 4.1 1.9 5.9 4.0 0.0 1.4 112.9 111.9 53.98 28.30 37 1.459 2329K 560 0.0 0.0 L5 1124/0 92.40 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1329/13 107.69 GB 0.0 10.5 5.3 5.1 13.1 8.0 4.0 4.7 110.5 138.9 96.92 55.85 95 1.020 6303K 6559 0.0 0.0 Int 0/0 0.00 KB 0.0 10.5 5.3 5.1 13.1 8.0 4.0 4.7 110.5 138.9 96.92 55.85 95 1.020 6303K 6559 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.5 5.3 5.1 10.4 5.2 0.0 0.0 127.9 127.0 83.70 47.84 54 1.550 6303K 6559 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 214.6 13.21 8.01 41 0.322 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.9 total, 15.7 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.15 GB write, 848.46 MB/s write, 10.46 GB read, 674.81 MB/s read, 96.9 seconds Interval compaction: 13.15 GB write, 858.70 MB/s write, 10.46 GB read, 682.96 MB/s read, 96.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 25, pending-compaction-bytes-delays: 4, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 25, interval: 28 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 991779 Average: 10.6046 StdDev: 127.11 Min: 1 Median: 1.7140 Max: 58174 Percentiles: P50: 1.71 P75: 2.52 P99: 189.71 P99.9: 1010.22 P99.99: 2302.02 ------------------------------------------------------ [ 0, 1 ] 113418 11.436% 11.436% ## ( 1, 2 ] 535684 54.012% 65.448% ########### ( 2, 3 ] 181887 18.339% 83.788% #### ( 3, 4 ] 60699 6.120% 89.908% # ( 4, 6 ] 62762 6.328% 96.236% # ( 6, 10 ] 14145 1.426% 97.662% ( 10, 15 ] 2256 0.227% 97.890% ( 15, 22 ] 300 0.030% 97.920% ( 22, 34 ] 30 0.003% 97.923% ( 34, 51 ] 36 0.004% 97.927% ( 51, 76 ] 33 0.003% 97.930% ( 76, 110 ] 2252 0.227% 98.157% ( 110, 170 ] 7341 0.740% 98.897% ( 170, 250 ] 4132 0.417% 99.314% ( 250, 380 ] 2348 0.237% 99.551% ( 380, 580 ] 2081 0.210% 99.761% ( 580, 870 ] 1183 0.119% 99.880% ( 870, 1300 ] 614 0.062% 99.942% ( 1300, 1900 ] 355 0.036% 99.978% ( 1900, 2900 ] 308 0.031% 100.009% ( 2900, 4400 ] 246 0.025% 100.033% ( 4400, 6600 ] 71 0.007% 100.041% ( 6600, 9900 ] 19 0.002% 100.042% ( 9900, 14000 ] 3 0.000% 100.043% ( 14000, 22000 ] 2 0.000% 100.043% ( 22000, 33000 ] 1 0.000% 100.043% ( 50000, 75000 ] 1 0.000% 100.043% ** Level 1 read latency histogram (micros): Count: 166017 Average: 6.3499 StdDev: 74.47 Min: 1 Median: 2.0758 Max: 8360 Percentiles: P50: 2.08 P75: 3.03 P99: 17.16 P99.9: 600.07 P99.99: 3288.09 ------------------------------------------------------ [ 0, 1 ] 5336 3.214% 3.214% # ( 1, 2 ] 74306 44.758% 47.972% ######### ( 2, 3 ] 44405 26.747% 74.719% ##### ( 3, 4 ] 15110 9.101% 83.821% ## ( 4, 6 ] 18545 11.171% 94.991% ## ( 6, 10 ] 5749 3.463% 98.454% # ( 10, 15 ] 807 0.486% 98.940% ( 15, 22 ] 324 0.195% 99.136% ( 22, 34 ] 32 0.019% 99.155% ( 34, 51 ] 5 0.003% 99.158% ( 51, 76 ] 2 0.001% 99.159% ( 76, 110 ] 150 0.090% 99.249% ( 110, 170 ] 492 0.296% 99.546% ( 170, 250 ] 254 0.153% 99.699% ( 250, 380 ] 178 0.107% 99.806% ( 380, 580 ] 152 0.092% 99.898% ( 580, 870 ] 72 0.043% 99.941% ( 870, 1300 ] 44 0.027% 99.967% ( 1300, 1900 ] 17 0.010% 99.978% ( 1900, 2900 ] 17 0.010% 99.988% ( 2900, 4400 ] 17 0.010% 99.998% ( 4400, 6600 ] 5 0.003% 100.001% ( 6600, 9900 ] 4 0.002% 100.004% ** Level 2 read latency histogram (micros): Count: 256520 Average: 9.9542 StdDev: 135.92 Min: 1 Median: 1.9844 Max: 8849 Percentiles: P50: 1.98 P75: 3.01 P99: 10.25 P99.9: 1397.56 P99.99: 2650.82 ------------------------------------------------------ [ 0, 1 ] 7349 2.865% 2.865% # ( 1, 2 ] 122830 47.883% 50.748% ########## ( 2, 3 ] 61829 24.103% 74.851% ##### ( 3, 4 ] 26853 10.468% 85.319% ## ( 4, 6 ] 27510 10.724% 96.044% ## ( 6, 10 ] 7522 2.932% 98.976% # ( 10, 15 ] 1220 0.476% 99.452% ( 15, 22 ] 187 0.073% 99.524% ( 22, 34 ] 47 0.018% 99.543% ( 34, 51 ] 13 0.005% 99.548% ( 51, 76 ] 11 0.004% 99.552% ( 76, 110 ] 10 0.004% 99.556% ( 110, 170 ] 48 0.019% 99.575% ( 170, 250 ] 60 0.023% 99.598% ( 250, 380 ] 127 0.050% 99.648% ( 380, 580 ] 156 0.061% 99.708% ( 580, 870 ] 233 0.091% 99.799% ( 870, 1300 ] 231 0.090% 99.889% ( 1300, 1900 ] 169 0.066% 99.955% ( 1900, 2900 ] 119 0.046% 100.002% ( 2900, 4400 ] 111 0.043% 100.045% ( 4400, 6600 ] 41 0.016% 100.061% ( 6600, 9900 ] 12 0.005% 100.065% ** Level 3 read latency histogram (micros): Count: 1109114 Average: 12.1373 StdDev: 188.46 Min: 0 Median: 1.9949 Max: 55559 Percentiles: P50: 1.99 P75: 2.99 P99: 9.82 P99.9: 1027.24 P99.99: 1520.84 ------------------------------------------------------ [ 0, 1 ] 17808 1.606% 1.606% ( 1, 2 ] 539501 48.643% 50.248% ########## ( 2, 3 ] 278345 25.096% 75.344% ##### ( 3, 4 ] 118382 10.674% 86.018% ## ( 4, 6 ] 118295 10.666% 96.684% ## ( 6, 10 ] 26879 2.423% 99.107% ( 10, 15 ] 4230 0.381% 99.488% ( 15, 22 ] 575 0.052% 99.540% ( 22, 34 ] 86 0.008% 99.548% ( 34, 51 ] 75 0.007% 99.555% ( 51, 76 ] 59 0.005% 99.560% ( 76, 110 ] 159 0.014% 99.574% ( 110, 170 ] 516 0.047% 99.621% ( 170, 250 ] 360 0.032% 99.653% ( 250, 380 ] 479 0.043% 99.697% ( 380, 580 ] 749 0.068% 99.764% ( 580, 870 ] 1101 0.099% 99.863% ( 870, 1300 ] 1110 0.100% 99.963% ( 1300, 1900 ] 799 0.072% 100.036% ( 1900, 2900 ] 751 0.068% 100.103% ( 2900, 4400 ] 766 0.069% 100.172% ( 4400, 6600 ] 217 0.020% 100.192% ( 6600, 9900 ] 38 0.003% 100.195% ( 9900, 14000 ] 4 0.000% 100.196% ( 14000, 22000 ] 6 0.001% 100.196% ( 22000, 33000 ] 3 0.000% 100.196% ( 33000, 50000 ] 2 0.000% 100.197% ( 50000, 75000 ] 2 0.000% 100.197% ** Level 4 read latency histogram (micros): Count: 497219 Average: 27.5496 StdDev: 260.78 Min: 1 Median: 2.0126 Max: 54619 Percentiles: P50: 2.01 P75: 3.14 P99: 590.17 P99.9: 2858.02 P99.99: 3880.34 ------------------------------------------------------ [ 0, 1 ] 9106 1.831% 1.831% ( 1, 2 ] 238012 47.869% 49.700% ########## ( 2, 3 ] 118669 23.867% 73.567% ##### ( 3, 4 ] 51310 10.319% 83.886% ## ( 4, 6 ] 48561 9.767% 93.652% ## ( 6, 10 ] 11073 2.227% 95.879% ( 10, 15 ] 1525 0.307% 96.186% ( 15, 22 ] 213 0.043% 96.229% ( 22, 34 ] 25 0.005% 96.234% ( 34, 51 ] 40 0.008% 96.242% ( 51, 76 ] 32 0.006% 96.249% ( 76, 110 ] 1629 0.328% 96.576% ( 110, 170 ] 5240 1.054% 97.630% ( 170, 250 ] 2952 0.594% 98.224% ( 250, 380 ] 1818 0.366% 98.589% ( 380, 580 ] 1984 0.399% 98.988% ( 580, 870 ] 1649 0.332% 99.320% ( 870, 1300 ] 1356 0.273% 99.593% ( 1300, 1900 ] 861 0.173% 99.766% ( 1900, 2900 ] 696 0.140% 99.906% ( 2900, 4400 ] 640 0.129% 100.035% ( 4400, 6600 ] 181 0.036% 100.071% ( 6600, 9900 ] 22 0.004% 100.075% ( 9900, 14000 ] 3 0.001% 100.076% ( 14000, 22000 ] 6 0.001% 100.077% ( 22000, 33000 ] 1 0.000% 100.077% ( 33000, 50000 ] 3 0.001% 100.078% ( 50000, 75000 ] 1 0.000% 100.078% ** Level 5 read latency histogram (micros): Count: 95934 Average: 455.2953 StdDev: 843.82 Min: 72 Median: 193.1148 Max: 57960 Percentiles: P50: 193.11 P75: 414.57 P99: 4294.90 P99.9: 7318.09 P99.99: 11895.29 ------------------------------------------------------ ( 51, 76 ] 12 0.013% 0.013% ( 76, 110 ] 10015 10.439% 10.452% ## ( 110, 170 ] 32864 34.257% 44.709% ####### ( 170, 250 ] 17568 18.313% 63.021% #### ( 250, 380 ] 9876 10.295% 73.316% ## ( 380, 580 ] 9345 9.741% 83.057% ## ( 580, 870 ] 5989 6.243% 89.300% # ( 870, 1300 ] 3691 3.847% 93.147% # ( 1300, 1900 ] 2130 2.220% 95.368% ( 1900, 2900 ] 1768 1.843% 97.211% ( 2900, 4400 ] 1846 1.924% 99.135% ( 4400, 6600 ] 711 0.741% 99.876% ( 6600, 9900 ] 106 0.110% 99.986% ( 9900, 14000 ] 7 0.007% 99.994% ( 14000, 22000 ] 5 0.005% 99.999% ( 22000, 33000 ] 6 0.006% 100.005% ( 33000, 50000 ] 2 0.002% 100.007% ( 50000, 75000 ] 1 0.001% 100.008% ** DB Stats ** Uptime(secs): 15.9 total, 3.5 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 186.08 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.864 H:M:S, 30.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 283.91 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/17-08:45:50 ... thread 0: (1000000,4000000) ops and (96416.2,153564.6) ops/second in (10.371704,26.047675) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/15 3.81 GB 6.3 3.7 0.0 3.7 7.5 3.8 0.0 2.0 117.7 239.2 32.11 25.46 72 0.446 3837K 4021 0.0 0.0 L1 21/21 1.93 GB 0.0 1.9 1.2 0.7 1.9 1.2 0.0 1.6 118.4 117.7 16.79 10.12 1 16.792 2018K 10K 0.0 0.0 L3 15/0 877.22 MB 0.9 2.0 1.2 0.8 2.0 1.2 0.0 1.6 110.3 109.1 18.99 9.69 8 2.373 1416K 3276 0.0 0.0 L4 167/0 9.73 GB 1.0 5.9 4.1 1.9 5.9 4.0 0.0 1.4 112.9 111.9 53.98 28.30 37 1.459 2329K 560 0.0 0.0 L5 1124/0 92.40 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1343/36 108.72 GB 0.0 13.6 6.5 7.1 17.4 10.2 4.0 4.5 114.5 145.8 121.87 73.57 118 1.033 9601K 18K 0.0 0.0 Int 0/0 0.00 KB 0.0 13.6 6.5 7.1 17.4 10.2 4.0 4.5 114.5 145.8 121.87 73.57 118 1.033 9601K 18K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.6 6.5 7.1 13.5 6.4 0.0 0.0 132.5 131.7 105.29 62.65 60 1.755 9601K 18K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 235.6 16.58 10.91 58 0.286 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 26.2 total, 26.0 interval Flush(GB): cumulative 3.814, interval 3.814 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 17.35 GB write, 677.19 MB/s write, 13.63 GB read, 531.84 MB/s read, 121.9 seconds Interval compaction: 17.35 GB write, 682.11 MB/s write, 13.63 GB read, 535.70 MB/s read, 121.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 2, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 2, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 25, pending-compaction-bytes-delays: 8, pending-compaction-bytes-stops: 0, total-delays: 10, total-stops: 25, interval: 34 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1405864 Average: 12.6810 StdDev: 111.42 Min: 0 Median: 1.6683 Max: 58174 Percentiles: P50: 1.67 P75: 2.47 P99: 202.72 P99.9: 817.88 P99.99: 1887.49 ------------------------------------------------------ [ 0, 1 ] 204485 14.545% 14.545% ### ( 1, 2 ] 745884 53.055% 67.600% ########### ( 2, 3 ] 222451 15.823% 83.423% ### ( 3, 4 ] 77396 5.505% 88.929% # ( 4, 6 ] 73043 5.196% 94.124% # ( 6, 10 ] 15532 1.105% 95.229% ( 10, 15 ] 2549 0.181% 95.410% ( 15, 22 ] 345 0.025% 95.435% ( 22, 34 ] 37 0.003% 95.438% ( 34, 51 ] 36 0.003% 95.440% ( 51, 76 ] 45 0.003% 95.443% ( 76, 110 ] 13523 0.962% 96.405% ( 110, 170 ] 32880 2.339% 98.744% ( 170, 250 ] 8802 0.626% 99.370% ( 250, 380 ] 3467 0.247% 99.617% ( 380, 580 ] 2830 0.201% 99.818% ( 580, 870 ] 1407 0.100% 99.918% ( 870, 1300 ] 656 0.047% 99.965% ( 1300, 1900 ] 364 0.026% 99.991% ( 1900, 2900 ] 324 0.023% 100.014% ( 2900, 4400 ] 265 0.019% 100.033% ( 4400, 6600 ] 71 0.005% 100.038% ( 6600, 9900 ] 19 0.001% 100.039% ( 9900, 14000 ] 3 0.000% 100.039% ( 14000, 22000 ] 2 0.000% 100.039% ( 22000, 33000 ] 1 0.000% 100.039% ( 50000, 75000 ] 1 0.000% 100.039% ** Level 1 read latency histogram (micros): Count: 226238 Average: 8.5742 StdDev: 69.58 Min: 1 Median: 2.1474 Max: 8360 Percentiles: P50: 2.15 P75: 3.07 P99: 156.07 P99.9: 578.25 P99.99: 2818.81 ------------------------------------------------------ [ 0, 1 ] 6818 3.014% 3.014% # ( 1, 2 ] 96751 42.765% 45.779% ######### ( 2, 3 ] 64808 28.646% 74.425% ###### ( 3, 4 ] 19435 8.591% 83.015% ## ( 4, 6 ] 23717 10.483% 93.498% ## ( 6, 10 ] 7264 3.211% 96.709% # ( 10, 15 ] 1025 0.453% 97.162% ( 15, 22 ] 414 0.183% 97.345% ( 22, 34 ] 39 0.017% 97.363% ( 34, 51 ] 6 0.003% 97.365% ( 51, 76 ] 3 0.001% 97.366% ( 76, 110 ] 1220 0.539% 97.906% ( 110, 170 ] 3224 1.425% 99.331% ( 170, 250 ] 719 0.318% 99.649% ( 250, 380 ] 315 0.139% 99.788% ( 380, 580 ] 256 0.113% 99.901% ( 580, 870 ] 100 0.044% 99.945% ( 870, 1300 ] 59 0.026% 99.971% ( 1300, 1900 ] 24 0.011% 99.982% ( 1900, 2900 ] 20 0.009% 99.991% ( 2900, 4400 ] 17 0.008% 99.998% ( 4400, 6600 ] 5 0.002% 100.000% ( 6600, 9900 ] 4 0.002% 100.002% ** Level 2 read latency histogram (micros): Count: 256520 Average: 9.9542 StdDev: 135.92 Min: 1 Median: 1.9844 Max: 8849 Percentiles: P50: 1.98 P75: 3.01 P99: 10.25 P99.9: 1397.56 P99.99: 2650.82 ------------------------------------------------------ [ 0, 1 ] 7349 2.865% 2.865% # ( 1, 2 ] 122830 47.883% 50.748% ########## ( 2, 3 ] 61829 24.103% 74.851% ##### ( 3, 4 ] 26853 10.468% 85.319% ## ( 4, 6 ] 27510 10.724% 96.044% ## ( 6, 10 ] 7522 2.932% 98.976% # ( 10, 15 ] 1220 0.476% 99.452% ( 15, 22 ] 187 0.073% 99.524% ( 22, 34 ] 47 0.018% 99.543% ( 34, 51 ] 13 0.005% 99.548% ( 51, 76 ] 11 0.004% 99.552% ( 76, 110 ] 10 0.004% 99.556% ( 110, 170 ] 48 0.019% 99.575% ( 170, 250 ] 60 0.023% 99.598% ( 250, 380 ] 127 0.050% 99.648% ( 380, 580 ] 156 0.061% 99.708% ( 580, 870 ] 233 0.091% 99.799% ( 870, 1300 ] 231 0.090% 99.889% ( 1300, 1900 ] 169 0.066% 99.955% ( 1900, 2900 ] 119 0.046% 100.002% ( 2900, 4400 ] 111 0.043% 100.045% ( 4400, 6600 ] 41 0.016% 100.061% ( 6600, 9900 ] 12 0.005% 100.065% ** Level 3 read latency histogram (micros): Count: 1111661 Average: 12.4755 StdDev: 188.49 Min: 0 Median: 1.9973 Max: 55559 Percentiles: P50: 2.00 P75: 2.99 P99: 11.58 P99.9: 1027.90 P99.99: 1522.90 ------------------------------------------------------ [ 0, 1 ] 17808 1.602% 1.602% ( 1, 2 ] 539501 48.531% 50.133% ########## ( 2, 3 ] 278345 25.039% 75.172% ##### ( 3, 4 ] 118382 10.649% 85.821% ## ( 4, 6 ] 118295 10.641% 96.462% ## ( 6, 10 ] 26879 2.418% 98.880% ( 10, 15 ] 4230 0.381% 99.260% ( 15, 22 ] 575 0.052% 99.312% ( 22, 34 ] 86 0.008% 99.320% ( 34, 51 ] 75 0.007% 99.327% ( 51, 76 ] 59 0.005% 99.332% ( 76, 110 ] 722 0.065% 99.397% ( 110, 170 ] 2021 0.182% 99.579% ( 170, 250 ] 647 0.058% 99.637% ( 250, 380 ] 570 0.051% 99.688% ( 380, 580 ] 824 0.074% 99.762% ( 580, 870 ] 1122 0.101% 99.863% ( 870, 1300 ] 1112 0.100% 99.963% ( 1300, 1900 ] 799 0.072% 100.035% ( 1900, 2900 ] 752 0.068% 100.103% ( 2900, 4400 ] 768 0.069% 100.172% ( 4400, 6600 ] 217 0.020% 100.191% ( 6600, 9900 ] 38 0.003% 100.195% ( 9900, 14000 ] 4 0.000% 100.195% ( 14000, 22000 ] 6 0.001% 100.196% ( 22000, 33000 ] 3 0.000% 100.196% ( 33000, 50000 ] 2 0.000% 100.196% ( 50000, 75000 ] 2 0.000% 100.196% ** Level 4 read latency histogram (micros): Count: 525990 Average: 34.5608 StdDev: 256.42 Min: 1 Median: 2.1338 Max: 54619 Percentiles: P50: 2.13 P75: 3.56 P99: 593.03 P99.9: 2821.67 P99.99: 3876.05 ------------------------------------------------------ [ 0, 1 ] 9106 1.731% 1.731% ( 1, 2 ] 238012 45.250% 46.982% ######### ( 2, 3 ] 118679 22.563% 69.544% ##### ( 3, 4 ] 51314 9.756% 79.300% ## ( 4, 6 ] 48573 9.235% 88.535% ## ( 6, 10 ] 11074 2.105% 90.640% ( 10, 15 ] 1525 0.290% 90.930% ( 15, 22 ] 213 0.040% 90.971% ( 22, 34 ] 25 0.005% 90.975% ( 34, 51 ] 40 0.008% 90.983% ( 51, 76 ] 38 0.007% 90.990% ( 76, 110 ] 8359 1.589% 92.579% ( 110, 170 ] 22124 4.206% 96.785% # ( 170, 250 ] 6201 1.179% 97.964% ( 250, 380 ] 2701 0.514% 98.478% ( 380, 580 ] 2661 0.506% 98.984% ( 580, 870 ] 1894 0.360% 99.344% ( 870, 1300 ] 1403 0.267% 99.611% ( 1300, 1900 ] 875 0.166% 99.777% ( 1900, 2900 ] 702 0.133% 99.910% ( 2900, 4400 ] 643 0.122% 100.033% ( 4400, 6600 ] 181 0.034% 100.067% ( 6600, 9900 ] 22 0.004% 100.071% ( 9900, 14000 ] 3 0.001% 100.072% ( 14000, 22000 ] 6 0.001% 100.073% ( 22000, 33000 ] 1 0.000% 100.073% ( 33000, 50000 ] 3 0.001% 100.074% ( 50000, 75000 ] 1 0.000% 100.074% ** Level 5 read latency histogram (micros): Count: 263061 Average: 264.8546 StdDev: 535.73 Min: 72 Median: 147.8148 Max: 57960 Percentiles: P50: 147.81 P75: 208.23 P99: 2938.34 P99.9: 6135.68 P99.99: 9298.96 ------------------------------------------------------ ( 51, 76 ] 70 0.027% 0.027% ( 76, 110 ] 49304 18.742% 18.769% #### ( 110, 170 ] 130356 49.554% 68.323% ########## ( 170, 250 ] 36759 13.974% 82.296% ### ( 250, 380 ] 15242 5.794% 88.090% # ( 380, 580 ] 13460 5.117% 93.207% # ( 580, 870 ] 7234 2.750% 95.957% # ( 870, 1300 ] 3955 1.503% 97.460% ( 1300, 1900 ] 2198 0.836% 98.296% ( 1900, 2900 ] 1805 0.686% 98.982% ( 2900, 4400 ] 1854 0.705% 99.687% ( 4400, 6600 ] 711 0.270% 99.957% ( 6600, 9900 ] 106 0.040% 99.997% ( 9900, 14000 ] 7 0.003% 100.000% ( 14000, 22000 ] 5 0.002% 100.002% ( 22000, 33000 ] 6 0.002% 100.004% ( 33000, 50000 ] 2 0.001% 100.005% ( 50000, 75000 ] 1 0.000% 100.005% ** DB Stats ** Uptime(secs): 26.2 total, 10.4 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 150.03 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:11.578 H:M:S, 44.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 94.89 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:6.714 H:M:S, 64.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 224.652 micros/op 17798 ops/sec 60.063 seconds 1068996 operations; 17.2 MB/s (267999 of 267999 found) Microseconds per read: Count: 1068996 Average: 224.6543 StdDev: 626.01 Min: 2 Median: 167.1405 Max: 100879 Percentiles: P50: 167.14 P75: 226.14 P99: 916.30 P99.9: 10435.62 P99.99: 25864.14 ------------------------------------------------------ ( 1, 2 ] 3 0.000% 0.000% ( 2, 3 ] 33 0.003% 0.003% ( 3, 4 ] 50 0.005% 0.008% ( 4, 6 ] 85 0.008% 0.016% ( 6, 10 ] 588 0.055% 0.071% ( 10, 15 ] 4476 0.419% 0.490% ( 15, 22 ] 4122 0.386% 0.875% ( 22, 34 ] 9581 0.896% 1.772% ( 34, 51 ] 3424 0.320% 2.092% ( 51, 76 ] 394 0.037% 2.129% ( 76, 110 ] 28048 2.624% 4.752% # ( 110, 170 ] 507900 47.512% 52.264% ########## ( 170, 250 ] 346345 32.399% 84.663% ###### ( 250, 380 ] 104070 9.735% 94.399% ## ( 380, 580 ] 35937 3.362% 97.761% # ( 580, 870 ] 12715 1.189% 98.950% ( 870, 1300 ] 4969 0.465% 99.415% ( 1300, 1900 ] 2007 0.188% 99.603% ( 1900, 2900 ] 1400 0.131% 99.733% ( 2900, 4400 ] 1146 0.107% 99.841% ( 4400, 6600 ] 385 0.036% 99.877% ( 6600, 9900 ] 191 0.018% 99.895% ( 9900, 14000 ] 444 0.042% 99.936% ( 14000, 22000 ] 535 0.050% 99.986% ( 22000, 33000 ] 117 0.011% 99.997% ( 33000, 50000 ] 27 0.003% 100.000% ( 50000, 75000 ] 3 0.000% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 19307208 rocksdb.bloom.filter.full.positive COUNT : 1256586 rocksdb.bloom.filter.full.true.positive COUNT : 1068559 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 437 rocksdb.memtable.miss COUNT : 1068559 rocksdb.l0.hit COUNT : 40455 rocksdb.l1.hit COUNT : 19234 rocksdb.l2andup.hit COUNT : 1008870 rocksdb.compaction.key.drop.new COUNT : 140268 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 4114269 rocksdb.number.keys.read COUNT : 1068996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 4245925608 rocksdb.bytes.read COUNT : 1068996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1541 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 44357595 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 16783147719 rocksdb.compact.write.bytes COUNT : 16643935645 rocksdb.flush.write.bytes COUNT : 4227771578 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 22 rocksdb.number.superversion_acquires COUNT : 795 rocksdb.number.superversion_releases COUNT : 767 rocksdb.number.superversion_cleanups COUNT : 198 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 196 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 4193565696 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1529856 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 21735251958 rocksdb.non.last.level.read.count COUNT : 5110247 rocksdb.block.checksum.compute.count COUNT : 5336004 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 166.313480 P95 : 414.036515 P99 : 914.357529 P100 : 100878.000000 COUNT : 1068996 SUM : 239219308 rocksdb.db.write.micros P50 : 1.576130 P95 : 4.807484 P99 : 12.831410 P100 : 821212.000000 COUNT : 4114269 SUM : 54921687 rocksdb.compaction.times.micros P50 : 1433333.333333 P95 : 2816666.666667 P99 : 16792038.000000 P100 : 16792038.000000 COUNT : 60 SUM : 105290824 rocksdb.compaction.times.cpu_micros P50 : 896428.571429 P95 : 1666666.666667 P99 : 10115519.000000 P100 : 10115519.000000 COUNT : 60 SUM : 62651326 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 838.928571 P95 : 110000.000000 P99 : 226000.000000 P100 : 247958.000000 COUNT : 60 SUM : 1001172 rocksdb.compaction.outfile.sync.micros P50 : 40727.272727 P95 : 232769.230769 P99 : 383800.000000 P100 : 456660.000000 COUNT : 196 SUM : 13530535 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 197.692308 P95 : 4858.333333 P99 : 7672.500000 P100 : 8139.000000 COUNT : 135 SUM : 166589 rocksdb.table.open.io.micros P50 : 9258.917197 P95 : 19516.666667 P99 : 21799.629630 P100 : 78057.000000 COUNT : 1541 SUM : 14486665 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.119696 P95 : 90.607045 P99 : 178.281186 P100 : 58179.000000 COUNT : 4065877 SUM : 71709620 rocksdb.read.block.get.micros P50 : 135.920239 P95 : 280.703041 P99 : 842.399255 P100 : 57979.000000 COUNT : 1268574 SUM : 202529661 rocksdb.write.raw.block.micros P50 : 0.533332 P95 : 1.292007 P99 : 3.452107 P100 : 51500.000000 COUNT : 5055286 SUM : 13922979 rocksdb.numfiles.in.singlecompaction P50 : 2.976190 P95 : 3.977941 P99 : 36.000000 P100 : 36.000000 COUNT : 75 SUM : 283 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.594071 P95 : 2185.854931 P99 : 2763.229552 P100 : 821050.000000 COUNT : 171538 SUM : 44376132 rocksdb.sst.read.micros P50 : 2.516196 P95 : 167.730269 P99 : 444.441332 P100 : 58174.000000 COUNT : 5110261 SUM : 260066355 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1068996 SUM : 1068996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 4114269 SUM : 4245925608 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 220000.000000 P95 : 860000.000000 P99 : 1051048.000000 P100 : 1051048.000000 COUNT : 60 SUM : 17040986 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 204 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0