[2024-12-14 12:50:32.433100] Starting SPDK v24.09.1-pre git sha1 b18e1bd62 / DPDK 23.11.0 initialization... [2024-12-14 12:50:32.433368] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1466519 ] Set seed to 1734177032426428 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-14 12:50:32.544340] app.c: 917:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-14 12:50:32.586107] app.c: 610:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-14 12:50:32.586155] app.c: 611:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1466519' to capture a snapshot of events at runtime. [2024-12-14 12:50:32.586166] app.c: 616:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-14 12:50:32.586191] app.c: 617:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-14 12:50:32.586199] app.c: 618:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1466519 for offline analysis/debug. [2024-12-14 12:50:32.586300] reactor.c: 990:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sat Dec 14 12:50:36 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/14-12:50:44 ... thread 0: (1000000,1000000) ops and (130159.5,130159.5) ops/second in (7.682883,7.682883) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/10 1.38 GB 44.0 0.4 0.0 0.4 1.2 0.8 0.0 1.5 37.6 119.1 10.05 4.02 10 1.005 383K 574 0.0 0.0 L1 7/7 542.51 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 2/2 205.90 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 22/8 2.08 GB 1.1 1.6 1.0 0.6 1.5 0.9 0.0 1.6 101.8 100.2 15.66 7.40 9 1.740 473K 0 0.0 0.0 L4 203/19 12.49 GB 1.2 4.0 3.4 0.6 4.0 3.4 0.1 1.2 106.2 105.9 38.32 18.91 31 1.236 1736K 216 0.0 0.0 L5 1072/0 88.15 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1318/46 104.84 GB 0.0 5.9 4.4 1.5 6.7 5.1 0.7 8.3 94.3 106.5 64.02 30.33 50 1.280 2594K 790 0.0 0.0 Int 0/0 0.00 KB 0.0 5.9 4.4 1.5 6.7 5.1 0.7 8.3 94.3 106.5 64.02 30.33 50 1.280 2594K 790 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.9 4.4 1.5 5.9 4.3 0.0 0.0 106.1 105.5 56.91 27.82 41 1.388 2594K 790 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 115.0 7.12 2.51 9 0.791 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.9 total, 7.7 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.66 GB write, 868.17 MB/s write, 5.90 GB read, 768.64 MB/s read, 64.0 seconds Interval compaction: 6.66 GB write, 887.73 MB/s write, 5.90 GB read, 785.95 MB/s read, 64.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 32, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 32, interval: 32 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 267368 Average: 17.0658 StdDev: 299.21 Min: 1 Median: 1.9086 Max: 44235 Percentiles: P50: 1.91 P75: 2.94 P99: 13.95 P99.9: 2930.88 P99.99: 5085.83 ------------------------------------------------------ [ 0, 1 ] 9743 3.644% 3.644% # ( 1, 2 ] 136417 51.022% 54.666% ########## ( 2, 3 ] 57907 21.658% 76.324% #### ( 3, 4 ] 26209 9.803% 86.127% ## ( 4, 6 ] 27318 10.217% 96.344% ## ( 6, 10 ] 6215 2.325% 98.669% ( 10, 15 ] 1124 0.420% 99.089% ( 15, 22 ] 141 0.053% 99.142% ( 22, 34 ] 16 0.006% 99.148% ( 34, 51 ] 6 0.002% 99.150% ( 51, 76 ] 12 0.004% 99.155% ( 76, 110 ] 3 0.001% 99.156% ( 110, 170 ] 23 0.009% 99.164% ( 170, 250 ] 57 0.021% 99.186% ( 250, 380 ] 158 0.059% 99.245% ( 380, 580 ] 335 0.125% 99.370% ( 580, 870 ] 476 0.178% 99.548% ( 870, 1300 ] 438 0.164% 99.712% ( 1300, 1900 ] 280 0.105% 99.817% ( 1900, 2900 ] 220 0.082% 99.899% ( 2900, 4400 ] 225 0.084% 99.983% ( 4400, 6600 ] 65 0.024% 100.007% ( 6600, 9900 ] 7 0.003% 100.010% ( 9900, 14000 ] 2 0.001% 100.011% ( 14000, 22000 ] 12 0.004% 100.015% ( 22000, 33000 ] 9 0.003% 100.019% ( 33000, 50000 ] 4 0.001% 100.020% ** Level 1 read latency histogram (micros): Count: 55990 Average: 14.6156 StdDev: 239.74 Min: 1 Median: 2.1503 Max: 27831 Percentiles: P50: 2.15 P75: 3.22 P99: 13.23 P99.9: 3070.80 P99.99: 6490.18 ------------------------------------------------------ [ 0, 1 ] 666 1.189% 1.189% ( 1, 2 ] 25089 44.810% 45.999% ######### ( 2, 3 ] 14904 26.619% 72.618% ##### ( 3, 4 ] 6169 11.018% 83.636% ## ( 4, 6 ] 6543 11.686% 95.322% ## ( 6, 10 ] 1834 3.276% 98.598% # ( 10, 15 ] 348 0.622% 99.220% ( 15, 22 ] 50 0.089% 99.309% ( 22, 34 ] 4 0.007% 99.316% ( 51, 76 ] 3 0.005% 99.321% ( 76, 110 ] 1 0.002% 99.323% ( 110, 170 ] 10 0.018% 99.341% ( 170, 250 ] 15 0.027% 99.368% ( 250, 380 ] 26 0.046% 99.414% ( 380, 580 ] 45 0.080% 99.495% ( 580, 870 ] 59 0.105% 99.600% ( 870, 1300 ] 73 0.130% 99.730% ( 1300, 1900 ] 45 0.080% 99.811% ( 1900, 2900 ] 45 0.080% 99.891% ( 2900, 4400 ] 44 0.079% 99.970% ( 4400, 6600 ] 12 0.021% 99.991% ( 6600, 9900 ] 1 0.002% 99.993% ( 9900, 14000 ] 1 0.002% 99.995% ( 14000, 22000 ] 1 0.002% 99.996% ( 22000, 33000 ] 2 0.004% 100.000% ** Level 2 read latency histogram (micros): Count: 157797 Average: 15.3028 StdDev: 248.13 Min: 1 Median: 1.9840 Max: 46660 Percentiles: P50: 1.98 P75: 3.12 P99: 13.21 P99.9: 2930.41 P99.99: 4278.67 ------------------------------------------------------ [ 0, 1 ] 4307 2.729% 2.729% # ( 1, 2 ] 75805 48.040% 50.769% ########## ( 2, 3 ] 36289 22.997% 73.766% ##### ( 3, 4 ] 16816 10.657% 84.423% ## ( 4, 6 ] 17495 11.087% 95.510% ## ( 6, 10 ] 4859 3.079% 98.589% # ( 10, 15 ] 1009 0.639% 99.229% ( 15, 22 ] 154 0.098% 99.326% ( 22, 34 ] 25 0.016% 99.342% ( 34, 51 ] 6 0.004% 99.346% ( 51, 76 ] 6 0.004% 99.350% ( 76, 110 ] 6 0.004% 99.354% ( 110, 170 ] 18 0.011% 99.365% ( 170, 250 ] 38 0.024% 99.389% ( 250, 380 ] 86 0.055% 99.444% ( 380, 580 ] 148 0.094% 99.537% ( 580, 870 ] 174 0.110% 99.648% ( 870, 1300 ] 152 0.096% 99.744% ( 1300, 1900 ] 122 0.077% 99.821% ( 1900, 2900 ] 121 0.077% 99.898% ( 2900, 4400 ] 158 0.100% 99.998% ( 4400, 6600 ] 51 0.032% 100.030% ( 6600, 9900 ] 9 0.006% 100.036% ( 9900, 14000 ] 1 0.001% 100.037% ( 14000, 22000 ] 5 0.003% 100.040% ( 22000, 33000 ] 1 0.001% 100.041% ( 33000, 50000 ] 1 0.001% 100.041% ** Level 3 read latency histogram (micros): Count: 1028823 Average: 12.5572 StdDev: 241.78 Min: 1 Median: 1.9961 Max: 53375 Percentiles: P50: 2.00 P75: 3.03 P99: 9.76 P99.9: 729.26 P99.99: 1092.53 ------------------------------------------------------ [ 0, 1 ] 18607 1.809% 1.809% ( 1, 2 ] 497727 48.378% 50.187% ########## ( 2, 3 ] 251688 24.464% 74.651% ##### ( 3, 4 ] 111884 10.875% 85.525% ## ( 4, 6 ] 114457 11.125% 96.651% ## ( 6, 10 ] 25685 2.497% 99.147% ( 10, 15 ] 5327 0.518% 99.665% ( 15, 22 ] 731 0.071% 99.736% ( 22, 34 ] 65 0.006% 99.742% ( 34, 51 ] 47 0.005% 99.747% ( 51, 76 ] 52 0.005% 99.752% ( 76, 110 ] 19 0.002% 99.754% ( 110, 170 ] 48 0.005% 99.758% ( 170, 250 ] 96 0.009% 99.768% ( 250, 380 ] 323 0.031% 99.799% ( 380, 580 ] 578 0.056% 99.855% ( 580, 870 ] 896 0.087% 99.942% ( 870, 1300 ] 949 0.092% 100.035% ( 1300, 1900 ] 711 0.069% 100.104% ( 1900, 2900 ] 695 0.068% 100.171% ( 2900, 4400 ] 671 0.065% 100.236% ( 4400, 6600 ] 204 0.020% 100.256% ( 6600, 9900 ] 40 0.004% 100.260% ( 9900, 14000 ] 14 0.001% 100.262% ( 14000, 22000 ] 23 0.002% 100.264% ( 22000, 33000 ] 17 0.002% 100.265% ( 33000, 50000 ] 8 0.001% 100.266% ( 50000, 75000 ] 2 0.000% 100.266% ** Level 4 read latency histogram (micros): Count: 227763 Average: 36.8376 StdDev: 330.61 Min: 1 Median: 2.0246 Max: 41887 Percentiles: P50: 2.02 P75: 3.12 P99: 1135.31 P99.9: 3692.49 P99.99: 4326.47 ------------------------------------------------------ [ 0, 1 ] 4305 1.890% 1.890% ( 1, 2 ] 108217 47.513% 49.403% ########## ( 2, 3 ] 55308 24.283% 73.686% ##### ( 3, 4 ] 25343 11.127% 84.813% ## ( 4, 6 ] 23624 10.372% 95.185% ## ( 6, 10 ] 4756 2.088% 97.273% ( 10, 15 ] 1078 0.473% 97.747% ( 15, 22 ] 121 0.053% 97.800% ( 22, 34 ] 18 0.008% 97.808% ( 34, 51 ] 10 0.004% 97.812% ( 51, 76 ] 9 0.004% 97.816% ( 76, 110 ] 5 0.002% 97.818% ( 110, 170 ] 68 0.030% 97.848% ( 170, 250 ] 155 0.068% 97.916% ( 250, 380 ] 293 0.129% 98.045% ( 380, 580 ] 587 0.258% 98.303% ( 580, 870 ] 952 0.418% 98.721% ( 870, 1300 ] 1033 0.454% 99.174% ( 1300, 1900 ] 784 0.344% 99.518% ( 1900, 2900 ] 614 0.270% 99.788% ( 2900, 4400 ] 485 0.213% 100.001% ( 4400, 6600 ] 150 0.066% 100.067% ( 6600, 9900 ] 12 0.005% 100.072% ( 9900, 14000 ] 3 0.001% 100.073% ( 14000, 22000 ] 3 0.001% 100.075% ( 22000, 33000 ] 3 0.001% 100.076% ( 33000, 50000 ] 2 0.001% 100.077% ** Level 5 read latency histogram (micros): Count: 12378 Average: 1740.2369 StdDev: 2174.17 Min: 82 Median: 1145.3475 Max: 55978 Percentiles: P50: 1145.35 P75: 2362.54 P99: 7168.26 P99.9: 33961.27 P99.99: 55978.00 ------------------------------------------------------ ( 76, 110 ] 14 0.113% 0.113% ( 110, 170 ] 80 0.646% 0.759% ( 170, 250 ] 241 1.947% 2.706% ( 250, 380 ] 699 5.647% 8.354% # ( 380, 580 ] 1491 12.046% 20.399% ## ( 580, 870 ] 2245 18.137% 38.536% #### ( 870, 1300 ] 2216 17.903% 56.439% #### ( 1300, 1900 ] 1606 12.975% 69.413% ### ( 1900, 2900 ] 1495 12.078% 81.491% ## ( 2900, 4400 ] 1527 12.336% 93.828% ## ( 4400, 6600 ] 623 5.033% 98.861% # ( 6600, 9900 ] 100 0.808% 99.669% ( 9900, 14000 ] 9 0.073% 99.741% ( 14000, 22000 ] 12 0.097% 99.838% ( 22000, 33000 ] 7 0.057% 99.895% ( 33000, 50000 ] 11 0.089% 99.984% ( 50000, 75000 ] 2 0.016% 100.000% ** DB Stats ** Uptime(secs): 7.9 total, 7.7 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 125.25 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.541 H:M:S, 45.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 128.06 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.541 H:M:S, 46.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/12/14-12:50:48 ... thread 0: (1000000,2000000) ops and (239895.3,168756.9) ops/second in (4.168486,11.851369) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/10 2.42 GB 163.6 1.5 0.0 1.5 3.4 1.8 0.0 1.8 80.8 177.7 19.49 11.77 30 0.650 1599K 1954 0.0 0.0 L1 7/7 542.51 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 11/0 884.78 MB 0.9 1.9 1.2 0.7 1.9 1.1 0.0 1.6 106.1 104.4 18.17 9.08 10 1.817 682K 0 0.0 0.0 L4 165/0 9.59 GB 1.0 6.1 4.7 1.4 6.1 4.7 0.2 1.3 112.8 112.3 55.50 29.87 42 1.321 2622K 1728 0.0 0.0 L5 1124/0 92.46 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1323/17 105.86 GB 0.0 9.5 5.9 3.7 11.3 7.7 5.2 6.1 104.8 124.5 93.16 50.73 82 1.136 4904K 3682 0.0 0.0 Int 0/0 0.00 KB 0.0 9.5 5.9 3.7 11.3 7.7 5.2 6.1 104.8 124.5 93.16 50.73 82 1.136 4904K 3682 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.5 5.9 3.7 9.5 5.8 0.0 0.0 119.2 118.5 81.89 45.22 57 1.437 4904K 3682 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 167.7 11.27 5.50 25 0.451 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.0 total, 11.9 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.32 GB write, 964.09 MB/s write, 9.53 GB read, 811.71 MB/s read, 93.2 seconds Interval compaction: 11.32 GB write, 978.17 MB/s write, 9.53 GB read, 823.56 MB/s read, 93.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 35, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 35, interval: 35 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 575045 Average: 15.3389 StdDev: 217.73 Min: 1 Median: 1.7350 Max: 44235 Percentiles: P50: 1.74 P75: 2.60 P99: 265.82 P99.9: 1886.51 P99.99: 3786.82 ------------------------------------------------------ [ 0, 1 ] 56004 9.739% 9.739% ## ( 1, 2 ] 314992 54.777% 64.516% ########### ( 2, 3 ] 99970 17.385% 81.901% ### ( 3, 4 ] 40927 7.117% 89.018% # ( 4, 6 ] 40532 7.048% 96.066% # ( 6, 10 ] 8602 1.496% 97.562% ( 10, 15 ] 1719 0.299% 97.861% ( 15, 22 ] 225 0.039% 97.900% ( 22, 34 ] 25 0.004% 97.905% ( 34, 51 ] 15 0.003% 97.907% ( 51, 76 ] 15 0.003% 97.910% ( 76, 110 ] 801 0.139% 98.049% ( 110, 170 ] 3178 0.553% 98.602% ( 170, 250 ] 2123 0.369% 98.971% ( 250, 380 ] 1385 0.241% 99.212% ( 380, 580 ] 1506 0.262% 99.474% ( 580, 870 ] 1221 0.212% 99.686% ( 870, 1300 ] 795 0.138% 99.824% ( 1300, 1900 ] 447 0.078% 99.902% ( 1900, 2900 ] 339 0.059% 99.961% ( 2900, 4400 ] 285 0.050% 100.011% ( 4400, 6600 ] 75 0.013% 100.024% ( 6600, 9900 ] 7 0.001% 100.025% ( 9900, 14000 ] 2 0.000% 100.025% ( 14000, 22000 ] 12 0.002% 100.027% ( 22000, 33000 ] 9 0.002% 100.029% ( 33000, 50000 ] 4 0.001% 100.030% ** Level 1 read latency histogram (micros): Count: 130747 Average: 9.6507 StdDev: 162.85 Min: 1 Median: 2.3642 Max: 27831 Percentiles: P50: 2.36 P75: 3.55 P99: 14.21 P99.9: 1758.37 P99.99: 5064.28 ------------------------------------------------------ [ 0, 1 ] 1559 1.192% 1.192% ( 1, 2 ] 50684 38.765% 39.957% ######## ( 2, 3 ] 36051 27.573% 67.530% ###### ( 3, 4 ] 17803 13.616% 81.147% ### ( 4, 6 ] 17729 13.560% 94.707% ### ( 6, 10 ] 4805 3.675% 98.382% # ( 10, 15 ] 961 0.735% 99.117% ( 15, 22 ] 246 0.188% 99.305% ( 22, 34 ] 26 0.020% 99.325% ( 34, 51 ] 11 0.008% 99.333% ( 51, 76 ] 7 0.005% 99.338% ( 76, 110 ] 48 0.037% 99.375% ( 110, 170 ] 175 0.134% 99.509% ( 170, 250 ] 103 0.079% 99.588% ( 250, 380 ] 83 0.063% 99.651% ( 380, 580 ] 105 0.080% 99.732% ( 580, 870 ] 92 0.070% 99.802% ( 870, 1300 ] 87 0.067% 99.868% ( 1300, 1900 ] 54 0.041% 99.910% ( 1900, 2900 ] 48 0.037% 99.946% ( 2900, 4400 ] 53 0.041% 99.987% ( 4400, 6600 ] 13 0.010% 99.997% ( 6600, 9900 ] 1 0.001% 99.998% ( 9900, 14000 ] 1 0.001% 99.998% ( 14000, 22000 ] 1 0.001% 99.999% ( 22000, 33000 ] 2 0.002% 100.001% ** Level 2 read latency histogram (micros): Count: 178516 Average: 14.3813 StdDev: 235.07 Min: 1 Median: 1.9776 Max: 46660 Percentiles: P50: 1.98 P75: 3.10 P99: 12.82 P99.9: 2782.45 P99.99: 4269.32 ------------------------------------------------------ [ 0, 1 ] 4725 2.647% 2.647% # ( 1, 2 ] 86473 48.440% 51.087% ########## ( 2, 3 ] 40869 22.894% 73.980% ##### ( 3, 4 ] 18986 10.635% 84.616% ## ( 4, 6 ] 19668 11.017% 95.633% ## ( 6, 10 ] 5398 3.024% 98.657% # ( 10, 15 ] 1083 0.607% 99.264% ( 15, 22 ] 164 0.092% 99.356% ( 22, 34 ] 25 0.014% 99.370% ( 34, 51 ] 6 0.003% 99.373% ( 51, 76 ] 6 0.003% 99.377% ( 76, 110 ] 7 0.004% 99.380% ( 110, 170 ] 18 0.010% 99.391% ( 170, 250 ] 39 0.022% 99.412% ( 250, 380 ] 93 0.052% 99.464% ( 380, 580 ] 164 0.092% 99.556% ( 580, 870 ] 196 0.110% 99.666% ( 870, 1300 ] 170 0.095% 99.761% ( 1300, 1900 ] 131 0.073% 99.835% ( 1900, 2900 ] 132 0.074% 99.909% ( 2900, 4400 ] 159 0.089% 99.998% ( 4400, 6600 ] 52 0.029% 100.027% ( 6600, 9900 ] 9 0.005% 100.032% ( 9900, 14000 ] 1 0.001% 100.032% ( 14000, 22000 ] 5 0.003% 100.035% ( 22000, 33000 ] 1 0.001% 100.036% ( 33000, 50000 ] 1 0.001% 100.036% ** Level 3 read latency histogram (micros): Count: 1259586 Average: 11.3897 StdDev: 220.93 Min: 1 Median: 1.9840 Max: 53375 Percentiles: P50: 1.98 P75: 2.99 P99: 9.72 P99.9: 667.46 P99.99: 1036.20 ------------------------------------------------------ [ 0, 1 ] 22615 1.795% 1.795% ( 1, 2 ] 617079 48.991% 50.786% ########## ( 2, 3 ] 307071 24.379% 75.165% ##### ( 3, 4 ] 134577 10.684% 85.849% ## ( 4, 6 ] 137346 10.904% 96.753% ## ( 6, 10 ] 30418 2.415% 99.168% ( 10, 15 ] 6158 0.489% 99.657% ( 15, 22 ] 815 0.065% 99.722% ( 22, 34 ] 73 0.006% 99.727% ( 34, 51 ] 69 0.005% 99.733% ( 51, 76 ] 63 0.005% 99.738% ( 76, 110 ] 91 0.007% 99.745% ( 110, 170 ] 263 0.021% 99.766% ( 170, 250 ] 224 0.018% 99.784% ( 250, 380 ] 428 0.034% 99.818% ( 380, 580 ] 727 0.058% 99.875% ( 580, 870 ] 1026 0.081% 99.957% ( 870, 1300 ] 1079 0.086% 100.043% ( 1300, 1900 ] 774 0.061% 100.104% ( 1900, 2900 ] 741 0.059% 100.163% ( 2900, 4400 ] 702 0.056% 100.219% ( 4400, 6600 ] 212 0.017% 100.235% ( 6600, 9900 ] 41 0.003% 100.239% ( 9900, 14000 ] 14 0.001% 100.240% ( 14000, 22000 ] 23 0.002% 100.242% ( 22000, 33000 ] 17 0.001% 100.243% ( 33000, 50000 ] 8 0.001% 100.244% ( 50000, 75000 ] 2 0.000% 100.244% ** Level 4 read latency histogram (micros): Count: 369037 Average: 31.3289 StdDev: 275.97 Min: 0 Median: 1.9828 Max: 41887 Percentiles: P50: 1.98 P75: 3.07 P99: 857.28 P99.9: 3202.65 P99.99: 4116.78 ------------------------------------------------------ [ 0, 1 ] 7363 1.995% 1.995% ( 1, 2 ] 180262 48.847% 50.842% ########## ( 2, 3 ] 86632 23.475% 74.317% ##### ( 3, 4 ] 38011 10.300% 84.617% ## ( 4, 6 ] 35696 9.673% 94.290% ## ( 6, 10 ] 7941 2.152% 96.442% ( 10, 15 ] 1538 0.417% 96.858% ( 15, 22 ] 180 0.049% 96.907% ( 22, 34 ] 22 0.006% 96.913% ( 34, 51 ] 14 0.004% 96.917% ( 51, 76 ] 19 0.005% 96.922% ( 76, 110 ] 619 0.168% 97.090% ( 110, 170 ] 2045 0.554% 97.644% ( 170, 250 ] 1299 0.352% 97.996% ( 250, 380 ] 985 0.267% 98.263% ( 380, 580 ] 1296 0.351% 98.614% ( 580, 870 ] 1490 0.404% 99.018% ( 870, 1300 ] 1416 0.384% 99.401% ( 1300, 1900 ] 989 0.268% 99.669% ( 1900, 2900 ] 741 0.201% 99.870% ( 2900, 4400 ] 545 0.148% 100.018% ( 4400, 6600 ] 161 0.044% 100.062% ( 6600, 9900 ] 12 0.003% 100.065% ( 9900, 14000 ] 3 0.001% 100.066% ( 14000, 22000 ] 3 0.001% 100.066% ( 22000, 33000 ] 3 0.001% 100.067% ( 33000, 50000 ] 2 0.001% 100.068% ** Level 5 read latency histogram (micros): Count: 47171 Average: 696.2622 StdDev: 1327.90 Min: 77 Median: 256.5740 Max: 55978 Percentiles: P50: 256.57 P75: 728.32 P99: 5492.57 P99.9: 9665.67 P99.99: 44255.24 ------------------------------------------------------ ( 76, 110 ] 3749 7.948% 7.948% ## ( 110, 170 ] 12458 26.410% 34.358% ##### ( 170, 250 ] 7139 15.134% 49.492% ### ( 250, 380 ] 4746 10.061% 59.554% ## ( 380, 580 ] 5077 10.763% 70.317% ## ( 580, 870 ] 4321 9.160% 79.477% ## ( 870, 1300 ] 3279 6.951% 86.428% # ( 1300, 1900 ] 2094 4.439% 90.867% # ( 1900, 2900 ] 1788 3.790% 94.658% # ( 2900, 4400 ] 1723 3.653% 98.310% # ( 4400, 6600 ] 657 1.393% 99.703% ( 6600, 9900 ] 101 0.214% 99.917% ( 9900, 14000 ] 9 0.019% 99.936% ( 14000, 22000 ] 12 0.025% 99.962% ( 22000, 33000 ] 7 0.015% 99.977% ( 33000, 50000 ] 11 0.023% 100.000% ( 50000, 75000 ] 2 0.004% 100.004% ** DB Stats ** Uptime(secs): 12.0 total, 4.2 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 163.68 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.142 H:M:S, 34.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 236.11 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.601 H:M:S, 14.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/14-12:50:52 ... thread 0: (1000000,3000000) ops and (294430.5,196750.3) ops/second in (3.396387,15.247756) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/4 2.83 GB 264.3 2.5 0.0 2.5 5.3 2.8 0.0 1.9 97.8 207.5 26.40 18.19 50 0.528 2622K 2964 0.0 0.0 L1 10/0 1.10 GB 113.0 1.1 0.6 0.5 1.1 0.6 0.0 1.9 93.1 92.7 12.18 5.85 1 12.183 1151K 4589 0.0 0.0 L3 11/0 884.78 MB 0.9 1.9 1.2 0.7 1.9 1.1 0.0 1.6 106.1 104.4 18.17 9.08 10 1.817 682K 0 0.0 0.0 L4 165/0 9.59 GB 1.0 6.1 4.7 1.4 6.1 4.7 0.2 1.3 112.8 112.3 55.50 29.87 42 1.321 2622K 1728 0.0 0.0 L5 1124/0 92.46 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1324/4 106.84 GB 0.0 11.6 6.5 5.2 14.4 9.2 5.2 5.1 106.0 131.3 112.26 62.99 103 1.090 7080K 9281 0.0 0.0 Int 0/0 0.00 KB 0.0 11.6 6.5 5.2 14.4 9.2 5.2 5.1 106.0 131.3 112.26 62.99 103 1.090 7080K 9281 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.6 6.5 5.2 11.6 6.4 0.0 0.0 121.6 120.9 97.90 54.83 62 1.579 7080K 9281 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 201.8 14.36 8.16 41 0.350 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.4 total, 15.2 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.39 GB write, 955.68 MB/s write, 11.62 GB read, 771.86 MB/s read, 112.3 seconds Interval compaction: 14.39 GB write, 966.52 MB/s write, 11.62 GB read, 780.62 MB/s read, 112.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 35, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 35, interval: 35 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 835219 Average: 12.1412 StdDev: 181.67 Min: 0 Median: 1.6035 Max: 44235 Percentiles: P50: 1.60 P75: 2.22 P99: 206.37 P99.9: 1438.78 P99.99: 3227.68 ------------------------------------------------------ [ 0, 1 ] 138575 16.591% 16.591% ### ( 1, 2 ] 462343 55.356% 71.947% ########### ( 2, 3 ] 116811 13.986% 85.933% ### ( 3, 4 ] 46582 5.577% 91.510% # ( 4, 6 ] 43517 5.210% 96.721% # ( 6, 10 ] 9262 1.109% 97.829% ( 10, 15 ] 1981 0.237% 98.067% ( 15, 22 ] 252 0.030% 98.097% ( 22, 34 ] 28 0.003% 98.100% ( 34, 51 ] 16 0.002% 98.102% ( 51, 76 ] 15 0.002% 98.104% ( 76, 110 ] 1282 0.153% 98.257% ( 110, 170 ] 4818 0.577% 98.834% ( 170, 250 ] 3048 0.365% 99.199% ( 250, 380 ] 1796 0.215% 99.414% ( 380, 580 ] 1811 0.217% 99.631% ( 580, 870 ] 1328 0.159% 99.790% ( 870, 1300 ] 815 0.098% 99.888% ( 1300, 1900 ] 453 0.054% 99.942% ( 1900, 2900 ] 341 0.041% 99.983% ( 2900, 4400 ] 286 0.034% 100.017% ( 4400, 6600 ] 75 0.009% 100.026% ( 6600, 9900 ] 7 0.001% 100.027% ( 9900, 14000 ] 2 0.000% 100.027% ( 14000, 22000 ] 12 0.001% 100.028% ( 22000, 33000 ] 9 0.001% 100.029% ( 33000, 50000 ] 4 0.000% 100.030% ** Level 1 read latency histogram (micros): Count: 139164 Average: 10.8417 StdDev: 160.37 Min: 1 Median: 2.3648 Max: 27831 Percentiles: P50: 2.36 P75: 3.56 P99: 133.80 P99.9: 1712.10 P99.99: 4752.61 ------------------------------------------------------ [ 0, 1 ] 1873 1.346% 1.346% ( 1, 2 ] 53723 38.604% 39.950% ######## ( 2, 3 ] 38343 27.552% 67.502% ###### ( 3, 4 ] 18598 13.364% 80.866% ### ( 4, 6 ] 18473 13.274% 94.141% ### ( 6, 10 ] 4999 3.592% 97.733% # ( 10, 15 ] 1000 0.719% 98.451% ( 15, 22 ] 261 0.188% 98.639% ( 22, 34 ] 26 0.019% 98.658% ( 34, 51 ] 11 0.008% 98.666% ( 51, 76 ] 7 0.005% 98.671% ( 76, 110 ] 212 0.152% 98.823% ( 110, 170 ] 621 0.446% 99.269% ( 170, 250 ] 271 0.195% 99.464% ( 250, 380 ] 166 0.119% 99.583% ( 380, 580 ] 178 0.128% 99.711% ( 580, 870 ] 123 0.088% 99.800% ( 870, 1300 ] 100 0.072% 99.871% ( 1300, 1900 ] 58 0.042% 99.913% ( 1900, 2900 ] 50 0.036% 99.949% ( 2900, 4400 ] 55 0.040% 99.989% ( 4400, 6600 ] 13 0.009% 99.998% ( 6600, 9900 ] 1 0.001% 99.999% ( 9900, 14000 ] 1 0.001% 99.999% ( 14000, 22000 ] 1 0.001% 100.000% ( 22000, 33000 ] 2 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 178516 Average: 14.3813 StdDev: 235.07 Min: 1 Median: 1.9776 Max: 46660 Percentiles: P50: 1.98 P75: 3.10 P99: 12.82 P99.9: 2782.45 P99.99: 4269.32 ------------------------------------------------------ [ 0, 1 ] 4725 2.647% 2.647% # ( 1, 2 ] 86473 48.440% 51.087% ########## ( 2, 3 ] 40869 22.894% 73.980% ##### ( 3, 4 ] 18986 10.635% 84.616% ## ( 4, 6 ] 19668 11.017% 95.633% ## ( 6, 10 ] 5398 3.024% 98.657% # ( 10, 15 ] 1083 0.607% 99.264% ( 15, 22 ] 164 0.092% 99.356% ( 22, 34 ] 25 0.014% 99.370% ( 34, 51 ] 6 0.003% 99.373% ( 51, 76 ] 6 0.003% 99.377% ( 76, 110 ] 7 0.004% 99.380% ( 110, 170 ] 18 0.010% 99.391% ( 170, 250 ] 39 0.022% 99.412% ( 250, 380 ] 93 0.052% 99.464% ( 380, 580 ] 164 0.092% 99.556% ( 580, 870 ] 196 0.110% 99.666% ( 870, 1300 ] 170 0.095% 99.761% ( 1300, 1900 ] 131 0.073% 99.835% ( 1900, 2900 ] 132 0.074% 99.909% ( 2900, 4400 ] 159 0.089% 99.998% ( 4400, 6600 ] 52 0.029% 100.027% ( 6600, 9900 ] 9 0.005% 100.032% ( 9900, 14000 ] 1 0.001% 100.032% ( 14000, 22000 ] 5 0.003% 100.035% ( 22000, 33000 ] 1 0.001% 100.036% ( 33000, 50000 ] 1 0.001% 100.036% ** Level 3 read latency histogram (micros): Count: 1260302 Average: 11.4942 StdDev: 220.94 Min: 1 Median: 1.9845 Max: 53375 Percentiles: P50: 1.98 P75: 2.99 P99: 9.81 P99.9: 672.50 P99.99: 1038.18 ------------------------------------------------------ [ 0, 1 ] 22615 1.794% 1.794% ( 1, 2 ] 617085 48.963% 50.758% ########## ( 2, 3 ] 307078 24.365% 75.123% ##### ( 3, 4 ] 134584 10.679% 85.802% ## ( 4, 6 ] 137347 10.898% 96.700% ## ( 6, 10 ] 30418 2.414% 99.113% ( 10, 15 ] 6158 0.489% 99.602% ( 15, 22 ] 815 0.065% 99.667% ( 22, 34 ] 73 0.006% 99.672% ( 34, 51 ] 69 0.005% 99.678% ( 51, 76 ] 63 0.005% 99.683% ( 76, 110 ] 207 0.016% 99.699% ( 110, 170 ] 584 0.046% 99.746% ( 170, 250 ] 357 0.028% 99.774% ( 250, 380 ] 487 0.039% 99.813% ( 380, 580 ] 769 0.061% 99.874% ( 580, 870 ] 1043 0.083% 99.956% ( 870, 1300 ] 1084 0.086% 100.042% ( 1300, 1900 ] 776 0.062% 100.104% ( 1900, 2900 ] 741 0.059% 100.163% ( 2900, 4400 ] 702 0.056% 100.218% ( 4400, 6600 ] 212 0.017% 100.235% ( 6600, 9900 ] 41 0.003% 100.239% ( 9900, 14000 ] 14 0.001% 100.240% ( 14000, 22000 ] 23 0.002% 100.241% ( 22000, 33000 ] 17 0.001% 100.243% ( 33000, 50000 ] 8 0.001% 100.243% ( 50000, 75000 ] 2 0.000% 100.244% ** Level 4 read latency histogram (micros): Count: 376870 Average: 34.4615 StdDev: 274.89 Min: 0 Median: 2.0082 Max: 41887 Percentiles: P50: 2.01 P75: 3.21 P99: 853.92 P99.9: 3180.58 P99.99: 4112.40 ------------------------------------------------------ [ 0, 1 ] 7363 1.954% 1.954% ( 1, 2 ] 180362 47.858% 49.812% ########## ( 2, 3 ] 86921 23.064% 72.876% ##### ( 3, 4 ] 38189 10.133% 83.009% ## ( 4, 6 ] 35835 9.509% 92.517% ## ( 6, 10 ] 7961 2.112% 94.630% ( 10, 15 ] 1541 0.409% 95.039% ( 15, 22 ] 180 0.048% 95.086% ( 22, 34 ] 22 0.006% 95.092% ( 34, 51 ] 14 0.004% 95.096% ( 51, 76 ] 19 0.005% 95.101% ( 76, 110 ] 1881 0.499% 95.600% ( 110, 170 ] 5375 1.426% 97.026% ( 170, 250 ] 2489 0.660% 97.687% ( 250, 380 ] 1581 0.420% 98.106% ( 380, 580 ] 1806 0.479% 98.585% ( 580, 870 ] 1654 0.439% 99.024% ( 870, 1300 ] 1460 0.387% 99.412% ( 1300, 1900 ] 995 0.264% 99.676% ( 1900, 2900 ] 743 0.197% 99.873% ( 2900, 4400 ] 546 0.145% 100.018% ( 4400, 6600 ] 161 0.043% 100.060% ( 6600, 9900 ] 12 0.003% 100.064% ( 9900, 14000 ] 3 0.001% 100.064% ( 14000, 22000 ] 3 0.001% 100.065% ( 22000, 33000 ] 3 0.001% 100.066% ( 33000, 50000 ] 2 0.001% 100.067% ** Level 5 read latency histogram (micros): Count: 91758 Average: 455.6831 StdDev: 989.89 Min: 74 Median: 177.1201 Max: 55978 Percentiles: P50: 177.12 P75: 395.33 P99: 4292.66 P99.9: 8078.20 P99.99: 31152.31 ------------------------------------------------------ ( 51, 76 ] 6 0.007% 0.007% ( 76, 110 ] 11683 12.732% 12.739% ### ( 110, 170 ] 32850 35.801% 48.540% ####### ( 170, 250 ] 15056 16.408% 64.948% ### ( 250, 380 ] 8599 9.371% 74.319% ## ( 380, 580 ] 8145 8.877% 83.196% ## ( 580, 870 ] 5428 5.916% 89.112% # ( 870, 1300 ] 3524 3.841% 92.952% # ( 1300, 1900 ] 2130 2.321% 95.273% ( 1900, 2900 ] 1816 1.979% 97.253% ( 2900, 4400 ] 1727 1.882% 99.135% ( 4400, 6600 ] 657 0.716% 99.851% ( 6600, 9900 ] 101 0.110% 99.961% ( 9900, 14000 ] 9 0.010% 99.971% ( 14000, 22000 ] 12 0.013% 99.984% ( 22000, 33000 ] 7 0.008% 99.991% ( 33000, 50000 ] 11 0.012% 100.003% ( 50000, 75000 ] 2 0.002% 100.005% ** DB Stats ** Uptime(secs): 15.4 total, 3.4 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 191.44 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.142 H:M:S, 26.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 289.77 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 2024/12/14-12:50:55 ... thread 0: (1000000,4000000) ops and (283535.2,213053.2) ops/second in (3.526899,18.774655) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/4 3.75 GB 358.7 3.5 0.0 3.5 7.3 3.7 0.0 1.9 108.4 224.2 33.14 24.44 69 0.480 3646K 3985 0.0 0.0 L1 10/0 1.10 GB 113.0 1.1 0.6 0.5 1.1 0.6 0.0 1.9 93.1 92.7 12.18 5.85 1 12.183 1151K 4589 0.0 0.0 L3 11/0 884.78 MB 0.9 1.9 1.2 0.7 1.9 1.1 0.0 1.6 106.1 104.4 18.17 9.08 10 1.817 682K 0 0.0 0.0 L4 165/0 9.59 GB 1.0 6.1 4.7 1.4 6.1 4.7 0.2 1.3 112.8 112.3 55.50 29.87 42 1.321 2622K 1728 0.0 0.0 L5 1124/0 92.46 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.9 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1327/4 107.76 GB 0.0 12.6 6.5 6.2 16.3 10.1 5.2 4.3 108.5 140.3 118.99 69.24 122 0.975 8103K 10K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.6 6.5 6.2 16.3 10.1 5.2 4.3 108.5 140.3 118.99 69.24 122 0.975 8103K 10K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.6 6.5 6.2 12.5 6.4 0.0 0.0 126.9 126.3 101.75 58.58 66 1.542 8103K 10K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 222.9 17.24 10.66 56 0.308 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 18.9 total, 18.8 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.30 GB write, 880.81 MB/s write, 12.61 GB read, 681.39 MB/s read, 119.0 seconds Interval compaction: 16.30 GB write, 888.93 MB/s write, 12.61 GB read, 687.67 MB/s read, 119.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 1, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 1, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 35, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 1, total-stops: 35, interval: 36 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1101579 Average: 10.2653 StdDev: 158.86 Min: 0 Median: 1.5318 Max: 44235 Percentiles: P50: 1.53 P75: 1.98 P99: 171.09 P99.9: 1174.67 P99.99: 2753.77 ------------------------------------------------------ [ 0, 1 ] 225645 20.484% 20.484% #### ( 1, 2 ] 611387 55.501% 75.985% ########### ( 2, 3 ] 134253 12.187% 88.172% ## ( 3, 4 ] 52498 4.766% 92.938% # ( 4, 6 ] 46260 4.199% 97.137% # ( 6, 10 ] 9849 0.894% 98.031% ( 10, 15 ] 2171 0.197% 98.228% ( 15, 22 ] 282 0.026% 98.254% ( 22, 34 ] 31 0.003% 98.257% ( 34, 51 ] 16 0.001% 98.258% ( 51, 76 ] 16 0.001% 98.260% ( 76, 110 ] 1730 0.157% 98.417% ( 110, 170 ] 6374 0.579% 98.995% ( 170, 250 ] 3892 0.353% 99.349% ( 250, 380 ] 2093 0.190% 99.539% ( 380, 580 ] 2008 0.182% 99.721% ( 580, 870 ] 1392 0.126% 99.847% ( 870, 1300 ] 822 0.075% 99.922% ( 1300, 1900 ] 459 0.042% 99.964% ( 1900, 2900 ] 343 0.031% 99.995% ( 2900, 4400 ] 288 0.026% 100.021% ( 4400, 6600 ] 75 0.007% 100.028% ( 6600, 9900 ] 7 0.001% 100.028% ( 9900, 14000 ] 2 0.000% 100.029% ( 14000, 22000 ] 12 0.001% 100.030% ( 22000, 33000 ] 9 0.001% 100.030% ( 33000, 50000 ] 4 0.000% 100.031% ** Level 1 read latency histogram (micros): Count: 140451 Average: 12.1781 StdDev: 160.81 Min: 1 Median: 2.3802 Max: 27831 Percentiles: P50: 2.38 P75: 3.61 P99: 166.82 P99.9: 1709.13 P99.99: 4730.83 ------------------------------------------------------ [ 0, 1 ] 1873 1.334% 1.334% ( 1, 2 ] 53750 38.270% 39.603% ######## ( 2, 3 ] 38410 27.348% 66.951% ##### ( 3, 4 ] 18630 13.264% 80.215% ### ( 4, 6 ] 18504 13.175% 93.390% ### ( 6, 10 ] 5003 3.562% 96.952% # ( 10, 15 ] 1001 0.713% 97.665% ( 15, 22 ] 261 0.186% 97.850% ( 22, 34 ] 26 0.019% 97.869% ( 34, 51 ] 11 0.008% 97.877% ( 51, 76 ] 7 0.005% 97.882% ( 76, 110 ] 437 0.311% 98.193% ( 110, 170 ] 1197 0.852% 99.045% ( 170, 250 ] 445 0.317% 99.362% ( 250, 380 ] 241 0.172% 99.534% ( 380, 580 ] 235 0.167% 99.701% ( 580, 870 ] 138 0.098% 99.799% ( 870, 1300 ] 102 0.073% 99.872% ( 1300, 1900 ] 58 0.041% 99.913% ( 1900, 2900 ] 51 0.036% 99.949% ( 2900, 4400 ] 55 0.039% 99.989% ( 4400, 6600 ] 13 0.009% 99.998% ( 6600, 9900 ] 1 0.001% 99.999% ( 9900, 14000 ] 1 0.001% 99.999% ( 14000, 22000 ] 1 0.001% 100.000% ( 22000, 33000 ] 2 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 178516 Average: 14.3813 StdDev: 235.07 Min: 1 Median: 1.9776 Max: 46660 Percentiles: P50: 1.98 P75: 3.10 P99: 12.82 P99.9: 2782.45 P99.99: 4269.32 ------------------------------------------------------ [ 0, 1 ] 4725 2.647% 2.647% # ( 1, 2 ] 86473 48.440% 51.087% ########## ( 2, 3 ] 40869 22.894% 73.980% ##### ( 3, 4 ] 18986 10.635% 84.616% ## ( 4, 6 ] 19668 11.017% 95.633% ## ( 6, 10 ] 5398 3.024% 98.657% # ( 10, 15 ] 1083 0.607% 99.264% ( 15, 22 ] 164 0.092% 99.356% ( 22, 34 ] 25 0.014% 99.370% ( 34, 51 ] 6 0.003% 99.373% ( 51, 76 ] 6 0.003% 99.377% ( 76, 110 ] 7 0.004% 99.380% ( 110, 170 ] 18 0.010% 99.391% ( 170, 250 ] 39 0.022% 99.412% ( 250, 380 ] 93 0.052% 99.464% ( 380, 580 ] 164 0.092% 99.556% ( 580, 870 ] 196 0.110% 99.666% ( 870, 1300 ] 170 0.095% 99.761% ( 1300, 1900 ] 131 0.073% 99.835% ( 1900, 2900 ] 132 0.074% 99.909% ( 2900, 4400 ] 159 0.089% 99.998% ( 4400, 6600 ] 52 0.029% 100.027% ( 6600, 9900 ] 9 0.005% 100.032% ( 9900, 14000 ] 1 0.001% 100.032% ( 14000, 22000 ] 5 0.003% 100.035% ( 22000, 33000 ] 1 0.001% 100.036% ( 33000, 50000 ] 1 0.001% 100.036% ** Level 3 read latency histogram (micros): Count: 1261135 Average: 11.6067 StdDev: 220.93 Min: 1 Median: 1.9852 Max: 53375 Percentiles: P50: 1.99 P75: 3.00 P99: 9.92 P99.9: 675.17 P99.99: 1038.39 ------------------------------------------------------ [ 0, 1 ] 22615 1.793% 1.793% ( 1, 2 ] 617085 48.931% 50.724% ########## ( 2, 3 ] 307078 24.349% 75.073% ##### ( 3, 4 ] 134584 10.672% 85.745% ## ( 4, 6 ] 137347 10.891% 96.636% ## ( 6, 10 ] 30418 2.412% 99.048% ( 10, 15 ] 6158 0.488% 99.536% ( 15, 22 ] 815 0.065% 99.601% ( 22, 34 ] 73 0.006% 99.607% ( 34, 51 ] 69 0.005% 99.612% ( 51, 76 ] 64 0.005% 99.617% ( 76, 110 ] 373 0.030% 99.647% ( 110, 170 ] 994 0.079% 99.725% ( 170, 250 ] 485 0.038% 99.764% ( 250, 380 ] 551 0.044% 99.808% ( 380, 580 ] 818 0.065% 99.872% ( 580, 870 ] 1057 0.084% 99.956% ( 870, 1300 ] 1085 0.086% 100.042% ( 1300, 1900 ] 776 0.062% 100.104% ( 1900, 2900 ] 741 0.059% 100.163% ( 2900, 4400 ] 702 0.056% 100.218% ( 4400, 6600 ] 212 0.017% 100.235% ( 6600, 9900 ] 41 0.003% 100.238% ( 9900, 14000 ] 14 0.001% 100.239% ( 14000, 22000 ] 23 0.002% 100.241% ( 22000, 33000 ] 17 0.001% 100.243% ( 33000, 50000 ] 8 0.001% 100.243% ( 50000, 75000 ] 2 0.000% 100.243% ** Level 4 read latency histogram (micros): Count: 385681 Average: 37.7310 StdDev: 273.41 Min: 0 Median: 2.0583 Max: 41887 Percentiles: P50: 2.06 P75: 3.38 P99: 846.83 P99.9: 3160.91 P99.99: 4111.04 ------------------------------------------------------ [ 0, 1 ] 7363 1.909% 1.909% ( 1, 2 ] 180405 46.776% 48.685% ######### ( 2, 3 ] 87077 22.577% 71.262% ##### ( 3, 4 ] 38250 9.918% 81.180% ## ( 4, 6 ] 35892 9.306% 90.486% ## ( 6, 10 ] 7965 2.065% 92.551% ( 10, 15 ] 1544 0.400% 92.951% ( 15, 22 ] 181 0.047% 92.998% ( 22, 34 ] 22 0.006% 93.004% ( 34, 51 ] 14 0.004% 93.008% ( 51, 76 ] 22 0.006% 93.013% ( 76, 110 ] 3593 0.932% 93.945% ( 110, 170 ] 9551 2.476% 96.421% ( 170, 250 ] 3866 1.002% 97.424% ( 250, 380 ] 2142 0.555% 97.979% ( 380, 580 ] 2281 0.591% 98.571% ( 580, 870 ] 1800 0.467% 99.037% ( 870, 1300 ] 1488 0.386% 99.423% ( 1300, 1900 ] 999 0.259% 99.682% ( 1900, 2900 ] 745 0.193% 99.875% ( 2900, 4400 ] 548 0.142% 100.017% ( 4400, 6600 ] 161 0.042% 100.059% ( 6600, 9900 ] 12 0.003% 100.062% ( 9900, 14000 ] 3 0.001% 100.063% ( 14000, 22000 ] 3 0.001% 100.064% ( 22000, 33000 ] 3 0.001% 100.065% ( 33000, 50000 ] 2 0.001% 100.065% ** Level 5 read latency histogram (micros): Count: 141115 Average: 360.7345 StdDev: 812.67 Min: 74 Median: 161.6054 Max: 55978 Percentiles: P50: 161.61 P75: 294.90 P99: 3864.96 P99.9: 6582.87 P99.99: 21925.67 ------------------------------------------------------ ( 51, 76 ] 21 0.015% 0.015% ( 76, 110 ] 21815 15.459% 15.474% ### ( 110, 170 ] 56647 40.142% 55.616% ######## ( 170, 250 ] 23217 16.453% 72.069% ### ( 250, 380 ] 11976 8.487% 80.556% ## ( 380, 580 ] 11065 7.841% 88.397% ## ( 580, 870 ] 6209 4.400% 92.797% # ( 870, 1300 ] 3644 2.582% 95.379% # ( 1300, 1900 ] 2163 1.533% 96.912% ( 1900, 2900 ] 1832 1.298% 98.210% ( 2900, 4400 ] 1733 1.228% 99.438% ( 4400, 6600 ] 657 0.466% 99.904% ( 6600, 9900 ] 101 0.072% 99.975% ( 9900, 14000 ] 9 0.006% 99.982% ( 14000, 22000 ] 12 0.009% 99.990% ( 22000, 33000 ] 7 0.005% 99.995% ( 33000, 50000 ] 11 0.008% 100.003% ( 50000, 75000 ] 2 0.001% 100.004% ** DB Stats ** Uptime(secs): 18.9 total, 3.5 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 207.75 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.201 H:M:S, 22.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 279.06 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.059 H:M:S, 1.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 226.048 micros/op 17674 ops/sec 60.141 seconds 1062996 operations; 17.1 MB/s (271999 of 271999 found) Microseconds per read: Count: 1062996 Average: 226.0494 StdDev: 677.02 Min: 2 Median: 168.1645 Max: 85009 Percentiles: P50: 168.16 P75: 226.62 P99: 891.75 P99.9: 10397.29 P99.99: 29360.43 ------------------------------------------------------ ( 1, 2 ] 3 0.000% 0.000% ( 2, 3 ] 30 0.003% 0.003% ( 3, 4 ] 73 0.007% 0.010% ( 4, 6 ] 106 0.010% 0.020% ( 6, 10 ] 407 0.038% 0.058% ( 10, 15 ] 3776 0.355% 0.413% ( 15, 22 ] 6452 0.607% 1.020% ( 22, 34 ] 11019 1.037% 2.057% ( 34, 51 ] 3084 0.290% 2.347% ( 51, 76 ] 441 0.041% 2.389% ( 76, 110 ] 23939 2.252% 4.641% ( 110, 170 ] 497384 46.791% 51.431% ######### ( 170, 250 ] 354009 33.303% 84.734% ####### ( 250, 380 ] 103103 9.699% 94.434% ## ( 380, 580 ] 36200 3.405% 97.839% # ( 580, 870 ] 12110 1.139% 98.978% ( 870, 1300 ] 4547 0.428% 99.406% ( 1300, 1900 ] 2040 0.192% 99.598% ( 1900, 2900 ] 1480 0.139% 99.737% ( 2900, 4400 ] 1080 0.102% 99.839% ( 4400, 6600 ] 363 0.034% 99.873% ( 6600, 9900 ] 234 0.022% 99.895% ( 9900, 14000 ] 437 0.041% 99.936% ( 14000, 22000 ] 473 0.044% 99.981% ( 22000, 33000 ] 149 0.014% 99.995% ( 33000, 50000 ] 43 0.004% 99.999% ( 50000, 75000 ] 13 0.001% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 22564593 rocksdb.bloom.filter.full.positive COUNT : 1282012 rocksdb.bloom.filter.full.true.positive COUNT : 1062523 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 473 rocksdb.memtable.miss COUNT : 1062523 rocksdb.l0.hit COUNT : 44818 rocksdb.l1.hit COUNT : 11284 rocksdb.l2andup.hit COUNT : 1006421 rocksdb.compaction.key.drop.new COUNT : 103037 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 4788086 rocksdb.number.keys.read COUNT : 1062996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 4941304752 rocksdb.bytes.read COUNT : 1062996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1551 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 41827000 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 16146398180 rocksdb.compact.write.bytes COUNT : 16041806883 rocksdb.flush.write.bytes COUNT : 4888454600 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 24 rocksdb.number.superversion_acquires COUNT : 919 rocksdb.number.superversion_releases COUNT : 885 rocksdb.number.superversion_cleanups COUNT : 229 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 212 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 4848866304 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1730560 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 21016321764 rocksdb.non.last.level.read.count COUNT : 4936277 rocksdb.block.checksum.compute.count COUNT : 5207782 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 167.261651 P95 : 411.455219 P99 : 889.480159 P100 : 85008.000000 COUNT : 1062996 SUM : 239355363 rocksdb.db.write.micros P50 : 1.521893 P95 : 4.384801 P99 : 9.682373 P100 : 896880.000000 COUNT : 4788086 SUM : 53770274 rocksdb.compaction.times.micros P50 : 1337500.000000 P95 : 2755000.000000 P99 : 11102000.000000 P100 : 12182728.000000 COUNT : 69 SUM : 104541743 rocksdb.compaction.times.cpu_micros P50 : 823750.000000 P95 : 1709444.444444 P99 : 4982000.000000 P100 : 5851121.000000 COUNT : 69 SUM : 61323405 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 771.911765 P95 : 94250.000000 P99 : 407935.000000 P100 : 407935.000000 COUNT : 69 SUM : 1229399 rocksdb.compaction.outfile.sync.micros P50 : 29562.500000 P95 : 292611.111111 P99 : 493050.000000 P100 : 1127083.000000 COUNT : 181 SUM : 13873423 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 169.142857 P95 : 4198.076923 P99 : 11745.000000 P100 : 35955.000000 COUNT : 155 SUM : 190638 rocksdb.table.open.io.micros P50 : 8766.908714 P95 : 19796.449704 P99 : 27496.451613 P100 : 31537.000000 COUNT : 1551 SUM : 14070389 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.959676 P95 : 81.271166 P99 : 186.868382 P100 : 53379.000000 COUNT : 3910897 SUM : 70073564 rocksdb.read.block.get.micros P50 : 133.629490 P95 : 264.558825 P99 : 827.382609 P100 : 55983.000000 COUNT : 1295344 SUM : 200443411 rocksdb.write.raw.block.micros P50 : 0.547810 P95 : 1.565256 P99 : 3.919512 P100 : 95902.000000 COUNT : 5069516 SUM : 15711979 rocksdb.numfiles.in.singlecompaction P50 : 2.780000 P95 : 3.987143 P99 : 23.320000 P100 : 27.000000 COUNT : 89 SUM : 313 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.593589 P95 : 2308.113435 P99 : 2782.266291 P100 : 896693.000000 COUNT : 147343 SUM : 41840805 rocksdb.sst.read.micros P50 : 2.573246 P95 : 167.704087 P99 : 454.565639 P100 : 55978.000000 COUNT : 4936327 SUM : 256269885 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1062996 SUM : 1062996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 4788086 SUM : 4941304752 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 219285.714286 P95 : 906750.000000 P99 : 1141350.000000 P100 : 1148611.000000 COUNT : 69 SUM : 19826563 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 222 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0