[2024-12-13 15:26:44.454326] Starting SPDK v24.09.1-pre git sha1 b18e1bd62 / DPDK 22.11.4 initialization... [2024-12-13 15:26:44.454468] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1217725 ] Set seed to 1734100004450835 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-13 15:26:44.542538] app.c: 917:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-13 15:26:44.580110] app.c: 610:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-13 15:26:44.580156] app.c: 611:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1217725' to capture a snapshot of events at runtime. [2024-12-13 15:26:44.580166] app.c: 616:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-13 15:26:44.580177] app.c: 617:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-13 15:26:44.580185] app.c: 618:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1217725 for offline analysis/debug. [2024-12-13 15:26:44.580272] reactor.c: 990:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Fri Dec 13 15:26:48 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/13-15:26:54 ... thread 0: (1000000,1000000) ops and (175938.6,175938.6) ops/second in (5.683800,5.683800) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/12 2.52 GB 88.1 0.6 0.0 0.6 1.4 0.9 0.0 1.6 84.6 216.0 6.70 4.65 14 0.479 575K 620 0.0 0.0 L1 9/9 1.10 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 846.15 MB 0.8 0.2 0.1 0.1 0.2 0.1 0.1 2.0 155.1 154.2 1.56 1.12 1 1.561 0 0 0.0 0.0 L4 175/0 9.74 GB 1.0 3.1 1.9 1.2 3.0 1.9 0.0 1.6 118.4 117.3 26.57 14.74 17 1.563 158K 0 0.0 0.0 L5 1116/0 91.97 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1334/21 106.15 GB 0.0 3.9 2.0 1.8 4.7 2.9 3.4 5.4 113.5 137.9 34.83 20.51 32 1.088 734K 620 0.0 0.0 Int 0/0 0.00 KB 0.0 3.9 2.0 1.8 4.7 2.9 3.4 5.4 113.5 137.9 34.83 20.51 32 1.088 734K 620 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.9 2.0 1.8 3.8 2.0 0.0 0.0 129.8 128.8 30.45 18.09 20 1.523 734K 620 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 201.5 4.38 2.42 12 0.365 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 5.8 total, 5.7 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 4.69 GB write, 823.54 MB/s write, 3.86 GB read, 677.60 MB/s read, 34.8 seconds Interval compaction: 4.69 GB write, 845.11 MB/s write, 3.86 GB read, 695.35 MB/s read, 34.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 5, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 5, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 7, pending-compaction-bytes-delays: 2, pending-compaction-bytes-stops: 0, total-delays: 7, total-stops: 7, interval: 14 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 323791 Average: 12.7632 StdDev: 137.48 Min: 1 Median: 1.7422 Max: 24166 Percentiles: P50: 1.74 P75: 2.53 P99: 187.72 P99.9: 1529.98 P99.99: 2713.22 ------------------------------------------------------ [ 0, 1 ] 21353 6.595% 6.595% # ( 1, 2 ] 189355 58.481% 65.075% ############ ( 2, 3 ] 60487 18.681% 83.756% #### ( 3, 4 ] 21724 6.709% 90.465% # ( 4, 6 ] 18141 5.603% 96.068% # ( 6, 10 ] 4975 1.536% 97.605% ( 10, 15 ] 765 0.236% 97.841% ( 15, 22 ] 87 0.027% 97.868% ( 22, 34 ] 13 0.004% 97.872% ( 34, 51 ] 4 0.001% 97.873% ( 51, 76 ] 4 0.001% 97.874% ( 76, 110 ] 858 0.265% 98.139% ( 110, 170 ] 2571 0.794% 98.933% ( 170, 250 ] 980 0.303% 99.236% ( 250, 380 ] 607 0.187% 99.423% ( 380, 580 ] 590 0.182% 99.606% ( 580, 870 ] 502 0.155% 99.761% ( 870, 1300 ] 356 0.110% 99.871% ( 1300, 1900 ] 251 0.078% 99.948% ( 1900, 2900 ] 168 0.052% 100.000% ( 2900, 4400 ] 118 0.036% 100.036% ( 4400, 6600 ] 39 0.012% 100.048% ( 6600, 9900 ] 6 0.002% 100.050% ( 9900, 14000 ] 1 0.000% 100.051% ( 22000, 33000 ] 1 0.000% 100.051% ** Level 1 read latency histogram (micros): Count: 276 Average: 551.1268 StdDev: 889.87 Min: 82 Median: 220.9091 Max: 6045 Percentiles: P50: 220.91 P75: 510.43 P99: 4576.00 P99.9: 6045.00 P99.99: 6045.00 ------------------------------------------------------ ( 76, 110 ] 20 7.246% 7.246% # ( 110, 170 ] 90 32.609% 39.855% ####### ( 170, 250 ] 44 15.942% 55.797% ### ( 250, 380 ] 38 13.768% 69.565% ### ( 380, 580 ] 23 8.333% 77.899% ## ( 580, 870 ] 21 7.609% 85.507% ## ( 870, 1300 ] 12 4.348% 89.855% # ( 1300, 1900 ] 10 3.623% 93.478% # ( 1900, 2900 ] 8 2.899% 96.377% # ( 2900, 4400 ] 7 2.536% 98.913% # ( 4400, 6600 ] 3 1.087% 100.000% ** Level 2 read latency histogram (micros): Count: 31174 Average: 7.8387 StdDev: 85.08 Min: 1 Median: 1.8216 Max: 4431 Percentiles: P50: 1.82 P75: 2.68 P99: 11.54 P99.9: 1245.41 P99.99: 3764.78 ------------------------------------------------------ [ 0, 1 ] 811 2.602% 2.602% # ( 1, 2 ] 17985 57.692% 60.294% ############ ( 2, 3 ] 6769 21.714% 82.007% #### ( 3, 4 ] 2345 7.522% 89.530% ## ( 4, 6 ] 2134 6.845% 96.375% # ( 6, 10 ] 785 2.518% 98.893% # ( 10, 15 ] 108 0.346% 99.240% ( 15, 22 ] 19 0.061% 99.301% ( 22, 34 ] 1 0.003% 99.304% ( 34, 51 ] 1 0.003% 99.307% ( 51, 76 ] 2 0.006% 99.314% ( 76, 110 ] 1 0.003% 99.317% ( 110, 170 ] 13 0.042% 99.358% ( 170, 250 ] 24 0.077% 99.435% ( 250, 380 ] 43 0.138% 99.573% ( 380, 580 ] 39 0.125% 99.698% ( 580, 870 ] 41 0.132% 99.830% ( 870, 1300 ] 25 0.080% 99.910% ( 1300, 1900 ] 14 0.045% 99.955% ( 1900, 2900 ] 8 0.026% 99.981% ( 2900, 4400 ] 5 0.016% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 263016 Average: 9.6498 StdDev: 126.44 Min: 1 Median: 1.8714 Max: 25853 Percentiles: P50: 1.87 P75: 2.75 P99: 9.97 P99.9: 1182.58 P99.99: 1715.40 ------------------------------------------------------ [ 0, 1 ] 6611 2.514% 2.514% # ( 1, 2 ] 143327 54.494% 57.007% ########### ( 2, 3 ] 62718 23.846% 80.853% ##### ( 3, 4 ] 21648 8.231% 89.084% ## ( 4, 6 ] 20401 7.757% 96.840% ## ( 6, 10 ] 5727 2.177% 99.018% ( 10, 15 ] 930 0.354% 99.371% ( 15, 22 ] 122 0.046% 99.418% ( 22, 34 ] 11 0.004% 99.422% ( 34, 51 ] 9 0.003% 99.425% ( 51, 76 ] 6 0.002% 99.427% ( 76, 110 ] 39 0.015% 99.442% ( 110, 170 ] 124 0.047% 99.489% ( 170, 250 ] 113 0.043% 99.532% ( 250, 380 ] 168 0.064% 99.596% ( 380, 580 ] 267 0.102% 99.698% ( 580, 870 ] 311 0.118% 99.816% ( 870, 1300 ] 304 0.116% 99.932% ( 1300, 1900 ] 222 0.084% 100.016% ( 1900, 2900 ] 118 0.045% 100.061% ( 2900, 4400 ] 69 0.026% 100.087% ( 4400, 6600 ] 22 0.008% 100.095% ( 6600, 9900 ] 4 0.002% 100.097% ( 9900, 14000 ] 1 0.000% 100.097% ( 22000, 33000 ] 1 0.000% 100.098% ** Level 4 read latency histogram (micros): Count: 243440 Average: 21.7776 StdDev: 195.56 Min: 1 Median: 1.9373 Max: 8445 Percentiles: P50: 1.94 P75: 2.92 P99: 463.76 P99.9: 2093.29 P99.99: 2749.27 ------------------------------------------------------ [ 0, 1 ] 6380 2.621% 2.621% # ( 1, 2 ] 123054 50.548% 53.169% ########## ( 2, 3 ] 57819 23.751% 76.920% ##### ( 3, 4 ] 22670 9.312% 86.232% ## ( 4, 6 ] 20952 8.607% 94.839% ## ( 6, 10 ] 6264 2.573% 97.412% # ( 10, 15 ] 1119 0.460% 97.871% ( 15, 22 ] 161 0.066% 97.937% ( 22, 34 ] 35 0.014% 97.952% ( 34, 51 ] 8 0.003% 97.955% ( 51, 76 ] 6 0.002% 97.958% ( 76, 110 ] 271 0.111% 98.069% ( 110, 170 ] 912 0.375% 98.444% ( 170, 250 ] 568 0.233% 98.677% ( 250, 380 ] 524 0.215% 98.892% ( 380, 580 ] 627 0.258% 99.150% ( 580, 870 ] 658 0.270% 99.420% ( 870, 1300 ] 648 0.266% 99.686% ( 1300, 1900 ] 456 0.187% 99.873% ( 1900, 2900 ] 334 0.137% 100.011% ( 2900, 4400 ] 242 0.099% 100.110% ( 4400, 6600 ] 63 0.026% 100.136% ( 6600, 9900 ] 8 0.003% 100.139% ** Level 5 read latency histogram (micros): Count: 22624 Average: 692.4922 StdDev: 1111.89 Min: 73 Median: 243.1739 Max: 29814 Percentiles: P50: 243.17 P75: 698.70 P99: 5682.47 P99.9: 8864.92 P99.99: 13731.04 ------------------------------------------------------ ( 51, 76 ] 2 0.009% 0.009% ( 76, 110 ] 1965 8.685% 8.694% ## ( 110, 170 ] 6515 28.797% 37.491% ###### ( 170, 250 ] 3094 13.676% 51.167% ### ( 250, 380 ] 2314 10.228% 61.395% ## ( 380, 580 ] 2313 10.224% 71.619% ## ( 580, 870 ] 1869 8.261% 79.880% ## ( 870, 1300 ] 1364 6.029% 85.909% # ( 1300, 1900 ] 931 4.115% 90.024% # ( 1900, 2900 ] 942 4.164% 94.188% # ( 2900, 4400 ] 855 3.779% 97.967% # ( 4400, 6600 ] 401 1.772% 99.739% ( 6600, 9900 ] 53 0.234% 99.973% ( 9900, 14000 ] 4 0.018% 99.991% ( 14000, 22000 ] 1 0.004% 99.996% ( 22000, 33000 ] 2 0.009% 100.004% ** DB Stats ** Uptime(secs): 5.8 total, 5.7 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 168.69 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.763 H:M:S, 30.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 173.07 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.763 H:M:S, 31.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/13-15:27:02 ... thread 0: (1000000,2000000) ops and (128979.1,148842.8) ops/second in (7.753193,13.436993) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/12 3.50 GB 188.5 2.4 0.0 2.4 4.2 1.8 0.0 2.3 144.8 255.9 16.96 14.35 35 0.485 2494K 4588 0.0 0.0 L1 9/9 1.10 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 846.15 MB 0.8 0.2 0.1 0.1 0.2 0.1 0.1 2.0 155.1 154.2 1.56 1.12 1 1.561 0 0 0.0 0.0 L4 175/0 9.74 GB 1.0 3.1 1.9 1.2 3.0 1.9 0.0 1.6 118.4 117.3 26.57 14.74 17 1.563 158K 0 0.0 0.0 L5 1116/0 91.97 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1335/21 107.13 GB 0.0 5.7 2.0 3.7 7.5 3.8 3.4 4.1 129.6 170.7 45.09 30.21 53 0.851 2652K 4588 0.0 0.0 Int 0/0 0.00 KB 0.0 5.7 2.0 3.7 7.5 3.8 3.4 4.1 129.6 170.7 45.09 30.21 53 0.851 2652K 4588 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.7 2.0 3.7 5.7 2.0 0.0 0.0 155.6 154.6 37.56 25.07 25 1.502 2652K 4588 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 251.0 7.53 5.13 28 0.269 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 13.6 total, 13.4 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.52 GB write, 566.55 MB/s write, 5.71 GB read, 430.00 MB/s read, 45.1 seconds Interval compaction: 7.52 GB write, 572.83 MB/s write, 5.71 GB read, 434.76 MB/s read, 45.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 14, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 14, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 7, pending-compaction-bytes-delays: 2, pending-compaction-bytes-stops: 0, total-delays: 16, total-stops: 7, interval: 23 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 869790 Average: 17.3580 StdDev: 99.32 Min: 0 Median: 1.5741 Max: 24166 Percentiles: P50: 1.57 P75: 2.11 P99: 245.72 P99.9: 774.13 P99.99: 1341.96 ------------------------------------------------------ [ 0, 1 ] 156740 18.020% 18.020% #### ( 1, 2 ] 484537 55.707% 73.728% ########### ( 2, 3 ] 98500 11.325% 85.052% ## ( 3, 4 ] 33512 3.853% 88.905% # ( 4, 6 ] 24742 2.845% 91.750% # ( 6, 10 ] 6218 0.715% 92.465% ( 10, 15 ] 1064 0.122% 92.587% ( 15, 22 ] 117 0.013% 92.601% ( 22, 34 ] 15 0.002% 92.602% ( 34, 51 ] 4 0.000% 92.603% ( 51, 76 ] 20 0.002% 92.605% ( 76, 110 ] 10046 1.155% 93.760% ( 110, 170 ] 34805 4.002% 97.762% # ( 170, 250 ] 11381 1.308% 99.070% ( 250, 380 ] 3679 0.423% 99.493% ( 380, 580 ] 2836 0.326% 99.819% ( 580, 870 ] 1052 0.121% 99.940% ( 870, 1300 ] 416 0.048% 99.988% ( 1300, 1900 ] 272 0.031% 100.019% ( 1900, 2900 ] 201 0.023% 100.042% ( 2900, 4400 ] 135 0.016% 100.058% ( 4400, 6600 ] 39 0.004% 100.062% ( 6600, 9900 ] 6 0.001% 100.063% ( 9900, 14000 ] 1 0.000% 100.063% ( 22000, 33000 ] 1 0.000% 100.063% ** Level 1 read latency histogram (micros): Count: 1764 Average: 229.6933 StdDev: 388.52 Min: 76 Median: 149.9594 Max: 6045 Percentiles: P50: 149.96 P75: 200.17 P99: 1945.00 P99.9: 5306.40 P99.99: 6045.00 ------------------------------------------------------ ( 51, 76 ] 1 0.057% 0.057% ( 76, 110 ] 225 12.755% 12.812% ### ( 110, 170 ] 985 55.839% 68.651% ########### ( 170, 250 ] 297 16.837% 85.488% ### ( 250, 380 ] 90 5.102% 90.590% # ( 380, 580 ] 91 5.159% 95.748% # ( 580, 870 ] 34 1.927% 97.676% ( 870, 1300 ] 13 0.737% 98.413% ( 1300, 1900 ] 10 0.567% 98.980% ( 1900, 2900 ] 8 0.454% 99.433% ( 2900, 4400 ] 7 0.397% 99.830% ( 4400, 6600 ] 3 0.170% 100.000% ** Level 2 read latency histogram (micros): Count: 31174 Average: 7.8387 StdDev: 85.08 Min: 1 Median: 1.8216 Max: 4431 Percentiles: P50: 1.82 P75: 2.68 P99: 11.54 P99.9: 1245.41 P99.99: 3764.78 ------------------------------------------------------ [ 0, 1 ] 811 2.602% 2.602% # ( 1, 2 ] 17985 57.692% 60.294% ############ ( 2, 3 ] 6769 21.714% 82.007% #### ( 3, 4 ] 2345 7.522% 89.530% ## ( 4, 6 ] 2134 6.845% 96.375% # ( 6, 10 ] 785 2.518% 98.893% # ( 10, 15 ] 108 0.346% 99.240% ( 15, 22 ] 19 0.061% 99.301% ( 22, 34 ] 1 0.003% 99.304% ( 34, 51 ] 1 0.003% 99.307% ( 51, 76 ] 2 0.006% 99.314% ( 76, 110 ] 1 0.003% 99.317% ( 110, 170 ] 13 0.042% 99.358% ( 170, 250 ] 24 0.077% 99.435% ( 250, 380 ] 43 0.138% 99.573% ( 380, 580 ] 39 0.125% 99.698% ( 580, 870 ] 41 0.132% 99.830% ( 870, 1300 ] 25 0.080% 99.910% ( 1300, 1900 ] 14 0.045% 99.955% ( 1900, 2900 ] 8 0.026% 99.981% ( 2900, 4400 ] 5 0.016% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 264646 Average: 10.5114 StdDev: 126.86 Min: 1 Median: 1.8770 Max: 25853 Percentiles: P50: 1.88 P75: 2.77 P99: 112.51 P99.9: 1182.84 P99.99: 1717.66 ------------------------------------------------------ [ 0, 1 ] 6611 2.498% 2.498% ( 1, 2 ] 143337 54.162% 56.660% ########### ( 2, 3 ] 62772 23.719% 80.379% ##### ( 3, 4 ] 21699 8.199% 88.578% ## ( 4, 6 ] 20473 7.736% 96.314% ## ( 6, 10 ] 5738 2.168% 98.483% ( 10, 15 ] 936 0.354% 98.836% ( 15, 22 ] 122 0.046% 98.882% ( 22, 34 ] 11 0.004% 98.886% ( 34, 51 ] 9 0.003% 98.890% ( 51, 76 ] 6 0.002% 98.892% ( 76, 110 ] 246 0.093% 98.985% ( 110, 170 ] 947 0.358% 99.343% ( 170, 250 ] 375 0.142% 99.485% ( 250, 380 ] 239 0.090% 99.575% ( 380, 580 ] 316 0.119% 99.694% ( 580, 870 ] 321 0.121% 99.816% ( 870, 1300 ] 307 0.116% 99.932% ( 1300, 1900 ] 222 0.084% 100.015% ( 1900, 2900 ] 119 0.045% 100.060% ( 2900, 4400 ] 69 0.026% 100.087% ( 4400, 6600 ] 22 0.008% 100.095% ( 6600, 9900 ] 4 0.002% 100.096% ( 9900, 14000 ] 1 0.000% 100.097% ( 22000, 33000 ] 1 0.000% 100.097% ** Level 4 read latency histogram (micros): Count: 258753 Average: 29.8860 StdDev: 194.74 Min: 1 Median: 1.9989 Max: 8445 Percentiles: P50: 2.00 P75: 3.28 P99: 524.86 P99.9: 2068.37 P99.99: 2753.31 ------------------------------------------------------ [ 0, 1 ] 6380 2.466% 2.466% ( 1, 2 ] 123138 47.589% 50.055% ########## ( 2, 3 ] 58166 22.479% 72.534% #### ( 3, 4 ] 22996 8.887% 81.421% ## ( 4, 6 ] 21451 8.290% 89.711% ## ( 6, 10 ] 6318 2.442% 92.153% ( 10, 15 ] 1138 0.440% 92.593% ( 15, 22 ] 163 0.063% 92.656% ( 22, 34 ] 35 0.014% 92.669% ( 34, 51 ] 8 0.003% 92.673% ( 51, 76 ] 8 0.003% 92.676% ( 76, 110 ] 2515 0.972% 93.648% ( 110, 170 ] 8727 3.373% 97.020% # ( 170, 250 ] 3043 1.176% 98.196% ( 250, 380 ] 1219 0.471% 98.667% ( 380, 580 ] 1188 0.459% 99.127% ( 580, 870 ] 811 0.313% 99.440% ( 870, 1300 ] 670 0.259% 99.699% ( 1300, 1900 ] 463 0.179% 99.878% ( 1900, 2900 ] 340 0.131% 100.009% ( 2900, 4400 ] 244 0.094% 100.104% ( 4400, 6600 ] 63 0.024% 100.128% ( 6600, 9900 ] 8 0.003% 100.131% ** Level 5 read latency histogram (micros): Count: 132854 Average: 260.3456 StdDev: 508.52 Min: 73 Median: 150.6530 Max: 29814 Percentiles: P50: 150.65 P75: 210.50 P99: 2891.11 P99.9: 6189.33 P99.99: 9384.12 ------------------------------------------------------ ( 51, 76 ] 7 0.005% 0.005% ( 76, 110 ] 19934 15.004% 15.010% ### ( 110, 170 ] 68609 51.642% 66.652% ########## ( 170, 250 ] 21907 16.490% 83.142% ### ( 250, 380 ] 7912 5.955% 89.097% # ( 380, 580 ] 6775 5.100% 94.197% # ( 580, 870 ] 2967 2.233% 96.430% ( 870, 1300 ] 1493 1.124% 97.554% ( 1300, 1900 ] 969 0.729% 98.283% ( 1900, 2900 ] 961 0.723% 99.006% ( 2900, 4400 ] 861 0.648% 99.655% ( 4400, 6600 ] 401 0.302% 99.956% ( 6600, 9900 ] 53 0.040% 99.996% ( 9900, 14000 ] 4 0.003% 99.999% ( 14000, 22000 ] 1 0.001% 100.000% ( 22000, 33000 ] 2 0.002% 100.002% ** DB Stats ** Uptime(secs): 13.6 total, 7.8 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 144.87 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.044 H:M:S, 44.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 126.94 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.281 H:M:S, 55.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/13-15:27:12 ... thread 0: (1000000,3000000) ops and (95668.9,125577.1) ops/second in (10.452715,23.889708) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/16 4.42 GB 200.9 4.2 0.0 4.2 6.9 2.8 0.0 2.5 161.4 267.9 26.54 23.44 55 0.483 4348K 8065 0.0 0.0 L1 9/9 1.10 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 846.15 MB 0.8 0.2 0.1 0.1 0.2 0.1 0.1 2.0 155.1 154.2 1.56 1.12 1 1.561 0 0 0.0 0.0 L4 175/0 9.74 GB 1.0 3.1 1.9 1.2 3.0 1.9 0.0 1.6 118.4 117.3 26.57 14.74 17 1.563 158K 0 0.0 0.0 L5 1116/0 91.97 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1335/25 108.05 GB 0.0 7.5 2.0 5.5 10.2 4.8 3.4 3.7 140.3 191.5 54.66 39.30 73 0.749 4507K 8065 0.0 0.0 Int 0/0 0.00 KB 0.0 7.5 2.0 5.5 10.2 4.8 3.4 3.7 140.3 191.5 54.66 39.30 73 0.749 4507K 8065 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 7.5 2.0 5.5 7.5 2.0 0.0 0.0 173.4 172.5 44.24 31.67 30 1.475 4507K 8065 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 271.9 10.43 7.64 43 0.243 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 24.0 total, 23.9 interval Flush(GB): cumulative 2.769, interval 2.769 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.22 GB write, 435.34 MB/s write, 7.49 GB read, 319.00 MB/s read, 54.7 seconds Interval compaction: 10.22 GB write, 438.06 MB/s write, 7.49 GB read, 320.98 MB/s read, 54.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 27, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 27, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 7, pending-compaction-bytes-delays: 2, pending-compaction-bytes-stops: 0, total-delays: 29, total-stops: 7, interval: 36 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1446709 Average: 20.5193 StdDev: 89.10 Min: 0 Median: 1.5295 Max: 24166 Percentiles: P50: 1.53 P75: 2.00 P99: 248.89 P99.9: 668.09 P99.99: 1028.11 ------------------------------------------------------ [ 0, 1 ] 315745 21.825% 21.825% #### ( 1, 2 ] 769864 53.215% 75.040% ########### ( 2, 3 ] 127883 8.840% 83.879% ## ( 3, 4 ] 44155 3.052% 86.932% # ( 4, 6 ] 31249 2.160% 89.092% ( 6, 10 ] 7440 0.514% 89.606% ( 10, 15 ] 1416 0.098% 89.704% ( 15, 22 ] 148 0.010% 89.714% ( 22, 34 ] 17 0.001% 89.715% ( 34, 51 ] 9 0.001% 89.716% ( 51, 76 ] 91 0.006% 89.722% ( 76, 110 ] 26747 1.849% 91.571% ( 110, 170 ] 83585 5.778% 97.348% # ( 170, 250 ] 24231 1.675% 99.023% ( 250, 380 ] 6969 0.482% 99.505% ( 380, 580 ] 5224 0.361% 99.866% ( 580, 870 ] 1614 0.112% 99.978% ( 870, 1300 ] 485 0.034% 100.011% ( 1300, 1900 ] 286 0.020% 100.031% ( 1900, 2900 ] 239 0.017% 100.048% ( 2900, 4400 ] 152 0.011% 100.058% ( 4400, 6600 ] 39 0.003% 100.061% ( 6600, 9900 ] 6 0.000% 100.061% ( 9900, 14000 ] 1 0.000% 100.061% ( 22000, 33000 ] 1 0.000% 100.061% ** Level 1 read latency histogram (micros): Count: 3818 Average: 194.5943 StdDev: 278.24 Min: 76 Median: 146.0822 Max: 6045 Percentiles: P50: 146.08 P75: 180.55 P99: 1079.47 P99.9: 4224.71 P99.99: 6045.00 ------------------------------------------------------ ( 51, 76 ] 1 0.026% 0.026% ( 76, 110 ] 591 15.479% 15.506% ### ( 110, 170 ] 2190 57.360% 72.865% ########### ( 170, 250 ] 618 16.186% 89.052% ### ( 250, 380 ] 164 4.295% 93.347% # ( 380, 580 ] 162 4.243% 97.590% # ( 580, 870 ] 47 1.231% 98.821% ( 870, 1300 ] 14 0.367% 99.188% ( 1300, 1900 ] 12 0.314% 99.502% ( 1900, 2900 ] 9 0.236% 99.738% ( 2900, 4400 ] 7 0.183% 99.921% ( 4400, 6600 ] 3 0.079% 100.000% ** Level 2 read latency histogram (micros): Count: 31174 Average: 7.8387 StdDev: 85.08 Min: 1 Median: 1.8216 Max: 4431 Percentiles: P50: 1.82 P75: 2.68 P99: 11.54 P99.9: 1245.41 P99.99: 3764.78 ------------------------------------------------------ [ 0, 1 ] 811 2.602% 2.602% # ( 1, 2 ] 17985 57.692% 60.294% ############ ( 2, 3 ] 6769 21.714% 82.007% #### ( 3, 4 ] 2345 7.522% 89.530% ## ( 4, 6 ] 2134 6.845% 96.375% # ( 6, 10 ] 785 2.518% 98.893% # ( 10, 15 ] 108 0.346% 99.240% ( 15, 22 ] 19 0.061% 99.301% ( 22, 34 ] 1 0.003% 99.304% ( 34, 51 ] 1 0.003% 99.307% ( 51, 76 ] 2 0.006% 99.314% ( 76, 110 ] 1 0.003% 99.317% ( 110, 170 ] 13 0.042% 99.358% ( 170, 250 ] 24 0.077% 99.435% ( 250, 380 ] 43 0.138% 99.573% ( 380, 580 ] 39 0.125% 99.698% ( 580, 870 ] 41 0.132% 99.830% ( 870, 1300 ] 25 0.080% 99.910% ( 1300, 1900 ] 14 0.045% 99.955% ( 1900, 2900 ] 8 0.026% 99.981% ( 2900, 4400 ] 5 0.016% 99.997% ( 4400, 6600 ] 1 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 266908 Average: 11.8141 StdDev: 127.67 Min: 1 Median: 1.8849 Max: 25853 Percentiles: P50: 1.88 P75: 2.80 P99: 159.48 P99.9: 1183.87 P99.99: 1723.25 ------------------------------------------------------ [ 0, 1 ] 6611 2.477% 2.477% ( 1, 2 ] 143343 53.705% 56.182% ########### ( 2, 3 ] 62780 23.521% 79.703% ##### ( 3, 4 ] 21707 8.133% 87.836% ## ( 4, 6 ] 20486 7.675% 95.511% ## ( 6, 10 ] 5740 2.151% 97.662% ( 10, 15 ] 936 0.351% 98.012% ( 15, 22 ] 122 0.046% 98.058% ( 22, 34 ] 11 0.004% 98.062% ( 34, 51 ] 9 0.003% 98.066% ( 51, 76 ] 6 0.002% 98.068% ( 76, 110 ] 639 0.239% 98.307% ( 110, 170 ] 2242 0.840% 99.147% ( 170, 250 ] 723 0.271% 99.418% ( 250, 380 ] 334 0.125% 99.543% ( 380, 580 ] 382 0.143% 99.686% ( 580, 870 ] 346 0.130% 99.816% ( 870, 1300 ] 307 0.115% 99.931% ( 1300, 1900 ] 223 0.084% 100.015% ( 1900, 2900 ] 120 0.045% 100.060% ( 2900, 4400 ] 70 0.026% 100.086% ( 4400, 6600 ] 22 0.008% 100.094% ( 6600, 9900 ] 4 0.001% 100.096% ( 9900, 14000 ] 1 0.000% 100.096% ( 22000, 33000 ] 1 0.000% 100.096% ** Level 4 read latency histogram (micros): Count: 280590 Average: 39.4759 StdDev: 191.78 Min: 1 Median: 2.1824 Max: 8445 Percentiles: P50: 2.18 P75: 3.96 P99: 536.95 P99.9: 2014.56 P99.99: 2748.67 ------------------------------------------------------ [ 0, 1 ] 6380 2.274% 2.274% ( 1, 2 ] 123250 43.925% 46.199% ######### ( 2, 3 ] 58456 20.833% 67.032% #### ( 3, 4 ] 23259 8.289% 75.322% ## ( 4, 6 ] 21771 7.759% 83.081% ## ( 6, 10 ] 6343 2.261% 85.341% ( 10, 15 ] 1150 0.410% 85.751% ( 15, 22 ] 165 0.059% 85.810% ( 22, 34 ] 36 0.013% 85.823% ( 34, 51 ] 8 0.003% 85.826% ( 51, 76 ] 11 0.004% 85.830% ( 76, 110 ] 6352 2.264% 88.093% ( 110, 170 ] 20990 7.481% 95.574% # ( 170, 250 ] 6237 2.223% 97.797% ( 250, 380 ] 1991 0.710% 98.506% ( 380, 580 ] 1765 0.629% 99.135% ( 580, 870 ] 956 0.341% 99.476% ( 870, 1300 ] 683 0.243% 99.720% ( 1300, 1900 ] 467 0.166% 99.886% ( 1900, 2900 ] 344 0.123% 100.009% ( 2900, 4400 ] 244 0.087% 100.096% ( 4400, 6600 ] 63 0.022% 100.118% ( 6600, 9900 ] 8 0.003% 100.121% ** Level 5 read latency histogram (micros): Count: 290249 Average: 206.6991 StdDev: 355.12 Min: 72 Median: 145.6833 Max: 29814 Percentiles: P50: 145.68 P75: 182.80 P99: 1552.46 P99.9: 5303.87 P99.99: 8155.05 ------------------------------------------------------ ( 51, 76 ] 25 0.009% 0.009% ( 76, 110 ] 49288 16.981% 16.990% ### ( 110, 170 ] 161103 55.505% 72.495% ########### ( 170, 250 ] 45453 15.660% 88.155% ### ( 250, 380 ] 13934 4.801% 92.956% # ( 380, 580 ] 11440 3.941% 96.897% # ( 580, 870 ] 4068 1.402% 98.299% ( 870, 1300 ] 1616 0.557% 98.855% ( 1300, 1900 ] 997 0.343% 99.199% ( 1900, 2900 ] 996 0.343% 99.542% ( 2900, 4400 ] 874 0.301% 99.843% ( 4400, 6600 ] 401 0.138% 99.981% ( 6600, 9900 ] 53 0.018% 100.000% ( 9900, 14000 ] 4 0.001% 100.001% ( 14000, 22000 ] 1 0.000% 100.001% ( 22000, 33000 ] 2 0.001% 100.002% ** DB Stats ** Uptime(secs): 24.0 total, 10.5 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 122.82 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:12.668 H:M:S, 52.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 94.16 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:6.625 H:M:S, 63.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 216.043 micros/op 18501 ops/sec 60.156 seconds 1112996 operations; 17.9 MB/s (294999 of 294999 found) Microseconds per read: Count: 1112996 Average: 216.0448 StdDev: 524.51 Min: 2 Median: 170.2737 Max: 65100 Percentiles: P50: 170.27 P75: 232.44 P99: 731.60 P99.9: 6273.73 P99.99: 23352.92 ------------------------------------------------------ ( 1, 2 ] 6 0.001% 0.001% ( 2, 3 ] 40 0.004% 0.004% ( 3, 4 ] 68 0.006% 0.010% ( 4, 6 ] 79 0.007% 0.017% ( 6, 10 ] 717 0.064% 0.082% ( 10, 15 ] 6154 0.553% 0.635% ( 15, 22 ] 3464 0.311% 0.946% ( 22, 34 ] 9955 0.894% 1.840% ( 34, 51 ] 2246 0.202% 2.042% ( 51, 76 ] 293 0.026% 2.068% ( 76, 110 ] 29127 2.617% 4.685% # ( 110, 170 ] 503124 45.204% 49.890% ######### ( 170, 250 ] 358070 32.172% 82.062% ###### ( 250, 380 ] 136169 12.234% 94.296% ## ( 380, 580 ] 45879 4.122% 98.418% # ( 580, 870 ] 12386 1.113% 99.531% ( 870, 1300 ] 2686 0.241% 99.772% ( 1300, 1900 ] 548 0.049% 99.822% ( 1900, 2900 ] 420 0.038% 99.859% ( 2900, 4400 ] 274 0.025% 99.884% ( 4400, 6600 ] 209 0.019% 99.903% ( 6600, 9900 ] 324 0.029% 99.932% ( 9900, 14000 ] 310 0.028% 99.960% ( 14000, 22000 ] 326 0.029% 99.989% ( 22000, 33000 ] 87 0.008% 99.997% ( 33000, 50000 ] 30 0.003% 100.000% ( 50000, 75000 ] 5 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 21210707 rocksdb.bloom.filter.full.positive COUNT : 1316822 rocksdb.bloom.filter.full.true.positive COUNT : 1112549 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 447 rocksdb.memtable.miss COUNT : 1112549 rocksdb.l0.hit COUNT : 48964 rocksdb.l1.hit COUNT : 12204 rocksdb.l2andup.hit COUNT : 1051381 rocksdb.compaction.key.drop.new COUNT : 71632 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3261622 rocksdb.number.keys.read COUNT : 1112996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3365993904 rocksdb.bytes.read COUNT : 1112996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1448 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 47024555 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 12002770170 rocksdb.compact.write.bytes COUNT : 11934254296 rocksdb.flush.write.bytes COUNT : 3303086059 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 62 rocksdb.number.superversion_acquires COUNT : 554 rocksdb.number.superversion_releases COUNT : 533 rocksdb.number.superversion_cleanups COUNT : 139 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3276273664 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1114112 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 16655539653 rocksdb.non.last.level.read.count COUNT : 3881505 rocksdb.block.checksum.compute.count COUNT : 4238098 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 169.301314 P95 : 412.329128 P99 : 730.270884 P100 : 65096.000000 COUNT : 1112996 SUM : 239544128 rocksdb.db.write.micros P50 : 1.540556 P95 : 5.212620 P99 : 206.815094 P100 : 669037.000000 COUNT : 3261622 SUM : 55442583 rocksdb.compaction.times.micros P50 : 1355555.555556 P95 : 3284878.000000 P99 : 3284878.000000 P100 : 3284878.000000 COUNT : 32 SUM : 48198458 rocksdb.compaction.times.cpu_micros P50 : 1003157.894737 P95 : 3253353.000000 P99 : 3253353.000000 P100 : 3253353.000000 COUNT : 32 SUM : 35581803 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 756.071429 P95 : 38100.000000 P99 : 141200.000000 P100 : 166253.000000 COUNT : 48 SUM : 306056 rocksdb.compaction.outfile.sync.micros P50 : 3025.000000 P95 : 111500.000000 P99 : 251300.000000 P100 : 256430.000000 COUNT : 99 SUM : 2775039 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 146.315789 P95 : 4675.000000 P99 : 6707.000000 P100 : 6707.000000 COUNT : 90 SUM : 77405 rocksdb.table.open.io.micros P50 : 7915.489749 P95 : 18616.793893 P99 : 23980.000000 P100 : 44953.000000 COUNT : 1448 SUM : 12225046 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.841155 P95 : 138.185966 P99 : 232.058768 P100 : 25862.000000 COUNT : 2908124 SUM : 61967919 rocksdb.read.block.get.micros P50 : 138.129214 P95 : 249.014331 P99 : 563.025407 P100 : 29859.000000 COUNT : 1328492 SUM : 203115743 rocksdb.write.raw.block.micros P50 : 0.556176 P95 : 1.644750 P99 : 3.595615 P100 : 32018.000000 COUNT : 3690403 SUM : 8997865 rocksdb.numfiles.in.singlecompaction P50 : 3.285714 P95 : 4.700000 P99 : 18.780000 P100 : 21.000000 COUNT : 46 SUM : 178 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.579580 P95 : 1284.270925 P99 : 2686.433814 P100 : 668873.000000 COUNT : 236813 SUM : 47046664 rocksdb.sst.read.micros P50 : 2.808907 P95 : 197.393750 P99 : 426.798618 P100 : 29814.000000 COUNT : 3881548 SUM : 249391147 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1112996 SUM : 1112996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3261622 SUM : 3365993904 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 214878.048780 P95 : 503500.000000 P99 : 926399.000000 P100 : 926399.000000 COUNT : 48 SUM : 11430277 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 122 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0