[2024-12-14 06:02:28.995806] Starting SPDK v24.09.1-pre git sha1 b18e1bd62 / DPDK 23.11.0 initialization... [2024-12-14 06:02:28.996069] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1473060 ] Set seed to 1734152548989203 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-14 06:02:29.104794] app.c: 917:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-14 06:02:29.142765] app.c: 610:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-14 06:02:29.142810] app.c: 611:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1473060' to capture a snapshot of events at runtime. [2024-12-14 06:02:29.142820] app.c: 616:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-14 06:02:29.142830] app.c: 617:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-14 06:02:29.142838] app.c: 618:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1473060 for offline analysis/debug. [2024-12-14 06:02:29.142925] reactor.c: 990:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sat Dec 14 06:02:33 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/14-06:02:40 ... thread 0: (1000000,1000000) ops and (145275.8,145275.8) ops/second in (6.883458,6.883458) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/15 2.43 GB 56.6 0.4 0.0 0.4 1.2 0.9 0.0 1.4 42.8 142.5 8.84 4.28 11 0.803 383K 542 0.0 0.0 L1 11/11 1.24 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 18/4 1.19 GB 1.0 0.6 0.3 0.3 0.6 0.3 0.0 2.1 117.9 117.1 5.55 3.07 3 1.851 175K 0 0.0 0.0 L4 175/8 9.70 GB 1.0 4.9 2.0 2.8 4.8 2.0 0.1 2.3 109.9 108.1 45.25 23.52 19 2.381 739K 0 0.0 0.0 L5 1120/0 92.21 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1342/38 106.77 GB 0.0 5.9 2.3 3.5 6.6 3.1 2.1 7.7 100.7 114.1 59.64 30.88 33 1.807 1298K 542 0.0 0.0 Int 0/0 0.00 KB 0.0 5.9 2.3 3.5 6.6 3.1 2.1 7.7 100.7 114.1 59.64 30.88 33 1.807 1298K 542 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.9 2.3 3.5 5.8 2.2 0.0 0.0 112.7 111.1 53.29 28.24 23 2.317 1298K 542 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 138.9 6.35 2.64 10 0.635 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.1 total, 6.9 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.64 GB write, 962.24 MB/s write, 5.87 GB read, 849.64 MB/s read, 59.6 seconds Interval compaction: 6.64 GB write, 987.91 MB/s write, 5.87 GB read, 872.31 MB/s read, 59.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 21, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 21, interval: 21 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 231399 Average: 23.3565 StdDev: 239.89 Min: 1 Median: 1.8447 Max: 51877 Percentiles: P50: 1.84 P75: 2.82 P99: 692.03 P99.9: 2762.66 P99.99: 4346.14 ------------------------------------------------------ [ 0, 1 ] 11161 4.823% 4.823% # ( 1, 2 ] 123761 53.484% 58.307% ########### ( 2, 3 ] 47001 20.312% 78.619% #### ( 3, 4 ] 20126 8.698% 87.316% ## ( 4, 6 ] 19323 8.351% 95.667% ## ( 6, 10 ] 4500 1.945% 97.611% ( 10, 15 ] 739 0.319% 97.931% ( 15, 22 ] 93 0.040% 97.971% ( 22, 34 ] 19 0.008% 97.979% ( 34, 51 ] 7 0.003% 97.982% ( 51, 76 ] 11 0.005% 97.987% ( 76, 110 ] 28 0.012% 97.999% ( 110, 170 ] 189 0.082% 98.081% ( 170, 250 ] 379 0.164% 98.245% ( 250, 380 ] 604 0.261% 98.506% ( 380, 580 ] 795 0.344% 98.849% ( 580, 870 ] 906 0.392% 99.241% ( 870, 1300 ] 805 0.348% 99.589% ( 1300, 1900 ] 449 0.194% 99.783% ( 1900, 2900 ] 316 0.137% 99.919% ( 2900, 4400 ] 171 0.074% 99.993% ( 4400, 6600 ] 64 0.028% 100.021% ( 6600, 9900 ] 10 0.004% 100.025% ( 9900, 14000 ] 2 0.001% 100.026% ( 14000, 22000 ] 1 0.000% 100.026% ( 22000, 33000 ] 2 0.001% 100.027% ( 50000, 75000 ] 1 0.000% 100.028% ** Level 1 read latency histogram (micros): Count: 19613 Average: 20.9434 StdDev: 211.73 Min: 1 Median: 1.9580 Max: 8918 Percentiles: P50: 1.96 P75: 3.02 P99: 541.29 P99.9: 3736.69 P99.99: 6335.64 ------------------------------------------------------ [ 0, 1 ] 459 2.340% 2.340% ( 1, 2 ] 9757 49.748% 52.088% ########## ( 2, 3 ] 4451 22.694% 74.782% ##### ( 3, 4 ] 1895 9.662% 84.444% ## ( 4, 6 ] 2124 10.830% 95.274% ## ( 6, 10 ] 540 2.753% 98.027% # ( 10, 15 ] 77 0.393% 98.419% ( 15, 22 ] 16 0.082% 98.501% ( 22, 34 ] 1 0.005% 98.506% ( 34, 51 ] 1 0.005% 98.511% ( 51, 76 ] 4 0.020% 98.532% ( 76, 110 ] 3 0.015% 98.547% ( 110, 170 ] 14 0.071% 98.618% ( 170, 250 ] 12 0.061% 98.679% ( 250, 380 ] 29 0.148% 98.827% ( 380, 580 ] 42 0.214% 99.041% ( 580, 870 ] 58 0.296% 99.337% ( 870, 1300 ] 50 0.255% 99.592% ( 1300, 1900 ] 26 0.133% 99.725% ( 1900, 2900 ] 21 0.107% 99.832% ( 2900, 4400 ] 24 0.122% 99.954% ( 4400, 6600 ] 8 0.041% 99.995% ( 6600, 9900 ] 1 0.005% 100.000% ** Level 2 read latency histogram (micros): Count: 32738 Average: 8.5228 StdDev: 112.97 Min: 1 Median: 1.8784 Max: 7552 Percentiles: P50: 1.88 P75: 2.83 P99: 9.67 P99.9: 1542.07 P99.99: 4226.30 ------------------------------------------------------ [ 0, 1 ] 770 2.352% 2.352% ( 1, 2 ] 17758 54.243% 56.595% ########### ( 2, 3 ] 7227 22.075% 78.670% #### ( 3, 4 ] 2959 9.038% 87.708% ## ( 4, 6 ] 2939 8.977% 96.686% ## ( 6, 10 ] 825 2.520% 99.206% # ( 10, 15 ] 98 0.299% 99.505% ( 15, 22 ] 11 0.034% 99.539% ( 22, 34 ] 1 0.003% 99.542% ( 34, 51 ] 1 0.003% 99.545% ( 51, 76 ] 1 0.003% 99.548% ( 110, 170 ] 2 0.006% 99.554% ( 170, 250 ] 7 0.021% 99.575% ( 250, 380 ] 7 0.021% 99.597% ( 380, 580 ] 18 0.055% 99.652% ( 580, 870 ] 42 0.128% 99.780% ( 870, 1300 ] 32 0.098% 99.878% ( 1300, 1900 ] 18 0.055% 99.933% ( 1900, 2900 ] 9 0.027% 99.960% ( 2900, 4400 ] 11 0.034% 99.994% ( 4400, 6600 ] 2 0.006% 100.000% ( 6600, 9900 ] 1 0.003% 100.003% ** Level 3 read latency histogram (micros): Count: 410270 Average: 9.2369 StdDev: 162.78 Min: 1 Median: 1.9421 Max: 35574 Percentiles: P50: 1.94 P75: 2.94 P99: 9.74 P99.9: 867.22 P99.99: 1278.75 ------------------------------------------------------ [ 0, 1 ] 10099 2.462% 2.462% ( 1, 2 ] 207018 50.459% 52.921% ########## ( 2, 3 ] 96198 23.447% 76.368% ##### ( 3, 4 ] 40825 9.951% 86.319% ## ( 4, 6 ] 41477 10.110% 96.428% ## ( 6, 10 ] 11289 2.752% 99.180% # ( 10, 15 ] 1795 0.438% 99.618% ( 15, 22 ] 269 0.066% 99.683% ( 22, 34 ] 38 0.009% 99.692% ( 34, 51 ] 18 0.004% 99.697% ( 51, 76 ] 25 0.006% 99.703% ( 76, 110 ] 12 0.003% 99.706% ( 110, 170 ] 27 0.007% 99.712% ( 170, 250 ] 66 0.016% 99.728% ( 250, 380 ] 109 0.027% 99.755% ( 380, 580 ] 257 0.063% 99.818% ( 580, 870 ] 341 0.083% 99.901% ( 870, 1300 ] 385 0.094% 99.995% ( 1300, 1900 ] 257 0.063% 100.057% ( 1900, 2900 ] 152 0.037% 100.094% ( 2900, 4400 ] 157 0.038% 100.133% ( 4400, 6600 ] 35 0.009% 100.141% ( 6600, 9900 ] 12 0.003% 100.144% ( 9900, 14000 ] 1 0.000% 100.144% ( 14000, 22000 ] 1 0.000% 100.145% ( 22000, 33000 ] 5 0.001% 100.146% ( 33000, 50000 ] 1 0.000% 100.146% ** Level 4 read latency histogram (micros): Count: 677580 Average: 18.8397 StdDev: 267.44 Min: 1 Median: 1.9455 Max: 64083 Percentiles: P50: 1.95 P75: 2.95 P99: 14.79 P99.9: 1321.20 P99.99: 1666.71 ------------------------------------------------------ [ 0, 1 ] 17133 2.529% 2.529% # ( 1, 2 ] 340190 50.207% 52.735% ########## ( 2, 3 ] 158153 23.341% 76.076% ##### ( 3, 4 ] 68956 10.177% 86.253% ## ( 4, 6 ] 65541 9.673% 95.926% ## ( 6, 10 ] 18013 2.658% 98.584% # ( 10, 15 ] 2939 0.434% 99.018% ( 15, 22 ] 437 0.064% 99.082% ( 22, 34 ] 61 0.009% 99.091% ( 34, 51 ] 33 0.005% 99.096% ( 51, 76 ] 48 0.007% 99.103% ( 76, 110 ] 41 0.006% 99.109% ( 110, 170 ] 211 0.031% 99.140% ( 170, 250 ] 437 0.064% 99.205% ( 250, 380 ] 647 0.095% 99.300% ( 380, 580 ] 1018 0.150% 99.451% ( 580, 870 ] 1534 0.226% 99.677% ( 870, 1300 ] 1473 0.217% 99.894% ( 1300, 1900 ] 1059 0.156% 100.051% ( 1900, 2900 ] 692 0.102% 100.153% ( 2900, 4400 ] 567 0.084% 100.237% ( 4400, 6600 ] 202 0.030% 100.266% ( 6600, 9900 ] 37 0.005% 100.272% ( 9900, 14000 ] 5 0.001% 100.273% ( 14000, 22000 ] 10 0.001% 100.274% ( 22000, 33000 ] 5 0.001% 100.275% ( 33000, 50000 ] 3 0.000% 100.275% ( 50000, 75000 ] 4 0.001% 100.276% ** Level 5 read latency histogram (micros): Count: 13172 Average: 1437.9976 StdDev: 1709.08 Min: 82 Median: 886.7766 Max: 52949 Percentiles: P50: 886.78 P75: 1831.40 P99: 6547.23 P99.9: 11699.53 P99.99: 48202.53 ------------------------------------------------------ ( 76, 110 ] 41 0.311% 0.311% ( 110, 170 ] 399 3.029% 3.340% # ( 170, 250 ] 751 5.701% 9.042% # ( 250, 380 ] 1174 8.913% 17.955% ## ( 380, 580 ] 1843 13.992% 31.947% ### ( 580, 870 ] 2294 17.416% 49.362% ### ( 870, 1300 ] 2153 16.345% 65.708% ### ( 1300, 1900 ] 1382 10.492% 76.200% ## ( 1900, 2900 ] 1228 9.323% 85.522% ## ( 2900, 4400 ] 1217 9.239% 94.762% ## ( 4400, 6600 ] 572 4.343% 99.104% # ( 6600, 9900 ] 100 0.759% 99.863% ( 9900, 14000 ] 11 0.084% 99.947% ( 14000, 22000 ] 2 0.015% 99.962% ( 22000, 33000 ] 1 0.008% 99.970% ( 33000, 50000 ] 3 0.023% 99.992% ( 50000, 75000 ] 2 0.015% 100.008% ** DB Stats ** Uptime(secs): 7.1 total, 6.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 139.23 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.037 H:M:S, 43.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 142.92 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.037 H:M:S, 44.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 6 num-running-flushes: 1 2024/12/14-06:02:48 ... thread 0: (1000000,2000000) ops and (119940.2,131397.9) ops/second in (8.337489,15.220947) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/15 3.41 GB 163.3 2.3 0.0 2.3 4.2 1.8 0.0 2.3 122.3 218.7 19.57 14.37 32 0.611 2430K 5026 0.0 0.0 L1 11/11 1.24 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 14/0 975.25 MB 1.0 0.6 0.3 0.3 0.6 0.3 0.0 2.1 117.9 117.1 5.55 3.07 3 1.851 175K 0 0.0 0.0 L4 175/0 9.61 GB 1.0 5.4 2.3 3.1 5.3 2.2 0.1 2.3 114.5 112.6 47.99 25.87 23 2.086 982K 0 0.0 0.0 L5 1124/0 92.52 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1342/26 107.74 GB 0.0 8.3 2.6 5.8 10.1 4.3 2.5 5.5 116.9 141.3 73.10 43.31 58 1.260 3588K 5026 0.0 0.0 Int 0/0 0.00 KB 0.0 8.3 2.6 5.8 10.1 4.3 2.5 5.5 116.9 141.3 73.10 43.31 58 1.260 3588K 5026 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 8.3 2.6 5.8 8.2 2.5 0.0 0.0 134.4 132.8 63.58 37.98 32 1.987 3588K 5026 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 198.4 9.53 5.33 26 0.366 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.4 total, 15.2 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.09 GB write, 670.69 MB/s write, 8.34 GB read, 554.63 MB/s read, 73.1 seconds Interval compaction: 10.09 GB write, 678.79 MB/s write, 8.34 GB read, 561.32 MB/s read, 73.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 16, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 16, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 21, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 16, total-stops: 21, interval: 37 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 768457 Average: 22.6101 StdDev: 147.23 Min: 0 Median: 1.5518 Max: 51877 Percentiles: P50: 1.55 P75: 2.23 P99: 338.86 P99.9: 1340.49 P99.99: 2483.51 ------------------------------------------------------ [ 0, 1 ] 169226 22.022% 22.022% #### ( 1, 2 ] 389669 50.708% 72.730% ########## ( 2, 3 ] 77014 10.022% 82.751% ## ( 3, 4 ] 31372 4.082% 86.834% # ( 4, 6 ] 26503 3.449% 90.283% # ( 6, 10 ] 5809 0.756% 91.039% ( 10, 15 ] 1247 0.162% 91.201% ( 15, 22 ] 165 0.021% 91.222% ( 22, 34 ] 21 0.003% 91.225% ( 34, 51 ] 9 0.001% 91.226% ( 51, 76 ] 20 0.003% 91.229% ( 76, 110 ] 11115 1.446% 92.675% ( 110, 170 ] 35432 4.611% 97.286% # ( 170, 250 ] 10639 1.384% 98.671% ( 250, 380 ] 3705 0.482% 99.153% ( 380, 580 ] 3147 0.410% 99.562% ( 580, 870 ] 1625 0.211% 99.774% ( 870, 1300 ] 938 0.122% 99.896% ( 1300, 1900 ] 497 0.065% 99.960% ( 1900, 2900 ] 391 0.051% 100.011% ( 2900, 4400 ] 213 0.028% 100.039% ( 4400, 6600 ] 65 0.008% 100.047% ( 6600, 9900 ] 10 0.001% 100.049% ( 9900, 14000 ] 2 0.000% 100.049% ( 14000, 22000 ] 1 0.000% 100.049% ( 22000, 33000 ] 2 0.000% 100.049% ( 50000, 75000 ] 1 0.000% 100.050% ** Level 1 read latency histogram (micros): Count: 21779 Average: 36.0573 StdDev: 210.68 Min: 1 Median: 2.1513 Max: 8918 Percentiles: P50: 2.15 P75: 3.88 P99: 565.71 P99.9: 3633.26 P99.99: 6276.08 ------------------------------------------------------ [ 0, 1 ] 459 2.108% 2.108% ( 1, 2 ] 9757 44.800% 46.908% ######### ( 2, 3 ] 4451 20.437% 67.345% #### ( 3, 4 ] 1895 8.701% 76.046% ## ( 4, 6 ] 2124 9.753% 85.798% ## ( 6, 10 ] 540 2.479% 88.278% ( 10, 15 ] 77 0.354% 88.631% ( 15, 22 ] 16 0.073% 88.705% ( 22, 34 ] 1 0.005% 88.709% ( 34, 51 ] 1 0.005% 88.714% ( 51, 76 ] 4 0.018% 88.732% ( 76, 110 ] 345 1.584% 90.316% ( 110, 170 ] 1260 5.785% 96.102% # ( 170, 250 ] 360 1.653% 97.755% ( 250, 380 ] 157 0.721% 98.476% ( 380, 580 ] 123 0.565% 99.040% ( 580, 870 ] 74 0.340% 99.380% ( 870, 1300 ] 51 0.234% 99.614% ( 1300, 1900 ] 26 0.119% 99.734% ( 1900, 2900 ] 24 0.110% 99.844% ( 2900, 4400 ] 25 0.115% 99.959% ( 4400, 6600 ] 8 0.037% 99.995% ( 6600, 9900 ] 1 0.005% 100.000% ** Level 2 read latency histogram (micros): Count: 32820 Average: 8.9262 StdDev: 113.21 Min: 1 Median: 1.8807 Max: 7552 Percentiles: P50: 1.88 P75: 2.84 P99: 10.70 P99.9: 1539.33 P99.99: 4225.18 ------------------------------------------------------ [ 0, 1 ] 770 2.346% 2.346% ( 1, 2 ] 17758 54.107% 56.453% ########### ( 2, 3 ] 7227 22.020% 78.473% #### ( 3, 4 ] 2959 9.016% 87.489% ## ( 4, 6 ] 2939 8.955% 96.444% ## ( 6, 10 ] 825 2.514% 98.958% # ( 10, 15 ] 98 0.299% 99.257% ( 15, 22 ] 11 0.034% 99.290% ( 22, 34 ] 1 0.003% 99.293% ( 34, 51 ] 1 0.003% 99.296% ( 51, 76 ] 1 0.003% 99.299% ( 76, 110 ] 13 0.040% 99.339% ( 110, 170 ] 46 0.140% 99.479% ( 170, 250 ] 24 0.073% 99.552% ( 250, 380 ] 11 0.034% 99.586% ( 380, 580 ] 21 0.064% 99.650% ( 580, 870 ] 43 0.131% 99.781% ( 870, 1300 ] 32 0.098% 99.878% ( 1300, 1900 ] 18 0.055% 99.933% ( 1900, 2900 ] 9 0.027% 99.960% ( 2900, 4400 ] 11 0.034% 99.994% ( 4400, 6600 ] 2 0.006% 100.000% ( 6600, 9900 ] 1 0.003% 100.003% ** Level 3 read latency histogram (micros): Count: 453160 Average: 9.2779 StdDev: 155.63 Min: 1 Median: 1.9355 Max: 35574 Percentiles: P50: 1.94 P75: 2.94 P99: 11.31 P99.9: 779.19 P99.99: 1195.44 ------------------------------------------------------ [ 0, 1 ] 11052 2.439% 2.439% ( 1, 2 ] 230383 50.839% 53.278% ########## ( 2, 3 ] 105238 23.223% 76.501% ##### ( 3, 4 ] 44751 9.875% 86.377% ## ( 4, 6 ] 45042 9.940% 96.316% ## ( 6, 10 ] 11685 2.579% 98.895% # ( 10, 15 ] 1825 0.403% 99.297% ( 15, 22 ] 273 0.060% 99.358% ( 22, 34 ] 40 0.009% 99.366% ( 34, 51 ] 22 0.005% 99.371% ( 51, 76 ] 27 0.006% 99.377% ( 76, 110 ] 287 0.063% 99.441% ( 110, 170 ] 985 0.217% 99.658% ( 170, 250 ] 343 0.076% 99.734% ( 250, 380 ] 181 0.040% 99.774% ( 380, 580 ] 329 0.073% 99.846% ( 580, 870 ] 355 0.078% 99.925% ( 870, 1300 ] 392 0.087% 100.011% ( 1300, 1900 ] 258 0.057% 100.068% ( 1900, 2900 ] 156 0.034% 100.102% ( 2900, 4400 ] 158 0.035% 100.137% ( 4400, 6600 ] 35 0.008% 100.145% ( 6600, 9900 ] 12 0.003% 100.148% ( 9900, 14000 ] 1 0.000% 100.148% ( 14000, 22000 ] 1 0.000% 100.148% ( 22000, 33000 ] 5 0.001% 100.149% ( 33000, 50000 ] 1 0.000% 100.149% ** Level 4 read latency histogram (micros): Count: 743353 Average: 20.7802 StdDev: 257.34 Min: 1 Median: 1.9610 Max: 64083 Percentiles: P50: 1.96 P75: 3.03 P99: 230.44 P99.9: 1268.67 P99.99: 1614.42 ------------------------------------------------------ [ 0, 1 ] 18269 2.458% 2.458% ( 1, 2 ] 367737 49.470% 51.928% ########## ( 2, 3 ] 169309 22.776% 74.704% ##### ( 3, 4 ] 74158 9.976% 84.680% ## ( 4, 6 ] 70459 9.479% 94.159% ## ( 6, 10 ] 19268 2.592% 96.751% # ( 10, 15 ] 3049 0.410% 97.161% ( 15, 22 ] 448 0.060% 97.221% ( 22, 34 ] 62 0.008% 97.230% ( 34, 51 ] 35 0.005% 97.234% ( 51, 76 ] 49 0.007% 97.241% ( 76, 110 ] 2498 0.336% 97.577% ( 110, 170 ] 8371 1.126% 98.703% ( 170, 250 ] 2922 0.393% 99.096% ( 250, 380 ] 1339 0.180% 99.276% ( 380, 580 ] 1545 0.208% 99.484% ( 580, 870 ] 1700 0.229% 99.713% ( 870, 1300 ] 1501 0.202% 99.915% ( 1300, 1900 ] 1068 0.144% 100.058% ( 1900, 2900 ] 701 0.094% 100.153% ( 2900, 4400 ] 573 0.077% 100.230% ( 4400, 6600 ] 203 0.027% 100.257% ( 6600, 9900 ] 37 0.005% 100.262% ( 9900, 14000 ] 6 0.001% 100.263% ( 14000, 22000 ] 10 0.001% 100.264% ( 22000, 33000 ] 5 0.001% 100.265% ( 33000, 50000 ] 3 0.000% 100.265% ( 50000, 75000 ] 4 0.001% 100.266% ** Level 5 read latency histogram (micros): Count: 130668 Average: 301.7691 StdDev: 677.00 Min: 74 Median: 150.4257 Max: 52949 Percentiles: P50: 150.43 P75: 214.41 P99: 3660.54 P99.9: 6536.12 P99.99: 10247.83 ------------------------------------------------------ ( 51, 76 ] 9 0.007% 0.007% ( 76, 110 ] 20381 15.598% 15.604% ### ( 110, 170 ] 66706 51.050% 66.654% ########## ( 170, 250 ] 19643 15.033% 81.687% ### ( 250, 380 ] 6856 5.247% 86.934% # ( 380, 580 ] 6297 4.819% 91.753% # ( 580, 870 ] 3618 2.769% 94.522% # ( 870, 1300 ] 2399 1.836% 96.358% ( 1300, 1900 ] 1469 1.124% 97.482% ( 1900, 2900 ] 1347 1.031% 98.513% ( 2900, 4400 ] 1255 0.960% 99.473% ( 4400, 6600 ] 574 0.439% 99.913% ( 6600, 9900 ] 100 0.077% 99.989% ( 9900, 14000 ] 11 0.008% 99.998% ( 14000, 22000 ] 2 0.002% 99.999% ( 22000, 33000 ] 1 0.001% 100.000% ( 33000, 50000 ] 3 0.002% 100.002% ( 50000, 75000 ] 2 0.002% 100.004% ** DB Stats ** Uptime(secs): 15.4 total, 8.3 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 127.76 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.733 H:M:S, 50.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 118.05 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.696 H:M:S, 56.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/14-06:02:57 ... thread 0: (1000000,3000000) ops and (116296.4,125946.4) ops/second in (8.598716,23.819663) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/11 4.40 GB 288.9 4.1 0.0 4.1 6.9 2.8 0.0 2.5 143.2 241.3 29.46 23.70 53 0.556 4284K 8458 0.0 0.0 L1 11/11 1.24 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 14/0 975.25 MB 1.0 0.6 0.3 0.3 0.6 0.3 0.0 2.1 117.9 117.1 5.55 3.07 3 1.851 175K 0 0.0 0.0 L4 175/0 9.61 GB 1.0 5.4 2.3 3.1 5.3 2.2 0.1 2.3 114.5 112.6 47.99 25.87 23 2.086 982K 0 0.0 0.0 L5 1124/0 92.52 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1343/22 108.72 GB 0.0 10.1 2.6 7.6 12.9 5.3 2.5 4.5 124.9 158.6 83.00 52.64 79 1.051 5442K 8458 0.0 0.0 Int 0/0 0.00 KB 0.0 10.1 2.6 7.6 12.9 5.3 2.5 4.5 124.9 158.6 83.00 52.64 79 1.051 5442K 8458 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.1 2.6 7.6 10.0 2.5 0.0 0.0 147.5 146.0 70.33 44.59 37 1.901 5442K 8458 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 228.6 12.68 8.05 42 0.302 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 24.0 total, 23.8 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.86 GB write, 548.37 MB/s write, 10.13 GB read, 432.04 MB/s read, 83.0 seconds Interval compaction: 12.86 GB write, 552.60 MB/s write, 10.13 GB read, 435.37 MB/s read, 83.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 28, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 28, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 21, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 28, total-stops: 21, interval: 49 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1308413 Average: 22.6209 StdDev: 121.45 Min: 0 Median: 1.4916 Max: 51877 Percentiles: P50: 1.49 P75: 1.99 P99: 304.38 P99.9: 976.80 P99.99: 1774.90 ------------------------------------------------------ [ 0, 1 ] 332535 25.415% 25.415% ##### ( 1, 2 ] 654372 50.013% 75.428% ########## ( 2, 3 ] 104016 7.950% 83.378% ## ( 3, 4 ] 41456 3.168% 86.546% # ( 4, 6 ] 32217 2.462% 89.008% ( 6, 10 ] 6912 0.528% 89.537% ( 10, 15 ] 1511 0.115% 89.652% ( 15, 22 ] 203 0.016% 89.668% ( 22, 34 ] 25 0.002% 89.669% ( 34, 51 ] 11 0.001% 89.670% ( 51, 76 ] 34 0.003% 89.673% ( 76, 110 ] 23480 1.795% 91.467% ( 110, 170 ] 74127 5.665% 97.133% # ( 170, 250 ] 21602 1.651% 98.784% ( 250, 380 ] 6760 0.517% 99.301% ( 380, 580 ] 5418 0.414% 99.715% ( 580, 870 ] 2173 0.166% 99.881% ( 870, 1300 ] 1017 0.078% 99.958% ( 1300, 1900 ] 522 0.040% 99.998% ( 1900, 2900 ] 421 0.032% 100.030% ( 2900, 4400 ] 234 0.018% 100.048% ( 4400, 6600 ] 65 0.005% 100.053% ( 6600, 9900 ] 10 0.001% 100.054% ( 9900, 14000 ] 2 0.000% 100.054% ( 14000, 22000 ] 1 0.000% 100.054% ( 22000, 33000 ] 2 0.000% 100.054% ( 50000, 75000 ] 1 0.000% 100.055% ** Level 1 read latency histogram (micros): Count: 24007 Average: 48.2758 StdDev: 207.35 Min: 1 Median: 2.4016 Max: 8918 Percentiles: P50: 2.40 P75: 5.36 P99: 574.83 P99.9: 3499.58 P99.99: 6214.81 ------------------------------------------------------ [ 0, 1 ] 459 1.912% 1.912% ( 1, 2 ] 9757 40.642% 42.554% ######## ( 2, 3 ] 4451 18.540% 61.095% #### ( 3, 4 ] 1895 7.894% 68.988% ## ( 4, 6 ] 2124 8.847% 77.836% ## ( 6, 10 ] 540 2.249% 80.085% ( 10, 15 ] 77 0.321% 80.406% ( 15, 22 ] 16 0.067% 80.472% ( 22, 34 ] 1 0.004% 80.477% ( 34, 51 ] 1 0.004% 80.481% ( 51, 76 ] 4 0.017% 80.497% ( 76, 110 ] 741 3.087% 83.584% # ( 110, 170 ] 2555 10.643% 94.227% ## ( 170, 250 ] 688 2.866% 97.093% # ( 250, 380 ] 267 1.112% 98.205% ( 380, 580 ] 196 0.816% 99.021% ( 580, 870 ] 97 0.404% 99.425% ( 870, 1300 ] 51 0.212% 99.638% ( 1300, 1900 ] 27 0.112% 99.750% ( 1900, 2900 ] 26 0.108% 99.858% ( 2900, 4400 ] 25 0.104% 99.963% ( 4400, 6600 ] 8 0.033% 99.996% ( 6600, 9900 ] 1 0.004% 100.000% ** Level 2 read latency histogram (micros): Count: 32924 Average: 9.4766 StdDev: 113.66 Min: 1 Median: 1.8837 Max: 7552 Percentiles: P50: 1.88 P75: 2.85 P99: 81.06 P99.9: 1535.87 P99.99: 4223.76 ------------------------------------------------------ [ 0, 1 ] 770 2.339% 2.339% ( 1, 2 ] 17758 53.936% 56.275% ########### ( 2, 3 ] 7227 21.951% 78.226% #### ( 3, 4 ] 2959 8.987% 87.213% ## ( 4, 6 ] 2939 8.927% 96.140% ## ( 6, 10 ] 825 2.506% 98.645% # ( 10, 15 ] 98 0.298% 98.943% ( 15, 22 ] 11 0.033% 98.976% ( 22, 34 ] 1 0.003% 98.979% ( 34, 51 ] 1 0.003% 98.983% ( 51, 76 ] 1 0.003% 98.986% ( 76, 110 ] 32 0.097% 99.083% ( 110, 170 ] 101 0.307% 99.390% ( 170, 250 ] 39 0.118% 99.508% ( 250, 380 ] 19 0.058% 99.566% ( 380, 580 ] 27 0.082% 99.648% ( 580, 870 ] 44 0.134% 99.781% ( 870, 1300 ] 32 0.097% 99.879% ( 1300, 1900 ] 18 0.055% 99.933% ( 1900, 2900 ] 9 0.027% 99.961% ( 2900, 4400 ] 11 0.033% 99.994% ( 4400, 6600 ] 2 0.006% 100.000% ( 6600, 9900 ] 1 0.003% 100.003% ** Level 3 read latency histogram (micros): Count: 454833 Average: 9.8744 StdDev: 155.79 Min: 1 Median: 1.9392 Max: 35574 Percentiles: P50: 1.94 P75: 2.95 P99: 32.70 P99.9: 785.61 P99.99: 1198.81 ------------------------------------------------------ [ 0, 1 ] 11052 2.430% 2.430% ( 1, 2 ] 230383 50.652% 53.082% ########## ( 2, 3 ] 105238 23.138% 76.220% ##### ( 3, 4 ] 44751 9.839% 86.059% ## ( 4, 6 ] 45042 9.903% 95.962% ## ( 6, 10 ] 11685 2.569% 98.531% # ( 10, 15 ] 1825 0.401% 98.932% ( 15, 22 ] 273 0.060% 98.992% ( 22, 34 ] 40 0.009% 99.001% ( 34, 51 ] 22 0.005% 99.006% ( 51, 76 ] 27 0.006% 99.012% ( 76, 110 ] 582 0.128% 99.140% ( 110, 170 ] 1905 0.419% 99.559% ( 170, 250 ] 634 0.139% 99.698% ( 250, 380 ] 262 0.058% 99.756% ( 380, 580 ] 392 0.086% 99.842% ( 580, 870 ] 374 0.082% 99.924% ( 870, 1300 ] 393 0.086% 100.010% ( 1300, 1900 ] 261 0.057% 100.068% ( 1900, 2900 ] 156 0.034% 100.102% ( 2900, 4400 ] 158 0.035% 100.137% ( 4400, 6600 ] 35 0.008% 100.144% ( 6600, 9900 ] 12 0.003% 100.147% ( 9900, 14000 ] 1 0.000% 100.147% ( 14000, 22000 ] 1 0.000% 100.148% ( 22000, 33000 ] 5 0.001% 100.149% ( 33000, 50000 ] 1 0.000% 100.149% ** Level 4 read latency histogram (micros): Count: 761647 Average: 23.9443 StdDev: 255.59 Min: 1 Median: 1.9856 Max: 64083 Percentiles: P50: 1.99 P75: 3.21 P99: 286.89 P99.9: 1266.00 P99.99: 1616.70 ------------------------------------------------------ [ 0, 1 ] 18269 2.399% 2.399% ( 1, 2 ] 367843 48.296% 50.694% ########## ( 2, 3 ] 169792 22.293% 72.987% #### ( 3, 4 ] 74532 9.786% 82.773% ## ( 4, 6 ] 70981 9.319% 92.092% ## ( 6, 10 ] 19334 2.538% 94.631% # ( 10, 15 ] 3065 0.402% 95.033% ( 15, 22 ] 450 0.059% 95.092% ( 22, 34 ] 63 0.008% 95.100% ( 34, 51 ] 35 0.005% 95.105% ( 51, 76 ] 53 0.007% 95.112% ( 76, 110 ] 5317 0.698% 95.810% ( 110, 170 ] 18048 2.370% 98.180% ( 170, 250 ] 5654 0.742% 98.922% ( 250, 380 ] 2095 0.275% 99.197% ( 380, 580 ] 2132 0.280% 99.477% ( 580, 870 ] 1829 0.240% 99.717% ( 870, 1300 ] 1513 0.199% 99.916% ( 1300, 1900 ] 1072 0.141% 100.056% ( 1900, 2900 ] 704 0.092% 100.149% ( 2900, 4400 ] 575 0.075% 100.224% ( 4400, 6600 ] 203 0.027% 100.251% ( 6600, 9900 ] 37 0.005% 100.256% ( 9900, 14000 ] 6 0.001% 100.257% ( 14000, 22000 ] 10 0.001% 100.258% ( 22000, 33000 ] 5 0.001% 100.259% ( 33000, 50000 ] 3 0.000% 100.259% ( 50000, 75000 ] 4 0.001% 100.260% ** Level 5 read latency histogram (micros): Count: 254913 Average: 235.9727 StdDev: 494.94 Min: 74 Median: 147.0275 Max: 52949 Percentiles: P50: 147.03 P75: 192.15 P99: 2464.87 P99.9: 6052.25 P99.99: 9454.79 ------------------------------------------------------ ( 51, 76 ] 13 0.005% 0.005% ( 76, 110 ] 42382 16.626% 16.631% ### ( 110, 170 ] 137835 54.071% 70.703% ########### ( 170, 250 ] 39574 15.525% 86.227% ### ( 250, 380 ] 12464 4.890% 91.117% # ( 380, 580 ] 10603 4.159% 95.276% # ( 580, 870 ] 4686 1.838% 97.114% ( 870, 1300 ] 2513 0.986% 98.100% ( 1300, 1900 ] 1507 0.591% 98.691% ( 1900, 2900 ] 1393 0.546% 99.238% ( 2900, 4400 ] 1257 0.493% 99.731% ( 4400, 6600 ] 574 0.225% 99.956% ( 6600, 9900 ] 100 0.039% 99.995% ( 9900, 14000 ] 11 0.004% 100.000% ( 14000, 22000 ] 2 0.001% 100.000% ( 22000, 33000 ] 1 0.000% 100.001% ( 33000, 50000 ] 3 0.001% 100.002% ( 50000, 75000 ] 2 0.001% 100.003% ** DB Stats ** Uptime(secs): 24.0 total, 8.6 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 123.00 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:12.743 H:M:S, 53.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 114.46 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.011 H:M:S, 58.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 224.117 micros/op 17819 ops/sec 60.157 seconds 1071996 operations; 17.3 MB/s (277999 of 277999 found) Microseconds per read: Count: 1071996 Average: 224.1182 StdDev: 660.56 Min: 1 Median: 173.2363 Max: 153478 Percentiles: P50: 173.24 P75: 232.83 P99: 780.49 P99.9: 9821.51 P99.99: 28886.98 ------------------------------------------------------ [ 0, 1 ] 1 0.000% 0.000% ( 1, 2 ] 6 0.001% 0.001% ( 2, 3 ] 47 0.004% 0.005% ( 3, 4 ] 61 0.006% 0.011% ( 4, 6 ] 86 0.008% 0.019% ( 6, 10 ] 447 0.042% 0.060% ( 10, 15 ] 5027 0.469% 0.529% ( 15, 22 ] 4081 0.381% 0.910% ( 22, 34 ] 11337 1.058% 1.968% ( 34, 51 ] 3497 0.326% 2.294% ( 51, 76 ] 376 0.035% 2.329% ( 76, 110 ] 22425 2.092% 4.421% ( 110, 170 ] 474054 44.222% 48.642% ######### ( 170, 250 ] 359743 33.558% 82.201% ####### ( 250, 380 ] 130380 12.162% 94.363% ## ( 380, 580 ] 41851 3.904% 98.267% # ( 580, 870 ] 11365 1.060% 99.327% ( 870, 1300 ] 3228 0.301% 99.628% ( 1300, 1900 ] 1119 0.104% 99.733% ( 1900, 2900 ] 896 0.084% 99.816% ( 2900, 4400 ] 542 0.051% 99.867% ( 4400, 6600 ] 191 0.018% 99.885% ( 6600, 9900 ] 168 0.016% 99.900% ( 9900, 14000 ] 429 0.040% 99.940% ( 14000, 22000 ] 431 0.040% 99.981% ( 22000, 33000 ] 161 0.015% 99.996% ( 33000, 50000 ] 34 0.003% 99.999% ( 50000, 75000 ] 9 0.001% 100.000% ( 75000, 110000 ] 2 0.000% 100.000% ( 110000, 170000 ] 2 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 21441609 rocksdb.bloom.filter.full.positive COUNT : 1281008 rocksdb.bloom.filter.full.true.positive COUNT : 1071601 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 395 rocksdb.memtable.miss COUNT : 1071601 rocksdb.l0.hit COUNT : 46835 rocksdb.l1.hit COUNT : 13243 rocksdb.l2andup.hit COUNT : 1011523 rocksdb.compaction.key.drop.new COUNT : 131078 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3241730 rocksdb.number.keys.read COUNT : 1071996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3345465360 rocksdb.bytes.read COUNT : 1071996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1511 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 47316547 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 13513199368 rocksdb.compact.write.bytes COUNT : 13381046648 rocksdb.flush.write.bytes COUNT : 3303036254 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 59 rocksdb.number.superversion_acquires COUNT : 581 rocksdb.number.superversion_releases COUNT : 555 rocksdb.number.superversion_cleanups COUNT : 143 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3276270592 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1148928 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 17978390632 rocksdb.non.last.level.read.count COUNT : 4200457 rocksdb.block.checksum.compute.count COUNT : 4567768 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 171.652134 P95 : 410.538153 P99 : 779.612085 P100 : 153476.000000 COUNT : 1071996 SUM : 239340384 rocksdb.db.write.micros P50 : 1.550706 P95 : 5.136330 P99 : 90.795817 P100 : 875038.000000 COUNT : 3241730 SUM : 55725660 rocksdb.compaction.times.micros P50 : 1666666.666667 P95 : 3579237.000000 P99 : 3579237.000000 P100 : 3579237.000000 COUNT : 38 SUM : 71268760 rocksdb.compaction.times.cpu_micros P50 : 1132000.000000 P95 : 2970000.000000 P99 : 3421690.000000 P100 : 3421690.000000 COUNT : 38 SUM : 45520760 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 809.583333 P95 : 87250.000000 P99 : 199231.000000 P100 : 199231.000000 COUNT : 46 SUM : 506918 rocksdb.compaction.outfile.sync.micros P50 : 33000.000000 P95 : 228000.000000 P99 : 360066.666667 P100 : 549470.000000 COUNT : 146 SUM : 9704355 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 166.129032 P95 : 4565.000000 P99 : 6798.000000 P100 : 8380.000000 COUNT : 94 SUM : 81966 rocksdb.table.open.io.micros P50 : 9220.186335 P95 : 20218.502203 P99 : 26945.000000 P100 : 84042.000000 COUNT : 1511 SUM : 14658813 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.920854 P95 : 135.102945 P99 : 230.757580 P100 : 64090.000000 COUNT : 3273500 SUM : 74019433 rocksdb.read.block.get.micros P50 : 137.823968 P95 : 247.281415 P99 : 638.679991 P100 : 52954.000000 COUNT : 1292753 SUM : 200830922 rocksdb.write.raw.block.micros P50 : 0.535968 P95 : 1.372057 P99 : 3.749225 P100 : 90404.000000 COUNT : 4041089 SUM : 11568718 rocksdb.numfiles.in.singlecompaction P50 : 3.550000 P95 : 8.600000 P99 : 18.640000 P100 : 22.000000 COUNT : 48 SUM : 220 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.574859 P95 : 1947.811280 P99 : 2710.160312 P100 : 874881.000000 COUNT : 229448 SUM : 47337808 rocksdb.sst.read.micros P50 : 2.702094 P95 : 185.989624 P99 : 446.978872 P100 : 64083.000000 COUNT : 4200543 SUM : 260601478 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1071996 SUM : 1071996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3241730 SUM : 3345465360 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 221111.111111 P95 : 939333.333333 P99 : 1105860.000000 P100 : 1105860.000000 COUNT : 46 SUM : 13525002 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 174 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0