[2024-11-18 09:20:45.427876] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 23.11.0 initialization... [2024-11-18 09:20:45.428005] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid962109 ] Set seed to 1731918045425404 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-18 09:20:45.528251] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-18 09:20:45.568181] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-18 09:20:45.568235] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 962109' to capture a snapshot of events at runtime. [2024-11-18 09:20:45.568247] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid962109 for offline analysis/debug. [2024-11-18 09:20:45.568270] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Nov 18 09:20:49 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/18-09:20:54 ... thread 0: (1000000,1000000) ops and (204601.8,204601.8) ops/second in (4.887543,4.887543) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/12 1.73 GB 63.0 0.3 0.0 0.3 1.2 0.9 0.0 1.4 57.2 217.4 5.50 3.60 13 0.423 319K 360 0.0 0.0 L1 5/5 455.76 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 2/0 98.57 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 19/0 986.58 MB 1.0 0.7 0.2 0.5 0.7 0.2 0.1 3.6 132.4 131.8 5.68 3.58 4 1.420 209K 0 0.0 0.0 L4 170/0 9.76 GB 1.0 1.8 0.9 0.9 1.8 0.9 0.1 2.0 117.9 116.3 15.83 8.58 13 1.218 335K 0 0.0 0.0 L5 1120/0 92.09 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1333/17 105.08 GB 0.0 2.9 1.1 1.8 3.7 1.9 1.8 4.3 108.6 140.2 27.01 15.76 30 0.900 864K 360 0.0 0.0 Int 0/0 0.00 KB 0.0 2.9 1.1 1.8 3.7 1.9 1.8 4.3 108.6 140.2 27.01 15.76 30 0.900 864K 360 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.9 1.1 1.8 2.8 1.1 0.0 0.0 128.9 127.6 22.76 13.36 18 1.265 864K 360 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 207.4 4.25 2.40 12 0.354 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 5.0 total, 4.9 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.70 GB write, 751.39 MB/s write, 2.86 GB read, 582.06 MB/s read, 27.0 seconds Interval compaction: 3.70 GB write, 774.42 MB/s write, 2.86 GB read, 599.90 MB/s read, 27.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 17, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 17, interval: 17 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 311990 Average: 7.0489 StdDev: 77.77 Min: 0 Median: 1.6171 Max: 8738 Percentiles: P50: 1.62 P75: 2.23 P99: 14.91 P99.9: 821.51 P99.99: 1517.51 ------------------------------------------------------ [ 0, 1 ] 45508 14.586% 14.586% ### ( 1, 2 ] 179050 57.390% 71.976% ########### ( 2, 3 ] 41713 13.370% 85.346% ### ( 3, 4 ] 19110 6.125% 91.471% # ( 4, 6 ] 17710 5.676% 97.148% # ( 6, 10 ] 4760 1.526% 98.673% ( 10, 15 ] 1039 0.333% 99.006% ( 15, 22 ] 142 0.046% 99.052% ( 22, 34 ] 22 0.007% 99.059% ( 34, 51 ] 13 0.004% 99.063% ( 51, 76 ] 11 0.004% 99.067% ( 76, 110 ] 221 0.071% 99.137% ( 110, 170 ] 759 0.243% 99.381% ( 170, 250 ] 539 0.173% 99.554% ( 250, 380 ] 436 0.140% 99.693% ( 380, 580 ] 403 0.129% 99.822% ( 580, 870 ] 293 0.094% 99.916% ( 870, 1300 ] 197 0.063% 99.979% ( 1300, 1900 ] 96 0.031% 100.010% ( 1900, 2900 ] 64 0.021% 100.031% ( 2900, 4400 ] 32 0.010% 100.041% ( 4400, 6600 ] 7 0.002% 100.043% ( 6600, 9900 ] 3 0.001% 100.044% ** Level 1 read latency histogram (micros): Count: 13009 Average: 18.0304 StdDev: 170.57 Min: 1 Median: 1.8451 Max: 5080 Percentiles: P50: 1.85 P75: 2.89 P99: 366.61 P99.9: 2827.93 P99.99: 5080.00 ------------------------------------------------------ [ 0, 1 ] 960 7.380% 7.380% # ( 1, 2 ] 6561 50.434% 57.814% ########## ( 2, 3 ] 2501 19.225% 77.039% #### ( 3, 4 ] 1119 8.602% 85.641% ## ( 4, 6 ] 1148 8.825% 94.465% ## ( 6, 10 ] 340 2.614% 97.079% # ( 10, 15 ] 72 0.553% 97.632% ( 15, 22 ] 17 0.131% 97.763% ( 22, 34 ] 5 0.038% 97.802% ( 34, 51 ] 2 0.015% 97.817% ( 76, 110 ] 21 0.161% 97.978% ( 110, 170 ] 68 0.523% 98.501% ( 170, 250 ] 38 0.292% 98.793% ( 250, 380 ] 30 0.231% 99.024% ( 380, 580 ] 30 0.231% 99.254% ( 580, 870 ] 28 0.215% 99.470% ( 870, 1300 ] 23 0.177% 99.646% ( 1300, 1900 ] 20 0.154% 99.800% ( 1900, 2900 ] 14 0.108% 99.908% ( 2900, 4400 ] 7 0.054% 99.962% ( 4400, 6600 ] 5 0.038% 100.000% ** Level 2 read latency histogram (micros): Count: 27376 Average: 12.0803 StdDev: 128.03 Min: 1 Median: 1.9310 Max: 6627 Percentiles: P50: 1.93 P75: 2.98 P99: 148.13 P99.9: 1735.90 P99.99: 3777.07 ------------------------------------------------------ [ 0, 1 ] 1685 6.155% 6.155% # ( 1, 2 ] 12892 47.092% 53.247% ######### ( 2, 3 ] 6056 22.122% 75.369% #### ( 3, 4 ] 2772 10.126% 85.495% ## ( 4, 6 ] 2648 9.673% 95.167% ## ( 6, 10 ] 747 2.729% 97.896% # ( 10, 15 ] 218 0.796% 98.692% ( 15, 22 ] 39 0.142% 98.835% ( 22, 34 ] 8 0.029% 98.864% ( 34, 51 ] 3 0.011% 98.875% ( 51, 76 ] 2 0.007% 98.882% ( 76, 110 ] 10 0.037% 98.919% ( 110, 170 ] 35 0.128% 99.047% ( 170, 250 ] 28 0.102% 99.149% ( 250, 380 ] 29 0.106% 99.255% ( 380, 580 ] 74 0.270% 99.525% ( 580, 870 ] 47 0.172% 99.697% ( 870, 1300 ] 44 0.161% 99.858% ( 1300, 1900 ] 16 0.058% 99.916% ( 1900, 2900 ] 15 0.055% 99.971% ( 2900, 4400 ] 9 0.033% 100.004% ( 4400, 6600 ] 3 0.011% 100.015% ( 6600, 9900 ] 1 0.004% 100.018% ** Level 3 read latency histogram (micros): Count: 198279 Average: 7.7835 StdDev: 102.32 Min: 1 Median: 1.8420 Max: 7919 Percentiles: P50: 1.84 P75: 2.91 P99: 9.94 P99.9: 847.46 P99.99: 1378.73 ------------------------------------------------------ [ 0, 1 ] 11833 5.968% 5.968% # ( 1, 2 ] 103689 52.294% 58.262% ########## ( 2, 3 ] 36516 18.416% 76.679% #### ( 3, 4 ] 19206 9.686% 86.365% ## ( 4, 6 ] 20519 10.349% 96.714% ## ( 6, 10 ] 4605 2.322% 99.036% ( 10, 15 ] 1016 0.512% 99.549% ( 15, 22 ] 140 0.071% 99.619% ( 22, 34 ] 19 0.010% 99.629% ( 34, 51 ] 13 0.007% 99.635% ( 51, 76 ] 12 0.006% 99.641% ( 76, 110 ] 39 0.020% 99.661% ( 110, 170 ] 74 0.037% 99.698% ( 170, 250 ] 66 0.033% 99.732% ( 250, 380 ] 64 0.032% 99.764% ( 380, 580 ] 124 0.063% 99.827% ( 580, 870 ] 158 0.080% 99.906% ( 870, 1300 ] 152 0.077% 99.983% ( 1300, 1900 ] 108 0.054% 100.037% ( 1900, 2900 ] 64 0.032% 100.070% ( 2900, 4400 ] 50 0.025% 100.095% ( 4400, 6600 ] 17 0.009% 100.103% ( 6600, 9900 ] 2 0.001% 100.104% ** Level 4 read latency histogram (micros): Count: 219860 Average: 22.0487 StdDev: 211.51 Min: 1 Median: 1.8779 Max: 13664 Percentiles: P50: 1.88 P75: 3.03 P99: 393.68 P99.9: 2067.15 P99.99: 2754.22 ------------------------------------------------------ [ 0, 1 ] 12049 5.480% 5.480% # ( 1, 2 ] 111500 50.714% 56.194% ########## ( 2, 3 ] 40619 18.475% 74.669% #### ( 3, 4 ] 21237 9.659% 84.329% ## ( 4, 6 ] 22859 10.397% 94.726% ## ( 6, 10 ] 6020 2.738% 97.464% # ( 10, 15 ] 1564 0.711% 98.175% ( 15, 22 ] 218 0.099% 98.274% ( 22, 34 ] 27 0.012% 98.287% ( 34, 51 ] 13 0.006% 98.293% ( 51, 76 ] 10 0.005% 98.297% ( 76, 110 ] 169 0.077% 98.374% ( 110, 170 ] 596 0.271% 98.645% ( 170, 250 ] 419 0.191% 98.836% ( 250, 380 ] 327 0.149% 98.984% ( 380, 580 ] 503 0.229% 99.213% ( 580, 870 ] 554 0.252% 99.465% ( 870, 1300 ] 499 0.227% 99.692% ( 1300, 1900 ] 409 0.186% 99.878% ( 1900, 2900 ] 288 0.131% 100.009% ( 2900, 4400 ] 271 0.123% 100.132% ( 4400, 6600 ] 79 0.036% 100.168% ( 6600, 9900 ] 10 0.005% 100.173% ( 9900, 14000 ] 1 0.000% 100.173% ** Level 5 read latency histogram (micros): Count: 18784 Average: 733.5733 StdDev: 1143.07 Min: 78 Median: 252.8044 Max: 11088 Percentiles: P50: 252.80 P75: 703.13 P99: 5881.37 P99.9: 8997.12 P99.99: 9855.40 ------------------------------------------------------ ( 76, 110 ] 1395 7.427% 7.427% # ( 110, 170 ] 4989 26.560% 33.986% ##### ( 170, 250 ] 2963 15.774% 49.760% ### ( 250, 380 ] 2086 11.105% 60.866% ## ( 380, 580 ] 2061 10.972% 71.838% ## ( 580, 870 ] 1399 7.448% 79.286% # ( 870, 1300 ] 1000 5.324% 84.609% # ( 1300, 1900 ] 780 4.152% 88.762% # ( 1900, 2900 ] 811 4.318% 93.079% # ( 2900, 4400 ] 861 4.584% 97.663% # ( 4400, 6600 ] 373 1.986% 99.649% ( 6600, 9900 ] 65 0.346% 99.995% ( 9900, 14000 ] 1 0.005% 100.000% ** DB Stats ** Uptime(secs): 5.0 total, 4.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 195.27 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.190 H:M:S, 23.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 201.22 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.190 H:M:S, 24.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/18-09:20:58 ... thread 0: (1000000,2000000) ops and (240205.7,220978.8) ops/second in (4.163098,9.050641) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/8 2.71 GB 188.8 1.5 0.0 1.5 3.4 1.8 0.0 1.8 117.6 258.5 13.40 10.94 34 0.394 1599K 1534 0.0 0.0 L1 5/5 455.76 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 2/0 98.57 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 19/0 986.58 MB 1.0 0.7 0.2 0.5 0.7 0.2 0.1 3.6 132.4 131.8 5.68 3.58 4 1.420 209K 0 0.0 0.0 L4 170/0 9.76 GB 1.0 1.8 0.9 0.9 1.8 0.9 0.1 2.0 117.9 116.3 15.83 8.58 13 1.218 335K 0 0.0 0.0 L5 1120/0 92.09 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1334/13 106.06 GB 0.0 4.1 1.1 3.0 5.9 2.9 1.8 3.2 120.1 173.4 34.91 23.10 51 0.684 2143K 1534 0.0 0.0 Int 0/0 0.00 KB 0.0 4.1 1.1 3.0 5.9 2.9 1.8 3.2 120.1 173.4 34.91 23.10 51 0.684 2143K 1534 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.1 1.1 3.0 4.1 1.1 0.0 0.0 152.2 151.1 27.55 18.03 23 1.198 2143K 1534 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 256.9 7.36 5.07 28 0.263 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.2 total, 9.1 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.91 GB write, 657.81 MB/s write, 4.10 GB read, 455.68 MB/s read, 34.9 seconds Interval compaction: 5.91 GB write, 668.70 MB/s write, 4.10 GB read, 463.22 MB/s read, 34.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 17, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 17, interval: 21 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 610948 Average: 11.5425 StdDev: 72.97 Min: 0 Median: 1.5170 Max: 8738 Percentiles: P50: 1.52 P75: 1.97 P99: 227.53 P99.9: 683.21 P99.99: 1080.79 ------------------------------------------------------ [ 0, 1 ] 132516 21.690% 21.690% #### ( 1, 2 ] 334550 54.759% 76.449% ########### ( 2, 3 ] 61746 10.107% 86.556% ## ( 3, 4 ] 27231 4.457% 91.013% # ( 4, 6 ] 22547 3.690% 94.704% # ( 6, 10 ] 5744 0.940% 95.644% ( 10, 15 ] 1354 0.222% 95.865% ( 15, 22 ] 188 0.031% 95.896% ( 22, 34 ] 32 0.005% 95.901% ( 34, 51 ] 14 0.002% 95.904% ( 51, 76 ] 18 0.003% 95.907% ( 76, 110 ] 3384 0.554% 96.461% ( 110, 170 ] 11819 1.935% 98.395% ( 170, 250 ] 5139 0.841% 99.236% ( 250, 380 ] 2071 0.339% 99.575% ( 380, 580 ] 1752 0.287% 99.862% ( 580, 870 ] 652 0.107% 99.969% ( 870, 1300 ] 265 0.043% 100.012% ( 1300, 1900 ] 118 0.019% 100.031% ( 1900, 2900 ] 76 0.012% 100.044% ( 2900, 4400 ] 47 0.008% 100.052% ( 4400, 6600 ] 7 0.001% 100.053% ( 6600, 9900 ] 3 0.000% 100.053% ** Level 1 read latency histogram (micros): Count: 27463 Average: 14.1738 StdDev: 120.97 Min: 1 Median: 2.7696 Max: 5080 Percentiles: P50: 2.77 P75: 4.54 P99: 224.05 P99.9: 1856.11 P99.99: 5080.00 ------------------------------------------------------ [ 0, 1 ] 1034 3.765% 3.765% # ( 1, 2 ] 8540 31.096% 34.861% ###### ( 2, 3 ] 5402 19.670% 54.532% #### ( 3, 4 ] 4083 14.867% 69.399% ### ( 4, 6 ] 5692 20.726% 90.125% #### ( 6, 10 ] 1408 5.127% 95.252% # ( 10, 15 ] 331 1.205% 96.457% ( 15, 22 ] 166 0.604% 97.062% ( 22, 34 ] 28 0.102% 97.163% ( 34, 51 ] 7 0.025% 97.189% ( 76, 110 ] 103 0.375% 97.564% ( 110, 170 ] 316 1.151% 98.715% ( 170, 250 ] 116 0.422% 99.137% ( 250, 380 ] 64 0.233% 99.370% ( 380, 580 ] 68 0.248% 99.618% ( 580, 870 ] 36 0.131% 99.749% ( 870, 1300 ] 23 0.084% 99.833% ( 1300, 1900 ] 20 0.073% 99.905% ( 1900, 2900 ] 14 0.051% 99.956% ( 2900, 4400 ] 7 0.025% 99.982% ( 4400, 6600 ] 5 0.018% 100.000% ** Level 2 read latency histogram (micros): Count: 27631 Average: 13.7618 StdDev: 129.26 Min: 1 Median: 1.9409 Max: 6627 Percentiles: P50: 1.94 P75: 3.03 P99: 251.66 P99.9: 1726.34 P99.99: 3772.82 ------------------------------------------------------ [ 0, 1 ] 1685 6.098% 6.098% # ( 1, 2 ] 12892 46.658% 52.756% ######### ( 2, 3 ] 6056 21.917% 74.673% #### ( 3, 4 ] 2772 10.032% 84.706% ## ( 4, 6 ] 2648 9.583% 94.289% ## ( 6, 10 ] 747 2.703% 96.993% # ( 10, 15 ] 218 0.789% 97.781% ( 15, 22 ] 39 0.141% 97.923% ( 22, 34 ] 8 0.029% 97.952% ( 34, 51 ] 3 0.011% 97.962% ( 51, 76 ] 2 0.007% 97.970% ( 76, 110 ] 46 0.166% 98.136% ( 110, 170 ] 155 0.561% 98.697% ( 170, 250 ] 83 0.300% 98.998% ( 250, 380 ] 54 0.195% 99.193% ( 380, 580 ] 85 0.308% 99.501% ( 580, 870 ] 54 0.195% 99.696% ( 870, 1300 ] 45 0.163% 99.859% ( 1300, 1900 ] 16 0.058% 99.917% ( 1900, 2900 ] 15 0.054% 99.971% ( 2900, 4400 ] 9 0.033% 100.004% ( 4400, 6600 ] 3 0.011% 100.014% ( 6600, 9900 ] 1 0.004% 100.018% ** Level 3 read latency histogram (micros): Count: 199251 Average: 8.6472 StdDev: 103.47 Min: 1 Median: 1.8467 Max: 7919 Percentiles: P50: 1.85 P75: 2.93 P99: 14.36 P99.9: 852.30 P99.99: 1383.75 ------------------------------------------------------ [ 0, 1 ] 11833 5.939% 5.939% # ( 1, 2 ] 103689 52.039% 57.978% ########## ( 2, 3 ] 36518 18.328% 76.306% #### ( 3, 4 ] 19207 9.640% 85.945% ## ( 4, 6 ] 20520 10.299% 96.244% ## ( 6, 10 ] 4606 2.312% 98.556% ( 10, 15 ] 1016 0.510% 99.066% ( 15, 22 ] 140 0.070% 99.136% ( 22, 34 ] 19 0.010% 99.145% ( 34, 51 ] 13 0.007% 99.152% ( 51, 76 ] 12 0.006% 99.158% ( 76, 110 ] 187 0.094% 99.252% ( 110, 170 ] 569 0.286% 99.537% ( 170, 250 ] 265 0.133% 99.670% ( 250, 380 ] 118 0.059% 99.729% ( 380, 580 ] 182 0.091% 99.821% ( 580, 870 ] 168 0.084% 99.905% ( 870, 1300 ] 154 0.077% 99.982% ( 1300, 1900 ] 108 0.054% 100.037% ( 1900, 2900 ] 64 0.032% 100.069% ( 2900, 4400 ] 51 0.026% 100.094% ( 4400, 6600 ] 17 0.009% 100.103% ( 6600, 9900 ] 2 0.001% 100.104% ** Level 4 read latency histogram (micros): Count: 227093 Average: 26.7386 StdDev: 211.44 Min: 1 Median: 1.9098 Max: 13664 Percentiles: P50: 1.91 P75: 3.27 P99: 485.57 P99.9: 2063.51 P99.99: 2761.06 ------------------------------------------------------ [ 0, 1 ] 12049 5.306% 5.306% # ( 1, 2 ] 111566 49.128% 54.434% ########## ( 2, 3 ] 40872 17.998% 72.432% #### ( 3, 4 ] 21392 9.420% 81.851% ## ( 4, 6 ] 23085 10.165% 92.017% ## ( 6, 10 ] 6049 2.664% 94.681% # ( 10, 15 ] 1577 0.694% 95.375% ( 15, 22 ] 218 0.096% 95.471% ( 22, 34 ] 27 0.012% 95.483% ( 34, 51 ] 13 0.006% 95.489% ( 51, 76 ] 10 0.004% 95.493% ( 76, 110 ] 1208 0.532% 96.025% ( 110, 170 ] 3916 1.724% 97.749% ( 170, 250 ] 1635 0.720% 98.469% ( 250, 380 ] 749 0.330% 98.799% ( 380, 580 ] 864 0.380% 99.180% ( 580, 870 ] 656 0.289% 99.468% ( 870, 1300 ] 519 0.229% 99.697% ( 1300, 1900 ] 413 0.182% 99.879% ( 1900, 2900 ] 293 0.129% 100.008% ( 2900, 4400 ] 273 0.120% 100.128% ( 4400, 6600 ] 79 0.035% 100.163% ( 6600, 9900 ] 10 0.004% 100.167% ( 9900, 14000 ] 1 0.000% 100.168% ** Level 5 read latency histogram (micros): Count: 75958 Average: 323.3195 StdDev: 628.07 Min: 75 Median: 158.4290 Max: 11088 Percentiles: P50: 158.43 P75: 242.37 P99: 3847.64 P99.9: 6529.47 P99.99: 9463.60 ------------------------------------------------------ ( 51, 76 ] 4 0.005% 0.005% ( 76, 110 ] 10293 13.551% 13.556% ### ( 110, 170 ] 34296 45.151% 58.707% ######### ( 170, 250 ] 13681 18.011% 76.719% #### ( 250, 380 ] 5976 7.868% 84.586% ## ( 380, 580 ] 5245 6.905% 91.491% # ( 580, 870 ] 2317 3.050% 94.542% # ( 870, 1300 ] 1159 1.526% 96.068% ( 1300, 1900 ] 831 1.094% 97.162% ( 1900, 2900 ] 843 1.110% 98.271% ( 2900, 4400 ] 876 1.153% 99.425% ( 4400, 6600 ] 373 0.491% 99.916% ( 6600, 9900 ] 65 0.086% 100.001% ( 9900, 14000 ] 1 0.001% 100.003% ** DB Stats ** Uptime(secs): 9.2 total, 4.2 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 213.88 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.915 H:M:S, 20.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 236.41 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.726 H:M:S, 17.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/18-09:21:08 ... thread 0: (1000000,3000000) ops and (99503.9,157063.9) ops/second in (10.049861,19.100502) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/7 2.70 GB 119.6 2.3 0.0 2.3 5.0 2.7 0.0 1.8 113.5 248.3 20.54 16.25 50 0.411 2366K 2232 0.0 0.0 L1 3/0 221.81 MB 22.2 1.3 0.9 0.4 1.3 0.9 0.0 1.5 82.4 81.9 16.31 7.93 1 16.314 1364K 7139 0.0 0.0 L2 2/2 178.25 MB 0.0 0.0 0.0 0.0 0.0 0.0 1.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 32/13 1.97 GB 1.0 2.4 1.1 1.3 2.4 1.1 0.2 2.2 131.7 131.2 18.77 11.56 13 1.444 1618K 4184 0.0 0.0 L4 170/21 9.76 GB 0.9 1.8 0.9 0.9 1.8 0.9 0.1 2.0 117.9 116.3 15.83 8.58 13 1.218 335K 0 0.0 0.0 L5 1120/0 92.09 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1341/43 106.91 GB 0.0 7.8 2.8 5.0 10.5 5.5 3.0 3.9 112.2 150.3 71.45 44.31 77 0.928 5684K 13K 0.0 0.0 Int 0/0 0.00 KB 0.0 7.8 2.8 5.0 10.5 5.5 3.0 3.9 112.2 150.3 71.45 44.31 77 0.928 5684K 13K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 7.8 2.8 5.0 7.8 2.8 0.0 0.0 133.4 132.6 60.08 36.80 36 1.669 5684K 13K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.7 2.7 0.0 0.0 0.0 243.6 11.38 7.51 41 0.278 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 19.3 total, 19.1 interval Flush(GB): cumulative 2.707, interval 2.707 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.49 GB write, 557.85 MB/s write, 7.83 GB read, 416.23 MB/s read, 71.5 seconds Interval compaction: 10.49 GB write, 562.23 MB/s write, 7.83 GB read, 419.50 MB/s read, 71.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 12, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 12, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 24, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 12, total-stops: 24, interval: 36 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1247340 Average: 14.7161 StdDev: 95.49 Min: 0 Median: 1.4894 Max: 8738 Percentiles: P50: 1.49 P75: 1.96 P99: 242.41 P99.9: 819.98 P99.99: 1475.10 ------------------------------------------------------ [ 0, 1 ] 301567 24.177% 24.177% ##### ( 1, 2 ] 658189 52.767% 76.944% ########### ( 2, 3 ] 112608 9.028% 85.972% ## ( 3, 4 ] 51520 4.130% 90.102% # ( 4, 6 ] 42808 3.432% 93.534% # ( 6, 10 ] 9369 0.751% 94.286% ( 10, 15 ] 2057 0.165% 94.450% ( 15, 22 ] 276 0.022% 94.473% ( 22, 34 ] 45 0.004% 94.476% ( 34, 51 ] 22 0.002% 94.478% ( 51, 76 ] 27 0.002% 94.480% ( 76, 110 ] 10684 0.857% 95.337% ( 110, 170 ] 34434 2.761% 98.097% # ( 170, 250 ] 12441 0.997% 99.095% ( 250, 380 ] 4678 0.375% 99.470% ( 380, 580 ] 3994 0.320% 99.790% ( 580, 870 ] 1660 0.133% 99.923% ( 870, 1300 ] 735 0.059% 99.982% ( 1300, 1900 ] 347 0.028% 100.010% ( 1900, 2900 ] 219 0.018% 100.027% ( 2900, 4400 ] 255 0.020% 100.048% ( 4400, 6600 ] 59 0.005% 100.052% ( 6600, 9900 ] 8 0.001% 100.053% ** Level 1 read latency histogram (micros): Count: 334420 Average: 8.2161 StdDev: 109.55 Min: 1 Median: 2.0452 Max: 13160 Percentiles: P50: 2.05 P75: 3.09 P99: 14.33 P99.9: 641.23 P99.99: 1353.10 ------------------------------------------------------ [ 0, 1 ] 9580 2.865% 2.865% # ( 1, 2 ] 153812 45.994% 48.858% ######### ( 2, 3 ] 84458 25.255% 74.113% ##### ( 3, 4 ] 34120 10.203% 84.316% ## ( 4, 6 ] 38491 11.510% 95.826% ## ( 6, 10 ] 8738 2.613% 98.439% # ( 10, 15 ] 2168 0.648% 99.087% ( 15, 22 ] 462 0.138% 99.225% ( 22, 34 ] 52 0.016% 99.241% ( 34, 51 ] 40 0.012% 99.253% ( 51, 76 ] 25 0.007% 99.260% ( 76, 110 ] 354 0.106% 99.366% ( 110, 170 ] 969 0.290% 99.656% ( 170, 250 ] 317 0.095% 99.751% ( 250, 380 ] 245 0.073% 99.824% ( 380, 580 ] 217 0.065% 99.889% ( 580, 870 ] 178 0.053% 99.942% ( 870, 1300 ] 152 0.045% 99.987% ( 1300, 1900 ] 108 0.032% 100.020% ( 1900, 2900 ] 90 0.027% 100.047% ( 2900, 4400 ] 134 0.040% 100.087% ( 4400, 6600 ] 25 0.007% 100.094% ( 6600, 9900 ] 3 0.001% 100.095% ( 9900, 14000 ] 2 0.001% 100.096% ** Level 2 read latency histogram (micros): Count: 68916 Average: 12.5754 StdDev: 127.24 Min: 1 Median: 1.9311 Max: 6627 Percentiles: P50: 1.93 P75: 3.11 P99: 197.67 P99.9: 1777.60 P99.99: 4076.96 ------------------------------------------------------ [ 0, 1 ] 3035 4.404% 4.404% # ( 1, 2 ] 33750 48.973% 53.377% ########## ( 2, 3 ] 14174 20.567% 73.944% #### ( 3, 4 ] 6885 9.990% 83.934% ## ( 4, 6 ] 7279 10.562% 94.496% ## ( 6, 10 ] 1835 2.663% 97.159% # ( 10, 15 ] 488 0.708% 97.867% ( 15, 22 ] 66 0.096% 97.963% ( 22, 34 ] 14 0.020% 97.983% ( 34, 51 ] 9 0.013% 97.996% ( 51, 76 ] 14 0.020% 98.016% ( 76, 110 ] 144 0.209% 98.225% ( 110, 170 ] 465 0.675% 98.900% ( 170, 250 ] 199 0.289% 99.189% ( 250, 380 ] 142 0.206% 99.395% ( 380, 580 ] 165 0.239% 99.634% ( 580, 870 ] 90 0.131% 99.765% ( 870, 1300 ] 70 0.102% 99.867% ( 1300, 1900 ] 29 0.042% 99.909% ( 1900, 2900 ] 31 0.045% 99.954% ( 2900, 4400 ] 32 0.046% 100.000% ( 4400, 6600 ] 10 0.015% 100.015% ( 6600, 9900 ] 1 0.001% 100.016% ** Level 3 read latency histogram (micros): Count: 541565 Average: 10.1373 StdDev: 132.09 Min: 1 Median: 1.8917 Max: 9080 Percentiles: P50: 1.89 P75: 2.99 P99: 14.60 P99.9: 844.92 P99.99: 1340.28 ------------------------------------------------------ [ 0, 1 ] 21286 3.930% 3.930% # ( 1, 2 ] 279800 51.665% 55.596% ########## ( 2, 3 ] 106437 19.654% 75.249% #### ( 3, 4 ] 54420 10.049% 85.298% ## ( 4, 6 ] 59122 10.917% 96.215% ## ( 6, 10 ] 12240 2.260% 98.475% ( 10, 15 ] 3091 0.571% 99.046% ( 15, 22 ] 366 0.068% 99.113% ( 22, 34 ] 40 0.007% 99.121% ( 34, 51 ] 52 0.010% 99.130% ( 51, 76 ] 69 0.013% 99.143% ( 76, 110 ] 525 0.097% 99.240% ( 110, 170 ] 1613 0.298% 99.538% ( 170, 250 ] 663 0.122% 99.660% ( 250, 380 ] 375 0.069% 99.729% ( 380, 580 ] 497 0.092% 99.821% ( 580, 870 ] 469 0.087% 99.908% ( 870, 1300 ] 431 0.080% 99.987% ( 1300, 1900 ] 236 0.044% 100.031% ( 1900, 2900 ] 195 0.036% 100.067% ( 2900, 4400 ] 265 0.049% 100.116% ( 4400, 6600 ] 83 0.015% 100.131% ( 6600, 9900 ] 20 0.004% 100.135% ** Level 4 read latency histogram (micros): Count: 324854 Average: 33.0914 StdDev: 227.24 Min: 1 Median: 1.9595 Max: 13664 Percentiles: P50: 1.96 P75: 3.57 P99: 546.23 P99.9: 2378.30 P99.99: 3172.93 ------------------------------------------------------ [ 0, 1 ] 14595 4.493% 4.493% # ( 1, 2 ] 154078 47.430% 51.923% ######### ( 2, 3 ] 57895 17.822% 69.745% #### ( 3, 4 ] 30165 9.286% 79.030% ## ( 4, 6 ] 32689 10.063% 89.093% ## ( 6, 10 ] 8156 2.511% 91.604% # ( 10, 15 ] 2208 0.680% 92.283% ( 15, 22 ] 292 0.090% 92.373% ( 22, 34 ] 31 0.010% 92.383% ( 34, 51 ] 23 0.007% 92.390% ( 51, 76 ] 28 0.009% 92.398% ( 76, 110 ] 3484 1.072% 93.471% ( 110, 170 ] 11135 3.428% 96.899% # ( 170, 250 ] 3919 1.206% 98.105% ( 250, 380 ] 1571 0.484% 98.589% ( 380, 580 ] 1608 0.495% 99.084% ( 580, 870 ] 1083 0.333% 99.417% ( 870, 1300 ] 816 0.251% 99.668% ( 1300, 1900 ] 558 0.172% 99.840% ( 1900, 2900 ] 408 0.126% 99.966% ( 2900, 4400 ] 437 0.135% 100.100% ( 4400, 6600 ] 146 0.045% 100.145% ( 6600, 9900 ] 23 0.007% 100.152% ( 9900, 14000 ] 1 0.000% 100.152% ** Level 5 read latency histogram (micros): Count: 190990 Average: 268.3716 StdDev: 507.21 Min: 74 Median: 151.6968 Max: 11088 Percentiles: P50: 151.70 P75: 217.71 P99: 3126.06 P99.9: 6181.92 P99.99: 8927.97 ------------------------------------------------------ ( 51, 76 ] 15 0.008% 0.008% ( 76, 110 ] 29836 15.622% 15.630% ### ( 110, 170 ] 94459 49.458% 65.087% ########## ( 170, 250 ] 31747 16.622% 81.710% ### ( 250, 380 ] 12301 6.441% 88.150% # ( 380, 580 ] 10468 5.481% 93.631% # ( 580, 870 ] 4830 2.529% 96.160% # ( 870, 1300 ] 2525 1.322% 97.482% ( 1300, 1900 ] 1459 0.764% 98.246% ( 1900, 2900 ] 1216 0.637% 98.883% ( 2900, 4400 ] 1487 0.779% 99.661% ( 4400, 6600 ] 563 0.295% 99.956% ( 6600, 9900 ] 92 0.048% 100.004% ( 9900, 14000 ] 2 0.001% 100.005% ** DB Stats ** Uptime(secs): 19.3 total, 10.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 153.36 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.943 H:M:S, 41.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 97.93 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:6.027 H:M:S, 60.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/11/18-09:21:14 ... thread 0: (1000000,4000000) ops and (168846.5,159852.7) ops/second in (5.922539,25.023041) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/11 3.81 GB 132.2 3.4 0.0 3.4 7.2 3.8 0.0 1.9 113.3 240.9 30.58 23.67 70 0.437 3518K 3466 0.0 0.0 L1 3/0 221.81 MB 22.2 1.3 0.9 0.4 1.3 0.9 0.0 1.5 82.4 81.9 16.31 7.93 1 16.314 1364K 7139 0.0 0.0 L3 19/0 969.13 MB 1.0 2.9 1.3 1.7 2.9 1.2 0.2 2.3 134.9 134.3 22.34 14.03 15 1.490 1993K 4184 0.0 0.0 L4 166/0 9.55 GB 1.0 5.5 2.0 3.5 5.5 2.0 0.2 2.7 133.8 132.1 42.42 26.10 27 1.571 2381K 0 0.0 0.0 L5 1136/0 93.44 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1342/11 107.97 GB 0.0 13.2 4.2 9.0 16.9 7.9 3.3 4.4 120.9 155.0 111.66 71.72 113 0.988 9258K 14K 0.0 0.0 Int 0/0 0.00 KB 0.0 13.2 4.2 9.0 16.9 7.9 3.3 4.4 120.9 155.0 111.66 71.72 113 0.988 9258K 14K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.2 4.2 9.0 13.1 4.1 0.0 0.0 141.8 140.7 95.22 61.13 56 1.700 9258K 14K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 237.6 16.44 10.60 57 0.288 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 25.2 total, 25.0 interval Flush(GB): cumulative 3.814, interval 3.814 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 16.90 GB write, 687.47 MB/s write, 13.18 GB read, 536.15 MB/s read, 111.7 seconds Interval compaction: 16.90 GB write, 691.58 MB/s write, 13.18 GB read, 539.36 MB/s read, 111.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 19, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 19, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 34, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 19, total-stops: 34, interval: 53 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1752038 Average: 16.1674 StdDev: 116.98 Min: 0 Median: 1.5273 Max: 8738 Percentiles: P50: 1.53 P75: 1.99 P99: 286.75 P99.9: 1133.94 P99.99: 2112.30 ------------------------------------------------------ [ 0, 1 ] 375006 21.404% 21.404% #### ( 1, 2 ] 950217 54.235% 75.639% ########### ( 2, 3 ] 171919 9.813% 85.451% ## ( 3, 4 ] 81675 4.662% 90.113% # ( 4, 6 ] 68212 3.893% 94.006% # ( 6, 10 ] 14576 0.832% 94.838% ( 10, 15 ] 2854 0.163% 95.001% ( 15, 22 ] 346 0.020% 95.021% ( 22, 34 ] 57 0.003% 95.024% ( 34, 51 ] 35 0.002% 95.026% ( 51, 76 ] 41 0.002% 95.029% ( 76, 110 ] 11785 0.673% 95.701% ( 110, 170 ] 39969 2.281% 97.983% ( 170, 250 ] 15881 0.906% 98.889% ( 250, 380 ] 6886 0.393% 99.282% ( 380, 580 ] 6239 0.356% 99.638% ( 580, 870 ] 3498 0.200% 99.838% ( 870, 1300 ] 1779 0.102% 99.939% ( 1300, 1900 ] 769 0.044% 99.983% ( 1900, 2900 ] 569 0.032% 100.016% ( 2900, 4400 ] 601 0.034% 100.050% ( 4400, 6600 ] 127 0.007% 100.057% ( 6600, 9900 ] 15 0.001% 100.058% ** Level 1 read latency histogram (micros): Count: 367310 Average: 8.3365 StdDev: 110.50 Min: 1 Median: 2.0195 Max: 13160 Percentiles: P50: 2.02 P75: 3.07 P99: 14.39 P99.9: 716.03 P99.99: 1563.54 ------------------------------------------------------ [ 0, 1 ] 10336 2.814% 2.814% # ( 1, 2 ] 171538 46.701% 49.515% ######### ( 2, 3 ] 91143 24.814% 74.329% ##### ( 3, 4 ] 37341 10.166% 84.495% ## ( 4, 6 ] 41693 11.351% 95.846% ## ( 6, 10 ] 9525 2.593% 98.439% # ( 10, 15 ] 2346 0.639% 99.078% ( 15, 22 ] 474 0.129% 99.207% ( 22, 34 ] 54 0.015% 99.221% ( 34, 51 ] 51 0.014% 99.235% ( 51, 76 ] 34 0.009% 99.245% ( 76, 110 ] 364 0.099% 99.344% ( 110, 170 ] 1048 0.285% 99.629% ( 170, 250 ] 364 0.099% 99.728% ( 250, 380 ] 286 0.078% 99.806% ( 380, 580 ] 250 0.068% 99.874% ( 580, 870 ] 204 0.056% 99.929% ( 870, 1300 ] 170 0.046% 99.976% ( 1300, 1900 ] 119 0.032% 100.008% ( 1900, 2900 ] 111 0.030% 100.038% ( 2900, 4400 ] 145 0.039% 100.078% ( 4400, 6600 ] 28 0.008% 100.085% ( 6600, 9900 ] 4 0.001% 100.087% ( 9900, 14000 ] 2 0.001% 100.087% ** Level 2 read latency histogram (micros): Count: 69075 Average: 12.5648 StdDev: 127.13 Min: 1 Median: 1.9307 Max: 6627 Percentiles: P50: 1.93 P75: 3.10 P99: 197.44 P99.9: 1774.31 P99.99: 4076.21 ------------------------------------------------------ [ 0, 1 ] 3044 4.407% 4.407% # ( 1, 2 ] 33837 48.986% 53.393% ########## ( 2, 3 ] 14208 20.569% 73.962% #### ( 3, 4 ] 6897 9.985% 83.946% ## ( 4, 6 ] 7293 10.558% 94.505% ## ( 6, 10 ] 1836 2.658% 97.163% # ( 10, 15 ] 489 0.708% 97.870% ( 15, 22 ] 66 0.096% 97.966% ( 22, 34 ] 14 0.020% 97.986% ( 34, 51 ] 9 0.013% 97.999% ( 51, 76 ] 14 0.020% 98.020% ( 76, 110 ] 144 0.208% 98.228% ( 110, 170 ] 465 0.673% 98.901% ( 170, 250 ] 199 0.288% 99.189% ( 250, 380 ] 142 0.206% 99.395% ( 380, 580 ] 165 0.239% 99.634% ( 580, 870 ] 91 0.132% 99.765% ( 870, 1300 ] 70 0.101% 99.867% ( 1300, 1900 ] 29 0.042% 99.909% ( 1900, 2900 ] 31 0.045% 99.954% ( 2900, 4400 ] 32 0.046% 100.000% ( 4400, 6600 ] 10 0.014% 100.014% ( 6600, 9900 ] 1 0.001% 100.016% ** Level 3 read latency histogram (micros): Count: 798622 Average: 9.2492 StdDev: 122.61 Min: 1 Median: 1.8804 Max: 9080 Percentiles: P50: 1.88 P75: 2.95 P99: 13.11 P99.9: 840.03 P99.99: 1439.88 ------------------------------------------------------ [ 0, 1 ] 30609 3.833% 3.833% # ( 1, 2 ] 418795 52.440% 56.272% ########## ( 2, 3 ] 156780 19.631% 75.904% #### ( 3, 4 ] 79283 9.927% 85.831% ## ( 4, 6 ] 84702 10.606% 96.437% ## ( 6, 10 ] 17870 2.238% 98.675% ( 10, 15 ] 4169 0.522% 99.197% ( 15, 22 ] 463 0.058% 99.255% ( 22, 34 ] 56 0.007% 99.262% ( 34, 51 ] 66 0.008% 99.270% ( 51, 76 ] 99 0.012% 99.283% ( 76, 110 ] 563 0.070% 99.353% ( 110, 170 ] 1783 0.223% 99.576% ( 170, 250 ] 804 0.101% 99.677% ( 250, 380 ] 513 0.064% 99.741% ( 380, 580 ] 673 0.084% 99.825% ( 580, 870 ] 664 0.083% 99.909% ( 870, 1300 ] 576 0.072% 99.981% ( 1300, 1900 ] 318 0.040% 100.021% ( 1900, 2900 ] 275 0.034% 100.055% ( 2900, 4400 ] 328 0.041% 100.096% ( 4400, 6600 ] 108 0.014% 100.110% ( 6600, 9900 ] 22 0.003% 100.112% ** Level 4 read latency histogram (micros): Count: 895601 Average: 20.6092 StdDev: 179.55 Min: 1 Median: 1.9224 Max: 13664 Percentiles: P50: 1.92 P75: 3.17 P99: 315.99 P99.9: 1484.47 P99.99: 2121.93 ------------------------------------------------------ [ 0, 1 ] 30761 3.435% 3.435% # ( 1, 2 ] 452105 50.481% 53.915% ########## ( 2, 3 ] 174504 19.485% 73.400% #### ( 3, 4 ] 85576 9.555% 82.955% ## ( 4, 6 ] 92133 10.287% 93.242% ## ( 6, 10 ] 22466 2.508% 95.751% # ( 10, 15 ] 5793 0.647% 96.398% ( 15, 22 ] 656 0.073% 96.471% ( 22, 34 ] 81 0.009% 96.480% ( 34, 51 ] 93 0.010% 96.490% ( 51, 76 ] 133 0.015% 96.505% ( 76, 110 ] 3773 0.421% 96.926% ( 110, 170 ] 12402 1.385% 98.311% ( 170, 250 ] 4899 0.547% 98.858% ( 250, 380 ] 2502 0.279% 99.138% ( 380, 580 ] 2717 0.303% 99.441% ( 580, 870 ] 2198 0.245% 99.686% ( 870, 1300 ] 1633 0.182% 99.869% ( 1300, 1900 ] 912 0.102% 99.971% ( 1900, 2900 ] 786 0.088% 100.058% ( 2900, 4400 ] 790 0.088% 100.146% ( 4400, 6600 ] 229 0.026% 100.172% ( 6600, 9900 ] 30 0.003% 100.175% ( 9900, 14000 ] 2 0.000% 100.176% ** Level 5 read latency histogram (micros): Count: 224239 Average: 300.1085 StdDev: 542.09 Min: 74 Median: 156.1660 Max: 11088 Percentiles: P50: 156.17 P75: 237.85 P99: 3355.96 P99.9: 6183.97 P99.99: 8839.43 ------------------------------------------------------ ( 51, 76 ] 15 0.007% 0.007% ( 76, 110 ] 31901 14.226% 14.233% ### ( 110, 170 ] 104237 46.485% 60.718% ######### ( 170, 250 ] 37762 16.840% 77.558% ### ( 250, 380 ] 16172 7.212% 84.770% # ( 380, 580 ] 14499 6.466% 91.236% # ( 580, 870 ] 8141 3.631% 94.866% # ( 870, 1300 ] 4500 2.007% 96.873% ( 1300, 1900 ] 2261 1.008% 97.881% ( 1900, 2900 ] 1873 0.835% 98.717% ( 2900, 4400 ] 2091 0.932% 99.649% ( 4400, 6600 ] 694 0.309% 99.959% ( 6600, 9900 ] 104 0.046% 100.005% ( 9900, 14000 ] 2 0.001% 100.006% ** DB Stats ** Uptime(secs): 25.2 total, 5.9 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 156.37 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:10.080 H:M:S, 40.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 166.18 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.138 H:M:S, 36.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 2024/11/18-09:21:18 ... thread 0: (1000000,5000000) ops and (270639.2,174106.9) ops/second in (3.694956,28.717997) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/15 4.72 GB 37.8 6.4 0.0 6.4 11.1 4.7 0.0 2.3 122.4 212.8 53.49 38.22 89 0.601 6650K 18K 0.0 0.0 L1 3/3 221.81 MB 0.0 1.3 0.9 0.4 1.3 0.9 0.0 1.5 82.4 81.9 16.31 7.93 1 16.314 1364K 7139 0.0 0.0 L3 19/0 969.13 MB 1.0 2.9 1.3 1.7 2.9 1.2 0.2 2.3 134.9 134.3 22.34 14.03 15 1.490 1993K 4184 0.0 0.0 L4 166/0 9.55 GB 1.0 5.5 2.0 3.5 5.5 2.0 0.2 2.7 133.8 132.1 42.42 26.10 27 1.571 2381K 0 0.0 0.0 L5 1136/0 93.44 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1342/18 108.87 GB 0.0 16.2 4.2 12.0 20.8 8.8 3.3 4.4 123.2 158.4 134.57 86.28 132 1.019 12M 29K 0.0 0.0 Int 0/0 0.00 KB 0.0 16.2 4.2 12.0 20.8 8.8 3.3 4.4 123.2 158.4 134.57 86.28 132 1.019 12M 29K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 16.2 4.2 12.0 16.1 4.1 0.0 0.0 143.9 143.0 115.20 73.14 60 1.920 12M 29K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 250.4 19.37 13.14 72 0.269 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 28.9 total, 28.7 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 20.82 GB write, 738.53 MB/s write, 16.19 GB read, 574.37 MB/s read, 134.6 seconds Interval compaction: 20.82 GB write, 742.39 MB/s write, 16.19 GB read, 577.36 MB/s read, 134.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 20, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 20, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 34, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 20, total-stops: 34, interval: 54 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2004990 Average: 16.0560 StdDev: 111.79 Min: 0 Median: 1.5154 Max: 8738 Percentiles: P50: 1.52 P75: 1.98 P99: 284.50 P99.9: 1044.24 P99.99: 1846.47 ------------------------------------------------------ [ 0, 1 ] 444202 22.155% 22.155% #### ( 1, 2 ] 1083299 54.030% 76.185% ########### ( 2, 3 ] 190015 9.477% 85.662% ## ( 3, 4 ] 90504 4.514% 90.176% # ( 4, 6 ] 73138 3.648% 93.824% # ( 6, 10 ] 16410 0.818% 94.642% ( 10, 15 ] 3163 0.158% 94.800% ( 15, 22 ] 380 0.019% 94.819% ( 22, 34 ] 60 0.003% 94.822% ( 34, 51 ] 37 0.002% 94.824% ( 51, 76 ] 45 0.002% 94.826% ( 76, 110 ] 13841 0.690% 95.516% ( 110, 170 ] 48297 2.409% 97.925% ( 170, 250 ] 19346 0.965% 98.890% ( 250, 380 ] 8306 0.414% 99.304% ( 380, 580 ] 7324 0.365% 99.670% ( 580, 870 ] 3873 0.193% 99.863% ( 870, 1300 ] 1841 0.092% 99.955% ( 1300, 1900 ] 779 0.039% 99.994% ( 1900, 2900 ] 588 0.029% 100.023% ( 2900, 4400 ] 604 0.030% 100.053% ( 4400, 6600 ] 128 0.006% 100.059% ( 6600, 9900 ] 15 0.001% 100.060% ** Level 1 read latency histogram (micros): Count: 367650 Average: 8.5104 StdDev: 110.68 Min: 1 Median: 2.0214 Max: 13160 Percentiles: P50: 2.02 P75: 3.07 P99: 15.76 P99.9: 725.48 P99.99: 1563.37 ------------------------------------------------------ [ 0, 1 ] 10336 2.811% 2.811% # ( 1, 2 ] 171538 46.658% 49.469% ######### ( 2, 3 ] 91143 24.791% 74.260% ##### ( 3, 4 ] 37341 10.157% 84.417% ## ( 4, 6 ] 41693 11.340% 95.757% ## ( 6, 10 ] 9525 2.591% 98.348% # ( 10, 15 ] 2346 0.638% 98.986% ( 15, 22 ] 474 0.129% 99.115% ( 22, 34 ] 54 0.015% 99.130% ( 34, 51 ] 51 0.014% 99.143% ( 51, 76 ] 34 0.009% 99.153% ( 76, 110 ] 412 0.112% 99.265% ( 110, 170 ] 1229 0.334% 99.599% ( 170, 250 ] 418 0.114% 99.713% ( 250, 380 ] 306 0.083% 99.796% ( 380, 580 ] 276 0.075% 99.871% ( 580, 870 ] 212 0.058% 99.929% ( 870, 1300 ] 173 0.047% 99.976% ( 1300, 1900 ] 119 0.032% 100.008% ( 1900, 2900 ] 111 0.030% 100.038% ( 2900, 4400 ] 145 0.039% 100.078% ( 4400, 6600 ] 28 0.008% 100.085% ( 6600, 9900 ] 4 0.001% 100.086% ( 9900, 14000 ] 2 0.001% 100.087% ** Level 2 read latency histogram (micros): Count: 69075 Average: 12.5648 StdDev: 127.13 Min: 1 Median: 1.9307 Max: 6627 Percentiles: P50: 1.93 P75: 3.10 P99: 197.44 P99.9: 1774.31 P99.99: 4076.21 ------------------------------------------------------ [ 0, 1 ] 3044 4.407% 4.407% # ( 1, 2 ] 33837 48.986% 53.393% ########## ( 2, 3 ] 14208 20.569% 73.962% #### ( 3, 4 ] 6897 9.985% 83.946% ## ( 4, 6 ] 7293 10.558% 94.505% ## ( 6, 10 ] 1836 2.658% 97.163% # ( 10, 15 ] 489 0.708% 97.870% ( 15, 22 ] 66 0.096% 97.966% ( 22, 34 ] 14 0.020% 97.986% ( 34, 51 ] 9 0.013% 97.999% ( 51, 76 ] 14 0.020% 98.020% ( 76, 110 ] 144 0.208% 98.228% ( 110, 170 ] 465 0.673% 98.901% ( 170, 250 ] 199 0.288% 99.189% ( 250, 380 ] 142 0.206% 99.395% ( 380, 580 ] 165 0.239% 99.634% ( 580, 870 ] 91 0.132% 99.765% ( 870, 1300 ] 70 0.101% 99.867% ( 1300, 1900 ] 29 0.042% 99.909% ( 1900, 2900 ] 31 0.045% 99.954% ( 2900, 4400 ] 32 0.046% 100.000% ( 4400, 6600 ] 10 0.014% 100.014% ( 6600, 9900 ] 1 0.001% 100.016% ** Level 3 read latency histogram (micros): Count: 799329 Average: 9.4100 StdDev: 122.73 Min: 1 Median: 1.8812 Max: 9080 Percentiles: P50: 1.88 P75: 2.96 P99: 13.95 P99.9: 841.71 P99.99: 1439.75 ------------------------------------------------------ [ 0, 1 ] 30609 3.829% 3.829% # ( 1, 2 ] 418795 52.393% 56.223% ########## ( 2, 3 ] 156780 19.614% 75.837% #### ( 3, 4 ] 79283 9.919% 85.755% ## ( 4, 6 ] 84702 10.597% 96.352% ## ( 6, 10 ] 17870 2.236% 98.588% ( 10, 15 ] 4169 0.522% 99.109% ( 15, 22 ] 463 0.058% 99.167% ( 22, 34 ] 56 0.007% 99.174% ( 34, 51 ] 66 0.008% 99.182% ( 51, 76 ] 99 0.012% 99.195% ( 76, 110 ] 676 0.085% 99.279% ( 110, 170 ] 2135 0.267% 99.546% ( 170, 250 ] 929 0.116% 99.663% ( 250, 380 ] 573 0.072% 99.734% ( 380, 580 ] 711 0.089% 99.823% ( 580, 870 ] 680 0.085% 99.908% ( 870, 1300 ] 579 0.072% 99.981% ( 1300, 1900 ] 318 0.040% 100.021% ( 1900, 2900 ] 275 0.034% 100.055% ( 2900, 4400 ] 328 0.041% 100.096% ( 4400, 6600 ] 108 0.014% 100.109% ( 6600, 9900 ] 22 0.003% 100.112% ** Level 4 read latency histogram (micros): Count: 902309 Average: 21.7822 StdDev: 179.89 Min: 1 Median: 1.9298 Max: 13664 Percentiles: P50: 1.93 P75: 3.22 P99: 348.63 P99.9: 1484.36 P99.99: 2122.35 ------------------------------------------------------ [ 0, 1 ] 30761 3.409% 3.409% # ( 1, 2 ] 452153 50.111% 53.520% ########## ( 2, 3 ] 174718 19.363% 72.883% #### ( 3, 4 ] 85727 9.501% 82.384% ## ( 4, 6 ] 92396 10.240% 92.624% ## ( 6, 10 ] 22497 2.493% 95.117% ( 10, 15 ] 5801 0.643% 95.760% ( 15, 22 ] 660 0.073% 95.833% ( 22, 34 ] 81 0.009% 95.842% ( 34, 51 ] 93 0.010% 95.853% ( 51, 76 ] 133 0.015% 95.867% ( 76, 110 ] 4630 0.513% 96.381% ( 110, 170 ] 15376 1.704% 98.085% ( 170, 250 ] 6018 0.667% 98.752% ( 250, 380 ] 2955 0.327% 99.079% ( 380, 580 ] 3124 0.346% 99.425% ( 580, 870 ] 2345 0.260% 99.685% ( 870, 1300 ] 1656 0.184% 99.869% ( 1300, 1900 ] 920 0.102% 99.971% ( 1900, 2900 ] 786 0.087% 100.058% ( 2900, 4400 ] 791 0.088% 100.145% ( 4400, 6600 ] 229 0.025% 100.171% ( 6600, 9900 ] 30 0.003% 100.174% ( 9900, 14000 ] 2 0.000% 100.174% ** Level 5 read latency histogram (micros): Count: 273142 Average: 281.6648 StdDev: 496.59 Min: 74 Median: 155.6023 Max: 11088 Percentiles: P50: 155.60 P75: 233.62 P99: 3004.90 P99.9: 6013.09 P99.99: 8525.61 ------------------------------------------------------ ( 51, 76 ] 16 0.006% 0.006% ( 76, 110 ] 38922 14.250% 14.256% ### ( 110, 170 ] 128458 47.030% 61.285% ######### ( 170, 250 ] 47104 17.245% 78.531% ### ( 250, 380 ] 20042 7.338% 85.868% # ( 380, 580 ] 17586 6.438% 92.307% # ( 580, 870 ] 9255 3.388% 95.695% # ( 870, 1300 ] 4702 1.721% 97.416% ( 1300, 1900 ] 2283 0.836% 98.252% ( 1900, 2900 ] 1896 0.694% 98.946% ( 2900, 4400 ] 2096 0.767% 99.714% ( 4400, 6600 ] 694 0.254% 99.968% ( 6600, 9900 ] 104 0.038% 100.006% ( 9900, 14000 ] 2 0.001% 100.007% ** DB Stats ** Uptime(secs): 28.9 total, 3.7 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 170.45 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:10.358 H:M:S, 35.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 266.36 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.277 H:M:S, 7.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 241.488 micros/op 16549 ops/sec 60.124 seconds 994996 operations; 16.0 MB/s (245999 of 245999 found) Microseconds per read: Count: 994996 Average: 241.4939 StdDev: 600.61 Min: 2 Median: 172.6391 Max: 72082 Percentiles: P50: 172.64 P75: 233.78 P99: 1214.07 P99.9: 7762.01 P99.99: 26125.03 ------------------------------------------------------ ( 1, 2 ] 5 0.001% 0.001% ( 2, 3 ] 38 0.004% 0.004% ( 3, 4 ] 55 0.006% 0.010% ( 4, 6 ] 87 0.009% 0.019% ( 6, 10 ] 515 0.052% 0.070% ( 10, 15 ] 4900 0.492% 0.563% ( 15, 22 ] 5599 0.563% 1.126% ( 22, 34 ] 8019 0.806% 1.931% ( 34, 51 ] 3854 0.387% 2.319% ( 51, 76 ] 569 0.057% 2.376% ( 76, 110 ] 22406 2.252% 4.628% ( 110, 170 ] 440714 44.293% 48.921% ######### ( 170, 250 ] 325479 32.712% 81.632% ####### ( 250, 380 ] 102148 10.266% 91.899% ## ( 380, 580 ] 45994 4.623% 96.521% # ( 580, 870 ] 18930 1.903% 98.424% ( 870, 1300 ] 7166 0.720% 99.144% ( 1300, 1900 ] 2744 0.276% 99.420% ( 1900, 2900 ] 1925 0.193% 99.613% ( 2900, 4400 ] 1868 0.188% 99.801% ( 4400, 6600 ] 861 0.087% 99.887% ( 6600, 9900 ] 355 0.036% 99.923% ( 9900, 14000 ] 261 0.026% 99.949% ( 14000, 22000 ] 355 0.036% 99.985% ( 22000, 33000 ] 132 0.013% 99.998% ( 33000, 50000 ] 12 0.001% 99.999% ( 50000, 75000 ] 5 0.001% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 19099521 rocksdb.bloom.filter.full.positive COUNT : 1180623 rocksdb.bloom.filter.full.true.positive COUNT : 994560 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 436 rocksdb.memtable.miss COUNT : 994560 rocksdb.l0.hit COUNT : 47522 rocksdb.l1.hit COUNT : 2587 rocksdb.l2andup.hit COUNT : 944451 rocksdb.compaction.key.drop.new COUNT : 138283 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 5857284 rocksdb.number.keys.read COUNT : 994996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 6044717088 rocksdb.bytes.read COUNT : 994996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1599 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 37652430 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 21231088805 rocksdb.compact.write.bytes COUNT : 21085562580 rocksdb.flush.write.bytes COUNT : 6011469797 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 40 rocksdb.number.superversion_acquires COUNT : 1049 rocksdb.number.superversion_releases COUNT : 1001 rocksdb.number.superversion_cleanups COUNT : 258 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 234 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 5962750976 rocksdb.memtable.garbage.bytes.at.flush COUNT : 2099200 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 25904695509 rocksdb.non.last.level.read.count COUNT : 6116025 rocksdb.block.checksum.compute.count COUNT : 6334253 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 170.967846 P95 : 513.170956 P99 : 1213.159860 P100 : 72081.000000 COUNT : 994996 SUM : 239425324 rocksdb.db.write.micros P50 : 1.533371 P95 : 4.683314 P99 : 10.075802 P100 : 635639.000000 COUNT : 5857284 SUM : 52604042 rocksdb.compaction.times.micros P50 : 1243750.000000 P95 : 4160000.000000 P99 : 16314139.000000 P100 : 16314139.000000 COUNT : 66 SUM : 125515845 rocksdb.compaction.times.cpu_micros P50 : 996000.000000 P95 : 3880000.000000 P99 : 7622000.000000 P100 : 7925830.000000 COUNT : 66 SUM : 83284166 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 788.800000 P95 : 20800.000000 P99 : 52621.000000 P100 : 52621.000000 COUNT : 86 SUM : 287712 rocksdb.compaction.outfile.sync.micros P50 : 6545.000000 P95 : 118333.333333 P99 : 172000.000000 P100 : 175000.000000 COUNT : 195 SUM : 3750995 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 156.935484 P95 : 3287.500000 P99 : 6094.000000 P100 : 8737.000000 COUNT : 169 SUM : 110570 rocksdb.table.open.io.micros P50 : 6300.299401 P95 : 17690.625000 P99 : 21688.125000 P100 : 25379.000000 COUNT : 1599 SUM : 11499504 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.746827 P95 : 14.816112 P99 : 212.803756 P100 : 20587.000000 COUNT : 5140637 SUM : 82636614 rocksdb.read.block.get.micros P50 : 138.734851 P95 : 382.098192 P99 : 1070.241042 P100 : 20406.000000 COUNT : 1191990 SUM : 205794557 rocksdb.write.raw.block.micros P50 : 0.530991 P95 : 1.216895 P99 : 3.492184 P100 : 14698.000000 COUNT : 6562421 SUM : 17268834 rocksdb.numfiles.in.singlecompaction P50 : 3.222222 P95 : 8.240000 P99 : 12.900000 P100 : 14.000000 COUNT : 84 SUM : 329 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.570446 P95 : 1201.611630 P99 : 2548.630003 P100 : 635431.000000 COUNT : 216952 SUM : 37675584 rocksdb.sst.read.micros P50 : 1.895348 P95 : 169.148949 P99 : 487.224594 P100 : 13664.000000 COUNT : 6116045 SUM : 269996803 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 994996 SUM : 994996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 5857284 SUM : 6044717088 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 220588.235294 P95 : 637666.666667 P99 : 939499.000000 P100 : 939499.000000 COUNT : 86 SUM : 22162998 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 263 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0