[2024-07-21 18:26:46.769220] Starting SPDK v24.09-pre git sha1 89fd17309 / DPDK 24.03.0 initialization... [2024-07-21 18:26:46.769556] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3511735 ] Set seed to 1721579206761885 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-21 18:26:46.911902] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-21 18:26:47.117417] app.c: 603:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-21 18:26:47.117469] app.c: 604:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3511735' to capture a snapshot of events at runtime. [2024-07-21 18:26:47.117482] app.c: 609:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-21 18:26:47.117495] app.c: 610:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-21 18:26:47.117505] app.c: 611:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3511735 for offline analysis/debug. [2024-07-21 18:26:47.117600] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sun Jul 21 18:26:50 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/07/21-18:26:56 ... thread 0: (1000000,1000000) ops and (159988.1,159988.1) ops/second in (6.250464,6.250464) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/6 1.49 GB 81.4 0.6 0.0 0.6 1.3 0.8 0.0 1.7 56.8 138.6 9.96 5.32 13 0.766 574K 3073 0.0 0.0 L1 6/6 458.81 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 13/13 1.54 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 24/7 1.31 GB 0.9 2.0 1.4 0.6 1.9 1.4 0.0 1.4 135.4 133.6 14.81 9.66 10 1.481 425K 0 0.0 0.0 L4 134/18 9.74 GB 0.9 1.1 0.2 0.9 0.9 0.1 0.8 4.1 124.3 105.5 9.00 5.31 4 2.250 232K 0 0.0 0.0 L5 124/0 9.72 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 312/50 24.25 GB 0.0 3.6 1.6 2.0 4.2 2.2 1.4 5.3 109.3 127.6 33.77 20.29 27 1.251 1233K 3073 0.0 0.0 Int 0/0 0.00 KB 0.0 3.6 1.6 2.0 4.2 2.2 1.4 5.3 109.3 127.6 33.77 20.29 27 1.251 1233K 3073 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.6 1.6 2.0 3.4 1.4 0.0 0.0 131.3 124.2 28.11 17.74 16 1.757 1233K 3073 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 144.4 5.66 2.54 11 0.515 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.3 total, 6.3 interval Flush(GB): cumulative 0.798, interval 0.798 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 4.21 GB write, 681.72 MB/s write, 3.60 GB read, 583.88 MB/s read, 33.8 seconds Interval compaction: 4.21 GB write, 689.21 MB/s write, 3.60 GB read, 590.30 MB/s read, 33.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 6, interval: 6 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 283350 Average: 29.4673 StdDev: 469.07 Min: 1 Median: 2.1101 Max: 59019 Percentiles: P50: 2.11 P75: 2.99 P99: 639.64 P99.9: 3404.11 P99.99: 4870.04 ------------------------------------------------------ [ 0, 1 ] 6264 2.211% 2.211% ( 1, 2 ] 126530 44.655% 46.866% ######### ( 2, 3 ] 80667 28.469% 75.335% ###### ( 3, 4 ] 26639 9.401% 84.736% ## ( 4, 6 ] 27547 9.722% 94.458% ## ( 6, 10 ] 7508 2.650% 97.108% # ( 10, 15 ] 1840 0.649% 97.757% ( 15, 22 ] 225 0.079% 97.837% ( 22, 34 ] 23 0.008% 97.845% ( 34, 51 ] 63 0.022% 97.867% ( 51, 76 ] 37 0.013% 97.880% ( 76, 110 ] 7 0.002% 97.882% ( 110, 170 ] 48 0.017% 97.899% ( 170, 250 ] 264 0.093% 97.993% ( 250, 380 ] 880 0.311% 98.303% ( 380, 580 ] 1672 0.590% 98.893% ( 580, 870 ] 1471 0.519% 99.412% ( 870, 1300 ] 723 0.255% 99.668% ( 1300, 1900 ] 257 0.091% 99.758% ( 1900, 2900 ] 281 0.099% 99.857% ( 2900, 4400 ] 359 0.127% 99.984% ( 4400, 6600 ] 78 0.028% 100.012% ( 6600, 9900 ] 20 0.007% 100.019% ( 9900, 14000 ] 13 0.005% 100.023% ( 14000, 22000 ] 22 0.008% 100.031% ( 22000, 33000 ] 16 0.006% 100.037% ( 33000, 50000 ] 16 0.006% 100.042% ( 50000, 75000 ] 1 0.000% 100.043% ** Level 1 read latency histogram (micros): Count: 465 Average: 1106.3484 StdDev: 1813.75 Min: 121 Median: 563.6879 Max: 15987 Percentiles: P50: 563.69 P75: 962.45 P99: 11745.00 P99.9: 15987.00 P99.99: 15987.00 ------------------------------------------------------ ( 110, 170 ] 11 2.366% 2.366% ( 170, 250 ] 18 3.871% 6.237% # ( 250, 380 ] 74 15.914% 22.151% ### ( 380, 580 ] 141 30.323% 52.473% ###### ( 580, 870 ] 94 20.215% 72.688% #### ( 870, 1300 ] 50 10.753% 83.441% ## ( 1300, 1900 ] 17 3.656% 87.097% # ( 1900, 2900 ] 18 3.871% 90.968% # ( 2900, 4400 ] 27 5.806% 96.774% # ( 4400, 6600 ] 7 1.505% 98.280% ( 6600, 9900 ] 2 0.430% 98.710% ( 9900, 14000 ] 3 0.645% 99.355% ( 14000, 22000 ] 3 0.645% 100.000% ** Level 2 read latency histogram (micros): Count: 580053 Average: 13.3519 StdDev: 255.13 Min: 1 Median: 2.2264 Max: 49966 Percentiles: P50: 2.23 P75: 3.17 P99: 13.45 P99.9: 811.11 P99.99: 1251.28 ------------------------------------------------------ [ 0, 1 ] 5179 0.893% 0.893% ( 1, 2 ] 245562 42.334% 43.227% ######## ( 2, 3 ] 173517 29.914% 73.141% ###### ( 3, 4 ] 63545 10.955% 84.096% ## ( 4, 6 ] 66043 11.386% 95.482% ## ( 6, 10 ] 17183 2.962% 98.444% # ( 10, 15 ] 4672 0.805% 99.250% ( 15, 22 ] 681 0.117% 99.367% ( 22, 34 ] 91 0.016% 99.383% ( 34, 51 ] 158 0.027% 99.410% ( 51, 76 ] 115 0.020% 99.430% ( 76, 110 ] 40 0.007% 99.437% ( 110, 170 ] 72 0.012% 99.449% ( 170, 250 ] 234 0.040% 99.490% ( 250, 380 ] 667 0.115% 99.605% ( 380, 580 ] 1078 0.186% 99.790% ( 580, 870 ] 798 0.138% 99.928% ( 870, 1300 ] 406 0.070% 99.998% ( 1300, 1900 ] 286 0.049% 100.047% ( 1900, 2900 ] 294 0.051% 100.098% ( 2900, 4400 ] 348 0.060% 100.158% ( 4400, 6600 ] 97 0.017% 100.175% ( 6600, 9900 ] 26 0.004% 100.179% ( 9900, 14000 ] 21 0.004% 100.183% ( 14000, 22000 ] 12 0.002% 100.185% ( 22000, 33000 ] 15 0.003% 100.187% ( 33000, 50000 ] 5 0.001% 100.188% ** Level 3 read latency histogram (micros): Count: 237903 Average: 22.2612 StdDev: 406.71 Min: 1 Median: 2.2465 Max: 51128 Percentiles: P50: 2.25 P75: 3.19 P99: 409.88 P99.9: 2371.11 P99.99: 3641.69 ------------------------------------------------------ [ 0, 1 ] 2309 0.971% 0.971% ( 1, 2 ] 98841 41.547% 42.517% ######## ( 2, 3 ] 72216 30.355% 72.873% ###### ( 3, 4 ] 26279 11.046% 83.919% ## ( 4, 6 ] 26433 11.111% 95.029% ## ( 6, 10 ] 6526 2.743% 97.773% # ( 10, 15 ] 1577 0.663% 98.435% ( 15, 22 ] 248 0.104% 98.540% ( 22, 34 ] 46 0.019% 98.559% ( 34, 51 ] 79 0.033% 98.592% ( 51, 76 ] 62 0.026% 98.618% ( 76, 110 ] 21 0.009% 98.627% ( 110, 170 ] 35 0.015% 98.642% ( 170, 250 ] 211 0.089% 98.731% ( 250, 380 ] 504 0.212% 98.942% ( 380, 580 ] 937 0.394% 99.336% ( 580, 870 ] 784 0.330% 99.666% ( 870, 1300 ] 342 0.144% 99.810% ( 1300, 1900 ] 130 0.055% 99.864% ( 1900, 2900 ] 187 0.079% 99.943% ( 2900, 4400 ] 233 0.098% 100.041% ( 4400, 6600 ] 49 0.021% 100.061% ( 6600, 9900 ] 20 0.008% 100.070% ( 9900, 14000 ] 7 0.003% 100.073% ( 14000, 22000 ] 13 0.005% 100.078% ( 22000, 33000 ] 14 0.006% 100.084% ( 33000, 50000 ] 10 0.004% 100.088% ( 50000, 75000 ] 1 0.000% 100.089% ** Level 4 read latency histogram (micros): Count: 398051 Average: 38.0889 StdDev: 514.70 Min: 1 Median: 2.3432 Max: 59001 Percentiles: P50: 2.34 P75: 3.46 P99: 662.71 P99.9: 2645.82 P99.99: 3347.12 ------------------------------------------------------ [ 0, 1 ] 3902 0.980% 0.980% ( 1, 2 ] 154161 38.729% 39.709% ######## ( 2, 3 ] 119347 29.983% 69.692% ###### ( 3, 4 ] 46104 11.582% 81.275% ## ( 4, 6 ] 48726 12.241% 93.516% ## ( 6, 10 ] 12326 3.097% 96.612% # ( 10, 15 ] 3220 0.809% 97.421% ( 15, 22 ] 441 0.111% 97.532% ( 22, 34 ] 63 0.016% 97.548% ( 34, 51 ] 104 0.026% 97.574% ( 51, 76 ] 142 0.036% 97.610% ( 76, 110 ] 31 0.008% 97.617% ( 110, 170 ] 104 0.026% 97.644% ( 170, 250 ] 470 0.118% 97.762% ( 250, 380 ] 1536 0.386% 98.147% ( 380, 580 ] 2713 0.682% 98.829% ( 580, 870 ] 2386 0.599% 99.428% ( 870, 1300 ] 1016 0.255% 99.684% ( 1300, 1900 ] 450 0.113% 99.797% ( 1900, 2900 ] 551 0.138% 99.935% ( 2900, 4400 ] 732 0.184% 100.119% ( 4400, 6600 ] 177 0.044% 100.164% ( 6600, 9900 ] 66 0.017% 100.180% ( 9900, 14000 ] 26 0.007% 100.187% ( 14000, 22000 ] 55 0.014% 100.200% ( 22000, 33000 ] 53 0.013% 100.214% ( 33000, 50000 ] 14 0.004% 100.217% ( 50000, 75000 ] 2 0.001% 100.218% ** Level 5 read latency histogram (micros): Count: 7362 Average: 1048.4027 StdDev: 2025.08 Min: 108 Median: 578.9200 Max: 50028 Percentiles: P50: 578.92 P75: 944.25 P99: 6237.72 P99.9: 34549.43 P99.99: 50028.00 ------------------------------------------------------ ( 76, 110 ] 2 0.027% 0.027% ( 110, 170 ] 64 0.869% 0.896% ( 170, 250 ] 404 5.488% 6.384% # ( 250, 380 ] 1185 16.096% 22.480% ### ( 380, 580 ] 2037 27.669% 50.149% ###### ( 580, 870 ] 1700 23.092% 73.241% ##### ( 870, 1300 ] 750 10.187% 83.428% ## ( 1300, 1900 ] 307 4.170% 87.598% # ( 1900, 2900 ] 322 4.374% 91.972% # ( 2900, 4400 ] 428 5.814% 97.786% # ( 4400, 6600 ] 107 1.453% 99.239% ( 6600, 9900 ] 25 0.340% 99.579% ( 9900, 14000 ] 2 0.027% 99.606% ( 14000, 22000 ] 10 0.136% 99.742% ( 22000, 33000 ] 11 0.149% 99.891% ( 33000, 50000 ] 7 0.095% 99.986% ( 50000, 75000 ] 1 0.014% 100.000% ** DB Stats ** Uptime(secs): 6.3 total, 6.3 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 155.71 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.396 H:M:S, 22.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 157.40 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.396 H:M:S, 22.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 8 num-running-flushes: 1 2024/07/21-18:27:03 ... thread 0: (1000000,2000000) ops and (162373.7,161172.1) ops/second in (6.158632,12.409096) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/14 2.53 GB 125.1 1.1 0.0 1.1 2.9 1.8 0.0 1.6 63.2 168.1 17.92 10.67 29 0.618 1149K 6277 0.0 0.0 L1 6/6 458.81 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 15/0 886.02 MB 0.9 3.6 2.9 0.7 3.6 2.9 0.0 1.2 130.2 128.5 28.37 18.33 12 2.364 1394K 0 0.0 0.0 L4 147/0 9.76 GB 1.0 5.7 2.2 3.5 5.0 1.6 0.8 2.3 131.0 116.3 44.19 28.75 19 2.326 2430K 2333 0.0 0.0 L5 144/0 11.20 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 331/20 24.80 GB 0.0 10.4 5.1 5.3 11.5 6.3 2.9 6.3 117.3 130.4 90.48 57.75 60 1.508 4974K 8610 0.0 0.0 Int 0/0 0.00 KB 0.0 10.4 5.1 5.3 11.5 6.3 2.9 6.3 117.3 130.4 90.48 57.75 60 1.508 4974K 8610 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.4 5.1 5.3 9.7 4.4 0.0 0.0 133.4 124.6 79.57 51.99 35 2.273 4974K 8610 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 173.0 10.90 5.76 25 0.436 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.5 total, 12.4 interval Flush(GB): cumulative 1.842, interval 1.842 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.52 GB write, 945.58 MB/s write, 10.37 GB read, 850.59 MB/s read, 90.5 seconds Interval compaction: 11.52 GB write, 950.82 MB/s write, 10.37 GB read, 855.30 MB/s read, 90.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 8, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 8, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 17, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 8, total-stops: 17, interval: 25 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 519937 Average: 46.6201 StdDev: 505.11 Min: 1 Median: 1.8653 Max: 75071 Percentiles: P50: 1.87 P75: 2.89 P99: 771.79 P99.9: 3355.03 P99.99: 4272.56 ------------------------------------------------------ [ 0, 1 ] 38448 7.395% 7.395% # ( 1, 2 ] 256012 49.239% 56.634% ########## ( 2, 3 ] 107349 20.647% 77.280% #### ( 3, 4 ] 34454 6.627% 83.907% # ( 4, 6 ] 32350 6.222% 90.129% # ( 6, 10 ] 10092 1.941% 92.070% ( 10, 15 ] 2067 0.398% 92.467% ( 15, 22 ] 276 0.053% 92.520% ( 22, 34 ] 30 0.006% 92.526% ( 34, 51 ] 71 0.014% 92.540% ( 51, 76 ] 38 0.007% 92.547% ( 76, 110 ] 103 0.020% 92.567% ( 110, 170 ] 3688 0.709% 93.276% ( 170, 250 ] 12801 2.462% 95.738% ( 250, 380 ] 7736 1.488% 97.226% ( 380, 580 ] 6328 1.217% 98.443% ( 580, 870 ] 4380 0.842% 99.286% ( 870, 1300 ] 1763 0.339% 99.625% ( 1300, 1900 ] 580 0.112% 99.736% ( 1900, 2900 ] 621 0.119% 99.856% ( 2900, 4400 ] 765 0.147% 100.003% ( 4400, 6600 ] 154 0.030% 100.033% ( 6600, 9900 ] 39 0.008% 100.040% ( 9900, 14000 ] 17 0.003% 100.043% ( 14000, 22000 ] 39 0.008% 100.051% ( 22000, 33000 ] 39 0.008% 100.058% ( 33000, 50000 ] 26 0.005% 100.063% ( 50000, 75000 ] 4 0.001% 100.064% ( 75000, 110000 ] 1 0.000% 100.064% ** Level 1 read latency histogram (micros): Count: 1491 Average: 690.9457 StdDev: 2039.49 Min: 108 Median: 329.0794 Max: 46306 Percentiles: P50: 329.08 P75: 578.68 P99: 5888.67 P99.9: 37326.50 P99.99: 46306.00 ------------------------------------------------------ ( 76, 110 ] 5 0.335% 0.335% ( 110, 170 ] 123 8.249% 8.585% ## ( 170, 250 ] 449 30.114% 38.699% ###### ( 250, 380 ] 277 18.578% 57.277% #### ( 380, 580 ] 266 17.840% 75.117% #### ( 580, 870 ] 185 12.408% 87.525% ## ( 870, 1300 ] 76 5.097% 92.622% # ( 1300, 1900 ] 28 1.878% 94.500% ( 1900, 2900 ] 23 1.543% 96.043% ( 2900, 4400 ] 38 2.549% 98.592% # ( 4400, 6600 ] 9 0.604% 99.195% ( 6600, 9900 ] 3 0.201% 99.396% ( 9900, 14000 ] 3 0.201% 99.598% ( 14000, 22000 ] 3 0.201% 99.799% ( 22000, 33000 ] 1 0.067% 99.866% ( 33000, 50000 ] 2 0.134% 100.000% ** Level 2 read latency histogram (micros): Count: 628361 Average: 12.9920 StdDev: 249.35 Min: 1 Median: 2.2332 Max: 49966 Percentiles: P50: 2.23 P75: 3.18 P99: 13.33 P99.9: 810.64 P99.99: 1274.39 ------------------------------------------------------ [ 0, 1 ] 5388 0.857% 0.857% ( 1, 2 ] 264820 42.145% 43.002% ######## ( 2, 3 ] 188569 30.010% 73.012% ###### ( 3, 4 ] 69042 10.988% 83.999% ## ( 4, 6 ] 72328 11.511% 95.510% ## ( 6, 10 ] 18667 2.971% 98.481% # ( 10, 15 ] 4904 0.780% 99.261% ( 15, 22 ] 707 0.113% 99.374% ( 22, 34 ] 100 0.016% 99.390% ( 34, 51 ] 164 0.026% 99.416% ( 51, 76 ] 122 0.019% 99.435% ( 76, 110 ] 42 0.007% 99.442% ( 110, 170 ] 76 0.012% 99.454% ( 170, 250 ] 256 0.041% 99.495% ( 250, 380 ] 731 0.116% 99.611% ( 380, 580 ] 1143 0.182% 99.793% ( 580, 870 ] 847 0.135% 99.928% ( 870, 1300 ] 417 0.066% 99.994% ( 1300, 1900 ] 298 0.047% 100.041% ( 1900, 2900 ] 306 0.049% 100.090% ( 2900, 4400 ] 358 0.057% 100.147% ( 4400, 6600 ] 102 0.016% 100.163% ( 6600, 9900 ] 27 0.004% 100.168% ( 9900, 14000 ] 23 0.004% 100.171% ( 14000, 22000 ] 12 0.002% 100.173% ( 22000, 33000 ] 16 0.003% 100.176% ( 33000, 50000 ] 5 0.001% 100.176% ** Level 3 read latency histogram (micros): Count: 722104 Average: 18.4037 StdDev: 363.08 Min: 1 Median: 2.2355 Max: 51128 Percentiles: P50: 2.24 P75: 3.15 P99: 203.24 P99.9: 1090.24 P99.99: 1837.42 ------------------------------------------------------ [ 0, 1 ] 6618 0.916% 0.916% ( 1, 2 ] 302440 41.883% 42.800% ######## ( 2, 3 ] 220768 30.573% 73.373% ###### ( 3, 4 ] 78231 10.834% 84.206% ## ( 4, 6 ] 79459 11.004% 95.210% ## ( 6, 10 ] 20630 2.857% 98.067% # ( 10, 15 ] 4432 0.614% 98.681% ( 15, 22 ] 596 0.083% 98.763% ( 22, 34 ] 115 0.016% 98.779% ( 34, 51 ] 204 0.028% 98.808% ( 51, 76 ] 146 0.020% 98.828% ( 76, 110 ] 55 0.008% 98.835% ( 110, 170 ] 500 0.069% 98.905% ( 170, 250 ] 1658 0.230% 99.134% ( 250, 380 ] 1545 0.214% 99.348% ( 380, 580 ] 1966 0.272% 99.620% ( 580, 870 ] 1645 0.228% 99.848% ( 870, 1300 ] 730 0.101% 99.949% ( 1300, 1900 ] 328 0.045% 99.995% ( 1900, 2900 ] 394 0.055% 100.049% ( 2900, 4400 ] 454 0.063% 100.112% ( 4400, 6600 ] 99 0.014% 100.126% ( 6600, 9900 ] 68 0.009% 100.135% ( 9900, 14000 ] 35 0.005% 100.140% ( 14000, 22000 ] 62 0.009% 100.149% ( 22000, 33000 ] 41 0.006% 100.154% ( 33000, 50000 ] 16 0.002% 100.157% ( 50000, 75000 ] 1 0.000% 100.157% ** Level 4 read latency histogram (micros): Count: 926284 Average: 31.0713 StdDev: 432.02 Min: 0 Median: 2.2614 Max: 60024 Percentiles: P50: 2.26 P75: 3.34 P99: 516.27 P99.9: 1823.36 P99.99: 2703.45 ------------------------------------------------------ [ 0, 1 ] 11418 1.233% 1.233% ( 1, 2 ] 381950 41.235% 42.467% ######## ( 2, 3 ] 266968 28.821% 71.289% ###### ( 3, 4 ] 100299 10.828% 82.117% ## ( 4, 6 ] 102432 11.058% 93.175% ## ( 6, 10 ] 26221 2.831% 96.006% # ( 10, 15 ] 6044 0.652% 96.658% ( 15, 22 ] 829 0.089% 96.748% ( 22, 34 ] 139 0.015% 96.763% ( 34, 51 ] 232 0.025% 96.788% ( 51, 76 ] 266 0.029% 96.817% ( 76, 110 ] 116 0.013% 96.829% ( 110, 170 ] 2424 0.262% 97.091% ( 170, 250 ] 8347 0.901% 97.992% ( 250, 380 ] 5603 0.605% 98.597% ( 380, 580 ] 5479 0.592% 99.188% ( 580, 870 ] 4210 0.455% 99.643% ( 870, 1300 ] 1730 0.187% 99.830% ( 1300, 1900 ] 746 0.081% 99.910% ( 1900, 2900 ] 919 0.099% 100.010% ( 2900, 4400 ] 1083 0.117% 100.126% ( 4400, 6600 ] 286 0.031% 100.157% ( 6600, 9900 ] 104 0.011% 100.169% ( 9900, 14000 ] 51 0.006% 100.174% ( 14000, 22000 ] 96 0.010% 100.184% ( 22000, 33000 ] 81 0.009% 100.193% ( 33000, 50000 ] 21 0.002% 100.195% ( 50000, 75000 ] 4 0.000% 100.196% ** Level 5 read latency histogram (micros): Count: 22780 Average: 647.7499 StdDev: 1620.94 Min: 96 Median: 336.5434 Max: 55406 Percentiles: P50: 336.54 P75: 603.26 P99: 4569.93 P99.9: 30110.00 P99.99: 49684.93 ------------------------------------------------------ ( 76, 110 ] 58 0.255% 0.255% ( 110, 170 ] 1992 8.745% 8.999% ## ( 170, 250 ] 6532 28.674% 37.673% ###### ( 250, 380 ] 4218 18.516% 56.190% #### ( 380, 580 ] 4051 17.783% 73.973% #### ( 580, 870 ] 2918 12.809% 86.782% ### ( 870, 1300 ] 1234 5.417% 92.199% # ( 1300, 1900 ] 460 2.019% 94.219% ( 1900, 2900 ] 480 2.107% 96.326% ( 2900, 4400 ] 598 2.625% 98.951% # ( 4400, 6600 ] 145 0.637% 99.587% ( 6600, 9900 ] 34 0.149% 99.737% ( 9900, 14000 ] 7 0.031% 99.767% ( 14000, 22000 ] 14 0.061% 99.829% ( 22000, 33000 ] 22 0.097% 99.925% ( 33000, 50000 ] 15 0.066% 99.991% ( 50000, 75000 ] 2 0.009% 100.000% ** DB Stats ** Uptime(secs): 12.5 total, 6.2 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 157.73 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.337 H:M:S, 26.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 159.81 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.941 H:M:S, 31.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/07/21-18:27:08 ... thread 0: (1000000,3000000) ops and (172564.5,164798.7) ops/second in (5.794936,18.204032) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 20/13 3.51 GB 143.8 2.5 0.0 2.5 5.3 2.8 0.0 1.9 80.6 172.8 31.23 19.04 50 0.625 2554K 13K 0.0 0.0 L1 6/6 458.81 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 15/0 886.02 MB 0.9 3.6 2.9 0.7 3.6 2.9 0.0 1.2 130.2 128.5 28.37 18.33 12 2.364 1394K 0 0.0 0.0 L4 147/0 9.76 GB 1.0 5.7 2.2 3.5 5.0 1.6 0.8 2.3 131.0 116.3 44.19 28.75 19 2.326 2430K 2333 0.0 0.0 L5 144/0 11.20 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 332/19 25.78 GB 0.0 11.7 5.1 6.6 13.9 7.2 2.9 4.9 115.6 136.7 103.79 66.12 81 1.281 6379K 16K 0.0 0.0 Int 0/0 0.00 KB 0.0 11.7 5.1 6.6 13.9 7.2 2.9 4.9 115.6 136.7 103.79 66.12 81 1.281 6379K 16K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.7 5.1 6.6 11.0 4.4 0.0 0.0 133.7 125.8 89.76 57.60 40 2.244 6379K 16K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 206.3 14.03 8.53 41 0.342 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 18.3 total, 18.2 interval Flush(GB): cumulative 2.825, interval 2.825 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.85 GB write, 776.09 MB/s write, 11.72 GB read, 656.56 MB/s read, 103.8 seconds Interval compaction: 13.85 GB write, 779.02 MB/s write, 11.72 GB read, 659.04 MB/s read, 103.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 14, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 14, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 17, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 14, total-stops: 17, interval: 31 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 867414 Average: 38.0976 StdDev: 393.54 Min: 1 Median: 1.7450 Max: 75071 Percentiles: P50: 1.75 P75: 2.73 P99: 561.50 P99.9: 2332.51 P99.99: 3739.72 ------------------------------------------------------ [ 0, 1 ] 100685 11.607% 11.607% ## ( 1, 2 ] 447005 51.533% 63.141% ########## ( 2, 3 ] 140613 16.211% 79.351% ### ( 3, 4 ] 45254 5.217% 84.568% # ( 4, 6 ] 39199 4.519% 89.087% # ( 6, 10 ] 11979 1.381% 90.468% ( 10, 15 ] 2494 0.288% 90.756% ( 15, 22 ] 353 0.041% 90.797% ( 22, 34 ] 68 0.008% 90.804% ( 34, 51 ] 74 0.009% 90.813% ( 51, 76 ] 38 0.004% 90.817% ( 76, 110 ] 697 0.080% 90.898% ( 110, 170 ] 16160 1.863% 92.761% ( 170, 250 ] 36313 4.186% 96.947% # ( 250, 380 ] 11224 1.294% 98.241% ( 380, 580 ] 7257 0.837% 99.078% ( 580, 870 ] 4504 0.519% 99.597% ( 870, 1300 ] 1777 0.205% 99.802% ( 1300, 1900 ] 586 0.068% 99.869% ( 1900, 2900 ] 621 0.072% 99.941% ( 2900, 4400 ] 765 0.088% 100.029% ( 4400, 6600 ] 154 0.018% 100.047% ( 6600, 9900 ] 39 0.004% 100.051% ( 9900, 14000 ] 17 0.002% 100.053% ( 14000, 22000 ] 39 0.004% 100.058% ( 22000, 33000 ] 39 0.004% 100.062% ( 33000, 50000 ] 26 0.003% 100.065% ( 50000, 75000 ] 4 0.000% 100.066% ( 75000, 110000 ] 1 0.000% 100.066% ** Level 1 read latency histogram (micros): Count: 55036 Average: 29.6079 StdDev: 355.50 Min: 1 Median: 2.8781 Max: 46306 Percentiles: P50: 2.88 P75: 4.53 P99: 466.32 P99.9: 3017.00 P99.99: 13311.75 ------------------------------------------------------ [ 0, 1 ] 1428 2.595% 2.595% # ( 1, 2 ] 14498 26.343% 28.937% ##### ( 2, 3 ] 13201 23.986% 52.924% ##### ( 3, 4 ] 9413 17.103% 70.027% ### ( 4, 6 ] 10260 18.642% 88.669% #### ( 6, 10 ] 1713 3.113% 91.782% # ( 10, 15 ] 511 0.928% 92.710% ( 15, 22 ] 380 0.690% 93.401% ( 22, 34 ] 56 0.102% 93.502% ( 34, 51 ] 8 0.015% 93.517% ( 76, 110 ] 37 0.067% 93.584% ( 110, 170 ] 739 1.343% 94.927% ( 170, 250 ] 1665 3.025% 97.952% # ( 250, 380 ] 445 0.809% 98.761% ( 380, 580 ] 305 0.554% 99.315% ( 580, 870 ] 192 0.349% 99.664% ( 870, 1300 ] 76 0.138% 99.802% ( 1300, 1900 ] 28 0.051% 99.853% ( 1900, 2900 ] 23 0.042% 99.895% ( 2900, 4400 ] 38 0.069% 99.964% ( 4400, 6600 ] 9 0.016% 99.980% ( 6600, 9900 ] 3 0.005% 99.985% ( 9900, 14000 ] 3 0.005% 99.991% ( 14000, 22000 ] 3 0.005% 99.996% ( 22000, 33000 ] 1 0.002% 99.998% ( 33000, 50000 ] 2 0.004% 100.002% ** Level 2 read latency histogram (micros): Count: 628361 Average: 12.9920 StdDev: 249.35 Min: 1 Median: 2.2332 Max: 49966 Percentiles: P50: 2.23 P75: 3.18 P99: 13.33 P99.9: 810.64 P99.99: 1274.39 ------------------------------------------------------ [ 0, 1 ] 5388 0.857% 0.857% ( 1, 2 ] 264820 42.145% 43.002% ######## ( 2, 3 ] 188569 30.010% 73.012% ###### ( 3, 4 ] 69042 10.988% 83.999% ## ( 4, 6 ] 72328 11.511% 95.510% ## ( 6, 10 ] 18667 2.971% 98.481% # ( 10, 15 ] 4904 0.780% 99.261% ( 15, 22 ] 707 0.113% 99.374% ( 22, 34 ] 100 0.016% 99.390% ( 34, 51 ] 164 0.026% 99.416% ( 51, 76 ] 122 0.019% 99.435% ( 76, 110 ] 42 0.007% 99.442% ( 110, 170 ] 76 0.012% 99.454% ( 170, 250 ] 256 0.041% 99.495% ( 250, 380 ] 731 0.116% 99.611% ( 380, 580 ] 1143 0.182% 99.793% ( 580, 870 ] 847 0.135% 99.928% ( 870, 1300 ] 417 0.066% 99.994% ( 1300, 1900 ] 298 0.047% 100.041% ( 1900, 2900 ] 306 0.049% 100.090% ( 2900, 4400 ] 358 0.057% 100.147% ( 4400, 6600 ] 102 0.016% 100.163% ( 6600, 9900 ] 27 0.004% 100.168% ( 9900, 14000 ] 23 0.004% 100.171% ( 14000, 22000 ] 12 0.002% 100.173% ( 22000, 33000 ] 16 0.003% 100.176% ( 33000, 50000 ] 5 0.001% 100.176% ** Level 3 read latency histogram (micros): Count: 728421 Average: 19.9456 StdDev: 361.93 Min: 1 Median: 2.2498 Max: 51128 Percentiles: P50: 2.25 P75: 3.21 P99: 243.69 P99.9: 1086.81 P99.99: 1836.26 ------------------------------------------------------ [ 0, 1 ] 6618 0.909% 0.909% ( 1, 2 ] 302440 41.520% 42.428% ######## ( 2, 3 ] 220768 30.308% 72.736% ###### ( 3, 4 ] 78231 10.740% 83.476% ## ( 4, 6 ] 79459 10.908% 94.384% ## ( 6, 10 ] 20630 2.832% 97.217% # ( 10, 15 ] 4432 0.608% 97.825% ( 15, 22 ] 596 0.082% 97.907% ( 22, 34 ] 115 0.016% 97.923% ( 34, 51 ] 204 0.028% 97.951% ( 51, 76 ] 146 0.020% 97.971% ( 76, 110 ] 136 0.019% 97.989% ( 110, 170 ] 2475 0.340% 98.329% ( 170, 250 ] 5305 0.728% 99.057% ( 250, 380 ] 2007 0.276% 99.333% ( 380, 580 ] 2093 0.287% 99.620% ( 580, 870 ] 1669 0.229% 99.849% ( 870, 1300 ] 731 0.100% 99.950% ( 1300, 1900 ] 328 0.045% 99.995% ( 1900, 2900 ] 394 0.054% 100.049% ( 2900, 4400 ] 454 0.062% 100.111% ( 4400, 6600 ] 99 0.014% 100.125% ( 6600, 9900 ] 68 0.009% 100.134% ( 9900, 14000 ] 35 0.005% 100.139% ( 14000, 22000 ] 62 0.009% 100.147% ( 22000, 33000 ] 41 0.006% 100.153% ( 33000, 50000 ] 16 0.002% 100.155% ( 50000, 75000 ] 1 0.000% 100.155% ** Level 4 read latency histogram (micros): Count: 967760 Average: 37.9669 StdDev: 424.16 Min: 0 Median: 2.3384 Max: 60024 Percentiles: P50: 2.34 P75: 3.65 P99: 515.64 P99.9: 1789.50 P99.99: 2697.85 ------------------------------------------------------ [ 0, 1 ] 11418 1.180% 1.180% ( 1, 2 ] 382023 39.475% 40.655% ######## ( 2, 3 ] 267270 27.617% 68.272% ###### ( 3, 4 ] 100472 10.382% 78.654% ## ( 4, 6 ] 102704 10.613% 89.267% ## ( 6, 10 ] 26242 2.712% 91.978% # ( 10, 15 ] 6059 0.626% 92.604% ( 15, 22 ] 830 0.086% 92.690% ( 22, 34 ] 142 0.015% 92.705% ( 34, 51 ] 232 0.024% 92.729% ( 51, 76 ] 266 0.027% 92.756% ( 76, 110 ] 755 0.078% 92.834% ( 110, 170 ] 15344 1.586% 94.420% ( 170, 250 ] 31309 3.235% 97.655% # ( 250, 380 ] 8715 0.901% 98.556% ( 380, 580 ] 6344 0.656% 99.211% ( 580, 870 ] 4314 0.446% 99.657% ( 870, 1300 ] 1744 0.180% 99.837% ( 1300, 1900 ] 748 0.077% 99.914% ( 1900, 2900 ] 919 0.095% 100.009% ( 2900, 4400 ] 1083 0.112% 100.121% ( 4400, 6600 ] 286 0.030% 100.151% ( 6600, 9900 ] 104 0.011% 100.162% ( 9900, 14000 ] 51 0.005% 100.167% ( 14000, 22000 ] 96 0.010% 100.177% ( 22000, 33000 ] 81 0.008% 100.185% ( 33000, 50000 ] 21 0.002% 100.187% ( 50000, 75000 ] 4 0.000% 100.188% ** Level 5 read latency histogram (micros): Count: 53653 Average: 387.6534 StdDev: 1080.78 Min: 91 Median: 217.9983 Max: 55406 Percentiles: P50: 218.00 P75: 326.60 P99: 3651.18 P99.9: 13031.81 P99.99: 45052.66 ------------------------------------------------------ ( 76, 110 ] 554 1.033% 1.033% ( 110, 170 ] 11981 22.331% 23.363% #### ( 170, 250 ] 23820 44.396% 67.759% ######### ( 250, 380 ] 6593 12.288% 80.048% ## ( 380, 580 ] 4685 8.732% 88.780% ## ( 580, 870 ] 2994 5.580% 94.360% # ( 870, 1300 ] 1247 2.324% 96.684% ( 1300, 1900 ] 462 0.861% 97.545% ( 1900, 2900 ] 481 0.897% 98.442% ( 2900, 4400 ] 598 1.115% 99.556% ( 4400, 6600 ] 145 0.270% 99.827% ( 6600, 9900 ] 34 0.063% 99.890% ( 9900, 14000 ] 7 0.013% 99.903% ( 14000, 22000 ] 14 0.026% 99.929% ( 22000, 33000 ] 22 0.041% 99.970% ( 33000, 50000 ] 15 0.028% 99.998% ( 50000, 75000 ] 2 0.004% 100.002% ** DB Stats ** Uptime(secs): 18.3 total, 5.8 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 161.56 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.289 H:M:S, 28.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 169.80 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.953 H:M:S, 33.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 423.418 micros/op 9409 ops/sec 20.193 seconds 189996 operations; 9.1 MB/s (47999 of 47999 found) Microseconds per read: Count: 189996 Average: 423.4312 StdDev: 1334.13 Min: 2 Median: 240.1387 Max: 93043 Percentiles: P50: 240.14 P75: 376.75 P99: 3661.48 P99.9: 20745.41 P99.99: 52941.76 ------------------------------------------------------ ( 1, 2 ] 13 0.007% 0.007% ( 2, 3 ] 54 0.028% 0.035% ( 3, 4 ] 75 0.039% 0.075% ( 4, 6 ] 161 0.085% 0.159% ( 6, 10 ] 247 0.130% 0.289% ( 10, 15 ] 1503 0.791% 1.081% ( 15, 22 ] 2282 1.201% 2.282% ( 22, 34 ] 3993 2.102% 4.383% ( 34, 51 ] 1393 0.733% 5.116% ( 51, 76 ] 225 0.118% 5.235% ( 76, 110 ] 74 0.039% 5.274% ( 110, 170 ] 13405 7.055% 12.329% # ( 170, 250 ] 81636 42.967% 55.296% ######### ( 250, 380 ] 38396 20.209% 75.505% #### ( 380, 580 ] 24280 12.779% 88.284% ### ( 580, 870 ] 11686 6.151% 94.435% # ( 870, 1300 ] 4751 2.501% 96.936% # ( 1300, 1900 ] 1778 0.936% 97.872% ( 1900, 2900 ] 1312 0.691% 98.562% ( 2900, 4400 ] 1639 0.863% 99.425% ( 4400, 6600 ] 474 0.249% 99.674% ( 6600, 9900 ] 231 0.122% 99.796% ( 9900, 14000 ] 112 0.059% 99.855% ( 14000, 22000 ] 102 0.054% 99.908% ( 22000, 33000 ] 102 0.054% 99.962% ( 33000, 50000 ] 51 0.027% 99.989% ( 50000, 75000 ] 17 0.009% 99.998% ( 75000, 110000 ] 4 0.002% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 3262927 rocksdb.bloom.filter.full.positive COUNT : 221189 rocksdb.bloom.filter.full.true.positive COUNT : 189252 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 744 rocksdb.memtable.miss COUNT : 189252 rocksdb.l0.hit COUNT : 24865 rocksdb.l1.hit COUNT : 3853 rocksdb.l2andup.hit COUNT : 160534 rocksdb.compaction.key.drop.new COUNT : 746323 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3013302 rocksdb.number.keys.read COUNT : 189996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3109727664 rocksdb.bytes.read COUNT : 189996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 486 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 7140398 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 14043306891 rocksdb.compact.write.bytes COUNT : 13277342631 rocksdb.flush.write.bytes COUNT : 3099518310 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 29 rocksdb.number.superversion_acquires COUNT : 583 rocksdb.number.superversion_releases COUNT : 561 rocksdb.number.superversion_cleanups COUNT : 141 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3079708672 rocksdb.memtable.garbage.bytes.at.flush COUNT : 5957632 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 14469224900 rocksdb.non.last.level.read.count COUNT : 3469237 rocksdb.block.checksum.compute.count COUNT : 3628024 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 239.486992 P95 : 965.765302 P99 : 3661.476510 P100 : 93043.000000 COUNT : 189996 SUM : 80287869 rocksdb.db.write.micros P50 : 1.625561 P95 : 5.244940 P99 : 8.905396 P100 : 595885.000000 COUNT : 3013302 SUM : 15910133 rocksdb.compaction.times.micros P50 : 2042857.142857 P95 : 6500000.000000 P99 : 6904242.000000 P100 : 6904242.000000 COUNT : 40 SUM : 89760573 rocksdb.compaction.times.cpu_micros P50 : 1200000.000000 P95 : 2900000.000000 P99 : 4584129.000000 P100 : 4584129.000000 COUNT : 40 SUM : 57599090 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 1042.000000 P95 : 6490.000000 P99 : 39841.000000 P100 : 39841.000000 COUNT : 42 SUM : 101550 rocksdb.compaction.outfile.sync.micros P50 : 1790.909091 P95 : 6600.000000 P99 : 9617.142857 P100 : 55979.000000 COUNT : 160 SUM : 454729 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 248.000000 P95 : 1550.000000 P99 : 4510.000000 P100 : 5205.000000 COUNT : 95 SUM : 45025 rocksdb.table.open.io.micros P50 : 5600.000000 P95 : 19080.000000 P99 : 30346.250000 P100 : 44027.000000 COUNT : 486 SUM : 3131998 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.420329 P95 : 9.956354 P99 : 242.175357 P100 : 59024.000000 COUNT : 3404042 SUM : 63186006 rocksdb.read.block.get.micros P50 : 208.705582 P95 : 803.723807 P99 : 3166.827397 P100 : 75077.000000 COUNT : 224036 SUM : 73032953 rocksdb.write.raw.block.micros P50 : 0.537630 P95 : 1.429925 P99 : 3.968937 P100 : 101321.000000 COUNT : 3968779 SUM : 15709051 rocksdb.numfiles.in.singlecompaction P50 : 3.309524 P95 : 7.666667 P99 : 12.000000 P100 : 12.000000 COUNT : 55 SUM : 217 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.561381 P95 : 1121.190072 P99 : 1295.997747 P100 : 595650.000000 COUNT : 36547 SUM : 7144429 rocksdb.sst.read.micros P50 : 2.142526 P95 : 194.975729 P99 : 545.002674 P100 : 75071.000000 COUNT : 3470105 SUM : 124539270 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 189996 SUM : 189996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3013302 SUM : 3109727664 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 239565.217391 P95 : 796200.000000 P99 : 875241.000000 P100 : 875241.000000 COUNT : 42 SUM : 14220189 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 155 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0