[2024-07-15 15:54:48.959912] Starting SPDK v24.09-pre git sha1 a95bbf233 / DPDK 24.03.0 initialization... [2024-07-15 15:54:48.960051] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2623080 ] Set seed to 1721051688956580 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-15 15:54:49.062278] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-15 15:54:49.276070] app.c: 603:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-15 15:54:49.276117] app.c: 604:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2623080' to capture a snapshot of events at runtime. [2024-07-15 15:54:49.276127] app.c: 609:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-15 15:54:49.276137] app.c: 610:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-15 15:54:49.276144] app.c: 611:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2623080 for offline analysis/debug. [2024-07-15 15:54:49.276237] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Jul 15 15:54:55 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/07/15-15:55:02 ... thread 0: (1000000,1000000) ops and (141719.7,141719.7) ops/second in (7.056183,7.056183) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 8/4 1.31 GB 103.1 0.6 0.0 0.6 1.5 0.9 0.0 1.7 54.1 125.9 12.19 5.53 12 1.016 669K 4221 0.0 0.0 L2 17/15 2.07 GB 2.6 0.7 0.6 0.1 0.7 0.6 0.0 1.1 111.7 109.5 6.16 3.43 1 6.156 637K 0 0.0 0.0 L3 42/14 2.46 GB 1.6 3.0 2.1 0.9 2.9 2.0 0.2 1.4 112.1 108.7 27.63 15.34 13 2.125 540K 0 0.0 0.0 L4 128/11 9.81 GB 0.9 0.8 0.2 0.6 0.6 0.0 0.5 4.0 123.6 98.9 6.37 3.32 3 2.122 159K 0 0.0 0.0 L5 116/0 9.54 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 311/44 25.18 GB 0.0 5.1 2.9 2.2 5.7 3.5 1.1 6.6 99.9 111.6 52.34 27.61 29 1.805 2007K 4221 0.0 0.0 Int 0/0 0.00 KB 0.0 5.1 2.9 2.2 5.7 3.5 1.1 6.6 99.9 111.6 52.34 27.61 29 1.805 2007K 4221 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.1 2.9 2.2 4.8 2.6 0.0 0.0 114.1 108.2 45.85 25.06 19 2.413 2007K 4221 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 135.6 6.49 2.55 10 0.649 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.2 total, 7.1 interval Flush(GB): cumulative 0.859, interval 0.859 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.70 GB write, 815.67 MB/s write, 5.11 GB read, 730.44 MB/s read, 52.3 seconds Interval compaction: 5.70 GB write, 827.60 MB/s write, 5.11 GB read, 741.12 MB/s read, 52.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 17, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 17, interval: 17 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 170130 Average: 34.7585 StdDev: 375.67 Min: 1 Median: 2.0109 Max: 40599 Percentiles: P50: 2.01 P75: 2.99 P99: 1084.54 P99.9: 4134.62 P99.99: 15593.07 ------------------------------------------------------ [ 0, 1 ] 4766 2.801% 2.801% # ( 1, 2 ] 79825 46.920% 49.721% ######### ( 2, 3 ] 43316 25.461% 75.182% ##### ( 3, 4 ] 14976 8.803% 83.985% ## ( 4, 6 ] 18197 10.696% 94.681% ## ( 6, 10 ] 4703 2.764% 97.445% # ( 10, 15 ] 811 0.477% 97.922% ( 15, 22 ] 86 0.051% 97.972% ( 22, 34 ] 1 0.001% 97.973% ( 34, 51 ] 3 0.002% 97.974% ( 51, 76 ] 6 0.004% 97.978% ( 76, 110 ] 2 0.001% 97.979% ( 110, 170 ] 17 0.010% 97.989% ( 170, 250 ] 57 0.034% 98.023% ( 250, 380 ] 180 0.106% 98.128% ( 380, 580 ] 411 0.242% 98.370% ( 580, 870 ] 695 0.409% 98.779% ( 870, 1300 ] 755 0.444% 99.222% ( 1300, 1900 ] 515 0.303% 99.525% ( 1900, 2900 ] 400 0.235% 99.760% ( 2900, 4400 ] 289 0.170% 99.930% ( 4400, 6600 ] 80 0.047% 99.977% ( 6600, 9900 ] 11 0.006% 99.984% ( 9900, 14000 ] 8 0.005% 99.988% ( 14000, 22000 ] 15 0.009% 99.997% ( 22000, 33000 ] 5 0.003% 100.000% ( 33000, 50000 ] 3 0.002% 100.002% ** Level 1 read latency histogram (micros): Count: 159865 Average: 19.6245 StdDev: 270.45 Min: 1 Median: 2.2446 Max: 28539 Percentiles: P50: 2.24 P75: 3.23 P99: 14.91 P99.9: 3684.85 P99.99: 11665.05 ------------------------------------------------------ [ 0, 1 ] 1662 1.040% 1.040% ( 1, 2 ] 66563 41.637% 42.677% ######## ( 2, 3 ] 47860 29.938% 72.614% ###### ( 3, 4 ] 16514 10.330% 82.944% ## ( 4, 6 ] 19672 12.305% 95.250% ## ( 6, 10 ] 4890 3.059% 98.309% # ( 10, 15 ] 1126 0.704% 99.013% ( 15, 22 ] 124 0.078% 99.090% ( 22, 34 ] 11 0.007% 99.097% ( 34, 51 ] 10 0.006% 99.104% ( 51, 76 ] 9 0.006% 99.109% ( 76, 110 ] 6 0.004% 99.113% ( 110, 170 ] 11 0.007% 99.120% ( 170, 250 ] 19 0.012% 99.132% ( 250, 380 ] 68 0.043% 99.174% ( 380, 580 ] 153 0.096% 99.270% ( 580, 870 ] 243 0.152% 99.422% ( 870, 1300 ] 265 0.166% 99.588% ( 1300, 1900 ] 211 0.132% 99.720% ( 1900, 2900 ] 195 0.122% 99.842% ( 2900, 4400 ] 178 0.111% 99.953% ( 4400, 6600 ] 42 0.026% 99.979% ( 6600, 9900 ] 14 0.009% 99.988% ( 9900, 14000 ] 7 0.004% 99.992% ( 14000, 22000 ] 11 0.007% 99.999% ( 22000, 33000 ] 2 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 572155 Average: 19.8623 StdDev: 280.53 Min: 1 Median: 2.2727 Max: 42968 Percentiles: P50: 2.27 P75: 3.24 P99: 14.31 P99.9: 2024.59 P99.99: 2771.97 ------------------------------------------------------ [ 0, 1 ] 4412 0.771% 0.771% ( 1, 2 ] 233259 40.768% 41.540% ######## ( 2, 3 ] 177484 31.020% 72.560% ###### ( 3, 4 ] 59010 10.314% 82.874% ## ( 4, 6 ] 71206 12.445% 95.319% ## ( 6, 10 ] 17940 3.136% 98.454% # ( 10, 15 ] 3625 0.634% 99.088% ( 15, 22 ] 431 0.075% 99.163% ( 22, 34 ] 46 0.008% 99.171% ( 34, 51 ] 53 0.009% 99.180% ( 51, 76 ] 54 0.009% 99.190% ( 76, 110 ] 18 0.003% 99.193% ( 110, 170 ] 31 0.005% 99.198% ( 170, 250 ] 93 0.016% 99.215% ( 250, 380 ] 267 0.047% 99.261% ( 380, 580 ] 666 0.116% 99.378% ( 580, 870 ] 1064 0.186% 99.564% ( 870, 1300 ] 1106 0.193% 99.757% ( 1300, 1900 ] 733 0.128% 99.885% ( 1900, 2900 ] 689 0.120% 100.006% ( 2900, 4400 ] 625 0.109% 100.115% ( 4400, 6600 ] 177 0.031% 100.146% ( 6600, 9900 ] 33 0.006% 100.152% ( 9900, 14000 ] 17 0.003% 100.155% ( 14000, 22000 ] 18 0.003% 100.158% ( 22000, 33000 ] 20 0.003% 100.161% ( 33000, 50000 ] 1 0.000% 100.161% ** Level 3 read latency histogram (micros): Count: 272718 Average: 24.3349 StdDev: 346.70 Min: 1 Median: 2.2654 Max: 36413 Percentiles: P50: 2.27 P75: 3.28 P99: 459.64 P99.9: 2774.36 P99.99: 3941.41 ------------------------------------------------------ [ 0, 1 ] 2958 1.085% 1.085% ( 1, 2 ] 111733 40.970% 42.055% ######## ( 2, 3 ] 81640 29.936% 71.990% ###### ( 3, 4 ] 29408 10.783% 82.774% ## ( 4, 6 ] 34258 12.562% 95.335% ### ( 6, 10 ] 7862 2.883% 98.218% # ( 10, 15 ] 1412 0.518% 98.736% ( 15, 22 ] 172 0.063% 98.799% ( 22, 34 ] 13 0.005% 98.804% ( 34, 51 ] 9 0.003% 98.807% ( 51, 76 ] 21 0.008% 98.815% ( 76, 110 ] 8 0.003% 98.818% ( 110, 170 ] 31 0.011% 98.829% ( 170, 250 ] 94 0.034% 98.864% ( 250, 380 ] 199 0.073% 98.937% ( 380, 580 ] 434 0.159% 99.096% ( 580, 870 ] 687 0.252% 99.348% ( 870, 1300 ] 731 0.268% 99.616% ( 1300, 1900 ] 478 0.175% 99.791% ( 1900, 2900 ] 340 0.125% 99.916% ( 2900, 4400 ] 292 0.107% 100.023% ( 4400, 6600 ] 61 0.022% 100.045% ( 6600, 9900 ] 35 0.013% 100.058% ( 9900, 14000 ] 16 0.006% 100.064% ( 14000, 22000 ] 30 0.011% 100.075% ( 22000, 33000 ] 11 0.004% 100.079% ( 33000, 50000 ] 3 0.001% 100.080% ** Level 4 read latency histogram (micros): Count: 342096 Average: 45.4859 StdDev: 424.16 Min: 1 Median: 2.2994 Max: 43982 Percentiles: P50: 2.30 P75: 3.45 P99: 1166.66 P99.9: 3343.38 P99.99: 3817.54 ------------------------------------------------------ [ 0, 1 ] 4266 1.247% 1.247% ( 1, 2 ] 137272 40.127% 41.374% ######## ( 2, 3 ] 98580 28.816% 70.190% ###### ( 3, 4 ] 36889 10.783% 80.973% ## ( 4, 6 ] 44300 12.950% 93.923% ### ( 6, 10 ] 10514 3.073% 96.996% # ( 10, 15 ] 2109 0.616% 97.613% ( 15, 22 ] 291 0.085% 97.698% ( 22, 34 ] 12 0.004% 97.702% ( 34, 51 ] 17 0.005% 97.706% ( 51, 76 ] 27 0.008% 97.714% ( 76, 110 ] 12 0.004% 97.718% ( 110, 170 ] 34 0.010% 97.728% ( 170, 250 ] 145 0.042% 97.770% ( 250, 380 ] 405 0.118% 97.889% ( 380, 580 ] 958 0.280% 98.169% ( 580, 870 ] 1637 0.479% 98.647% ( 870, 1300 ] 1751 0.512% 99.159% ( 1300, 1900 ] 1191 0.348% 99.507% ( 1900, 2900 ] 1057 0.309% 99.816% ( 2900, 4400 ] 974 0.285% 100.101% ( 4400, 6600 ] 315 0.092% 100.193% ( 6600, 9900 ] 49 0.014% 100.207% ( 9900, 14000 ] 10 0.003% 100.210% ( 14000, 22000 ] 15 0.004% 100.215% ( 22000, 33000 ] 20 0.006% 100.220% ( 33000, 50000 ] 4 0.001% 100.222% ** Level 5 read latency histogram (micros): Count: 5567 Average: 1599.5789 StdDev: 1908.21 Min: 88 Median: 1079.0278 Max: 34359 Percentiles: P50: 1079.03 P75: 1940.08 P99: 6592.20 P99.9: 28433.00 P99.99: 34359.00 ------------------------------------------------------ ( 76, 110 ] 3 0.054% 0.054% ( 110, 170 ] 22 0.395% 0.449% ( 170, 250 ] 105 1.886% 2.335% ( 250, 380 ] 268 4.814% 7.149% # ( 380, 580 ] 673 12.089% 19.238% ## ( 580, 870 ] 1135 20.388% 39.626% #### ( 870, 1300 ] 1188 21.340% 60.966% #### ( 1300, 1900 ] 757 13.598% 74.564% ### ( 1900, 2900 ] 605 10.868% 85.432% ## ( 2900, 4400 ] 567 10.185% 95.617% ## ( 4400, 6600 ] 189 3.395% 99.012% # ( 6600, 9900 ] 26 0.467% 99.479% ( 9900, 14000 ] 6 0.108% 99.587% ( 14000, 22000 ] 11 0.198% 99.784% ( 22000, 33000 ] 11 0.198% 99.982% ( 33000, 50000 ] 1 0.018% 100.000% ** DB Stats ** Uptime(secs): 7.2 total, 7.1 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 137.43 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.244 H:M:S, 45.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 139.42 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.244 H:M:S, 46.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/07/15-15:55:09 ... thread 0: (1000000,2000000) ops and (143737.9,142721.7) ops/second in (6.957106,14.013289) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/8 2.23 GB 153.0 1.3 0.0 1.3 3.0 1.8 0.0 1.7 52.1 125.5 24.72 10.75 25 0.989 1307K 7952 0.0 0.0 L2 1/0 128.78 MB 1.3 0.7 0.6 0.1 0.7 0.6 0.0 1.1 111.7 109.5 6.16 3.43 1 6.156 637K 0 0.0 0.0 L3 33/15 2.59 GB 1.0 5.3 4.0 1.3 5.2 3.9 0.2 1.3 115.7 112.9 47.27 26.79 18 2.626 2553K 4879 0.0 0.0 L4 157/13 10.47 GB 1.0 6.8 1.8 5.1 5.6 0.5 0.7 3.2 131.5 107.3 53.26 27.62 25 2.130 2019K 47K 0.0 0.0 L5 116/0 9.54 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 320/36 24.95 GB 0.0 14.1 6.4 7.7 14.5 6.8 1.2 8.1 110.0 112.8 131.40 68.60 69 1.904 6518K 60K 0.0 0.0 Int 0/0 0.00 KB 0.0 14.1 6.4 7.7 14.5 6.8 1.2 8.1 110.0 112.8 131.40 68.60 69 1.904 6518K 60K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.1 6.4 7.7 12.7 5.0 0.0 0.0 122.2 110.0 118.20 63.46 48 2.463 6518K 60K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 138.1 13.20 5.14 21 0.628 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.1 total, 14.0 interval Flush(GB): cumulative 1.780, interval 1.780 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.48 GB write, 1050.22 MB/s write, 14.11 GB read, 1023.39 MB/s read, 131.4 seconds Interval compaction: 14.48 GB write, 1057.95 MB/s write, 14.11 GB read, 1030.92 MB/s read, 131.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 39, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 39, interval: 39 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 336468 Average: 51.6152 StdDev: 379.48 Min: 1 Median: 2.0855 Max: 40599 Percentiles: P50: 2.09 P75: 3.19 P99: 1581.82 P99.9: 4280.99 P99.99: 8452.98 ------------------------------------------------------ [ 0, 1 ] 8729 2.594% 2.594% # ( 1, 2 ] 152185 45.230% 47.824% ######### ( 2, 3 ] 85627 25.449% 73.273% ##### ( 3, 4 ] 30713 9.128% 82.401% ## ( 4, 6 ] 36660 10.896% 93.297% ## ( 6, 10 ] 9506 2.825% 96.122% # ( 10, 15 ] 1538 0.457% 96.579% ( 15, 22 ] 186 0.055% 96.634% ( 22, 34 ] 4 0.001% 96.636% ( 34, 51 ] 10 0.003% 96.639% ( 51, 76 ] 13 0.004% 96.642% ( 76, 110 ] 3 0.001% 96.643% ( 110, 170 ] 45 0.013% 96.657% ( 170, 250 ] 138 0.041% 96.698% ( 250, 380 ] 475 0.141% 96.839% ( 380, 580 ] 1334 0.396% 97.235% ( 580, 870 ] 2472 0.735% 97.970% ( 870, 1300 ] 2664 0.792% 98.762% ( 1300, 1900 ] 1706 0.507% 99.269% ( 1900, 2900 ] 1213 0.361% 99.629% ( 2900, 4400 ] 989 0.294% 99.923% ( 4400, 6600 ] 212 0.063% 99.986% ( 6600, 9900 ] 22 0.007% 99.993% ( 9900, 14000 ] 9 0.003% 99.996% ( 14000, 22000 ] 19 0.006% 100.001% ( 22000, 33000 ] 5 0.001% 100.003% ( 33000, 50000 ] 3 0.001% 100.004% ** Level 1 read latency histogram (micros): Count: 159865 Average: 19.6245 StdDev: 270.45 Min: 1 Median: 2.2446 Max: 28539 Percentiles: P50: 2.24 P75: 3.23 P99: 14.91 P99.9: 3684.85 P99.99: 11665.05 ------------------------------------------------------ [ 0, 1 ] 1662 1.040% 1.040% ( 1, 2 ] 66563 41.637% 42.677% ######## ( 2, 3 ] 47860 29.938% 72.614% ###### ( 3, 4 ] 16514 10.330% 82.944% ## ( 4, 6 ] 19672 12.305% 95.250% ## ( 6, 10 ] 4890 3.059% 98.309% # ( 10, 15 ] 1126 0.704% 99.013% ( 15, 22 ] 124 0.078% 99.090% ( 22, 34 ] 11 0.007% 99.097% ( 34, 51 ] 10 0.006% 99.104% ( 51, 76 ] 9 0.006% 99.109% ( 76, 110 ] 6 0.004% 99.113% ( 110, 170 ] 11 0.007% 99.120% ( 170, 250 ] 19 0.012% 99.132% ( 250, 380 ] 68 0.043% 99.174% ( 380, 580 ] 153 0.096% 99.270% ( 580, 870 ] 243 0.152% 99.422% ( 870, 1300 ] 265 0.166% 99.588% ( 1300, 1900 ] 211 0.132% 99.720% ( 1900, 2900 ] 195 0.122% 99.842% ( 2900, 4400 ] 178 0.111% 99.953% ( 4400, 6600 ] 42 0.026% 99.979% ( 6600, 9900 ] 14 0.009% 99.988% ( 9900, 14000 ] 7 0.004% 99.992% ( 14000, 22000 ] 11 0.007% 99.999% ( 22000, 33000 ] 2 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 871189 Average: 19.0911 StdDev: 251.42 Min: 1 Median: 2.2986 Max: 42968 Percentiles: P50: 2.30 P75: 3.29 P99: 14.57 P99.9: 2226.64 P99.99: 3048.23 ------------------------------------------------------ [ 0, 1 ] 6078 0.698% 0.698% ( 1, 2 ] 347985 39.944% 40.641% ######## ( 2, 3 ] 273079 31.346% 71.987% ###### ( 3, 4 ] 91163 10.464% 82.451% ## ( 4, 6 ] 111107 12.753% 95.205% ### ( 6, 10 ] 28004 3.214% 98.419% # ( 10, 15 ] 5536 0.635% 99.055% ( 15, 22 ] 690 0.079% 99.134% ( 22, 34 ] 61 0.007% 99.141% ( 34, 51 ] 74 0.008% 99.149% ( 51, 76 ] 73 0.008% 99.158% ( 76, 110 ] 21 0.002% 99.160% ( 110, 170 ] 39 0.004% 99.164% ( 170, 250 ] 119 0.014% 99.178% ( 250, 380 ] 370 0.042% 99.221% ( 380, 580 ] 1000 0.115% 99.335% ( 580, 870 ] 1674 0.192% 99.528% ( 870, 1300 ] 1723 0.198% 99.725% ( 1300, 1900 ] 1187 0.136% 99.862% ( 1900, 2900 ] 1025 0.118% 99.979% ( 2900, 4400 ] 950 0.109% 100.088% ( 4400, 6600 ] 241 0.028% 100.116% ( 6600, 9900 ] 38 0.004% 100.120% ( 9900, 14000 ] 18 0.002% 100.122% ( 14000, 22000 ] 18 0.002% 100.124% ( 22000, 33000 ] 20 0.002% 100.127% ( 33000, 50000 ] 1 0.000% 100.127% ** Level 3 read latency histogram (micros): Count: 809305 Average: 20.9233 StdDev: 254.80 Min: 1 Median: 2.2966 Max: 36413 Percentiles: P50: 2.30 P75: 3.32 P99: 292.60 P99.9: 2451.69 P99.99: 3332.62 ------------------------------------------------------ [ 0, 1 ] 7959 0.983% 0.983% ( 1, 2 ] 323256 39.942% 40.926% ######## ( 2, 3 ] 247579 30.592% 71.517% ###### ( 3, 4 ] 87085 10.760% 82.278% ## ( 4, 6 ] 104001 12.851% 95.129% ### ( 6, 10 ] 25563 3.159% 98.287% # ( 10, 15 ] 4607 0.569% 98.856% ( 15, 22 ] 614 0.076% 98.932% ( 22, 34 ] 47 0.006% 98.938% ( 34, 51 ] 57 0.007% 98.945% ( 51, 76 ] 56 0.007% 98.952% ( 76, 110 ] 17 0.002% 98.954% ( 110, 170 ] 54 0.007% 98.961% ( 170, 250 ] 179 0.022% 98.983% ( 250, 380 ] 427 0.053% 99.036% ( 380, 580 ] 1114 0.138% 99.173% ( 580, 870 ] 1895 0.234% 99.408% ( 870, 1300 ] 2067 0.255% 99.663% ( 1300, 1900 ] 1353 0.167% 99.830% ( 1900, 2900 ] 1029 0.127% 99.957% ( 2900, 4400 ] 926 0.114% 100.072% ( 4400, 6600 ] 176 0.022% 100.093% ( 6600, 9900 ] 46 0.006% 100.099% ( 9900, 14000 ] 16 0.002% 100.101% ( 14000, 22000 ] 30 0.004% 100.105% ( 22000, 33000 ] 11 0.001% 100.106% ( 33000, 50000 ] 3 0.000% 100.107% ** Level 4 read latency histogram (micros): Count: 1440694 Average: 28.9103 StdDev: 283.88 Min: 1 Median: 2.2800 Max: 43982 Percentiles: P50: 2.28 P75: 3.36 P99: 672.98 P99.9: 2254.14 P99.99: 2709.10 ------------------------------------------------------ [ 0, 1 ] 16299 1.131% 1.131% ( 1, 2 ] 586061 40.679% 41.810% ######## ( 2, 3 ] 421376 29.248% 71.059% ###### ( 3, 4 ] 158072 10.972% 82.030% ## ( 4, 6 ] 183372 12.728% 94.758% ### ( 6, 10 ] 45238 3.140% 97.899% # ( 10, 15 ] 8757 0.608% 98.506% ( 15, 22 ] 1168 0.081% 98.587% ( 22, 34 ] 64 0.004% 98.592% ( 34, 51 ] 104 0.007% 98.599% ( 51, 76 ] 99 0.007% 98.606% ( 76, 110 ] 48 0.003% 98.609% ( 110, 170 ] 71 0.005% 98.614% ( 170, 250 ] 373 0.026% 98.640% ( 250, 380 ] 963 0.067% 98.707% ( 380, 580 ] 2676 0.186% 98.893% ( 580, 870 ] 4822 0.335% 99.227% ( 870, 1300 ] 5195 0.361% 99.588% ( 1300, 1900 ] 3486 0.242% 99.830% ( 1900, 2900 ] 2850 0.198% 100.028% ( 2900, 4400 ] 2613 0.181% 100.209% ( 4400, 6600 ] 682 0.047% 100.256% ( 6600, 9900 ] 85 0.006% 100.262% ( 9900, 14000 ] 12 0.001% 100.263% ( 14000, 22000 ] 20 0.001% 100.265% ( 22000, 33000 ] 20 0.001% 100.266% ( 33000, 50000 ] 4 0.000% 100.266% ** Level 5 read latency histogram (micros): Count: 11360 Average: 1498.2969 StdDev: 1542.00 Min: 88 Median: 1076.8579 Max: 34359 Percentiles: P50: 1076.86 P75: 1826.90 P99: 6235.09 P99.9: 22640.00 P99.99: 32864.00 ------------------------------------------------------ ( 76, 110 ] 4 0.035% 0.035% ( 110, 170 ] 35 0.308% 0.343% ( 170, 250 ] 178 1.567% 1.910% ( 250, 380 ] 497 4.375% 6.285% # ( 380, 580 ] 1321 11.629% 17.914% ## ( 580, 870 ] 2362 20.792% 38.706% #### ( 870, 1300 ] 2667 23.477% 62.183% ##### ( 1300, 1900 ] 1658 14.595% 76.778% ### ( 1900, 2900 ] 1175 10.343% 87.121% ## ( 2900, 4400 ] 1105 9.727% 96.849% ## ( 4400, 6600 ] 293 2.579% 99.428% # ( 6600, 9900 ] 35 0.308% 99.736% ( 9900, 14000 ] 7 0.062% 99.798% ( 14000, 22000 ] 11 0.097% 99.894% ( 22000, 33000 ] 11 0.097% 99.991% ( 33000, 50000 ] 1 0.009% 100.000% ** DB Stats ** Uptime(secs): 14.1 total, 7.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 139.42 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.415 H:M:S, 45.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 141.47 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.171 H:M:S, 45.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/07/15-15:55:13 ... thread 0: (1000000,3000000) ops and (222833.6,162153.9) ops/second in (4.487654,18.500943) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/4 3.20 GB 303.0 2.6 0.0 2.6 5.4 2.7 0.0 1.9 62.9 129.1 42.48 19.15 43 0.988 2712K 16K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.7 0.6 0.1 0.7 0.6 0.0 1.1 111.7 109.5 6.16 3.43 1 6.156 637K 0 0.0 0.0 L3 15/0 824.78 MB 0.8 5.5 4.2 1.4 5.4 4.0 0.2 1.3 116.6 113.8 48.57 27.68 19 2.556 2617K 4879 0.0 0.0 L4 150/0 9.53 GB 1.0 9.5 3.5 6.0 8.0 2.1 0.8 2.3 134.7 113.9 72.25 39.44 33 2.189 3943K 51K 0.0 0.0 L5 156/0 12.18 GB 0.1 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 336/4 25.72 GB 0.0 18.3 8.3 10.0 19.5 9.5 4.0 7.0 110.7 117.5 169.45 89.70 96 1.765 9911K 72K 0.0 0.0 Int 0/0 0.00 KB 0.0 18.3 8.3 10.0 19.5 9.5 4.0 7.0 110.7 117.5 169.45 89.70 96 1.765 9911K 72K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 18.3 8.3 10.0 16.7 6.7 0.0 0.0 123.4 112.4 152.01 81.82 61 2.492 9911K 72K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 162.3 17.44 7.88 35 0.498 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 18.6 total, 18.5 interval Flush(GB): cumulative 2.763, interval 2.763 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 19.45 GB write, 1070.46 MB/s write, 18.31 GB read, 1007.84 MB/s read, 169.5 seconds Interval compaction: 19.45 GB write, 1076.43 MB/s write, 18.31 GB read, 1013.46 MB/s read, 169.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 1, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 1, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 46, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 1, total-stops: 46, interval: 47 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 744679 Average: 33.2310 StdDev: 276.76 Min: 1 Median: 1.8593 Max: 40599 Percentiles: P50: 1.86 P75: 2.80 P99: 974.39 P99.9: 3438.93 P99.99: 4290.18 ------------------------------------------------------ [ 0, 1 ] 42212 5.668% 5.668% # ( 1, 2 ] 384180 51.590% 57.258% ########## ( 2, 3 ] 165372 22.207% 79.466% #### ( 3, 4 ] 52420 7.039% 86.505% # ( 4, 6 ] 57857 7.769% 94.274% ## ( 6, 10 ] 15890 2.134% 96.408% ( 10, 15 ] 2359 0.317% 96.725% ( 15, 22 ] 277 0.037% 96.762% ( 22, 34 ] 13 0.002% 96.764% ( 34, 51 ] 15 0.002% 96.766% ( 51, 76 ] 29 0.004% 96.770% ( 76, 110 ] 327 0.044% 96.814% ( 110, 170 ] 3060 0.411% 97.225% ( 170, 250 ] 3177 0.427% 97.651% ( 250, 380 ] 2336 0.314% 97.965% ( 380, 580 ] 3083 0.414% 98.379% ( 580, 870 ] 3754 0.504% 98.883% ( 870, 1300 ] 3597 0.483% 99.366% ( 1300, 1900 ] 2120 0.285% 99.651% ( 1900, 2900 ] 1434 0.193% 99.843% ( 2900, 4400 ] 1181 0.159% 100.002% ( 4400, 6600 ] 242 0.032% 100.034% ( 6600, 9900 ] 24 0.003% 100.038% ( 9900, 14000 ] 11 0.001% 100.039% ( 14000, 22000 ] 20 0.003% 100.042% ( 22000, 33000 ] 5 0.001% 100.042% ( 33000, 50000 ] 3 0.000% 100.043% ** Level 1 read latency histogram (micros): Count: 159865 Average: 19.6245 StdDev: 270.45 Min: 1 Median: 2.2446 Max: 28539 Percentiles: P50: 2.24 P75: 3.23 P99: 14.91 P99.9: 3684.85 P99.99: 11665.05 ------------------------------------------------------ [ 0, 1 ] 1662 1.040% 1.040% ( 1, 2 ] 66563 41.637% 42.677% ######## ( 2, 3 ] 47860 29.938% 72.614% ###### ( 3, 4 ] 16514 10.330% 82.944% ## ( 4, 6 ] 19672 12.305% 95.250% ## ( 6, 10 ] 4890 3.059% 98.309% # ( 10, 15 ] 1126 0.704% 99.013% ( 15, 22 ] 124 0.078% 99.090% ( 22, 34 ] 11 0.007% 99.097% ( 34, 51 ] 10 0.006% 99.104% ( 51, 76 ] 9 0.006% 99.109% ( 76, 110 ] 6 0.004% 99.113% ( 110, 170 ] 11 0.007% 99.120% ( 170, 250 ] 19 0.012% 99.132% ( 250, 380 ] 68 0.043% 99.174% ( 380, 580 ] 153 0.096% 99.270% ( 580, 870 ] 243 0.152% 99.422% ( 870, 1300 ] 265 0.166% 99.588% ( 1300, 1900 ] 211 0.132% 99.720% ( 1900, 2900 ] 195 0.122% 99.842% ( 2900, 4400 ] 178 0.111% 99.953% ( 4400, 6600 ] 42 0.026% 99.979% ( 6600, 9900 ] 14 0.009% 99.988% ( 9900, 14000 ] 7 0.004% 99.992% ( 14000, 22000 ] 11 0.007% 99.999% ( 22000, 33000 ] 2 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 903987 Average: 18.7914 StdDev: 247.88 Min: 1 Median: 2.2997 Max: 42968 Percentiles: P50: 2.30 P75: 3.29 P99: 14.54 P99.9: 2220.86 P99.99: 3065.87 ------------------------------------------------------ [ 0, 1 ] 6155 0.681% 0.681% ( 1, 2 ] 360710 39.902% 40.583% ######## ( 2, 3 ] 284022 31.419% 72.002% ###### ( 3, 4 ] 94574 10.462% 82.464% ## ( 4, 6 ] 115324 12.757% 95.221% ### ( 6, 10 ] 28993 3.207% 98.428% # ( 10, 15 ] 5693 0.630% 99.058% ( 15, 22 ] 715 0.079% 99.137% ( 22, 34 ] 63 0.007% 99.144% ( 34, 51 ] 75 0.008% 99.152% ( 51, 76 ] 74 0.008% 99.160% ( 76, 110 ] 23 0.003% 99.163% ( 110, 170 ] 55 0.006% 99.169% ( 170, 250 ] 144 0.016% 99.185% ( 250, 380 ] 390 0.043% 99.228% ( 380, 580 ] 1041 0.115% 99.343% ( 580, 870 ] 1721 0.190% 99.534% ( 870, 1300 ] 1763 0.195% 99.729% ( 1300, 1900 ] 1214 0.134% 99.863% ( 1900, 2900 ] 1041 0.115% 99.978% ( 2900, 4400 ] 964 0.107% 100.085% ( 4400, 6600 ] 243 0.027% 100.112% ( 6600, 9900 ] 38 0.004% 100.116% ( 9900, 14000 ] 18 0.002% 100.118% ( 14000, 22000 ] 18 0.002% 100.120% ( 22000, 33000 ] 20 0.002% 100.122% ( 33000, 50000 ] 1 0.000% 100.122% ** Level 3 read latency histogram (micros): Count: 1182748 Average: 18.5188 StdDev: 224.95 Min: 1 Median: 2.2511 Max: 36413 Percentiles: P50: 2.25 P75: 3.18 P99: 152.49 P99.9: 1908.74 P99.99: 2735.84 ------------------------------------------------------ [ 0, 1 ] 11682 0.988% 0.988% ( 1, 2 ] 487729 41.237% 42.225% ######## ( 2, 3 ] 366198 30.962% 73.186% ###### ( 3, 4 ] 120688 10.204% 83.390% ## ( 4, 6 ] 141263 11.944% 95.334% ## ( 6, 10 ] 35514 3.003% 98.337% # ( 10, 15 ] 6221 0.526% 98.863% ( 15, 22 ] 791 0.067% 98.929% ( 22, 34 ] 62 0.005% 98.935% ( 34, 51 ] 80 0.007% 98.941% ( 51, 76 ] 81 0.007% 98.948% ( 76, 110 ] 97 0.008% 98.956% ( 110, 170 ] 728 0.062% 99.018% ( 170, 250 ] 773 0.065% 99.083% ( 250, 380 ] 920 0.078% 99.161% ( 380, 580 ] 1729 0.146% 99.307% ( 580, 870 ] 2625 0.222% 99.529% ( 870, 1300 ] 2668 0.226% 99.755% ( 1300, 1900 ] 1706 0.144% 99.899% ( 1900, 2900 ] 1287 0.109% 100.008% ( 2900, 4400 ] 1136 0.096% 100.104% ( 4400, 6600 ] 192 0.016% 100.120% ( 6600, 9900 ] 48 0.004% 100.124% ( 9900, 14000 ] 16 0.001% 100.126% ( 14000, 22000 ] 33 0.003% 100.128% ( 22000, 33000 ] 11 0.001% 100.129% ( 33000, 50000 ] 3 0.000% 100.130% ** Level 4 read latency histogram (micros): Count: 1523213 Average: 32.1863 StdDev: 285.41 Min: 1 Median: 2.2803 Max: 43982 Percentiles: P50: 2.28 P75: 3.40 P99: 763.21 P99.9: 2340.46 P99.99: 2779.99 ------------------------------------------------------ [ 0, 1 ] 18949 1.244% 1.244% ( 1, 2 ] 619509 40.671% 41.915% ######## ( 2, 3 ] 439313 28.841% 70.756% ###### ( 3, 4 ] 163225 10.716% 81.472% ## ( 4, 6 ] 188535 12.377% 93.850% ## ( 6, 10 ] 46814 3.073% 96.923% # ( 10, 15 ] 9002 0.591% 97.514% ( 15, 22 ] 1203 0.079% 97.593% ( 22, 34 ] 68 0.004% 97.598% ( 34, 51 ] 110 0.007% 97.605% ( 51, 76 ] 104 0.007% 97.612% ( 76, 110 ] 547 0.036% 97.647% ( 110, 170 ] 4450 0.292% 97.940% ( 170, 250 ] 4443 0.292% 98.231% ( 250, 380 ] 3120 0.205% 98.436% ( 380, 580 ] 4595 0.302% 98.738% ( 580, 870 ] 6322 0.415% 99.153% ( 870, 1300 ] 6096 0.400% 99.553% ( 1300, 1900 ] 3911 0.257% 99.810% ( 1900, 2900 ] 3119 0.205% 100.015% ( 2900, 4400 ] 2814 0.185% 100.199% ( 4400, 6600 ] 706 0.046% 100.246% ( 6600, 9900 ] 86 0.006% 100.251% ( 9900, 14000 ] 12 0.001% 100.252% ( 14000, 22000 ] 20 0.001% 100.253% ( 22000, 33000 ] 20 0.001% 100.255% ( 33000, 50000 ] 4 0.000% 100.255% ** Level 5 read latency histogram (micros): Count: 22231 Average: 973.4023 StdDev: 1286.55 Min: 85 Median: 603.4463 Max: 34359 Percentiles: P50: 603.45 P75: 1205.07 P99: 5479.60 P99.9: 14559.27 P99.99: 31776.90 ------------------------------------------------------ ( 76, 110 ] 363 1.633% 1.633% ( 110, 170 ] 3003 13.508% 15.141% ### ( 170, 250 ] 2891 13.004% 28.145% ### ( 250, 380 ] 1962 8.826% 36.971% ## ( 380, 580 ] 2628 11.821% 48.792% ## ( 580, 870 ] 3321 14.939% 63.731% ### ( 870, 1300 ] 3215 14.462% 78.193% ### ( 1300, 1900 ] 1916 8.619% 86.811% ## ( 1900, 2900 ] 1323 5.951% 92.762% # ( 2900, 4400 ] 1238 5.569% 98.331% # ( 4400, 6600 ] 303 1.363% 99.694% ( 6600, 9900 ] 37 0.166% 99.861% ( 9900, 14000 ] 8 0.036% 99.897% ( 14000, 22000 ] 11 0.049% 99.946% ( 22000, 33000 ] 11 0.049% 99.996% ( 33000, 50000 ] 1 0.004% 100.000% ** DB Stats ** Uptime(secs): 18.6 total, 4.5 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 158.69 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:7.236 H:M:S, 38.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 219.32 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.821 H:M:S, 18.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 848.912 micros/op 4692 ops/sec 20.244 seconds 94996 operations; 4.5 MB/s (23999 of 23999 found) Microseconds per read: Count: 94996 Average: 848.9317 StdDev: 1664.53 Min: 1 Median: 367.3228 Max: 54230 Percentiles: P50: 367.32 P75: 953.33 P99: 5886.89 P99.9: 25255.55 P99.99: 45750.31 ------------------------------------------------------ [ 0, 1 ] 1 0.001% 0.001% ( 1, 2 ] 3 0.003% 0.004% ( 2, 3 ] 22 0.023% 0.027% ( 3, 4 ] 33 0.035% 0.062% ( 4, 6 ] 120 0.126% 0.188% ( 6, 10 ] 176 0.185% 0.374% ( 10, 15 ] 505 0.532% 0.905% ( 15, 22 ] 1040 1.095% 2.000% ( 22, 34 ] 1564 1.646% 3.646% ( 34, 51 ] 945 0.995% 4.641% ( 51, 76 ] 110 0.116% 4.757% ( 76, 110 ] 125 0.132% 4.889% ( 110, 170 ] 10016 10.544% 15.432% ## ( 170, 250 ] 22408 23.588% 39.021% ##### ( 250, 380 ] 11557 12.166% 51.186% ## ( 380, 580 ] 10174 10.710% 61.896% ## ( 580, 870 ] 10586 11.144% 73.040% ## ( 870, 1300 ] 9608 10.114% 83.154% ## ( 1300, 1900 ] 6169 6.494% 89.648% # ( 1900, 2900 ] 4358 4.588% 94.236% # ( 2900, 4400 ] 3788 3.988% 98.223% # ( 4400, 6600 ] 1092 1.150% 99.373% ( 6600, 9900 ] 217 0.228% 99.601% ( 9900, 14000 ] 92 0.097% 99.698% ( 14000, 22000 ] 163 0.172% 99.869% ( 22000, 33000 ] 98 0.103% 99.973% ( 33000, 50000 ] 22 0.023% 99.996% ( 50000, 75000 ] 4 0.004% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 1318337 rocksdb.bloom.filter.full.positive COUNT : 107384 rocksdb.bloom.filter.full.true.positive COUNT : 94423 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 573 rocksdb.memtable.miss COUNT : 94423 rocksdb.l0.hit COUNT : 11499 rocksdb.l1.hit COUNT : 392 rocksdb.l2andup.hit COUNT : 82532 rocksdb.compaction.key.drop.new COUNT : 1694492 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3458821 rocksdb.number.keys.read COUNT : 94996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3569503272 rocksdb.bytes.read COUNT : 94996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 572 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 7414669 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 20184674373 rocksdb.compact.write.bytes COUNT : 18434950426 rocksdb.flush.write.bytes COUNT : 3494164262 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 35 rocksdb.number.superversion_acquires COUNT : 694 rocksdb.number.superversion_releases COUNT : 680 rocksdb.number.superversion_cleanups COUNT : 174 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 231 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3472812032 rocksdb.memtable.garbage.bytes.at.flush COUNT : 7640064 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 19390101783 rocksdb.non.last.level.read.count COUNT : 4658391 rocksdb.block.checksum.compute.count COUNT : 4999026 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 366.663165 P95 : 3187.170011 P99 : 5886.240147 P100 : 54229.000000 COUNT : 94996 SUM : 80554880 rocksdb.db.write.micros P50 : 1.541624 P95 : 3.898237 P99 : 5.788311 P100 : 1083858.000000 COUNT : 3458821 SUM : 15811824 rocksdb.compaction.times.micros P50 : 2223529.411765 P95 : 5290000.000000 P99 : 11396000.000000 P100 : 12686201.000000 COUNT : 62 SUM : 152974819 rocksdb.compaction.times.cpu_micros P50 : 1160769.230769 P95 : 1897200.000000 P99 : 7570904.000000 P100 : 7570904.000000 COUNT : 62 SUM : 82750472 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 6050.000000 P95 : 135500.000000 P99 : 274529.000000 P100 : 274529.000000 COUNT : 43 SUM : 1323942 rocksdb.compaction.outfile.sync.micros P50 : 13941.428571 P95 : 95854.166667 P99 : 320633.333333 P100 : 442303.000000 COUNT : 237 SUM : 7010182 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 462.352941 P95 : 2677.777778 P99 : 4300.000000 P100 : 4516.000000 COUNT : 120 SUM : 96058 rocksdb.table.open.io.micros P50 : 7803.125000 P95 : 20531.764706 P99 : 26162.000000 P100 : 26162.000000 COUNT : 572 SUM : 4665694 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.431822 P95 : 9.519239 P99 : 29.684046 P100 : 42973.000000 COUNT : 4889265 SUM : 85907066 rocksdb.read.block.get.micros P50 : 283.221912 P95 : 2815.745025 P99 : 4525.984749 P100 : 43989.000000 COUNT : 109243 SUM : 76037071 rocksdb.write.raw.block.micros P50 : 0.521016 P95 : 0.989930 P99 : 3.174487 P100 : 49946.000000 COUNT : 5328756 SUM : 19205178 rocksdb.numfiles.in.singlecompaction P50 : 3.260870 P95 : 5.000000 P99 : 13.000000 P100 : 13.000000 COUNT : 80 SUM : 306 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.563203 P95 : 1141.504132 P99 : 2585.000000 P100 : 1083674.000000 COUNT : 2504 SUM : 7414829 rocksdb.sst.read.micros P50 : 2.204271 P95 : 8.198578 P99 : 879.325415 P100 : 43982.000000 COUNT : 4658527 SUM : 144944593 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 94996 SUM : 94996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3458821 SUM : 3569503272 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 387916.666667 P95 : 1069666.666667 P99 : 1294951.000000 P100 : 1294951.000000 COUNT : 43 SUM : 19023404 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 231 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0