[2024-07-10 13:19:48.330069] Starting SPDK v24.05.1-pre git sha1 5fa2f5086 / DPDK 23.11.0 initialization... [2024-07-10 13:19:48.330202] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1158927 ] Set seed to 1720610388326793 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-10 13:19:48.479018] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-10 13:19:48.527495] app.c: 604:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-10 13:19:48.527556] app.c: 605:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1158927' to capture a snapshot of events at runtime. [2024-07-10 13:19:48.527572] app.c: 610:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-10 13:19:48.527592] app.c: 611:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-10 13:19:48.527605] app.c: 612:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1158927 for offline analysis/debug. [2024-07-10 13:19:48.527728] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Jul 10 13:19:52 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/07/10-13:20:00 ... thread 0: (1000000,1000000) ops and (134372.5,134372.5) ops/second in (7.441999,7.441999) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/11 2.47 GB 31.5 0.3 0.0 0.3 1.2 0.9 0.0 1.4 39.8 151.0 7.92 4.40 13 0.610 319K 366 0.0 0.0 L1 15/15 1.56 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 55.59 MB 0.6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 21/0 888.15 MB 0.9 1.1 0.5 0.6 1.1 0.5 0.1 2.2 79.3 78.5 14.46 7.83 7 2.066 200K 0 0.0 0.0 L4 163/0 9.63 GB 1.0 3.0 2.0 0.9 2.9 2.0 0.0 1.4 95.4 95.0 31.70 18.32 21 1.509 1154K 0 0.0 0.0 L5 1080/0 90.55 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1292/26 105.13 GB 0.0 4.4 2.5 1.8 5.2 3.4 2.1 6.1 83.0 98.8 54.08 30.55 41 1.319 1674K 366 0.0 0.0 Int 0/0 0.00 KB 0.0 4.4 2.5 1.8 5.2 3.4 2.1 6.1 83.0 98.8 54.08 30.55 41 1.319 1674K 366 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.4 2.5 1.8 4.4 2.5 0.0 0.0 93.7 93.1 47.91 27.65 29 1.652 1674K 366 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 142.8 6.18 2.89 12 0.515 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.7 total, 7.4 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.22 GB write, 696.80 MB/s write, 4.38 GB read, 585.01 MB/s read, 54.1 seconds Interval compaction: 5.22 GB write, 717.88 MB/s write, 4.38 GB read, 602.71 MB/s read, 54.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 14, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 14, interval: 14 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 375225 Average: 10.1893 StdDev: 143.45 Min: 1 Median: 2.6763 Max: 24914 Percentiles: P50: 2.68 P75: 4.22 P99: 16.68 P99.9: 786.72 P99.99: 1804.55 ------------------------------------------------------ [ 0, 1 ] 3697 0.985% 0.985% ( 1, 2 ] 89372 23.818% 24.804% ##### ( 2, 3 ] 139790 37.255% 62.058% ####### ( 3, 4 ] 41857 11.155% 73.214% ## ( 4, 6 ] 60813 16.207% 89.421% ### ( 6, 10 ] 29716 7.920% 97.340% ## ( 10, 15 ] 5896 1.571% 98.912% ( 15, 22 ] 1393 0.371% 99.283% ( 22, 34 ] 254 0.068% 99.351% ( 34, 51 ] 37 0.010% 99.360% ( 51, 76 ] 20 0.005% 99.366% ( 76, 110 ] 48 0.013% 99.379% ( 110, 170 ] 400 0.107% 99.485% ( 170, 250 ] 613 0.163% 99.648% ( 250, 380 ] 422 0.112% 99.761% ( 380, 580 ] 342 0.091% 99.852% ( 580, 870 ] 255 0.068% 99.920% ( 870, 1300 ] 156 0.042% 99.962% ( 1300, 1900 ] 129 0.034% 99.996% ( 1900, 2900 ] 119 0.032% 100.028% ( 2900, 4400 ] 159 0.042% 100.070% ( 4400, 6600 ] 59 0.016% 100.086% ( 6600, 9900 ] 15 0.004% 100.090% ( 9900, 14000 ] 1 0.000% 100.090% ( 14000, 22000 ] 2 0.001% 100.091% ( 22000, 33000 ] 2 0.001% 100.091% ** Level 1 read latency histogram (micros): Count: 409 Average: 998.6235 StdDev: 2273.73 Min: 59 Median: 324.6622 Max: 16890 Percentiles: P50: 324.66 P75: 736.03 P99: 16546.67 P99.9: 16890.00 P99.99: 16890.00 ------------------------------------------------------ ( 51, 76 ] 2 0.489% 0.489% ( 76, 110 ] 3 0.733% 1.222% ( 110, 170 ] 79 19.315% 20.538% #### ( 170, 250 ] 78 19.071% 39.609% #### ( 250, 380 ] 74 18.093% 57.702% #### ( 380, 580 ] 46 11.247% 68.949% ## ( 580, 870 ] 46 11.247% 80.196% ## ( 870, 1300 ] 23 5.623% 85.819% # ( 1300, 1900 ] 14 3.423% 89.242% # ( 1900, 2900 ] 13 3.178% 92.421% # ( 2900, 4400 ] 13 3.178% 95.599% # ( 4400, 6600 ] 8 1.956% 97.555% ( 6600, 9900 ] 2 0.489% 98.044% ( 9900, 14000 ] 2 0.489% 98.533% ( 14000, 22000 ] 6 1.467% 100.000% ** Level 2 read latency histogram (micros): Count: 14243 Average: 20.8995 StdDev: 257.15 Min: 1 Median: 2.8124 Max: 14362 Percentiles: P50: 2.81 P75: 4.64 P99: 21.95 P99.9: 4095.97 P99.99: 9199.90 ------------------------------------------------------ [ 0, 1 ] 145 1.018% 1.018% ( 1, 2 ] 2974 20.880% 21.898% #### ( 2, 3 ] 4927 34.592% 56.491% ####### ( 3, 4 ] 1842 12.933% 69.424% ### ( 4, 6 ] 2501 17.560% 86.983% #### ( 6, 10 ] 1384 9.717% 96.700% ## ( 10, 15 ] 267 1.875% 98.575% ( 15, 22 ] 61 0.428% 99.003% ( 22, 34 ] 12 0.084% 99.087% ( 34, 51 ] 3 0.021% 99.108% ( 51, 76 ] 2 0.014% 99.122% ( 110, 170 ] 10 0.070% 99.193% ( 170, 250 ] 12 0.084% 99.277% ( 250, 380 ] 10 0.070% 99.347% ( 380, 580 ] 7 0.049% 99.396% ( 580, 870 ] 9 0.063% 99.459% ( 870, 1300 ] 6 0.042% 99.502% ( 1300, 1900 ] 16 0.112% 99.614% ( 1900, 2900 ] 28 0.197% 99.810% ( 2900, 4400 ] 16 0.112% 99.923% ( 4400, 6600 ] 8 0.056% 99.979% ( 6600, 9900 ] 2 0.014% 99.993% ( 14000, 22000 ] 1 0.007% 100.000% ** Level 3 read latency histogram (micros): Count: 554905 Average: 12.4582 StdDev: 193.93 Min: 1 Median: 2.8210 Max: 39941 Percentiles: P50: 2.82 P75: 4.62 P99: 13.66 P99.9: 38.64 P99.99: 353.51 ------------------------------------------------------ [ 0, 1 ] 2178 0.392% 0.392% ( 1, 2 ] 97633 17.595% 17.987% #### ( 2, 3 ] 216363 38.991% 56.978% ######## ( 3, 4 ] 67339 12.135% 69.113% ## ( 4, 6 ] 105347 18.985% 88.098% #### ( 6, 10 ] 53416 9.626% 97.724% ## ( 10, 15 ] 9673 1.743% 99.467% ( 15, 22 ] 2052 0.370% 99.837% ( 22, 34 ] 324 0.058% 99.895% ( 34, 51 ] 92 0.017% 99.912% ( 51, 76 ] 43 0.008% 99.920% ( 76, 110 ] 38 0.007% 99.927% ( 110, 170 ] 95 0.017% 99.944% ( 170, 250 ] 153 0.028% 99.971% ( 250, 380 ] 130 0.023% 99.995% ( 380, 580 ] 189 0.034% 100.029% ( 580, 870 ] 235 0.042% 100.071% ( 870, 1300 ] 286 0.052% 100.123% ( 1300, 1900 ] 249 0.045% 100.168% ( 1900, 2900 ] 319 0.057% 100.225% ( 2900, 4400 ] 411 0.074% 100.299% ( 4400, 6600 ] 168 0.030% 100.329% ( 6600, 9900 ] 36 0.006% 100.336% ( 9900, 14000 ] 3 0.001% 100.336% ( 14000, 22000 ] 4 0.001% 100.337% ( 22000, 33000 ] 4 0.001% 100.338% ( 33000, 50000 ] 1 0.000% 100.338% ** Level 4 read latency histogram (micros): Count: 235193 Average: 36.6659 StdDev: 331.37 Min: 1 Median: 2.7449 Max: 39783 Percentiles: P50: 2.74 P75: 4.30 P99: 906.52 P99.9: 3477.27 P99.99: 4111.02 ------------------------------------------------------ [ 0, 1 ] 1012 0.430% 0.430% ( 1, 2 ] 45002 19.134% 19.564% #### ( 2, 3 ] 96096 40.858% 60.423% ######## ( 3, 4 ] 28496 12.116% 72.539% ## ( 4, 6 ] 38390 16.323% 88.861% ### ( 6, 10 ] 16371 6.961% 95.822% # ( 10, 15 ] 3751 1.595% 97.417% ( 15, 22 ] 708 0.301% 97.718% ( 22, 34 ] 94 0.040% 97.758% ( 34, 51 ] 25 0.011% 97.769% ( 51, 76 ] 26 0.011% 97.780% ( 76, 110 ] 43 0.018% 97.798% ( 110, 170 ] 488 0.207% 98.005% ( 170, 250 ] 659 0.280% 98.286% ( 250, 380 ] 533 0.227% 98.512% ( 380, 580 ] 536 0.228% 98.740% ( 580, 870 ] 555 0.236% 98.976% ( 870, 1300 ] 672 0.286% 99.262% ( 1300, 1900 ] 659 0.280% 99.542% ( 1900, 2900 ] 650 0.276% 99.818% ( 2900, 4400 ] 501 0.213% 100.031% ( 4400, 6600 ] 231 0.098% 100.130% ( 6600, 9900 ] 42 0.018% 100.148% ( 14000, 22000 ] 2 0.001% 100.148% ( 33000, 50000 ] 3 0.001% 100.150% ** Level 5 read latency histogram (micros): Count: 21989 Average: 947.9145 StdDev: 1738.51 Min: 87 Median: 337.7390 Max: 77547 Percentiles: P50: 337.74 P75: 900.51 P99: 6573.68 P99.9: 9904.10 P99.99: 56675.83 ------------------------------------------------------ ( 76, 110 ] 220 1.001% 1.001% ( 110, 170 ] 3751 17.059% 18.059% ### ( 170, 250 ] 4829 21.961% 40.020% #### ( 250, 380 ] 3253 14.794% 54.814% ### ( 380, 580 ] 2552 11.606% 66.420% ## ( 580, 870 ] 1795 8.163% 74.583% ## ( 870, 1300 ] 1307 5.944% 80.527% # ( 1300, 1900 ] 1099 4.998% 85.525% # ( 1900, 2900 ] 1052 4.784% 90.309% # ( 2900, 4400 ] 1178 5.357% 95.666% # ( 4400, 6600 ] 743 3.379% 99.045% # ( 6600, 9900 ] 189 0.860% 99.904% ( 9900, 14000 ] 11 0.050% 99.955% ( 14000, 22000 ] 6 0.027% 99.982% ( 33000, 50000 ] 2 0.009% 99.991% ( 50000, 75000 ] 3 0.014% 100.005% ( 75000, 110000 ] 1 0.005% 100.009% ** DB Stats ** Uptime(secs): 7.7 total, 7.4 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 128.31 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.197 H:M:S, 15.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 132.18 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.197 H:M:S, 16.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/07/10-13:20:09 ... thread 0: (1000000,2000000) ops and (115553.2,124254.3) ops/second in (8.654020,16.096019) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/7 3.46 GB 107.0 1.6 0.0 1.6 3.4 1.8 0.0 1.9 90.1 194.0 18.18 13.95 34 0.535 1663K 1716 0.0 0.0 L1 15/15 1.56 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 55.59 MB 0.6 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 21/0 888.15 MB 0.9 1.1 0.5 0.6 1.1 0.5 0.1 2.2 79.3 78.5 14.46 7.83 7 2.066 200K 0 0.0 0.0 L4 163/0 9.63 GB 1.0 3.0 2.0 0.9 2.9 2.0 0.0 1.4 95.4 95.0 31.70 18.32 21 1.509 1154K 0 0.0 0.0 L5 1080/0 90.55 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1293/22 106.12 GB 0.0 5.7 2.5 3.1 7.5 4.4 2.1 4.1 90.3 119.3 64.34 40.10 62 1.038 3018K 1716 0.0 0.0 Int 0/0 0.00 KB 0.0 5.7 2.5 3.1 7.5 4.4 2.1 4.1 90.3 119.3 64.34 40.10 62 1.038 3018K 1716 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.7 2.5 3.1 5.6 2.5 0.0 0.0 106.7 106.3 54.43 34.06 34 1.601 3018K 1716 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 190.7 9.91 6.04 28 0.354 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.3 total, 16.1 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.49 GB write, 470.12 MB/s write, 5.67 GB read, 355.92 MB/s read, 64.3 seconds Interval compaction: 7.49 GB write, 476.70 MB/s write, 5.67 GB read, 360.90 MB/s read, 64.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 14, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 14, interval: 14 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 962227 Average: 11.1243 StdDev: 98.49 Min: 1 Median: 2.2670 Max: 24914 Percentiles: P50: 2.27 P75: 2.94 P99: 224.53 P99.9: 576.30 P99.99: 867.49 ------------------------------------------------------ [ 0, 1 ] 23573 2.450% 2.450% ( 1, 2 ] 362197 37.642% 40.091% ######## ( 2, 3 ] 357050 37.107% 77.198% ####### ( 3, 4 ] 68464 7.115% 84.313% # ( 4, 6 ] 81141 8.433% 92.746% ## ( 6, 10 ] 34538 3.589% 96.335% # ( 10, 15 ] 7546 0.784% 97.119% ( 15, 22 ] 1530 0.159% 97.278% ( 22, 34 ] 264 0.027% 97.306% ( 34, 51 ] 41 0.004% 97.310% ( 51, 76 ] 22 0.002% 97.312% ( 76, 110 ] 528 0.055% 97.367% ( 110, 170 ] 9006 0.936% 98.303% ( 170, 250 ] 9838 1.022% 99.326% ( 250, 380 ] 3551 0.369% 99.695% ( 380, 580 ] 2014 0.209% 99.904% ( 580, 870 ] 836 0.087% 99.991% ( 870, 1300 ] 237 0.025% 100.015% ( 1300, 1900 ] 135 0.014% 100.030% ( 1900, 2900 ] 127 0.013% 100.043% ( 2900, 4400 ] 165 0.017% 100.060% ( 4400, 6600 ] 59 0.006% 100.066% ( 6600, 9900 ] 15 0.002% 100.068% ( 9900, 14000 ] 1 0.000% 100.068% ( 14000, 22000 ] 2 0.000% 100.068% ( 22000, 33000 ] 2 0.000% 100.068% ** Level 1 read latency histogram (micros): Count: 118935 Average: 11.0785 StdDev: 149.89 Min: 1 Median: 1.9323 Max: 16890 Percentiles: P50: 1.93 P75: 2.82 P99: 218.67 P99.9: 780.50 P99.99: 5254.29 ------------------------------------------------------ [ 0, 1 ] 8790 7.391% 7.391% # ( 1, 2 ] 54357 45.703% 53.094% ######### ( 2, 3 ] 31699 26.652% 79.746% ##### ( 3, 4 ] 8202 6.896% 86.642% # ( 4, 6 ] 9063 7.620% 94.262% ## ( 6, 10 ] 2835 2.384% 96.646% ( 10, 15 ] 635 0.534% 97.180% ( 15, 22 ] 191 0.161% 97.341% ( 22, 34 ] 15 0.013% 97.353% ( 34, 51 ] 3 0.003% 97.356% ( 51, 76 ] 3 0.003% 97.358% ( 76, 110 ] 76 0.064% 97.422% ( 110, 170 ] 1244 1.046% 98.468% ( 170, 250 ] 1040 0.874% 99.342% ( 250, 380 ] 396 0.333% 99.675% ( 380, 580 ] 200 0.168% 99.844% ( 580, 870 ] 97 0.082% 99.925% ( 870, 1300 ] 34 0.029% 99.954% ( 1300, 1900 ] 14 0.012% 99.966% ( 1900, 2900 ] 13 0.011% 99.976% ( 2900, 4400 ] 13 0.011% 99.987% ( 4400, 6600 ] 8 0.007% 99.994% ( 6600, 9900 ] 2 0.002% 99.996% ( 9900, 14000 ] 2 0.002% 99.997% ( 14000, 22000 ] 6 0.005% 100.003% ** Level 2 read latency histogram (micros): Count: 14468 Average: 24.0923 StdDev: 256.97 Min: 1 Median: 2.8352 Max: 14362 Percentiles: P50: 2.84 P75: 4.77 P99: 280.04 P99.9: 4074.88 P99.99: 9162.78 ------------------------------------------------------ [ 0, 1 ] 145 1.002% 1.002% ( 1, 2 ] 2974 20.556% 21.558% #### ( 2, 3 ] 4927 34.054% 55.612% ####### ( 3, 4 ] 1842 12.732% 68.344% ### ( 4, 6 ] 2501 17.286% 85.630% ### ( 6, 10 ] 1384 9.566% 95.196% ## ( 10, 15 ] 267 1.845% 97.042% ( 15, 22 ] 61 0.422% 97.463% ( 22, 34 ] 12 0.083% 97.546% ( 34, 51 ] 3 0.021% 97.567% ( 51, 76 ] 2 0.014% 97.581% ( 76, 110 ] 5 0.035% 97.615% ( 110, 170 ] 101 0.698% 98.314% ( 170, 250 ] 91 0.629% 98.942% ( 250, 380 ] 36 0.249% 99.191% ( 380, 580 ] 22 0.152% 99.343% ( 580, 870 ] 16 0.111% 99.454% ( 870, 1300 ] 8 0.055% 99.509% ( 1300, 1900 ] 16 0.111% 99.620% ( 1900, 2900 ] 28 0.194% 99.813% ( 2900, 4400 ] 16 0.111% 99.924% ( 4400, 6600 ] 8 0.055% 99.979% ( 6600, 9900 ] 2 0.014% 99.993% ( 14000, 22000 ] 1 0.007% 100.000% ** Level 3 read latency histogram (micros): Count: 556517 Average: 13.0484 StdDev: 194.07 Min: 1 Median: 2.8248 Max: 39941 Percentiles: P50: 2.82 P75: 4.64 P99: 14.48 P99.9: 240.10 P99.99: 457.67 ------------------------------------------------------ [ 0, 1 ] 2178 0.391% 0.391% ( 1, 2 ] 97633 17.544% 17.935% #### ( 2, 3 ] 216363 38.878% 56.813% ######## ( 3, 4 ] 67339 12.100% 68.913% ## ( 4, 6 ] 105347 18.930% 87.843% #### ( 6, 10 ] 53416 9.598% 97.441% ## ( 10, 15 ] 9673 1.738% 99.179% ( 15, 22 ] 2052 0.369% 99.548% ( 22, 34 ] 324 0.058% 99.606% ( 34, 51 ] 92 0.017% 99.623% ( 51, 76 ] 43 0.008% 99.630% ( 76, 110 ] 72 0.013% 99.643% ( 110, 170 ] 766 0.138% 99.781% ( 170, 250 ] 756 0.136% 99.917% ( 250, 380 ] 299 0.054% 99.971% ( 380, 580 ] 279 0.050% 100.021% ( 580, 870 ] 275 0.049% 100.070% ( 870, 1300 ] 291 0.052% 100.122% ( 1300, 1900 ] 249 0.045% 100.167% ( 1900, 2900 ] 319 0.057% 100.224% ( 2900, 4400 ] 411 0.074% 100.298% ( 4400, 6600 ] 168 0.030% 100.328% ( 6600, 9900 ] 36 0.006% 100.335% ( 9900, 14000 ] 3 0.001% 100.335% ( 14000, 22000 ] 4 0.001% 100.336% ( 22000, 33000 ] 4 0.001% 100.337% ( 33000, 50000 ] 1 0.000% 100.337% ** Level 4 read latency histogram (micros): Count: 250618 Average: 46.6933 StdDev: 325.16 Min: 1 Median: 2.8194 Max: 39783 Percentiles: P50: 2.82 P75: 4.85 P99: 856.64 P99.9: 3428.09 P99.99: 4103.41 ------------------------------------------------------ [ 0, 1 ] 1012 0.404% 0.404% ( 1, 2 ] 45167 18.022% 18.426% #### ( 2, 3 ] 96576 38.535% 56.961% ######## ( 3, 4 ] 28825 11.502% 68.463% ## ( 4, 6 ] 38703 15.443% 83.906% ### ( 6, 10 ] 16418 6.551% 90.457% # ( 10, 15 ] 3763 1.501% 91.958% ( 15, 22 ] 708 0.283% 92.241% ( 22, 34 ] 95 0.038% 92.279% ( 34, 51 ] 25 0.010% 92.289% ( 51, 76 ] 26 0.010% 92.299% ( 76, 110 ] 390 0.156% 92.455% ( 110, 170 ] 6214 2.479% 94.934% ( 170, 250 ] 5772 2.303% 97.237% ( 250, 380 ] 2170 0.866% 98.103% ( 380, 580 ] 1417 0.565% 98.668% ( 580, 870 ] 872 0.348% 99.016% ( 870, 1300 ] 724 0.289% 99.305% ( 1300, 1900 ] 662 0.264% 99.569% ( 1900, 2900 ] 653 0.261% 99.830% ( 2900, 4400 ] 501 0.200% 100.030% ( 4400, 6600 ] 231 0.092% 100.122% ( 6600, 9900 ] 42 0.017% 100.139% ( 14000, 22000 ] 2 0.001% 100.140% ( 33000, 50000 ] 3 0.001% 100.141% ** Level 5 read latency histogram (micros): Count: 132796 Average: 336.6965 StdDev: 766.60 Min: 85 Median: 194.4038 Max: 77547 Percentiles: P50: 194.40 P75: 265.50 P99: 3925.03 P99.9: 7965.47 P99.99: 13150.33 ------------------------------------------------------ ( 76, 110 ] 3046 2.294% 2.294% ( 110, 170 ] 49602 37.352% 39.646% ####### ( 170, 250 ] 45075 33.943% 73.589% ####### ( 250, 380 ] 15717 11.835% 85.424% ## ( 380, 580 ] 9340 7.033% 92.458% # ( 580, 870 ] 4052 3.051% 95.509% # ( 870, 1300 ] 1622 1.221% 96.730% ( 1300, 1900 ] 1137 0.856% 97.587% ( 1900, 2900 ] 1070 0.806% 98.392% ( 2900, 4400 ] 1181 0.889% 99.282% ( 4400, 6600 ] 743 0.560% 99.841% ( 6600, 9900 ] 189 0.142% 99.983% ( 9900, 14000 ] 11 0.008% 99.992% ( 14000, 22000 ] 6 0.005% 99.996% ( 33000, 50000 ] 2 0.002% 99.998% ( 50000, 75000 ] 3 0.002% 100.000% ( 75000, 110000 ] 1 0.001% 100.001% ** DB Stats ** Uptime(secs): 16.3 total, 8.7 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 120.58 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.197 H:M:S, 7.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 113.73 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/10-13:20:41 ... thread 0: (1000000,3000000) ops and (30671.5,61602.1) ops/second in (32.603602,48.699621) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/8 2.82 GB 75.5 3.2 0.0 3.2 6.0 2.8 0.0 2.1 96.8 182.4 33.82 25.79 53 0.638 3325K 5118 0.0 0.0 L1 15/15 1.76 GB 0.0 3.2 1.6 1.6 3.1 1.6 0.0 1.9 80.0 79.2 40.55 23.68 1 40.551 3293K 32K 0.0 0.0 L2 1/0 49.38 MB 0.5 0.0 0.0 0.0 0.0 0.0 1.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 21/0 975.78 MB 1.0 2.6 1.2 1.4 2.6 1.2 0.7 2.1 67.7 67.1 39.57 23.49 19 2.083 1371K 7065 0.0 0.0 L4 164/0 9.57 GB 1.0 5.9 3.3 2.6 5.9 3.2 0.0 1.8 100.3 99.2 60.70 38.30 41 1.480 3126K 10K 0.0 0.0 L5 1096/0 91.84 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1310/23 107.00 GB 0.0 14.9 6.2 8.8 17.6 8.9 5.5 6.2 87.5 103.4 174.64 111.26 114 1.532 11M 55K 0.0 0.0 Int 0/0 0.00 KB 0.0 14.9 6.2 8.8 17.6 8.9 5.5 6.2 87.5 103.4 174.64 111.26 114 1.532 11M 55K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.9 6.2 8.8 14.8 6.0 0.0 0.0 97.3 96.5 157.13 101.15 71 2.213 11M 55K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 165.5 17.51 10.11 43 0.407 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 48.9 total, 48.7 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 17.63 GB write, 368.99 MB/s write, 14.93 GB read, 312.37 MB/s read, 174.6 seconds Interval compaction: 17.63 GB write, 370.69 MB/s write, 14.93 GB read, 313.82 MB/s read, 174.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 23, pending-compaction-bytes-delays: 21, pending-compaction-bytes-stops: 0, total-delays: 21, total-stops: 23, interval: 44 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1354211 Average: 30.2485 StdDev: 244.81 Min: 1 Median: 2.3579 Max: 49941 Percentiles: P50: 2.36 P75: 3.60 P99: 343.11 P99.9: 1254.46 P99.99: 2550.30 ------------------------------------------------------ [ 0, 1 ] 44641 3.296% 3.296% # ( 1, 2 ] 473939 34.997% 38.294% ####### ( 2, 3 ] 442922 32.707% 71.001% ####### ( 3, 4 ] 90096 6.653% 77.654% # ( 4, 6 ] 103011 7.607% 85.261% ## ( 6, 10 ] 42453 3.135% 88.396% # ( 10, 15 ] 8325 0.615% 89.010% ( 15, 22 ] 1641 0.121% 89.131% ( 22, 34 ] 278 0.021% 89.152% ( 34, 51 ] 44 0.003% 89.155% ( 51, 76 ] 28 0.002% 89.157% ( 76, 110 ] 2803 0.207% 89.364% ( 110, 170 ] 64892 4.792% 94.156% # ( 170, 250 ] 58250 4.301% 98.458% # ( 250, 380 ] 10256 0.757% 99.215% ( 380, 580 ] 5184 0.383% 99.598% ( 580, 870 ] 2716 0.201% 99.798% ( 870, 1300 ] 1541 0.114% 99.912% ( 1300, 1900 ] 774 0.057% 99.969% ( 1900, 2900 ] 433 0.032% 100.001% ( 2900, 4400 ] 450 0.033% 100.034% ( 4400, 6600 ] 192 0.014% 100.049% ( 6600, 9900 ] 29 0.002% 100.051% ( 9900, 14000 ] 6 0.000% 100.051% ( 14000, 22000 ] 25 0.002% 100.053% ( 22000, 33000 ] 32 0.002% 100.055% ( 33000, 50000 ] 16 0.001% 100.057% ** Level 1 read latency histogram (micros): Count: 669423 Average: 17.2453 StdDev: 383.27 Min: 1 Median: 2.9766 Max: 47319 Percentiles: P50: 2.98 P75: 4.42 P99: 180.00 P99.9: 789.12 P99.99: 1393.37 ------------------------------------------------------ [ 0, 1 ] 10950 1.636% 1.636% ( 1, 2 ] 141746 21.174% 22.810% #### ( 2, 3 ] 186378 27.842% 50.652% ###### ( 3, 4 ] 133315 19.915% 70.567% #### ( 4, 6 ] 141484 21.135% 91.702% #### ( 6, 10 ] 30150 4.504% 96.206% # ( 10, 15 ] 7031 1.050% 97.256% ( 15, 22 ] 3961 0.592% 97.848% ( 22, 34 ] 219 0.033% 97.880% ( 34, 51 ] 56 0.008% 97.889% ( 51, 76 ] 88 0.013% 97.902% ( 76, 110 ] 371 0.055% 97.957% ( 110, 170 ] 6395 0.955% 98.913% ( 170, 250 ] 4679 0.699% 99.612% ( 250, 380 ] 889 0.133% 99.744% ( 380, 580 ] 632 0.094% 99.839% ( 580, 870 ] 568 0.085% 99.924% ( 870, 1300 ] 412 0.062% 99.985% ( 1300, 1900 ] 206 0.031% 100.016% ( 1900, 2900 ] 99 0.015% 100.031% ( 2900, 4400 ] 148 0.022% 100.053% ( 4400, 6600 ] 55 0.008% 100.061% ( 6600, 9900 ] 44 0.007% 100.068% ( 9900, 14000 ] 38 0.006% 100.073% ( 14000, 22000 ] 92 0.014% 100.087% ( 22000, 33000 ] 66 0.010% 100.097% ( 33000, 50000 ] 7 0.001% 100.098% ** Level 2 read latency histogram (micros): Count: 29481 Average: 45.1595 StdDev: 780.63 Min: 1 Median: 2.8101 Max: 55553 Percentiles: P50: 2.81 P75: 4.77 P99: 265.32 P99.9: 10678.49 P99.99: 33441.15 ------------------------------------------------------ [ 0, 1 ] 224 0.760% 0.760% ( 1, 2 ] 5439 18.449% 19.209% #### ( 2, 3 ] 11205 38.008% 57.217% ######## ( 3, 4 ] 3430 11.635% 68.851% ## ( 4, 6 ] 4716 15.997% 84.848% ### ( 6, 10 ] 2593 8.795% 93.643% ## ( 10, 15 ] 497 1.686% 95.329% ( 15, 22 ] 77 0.261% 95.590% ( 22, 34 ] 14 0.047% 95.638% ( 34, 51 ] 7 0.024% 95.662% ( 51, 76 ] 11 0.037% 95.699% ( 76, 110 ] 29 0.098% 95.797% ( 110, 170 ] 541 1.835% 97.632% ( 170, 250 ] 396 1.343% 98.976% ( 250, 380 ] 61 0.207% 99.183% ( 380, 580 ] 42 0.142% 99.325% ( 580, 870 ] 31 0.105% 99.430% ( 870, 1300 ] 30 0.102% 99.532% ( 1300, 1900 ] 35 0.119% 99.651% ( 1900, 2900 ] 31 0.105% 99.756% ( 2900, 4400 ] 26 0.088% 99.844% ( 4400, 6600 ] 11 0.037% 99.881% ( 6600, 9900 ] 4 0.014% 99.895% ( 9900, 14000 ] 8 0.027% 99.922% ( 14000, 22000 ] 10 0.034% 99.956% ( 22000, 33000 ] 10 0.034% 99.990% ( 33000, 50000 ] 2 0.007% 99.997% ( 50000, 75000 ] 1 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 1012176 Average: 16.4412 StdDev: 370.13 Min: 1 Median: 2.8469 Max: 60309 Percentiles: P50: 2.85 P75: 4.71 P99: 20.19 P99.9: 247.99 P99.99: 451.97 ------------------------------------------------------ [ 0, 1 ] 4225 0.417% 0.417% ( 1, 2 ] 176456 17.433% 17.851% ### ( 2, 3 ] 384255 37.963% 55.814% ######## ( 3, 4 ] 124226 12.273% 68.087% ## ( 4, 6 ] 196218 19.386% 87.473% #### ( 6, 10 ] 97695 9.652% 97.125% ## ( 10, 15 ] 16461 1.626% 98.751% ( 15, 22 ] 3399 0.336% 99.087% ( 22, 34 ] 466 0.046% 99.133% ( 34, 51 ] 205 0.020% 99.153% ( 51, 76 ] 219 0.022% 99.175% ( 76, 110 ] 337 0.033% 99.208% ( 110, 170 ] 4122 0.407% 99.615% ( 170, 250 ] 2954 0.292% 99.907% ( 250, 380 ] 627 0.062% 99.969% ( 380, 580 ] 583 0.058% 100.027% ( 580, 870 ] 537 0.053% 100.080% ( 870, 1300 ] 527 0.052% 100.132% ( 1300, 1900 ] 377 0.037% 100.169% ( 1900, 2900 ] 403 0.040% 100.209% ( 2900, 4400 ] 534 0.053% 100.262% ( 4400, 6600 ] 238 0.024% 100.285% ( 6600, 9900 ] 70 0.007% 100.292% ( 9900, 14000 ] 38 0.004% 100.296% ( 14000, 22000 ] 92 0.009% 100.305% ( 22000, 33000 ] 72 0.007% 100.312% ( 33000, 50000 ] 17 0.002% 100.314% ( 50000, 75000 ] 2 0.000% 100.314% ** Level 4 read latency histogram (micros): Count: 753837 Average: 40.4760 StdDev: 321.08 Min: 0 Median: 2.9604 Max: 43064 Percentiles: P50: 2.96 P75: 5.45 P99: 412.40 P99.9: 1597.31 P99.99: 1987.06 ------------------------------------------------------ [ 0, 1 ] 3723 0.494% 0.494% ( 1, 2 ] 127839 16.958% 17.452% ### ( 2, 3 ] 255481 33.891% 51.343% ####### ( 3, 4 ] 85855 11.389% 62.732% ## ( 4, 6 ] 127540 16.919% 79.651% ### ( 6, 10 ] 59654 7.913% 87.564% ## ( 10, 15 ] 12331 1.636% 89.200% ( 15, 22 ] 2431 0.322% 89.523% ( 22, 34 ] 339 0.045% 89.568% ( 34, 51 ] 115 0.015% 89.583% ( 51, 76 ] 201 0.027% 89.609% ( 76, 110 ] 2331 0.309% 89.919% ( 110, 170 ] 37678 4.998% 94.917% # ( 170, 250 ] 25673 3.406% 98.322% # ( 250, 380 ] 4633 0.615% 98.937% ( 380, 580 ] 2930 0.389% 99.326% ( 580, 870 ] 2035 0.270% 99.596% ( 870, 1300 ] 1705 0.226% 99.822% ( 1300, 1900 ] 1189 0.158% 99.980% ( 1900, 2900 ] 903 0.120% 100.099% ( 2900, 4400 ] 856 0.114% 100.213% ( 4400, 6600 ] 346 0.046% 100.259% ( 6600, 9900 ] 70 0.009% 100.268% ( 9900, 14000 ] 17 0.002% 100.270% ( 14000, 22000 ] 39 0.005% 100.276% ( 22000, 33000 ] 20 0.003% 100.278% ( 33000, 50000 ] 7 0.001% 100.279% ** Level 5 read latency histogram (micros): Count: 551575 Average: 245.6111 StdDev: 693.03 Min: 83 Median: 166.7489 Max: 77547 Percentiles: P50: 166.75 P75: 221.89 P99: 1807.14 P99.9: 6440.09 P99.99: 31365.37 ------------------------------------------------------ ( 76, 110 ] 16828 3.051% 3.051% # ( 110, 170 ] 273795 49.639% 52.690% ########## ( 170, 250 ] 189721 34.396% 87.086% ####### ( 250, 380 ] 32770 5.941% 93.027% # ( 380, 580 ] 17555 3.183% 96.210% # ( 580, 870 ] 8557 1.551% 97.761% ( 870, 1300 ] 4591 0.832% 98.593% ( 1300, 1900 ] 2654 0.481% 99.075% ( 1900, 2900 ] 1812 0.329% 99.403% ( 2900, 4400 ] 1790 0.325% 99.728% ( 4400, 6600 ] 1026 0.186% 99.914% ( 6600, 9900 ] 243 0.044% 99.958% ( 9900, 14000 ] 33 0.006% 99.964% ( 14000, 22000 ] 59 0.011% 99.974% ( 22000, 33000 ] 102 0.018% 99.993% ( 33000, 50000 ] 32 0.006% 99.999% ( 50000, 75000 ] 10 0.002% 100.001% ( 75000, 110000 ] 1 0.000% 100.001% ** DB Stats ** Uptime(secs): 48.9 total, 32.6 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 60.35 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:27.669 H:M:S, 56.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 30.19 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:26.472 H:M:S, 81.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 285.772 micros/op 13989 ops/sec 60.118 seconds 840996 operations; 13.6 MB/s (209999 of 209999 found) Microseconds per read: Count: 840996 Average: 285.7750 StdDev: 1013.66 Min: 2 Median: 209.1315 Max: 166523 Percentiles: P50: 209.13 P75: 248.12 P99: 1229.65 P99.9: 16480.56 P99.99: 42511.27 ------------------------------------------------------ ( 1, 2 ] 8 0.001% 0.001% ( 2, 3 ] 38 0.005% 0.005% ( 3, 4 ] 77 0.009% 0.015% ( 4, 6 ] 68 0.008% 0.023% ( 6, 10 ] 384 0.046% 0.068% ( 10, 15 ] 2915 0.347% 0.415% ( 15, 22 ] 2857 0.340% 0.755% ( 22, 34 ] 4923 0.585% 1.340% ( 34, 51 ] 2258 0.268% 1.609% ( 51, 76 ] 225 0.027% 1.635% ( 76, 110 ] 1057 0.126% 1.761% ( 110, 170 ] 194665 23.147% 24.908% ##### ( 170, 250 ] 431413 51.298% 76.206% ########## ( 250, 380 ] 113967 13.551% 89.757% ### ( 380, 580 ] 53586 6.372% 96.129% # ( 580, 870 ] 18563 2.207% 98.336% ( 870, 1300 ] 6674 0.794% 99.130% ( 1300, 1900 ] 2673 0.318% 99.448% ( 1900, 2900 ] 1510 0.180% 99.627% ( 2900, 4400 ] 1072 0.127% 99.755% ( 4400, 6600 ] 607 0.072% 99.827% ( 6600, 9900 ] 242 0.029% 99.856% ( 9900, 14000 ] 222 0.026% 99.882% ( 14000, 22000 ] 487 0.058% 99.940% ( 22000, 33000 ] 356 0.042% 99.982% ( 33000, 50000 ] 116 0.014% 99.996% ( 50000, 75000 ] 26 0.003% 99.999% ( 75000, 110000 ] 3 0.000% 100.000% ( 110000, 170000 ] 4 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 10668079 rocksdb.bloom.filter.full.positive COUNT : 944842 rocksdb.bloom.filter.full.true.positive COUNT : 840634 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 362 rocksdb.memtable.miss COUNT : 840634 rocksdb.l0.hit COUNT : 29085 rocksdb.l1.hit COUNT : 13616 rocksdb.l2andup.hit COUNT : 797933 rocksdb.compaction.key.drop.new COUNT : 139183 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3566228 rocksdb.number.keys.read COUNT : 840996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3680347296 rocksdb.bytes.read COUNT : 840996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1547 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 35953458 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 18474413184 rocksdb.compact.write.bytes COUNT : 18324601144 rocksdb.flush.write.bytes COUNT : 3633315377 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 34 rocksdb.number.superversion_acquires COUNT : 798 rocksdb.number.superversion_releases COUNT : 765 rocksdb.number.superversion_cleanups COUNT : 199 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 131 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3603876864 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1275904 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 22011464115 rocksdb.non.last.level.read.count COUNT : 5174406 rocksdb.block.checksum.compute.count COUNT : 5426529 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 208.481485 P95 : 543.815392 P99 : 1228.745444 P100 : 166520.000000 COUNT : 840996 SUM : 239655112 rocksdb.db.write.micros P50 : 2.966734 P95 : 9.167382 P99 : 17.263633 P100 : 1979679.000000 COUNT : 3566228 SUM : 51826549 rocksdb.compaction.times.micros P50 : 1627777.777778 P95 : 3483333.333333 P99 : 37000000.000000 P100 : 40551096.000000 COUNT : 75 SUM : 164972772 rocksdb.compaction.times.cpu_micros P50 : 1051250.000000 P95 : 2650000.000000 P99 : 23680582.000000 P100 : 23680582.000000 COUNT : 75 SUM : 108876138 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 1192.500000 P95 : 11540.000000 P99 : 15483.000000 P100 : 15483.000000 COUNT : 52 SUM : 131361 rocksdb.compaction.outfile.sync.micros P50 : 4950.000000 P95 : 30983.333333 P99 : 44936.000000 P100 : 44936.000000 COUNT : 213 SUM : 1714516 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 455.000000 P95 : 5458.750000 P99 : 9190.500000 P100 : 30256.000000 COUNT : 143 SUM : 198992 rocksdb.table.open.io.micros P50 : 9516.732283 P95 : 25516.666667 P99 : 32392.222222 P100 : 144436.000000 COUNT : 1547 SUM : 16592240 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 5.234591 P95 : 14.607173 P99 : 236.727077 P100 : 60314.000000 COUNT : 4472368 SUM : 95271629 rocksdb.read.block.get.micros P50 : 170.930143 P95 : 434.029918 P99 : 1203.570821 P100 : 77603.000000 COUNT : 952587 SUM : 210422810 rocksdb.write.raw.block.micros P50 : 0.591267 P95 : 1.836232 P99 : 4.585751 P100 : 92222.000000 COUNT : 5318173 SUM : 22391473 rocksdb.numfiles.in.singlecompaction P50 : 2.700000 P95 : 4.100000 P99 : 22.360000 P100 : 23.000000 COUNT : 97 SUM : 332 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.567013 P95 : 1221.770991 P99 : 2624.123981 P100 : 1979475.000000 COUNT : 201082 SUM : 35973217 rocksdb.sst.read.micros P50 : 2.845352 P95 : 225.998678 P99 : 523.996728 P100 : 77547.000000 COUNT : 5174417 SUM : 286059410 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 840996 SUM : 840996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3566228 SUM : 3680347296 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 239333.333333 P95 : 840666.666667 P99 : 2347570.000000 P100 : 2347570.000000 COUNT : 52 SUM : 19734927 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 241 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0