[2024-12-17 00:00:12.157327] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 22.11.4 initialization... [2024-12-17 00:00:12.157472] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid705674 ] Set seed to 1734390012153845 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-17 00:00:12.243629] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-17 00:00:12.280930] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-17 00:00:12.280975] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 705674' to capture a snapshot of events at runtime. [2024-12-17 00:00:12.280987] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid705674 for offline analysis/debug. [2024-12-17 00:00:12.281011] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Tue Dec 17 00:00:16 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/17-00:00:26 ... thread 0: (1000000,1000000) ops and (100427.7,100427.7) ops/second in (9.957413,9.957413) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/7 1.98 GB 44.0 0.4 0.0 0.4 1.2 0.8 0.0 1.5 37.5 118.7 10.08 4.48 10 1.008 383K 543 0.0 0.0 L1 7/7 823.05 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 44.60 MB 0.4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 21/5 1.46 GB 0.9 3.2 1.8 1.5 3.2 1.7 0.0 1.8 110.5 109.3 30.11 15.67 13 2.316 1495K 0 0.0 0.0 L4 175/13 10.26 GB 1.0 4.6 3.4 1.1 4.5 3.4 0.0 1.3 99.2 98.3 47.13 23.99 31 1.520 2489K 610 0.0 0.0 L5 1112/0 91.35 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1325/32 105.90 GB 0.0 8.2 5.2 3.0 8.9 5.9 2.6 11.1 95.9 104.4 87.32 44.14 54 1.617 4368K 1153 0.0 0.0 Int 0/0 0.00 KB 0.0 8.2 5.2 3.0 8.9 5.9 2.6 11.1 95.9 104.4 87.32 44.14 54 1.617 4368K 1153 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 8.2 5.2 3.0 8.1 5.1 0.0 0.0 104.8 103.9 79.93 41.54 45 1.776 4368K 1153 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 110.8 7.39 2.61 9 0.821 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 10.1 total, 10.0 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.91 GB write, 900.75 MB/s write, 8.18 GB read, 827.39 MB/s read, 87.3 seconds Interval compaction: 8.91 GB write, 915.79 MB/s write, 8.18 GB read, 841.21 MB/s read, 87.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 33, pending-compaction-bytes-delays: 12, pending-compaction-bytes-stops: 0, total-delays: 12, total-stops: 33, interval: 44 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 398940 Average: 14.5716 StdDev: 287.52 Min: 1 Median: 1.8743 Max: 41880 Percentiles: P50: 1.87 P75: 2.97 P99: 11.70 P99.9: 2724.29 P99.99: 6417.27 ------------------------------------------------------ [ 0, 1 ] 20185 5.060% 5.060% # ( 1, 2 ] 205053 51.399% 56.459% ########## ( 2, 3 ] 76613 19.204% 75.663% #### ( 3, 4 ] 39462 9.892% 85.555% ## ( 4, 6 ] 42871 10.746% 96.301% ## ( 6, 10 ] 10028 2.514% 98.815% # ( 10, 15 ] 2177 0.546% 99.361% ( 15, 22 ] 274 0.069% 99.429% ( 22, 34 ] 28 0.007% 99.436% ( 34, 51 ] 34 0.009% 99.445% ( 51, 76 ] 30 0.008% 99.452% ( 76, 110 ] 14 0.004% 99.456% ( 110, 170 ] 22 0.006% 99.461% ( 170, 250 ] 46 0.012% 99.473% ( 250, 380 ] 121 0.030% 99.503% ( 380, 580 ] 244 0.061% 99.564% ( 580, 870 ] 439 0.110% 99.674% ( 870, 1300 ] 408 0.102% 99.777% ( 1300, 1900 ] 300 0.075% 99.852% ( 1900, 2900 ] 233 0.058% 99.910% ( 2900, 4400 ] 242 0.061% 99.971% ( 4400, 6600 ] 83 0.021% 99.992% ( 6600, 9900 ] 33 0.008% 100.000% ( 9900, 14000 ] 19 0.005% 100.005% ( 14000, 22000 ] 33 0.008% 100.013% ( 22000, 33000 ] 13 0.003% 100.016% ( 33000, 50000 ] 2 0.001% 100.017% ** Level 1 read latency histogram (micros): Count: 4770 Average: 103.2411 StdDev: 772.01 Min: 1 Median: 1.9594 Max: 31752 Percentiles: P50: 1.96 P75: 3.35 P99: 2787.50 P99.9: 11581.00 P99.99: 27753.00 ------------------------------------------------------ [ 0, 1 ] 139 2.914% 2.914% # ( 1, 2 ] 2341 49.078% 51.992% ########## ( 2, 3 ] 957 20.063% 72.055% #### ( 3, 4 ] 398 8.344% 80.398% ## ( 4, 6 ] 534 11.195% 91.593% ## ( 6, 10 ] 128 2.683% 94.277% # ( 10, 15 ] 22 0.461% 94.738% ( 34, 51 ] 4 0.084% 94.822% ( 51, 76 ] 4 0.084% 94.906% ( 110, 170 ] 1 0.021% 94.927% ( 170, 250 ] 3 0.063% 94.990% ( 250, 380 ] 8 0.168% 95.157% ( 380, 580 ] 27 0.566% 95.723% ( 580, 870 ] 54 1.132% 96.855% ( 870, 1300 ] 48 1.006% 97.862% ( 1300, 1900 ] 33 0.692% 98.553% ( 1900, 2900 ] 24 0.503% 99.057% ( 2900, 4400 ] 28 0.587% 99.644% ( 4400, 6600 ] 9 0.189% 99.832% ( 6600, 9900 ] 2 0.042% 99.874% ( 9900, 14000 ] 3 0.063% 99.937% ( 14000, 22000 ] 2 0.042% 99.979% ( 22000, 33000 ] 1 0.021% 100.000% ** Level 2 read latency histogram (micros): Count: 318401 Average: 13.8157 StdDev: 217.62 Min: 1 Median: 1.9152 Max: 46994 Percentiles: P50: 1.92 P75: 3.10 P99: 12.79 P99.9: 2174.89 P99.99: 3641.45 ------------------------------------------------------ [ 0, 1 ] 14632 4.595% 4.595% # ( 1, 2 ] 157968 49.613% 54.208% ########## ( 2, 3 ] 62911 19.758% 73.967% #### ( 3, 4 ] 33217 10.432% 84.399% ## ( 4, 6 ] 36009 11.309% 95.709% ## ( 6, 10 ] 9239 2.902% 98.610% # ( 10, 15 ] 2224 0.698% 99.309% ( 15, 22 ] 295 0.093% 99.401% ( 22, 34 ] 44 0.014% 99.415% ( 34, 51 ] 17 0.005% 99.421% ( 51, 76 ] 23 0.007% 99.428% ( 76, 110 ] 5 0.002% 99.429% ( 110, 170 ] 21 0.007% 99.436% ( 170, 250 ] 43 0.014% 99.449% ( 250, 380 ] 93 0.029% 99.479% ( 380, 580 ] 248 0.078% 99.557% ( 580, 870 ] 315 0.099% 99.655% ( 870, 1300 ] 398 0.125% 99.780% ( 1300, 1900 ] 316 0.099% 99.880% ( 1900, 2900 ] 235 0.074% 99.954% ( 2900, 4400 ] 235 0.074% 100.027% ( 4400, 6600 ] 63 0.020% 100.047% ( 6600, 9900 ] 25 0.008% 100.055% ( 9900, 14000 ] 4 0.001% 100.056% ( 14000, 22000 ] 6 0.002% 100.058% ( 33000, 50000 ] 2 0.001% 100.059% ** Level 3 read latency histogram (micros): Count: 1214490 Average: 12.9289 StdDev: 253.50 Min: 1 Median: 1.9127 Max: 61745 Percentiles: P50: 1.91 P75: 3.03 P99: 9.87 P99.9: 1009.63 P99.99: 1573.39 ------------------------------------------------------ [ 0, 1 ] 45458 3.743% 3.743% # ( 1, 2 ] 615504 50.680% 54.423% ########## ( 2, 3 ] 245638 20.226% 74.649% #### ( 3, 4 ] 128756 10.602% 85.250% ## ( 4, 6 ] 138823 11.431% 96.681% ## ( 6, 10 ] 29141 2.399% 99.080% ( 10, 15 ] 6595 0.543% 99.623% ( 15, 22 ] 800 0.066% 99.689% ( 22, 34 ] 102 0.008% 99.698% ( 34, 51 ] 65 0.005% 99.703% ( 51, 76 ] 80 0.007% 99.710% ( 76, 110 ] 39 0.003% 99.713% ( 110, 170 ] 49 0.004% 99.717% ( 170, 250 ] 102 0.008% 99.725% ( 250, 380 ] 265 0.022% 99.747% ( 380, 580 ] 571 0.047% 99.794% ( 580, 870 ] 942 0.078% 99.872% ( 870, 1300 ] 1064 0.088% 99.959% ( 1300, 1900 ] 822 0.068% 100.027% ( 1900, 2900 ] 768 0.063% 100.090% ( 2900, 4400 ] 819 0.067% 100.158% ( 4400, 6600 ] 242 0.020% 100.177% ( 6600, 9900 ] 82 0.007% 100.184% ( 9900, 14000 ] 27 0.002% 100.186% ( 14000, 22000 ] 48 0.004% 100.190% ( 22000, 33000 ] 27 0.002% 100.193% ( 33000, 50000 ] 5 0.000% 100.193% ( 50000, 75000 ] 1 0.000% 100.193% ** Level 4 read latency histogram (micros): Count: 375296 Average: 33.3457 StdDev: 401.55 Min: 1 Median: 1.9179 Max: 56153 Percentiles: P50: 1.92 P75: 3.07 P99: 858.36 P99.9: 3689.87 P99.99: 4635.40 ------------------------------------------------------ [ 0, 1 ] 13741 3.661% 3.661% # ( 1, 2 ] 189468 50.485% 54.146% ########## ( 2, 3 ] 75420 20.096% 74.242% #### ( 3, 4 ] 39616 10.556% 84.798% ## ( 4, 6 ] 39273 10.465% 95.263% ## ( 6, 10 ] 9171 2.444% 97.707% ( 10, 15 ] 2131 0.568% 98.274% ( 15, 22 ] 247 0.066% 98.340% ( 22, 34 ] 21 0.006% 98.346% ( 34, 51 ] 20 0.005% 98.351% ( 51, 76 ] 28 0.007% 98.359% ( 76, 110 ] 22 0.006% 98.364% ( 110, 170 ] 48 0.013% 98.377% ( 170, 250 ] 173 0.046% 98.423% ( 250, 380 ] 276 0.074% 98.497% ( 380, 580 ] 669 0.178% 98.675% ( 580, 870 ] 1270 0.338% 99.014% ( 870, 1300 ] 1310 0.349% 99.363% ( 1300, 1900 ] 969 0.258% 99.621% ( 1900, 2900 ] 697 0.186% 99.807% ( 2900, 4400 ] 666 0.177% 99.984% ( 4400, 6600 ] 210 0.056% 100.040% ( 6600, 9900 ] 51 0.014% 100.054% ( 9900, 14000 ] 27 0.007% 100.061% ( 14000, 22000 ] 43 0.011% 100.072% ( 22000, 33000 ] 19 0.005% 100.077% ( 33000, 50000 ] 2 0.001% 100.078% ( 50000, 75000 ] 1 0.000% 100.078% ** Level 5 read latency histogram (micros): Count: 15679 Average: 1895.4063 StdDev: 2657.52 Min: 89 Median: 1196.6025 Max: 88718 Percentiles: P50: 1196.60 P75: 2455.69 P99: 9261.73 P99.9: 32623.08 P99.99: 65200.62 ------------------------------------------------------ ( 76, 110 ] 10 0.064% 0.064% ( 110, 170 ] 81 0.517% 0.580% ( 170, 250 ] 200 1.276% 1.856% ( 250, 380 ] 665 4.241% 6.097% # ( 380, 580 ] 1720 10.970% 17.067% ## ( 580, 870 ] 2815 17.954% 35.021% #### ( 870, 1300 ] 3092 19.721% 54.742% #### ( 1300, 1900 ] 2161 13.783% 68.525% ### ( 1900, 2900 ] 1827 11.653% 80.177% ## ( 2900, 4400 ] 2016 12.858% 93.035% ### ( 4400, 6600 ] 836 5.332% 98.367% # ( 6600, 9900 ] 123 0.784% 99.152% ( 9900, 14000 ] 23 0.147% 99.298% ( 14000, 22000 ] 47 0.300% 99.598% ( 22000, 33000 ] 49 0.313% 99.911% ( 33000, 50000 ] 10 0.064% 99.974% ( 50000, 75000 ] 4 0.026% 100.000% ( 75000, 110000 ] 1 0.006% 100.006% ** DB Stats ** Uptime(secs): 10.1 total, 10.0 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 97.20 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.826 H:M:S, 57.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 98.81 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.826 H:M:S, 58.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 7 num-running-flushes: 1 2024/12/17-00:00:30 ... thread 0: (1000000,2000000) ops and (258295.0,144624.1) ops/second in (3.871542,13.828955) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/7 3.03 GB 163.6 1.5 0.0 1.5 3.4 1.8 0.0 1.8 83.2 183.0 18.92 12.29 30 0.631 1599K 1956 0.0 0.0 L1 7/7 823.05 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 44.60 MB 0.4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 929.46 MB 0.9 3.2 1.8 1.5 3.2 1.7 0.0 1.8 110.5 109.3 30.11 15.67 13 2.316 1495K 0 0.0 0.0 L4 167/0 9.45 GB 1.0 5.7 4.0 1.7 5.6 3.9 0.0 1.4 103.5 102.6 56.05 29.66 36 1.557 3043K 610 0.0 0.0 L5 1128/0 92.71 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1332/14 106.93 GB 0.0 10.5 5.7 4.7 12.2 7.5 4.0 6.6 101.9 119.0 105.08 57.62 79 1.330 6137K 2566 0.0 0.0 Int 0/0 0.00 KB 0.0 10.5 5.7 4.7 12.2 7.5 4.0 6.6 101.9 119.0 105.08 57.62 79 1.330 6137K 2566 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.5 5.7 4.7 10.4 5.7 0.0 0.0 114.0 113.1 93.86 51.95 54 1.738 6137K 2566 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 168.4 11.22 5.67 25 0.449 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.0 total, 13.8 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.21 GB write, 893.45 MB/s write, 10.45 GB read, 764.72 MB/s read, 105.1 seconds Interval compaction: 12.21 GB write, 904.20 MB/s write, 10.45 GB read, 773.91 MB/s read, 105.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 42, pending-compaction-bytes-delays: 12, pending-compaction-bytes-stops: 0, total-delays: 12, total-stops: 42, interval: 53 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 718207 Average: 12.3593 StdDev: 217.48 Min: 0 Median: 1.6932 Max: 41880 Percentiles: P50: 1.69 P75: 2.55 P99: 169.56 P99.9: 1440.79 P99.99: 3828.79 ------------------------------------------------------ [ 0, 1 ] 88460 12.317% 12.317% ## ( 1, 2 ] 390411 54.359% 66.676% ########### ( 2, 3 ] 108055 15.045% 81.721% ### ( 3, 4 ] 51582 7.182% 88.903% # ( 4, 6 ] 52512 7.312% 96.215% # ( 6, 10 ] 12092 1.684% 97.898% ( 10, 15 ] 2582 0.360% 98.258% ( 15, 22 ] 315 0.044% 98.302% ( 22, 34 ] 33 0.005% 98.306% ( 34, 51 ] 40 0.006% 98.312% ( 51, 76 ] 65 0.009% 98.321% ( 76, 110 ] 898 0.125% 98.446% ( 110, 170 ] 4010 0.558% 99.004% ( 170, 250 ] 2513 0.350% 99.354% ( 250, 380 ] 1318 0.184% 99.538% ( 380, 580 ] 1209 0.168% 99.706% ( 580, 870 ] 822 0.114% 99.820% ( 870, 1300 ] 497 0.069% 99.890% ( 1300, 1900 ] 323 0.045% 99.935% ( 1900, 2900 ] 245 0.034% 99.969% ( 2900, 4400 ] 249 0.035% 100.003% ( 4400, 6600 ] 84 0.012% 100.015% ( 6600, 9900 ] 33 0.005% 100.020% ( 9900, 14000 ] 19 0.003% 100.022% ( 14000, 22000 ] 33 0.005% 100.027% ( 22000, 33000 ] 13 0.002% 100.029% ( 33000, 50000 ] 2 0.000% 100.029% ** Level 1 read latency histogram (micros): Count: 93647 Average: 10.2784 StdDev: 179.71 Min: 1 Median: 2.2049 Max: 31752 Percentiles: P50: 2.20 P75: 3.71 P99: 21.13 P99.9: 1648.91 P99.99: 5777.52 ------------------------------------------------------ [ 0, 1 ] 2902 3.099% 3.099% # ( 1, 2 ] 40132 42.855% 45.953% ######### ( 2, 3 ] 18495 19.750% 65.703% #### ( 3, 4 ] 12336 13.173% 78.876% ### ( 4, 6 ] 13846 14.785% 93.661% ### ( 6, 10 ] 3913 4.178% 97.840% # ( 10, 15 ] 774 0.827% 98.666% ( 15, 22 ] 357 0.381% 99.047% ( 22, 34 ] 42 0.045% 99.092% ( 34, 51 ] 10 0.011% 99.103% ( 51, 76 ] 9 0.010% 99.113% ( 76, 110 ] 52 0.056% 99.168% ( 110, 170 ] 233 0.249% 99.417% ( 170, 250 ] 127 0.136% 99.553% ( 250, 380 ] 89 0.095% 99.648% ( 380, 580 ] 76 0.081% 99.729% ( 580, 870 ] 79 0.084% 99.813% ( 870, 1300 ] 61 0.065% 99.878% ( 1300, 1900 ] 35 0.037% 99.916% ( 1900, 2900 ] 33 0.035% 99.951% ( 2900, 4400 ] 31 0.033% 99.984% ( 4400, 6600 ] 9 0.010% 99.994% ( 6600, 9900 ] 2 0.002% 99.996% ( 9900, 14000 ] 3 0.003% 99.999% ( 14000, 22000 ] 2 0.002% 100.001% ( 22000, 33000 ] 1 0.001% 100.002% ** Level 2 read latency histogram (micros): Count: 318430 Average: 13.8292 StdDev: 217.62 Min: 1 Median: 1.9153 Max: 46994 Percentiles: P50: 1.92 P75: 3.10 P99: 12.85 P99.9: 2174.77 P99.99: 3641.43 ------------------------------------------------------ [ 0, 1 ] 14632 4.595% 4.595% # ( 1, 2 ] 157968 49.608% 54.203% ########## ( 2, 3 ] 62911 19.757% 73.960% #### ( 3, 4 ] 33217 10.431% 84.392% ## ( 4, 6 ] 36009 11.308% 95.700% ## ( 6, 10 ] 9239 2.901% 98.601% # ( 10, 15 ] 2224 0.698% 99.300% ( 15, 22 ] 295 0.093% 99.392% ( 22, 34 ] 44 0.014% 99.406% ( 34, 51 ] 17 0.005% 99.411% ( 51, 76 ] 23 0.007% 99.419% ( 76, 110 ] 12 0.004% 99.422% ( 110, 170 ] 35 0.011% 99.433% ( 170, 250 ] 48 0.015% 99.449% ( 250, 380 ] 95 0.030% 99.478% ( 380, 580 ] 249 0.078% 99.557% ( 580, 870 ] 315 0.099% 99.655% ( 870, 1300 ] 398 0.125% 99.780% ( 1300, 1900 ] 316 0.099% 99.880% ( 1900, 2900 ] 235 0.074% 99.954% ( 2900, 4400 ] 235 0.074% 100.027% ( 4400, 6600 ] 63 0.020% 100.047% ( 6600, 9900 ] 25 0.008% 100.055% ( 9900, 14000 ] 4 0.001% 100.056% ( 14000, 22000 ] 6 0.002% 100.058% ( 33000, 50000 ] 2 0.001% 100.059% ** Level 3 read latency histogram (micros): Count: 1266007 Average: 12.6876 StdDev: 248.62 Min: 1 Median: 1.9125 Max: 61745 Percentiles: P50: 1.91 P75: 3.03 P99: 9.92 P99.9: 983.39 P99.99: 1551.62 ------------------------------------------------------ [ 0, 1 ] 46849 3.701% 3.701% # ( 1, 2 ] 642338 50.737% 54.438% ########## ( 2, 3 ] 256260 20.242% 74.679% #### ( 3, 4 ] 133999 10.584% 85.264% ## ( 4, 6 ] 144278 11.396% 96.660% ## ( 6, 10 ] 30252 2.390% 99.050% ( 10, 15 ] 6809 0.538% 99.588% ( 15, 22 ] 814 0.064% 99.652% ( 22, 34 ] 103 0.008% 99.660% ( 34, 51 ] 73 0.006% 99.666% ( 51, 76 ] 97 0.008% 99.673% ( 76, 110 ] 102 0.008% 99.681% ( 110, 170 ] 305 0.024% 99.706% ( 170, 250 ] 236 0.019% 99.724% ( 250, 380 ] 331 0.026% 99.750% ( 380, 580 ] 638 0.050% 99.801% ( 580, 870 ] 973 0.077% 99.878% ( 870, 1300 ] 1077 0.085% 99.963% ( 1300, 1900 ] 826 0.065% 100.028% ( 1900, 2900 ] 777 0.061% 100.089% ( 2900, 4400 ] 828 0.065% 100.155% ( 4400, 6600 ] 242 0.019% 100.174% ( 6600, 9900 ] 82 0.006% 100.180% ( 9900, 14000 ] 27 0.002% 100.182% ( 14000, 22000 ] 48 0.004% 100.186% ( 22000, 33000 ] 27 0.002% 100.188% ( 33000, 50000 ] 5 0.000% 100.189% ( 50000, 75000 ] 1 0.000% 100.189% ** Level 4 read latency histogram (micros): Count: 445804 Average: 32.2644 StdDev: 371.35 Min: 1 Median: 1.9265 Max: 56153 Percentiles: P50: 1.93 P75: 3.13 P99: 749.84 P99.9: 3422.46 P99.99: 4306.21 ------------------------------------------------------ [ 0, 1 ] 15409 3.456% 3.456% # ( 1, 2 ] 223953 50.236% 53.692% ########## ( 2, 3 ] 89017 19.968% 73.660% #### ( 3, 4 ] 45680 10.247% 83.907% ## ( 4, 6 ] 45525 10.212% 94.118% ## ( 6, 10 ] 10809 2.425% 96.543% ( 10, 15 ] 2476 0.555% 97.099% ( 15, 22 ] 277 0.062% 97.161% ( 22, 34 ] 27 0.006% 97.167% ( 34, 51 ] 40 0.009% 97.176% ( 51, 76 ] 48 0.011% 97.186% ( 76, 110 ] 744 0.167% 97.353% ( 110, 170 ] 2568 0.576% 97.929% ( 170, 250 ] 1563 0.351% 98.280% ( 250, 380 ] 1014 0.227% 98.507% ( 380, 580 ] 1287 0.289% 98.796% ( 580, 870 ] 1552 0.348% 99.144% ( 870, 1300 ] 1414 0.317% 99.461% ( 1300, 1900 ] 999 0.224% 99.686% ( 1900, 2900 ] 719 0.161% 99.847% ( 2900, 4400 ] 681 0.153% 100.000% ( 4400, 6600 ] 210 0.047% 100.047% ( 6600, 9900 ] 52 0.012% 100.058% ( 9900, 14000 ] 27 0.006% 100.064% ( 14000, 22000 ] 43 0.010% 100.074% ( 22000, 33000 ] 19 0.004% 100.078% ( 33000, 50000 ] 2 0.000% 100.079% ( 50000, 75000 ] 1 0.000% 100.079% ** Level 5 read latency histogram (micros): Count: 57567 Average: 694.8396 StdDev: 1583.40 Min: 76 Median: 228.6213 Max: 88718 Percentiles: P50: 228.62 P75: 653.41 P99: 5755.89 P99.9: 22545.96 P99.99: 41913.44 ------------------------------------------------------ ( 51, 76 ] 2 0.003% 0.003% ( 76, 110 ] 4597 7.985% 7.989% ## ( 110, 170 ] 17510 30.417% 38.406% ###### ( 170, 250 ] 9110 15.825% 54.231% ### ( 250, 380 ] 5186 9.009% 63.239% ## ( 380, 580 ] 5617 9.757% 72.997% ## ( 580, 870 ] 4559 7.919% 80.916% ## ( 870, 1300 ] 3624 6.295% 87.211% # ( 1300, 1900 ] 2302 3.999% 91.210% # ( 1900, 2900 ] 1889 3.281% 94.492% # ( 2900, 4400 ] 2078 3.610% 98.101% # ( 4400, 6600 ] 841 1.461% 99.562% ( 6600, 9900 ] 123 0.214% 99.776% ( 9900, 14000 ] 23 0.040% 99.816% ( 14000, 22000 ] 47 0.082% 99.898% ( 22000, 33000 ] 49 0.085% 99.983% ( 33000, 50000 ] 10 0.017% 100.000% ( 50000, 75000 ] 4 0.007% 100.007% ( 75000, 110000 ] 1 0.002% 100.009% ** DB Stats ** Uptime(secs): 14.0 total, 3.9 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 140.63 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.161 H:M:S, 44.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 254.22 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.336 H:M:S, 8.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/17-00:00:33 ... thread 0: (1000000,3000000) ops and (286098.5,173167.6) ops/second in (3.495300,17.324255) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/7 3.95 GB 258.0 2.5 0.0 2.5 5.3 2.8 0.0 1.9 100.3 210.2 25.76 18.64 49 0.526 2622K 3005 0.0 0.0 L1 7/7 823.05 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 44.60 MB 0.4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 929.46 MB 0.9 3.2 1.8 1.5 3.2 1.7 0.0 1.8 110.5 109.3 30.11 15.67 13 2.316 1495K 0 0.0 0.0 L4 167/0 9.45 GB 1.0 5.7 4.0 1.7 5.6 3.9 0.0 1.4 103.5 102.6 56.05 29.66 36 1.557 3043K 610 0.0 0.0 L5 1128/0 92.71 GB 0.9 0.0 0.0 0.0 0.0 0.0 4.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1335/14 107.86 GB 0.0 11.4 5.7 5.7 14.1 8.4 4.0 5.1 104.6 129.2 111.92 63.98 98 1.142 7161K 3615 0.0 0.0 Int 0/0 0.00 KB 0.0 11.4 5.7 5.7 14.1 8.4 4.0 5.1 104.6 129.2 111.92 63.98 98 1.142 7161K 3615 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.4 5.7 5.7 11.4 5.7 0.0 0.0 119.9 119.0 97.67 55.69 58 1.684 7161K 3615 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 198.9 14.25 8.29 40 0.356 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 17.5 total, 17.3 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.12 GB write, 826.51 MB/s write, 11.44 GB read, 669.54 MB/s read, 111.9 seconds Interval compaction: 14.12 GB write, 834.44 MB/s write, 11.44 GB read, 675.96 MB/s read, 111.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 42, pending-compaction-bytes-delays: 12, pending-compaction-bytes-stops: 0, total-delays: 12, total-stops: 42, interval: 53 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 986011 Average: 12.2102 StdDev: 188.13 Min: 0 Median: 1.6176 Max: 41880 Percentiles: P50: 1.62 P75: 2.30 P99: 210.51 P99.9: 1077.29 P99.99: 2902.36 ------------------------------------------------------ [ 0, 1 ] 158359 16.061% 16.061% ### ( 1, 2 ] 541819 54.951% 71.011% ########### ( 2, 3 ] 128977 13.081% 84.092% ### ( 3, 4 ] 58814 5.965% 90.057% # ( 4, 6 ] 57631 5.845% 95.902% # ( 6, 10 ] 13051 1.324% 97.225% ( 10, 15 ] 2776 0.282% 97.507% ( 15, 22 ] 349 0.035% 97.542% ( 22, 34 ] 35 0.004% 97.546% ( 34, 51 ] 42 0.004% 97.550% ( 51, 76 ] 66 0.007% 97.557% ( 76, 110 ] 2266 0.230% 97.786% ( 110, 170 ] 9294 0.943% 98.729% ( 170, 250 ] 5277 0.535% 99.264% ( 250, 380 ] 2625 0.266% 99.530% ( 380, 580 ] 2222 0.225% 99.756% ( 580, 870 ] 1153 0.117% 99.873% ( 870, 1300 ] 558 0.057% 99.929% ( 1300, 1900 ] 343 0.035% 99.964% ( 1900, 2900 ] 255 0.026% 99.990% ( 2900, 4400 ] 254 0.026% 100.016% ( 4400, 6600 ] 84 0.009% 100.024% ( 6600, 9900 ] 33 0.003% 100.028% ( 9900, 14000 ] 19 0.002% 100.030% ( 14000, 22000 ] 33 0.003% 100.033% ( 22000, 33000 ] 13 0.001% 100.034% ( 33000, 50000 ] 2 0.000% 100.034% ** Level 1 read latency histogram (micros): Count: 155578 Average: 8.3438 StdDev: 140.48 Min: 1 Median: 2.3531 Max: 31752 Percentiles: P50: 2.35 P75: 3.71 P99: 21.41 P99.9: 1034.09 P99.99: 4373.01 ------------------------------------------------------ [ 0, 1 ] 4294 2.760% 2.760% # ( 1, 2 ] 60356 38.795% 41.555% ######## ( 2, 3 ] 37212 23.919% 65.473% ##### ( 3, 4 ] 20814 13.378% 78.852% ### ( 4, 6 ] 22959 14.757% 93.609% ### ( 6, 10 ] 6429 4.132% 97.741% # ( 10, 15 ] 1385 0.890% 98.632% ( 15, 22 ] 626 0.402% 99.034% ( 22, 34 ] 71 0.046% 99.080% ( 34, 51 ] 14 0.009% 99.089% ( 51, 76 ] 9 0.006% 99.094% ( 76, 110 ] 128 0.082% 99.177% ( 110, 170 ] 487 0.313% 99.490% ( 170, 250 ] 264 0.170% 99.659% ( 250, 380 ] 137 0.088% 99.747% ( 380, 580 ] 122 0.078% 99.826% ( 580, 870 ] 91 0.058% 99.884% ( 870, 1300 ] 64 0.041% 99.925% ( 1300, 1900 ] 36 0.023% 99.949% ( 1900, 2900 ] 34 0.022% 99.970% ( 2900, 4400 ] 31 0.020% 99.990% ( 4400, 6600 ] 9 0.006% 99.996% ( 6600, 9900 ] 2 0.001% 99.997% ( 9900, 14000 ] 3 0.002% 99.999% ( 14000, 22000 ] 2 0.001% 100.001% ( 22000, 33000 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 318464 Average: 13.8491 StdDev: 217.62 Min: 1 Median: 1.9154 Max: 46994 Percentiles: P50: 1.92 P75: 3.10 P99: 12.93 P99.9: 2174.62 P99.99: 3641.41 ------------------------------------------------------ [ 0, 1 ] 14632 4.595% 4.595% # ( 1, 2 ] 157968 49.603% 54.198% ########## ( 2, 3 ] 62911 19.755% 73.952% #### ( 3, 4 ] 33217 10.430% 84.383% ## ( 4, 6 ] 36009 11.307% 95.690% ## ( 6, 10 ] 9239 2.901% 98.591% # ( 10, 15 ] 2224 0.698% 99.289% ( 15, 22 ] 295 0.093% 99.382% ( 22, 34 ] 44 0.014% 99.396% ( 34, 51 ] 17 0.005% 99.401% ( 51, 76 ] 23 0.007% 99.408% ( 76, 110 ] 15 0.005% 99.413% ( 110, 170 ] 53 0.017% 99.429% ( 170, 250 ] 55 0.017% 99.447% ( 250, 380 ] 98 0.031% 99.477% ( 380, 580 ] 252 0.079% 99.557% ( 580, 870 ] 315 0.099% 99.656% ( 870, 1300 ] 398 0.125% 99.781% ( 1300, 1900 ] 316 0.099% 99.880% ( 1900, 2900 ] 235 0.074% 99.954% ( 2900, 4400 ] 235 0.074% 100.027% ( 4400, 6600 ] 63 0.020% 100.047% ( 6600, 9900 ] 25 0.008% 100.055% ( 9900, 14000 ] 4 0.001% 100.056% ( 14000, 22000 ] 6 0.002% 100.058% ( 33000, 50000 ] 2 0.001% 100.059% ** Level 3 read latency histogram (micros): Count: 1266728 Average: 12.7946 StdDev: 248.61 Min: 1 Median: 1.9131 Max: 61745 Percentiles: P50: 1.91 P75: 3.03 P99: 10.06 P99.9: 983.98 P99.99: 1551.57 ------------------------------------------------------ [ 0, 1 ] 46849 3.698% 3.698% # ( 1, 2 ] 642338 50.708% 54.407% ########## ( 2, 3 ] 256260 20.230% 74.637% #### ( 3, 4 ] 133999 10.578% 85.215% ## ( 4, 6 ] 144278 11.390% 96.605% ## ( 6, 10 ] 30252 2.388% 98.993% ( 10, 15 ] 6809 0.538% 99.531% ( 15, 22 ] 814 0.064% 99.595% ( 22, 34 ] 103 0.008% 99.603% ( 34, 51 ] 73 0.006% 99.609% ( 51, 76 ] 97 0.008% 99.617% ( 76, 110 ] 180 0.014% 99.631% ( 110, 170 ] 643 0.051% 99.682% ( 170, 250 ] 400 0.032% 99.713% ( 250, 380 ] 408 0.032% 99.745% ( 380, 580 ] 690 0.054% 99.800% ( 580, 870 ] 982 0.078% 99.877% ( 870, 1300 ] 1080 0.085% 99.963% ( 1300, 1900 ] 826 0.065% 100.028% ( 1900, 2900 ] 777 0.061% 100.089% ( 2900, 4400 ] 828 0.065% 100.155% ( 4400, 6600 ] 242 0.019% 100.174% ( 6600, 9900 ] 82 0.006% 100.180% ( 9900, 14000 ] 27 0.002% 100.182% ( 14000, 22000 ] 48 0.004% 100.186% ( 22000, 33000 ] 27 0.002% 100.188% ( 33000, 50000 ] 5 0.000% 100.189% ( 50000, 75000 ] 1 0.000% 100.189% ** Level 4 read latency histogram (micros): Count: 453158 Average: 34.8514 StdDev: 369.47 Min: 1 Median: 1.9427 Max: 56153 Percentiles: P50: 1.94 P75: 3.25 P99: 755.58 P99.9: 3406.26 P99.99: 4304.59 ------------------------------------------------------ [ 0, 1 ] 15410 3.401% 3.401% # ( 1, 2 ] 224001 49.431% 52.832% ########## ( 2, 3 ] 89189 19.682% 72.513% #### ( 3, 4 ] 45777 10.102% 82.615% ## ( 4, 6 ] 45642 10.072% 92.687% ## ( 6, 10 ] 10829 2.390% 95.077% ( 10, 15 ] 2478 0.547% 95.624% ( 15, 22 ] 278 0.061% 95.685% ( 22, 34 ] 27 0.006% 95.691% ( 34, 51 ] 40 0.009% 95.700% ( 51, 76 ] 48 0.011% 95.710% ( 76, 110 ] 1690 0.373% 96.083% ( 110, 170 ] 5802 1.280% 97.364% ( 170, 250 ] 3021 0.667% 98.030% ( 250, 380 ] 1601 0.353% 98.384% ( 380, 580 ] 1766 0.390% 98.773% ( 580, 870 ] 1697 0.374% 99.148% ( 870, 1300 ] 1446 0.319% 99.467% ( 1300, 1900 ] 1003 0.221% 99.688% ( 1900, 2900 ] 730 0.161% 99.849% ( 2900, 4400 ] 681 0.150% 100.000% ( 4400, 6600 ] 210 0.046% 100.046% ( 6600, 9900 ] 52 0.011% 100.057% ( 9900, 14000 ] 27 0.006% 100.063% ( 14000, 22000 ] 43 0.009% 100.073% ( 22000, 33000 ] 19 0.004% 100.077% ( 33000, 50000 ] 2 0.000% 100.077% ( 50000, 75000 ] 1 0.000% 100.078% ** Level 5 read latency histogram (micros): Count: 101920 Average: 482.3344 StdDev: 1218.99 Min: 76 Median: 180.7936 Max: 88718 Percentiles: P50: 180.79 P75: 384.71 P99: 4593.06 P99.9: 14694.47 P99.99: 32956.90 ------------------------------------------------------ ( 51, 76 ] 4 0.004% 0.004% ( 76, 110 ] 10687 10.486% 10.490% ## ( 110, 170 ] 37780 37.068% 47.558% ####### ( 170, 250 ] 18448 18.100% 65.658% #### ( 250, 380 ] 9313 9.138% 74.796% ## ( 380, 580 ] 8841 8.674% 83.470% ## ( 580, 870 ] 5582 5.477% 88.947% # ( 870, 1300 ] 3813 3.741% 92.688% # ( 1300, 1900 ] 2351 2.307% 94.995% ( 1900, 2900 ] 1918 1.882% 96.877% ( 2900, 4400 ] 2090 2.051% 98.928% ( 4400, 6600 ] 841 0.825% 99.753% ( 6600, 9900 ] 123 0.121% 99.873% ( 9900, 14000 ] 23 0.023% 99.896% ( 14000, 22000 ] 47 0.046% 99.942% ( 22000, 33000 ] 49 0.048% 99.990% ( 33000, 50000 ] 10 0.010% 100.000% ( 50000, 75000 ] 4 0.004% 100.004% ( 75000, 110000 ] 1 0.001% 100.005% ** DB Stats ** Uptime(secs): 17.5 total, 3.5 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 168.79 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.161 H:M:S, 35.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 281.57 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 228.989 micros/op 17453 ops/sec 60.160 seconds 1049996 operations; 16.9 MB/s (270999 of 270999 found) Microseconds per read: Count: 1049996 Average: 228.9920 StdDev: 762.67 Min: 2 Median: 165.5754 Max: 191171 Percentiles: P50: 165.58 P75: 223.60 P99: 1053.72 P99.9: 11708.86 P99.99: 28778.40 ------------------------------------------------------ ( 1, 2 ] 14 0.001% 0.001% ( 2, 3 ] 46 0.004% 0.006% ( 3, 4 ] 45 0.004% 0.010% ( 4, 6 ] 86 0.008% 0.018% ( 6, 10 ] 403 0.038% 0.057% ( 10, 15 ] 4307 0.410% 0.467% ( 15, 22 ] 3962 0.377% 0.844% ( 22, 34 ] 13398 1.276% 2.120% ( 34, 51 ] 2331 0.222% 2.342% ( 51, 76 ] 268 0.026% 2.368% ( 76, 110 ] 26117 2.487% 4.855% ( 110, 170 ] 511760 48.739% 53.594% ########## ( 170, 250 ] 335445 31.947% 85.541% ###### ( 250, 380 ] 91759 8.739% 94.280% ## ( 380, 580 ] 33965 3.235% 97.515% # ( 580, 870 ] 13232 1.260% 98.775% ( 870, 1300 ] 5519 0.526% 99.301% ( 1300, 1900 ] 2398 0.228% 99.529% ( 1900, 2900 ] 1561 0.149% 99.678% ( 2900, 4400 ] 1424 0.136% 99.814% ( 4400, 6600 ] 484 0.046% 99.860% ( 6600, 9900 ] 227 0.022% 99.881% ( 9900, 14000 ] 442 0.042% 99.924% ( 14000, 22000 ] 584 0.056% 99.979% ( 22000, 33000 ] 185 0.018% 99.997% ( 33000, 50000 ] 23 0.002% 99.999% ( 50000, 75000 ] 6 0.001% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% ( 170000, 250000 ] 3 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 13314019 rocksdb.bloom.filter.full.positive COUNT : 1180071 rocksdb.bloom.filter.full.true.positive COUNT : 1049604 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 392 rocksdb.memtable.miss COUNT : 1049604 rocksdb.l0.hit COUNT : 36675 rocksdb.l1.hit COUNT : 16400 rocksdb.l2andup.hit COUNT : 996529 rocksdb.compaction.key.drop.new COUNT : 146579 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3676167 rocksdb.number.keys.read COUNT : 1049996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3793804344 rocksdb.bytes.read COUNT : 1049996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1560 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 45709396 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 19305136602 rocksdb.compact.write.bytes COUNT : 19153188540 rocksdb.flush.write.bytes COUNT : 3765281283 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 23 rocksdb.number.superversion_acquires COUNT : 784 rocksdb.number.superversion_releases COUNT : 749 rocksdb.number.superversion_cleanups COUNT : 194 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 215 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3734885376 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1422336 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 23665097776 rocksdb.non.last.level.read.count COUNT : 5576613 rocksdb.block.checksum.compute.count COUNT : 5866022 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 164.610807 P95 : 422.710014 P99 : 1052.491312 P100 : 191168.000000 COUNT : 1049996 SUM : 239402317 rocksdb.db.write.micros P50 : 1.570679 P95 : 4.833613 P99 : 13.730466 P100 : 1191150.000000 COUNT : 3676167 SUM : 55198986 rocksdb.compaction.times.micros P50 : 1383333.333333 P95 : 3894000.000000 P99 : 16480000.000000 P100 : 20808695.000000 COUNT : 69 SUM : 131697240 rocksdb.compaction.times.cpu_micros P50 : 869444.444444 P95 : 1877500.000000 P99 : 11102000.000000 P100 : 11851190.000000 COUNT : 69 SUM : 80246806 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 780.769231 P95 : 55000.000000 P99 : 91800.000000 P100 : 95206.000000 COUNT : 52 SUM : 383339 rocksdb.compaction.outfile.sync.micros P50 : 38230.769231 P95 : 268909.090909 P99 : 367236.363636 P100 : 410985.000000 COUNT : 208 SUM : 14290401 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 210.000000 P95 : 4296.153846 P99 : 7373.000000 P100 : 7373.000000 COUNT : 138 SUM : 146353 rocksdb.table.open.io.micros P50 : 9275.869565 P95 : 19849.056604 P99 : 25494.117647 P100 : 186445.000000 COUNT : 1560 SUM : 15046022 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.904050 P95 : 14.023810 P99 : 172.060014 P100 : 61749.000000 COUNT : 4675125 SUM : 79382065 rocksdb.read.block.get.micros P50 : 138.851936 P95 : 316.897169 P99 : 1015.039128 P100 : 88846.000000 COUNT : 1189332 SUM : 204954603 rocksdb.write.raw.block.micros P50 : 0.540107 P95 : 1.455034 P99 : 3.689350 P100 : 83854.000000 COUNT : 5551015 SUM : 18534067 rocksdb.numfiles.in.singlecompaction P50 : 2.739130 P95 : 3.961765 P99 : 23.680000 P100 : 26.000000 COUNT : 86 SUM : 296 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.571845 P95 : 2074.623958 P99 : 2739.788013 P100 : 1190975.000000 COUNT : 213368 SUM : 45726698 rocksdb.sst.read.micros P50 : 2.245120 P95 : 166.580944 P99 : 445.871337 P100 : 88718.000000 COUNT : 5576643 SUM : 269133944 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1049996 SUM : 1049996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3676167 SUM : 3793804344 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 222000.000000 P95 : 1018666.666667 P99 : 1489582.000000 P100 : 1489582.000000 COUNT : 52 SUM : 16686841 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 216 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0