[2024-12-09 23:55:34.447594] Starting SPDK v25.01-pre git sha1 b6a18b192 / DPDK 24.03.0 initialization... [2024-12-09 23:55:34.447906] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid233032 ] Set seed to 1733784934440417 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-09 23:55:34.604765] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-09 23:55:34.680365] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-09 23:55:34.680400] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 233032' to capture a snapshot of events at runtime. [2024-12-09 23:55:34.680409] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-09 23:55:34.680417] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-09 23:55:34.680425] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid233032 for offline analysis/debug. [2024-12-09 23:55:34.681375] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Dec 9 23:55:36 2024 CPU: 144 * Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz CPUCache: 55296 KB 2024/12/09-23:55:41 ... thread 0: (1000000,1000000) ops and (206711.5,206711.5) ops/second in (4.837661,4.837661) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/14 1.52 GB 18.9 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 192.4 4.25 1.88 11 0.386 0 0 0.0 0.0 L1 3/3 325.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 31/7 1.85 GB 1.3 3.0 1.6 1.4 3.0 1.6 0.0 1.8 135.5 133.5 22.87 13.62 13 1.759 2008K 18K 0.0 0.0 L4 153/26 11.48 GB 1.0 2.0 1.3 0.7 2.0 1.3 0.6 1.5 157.3 154.4 13.18 8.44 11 1.198 542K 2609 0.0 0.0 L5 124/0 9.29 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 328/50 24.46 GB 0.0 5.1 3.0 2.1 5.8 3.7 0.6 7.2 128.3 146.5 40.30 23.94 35 1.151 2551K 20K 0.0 0.0 Int 0/0 0.00 KB 0.0 5.1 3.0 2.1 5.8 3.7 0.6 7.2 128.3 146.5 40.30 23.94 35 1.151 2551K 20K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.1 3.0 2.1 5.0 2.9 0.0 0.0 143.5 141.1 36.05 22.06 24 1.502 2551K 20K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 192.4 4.25 1.88 11 0.386 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 4.9 total, 4.8 interval Flush(GB): cumulative 0.798, interval 0.798 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.77 GB write, 1200.81 MB/s write, 5.05 GB read, 1051.67 MB/s read, 40.3 seconds Interval compaction: 5.77 GB write, 1219.56 MB/s write, 5.05 GB read, 1068.09 MB/s read, 40.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 10, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 10, interval: 10 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 81029 Average: 115.6547 StdDev: 861.32 Min: 0 Median: 1.6375 Max: 70334 Percentiles: P50: 1.64 P75: 2.77 P99: 2219.72 P99.9: 5342.03 P99.99: 32858.51 ------------------------------------------------------ [ 0, 1 ] 17770 21.930% 21.930% #### ( 1, 2 ] 35682 44.036% 65.967% ######### ( 2, 3 ] 9513 11.740% 77.707% ## ( 3, 4 ] 4955 6.115% 83.822% # ( 4, 6 ] 3062 3.779% 87.601% # ( 6, 10 ] 592 0.731% 88.331% ( 10, 15 ] 59 0.073% 88.404% ( 15, 22 ] 13 0.016% 88.420% ( 22, 34 ] 5 0.006% 88.426% ( 34, 51 ] 6 0.007% 88.434% ( 51, 76 ] 8 0.010% 88.444% ( 76, 110 ] 6 0.007% 88.451% ( 110, 170 ] 68 0.084% 88.535% ( 170, 250 ] 350 0.432% 88.967% ( 250, 380 ] 1452 1.792% 90.759% ( 380, 580 ] 2799 3.454% 94.213% # ( 580, 870 ] 2390 2.950% 97.163% # ( 870, 1300 ] 956 1.180% 98.343% ( 1300, 1900 ] 386 0.476% 98.819% ( 1900, 2900 ] 462 0.570% 99.389% ( 2900, 4400 ] 382 0.471% 99.861% ( 4400, 6600 ] 77 0.095% 99.956% ( 6600, 9900 ] 10 0.012% 99.968% ( 9900, 14000 ] 3 0.004% 99.972% ( 14000, 22000 ] 8 0.010% 99.981% ( 22000, 33000 ] 8 0.010% 99.991% ( 33000, 50000 ] 10 0.012% 100.004% ( 50000, 75000 ] 6 0.007% 100.011% ** Level 1 read latency histogram (micros): Count: 259 Average: 953.9498 StdDev: 1041.02 Min: 111 Median: 586.9048 Max: 7688 Percentiles: P50: 586.90 P75: 928.23 P99: 4353.42 P99.9: 7688.00 P99.99: 7688.00 ------------------------------------------------------ ( 110, 170 ] 4 1.544% 1.544% ( 170, 250 ] 8 3.089% 4.633% # ( 250, 380 ] 44 16.988% 21.622% ### ( 380, 580 ] 72 27.799% 49.421% ###### ( 580, 870 ] 63 24.324% 73.745% ##### ( 870, 1300 ] 24 9.266% 83.012% ## ( 1300, 1900 ] 12 4.633% 87.645% # ( 1900, 2900 ] 11 4.247% 91.892% # ( 2900, 4400 ] 19 7.336% 99.228% # ( 4400, 6600 ] 1 0.386% 99.614% ( 6600, 9900 ] 1 0.386% 100.000% ** Level 2 read latency histogram (micros): Count: 354419 Average: 12.8163 StdDev: 291.29 Min: 0 Median: 1.6141 Max: 62975 Percentiles: P50: 1.61 P75: 2.38 P99: 9.54 P99.9: 2443.95 P99.99: 5141.44 ------------------------------------------------------ [ 0, 1 ] 65529 18.489% 18.489% #### ( 1, 2 ] 181874 51.316% 69.805% ########## ( 2, 3 ] 48670 13.732% 83.538% ### ( 3, 4 ] 30413 8.581% 92.119% ## ( 4, 6 ] 20426 5.763% 97.882% # ( 6, 10 ] 4483 1.265% 99.147% ( 10, 15 ] 396 0.112% 99.259% ( 15, 22 ] 93 0.026% 99.285% ( 22, 34 ] 109 0.031% 99.315% ( 34, 51 ] 114 0.032% 99.348% ( 51, 76 ] 31 0.009% 99.356% ( 76, 110 ] 15 0.004% 99.361% ( 110, 170 ] 46 0.013% 99.374% ( 170, 250 ] 127 0.036% 99.409% ( 250, 380 ] 349 0.098% 99.508% ( 380, 580 ] 521 0.147% 99.655% ( 580, 870 ] 379 0.107% 99.762% ( 870, 1300 ] 195 0.055% 99.817% ( 1300, 1900 ] 176 0.050% 99.867% ( 1900, 2900 ] 218 0.062% 99.928% ( 2900, 4400 ] 199 0.056% 99.984% ( 4400, 6600 ] 61 0.017% 100.001% ( 6600, 9900 ] 24 0.007% 100.008% ( 9900, 14000 ] 8 0.002% 100.010% ( 14000, 22000 ] 21 0.006% 100.016% ( 22000, 33000 ] 8 0.002% 100.019% ( 33000, 50000 ] 3 0.001% 100.019% ( 50000, 75000 ] 1 0.000% 100.020% ** Level 3 read latency histogram (micros): Count: 633003 Average: 11.3103 StdDev: 253.62 Min: 0 Median: 1.5225 Max: 63222 Percentiles: P50: 1.52 P75: 2.15 P99: 9.22 P99.9: 1674.76 P99.99: 3386.33 ------------------------------------------------------ [ 0, 1 ] 157277 24.846% 24.846% ##### ( 1, 2 ] 304738 48.142% 72.988% ########## ( 2, 3 ] 83909 13.256% 86.244% ### ( 3, 4 ] 47830 7.556% 93.800% ## ( 4, 6 ] 28814 4.552% 98.352% # ( 6, 10 ] 5092 0.804% 99.156% ( 10, 15 ] 348 0.055% 99.211% ( 15, 22 ] 102 0.016% 99.227% ( 22, 34 ] 101 0.016% 99.243% ( 34, 51 ] 160 0.025% 99.268% ( 51, 76 ] 43 0.007% 99.275% ( 76, 110 ] 30 0.005% 99.280% ( 110, 170 ] 62 0.010% 99.290% ( 170, 250 ] 190 0.030% 99.320% ( 250, 380 ] 727 0.115% 99.434% ( 380, 580 ] 1287 0.203% 99.638% ( 580, 870 ] 1021 0.161% 99.799% ( 870, 1300 ] 441 0.070% 99.869% ( 1300, 1900 ] 317 0.050% 99.919% ( 1900, 2900 ] 357 0.056% 99.975% ( 2900, 4400 ] 289 0.046% 100.021% ( 4400, 6600 ] 70 0.011% 100.032% ( 6600, 9900 ] 15 0.002% 100.034% ( 9900, 14000 ] 6 0.001% 100.035% ( 14000, 22000 ] 10 0.002% 100.037% ( 22000, 33000 ] 9 0.001% 100.038% ( 33000, 50000 ] 7 0.001% 100.039% ( 50000, 75000 ] 2 0.000% 100.040% ** Level 4 read latency histogram (micros): Count: 269401 Average: 35.0233 StdDev: 465.39 Min: 0 Median: 1.4346 Max: 59635 Percentiles: P50: 1.43 P75: 2.01 P99: 797.96 P99.9: 3475.12 P99.99: 4388.92 ------------------------------------------------------ [ 0, 1 ] 83273 30.910% 30.910% ###### ( 1, 2 ] 118326 43.922% 74.832% ######### ( 2, 3 ] 32806 12.177% 87.010% ## ( 3, 4 ] 16505 6.127% 93.136% # ( 4, 6 ] 8655 3.213% 96.349% # ( 6, 10 ] 1206 0.448% 96.797% ( 10, 15 ] 86 0.032% 96.829% ( 15, 22 ] 19 0.007% 96.836% ( 22, 34 ] 30 0.011% 96.847% ( 34, 51 ] 66 0.024% 96.871% ( 51, 76 ] 11 0.004% 96.875% ( 76, 110 ] 28 0.010% 96.886% ( 110, 170 ] 87 0.032% 96.918% ( 170, 250 ] 382 0.142% 97.060% ( 250, 380 ] 1254 0.465% 97.525% ( 380, 580 ] 2428 0.901% 98.427% ( 580, 870 ] 2061 0.765% 99.192% ( 870, 1300 ] 868 0.322% 99.514% ( 1300, 1900 ] 411 0.153% 99.666% ( 1900, 2900 ] 481 0.179% 99.845% ( 2900, 4400 ] 398 0.148% 99.993% ( 4400, 6600 ] 86 0.032% 100.024% ( 6600, 9900 ] 16 0.006% 100.030% ( 9900, 14000 ] 2 0.001% 100.031% ( 14000, 22000 ] 13 0.005% 100.036% ( 22000, 33000 ] 5 0.002% 100.038% ( 33000, 50000 ] 13 0.005% 100.043% ( 50000, 75000 ] 5 0.002% 100.045% ** Level 5 read latency histogram (micros): Count: 6421 Average: 989.0076 StdDev: 2253.65 Min: 102 Median: 573.0710 Max: 63429 Percentiles: P50: 573.07 P75: 863.70 P99: 5758.71 P99.9: 40307.17 P99.99: 63429.00 ------------------------------------------------------ ( 76, 110 ] 6 0.093% 0.093% ( 110, 170 ] 63 0.981% 1.075% ( 170, 250 ] 325 5.062% 6.136% # ( 250, 380 ] 1075 16.742% 22.878% ### ( 380, 580 ] 1804 28.095% 50.973% ###### ( 580, 870 ] 1577 24.560% 75.533% ##### ( 870, 1300 ] 607 9.453% 84.987% ## ( 1300, 1900 ] 262 4.080% 89.067% # ( 1900, 2900 ] 331 5.155% 94.222% # ( 2900, 4400 ] 258 4.018% 98.240% # ( 4400, 6600 ] 79 1.230% 99.470% ( 6600, 9900 ] 10 0.156% 99.626% ( 9900, 14000 ] 3 0.047% 99.673% ( 14000, 22000 ] 4 0.062% 99.735% ( 22000, 33000 ] 8 0.125% 99.860% ( 33000, 50000 ] 6 0.093% 99.953% ( 50000, 75000 ] 3 0.047% 100.000% ** DB Stats ** Uptime(secs): 4.9 total, 4.8 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 200.13 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.770 H:M:S, 15.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 203.21 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.770 H:M:S, 15.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/12/09-23:55:46 ... thread 0: (1000000,2000000) ops and (208226.4,207466.1) ops/second in (4.802466,9.640127) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/6 2.55 GB 187.8 1.5 0.0 1.5 3.3 1.8 0.0 1.8 78.4 175.9 19.26 9.37 29 0.664 1532K 8509 0.0 0.0 L1 3/3 325.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 17/0 945.13 MB 0.9 3.0 1.6 1.4 3.0 1.6 0.0 1.8 135.5 133.5 22.87 13.62 13 1.759 2008K 18K 0.0 0.0 L4 141/0 9.76 GB 1.0 6.9 2.3 4.6 6.2 1.6 0.6 2.7 185.1 165.7 38.10 23.05 25 1.524 1498K 2609 0.0 0.0 L5 152/0 11.26 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 328/9 24.81 GB 0.0 11.4 3.9 7.5 12.5 5.0 2.6 6.8 145.3 158.9 80.22 46.03 67 1.197 5040K 29K 0.0 0.0 Int 0/0 0.00 KB 0.0 11.4 3.9 7.5 12.5 5.0 2.6 6.8 145.3 158.9 80.22 46.03 67 1.197 5040K 29K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.4 3.9 7.5 10.6 3.1 0.0 0.0 163.8 152.6 71.18 41.37 43 1.655 5040K 29K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 208.7 9.04 4.67 24 0.377 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.7 total, 9.6 interval Flush(GB): cumulative 1.842, interval 1.842 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.45 GB write, 1311.84 MB/s write, 11.38 GB read, 1199.29 MB/s read, 80.2 seconds Interval compaction: 12.45 GB write, 1322.12 MB/s write, 11.38 GB read, 1208.69 MB/s read, 80.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 29, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 29, interval: 29 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 440816 Average: 44.7469 StdDev: 438.32 Min: 0 Median: 1.2889 Max: 70334 Percentiles: P50: 1.29 P75: 1.94 P99: 834.83 P99.9: 3363.08 P99.99: 4309.19 ------------------------------------------------------ [ 0, 1 ] 171239 38.846% 38.846% ######## ( 1, 2 ] 170192 38.608% 77.454% ######## ( 2, 3 ] 39015 8.851% 86.305% ## ( 3, 4 ] 20250 4.594% 90.899% # ( 4, 6 ] 10815 2.453% 93.352% ( 6, 10 ] 1945 0.441% 93.793% ( 10, 15 ] 561 0.127% 93.921% ( 15, 22 ] 105 0.024% 93.944% ( 22, 34 ] 45 0.010% 93.955% ( 34, 51 ] 22 0.005% 93.960% ( 51, 76 ] 10 0.002% 93.962% ( 76, 110 ] 179 0.041% 94.002% ( 110, 170 ] 1741 0.395% 94.397% ( 170, 250 ] 2640 0.599% 94.996% ( 250, 380 ] 6423 1.457% 96.453% ( 380, 580 ] 6906 1.567% 98.020% ( 580, 870 ] 4916 1.115% 99.135% ( 870, 1300 ] 1661 0.377% 99.512% ( 1300, 1900 ] 648 0.147% 99.659% ( 1900, 2900 ] 868 0.197% 99.856% ( 2900, 4400 ] 629 0.143% 99.999% ( 4400, 6600 ] 128 0.029% 100.028% ( 6600, 9900 ] 25 0.006% 100.033% ( 9900, 14000 ] 8 0.002% 100.035% ( 14000, 22000 ] 12 0.003% 100.038% ( 22000, 33000 ] 12 0.003% 100.041% ( 33000, 50000 ] 13 0.003% 100.044% ( 50000, 75000 ] 7 0.002% 100.045% ** Level 1 read latency histogram (micros): Count: 730 Average: 653.0014 StdDev: 779.52 Min: 88 Median: 426.5116 Max: 7688 Percentiles: P50: 426.51 P75: 697.16 P99: 4119.57 P99.9: 7491.00 P99.99: 7688.00 ------------------------------------------------------ ( 76, 110 ] 9 1.233% 1.233% ( 110, 170 ] 50 6.849% 8.082% # ( 170, 250 ] 76 10.411% 18.493% ## ( 250, 380 ] 190 26.027% 44.521% ##### ( 380, 580 ] 172 23.562% 68.082% ##### ( 580, 870 ] 125 17.123% 85.205% ### ( 870, 1300 ] 44 6.027% 91.233% # ( 1300, 1900 ] 17 2.329% 93.562% ( 1900, 2900 ] 21 2.877% 96.438% # ( 2900, 4400 ] 23 3.151% 99.589% # ( 4400, 6600 ] 2 0.274% 99.863% ( 6600, 9900 ] 1 0.137% 100.000% ** Level 2 read latency histogram (micros): Count: 354419 Average: 12.8163 StdDev: 291.29 Min: 0 Median: 1.6141 Max: 62975 Percentiles: P50: 1.61 P75: 2.38 P99: 9.54 P99.9: 2443.95 P99.99: 5141.44 ------------------------------------------------------ [ 0, 1 ] 65529 18.489% 18.489% #### ( 1, 2 ] 181874 51.316% 69.805% ########## ( 2, 3 ] 48670 13.732% 83.538% ### ( 3, 4 ] 30413 8.581% 92.119% ## ( 4, 6 ] 20426 5.763% 97.882% # ( 6, 10 ] 4483 1.265% 99.147% ( 10, 15 ] 396 0.112% 99.259% ( 15, 22 ] 93 0.026% 99.285% ( 22, 34 ] 109 0.031% 99.315% ( 34, 51 ] 114 0.032% 99.348% ( 51, 76 ] 31 0.009% 99.356% ( 76, 110 ] 15 0.004% 99.361% ( 110, 170 ] 46 0.013% 99.374% ( 170, 250 ] 127 0.036% 99.409% ( 250, 380 ] 349 0.098% 99.508% ( 380, 580 ] 521 0.147% 99.655% ( 580, 870 ] 379 0.107% 99.762% ( 870, 1300 ] 195 0.055% 99.817% ( 1300, 1900 ] 176 0.050% 99.867% ( 1900, 2900 ] 218 0.062% 99.928% ( 2900, 4400 ] 199 0.056% 99.984% ( 4400, 6600 ] 61 0.017% 100.001% ( 6600, 9900 ] 24 0.007% 100.008% ( 9900, 14000 ] 8 0.002% 100.010% ( 14000, 22000 ] 21 0.006% 100.016% ( 22000, 33000 ] 8 0.002% 100.019% ( 33000, 50000 ] 3 0.001% 100.019% ( 50000, 75000 ] 1 0.000% 100.020% ** Level 3 read latency histogram (micros): Count: 827310 Average: 12.4042 StdDev: 241.82 Min: 0 Median: 1.5045 Max: 63222 Percentiles: P50: 1.50 P75: 2.13 P99: 120.76 P99.9: 1787.79 P99.99: 3568.60 ------------------------------------------------------ [ 0, 1 ] 217183 26.252% 26.252% ##### ( 1, 2 ] 389450 47.074% 73.326% ######### ( 2, 3 ] 108659 13.134% 86.460% ### ( 3, 4 ] 61612 7.447% 93.907% # ( 4, 6 ] 35291 4.266% 98.173% # ( 6, 10 ] 5852 0.707% 98.880% ( 10, 15 ] 388 0.047% 98.927% ( 15, 22 ] 121 0.015% 98.942% ( 22, 34 ] 116 0.014% 98.956% ( 34, 51 ] 191 0.023% 98.979% ( 51, 76 ] 54 0.007% 98.986% ( 76, 110 ] 67 0.008% 98.994% ( 110, 170 ] 295 0.036% 99.029% ( 170, 250 ] 538 0.065% 99.094% ( 250, 380 ] 1660 0.201% 99.295% ( 380, 580 ] 2264 0.274% 99.569% ( 580, 870 ] 1740 0.210% 99.779% ( 870, 1300 ] 683 0.083% 99.861% ( 1300, 1900 ] 392 0.047% 99.909% ( 1900, 2900 ] 501 0.061% 99.969% ( 2900, 4400 ] 382 0.046% 100.016% ( 4400, 6600 ] 84 0.010% 100.026% ( 6600, 9900 ] 21 0.003% 100.028% ( 9900, 14000 ] 8 0.001% 100.029% ( 14000, 22000 ] 15 0.002% 100.031% ( 22000, 33000 ] 11 0.001% 100.032% ( 33000, 50000 ] 7 0.001% 100.033% ( 50000, 75000 ] 2 0.000% 100.033% ** Level 4 read latency histogram (micros): Count: 1199533 Average: 18.6663 StdDev: 283.75 Min: 0 Median: 1.3795 Max: 59635 Percentiles: P50: 1.38 P75: 1.99 P99: 464.39 P99.9: 2109.90 P99.99: 3196.09 ------------------------------------------------------ [ 0, 1 ] 413827 34.499% 34.499% ####### ( 1, 2 ] 489907 40.841% 75.340% ######## ( 2, 3 ] 149267 12.444% 87.784% ## ( 3, 4 ] 77673 6.475% 94.260% # ( 4, 6 ] 37754 3.147% 97.407% # ( 6, 10 ] 5194 0.433% 97.840% ( 10, 15 ] 746 0.062% 97.902% ( 15, 22 ] 91 0.008% 97.910% ( 22, 34 ] 121 0.010% 97.920% ( 34, 51 ] 241 0.020% 97.940% ( 51, 76 ] 60 0.005% 97.945% ( 76, 110 ] 252 0.021% 97.966% ( 110, 170 ] 1576 0.131% 98.097% ( 170, 250 ] 2345 0.195% 98.293% ( 250, 380 ] 5793 0.483% 98.776% ( 380, 580 ] 6377 0.532% 99.307% ( 580, 870 ] 4525 0.377% 99.685% ( 870, 1300 ] 1579 0.132% 99.816% ( 1300, 1900 ] 758 0.063% 99.879% ( 1900, 2900 ] 1179 0.098% 99.978% ( 2900, 4400 ] 750 0.063% 100.040% ( 4400, 6600 ] 162 0.014% 100.054% ( 6600, 9900 ] 32 0.003% 100.056% ( 9900, 14000 ] 13 0.001% 100.057% ( 14000, 22000 ] 32 0.003% 100.060% ( 22000, 33000 ] 16 0.001% 100.061% ( 33000, 50000 ] 18 0.002% 100.063% ( 50000, 75000 ] 5 0.000% 100.063% ** Level 5 read latency histogram (micros): Count: 17267 Average: 680.5464 StdDev: 1545.75 Min: 82 Median: 417.3934 Max: 63429 Percentiles: P50: 417.39 P75: 678.23 P99: 4308.63 P99.9: 24382.88 P99.99: 60610.83 ------------------------------------------------------ ( 76, 110 ] 152 0.880% 0.880% ( 110, 170 ] 1261 7.303% 8.183% # ( 170, 250 ] 1907 11.044% 19.227% ## ( 250, 380 ] 4546 26.328% 45.555% ##### ( 380, 580 ] 4105 23.774% 69.329% ##### ( 580, 870 ] 2891 16.743% 86.072% ### ( 870, 1300 ] 939 5.438% 91.510% # ( 1300, 1900 ] 392 2.270% 93.780% ( 1900, 2900 ] 521 3.017% 96.797% # ( 2900, 4400 ] 405 2.346% 99.143% ( 4400, 6600 ] 105 0.608% 99.751% ( 6600, 9900 ] 11 0.064% 99.815% ( 9900, 14000 ] 6 0.035% 99.849% ( 14000, 22000 ] 7 0.041% 99.890% ( 22000, 33000 ] 8 0.046% 99.936% ( 33000, 50000 ] 8 0.046% 99.983% ( 50000, 75000 ] 3 0.017% 100.000% ** DB Stats ** Uptime(secs): 9.7 total, 4.8 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 202.50 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.484 H:M:S, 25.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 204.94 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.714 H:M:S, 35.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/09-23:55:50 ... thread 0: (1000000,3000000) ops and (239325.8,217099.8) ops/second in (4.178404,13.818531) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/6 3.47 GB 281.6 2.5 0.0 2.5 5.3 2.7 0.0 1.9 101.1 211.4 25.52 15.31 48 0.532 2618K 14K 0.0 0.0 L1 3/3 325.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 17/0 945.13 MB 0.9 3.0 1.6 1.4 3.0 1.6 0.0 1.8 135.5 133.5 22.87 13.62 13 1.759 2008K 18K 0.0 0.0 L4 141/0 9.76 GB 1.0 6.9 2.3 4.6 6.2 1.6 0.6 2.7 185.1 165.7 38.10 23.05 25 1.524 1498K 2609 0.0 0.0 L5 152/0 11.26 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 331/9 25.73 GB 0.0 12.4 3.9 8.5 14.4 5.9 2.6 5.2 147.2 170.7 86.48 51.97 86 1.006 6126K 34K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.4 3.9 8.5 14.4 5.9 2.6 5.2 147.2 170.7 86.48 51.97 86 1.006 6126K 34K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.4 3.9 8.5 11.6 3.1 0.0 0.0 171.1 160.3 74.40 44.54 47 1.583 6126K 34K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 234.2 12.08 7.44 39 0.310 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 13.9 total, 13.8 interval Flush(GB): cumulative 2.764, interval 2.764 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.41 GB write, 1061.89 MB/s write, 12.43 GB read, 915.68 MB/s read, 86.5 seconds Interval compaction: 14.41 GB write, 1067.70 MB/s write, 12.43 GB read, 920.69 MB/s read, 86.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 29, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 29, interval: 33 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 740968 Average: 35.1623 StdDev: 341.44 Min: 0 Median: 1.1943 Max: 70334 Percentiles: P50: 1.19 P75: 1.84 P99: 652.02 P99.9: 2521.00 P99.99: 3705.81 ------------------------------------------------------ [ 0, 1 ] 314856 42.493% 42.493% ######## ( 1, 2 ] 286265 38.634% 81.126% ######## ( 2, 3 ] 47610 6.425% 87.552% # ( 3, 4 ] 23620 3.188% 90.740% # ( 4, 6 ] 13131 1.772% 92.512% ( 6, 10 ] 3014 0.407% 92.918% ( 10, 15 ] 2389 0.322% 93.241% ( 15, 22 ] 236 0.032% 93.273% ( 22, 34 ] 124 0.017% 93.289% ( 34, 51 ] 29 0.004% 93.293% ( 51, 76 ] 11 0.001% 93.295% ( 76, 110 ] 945 0.128% 93.422% ( 110, 170 ] 7301 0.985% 94.408% ( 170, 250 ] 7535 1.017% 95.425% ( 250, 380 ] 16553 2.234% 97.659% ( 380, 580 ] 8684 1.172% 98.831% ( 580, 870 ] 5059 0.683% 99.513% ( 870, 1300 ] 1675 0.226% 99.739% ( 1300, 1900 ] 652 0.088% 99.827% ( 1900, 2900 ] 868 0.117% 99.945% ( 2900, 4400 ] 629 0.085% 100.029% ( 4400, 6600 ] 128 0.017% 100.047% ( 6600, 9900 ] 25 0.003% 100.050% ( 9900, 14000 ] 8 0.001% 100.051% ( 14000, 22000 ] 12 0.002% 100.053% ( 22000, 33000 ] 12 0.002% 100.054% ( 33000, 50000 ] 13 0.002% 100.056% ( 50000, 75000 ] 7 0.001% 100.057% ** Level 1 read latency histogram (micros): Count: 1437 Average: 456.7933 StdDev: 594.75 Min: 88 Median: 313.7606 Max: 7688 Percentiles: P50: 313.76 P75: 478.96 P99: 3658.48 P99.9: 6119.30 P99.99: 7688.00 ------------------------------------------------------ ( 76, 110 ] 31 2.157% 2.157% ( 110, 170 ] 223 15.518% 17.676% ### ( 170, 250 ] 233 16.214% 33.890% ### ( 250, 380 ] 472 32.846% 66.736% ####### ( 380, 580 ] 240 16.701% 83.438% ### ( 580, 870 ] 130 9.047% 92.484% ## ( 870, 1300 ] 44 3.062% 95.546% # ( 1300, 1900 ] 17 1.183% 96.729% ( 1900, 2900 ] 21 1.461% 98.191% ( 2900, 4400 ] 23 1.601% 99.791% ( 4400, 6600 ] 2 0.139% 99.930% ( 6600, 9900 ] 1 0.070% 100.000% ** Level 2 read latency histogram (micros): Count: 354419 Average: 12.8163 StdDev: 291.29 Min: 0 Median: 1.6141 Max: 62975 Percentiles: P50: 1.61 P75: 2.38 P99: 9.54 P99.9: 2443.95 P99.99: 5141.44 ------------------------------------------------------ [ 0, 1 ] 65529 18.489% 18.489% #### ( 1, 2 ] 181874 51.316% 69.805% ########## ( 2, 3 ] 48670 13.732% 83.538% ### ( 3, 4 ] 30413 8.581% 92.119% ## ( 4, 6 ] 20426 5.763% 97.882% # ( 6, 10 ] 4483 1.265% 99.147% ( 10, 15 ] 396 0.112% 99.259% ( 15, 22 ] 93 0.026% 99.285% ( 22, 34 ] 109 0.031% 99.315% ( 34, 51 ] 114 0.032% 99.348% ( 51, 76 ] 31 0.009% 99.356% ( 76, 110 ] 15 0.004% 99.361% ( 110, 170 ] 46 0.013% 99.374% ( 170, 250 ] 127 0.036% 99.409% ( 250, 380 ] 349 0.098% 99.508% ( 380, 580 ] 521 0.147% 99.655% ( 580, 870 ] 379 0.107% 99.762% ( 870, 1300 ] 195 0.055% 99.817% ( 1300, 1900 ] 176 0.050% 99.867% ( 1900, 2900 ] 218 0.062% 99.928% ( 2900, 4400 ] 199 0.056% 99.984% ( 4400, 6600 ] 61 0.017% 100.001% ( 6600, 9900 ] 24 0.007% 100.008% ( 9900, 14000 ] 8 0.002% 100.010% ( 14000, 22000 ] 21 0.006% 100.016% ( 22000, 33000 ] 8 0.002% 100.019% ( 33000, 50000 ] 3 0.001% 100.019% ( 50000, 75000 ] 1 0.000% 100.020% ** Level 3 read latency histogram (micros): Count: 830461 Average: 13.3259 StdDev: 241.91 Min: 0 Median: 1.5085 Max: 63222 Percentiles: P50: 1.51 P75: 2.15 P99: 285.93 P99.9: 1782.97 P99.99: 3567.36 ------------------------------------------------------ [ 0, 1 ] 217183 26.152% 26.152% ##### ( 1, 2 ] 389450 46.896% 73.048% ######### ( 2, 3 ] 108659 13.084% 86.132% ### ( 3, 4 ] 61612 7.419% 93.551% # ( 4, 6 ] 35291 4.250% 97.800% # ( 6, 10 ] 5852 0.705% 98.505% ( 10, 15 ] 388 0.047% 98.552% ( 15, 22 ] 121 0.015% 98.566% ( 22, 34 ] 116 0.014% 98.580% ( 34, 51 ] 191 0.023% 98.603% ( 51, 76 ] 54 0.007% 98.610% ( 76, 110 ] 173 0.021% 98.631% ( 110, 170 ] 1024 0.123% 98.754% ( 170, 250 ] 1203 0.145% 98.899% ( 250, 380 ] 3037 0.366% 99.265% ( 380, 580 ] 2506 0.302% 99.566% ( 580, 870 ] 1770 0.213% 99.780% ( 870, 1300 ] 685 0.082% 99.862% ( 1300, 1900 ] 392 0.047% 99.909% ( 1900, 2900 ] 501 0.060% 99.970% ( 2900, 4400 ] 382 0.046% 100.016% ( 4400, 6600 ] 84 0.010% 100.026% ( 6600, 9900 ] 21 0.003% 100.028% ( 9900, 14000 ] 8 0.001% 100.029% ( 14000, 22000 ] 15 0.002% 100.031% ( 22000, 33000 ] 11 0.001% 100.032% ( 33000, 50000 ] 7 0.001% 100.033% ( 50000, 75000 ] 2 0.000% 100.033% ** Level 4 read latency histogram (micros): Count: 1220399 Average: 22.5331 StdDev: 283.24 Min: 0 Median: 1.4008 Max: 59635 Percentiles: P50: 1.40 P75: 2.08 P99: 485.16 P99.9: 2092.03 P99.99: 3189.15 ------------------------------------------------------ [ 0, 1 ] 413829 33.909% 33.909% ####### ( 1, 2 ] 489933 40.145% 74.055% ######## ( 2, 3 ] 149328 12.236% 86.291% ## ( 3, 4 ] 77767 6.372% 92.663% # ( 4, 6 ] 37917 3.107% 95.770% # ( 6, 10 ] 5294 0.434% 96.204% ( 10, 15 ] 977 0.080% 96.284% ( 15, 22 ] 110 0.009% 96.293% ( 22, 34 ] 131 0.011% 96.303% ( 34, 51 ] 241 0.020% 96.323% ( 51, 76 ] 60 0.005% 96.328% ( 76, 110 ] 892 0.073% 96.401% ( 110, 170 ] 6469 0.530% 96.931% ( 170, 250 ] 6434 0.527% 97.458% ( 250, 380 ] 14658 1.201% 98.660% ( 380, 580 ] 7902 0.647% 99.307% ( 580, 870 ] 4667 0.382% 99.689% ( 870, 1300 ] 1583 0.130% 99.819% ( 1300, 1900 ] 760 0.062% 99.881% ( 1900, 2900 ] 1180 0.097% 99.978% ( 2900, 4400 ] 752 0.062% 100.040% ( 4400, 6600 ] 162 0.013% 100.053% ( 6600, 9900 ] 32 0.003% 100.056% ( 9900, 14000 ] 13 0.001% 100.057% ( 14000, 22000 ] 32 0.003% 100.059% ( 22000, 33000 ] 16 0.001% 100.061% ( 33000, 50000 ] 18 0.001% 100.062% ( 50000, 75000 ] 5 0.000% 100.063% ** Level 5 read latency histogram (micros): Count: 32986 Average: 476.3352 StdDev: 1140.92 Min: 82 Median: 313.8019 Max: 63429 Percentiles: P50: 313.80 P75: 472.86 P99: 3717.02 P99.9: 8704.20 P99.99: 42990.47 ------------------------------------------------------ ( 76, 110 ] 659 1.998% 1.998% ( 110, 170 ] 5075 15.385% 17.383% ### ( 170, 250 ] 5194 15.746% 33.129% ### ( 250, 380 ] 11339 34.375% 67.504% ####### ( 380, 580 ] 5325 16.143% 83.648% ### ( 580, 870 ] 2980 9.034% 92.682% ## ( 870, 1300 ] 950 2.880% 95.562% # ( 1300, 1900 ] 392 1.188% 96.750% ( 1900, 2900 ] 521 1.579% 98.330% ( 2900, 4400 ] 406 1.231% 99.560% ( 4400, 6600 ] 105 0.318% 99.879% ( 6600, 9900 ] 11 0.033% 99.912% ( 9900, 14000 ] 6 0.018% 99.930% ( 14000, 22000 ] 7 0.021% 99.951% ( 22000, 33000 ] 8 0.024% 99.976% ( 33000, 50000 ] 8 0.024% 100.000% ( 50000, 75000 ] 3 0.009% 100.009% ** DB Stats ** Uptime(secs): 13.9 total, 4.2 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 212.44 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.114 H:M:S, 22.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 235.54 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.630 H:M:S, 15.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 471.189 micros/op 8434 ops/sec 20.274 seconds 170996 operations; 8.2 MB/s (42999 of 42999 found) Microseconds per read: Count: 170996 Average: 471.2224 StdDev: 1183.24 Min: 3 Median: 354.4632 Max: 83011 Percentiles: P50: 354.46 P75: 518.34 P99: 3238.26 P99.9: 13074.46 P99.99: 56965.00 ------------------------------------------------------ ( 2, 3 ] 3 0.002% 0.002% ( 3, 4 ] 21 0.012% 0.014% ( 4, 6 ] 56 0.033% 0.047% ( 6, 10 ] 198 0.116% 0.163% ( 10, 15 ] 302 0.177% 0.339% ( 15, 22 ] 630 0.368% 0.708% ( 22, 34 ] 2584 1.511% 2.219% ( 34, 51 ] 2044 1.195% 3.414% ( 51, 76 ] 3000 1.754% 5.169% ( 76, 110 ] 1027 0.601% 5.769% ( 110, 170 ] 8833 5.166% 10.935% # ( 170, 250 ] 36308 21.233% 32.168% #### ( 250, 380 ] 37946 22.191% 54.359% #### ( 380, 580 ] 51025 29.840% 84.199% ###### ( 580, 870 ] 16613 9.715% 93.914% ## ( 870, 1300 ] 5347 3.127% 97.041% # ( 1300, 1900 ] 1637 0.957% 97.999% ( 1900, 2900 ] 1450 0.848% 98.847% ( 2900, 4400 ] 1162 0.680% 99.526% ( 4400, 6600 ] 422 0.247% 99.773% ( 6600, 9900 ] 169 0.099% 99.872% ( 9900, 14000 ] 62 0.036% 99.908% ( 14000, 22000 ] 67 0.039% 99.947% ( 22000, 33000 ] 45 0.026% 99.974% ( 33000, 50000 ] 24 0.014% 99.988% ( 50000, 75000 ] 14 0.008% 99.996% ( 75000, 110000 ] 7 0.004% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 2876346 rocksdb.bloom.filter.full.positive COUNT : 198415 rocksdb.bloom.filter.full.true.positive COUNT : 170292 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 704 rocksdb.memtable.miss COUNT : 170292 rocksdb.l0.hit COUNT : 24994 rocksdb.l1.hit COUNT : 2396 rocksdb.l2andup.hit COUNT : 142902 rocksdb.compaction.key.drop.new COUNT : 839667 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3673885 rocksdb.number.keys.read COUNT : 170996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3791449320 rocksdb.bytes.read COUNT : 170996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 512 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 6693228 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 15377770849 rocksdb.compact.write.bytes COUNT : 14509685522 rocksdb.flush.write.bytes COUNT : 3693181479 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 27 rocksdb.number.superversion_acquires COUNT : 665 rocksdb.number.superversion_releases COUNT : 645 rocksdb.number.superversion_cleanups COUNT : 166 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3669410816 rocksdb.memtable.garbage.bytes.at.flush COUNT : 6930432 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 15455068354 rocksdb.non.last.level.read.count COUNT : 3705458 rocksdb.block.checksum.compute.count COUNT : 3926051 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 348.559918 P95 : 1015.707317 P99 : 3236.258621 P100 : 83010.000000 COUNT : 170996 SUM : 80129028 rocksdb.db.write.micros P50 : 1.473209 P95 : 3.881767 P99 : 7.246588 P100 : 526459.000000 COUNT : 3673885 SUM : 15661178 rocksdb.compaction.times.micros P50 : 1508000.000000 P95 : 3250000.000000 P99 : 4660283.000000 P100 : 4660283.000000 COUNT : 50 SUM : 79628653 rocksdb.compaction.times.cpu_micros P50 : 925384.615385 P95 : 1725000.000000 P99 : 3479480.000000 P100 : 3479480.000000 COUNT : 50 SUM : 49722158 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 628.333333 P95 : 9192.857143 P99 : 10586.000000 P100 : 10586.000000 COUNT : 50 SUM : 111727 rocksdb.compaction.outfile.sync.micros P50 : 5356.521739 P95 : 95562.500000 P99 : 183600.000000 P100 : 222929.000000 COUNT : 166 SUM : 3384848 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 175.000000 P95 : 2383.333333 P99 : 4345.000000 P100 : 12025.000000 COUNT : 111 SUM : 71725 rocksdb.table.open.io.micros P50 : 3878.260870 P95 : 16133.333333 P99 : 48980.000000 P100 : 71540.000000 COUNT : 512 SUM : 2699224 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 2.734673 P95 : 6.130580 P99 : 301.634217 P100 : 63224.000000 COUNT : 3725114 SUM : 53550640 rocksdb.read.block.get.micros P50 : 267.123238 P95 : 783.232869 P99 : 2662.460568 P100 : 70339.000000 COUNT : 201050 SUM : 68761871 rocksdb.write.raw.block.micros P50 : 0.521181 P95 : 0.990245 P99 : 4.300861 P100 : 66824.000000 COUNT : 4419776 SUM : 11753295 rocksdb.numfiles.in.singlecompaction P50 : 3.227273 P95 : 5.816667 P99 : 8.760000 P100 : 9.000000 COUNT : 62 SUM : 227 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.598605 P95 : 1122.872988 P99 : 1271.402924 P100 : 526282.000000 COUNT : 32193 SUM : 6700803 rocksdb.sst.read.micros P50 : 1.436471 P95 : 179.432160 P99 : 528.757968 P100 : 70334.000000 COUNT : 3706404 SUM : 111098602 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 170996 SUM : 170996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3673885 SUM : 3791449320 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 231818.181818 P95 : 678750.000000 P99 : 725273.000000 P100 : 725273.000000 COUNT : 50 SUM : 14266091 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 180 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0