[2024-11-28 11:40:01.035812] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 23.11.0 initialization... [2024-11-28 11:40:01.035916] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1283088 ] Set seed to 1732790401033431 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-28 11:40:01.137479] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-28 11:40:01.339364] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-28 11:40:01.339413] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1283088' to capture a snapshot of events at runtime. [2024-11-28 11:40:01.339423] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1283088 for offline analysis/debug. [2024-11-28 11:40:01.339447] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Thu Nov 28 11:40:10 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/28-11:40:17 ... thread 0: (1000000,1000000) ops and (144618.8,144618.8) ops/second in (6.914730,6.914730) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/12 3.19 GB 50.3 0.4 0.0 0.4 1.2 0.9 0.0 1.4 43.3 144.3 8.73 4.21 11 0.793 383K 501 0.0 0.0 L1 2/2 77.42 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 14/3 1.11 GB 0.9 0.2 0.1 0.0 0.2 0.1 0.1 1.3 102.0 101.3 1.67 0.84 1 1.672 130K 0 0.0 0.0 L4 189/22 11.01 GB 1.0 3.7 1.5 2.1 3.6 1.5 0.0 2.4 95.8 94.7 39.27 18.40 18 2.182 985K 0 0.0 0.0 L5 1104/0 90.86 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1324/39 106.24 GB 0.0 4.2 1.7 2.6 5.0 2.5 0.8 5.8 86.8 103.6 49.67 23.45 30 1.656 1500K 501 0.0 0.0 Int 0/0 0.00 KB 0.0 4.2 1.7 2.6 5.0 2.5 0.8 5.8 86.8 103.6 49.67 23.45 30 1.656 1500K 501 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.2 1.7 2.6 4.2 1.6 0.0 0.0 99.8 98.7 43.20 20.76 20 2.160 1500K 501 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 136.3 6.47 2.69 10 0.647 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.1 total, 6.9 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.03 GB write, 724.60 MB/s write, 4.21 GB read, 607.22 MB/s read, 49.7 seconds Interval compaction: 5.03 GB write, 744.00 MB/s write, 4.21 GB read, 623.48 MB/s read, 49.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 19, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 19, interval: 19 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 323567 Average: 15.9953 StdDev: 242.41 Min: 1 Median: 1.9138 Max: 42630 Percentiles: P50: 1.91 P75: 2.89 P99: 14.14 P99.9: 2747.58 P99.99: 4728.40 ------------------------------------------------------ [ 0, 1 ] 12790 3.953% 3.953% # ( 1, 2 ] 163054 50.393% 54.345% ########## ( 2, 3 ] 74881 23.142% 77.488% ##### ( 3, 4 ] 27302 8.438% 85.926% ## ( 4, 6 ] 32430 10.023% 95.948% ## ( 6, 10 ] 8213 2.538% 98.487% # ( 10, 15 ] 2007 0.620% 99.107% ( 15, 22 ] 246 0.076% 99.183% ( 22, 34 ] 12 0.004% 99.187% ( 34, 51 ] 7 0.002% 99.189% ( 51, 76 ] 4 0.001% 99.190% ( 76, 110 ] 12 0.004% 99.194% ( 110, 170 ] 54 0.017% 99.210% ( 170, 250 ] 116 0.036% 99.246% ( 250, 380 ] 214 0.066% 99.312% ( 380, 580 ] 352 0.109% 99.421% ( 580, 870 ] 419 0.129% 99.551% ( 870, 1300 ] 438 0.135% 99.686% ( 1300, 1900 ] 390 0.121% 99.807% ( 1900, 2900 ] 358 0.111% 99.917% ( 2900, 4400 ] 225 0.070% 99.987% ( 4400, 6600 ] 78 0.024% 100.011% ( 6600, 9900 ] 21 0.006% 100.017% ( 9900, 14000 ] 5 0.002% 100.019% ( 14000, 22000 ] 9 0.003% 100.022% ( 22000, 33000 ] 3 0.001% 100.023% ( 33000, 50000 ] 2 0.001% 100.023% ** Level 1 read latency histogram (micros): Count: 12 Average: 853.3333 StdDev: 996.23 Min: 140 Median: 480.0000 Max: 3847 Percentiles: P50: 480.00 P75: 1085.00 P99: 3847.00 P99.9: 3847.00 P99.99: 3847.00 ------------------------------------------------------ ( 110, 170 ] 1 8.333% 8.333% ## ( 170, 250 ] 2 16.667% 25.000% ### ( 250, 380 ] 2 16.667% 41.667% ### ( 380, 580 ] 2 16.667% 58.333% ### ( 580, 870 ] 1 8.333% 66.667% ## ( 870, 1300 ] 2 16.667% 83.333% ### ( 1300, 1900 ] 1 8.333% 91.667% ## ( 2900, 4400 ] 1 8.333% 100.000% ## ** Level 2 read latency histogram (micros): Count: 65416 Average: 14.7007 StdDev: 277.12 Min: 1 Median: 1.8751 Max: 47875 Percentiles: P50: 1.88 P75: 2.96 P99: 13.13 P99.9: 2834.75 P99.99: 6260.85 ------------------------------------------------------ [ 0, 1 ] 3808 5.821% 5.821% # ( 1, 2 ] 33024 50.483% 56.304% ########## ( 2, 3 ] 12748 19.488% 75.792% #### ( 3, 4 ] 6306 9.640% 85.432% ## ( 4, 6 ] 6730 10.288% 95.720% ## ( 6, 10 ] 1818 2.779% 98.499% # ( 10, 15 ] 524 0.801% 99.300% ( 15, 22 ] 68 0.104% 99.404% ( 22, 34 ] 8 0.012% 99.416% ( 34, 51 ] 3 0.005% 99.421% ( 51, 76 ] 3 0.005% 99.425% ( 76, 110 ] 3 0.005% 99.430% ( 110, 170 ] 1 0.002% 99.431% ( 170, 250 ] 2 0.003% 99.434% ( 250, 380 ] 16 0.024% 99.459% ( 380, 580 ] 19 0.029% 99.488% ( 580, 870 ] 34 0.052% 99.540% ( 870, 1300 ] 66 0.101% 99.641% ( 1300, 1900 ] 92 0.141% 99.781% ( 1900, 2900 ] 83 0.127% 99.908% ( 2900, 4400 ] 45 0.069% 99.977% ( 4400, 6600 ] 10 0.015% 99.992% ( 6600, 9900 ] 2 0.003% 99.995% ( 14000, 22000 ] 1 0.002% 99.997% ( 22000, 33000 ] 1 0.002% 99.998% ( 33000, 50000 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 213043 Average: 14.5575 StdDev: 270.00 Min: 1 Median: 1.9161 Max: 48654 Percentiles: P50: 1.92 P75: 2.95 P99: 11.73 P99.9: 2371.32 P99.99: 3508.03 ------------------------------------------------------ [ 0, 1 ] 8217 3.857% 3.857% # ( 1, 2 ] 107302 50.366% 54.223% ########## ( 2, 3 ] 46778 21.957% 76.180% #### ( 3, 4 ] 20407 9.579% 85.759% ## ( 4, 6 ] 22114 10.380% 96.139% ## ( 6, 10 ] 5643 2.649% 98.788% # ( 10, 15 ] 1303 0.612% 99.400% ( 15, 22 ] 180 0.084% 99.484% ( 22, 34 ] 12 0.006% 99.490% ( 34, 51 ] 7 0.003% 99.493% ( 51, 76 ] 13 0.006% 99.499% ( 76, 110 ] 2 0.001% 99.500% ( 110, 170 ] 10 0.005% 99.505% ( 170, 250 ] 11 0.005% 99.510% ( 250, 380 ] 35 0.016% 99.526% ( 380, 580 ] 76 0.036% 99.562% ( 580, 870 ] 106 0.050% 99.612% ( 870, 1300 ] 215 0.101% 99.713% ( 1300, 1900 ] 275 0.129% 99.842% ( 1900, 2900 ] 263 0.123% 99.965% ( 2900, 4400 ] 130 0.061% 100.026% ( 4400, 6600 ] 37 0.017% 100.044% ( 6600, 9900 ] 12 0.006% 100.049% ( 9900, 14000 ] 3 0.001% 100.051% ( 14000, 22000 ] 4 0.002% 100.053% ( 22000, 33000 ] 4 0.002% 100.054% ( 33000, 50000 ] 2 0.001% 100.055% ** Level 4 read latency histogram (micros): Count: 714302 Average: 25.5524 StdDev: 377.21 Min: 1 Median: 1.9059 Max: 97446 Percentiles: P50: 1.91 P75: 2.93 P99: 18.35 P99.9: 2637.29 P99.99: 3222.61 ------------------------------------------------------ [ 0, 1 ] 28588 4.002% 4.002% # ( 1, 2 ] 362675 50.773% 54.776% ########## ( 2, 3 ] 155888 21.824% 76.599% #### ( 3, 4 ] 66976 9.376% 85.976% ## ( 4, 6 ] 69195 9.687% 95.663% ## ( 6, 10 ] 18598 2.604% 98.267% # ( 10, 15 ] 4899 0.686% 98.952% ( 15, 22 ] 713 0.100% 99.052% ( 22, 34 ] 60 0.008% 99.061% ( 34, 51 ] 30 0.004% 99.065% ( 51, 76 ] 23 0.003% 99.068% ( 76, 110 ] 30 0.004% 99.072% ( 110, 170 ] 89 0.012% 99.085% ( 170, 250 ] 218 0.031% 99.115% ( 250, 380 ] 310 0.043% 99.159% ( 380, 580 ] 517 0.072% 99.231% ( 580, 870 ] 864 0.121% 99.352% ( 870, 1300 ] 1273 0.178% 99.530% ( 1300, 1900 ] 1495 0.209% 99.739% ( 1900, 2900 ] 1558 0.218% 99.958% ( 2900, 4400 ] 1086 0.152% 100.110% ( 4400, 6600 ] 415 0.058% 100.168% ( 6600, 9900 ] 67 0.009% 100.177% ( 9900, 14000 ] 7 0.001% 100.178% ( 14000, 22000 ] 10 0.001% 100.179% ( 22000, 33000 ] 15 0.002% 100.182% ( 33000, 50000 ] 8 0.001% 100.183% ( 50000, 75000 ] 4 0.001% 100.183% ( 75000, 110000 ] 2 0.000% 100.184% ** Level 5 read latency histogram (micros): Count: 9560 Average: 2028.1628 StdDev: 2465.67 Min: 83 Median: 1377.7011 Max: 84542 Percentiles: P50: 1377.70 P75: 2869.09 P99: 8603.02 P99.9: 26840.00 P99.99: 84542.00 ------------------------------------------------------ ( 76, 110 ] 19 0.199% 0.199% ( 110, 170 ] 148 1.548% 1.747% ( 170, 250 ] 393 4.111% 5.858% # ( 250, 380 ] 619 6.475% 12.333% # ( 380, 580 ] 899 9.404% 21.736% ## ( 580, 870 ] 1201 12.563% 34.299% ### ( 870, 1300 ] 1332 13.933% 48.232% ### ( 1300, 1900 ] 1305 13.651% 61.883% ### ( 1900, 2900 ] 1294 13.536% 75.418% ### ( 2900, 4400 ] 1275 13.337% 88.755% ### ( 4400, 6600 ] 875 9.153% 97.908% ## ( 6600, 9900 ] 172 1.799% 99.707% ( 9900, 14000 ] 16 0.167% 99.874% ( 14000, 22000 ] 2 0.021% 99.895% ( 22000, 33000 ] 1 0.010% 99.906% ( 33000, 50000 ] 5 0.052% 99.958% ( 50000, 75000 ] 2 0.021% 99.979% ( 75000, 110000 ] 2 0.021% 100.000% ** DB Stats ** Uptime(secs): 7.1 total, 6.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 138.58 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.237 H:M:S, 45.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 142.27 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.237 H:M:S, 46.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 5 num-running-flushes: 1 2024/11/28-11:40:20 ... thread 0: (1000000,2000000) ops and (280138.1,190759.6) ops/second in (3.569668,10.484398) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/12 4.17 GB 163.6 1.5 0.0 1.5 3.3 1.8 0.0 1.8 91.4 205.5 16.54 11.32 31 0.534 1535K 1783 0.0 0.0 L1 2/2 77.42 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 11/0 913.19 MB 0.9 0.2 0.1 0.0 0.2 0.1 0.1 1.3 102.0 101.3 1.67 0.84 1 1.672 130K 0 0.0 0.0 L4 174/0 9.64 GB 1.0 5.4 1.7 3.7 5.3 1.6 0.0 3.0 105.5 102.8 52.36 25.98 21 2.493 1044K 0 0.0 0.0 L5 1124/0 92.35 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1330/14 107.13 GB 0.0 7.0 1.9 5.2 8.7 3.6 2.3 4.7 102.1 126.8 70.57 38.14 53 1.332 2710K 1783 0.0 0.0 Int 0/0 0.00 KB 0.0 7.0 1.9 5.2 8.7 3.6 2.3 4.7 102.1 126.8 70.57 38.14 53 1.332 2710K 1783 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 7.0 1.9 5.2 6.9 1.7 0.0 0.0 118.6 116.2 60.76 32.66 27 2.250 2710K 1783 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 192.5 9.82 5.48 26 0.378 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 10.7 total, 10.5 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.74 GB write, 838.80 MB/s write, 7.04 GB read, 675.38 MB/s read, 70.6 seconds Interval compaction: 8.74 GB write, 853.61 MB/s write, 7.04 GB read, 687.31 MB/s read, 70.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 1, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 1, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 19, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 1, total-stops: 19, interval: 20 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 656026 Average: 15.3570 StdDev: 178.41 Min: 1 Median: 1.7342 Max: 42630 Percentiles: P50: 1.73 P75: 2.61 P99: 265.25 P99.9: 1519.41 P99.99: 2707.14 ------------------------------------------------------ [ 0, 1 ] 68773 10.483% 10.483% ## ( 1, 2 ] 353072 53.820% 64.303% ########### ( 2, 3 ] 115067 17.540% 81.843% #### ( 3, 4 ] 42843 6.531% 88.374% # ( 4, 6 ] 44204 6.738% 95.112% # ( 6, 10 ] 10627 1.620% 96.732% ( 10, 15 ] 2640 0.402% 97.134% ( 15, 22 ] 371 0.057% 97.191% ( 22, 34 ] 40 0.006% 97.197% ( 34, 51 ] 15 0.002% 97.199% ( 51, 76 ] 7 0.001% 97.200% ( 76, 110 ] 475 0.072% 97.273% ( 110, 170 ] 5922 0.903% 98.175% ( 170, 250 ] 5141 0.784% 98.959% ( 250, 380 ] 2291 0.349% 99.308% ( 380, 580 ] 1904 0.290% 99.598% ( 580, 870 ] 1163 0.177% 99.776% ( 870, 1300 ] 657 0.100% 99.876% ( 1300, 1900 ] 432 0.066% 99.942% ( 1900, 2900 ] 392 0.060% 100.002% ( 2900, 4400 ] 240 0.037% 100.038% ( 4400, 6600 ] 78 0.012% 100.050% ( 6600, 9900 ] 21 0.003% 100.053% ( 9900, 14000 ] 5 0.001% 100.054% ( 14000, 22000 ] 9 0.001% 100.055% ( 22000, 33000 ] 3 0.000% 100.056% ( 33000, 50000 ] 2 0.000% 100.056% ** Level 1 read latency histogram (micros): Count: 68 Average: 410.1912 StdDev: 501.55 Min: 92 Median: 261.8182 Max: 3847 Percentiles: P50: 261.82 P75: 496.67 P99: 3380.00 P99.9: 3847.00 P99.99: 3847.00 ------------------------------------------------------ ( 76, 110 ] 1 1.471% 1.471% ( 110, 170 ] 15 22.059% 23.529% #### ( 170, 250 ] 17 25.000% 48.529% ##### ( 250, 380 ] 11 16.176% 64.706% ### ( 380, 580 ] 12 17.647% 82.353% #### ( 580, 870 ] 7 10.294% 92.647% ## ( 870, 1300 ] 3 4.412% 97.059% # ( 1300, 1900 ] 1 1.471% 98.529% ( 2900, 4400 ] 1 1.471% 100.000% ** Level 2 read latency histogram (micros): Count: 65416 Average: 14.7007 StdDev: 277.12 Min: 1 Median: 1.8751 Max: 47875 Percentiles: P50: 1.88 P75: 2.96 P99: 13.13 P99.9: 2834.75 P99.99: 6260.85 ------------------------------------------------------ [ 0, 1 ] 3808 5.821% 5.821% # ( 1, 2 ] 33024 50.483% 56.304% ########## ( 2, 3 ] 12748 19.488% 75.792% #### ( 3, 4 ] 6306 9.640% 85.432% ## ( 4, 6 ] 6730 10.288% 95.720% ## ( 6, 10 ] 1818 2.779% 98.499% # ( 10, 15 ] 524 0.801% 99.300% ( 15, 22 ] 68 0.104% 99.404% ( 22, 34 ] 8 0.012% 99.416% ( 34, 51 ] 3 0.005% 99.421% ( 51, 76 ] 3 0.005% 99.425% ( 76, 110 ] 3 0.005% 99.430% ( 110, 170 ] 1 0.002% 99.431% ( 170, 250 ] 2 0.003% 99.434% ( 250, 380 ] 16 0.024% 99.459% ( 380, 580 ] 19 0.029% 99.488% ( 580, 870 ] 34 0.052% 99.540% ( 870, 1300 ] 66 0.101% 99.641% ( 1300, 1900 ] 92 0.141% 99.781% ( 1900, 2900 ] 83 0.127% 99.908% ( 2900, 4400 ] 45 0.069% 99.977% ( 4400, 6600 ] 10 0.015% 99.992% ( 6600, 9900 ] 2 0.003% 99.995% ( 14000, 22000 ] 1 0.002% 99.997% ( 22000, 33000 ] 1 0.002% 99.998% ( 33000, 50000 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 225757 Average: 14.4948 StdDev: 262.71 Min: 1 Median: 1.9119 Max: 48654 Percentiles: P50: 1.91 P75: 2.94 P99: 13.36 P99.9: 2315.37 P99.99: 3470.28 ------------------------------------------------------ [ 0, 1 ] 8840 3.916% 3.916% # ( 1, 2 ] 114096 50.539% 54.455% ########## ( 2, 3 ] 49477 21.916% 76.371% #### ( 3, 4 ] 21320 9.444% 85.815% ## ( 4, 6 ] 22944 10.163% 95.978% ## ( 6, 10 ] 5904 2.615% 98.593% # ( 10, 15 ] 1366 0.605% 99.198% ( 15, 22 ] 185 0.082% 99.280% ( 22, 34 ] 12 0.005% 99.286% ( 34, 51 ] 7 0.003% 99.289% ( 51, 76 ] 13 0.006% 99.294% ( 76, 110 ] 12 0.005% 99.300% ( 110, 170 ] 199 0.088% 99.388% ( 170, 250 ] 177 0.078% 99.466% ( 250, 380 ] 98 0.043% 99.510% ( 380, 580 ] 142 0.063% 99.573% ( 580, 870 ] 132 0.058% 99.631% ( 870, 1300 ] 223 0.099% 99.730% ( 1300, 1900 ] 275 0.122% 99.852% ( 1900, 2900 ] 263 0.116% 99.968% ( 2900, 4400 ] 130 0.058% 100.026% ( 4400, 6600 ] 37 0.016% 100.042% ( 6600, 9900 ] 12 0.005% 100.047% ( 9900, 14000 ] 3 0.001% 100.049% ( 14000, 22000 ] 4 0.002% 100.050% ( 22000, 33000 ] 4 0.002% 100.052% ( 33000, 50000 ] 2 0.001% 100.053% ** Level 4 read latency histogram (micros): Count: 832531 Average: 23.8982 StdDev: 350.48 Min: 1 Median: 1.8927 Max: 97446 Percentiles: P50: 1.89 P75: 2.91 P99: 222.81 P99.9: 2486.40 P99.99: 3038.90 ------------------------------------------------------ [ 0, 1 ] 34449 4.138% 4.138% # ( 1, 2 ] 427702 51.374% 55.512% ########## ( 2, 3 ] 178831 21.480% 76.992% #### ( 3, 4 ] 75787 9.103% 86.095% ## ( 4, 6 ] 76902 9.237% 95.332% ## ( 6, 10 ] 21199 2.546% 97.879% # ( 10, 15 ] 5453 0.655% 98.534% ( 15, 22 ] 800 0.096% 98.630% ( 22, 34 ] 63 0.008% 98.637% ( 34, 51 ] 30 0.004% 98.641% ( 51, 76 ] 24 0.003% 98.644% ( 76, 110 ] 155 0.019% 98.662% ( 110, 170 ] 1693 0.203% 98.866% ( 170, 250 ] 1693 0.203% 99.069% ( 250, 380 ] 925 0.111% 99.180% ( 380, 580 ] 1053 0.126% 99.307% ( 580, 870 ] 1132 0.136% 99.443% ( 870, 1300 ] 1366 0.164% 99.607% ( 1300, 1900 ] 1522 0.183% 99.790% ( 1900, 2900 ] 1568 0.188% 99.978% ( 2900, 4400 ] 1088 0.131% 100.109% ( 4400, 6600 ] 415 0.050% 100.158% ( 6600, 9900 ] 67 0.008% 100.166% ( 9900, 14000 ] 7 0.001% 100.167% ( 14000, 22000 ] 10 0.001% 100.169% ( 22000, 33000 ] 15 0.002% 100.170% ( 33000, 50000 ] 8 0.001% 100.171% ( 50000, 75000 ] 4 0.000% 100.172% ( 75000, 110000 ] 2 0.000% 100.172% ** Level 5 read latency histogram (micros): Count: 44305 Average: 647.3250 StdDev: 1373.00 Min: 83 Median: 229.2944 Max: 84542 Percentiles: P50: 229.29 P75: 528.02 P99: 5992.37 P99.9: 9587.17 P99.99: 48536.30 ------------------------------------------------------ ( 76, 110 ] 1110 2.505% 2.505% # ( 110, 170 ] 12765 28.812% 31.317% ###### ( 170, 250 ] 11168 25.207% 56.524% ##### ( 250, 380 ] 4939 11.148% 67.672% ## ( 380, 580 ] 4387 9.902% 77.574% ## ( 580, 870 ] 2881 6.503% 84.076% # ( 870, 1300 ] 1843 4.160% 88.236% # ( 1300, 1900 ] 1453 3.280% 91.516% # ( 1900, 2900 ] 1368 3.088% 94.603% # ( 2900, 4400 ] 1311 2.959% 97.562% # ( 4400, 6600 ] 880 1.986% 99.549% ( 6600, 9900 ] 172 0.388% 99.937% ( 9900, 14000 ] 16 0.036% 99.973% ( 14000, 22000 ] 2 0.005% 99.977% ( 22000, 33000 ] 1 0.002% 99.980% ( 33000, 50000 ] 5 0.011% 99.991% ( 50000, 75000 ] 2 0.005% 99.995% ( 75000, 110000 ] 2 0.005% 100.000% ** DB Stats ** Uptime(secs): 10.7 total, 3.6 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 184.45 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.349 H:M:S, 31.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 275.72 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.112 H:M:S, 3.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/28-11:40:31 ... thread 0: (1000000,3000000) ops and (92796.3,141105.5) ops/second in (10.776287,21.260685) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/12 5.08 GB 256.9 4.0 0.0 4.0 6.8 2.8 0.0 2.4 139.8 236.1 29.29 23.34 51 0.574 4156K 13K 0.0 0.0 L1 2/2 77.42 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 11/0 913.19 MB 0.9 0.2 0.1 0.0 0.2 0.1 0.1 1.3 102.0 101.3 1.67 0.84 1 1.672 130K 0 0.0 0.0 L4 174/0 9.64 GB 1.0 5.4 1.7 3.7 5.3 1.6 0.0 3.0 105.5 102.8 52.36 25.98 21 2.493 1044K 0 0.0 0.0 L5 1124/0 92.35 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1327/14 108.04 GB 0.0 9.6 1.9 7.7 12.2 4.5 2.3 4.4 117.5 149.6 83.32 50.15 73 1.141 5331K 13K 0.0 0.0 Int 0/0 0.00 KB 0.0 9.6 1.9 7.7 12.2 4.5 2.3 4.4 117.5 149.6 83.32 50.15 73 1.141 5331K 13K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.6 1.9 7.7 9.4 1.7 0.0 0.0 138.8 136.6 70.53 42.16 32 2.204 5331K 13K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 221.6 12.79 8.00 41 0.312 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 21.4 total, 21.3 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.17 GB write, 581.21 MB/s write, 9.56 GB read, 456.40 MB/s read, 83.3 seconds Interval compaction: 12.17 GB write, 586.28 MB/s write, 9.56 GB read, 460.37 MB/s read, 83.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 14, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 14, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 19, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 14, total-stops: 19, interval: 33 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1393549 Average: 18.0172 StdDev: 131.98 Min: 0 Median: 1.5783 Max: 42630 Percentiles: P50: 1.58 P75: 2.24 P99: 265.14 P99.9: 831.20 P99.99: 1583.27 ------------------------------------------------------ [ 0, 1 ] 271574 19.488% 19.488% #### ( 1, 2 ] 735299 52.764% 72.252% ########### ( 2, 3 ] 158806 11.396% 83.648% ## ( 3, 4 ] 63585 4.563% 88.211% # ( 4, 6 ] 56844 4.079% 92.290% # ( 6, 10 ] 13019 0.934% 93.224% ( 10, 15 ] 3433 0.246% 93.471% ( 15, 22 ] 513 0.037% 93.508% ( 22, 34 ] 87 0.006% 93.514% ( 34, 51 ] 19 0.001% 93.515% ( 51, 76 ] 13 0.001% 93.516% ( 76, 110 ] 3920 0.281% 93.797% ( 110, 170 ] 43585 3.128% 96.925% # ( 170, 250 ] 28115 2.018% 98.942% ( 250, 380 ] 6898 0.495% 99.437% ( 380, 580 ] 4767 0.342% 99.780% ( 580, 870 ] 1940 0.139% 99.919% ( 870, 1300 ] 777 0.056% 99.975% ( 1300, 1900 ] 461 0.033% 100.008% ( 1900, 2900 ] 413 0.030% 100.037% ( 2900, 4400 ] 259 0.019% 100.056% ( 4400, 6600 ] 79 0.006% 100.061% ( 6600, 9900 ] 21 0.002% 100.063% ( 9900, 14000 ] 5 0.000% 100.063% ( 14000, 22000 ] 9 0.001% 100.064% ( 22000, 33000 ] 3 0.000% 100.064% ( 33000, 50000 ] 2 0.000% 100.064% ** Level 1 read latency histogram (micros): Count: 273 Average: 237.2051 StdDev: 281.20 Min: 92 Median: 169.7710 Max: 3847 Percentiles: P50: 169.77 P75: 232.30 P99: 1195.37 P99.9: 3847.00 P99.99: 3847.00 ------------------------------------------------------ ( 76, 110 ] 6 2.198% 2.198% ( 110, 170 ] 131 47.985% 50.183% ########## ( 170, 250 ] 87 31.868% 82.051% ###### ( 250, 380 ] 17 6.227% 88.278% # ( 380, 580 ] 17 6.227% 94.505% # ( 580, 870 ] 10 3.663% 98.168% # ( 870, 1300 ] 3 1.099% 99.267% ( 1300, 1900 ] 1 0.366% 99.634% ( 2900, 4400 ] 1 0.366% 100.000% ** Level 2 read latency histogram (micros): Count: 65416 Average: 14.7007 StdDev: 277.12 Min: 1 Median: 1.8751 Max: 47875 Percentiles: P50: 1.88 P75: 2.96 P99: 13.13 P99.9: 2834.75 P99.99: 6260.85 ------------------------------------------------------ [ 0, 1 ] 3808 5.821% 5.821% # ( 1, 2 ] 33024 50.483% 56.304% ########## ( 2, 3 ] 12748 19.488% 75.792% #### ( 3, 4 ] 6306 9.640% 85.432% ## ( 4, 6 ] 6730 10.288% 95.720% ## ( 6, 10 ] 1818 2.779% 98.499% # ( 10, 15 ] 524 0.801% 99.300% ( 15, 22 ] 68 0.104% 99.404% ( 22, 34 ] 8 0.012% 99.416% ( 34, 51 ] 3 0.005% 99.421% ( 51, 76 ] 3 0.005% 99.425% ( 76, 110 ] 3 0.005% 99.430% ( 110, 170 ] 1 0.002% 99.431% ( 170, 250 ] 2 0.003% 99.434% ( 250, 380 ] 16 0.024% 99.459% ( 380, 580 ] 19 0.029% 99.488% ( 580, 870 ] 34 0.052% 99.540% ( 870, 1300 ] 66 0.101% 99.641% ( 1300, 1900 ] 92 0.141% 99.781% ( 1900, 2900 ] 83 0.127% 99.908% ( 2900, 4400 ] 45 0.069% 99.977% ( 4400, 6600 ] 10 0.015% 99.992% ( 6600, 9900 ] 2 0.003% 99.995% ( 14000, 22000 ] 1 0.002% 99.997% ( 22000, 33000 ] 1 0.002% 99.998% ( 33000, 50000 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 227811 Average: 16.0294 StdDev: 262.21 Min: 1 Median: 1.9209 Max: 48654 Percentiles: P50: 1.92 P75: 2.97 P99: 167.06 P99.9: 2309.81 P99.99: 3467.91 ------------------------------------------------------ [ 0, 1 ] 8840 3.880% 3.880% # ( 1, 2 ] 114096 50.084% 53.964% ########## ( 2, 3 ] 49477 21.718% 75.682% #### ( 3, 4 ] 21320 9.359% 85.041% ## ( 4, 6 ] 22944 10.072% 95.113% ## ( 6, 10 ] 5904 2.592% 97.704% # ( 10, 15 ] 1366 0.600% 98.304% ( 15, 22 ] 185 0.081% 98.385% ( 22, 34 ] 12 0.005% 98.390% ( 34, 51 ] 7 0.003% 98.393% ( 51, 76 ] 13 0.006% 98.399% ( 76, 110 ] 123 0.054% 98.453% ( 110, 170 ] 1310 0.575% 99.028% ( 170, 250 ] 780 0.342% 99.371% ( 250, 380 ] 240 0.105% 99.476% ( 380, 580 ] 207 0.091% 99.567% ( 580, 870 ] 147 0.065% 99.631% ( 870, 1300 ] 229 0.101% 99.732% ( 1300, 1900 ] 275 0.121% 99.853% ( 1900, 2900 ] 264 0.116% 99.968% ( 2900, 4400 ] 130 0.057% 100.025% ( 4400, 6600 ] 37 0.016% 100.042% ( 6600, 9900 ] 12 0.005% 100.047% ( 9900, 14000 ] 3 0.001% 100.048% ( 14000, 22000 ] 4 0.002% 100.050% ( 22000, 33000 ] 4 0.002% 100.052% ( 33000, 50000 ] 2 0.001% 100.053% ** Level 4 read latency histogram (micros): Count: 853558 Average: 27.6409 StdDev: 347.41 Min: 1 Median: 1.9171 Max: 97446 Percentiles: P50: 1.92 P75: 2.99 P99: 336.47 P99.9: 2475.15 P99.99: 3035.87 ------------------------------------------------------ [ 0, 1 ] 34449 4.036% 4.036% # ( 1, 2 ] 427793 50.119% 54.155% ########## ( 2, 3 ] 179192 20.994% 75.148% #### ( 3, 4 ] 76071 8.912% 84.060% ## ( 4, 6 ] 77416 9.070% 93.130% ## ( 6, 10 ] 21251 2.490% 95.620% ( 10, 15 ] 5470 0.641% 96.261% ( 15, 22 ] 802 0.094% 96.355% ( 22, 34 ] 63 0.007% 96.362% ( 34, 51 ] 30 0.004% 96.366% ( 51, 76 ] 24 0.003% 96.368% ( 76, 110 ] 1144 0.134% 96.503% ( 110, 170 ] 12019 1.408% 97.911% ( 170, 250 ] 7849 0.920% 98.830% ( 250, 380 ] 2179 0.255% 99.085% ( 380, 580 ] 1805 0.211% 99.297% ( 580, 870 ] 1323 0.155% 99.452% ( 870, 1300 ] 1391 0.163% 99.615% ( 1300, 1900 ] 1527 0.179% 99.794% ( 1900, 2900 ] 1576 0.185% 99.978% ( 2900, 4400 ] 1089 0.128% 100.106% ( 4400, 6600 ] 415 0.049% 100.155% ( 6600, 9900 ] 67 0.008% 100.162% ( 9900, 14000 ] 7 0.001% 100.163% ( 14000, 22000 ] 10 0.001% 100.164% ( 22000, 33000 ] 15 0.002% 100.166% ( 33000, 50000 ] 8 0.001% 100.167% ( 50000, 75000 ] 4 0.000% 100.168% ( 75000, 110000 ] 2 0.000% 100.168% ** Level 5 read latency histogram (micros): Count: 190006 Average: 294.1822 StdDev: 697.27 Min: 81 Median: 167.6943 Max: 84542 Percentiles: P50: 167.69 P75: 232.74 P99: 3468.11 P99.9: 6791.75 P99.99: 12206.10 ------------------------------------------------------ ( 76, 110 ] 8549 4.499% 4.499% # ( 110, 170 ] 89909 47.319% 51.818% ######### ( 170, 250 ] 56162 29.558% 81.376% ###### ( 250, 380 ] 13664 7.191% 88.568% # ( 380, 580 ] 9995 5.260% 93.828% # ( 580, 870 ] 4393 2.312% 96.140% ( 870, 1300 ] 2039 1.073% 97.213% ( 1300, 1900 ] 1495 0.787% 98.000% ( 1900, 2900 ] 1400 0.737% 98.737% ( 2900, 4400 ] 1320 0.695% 99.432% ( 4400, 6600 ] 880 0.463% 99.895% ( 6600, 9900 ] 172 0.091% 99.985% ( 9900, 14000 ] 16 0.008% 99.994% ( 14000, 22000 ] 2 0.001% 99.995% ( 22000, 33000 ] 1 0.001% 99.995% ( 33000, 50000 ] 5 0.003% 99.998% ( 50000, 75000 ] 2 0.001% 99.999% ( 75000, 110000 ] 2 0.001% 100.000% ** DB Stats ** Uptime(secs): 21.4 total, 10.8 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 137.66 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:10.581 H:M:S, 49.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 91.33 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:7.233 H:M:S, 67.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 250.185 micros/op 15976 ops/sec 60.150 seconds 960996 operations; 15.5 MB/s (231999 of 231999 found) Microseconds per read: Count: 960996 Average: 250.1877 StdDev: 841.02 Min: 2 Median: 199.9648 Max: 170828 Percentiles: P50: 199.96 P75: 241.75 P99: 816.72 P99.9: 12595.67 P99.99: 32704.00 ------------------------------------------------------ ( 1, 2 ] 17 0.002% 0.002% ( 2, 3 ] 46 0.005% 0.007% ( 3, 4 ] 30 0.003% 0.010% ( 4, 6 ] 78 0.008% 0.018% ( 6, 10 ] 578 0.060% 0.078% ( 10, 15 ] 4532 0.472% 0.550% ( 15, 22 ] 2463 0.256% 0.806% ( 22, 34 ] 9729 1.012% 1.818% ( 34, 51 ] 2307 0.240% 2.058% ( 51, 76 ] 311 0.032% 2.091% ( 76, 110 ] 3728 0.388% 2.479% ( 110, 170 ] 284373 29.591% 32.070% ###### ( 170, 250 ] 460022 47.869% 79.939% ########## ( 250, 380 ] 117343 12.211% 92.150% ## ( 380, 580 ] 53922 5.611% 97.761% # ( 580, 870 ] 14587 1.518% 99.279% ( 870, 1300 ] 3452 0.359% 99.638% ( 1300, 1900 ] 986 0.103% 99.741% ( 1900, 2900 ] 685 0.071% 99.812% ( 2900, 4400 ] 408 0.042% 99.854% ( 4400, 6600 ] 147 0.015% 99.870% ( 6600, 9900 ] 172 0.018% 99.888% ( 9900, 14000 ] 181 0.019% 99.906% ( 14000, 22000 ] 510 0.053% 99.960% ( 22000, 33000 ] 301 0.031% 99.991% ( 33000, 50000 ] 72 0.007% 99.998% ( 50000, 75000 ] 9 0.001% 99.999% ( 75000, 110000 ] 2 0.000% 99.999% ( 110000, 170000 ] 4 0.000% 100.000% ( 170000, 250000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 17161376 rocksdb.bloom.filter.full.positive COUNT : 1126218 rocksdb.bloom.filter.full.true.positive COUNT : 960633 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 363 rocksdb.memtable.miss COUNT : 960633 rocksdb.l0.hit COUNT : 53760 rocksdb.l1.hit COUNT : 776 rocksdb.l2andup.hit COUNT : 906097 rocksdb.compaction.key.drop.new COUNT : 181173 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3689820 rocksdb.number.keys.read COUNT : 960996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3807894240 rocksdb.bytes.read COUNT : 960996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1492 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 46315855 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 14703666549 rocksdb.compact.write.bytes COUNT : 14520193871 rocksdb.flush.write.bytes COUNT : 3765321153 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 57 rocksdb.number.superversion_acquires COUNT : 630 rocksdb.number.superversion_releases COUNT : 597 rocksdb.number.superversion_cleanups COUNT : 154 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3734938624 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1441792 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 18706272623 rocksdb.non.last.level.read.count COUNT : 4375660 rocksdb.block.checksum.compute.count COUNT : 4699714 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 199.070494 P95 : 480.199296 P99 : 816.127362 P100 : 170826.000000 COUNT : 960996 SUM : 239612250 rocksdb.db.write.micros P50 : 1.533575 P95 : 4.614417 P99 : 12.217225 P100 : 854780.000000 COUNT : 3689820 SUM : 55470761 rocksdb.compaction.times.micros P50 : 1350000.000000 P95 : 5876666.666667 P99 : 6812221.000000 P100 : 6812221.000000 COUNT : 37 SUM : 78011058 rocksdb.compaction.times.cpu_micros P50 : 1042142.857143 P95 : 3705000.000000 P99 : 5686000.000000 P100 : 5768809.000000 COUNT : 37 SUM : 49538497 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 854.464286 P95 : 16800.000000 P99 : 33361.000000 P100 : 33361.000000 COUNT : 53 SUM : 154977 rocksdb.compaction.outfile.sync.micros P50 : 9900.000000 P95 : 114000.000000 P99 : 166800.000000 P100 : 281324.000000 COUNT : 132 SUM : 3124197 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 180.000000 P95 : 4473.333333 P99 : 8192.000000 P100 : 8192.000000 COUNT : 98 SUM : 85989 rocksdb.table.open.io.micros P50 : 11691.851852 P95 : 21953.363229 P99 : 39890.666667 P100 : 158943.000000 COUNT : 1492 SUM : 18219554 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.775788 P95 : 139.763548 P99 : 242.762626 P100 : 64179.000000 COUNT : 3561735 SUM : 84187240 rocksdb.read.block.get.micros P50 : 154.265683 P95 : 307.983079 P99 : 714.559231 P100 : 97571.000000 COUNT : 1136446 SUM : 203713598 rocksdb.write.raw.block.micros P50 : 0.536835 P95 : 1.392091 P99 : 3.454303 P100 : 69595.000000 COUNT : 4428689 SUM : 13260685 rocksdb.numfiles.in.singlecompaction P50 : 3.478261 P95 : 9.350000 P99 : 11.000000 P100 : 11.000000 COUNT : 46 SUM : 207 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.569050 P95 : 2192.777186 P99 : 2758.981876 P100 : 854611.000000 COUNT : 212440 SUM : 46332643 rocksdb.sst.read.micros P50 : 2.138062 P95 : 222.864878 P99 : 465.407571 P100 : 97446.000000 COUNT : 4375683 SUM : 272830849 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 960996 SUM : 960996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3689820 SUM : 3807894240 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 219302.325581 P95 : 899666.666667 P99 : 1060465.000000 P100 : 1060465.000000 COUNT : 53 SUM : 15231556 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 168 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0