[00:48:53] *** Joins: tomzawadzki (~tzawadzk@134.134.139.72) [01:00:30] *** Quits: ziyeyang_ (~ziyeyang@134.134.139.77) (Quit: Leaving) [01:46:44] *** Quits: tsuyoshi (b42b2067@gateway/web/freenode/ip.180.43.32.103) (Ping timeout: 260 seconds) [01:52:32] *** Joins: gila (~gila@5ED4FE92.cm-7-5d.dynamic.ziggo.nl) [02:36:50] *** Quits: changpeng (changpeng@nat/intel/x-fgtafscqyasqtbjv) (Remote host closed the connection) [03:30:08] *** Quits: tomzawadzki (~tzawadzk@134.134.139.72) (Remote host closed the connection) [03:30:13] *** Joins: Tomasz (~tzawadzk@134.134.139.72) [03:55:25] *** Quits: Tomasz (~tzawadzk@134.134.139.72) (Ping timeout: 255 seconds) [04:32:14] *** Quits: gila (~gila@5ED4FE92.cm-7-5d.dynamic.ziggo.nl) (Quit: My Mac Pro has gone to sleep. ZZZzzz…) [04:36:25] *** Joins: Tomasz (tzawadzk@nat/intel/x-knlvchaagqeldddm) [07:56:08] *** Joins: Edwinzrodriguez (d8f01e17@gateway/web/freenode/ip.216.240.30.23) [07:56:59] I'm looking at the stdinc.h changes and I was under the impression that the change would also remove the dpdk header dependencies from source files. [07:57:46] so FYI some website/doc changes I'm working on: (a) website development page: pull out specific copy-n-paste type elements and make this page only exist to provide the awesome overview that it does today (b) update README in repo to be super short/simple copy-n-pastable steps to go from zero to submitting a patch in one series of straightforward steps [07:58:30] what we have now rocks, but I think we can break it up this way to make it easier for both the novice and the expert and also avoid duplicating any of the detailed steps that will change over time (and have that part travel with the code in the repo) [07:58:42] any comments/concerns? [08:26:48] *** Joins: tomzawadzki (~tzawadzk@192.55.54.36) [08:47:26] peluse: sounds good to me [08:48:11] Edwinzrodriguez: stdinc.h is only intended to cover standard C/POSIX headers - but we do need to continue to remove the DPDK headers from SPDK libraries as well [08:48:42] which dependencies are causing you the most trouble right now? then we can prioritize getting those libraries fixed [09:01:57] *** Quits: Tomasz (tzawadzk@nat/intel/x-knlvchaagqeldddm) (Ping timeout: 240 seconds) [09:02:43] *** Quits: tomzawadzki (~tzawadzk@192.55.54.36) (Ping timeout: 260 seconds) [09:14:25] peluse: I think the instructions could be broken up into "how to submit a patch, if you're just going to submit one", and "how to get actually set up with GerritHub such that you can fully participate in the project" [09:22:09] bwalker, hmmm, how would one submit a patch w/o the use of GerritHub? We aren't going to allow pull requests right? [09:22:47] I hit it the first rte header in bdev.c, but its also a problem in event and trace. The trace one will be resolved with the trace abstraction change so is not as critical. [09:26:09] peluse: oh they have to use GerritHub either way, but if they are just submitting one patch they probably don't need to do like 3/4 of the GerritHub Configuration section, and they probably don't care about learning how to submit patch series [09:26:37] Edwinzrodriguez: We're currently working on removing all of those dependencies on DPDK. They'll get abstracted through the env.h header [09:26:59] I've been submitting patches working toward fixing bdev all week [09:27:24] as far as event - I thought you guys weren't going to use that library [09:29:10] bwalker, gotcha. yeah I think we're pretty close to being on the same page. README.md will be super simple on how to clone, install Ds, build, run through pre-submit checks, submit, address comments & resubmit. Everything else will be on spdk.io where it is today [09:29:30] yeah sounds good [09:42:48] bwalker, drv: want to sync up on upcoming submits - there's a whole series from darek that is ready to go in - i've added my +2 as well as +1s from tomek and pawel [09:43:10] we can commit these maybe later this afternoon - but should get them in before poland team gets in tomorrow [09:43:43] my iscsi test speedups i think are all ready (including moving the stub app to test/app) - will be pushing those out for review shortly [09:44:02] would be great to get these in today if only to cut more time off the test pool [09:44:11] drv on vacation [09:44:38] my io_channel series is now passing all tests [09:44:44] btw - i'll be on vacation starting next thursday through the following week [09:45:00] and I'm blocked there on fixing the bdev DPDK stuff [09:47:41] ok - how about i push my iscsi series now and then i'll review your patches while we wait for those to complete [09:48:24] btw - we can use this stub application to speed up the rocksdb tests too :) [09:48:34] ok [09:48:57] we take the dpdk initialization hit on each of the five workloads we run [09:59:50] that stub thing made me start thinking [10:00:02] you stopped thinking? [10:00:11] as much as possible [10:01:44] I think it is an interesting design for a system to create a storage service that does very little [10:02:09] except "own" the devices [10:02:22] called spdkd, which is essentially your stub [10:02:49] and it has JSON rpc config interfaces [10:02:58] and the nvme hotplug poller [10:02:59] maybe the bdev layer is bakend in there [10:03:03] baked [10:03:23] so you get logical volume management [10:03:29] as a separate service basically [10:03:47] and then NVMe-oF and iscsi can dynamically attach/detach [10:03:53] as secondary processes [10:03:59] and vhost [10:04:02] yes - exactly [10:04:07] I think that's a compelling design [10:04:30] has anyone done that for NICs with DPDK? [10:04:36] that's the approach i've wanted to take long term, rather than having to decide at link time which upper level protocols you want to support [10:04:37] I would think the same thing would make a lot of sense [10:05:46] brb rebooting [10:07:11] what we're really doing is using vfio to transform linux into a microkernel [10:07:49] where our IPC is JSON RPC for control messages plus shared memory for data [10:08:17] all protected by the IOMMU [10:08:33] i'd like to rethink our strategy on how to handle things like processes that segfault or are sigkill'd [10:09:11] I think if you have the central storage daemon, you handle that one as a special case [10:09:17] i'm thinking that trying to somehow recover from those conditions (i.e. some secondary process dies before freeing its nvme queues) if a slippery slope towards impossibility [10:09:18] but then all of the others are pretty standard [10:12:45] but if a process crashes, we shut the whole thing down? [10:13:14] is there really an alternative? [10:13:27] for the primary process, I don't think so [10:13:43] for the secondary, maybe there is some way we can detect that [10:13:44] imho it's the equivalent of a kernel panic [10:14:01] definitely for the primary I agree [10:14:26] there may be nothing we can do about the secondary either [10:14:38] we try to detect this for example in the nvme multi-process code [10:15:20] but it adds a ton of complexity - and my argument would be if once process segfaulted who knows how it may have hosed other secondary processes [10:15:52] if the process of restarting everything is fairly streamlined, then just starting over isn't a bad idea [10:15:54] i think it really breaks down as we move to the model you and i are discussing here [10:16:00] exactly [10:16:12] it's a lot faster than a reboot, which is the solution to a kernel panic [10:18:02] now what do we do about lack of process shared mutex on FreeBSD? [10:18:07] or did you say they added some in 11.0 [10:19:02] I guess the "real" deliverables of SPDK are the libraries themselves, and people can string those together however they need to [10:19:20] and we just write all of our example apps using this central storage service [10:20:18] jimharris: do you want to merge Darek's patches now? [10:20:35] I had to fix that == in my thread sanitizer patch, so my stuff has to all be re-run [10:20:44] but I'll wait if we're going to merge some stuff [10:21:29] i think you should push your updates - i was thinking i'd wait to commit darek's until cob today [10:22:02] should I wait for your iscsi series to merge? [10:22:07] or are we holding that for review [10:22:49] well if we get my series in (i think it is ready) and you rebase off of it, then your patches will run through quicker [10:22:58] I like that [10:23:00] let me review [11:39:20] fedora-03 is acting up again [11:44:07] bwalker: i'm just going to mark your tsan patch +2 - what's there is functional and we can change the operator to = in a later patch [11:44:39] ok, if it will still rebase without merging that will save some time [11:45:01] seth is pulling the NIC out of fedora-03 [11:45:09] I don't think any of the tests it is running actually need it [11:45:24] we do nvmf, iscsi, and vhost all on fedora-07 [12:47:59] bwalker: i'm not understanding this spdk_for_each_channel patch [12:48:17] let me know when you're in the lab and i'll come down with my questions [12:50:27] *** Joins: gila (~gila@5ED4FE92.cm-7-5d.dynamic.ziggo.nl) [12:55:50] I'm in the lab now [13:05:34] *** Joins: pvonstamwitz (c0f00e01@gateway/web/freenode/ip.192.240.14.1) [13:13:08] *** Parts: pvonstamwitz (c0f00e01@gateway/web/freenode/ip.192.240.14.1) () [14:23:25] *** Joins: pvonstamwitz (c0f00e01@gateway/web/freenode/ip.192.240.14.1) [14:23:47] *** Parts: pvonstamwitz (c0f00e01@gateway/web/freenode/ip.192.240.14.1) () [14:37:25] jimharris: patch just failed with some crazy vfio issue when running as a secondary process [14:37:27] http://spdk.intel.com/public/spdk/builds/review/18624ea5fca7013e340b0b276b3f3c25fe6c8731.1495747667/ [14:37:42] not sure if it is my changes or yours [14:38:06] although my patch did pass before rebasing [14:39:31] no - i'm sure this is something related to my patches [14:39:48] so i saw this same tailq issue on my system - I assumed I had fixed it with the spdk0_config file checking [14:40:01] maybe that's not an acceptable method [14:48:58] ok - i'll put something together a bit more sophisticated for this [15:30:26] i pushed a change out to explicitly wait until the stub app create a sentinel file noting it's done with dpdk and nvme intiailization [15:35:49] i also found an easy 5-10 second savings in the iscsi rpc tests [15:45:59] *** Quits: gila (~gila@5ED4FE92.cm-7-5d.dynamic.ziggo.nl) (Quit: My Mac Pro has gone to sleep. ZZZzzz…) [15:59:23] *** Joins: whitepa (~whitepa@2601:601:1200:f23b:28cf:18e0:5a0b:6d6f) [16:11:46] *** Joins: ziyeyang_ (~ziyeyang@134.134.137.71) [16:46:04] *** Joins: pvonstamwitz (c0f00e01@gateway/web/freenode/ip.192.240.14.1) [16:57:36] *** Joins: changpeng (~changpeng@192.55.54.42) [17:12:00] *** Quits: ziyeyang_ (~ziyeyang@134.134.137.71) (Ping timeout: 260 seconds) [17:29:19] *** Quits: pvonstamwitz (c0f00e01@gateway/web/freenode/ip.192.240.14.1) () [17:33:56] *** Joins: ziyeyang_ (ziyeyang@nat/intel/x-sbchgkqoblczrdci) [17:34:20] *** Quits: ziyeyang_ (ziyeyang@nat/intel/x-sbchgkqoblczrdci) (Client Quit) [21:06:56] *** Joins: ziyeyang_ (~ziyeyang@134.134.139.73)