12:06 <Segfault0> tagr: do you know if the upstream emc driver for t210 works? i'm having trouble with it

12:16 <kwizart> Segfault0, IIRC, the bootloader(s) doesn't pass the trained timing in the right (upstream) format, so upstream kernel cannot use them

12:17 <Segfault0> i'm running upstream uboot but i don't think that does the memory training on the jetson?

12:17 <kwizart> no, that's earlier stage bootloader, uboot only passes the tables to the kernel as I understand

12:19 <kwizart> I think using the downstream kernel with the upstream u-boot works to pass the table, (I don't know if there would be other missing features)

12:19 <kwizart> To be confirmed by nvidia devs anyway ...

12:19 <Segfault0> is there any way to get that data into an upstream kernel then? or is that still work in progress?

12:27 <kwizart> that's a good question.

12:27 <kwizart> is it better to have a uboot tool that could "translate" the table in the appropriate format for upstream (and what to do with downstream case) ?

12:27 <Segfault0> because at the default memory speed this soc isn't really usable :P

12:28 <Segfault0> does downstream even create training data? i thought it just put it in the dt

12:28 <Segfault0> same as the older socs

12:28 <kwizart> ddr4 seems to rely on memory training (then re-training) for higher frequencies

12:29 <kwizart> re-training would be done by the emc linux driver...

12:30 <kwizart> the other solution would be to import the downstream emc driver into upstream (I don't know how difficult would be this task)

12:32 <kwizart> but in the jetson-tx1 case (I don't know if it would behave the same with others board), I think the bootloader leave with reasonably high frequency, but maybe I'm mixing with the tk1 case...

12:33 <Segfault0> it's definitely not leaving the memory at high speed here, i'm pretty sure that's the k1

12:33 <Segfault0> i know the nexus 9 leaves the memory at maximum speed too

12:34 <kwizart> some basic benchmark I've made shows that even upstream perform better than downstream in some memory consumption, but then I need to double check...

12:34 <Segfault0> also i just checked and yeah the downstream dtb's in the l4t package, at least for the nano, contain an emc table very similar to the older socs

12:35 <Segfault0> i ran some benchmarks on my nano on upstream and the memory is hugely underperforming so i think the memory is being left at 400mhz or something

12:36 <kwizart> Segfault0, which is the higher frequency from downstream table ?

12:36 <Segfault0> ah good point, it only seems to go up to 204mhz

12:38 <kwizart> Segfault0, btw,I think I have your nexus9 ready to be tested at some point... I wish I would only need to update the dts and to rely on my (fedora) distro kernel, but tegrapt (at least) will be needed for the long run

12:39 <tagr> so the EMC story is a little troublesome...

12:39 <Segfault0> the n9 still has some major issues, biggest of which is that it doesn't even boot if you don't go through hboot first but there are also some other things going on

12:41 <tagr> back when I was working on upstreaming the EMC driver support, DT maintainers NAK'ed the existing DT bindings that downstream used, so we added a different mechanism (using the EMC table that upstream now supports), unfortunately by the time all of that was merged upstream, Tegra210 was software EOL'ed downstream, so no new releases are happening and the changes required in the early bootloaders to support the upstream DT bindings were never released

12:43 <Segfault0> ah

12:44 <Segfault0> so where do things go from here? seems like it's stuck between a rock and a hard place

12:44 <tagr> and yes, the tables that downstream DTS files have are for the lowest frequency, anything beyond needs training, which is done by an early bootloader (nvtboot), which is responsible for creating EMC table entries at higher frequencies

12:44 <tagr> the EMC driver applies "tweaks" on top of those trained values (so it doesn't fully retrain, because the full training algorithm is apparently not available under an open source license)

12:45 <tagr> Segfault0: yeah, pretty much

12:46 <tagr> I had at one point argued that perhaps we could do one more release because EMC scaling support is one of the last missing pieces on Tegra210, but again, software EOL makes that kind of thing pretty difficult

12:46 <kwizart> I've tried to escalate the issue on the forum at https://forums.developer.nvidia.com/t/efi-support-for-jetson-tx1-and-running-upstream-kernel/187945

12:46 <Segfault0> i guess the only real option would be adding support for the existing method to the upstream driver?

12:47 <kwizart> if a new release limited to upstream usage would be done, that would be very nice

12:47 <tagr> Jetson TX1 users are lucky in that it boots at the highest frequency by default, Jetson Nano doesn't, so you're stuck on a lower frequency there

12:47 <tagr> adding support for the existing method was already NAK'ed by the upstream DT maintainers

12:47 <tagr> admittedly for good reasons

12:48 <Segfault0> :/

12:48 <tagr> yeah, I think if we could've somehow released the nvtboot binary that has support for the upstream DT bindings that would've been nice

12:49 <tagr> however, the way that L4T releases work is that a significant amount of work needs to be done in order to QA a release

12:51 <tagr> kwizart: yeah, I had hoped that perhaps we could make a release for just nvtboot, disconnected from everything else, but it seems like that's not going to happen

12:54 <tagr> correction: I think Jetson TX1 doesn't run at the highest frequency it can, it runs at the highest frequency that doesn't need retraining

12:54 <tagr> not exactly sure what the numbers are, might have been 400 MHz for Jetson Nano and 800 MHz for Jetson TX1

12:55 <Segfault0> 200-400mhz seems about right for the nano, i was getting ~2GB/s of memory bw on a cpu based benchmark

12:59 <kwizart> I wonder if upstream could receive any static table for frequencies up to 800mhz for all SOCs ?

13:05 <Segfault0> i guess emc probably won't be working on t210 upstream any time soon then, that's a shame

13:08 <Segfault0> tagr: out of curiosity what do you think would be the easiest way to get it working in a local tree? port over the emc driver from the bsp kernel?

13:10 <tagr> Segfault0: one thing I had thought about in the past was to implement a conversion in U-Boot

13:10 <Segfault0> ah yeah

13:11 <tagr> the problem is that we can't create static table entries because apparently even on any given board you could have slight differences in some of the settings depending on environmental conditions and so on

13:12 <tagr> so the easiest would've probably been to parse the DTB that's passed to U-Boot (which should have all the data from nvtboot still) and then create a binary dump and add that statically into the DTS files that we use upstream

13:12 <tagr> but again, since there can be differences depending on environmental conditions, or just slightly different memory chips used depending on what exact board you have, that becomes impractical

13:13 <Segfault0> oh so does the bootloader add the higher speed options to the dtb along with the static low speed ones?

13:13 <tagr> but if you already have code in U-Boot to parse the old-style DTB, you could simply transcribe that into the binary table that upstream Linux expects, put it somewhere in system memory and write the corresponding DTB entries (i.e. the reserved-memory node and the phandle in the EMC controller node)

13:14 <tagr> IIRC the bootloader will use the low speed ones as sort of a template and then train the memory and use the replace the correct values

13:15 <tagr> or something like that

13:15 <Segfault0> ah

13:15 <tagr> in the end, the DTB that ends up in U-Boot should contain the same tables that the "new" nvtboot would've written into the new-style EMC table

13:16 <tagr> actually, you may have to look in more detail what DTB you use exactly in U-Boot, because U-Boot gets its own "control" DTB as well

13:17 <Segfault0> i'm just running upstream uboot with an extlinux.conf that sets the dtb to the one that comes with the upstream kernel

13:17 <Segfault0> i don't know what uboot itself is using

13:17 <tagr> U-Boot has a built-in DTB that's generated from a DTS that's part of the U-Boot source tree

13:18 <tagr> if you look at the code in arch/arm/mach-tegra/cboot.c it contains various functions that access the "cboot_blob", which is a pointer to the DTB passed by nvtboot (via cboot)

13:19 <tagr> see for example cboot_dram_init()

13:20 <tagr> so if you pass that cboot_blob (or, more specifically, the value in cboot_boot_x0) as first parameter to the FDT functions, it should be accessing the DTB that has the pre-trained EMC tables (in the old format)

13:20 <tagr> that layer is not as pretty as I would've wanted, but as discussed above, probably the best we can do at this point

13:20 <Segfault0> yeah

13:21 <tagr> we might still get the nvtboot released if there was a significant customer request behind it, but I don't see that at this point anymore as most big customers seem to have moved to newer chips

13:22 <Segfault0> yeah that doesn't seem likely

13:23 <tagr> it's very annoying because we came really close =(

13:24 <Segfault0> yeah the driver is there and should work, it just doesn't have the data it needs

13:24 <Segfault0> i'm a little surprised it even exists upstream if nothing can use it lol, although that said all of the t132 stuff still exists so i'll stay quiet and hope it doesn't get deleted :P

13:25 <tagr> no, that's the good thing about upstream, we don't usually remove support

13:26 <Segfault0> btw i can definitely tell the t132 stuff might as well have not been tested, there were some pretty major and simple oversights lol

13:26 <tagr> so technically the Linux kernel will use the EMC table, it's just that most people don't have access to a bootloader that will generate the data in the right format

13:26 <Segfault0> yeah

13:26 <tagr> I do, so I can run the Jetson Nano at high speed =P

13:27 <Segfault0> you have access to a bootloader like that? it's not publicly available is it? :P

13:27 <tagr> well, I wrote the code to make it generate the EMC table in the new format, and I might be the only person on this planet to have ever run that code, but yeah, not allowed to make it public

13:28 <Segfault0> darn

13:28 <tagr> and given that I likely am the only person to ever have run it, it'd be pretty obvious where it came from if I were to "accidentally" make it available somewhere

13:28 <Segfault0> heh

13:29 <Segfault0> i'm surprised just converting the data over would be problematic, or are you using a modified cboot or something?

13:30 <tagr> I don't understand

13:31 <Segfault0> you said you wrote code to generate an emc table in the new format, what is it that actually stops you from releasing that code? is it a part of some proprietary bootloader bit?

13:32 <tagr> yeah, that's part of nvtboot, which we don't provide source code for

13:32 <Segfault0> right

13:32 <Segfault0> hence you suggesting doing it in uboot instead and just using the data nvtboot already created

13:32 <tagr> right

13:33 <tagr> so technically nvtboot internally uses the binary table that the new-style bindings use, so the old-style tables are really already of a conversion

13:34 <tagr> so doing a conversion back in U-Boot never seemed right because it's effectively a double conversion, hence why doing this right in nvtboot is far superior

13:34 <Segfault0> it wouldn't happen to leave that data anywhere once it finishes running would it? :P

13:35 <tagr> I don't recall exactly, but I think it's some memory that's not necessarily at a fixed address

13:35 <tagr> in fact the patches I wrote were primarily just making sure the table was at a fixed address (and reasonably placed, so it wouldn't fragment memory unnecessarily) and create a reserved-memory node to point at that location

13:39 <Segfault0> damn, i wonder what could cause the address to change

13:40 <Segfault0> maybe if we're lucky it's static per device or something

13:42 <tagr> it might also just be in a memory region that ends up getting overwritten by subsequent bootloaders

13:42 <tagr> so even if the address remains fixed the data might be gone by the time you get to U-Boot

13:42 <Segfault0> yeah possibly

13:43 <Segfault0> my first thought would be that it just does it on the stack or something but i'm not sure if you commenting on that would be allowed lol

13:49 <Segfault0> well uh seeing as i'm here, tagr do you have any idea why a nexus 9 would hang if i try to enable interrupts on the pmic or why usb device mode wouldn't work but host mode does?

13:51 <tagr> do you have the nvidia,invert-interrupt property set in the pmc@7000e400 node?

13:51 <Segfault0> i don't, it didn't seem to be set for this board in the bsp kernel but i was meaning to try it anyway

13:51 <tagr> yeah, that's usually the culprit

13:52 <tagr> it's not technically a hang, it's just that the interrupt keeps firing if the polarity is wrong and then the system appears to be hanging

13:52 <Segfault0> yeah i had a feeling it was something along those lines

13:52 <tagr> no idea about USB device vs. host mode, though, not something we've ever tried upstream

13:53 <tagr> I think earlier iterations (like the Jetson TK1) had issues with OTG, so you couldn't make device mode work at all, or something

13:53 <Segfault0> interesting

13:53 <tagr> Nexus 9 would obviously be somewhat different, but who knows what the heck vendors did for USB on those devices

13:53 <Segfault0> i have a tk1 coming in the mail which i was hoping to use to debug exactly that issue

13:54 <tagr> I /think/ you may be able to get USB device mode working on Jetson TK1 by manually forcing the mode (I vaguely recall there being a sysfs know for that), but, again, not something we've tried exhaustively

13:55 <Segfault0> currently my only theory is that it's using the pmic extcon to know when to enable usb device mode and that it might be borked because i haven't got pmic interrupts going but it's a really far fetched theory

13:56 <tagr> actually, according to commit 96f1abf03e78df971b901aa8beba8c77e23a2617, USB device mode is something that we tested on Jetson TK1, kwizart seems to have verified that it works

13:56 <Segfault0> ah good

13:57 <tagr> again, it's not OTG, so only supports peripheral mode

13:57 <Segfault0> er, is this irc chat logged somewhere? i'd really like to have this conversation for future reference lol

13:57 <tagr> Nexus might be completely different

13:57 <kwizart> yep, I vaguely remember this test, I still have a tk1 to test again if needed (or even a nexus9)

13:57 <tagr> I suspect that some other hardware mechanism would exist on the Nexus, and it would have to be OTG as well, otherwise there'd be little purpose to it

13:58 <Segfault0> the n9 has a gpio for otg detection and power output is handled by the battery charger

13:58 <Segfault0> nice and simple

13:59 <tagr> ah... so is it detecting host/peripheral mode, then?

13:59 <Segfault0> yep

13:59 <Segfault0> it automatically switches modes and all, it just never shows up as a device when i plug it into another computer and configure the udc thing

14:00 <Segfault0> which is especially odd because doing the exact same thing on a nexus 7 works perfectly

14:00 <Segfault0> hence me buying a tk1 to test with :P

14:01 <tagr> I suppose it could have something to do with the XUSB padctl setup as well, since that's responsible for the PHY part

14:02 <tagr> if that's wrongly configured, I imagine no signal would be received at the host, or at least not a correct signal

14:02 <Segfault0> yeah the padctl bit is kind of confusing, my assumption at the moment is that if i just set the usb0 pad to the snps function it should let me just directly use the ehci controller and ignore xusb

14:03 <tagr> XUSB is host-only anyway, so if you want device mode you need the UDC driver

14:03 <Segfault0> yeah that's why i was trying to just ignore it and only use udc for the time being

14:04 <Segfault0> i think xusb's host controller might be faster (it is separate to udc right?) so in future i'd like to look into it but for now i just want things to work lol

14:04 <tagr> okay, but make sure to use nvidia,tegra124-udc and not nvidia,tegra124-ehci

14:04 <Segfault0> yep i am

14:06 <tagr> not sure if XUSB is faster, but it certainly support USB3 (at least on newer generations, not sure about Tegra124/Tegra132, but probably there as well)

14:06 <tagr> EHCI is 2.0 (and lower) only

14:06 <Segfault0> yep t124/132 do support usb 3 but this device doesn't expose it at all, just a single micro b 2.0 port

14:07 <tagr> so I guess it is ultimately faster, if you do attach SS devices

14:07 <Segfault0> but the udc controller only managed ~200mbit/s in my testing which is quite slow even for usb 2.0

14:07 <tagr> yeah, if it only has micro B, no need to use XUSB

14:07 <tagr> doesn't sound all that bad, honestly

14:08 <Segfault0> i mean it's fine enough for the time being

14:09 <tagr> yeah, you should be able to get a bit more out of it, but for starters that sounds great

14:10 <Segfault0> i also need to figure out why the device hangs during boot if i don't go through hboot first but i get the feeling that one's going to be really difficult to debug and not really much of a tegra thing

14:10 <Segfault0> my first thought is something clock or pmic related