10:30 <robmur01> HdkR: not quite Altra, but my eMAG has a WX5100 and it's behaving OK with standard Ubuntu kernels (other than HDMI audio last time I tried that, but that appeared to be the HDA driver having bizarre behaviour hard-coded for !x86)

10:31 <robmur01> what hacks did LX2160 require?

10:33 <HdkR> robmur01: Requires changing some IO WC memory to straight IO which hits unaligned access faults when newer GPUs try to load their firmware

10:33 <robmur01> ah, OK, that's liable to be a general arm64 thing then

10:33 <HdkR> They also have a patch for mesa to push some buffers through GTT if it is under 1MB which is hilariously bad

10:34 <HdkR> It would be shame if I nab an Ampere and then I can't use a GPU in it

10:37 <HdkR> https://github.com/Wooty-B/LX2K_Guide/blob/main/0001-radeonsi-On-Aarch64-force-persistent-buffers-to-GTT.patch mesa hack

10:37 <HdkR> https://gitlab.freedesktop.org/drm/amd/-/issues/2313#note_1803017 kernel hack

10:38 <HdkR> If the IO coherency is still busted on Ampere then it will be hard to justify

10:51 <robmur01> coherency should be less of an issue on Altra due to how CMN-600 works

10:53 <robmur01> how any given PCIe IP handles unaligned accesses will remain more of a crapshoot

11:01 <HdkR> Hm

11:02 <HdkR> Terrified of sticking an RX 6900XT in to the thing and it not working. Wasting many many thousands of monies

11:10 <HdkR> Which I guess if Grace uses the same CMN then I can't expect things to work there either

11:10 <robmur01> that does sound like an expensive way of running llvmpipe on top of EFI GOP, even if it would have plenty of CPU grunt for that :D

11:12 <HdkR> eesh, that would be rough

11:13 <robmur01> Grace is all about its own GPU though, no?

11:13 <HdkR> Grace is the CPU side component, where Grace+Hopper is the combo thingy

11:14 <HdkR> NVIDIA showed off a Grace+Grace config which would more match what I want

11:17 <robmur01> Ah, that was it. These days I struggle to remember what all our own codenames refer to, let alone anyone else's...

11:18 <HdkR> Orin was also a platform I tried testing but rebuilding NVIDIA's kernel is a nightmare so I gave up on it

11:18 <HdkR> and that platform can't use VFIO to just stick things in to a VM with passthrough

11:20 <robmur01> Was that the one that's mainly just a dodgy DT? I debugged some NVIDIA VFIO thing on-list recently.

11:21 <HdkR> Not sure, from a user's perspective it would generate virtual PCIe bridges that wouldn't ever bind to the vfio group

11:22 <HdkR> IOMMU grouping looked like it was working correctly

11:22 <robmur01> https://lore.kernel.org/linux-iommu/e182594b-5cdb-881d-59c8-a20dc90b62fb@arm.com/

11:23 <HdkR> Ouch

11:25 <HdkR> `So if you want to try SR-IOV on an ARM-based platform, don't use an NVIDIA Orin board` Yea, same board then

11:25 <HdkR> Same behaviour I saw as well

11:26 <robmur01> although IIRC the SoC has like 10 distinct root complexes, so it kind of looks "assign entire RC to a VM" might have actually been a considered usage model :/

11:26 <robmur01> *looks like

11:31 <HdkR> It certainly is a wacky AI/Car SoC

11:48 <HdkR> Maybe I'll just wait until there is some Neoverse-V3 or V4 thing on the market instead

11:56 <robmur01> or maybe Intel starts licensing PCIe IP for the true "works like x86" experience :D

11:56 <HdkR> haha

12:09 <bjdooks> someone got a good deal on designware pcie?

12:16 <geertu> bjdooks: shimoda-san, I guess?

13:51 <ukleinek> robher: I don't understand the dtc docs for -@. It says to generate a __local_fixups__ node, but either I don't understand something or the code disagrees?!

14:08 <robher> ukleinek: Does your overlay have a reference to a phandle node defined within the overlay?

14:15 <ukleinek> robher: I don't have an overlay

14:20 <ukleinek> robher: prototype patch: https://paste.debian.net/hidden/9b91686f/

14:28 <robher> ukleinek: No idea if that's correct or not offhand. Send it to the DTC list. I thought just having /plugin/ was enough to generate both types of fixup nodes.

14:32 <ukleinek> robher: For some reason I want a __local_fixups__ in a normal dtb, but actually that is orthogonal to the documentation strangeness. Will write a mail. Thanks

15:02 <marcan> hm, I just ran into a thing and I'm surprised nobody else has yet...

15:03 <marcan> as far as I can tell, the genpd core unconditionally powers off genpds in the suspend (not RPM) path once all dependent devices have been suspended, and there is no way for the device to prevent that without aborting the whole suspend cycle

15:03 <marcan> ... which is a bit of a problem for the UART driver if it has a power domain and no_console_suspend was passed, since then it'll get powered down anyway and then everything explodes on the next kernel message.

15:04 <marcan> am I missing something?

15:06 <marcan> in general, forcing genpds not to power off from the device driver seems like a desirable thing. e.g. right now we don't support proper save/restore for PCIe in s2idle, which is fine as long as those PDs never power off. that's worked around right now at the PD level (marking it always-on in the DT), but that means once we *do* implement this and remove that, older PCIe drivers will break on newer ...

15:06 <marcan> ... DTs.

15:06 <marcan> having the driver blocking genpd poweroff on suspend directly would make a lot more sense here too

16:06 <geertu> marcan: there should be some no_console_suspend handling, lemme check

16:08 <geertu> marcan: drivers/soc/renesas/rmobile-sysc.c has rmobile_pd_suspend_console() { return console_suspend_enabled ? 0 : -EBUSY; }

16:10 <geertu> which relies on the PM Domain that contains of_stdout to be handled special

16:10 <marcan> uh. so we need to special case this in the *power domain driver*?

16:10 <marcan> that seems completely backwards, the driver that knows about the console is the serial driver

16:11 <marcan> I don't want to start adding hacks to the (trivially simple) genpd driver we have for stuff like this

16:12 <marcan> if we want to do pcie properly etc, it'll become a giant mess when everything is now nicely described in the DT

16:12 <marcan> surely there *ought* to be a way for drivers to request their genpds not be powered down on suspend

16:33 <geertu> marcan: For wake-irqs, there is indeed such a mechanism

16:33 <marcan> yeah, device_wakeup_path / genpd_is_active_wakeup

16:34 <marcan> how inappropriate would it be to abuse that for stuff that needs to stay on? :p

16:35 <geertu> drivers/tty/serial/serial_core.c has some "console_suspend_enabled && uart_console(uport)" checks

16:35 <marcan> yeah, that doesn't do anything though

16:36 <marcan> it doesn't actually block the genpd suspend

16:43 <marcan> geertu: honestly, I almost feel like that wakeup_path stuff should just be renamed to powered_in_sleep or similar. it's basically exactly what I want, just named after a more specific use case than mine.