ChanServ changed the topic of #armlinux to: ARM kernel talk [Upstream kernel, find your vendor forums for questions about their kernels] | https://libera.irclog.whitequark.org/armlinux
hanetzer has quit [Ping timeout: 268 seconds]
hanetzer has joined #armlinux
hanetzer has quit [Ping timeout: 268 seconds]
hanetzer has joined #armlinux
hanetzer has quit [Ping timeout: 268 seconds]
XV8 has joined #armlinux
dliviu has quit [Remote host closed the connection]
elastic_dog has quit [Killed (calcium.libera.chat (Nickname regained by services))]
heat has quit [Remote host closed the connection]
heat has joined #armlinux
amitk has joined #armlinux
Tokamak_ has joined #armlinux
Tokamak has quit [Ping timeout: 260 seconds]
amitk_ has joined #armlinux
heat has quit [Ping timeout: 256 seconds]
iivanov has joined #armlinux
guillaume_g has joined #armlinux
jeeeun has joined #armlinux
monstr has joined #armlinux
apritzel has joined #armlinux
unixsmurf has quit [*.net *.split]
unixsmurf has joined #armlinux
rvalue has quit [Read error: Connection reset by peer]
rvalue has joined #armlinux
iokill has quit [*.net *.split]
georgem has quit [*.net *.split]
narmstrong has quit [*.net *.split]
iokill has joined #armlinux
georgem has joined #armlinux
narmstrong has joined #armlinux
apritzel has quit [Ping timeout: 260 seconds]
cleger has joined #armlinux
cbeznea has joined #armlinux
<milkylainen>
Can one override a pcie device dma bitmask from userspace or from the dts? Either overriding the default for the device or forcibly ignoring what the device driver thinks?
sszy has joined #armlinux
<bjdooks>
you might be able to control dma-ranges from dts, otherwise you'd probably be better off trying ot change the driver
<arnd>
milkylainen: you cannot override it to make it larger, the effective mask is always the intersection of what the dts and the driver support. The dts should describe limitations of the bus interconnect and the driver should tell the kernel what it thinks the device supports
<arnd>
if either the device or the bus is limited to 32-bit DMA, there is no way to tell the kernel to use a larger mask
<arnd>
actually the driver could use dma_coerce_mask_and_coherent() to set a larger mask than what the dts supports, but that generally leads to data corruption, so don't do that
headless has joined #armlinux
<lag>
arnd: It looks as though KASAN adds some Bytes (20-60) to function's stack-frames when enabled which blows the -Wframe-larger-than= limit on quite a few functions, seemingly unfairly
<lag>
arnd: Can you think of a sane way to bump the limit when KASAN is enabled?
<milkylainen>
hmm.
<lag>
arnd: Presently all I can come up with is an additional CONFIG value that gets added to CONFIG_FRAME_WARN when assigning -Wframe-larger-than=
<arnd>
lag: are you looking at 32-bit or 64-bit kernels right now?
<lag>
arnd: Right now, I'm working with Arm 32-bit
<arnd>
I think we can change the default for FRAME_WARN to be a bit higher when KASAN is enabled on 32-bit. On 64-bit I'd rather make the default lower for everyone first
<milkylainen>
arnd: I think someone here concluded that all pcie devices are >= 32-bit dma capable? iirc. I was looking for a way to open up the default mask when toying with uio or similar. Not my preferred way of doing things though. This is a project asking.
<lag>
arnd: I'm yet to see if this is exacerbated on 64-bit platforms
<lag>
arnd: Right, but the value needs adding on - since FRAME_WARN can be different values depending on system / compiler, etc
<milkylainen>
sure. The final mask is going to be the intersection of other things too. But I know the underlying hw is capable of a larger mask.
<lag>
arnd: Unless we just don't care about the frame-size when KASAN is enabled and we just allocated 2048 right out the gate !!
<arnd>
milkylainen: there are a couple of SoCs that have a smaller than 32 bit DMA capable PCI bus, so you can't just ignore what's in the bus and assume 32-bit is fine in general, though it almost always works. There are also a lot of systems that don't declare a dma-ranges property for their PCI bus in dts, so they end up being limited to 32-bit even though the hardware supports a wider access in principle. Those should be fixed by
<arnd>
correctly
<arnd>
listing the HW capabilities in DT
<arnd>
lag: I really want to ensure that an allmodconfig kernel has a small limit and warns about regressions when someone puts a large variable on the stack, so turning off the warning or making the limit much larger is not an option
<milkylainen>
arnd: But even if you declare a larger dma-range for the pci bus, the device starts of life as a 32-bit capable device, unless somebody else says so?
<lag>
arnd: Agree
<milkylainen>
32-bit dma mask capable even
<milkylainen>
"somebody" beeing the driver.
<arnd>
milkylainen: to get a larger than 32-bit DMA mask, you need both the dts to say that what the bus supports, and the driver to say what the device (and driver) supports
<milkylainen>
exactly.
<milkylainen>
And I was looking for a way to do what the driver usually does, when toying with uio (no driver).
<arnd>
If a driver knows that the particular version of the device supports 64-bit DMA addresses and the driver correctly passes them to the hardware (rather than truncating a dma_addr_t to 32 bit while writing a register), then the driver should call dma_set_mask_and_coherent()
<arnd>
milkylainen: I think uio is normally limited to non-DMA devices, if you need DMA on PCI devices you should use VFIO instead
<geertu>
arnd: That's an integration feature, which may not depend on the particular version of the DMA device?
<milkylainen>
I saw ther recent rebar from sysfs discussion. The driver has been capable to do thay earlier, but somebody wanted a sysfs knob for it. I was thinking the same here. :)
<milkylainen>
arnd: Mmm. This project uses some hacked up uio with dma support iirc.
<arnd>
lag: I would just put it into the set of 'default' lines in Kconfig
<lag>
arnd: Can you to arithmetic on default lines?
<milkylainen>
But non device specific. I guess one could send the "device dma mask" in question as a module parameter or something.
<arnd>
milkylainen: the problem with that is that this immediately compromises the kernel memory from user space that has access to the device, so I think such a hacked up uio driver won't make it into mainline kernels. OTOH if you already rely on a hacked up driver, just hack it up some more to do what you want
<milkylainen>
arnd: yeah. you're right. This entire thing is not perfectly sane. :\ But it is what it is. I'm not likely to sway their opinion or their current construct.
<arnd>
milkylainen: VFIO avoids this by using the IOMMU to give the PCI device access to your user address space, which is generally safe here
<arnd>
to keep the allmodconfig limit the same and just not use KASAN
<arnd>
we already do this for KASAN_STACK, which has an extreme effect on stack usage and is just broken with clang
<lag>
arnd: I like this idea better - although it's more likely to tread on more people's toes
<arnd>
it's a bit unfortunate as that would mean not being able to regression test KASAN itself
<lag>
Right, which seems a bit harsh
<milkylainen>
arnd: mm. I could try to talk them into trying VFIO.
<lag>
arnd: The first option certainly seems less intrusive
<milkylainen>
Not likely to succeed though.
<milkylainen>
:\
<geertu>
milkylainen: And VFIO also works without PCI
<geertu>
milkylainen: For VFIO, you also need a reset method for your device. I posted a patch to use "resets" from DT, the the virt people didn't like it, as they don't trust generic DT resets.
<arnd>
lag: have you tried narrowing the problem down to specific sub-features in KASAN? I managed to single out KASAN_STACK as the main culprit for this years ago, and we completely dropped -fsanitize-address-use-after-scope for similar reasons. It's possible that KASAN gained yet another feature that is generally useful for userland but just doesn't work for the kernel, and turning that one off makes the stack usage for KASAN-enabled \
<arnd>
kernels shrink enough again
<lag>
arnd: No, I've only tried pulling the big KASAN leaver
<arnd>
milkylainen: my impression is that using a hacked-up uio.ko driver is the same in the end as having a custom driver for the underlying hardware that just happens to look a lot like the upstream uio driver.
<milkylainen>
geertu: This is userspace. So trying to avoid kernel constructs as per the project req. Can't I push the reset from pci config writes?
<arnd>
if all you want to solve is the dma mask, then the easiest way would be to formalize this by renaming the custom driver, making it bind to the PCI ID you are interested in and setting calling dma_set_mask() in the init function
<lag>
arnd: KASAN_STACK is not enabled seemingly
<arnd>
lag: those Kconfig options are not what I meant, I was thinking of the compiler flags in scripts/Makefile.kasan
<geertu>
milkylainen: IIRC, VFIO forbids exporting devices to userspace if there is no reset method. But of course you can hack the core vfio driver to ignore that ;-)
<milkylainen>
hmm.
<milkylainen>
Is this a new addition ie, reset requirement?
<arnd>
Kasan is a generic framework that has a ton of more specific flags to enable or disable certain features. Some are enabled from the Makefile, some are enabled by default. It's possible that the defaults in clang-14 include something that we actually want disabled
<milkylainen>
Oh well. I guess it is solveable anyway.
<arnd>
milkylainen: if you have PCI device, don't worry about this, the pci vfio has a generic reset function that works on all of them
<lag>
arnd: I'm using Clang
<lag>
arnd: I hear there is a new flag which would be useful on it's way, but it won't be released for quite some time
<milkylainen>
arnd: ah.
<lag>
arnd: Let me see if I can remember it
<arnd>
milkylainen: what geertu is referring to is vfio support for platform devices, which cannot have a generic reset function, so you need a minimal stub driver for each vfio-platform device
<arnd>
lag, ok so between clang-12 and clang-14 the difference is only a few bytes. IIRC clang-11 is now the oldest supported release, so probably not much point looking at that.
<arnd>
trying clang-15 or clang-16 may help to see if that has any fixes in it, or regresses elsewhere, but I suppose it's most important to fix the ones here first
<lag>
arnd: Right, this is not a new issue AFAIK
<lag>
arnd: Let me see if TuxSuite supports that
<lag>
arnd: Not sure it's a bug - my assumption is the extra few Bytes are used for tracking?
<arnd>
right, I've been battling KASAN stack usage for many years, mainly needed to see if there were any fundamentally new issues here, or just the ones where my workarounds never made it upstream
<arnd>
lag: it's a mix. There are rare cases where a function legimately uses close to 1024 bytes of stack, and adding a few bytes pushes it over the limit
viorel_suman has joined #armlinux
<arnd>
the more common case however is an actual compiler misoptimization where a function is meant to take < 100 bytes of stack, and adding KASAN breaks the register allocator to the point where it takes several times more
<arnd>
KASAN disables a number of optimization passes in order to work as intended, and it makes life harder for the register allocator because it needs extra registers to check the memory references
wens has quit [Read error: Connection reset by peer]
<arnd>
another common case is code that is already badly optimized by one or both of the compilers, and KASAN pushes it over the limit, but the real fix is to rewrite the kernel code to be easier to optimize,or to fix one of the optimizer passes in the compiler to better work on that particular input
shoragan has quit [Read error: Connection reset by peer]
wens has joined #armlinux
<lag>
arnd: So in the Crypto case we looked at - do you think that KASAN is the cause of that optimisation?
<lag>
arnd: If so, is it possible to disable KASAN for individual drivers?
<lag>
s/optimisation/misoptimisation/
lag has quit [Changing host]
lag has joined #armlinux
shoragan has joined #armlinux
sakman has quit [Remote host closed the connection]
sakman has joined #armlinux
amitk_ has quit [Ping timeout: 268 seconds]
mraynal has quit [Remote host closed the connection]
mraynal has joined #armlinux
Amit_T has joined #armlinux
hanetzer has joined #armlinux
headless has quit [Quit: Konversation terminated!]
amitk_ has joined #armlinux
matthias_bgg has quit [Ping timeout: 260 seconds]
cleger has quit [Quit: Leaving]
matthias_bgg has joined #armlinux
amitk_ has quit [Ping timeout: 265 seconds]
elastic_dog has joined #armlinux
heat has joined #armlinux
<arnd>
mmind00: I'm a bit confused by your v6.2-rockchip-dts32-1 tag. The PR subject says "for 6.1", but the tag name says "6.2". From the contents I would guess 6.1 is correct, but it's not clear from the message
apritzel has quit [Ping timeout: 264 seconds]
elastic_1 has joined #armlinux
elastic_1 is now known as elastic_dog
elastic_dog has quit [Killed (erbium.libera.chat (Nickname regained by services))]
apritzel has joined #armlinux
derjanni_ has joined #armlinux
derjanni has quit [Ping timeout: 268 seconds]
naoki has quit [Quit: naoki]
linusw_ has quit [Quit: Connection closed for inactivity]
monstr has quit [Remote host closed the connection]
<mmind00>
arnd: sorry about that ... the tag is correct ... it looks like I misstyped the number in the email
<mmind00>
arnd: but then again ... that one commit, also could count as fix
<mmind00>
(the global-timer disabling)
matthias_bgg has quit [Ping timeout: 264 seconds]
<mmind00>
arnd: and now I see in my inbox that it already got applied as fix ... so just to say, all good like it is