14:14 <cpringle> Hi Folks. I'm upgrading U-boot on an IMX6Q from 2016.07 to 2021.07. The port is complete and working, but after booting Linux, I've noticed that memcpy speeds are about 30% longer with 2021.07. I've checked that the DCDs are the same across both, that the write cache policy is unchanged and that there are no platform-specific clock settings/patches that have been changed. Anyone got any thoughts on

14:14 <cpringle> what else would be worth looking into to explain the memcpy performance drop? [FYI, my kernel/OS/kernel-cmdline setup is exactly the same in both runs of the test, only the U-boot version is changed so pretty sure it's something to do with platform setup that is wrong). Also checked config settings between both to make sure things like errata workarounds are identical

15:10 <milkylainen> cpringle: Memory controller settings? I mean. The DCD settings might or might not be used / overwritten etc. Just a guess.

15:10 <milkylainen> 30% sounds a bit steep though.

15:14 <cpringle> I checked that the DCD settings were present (and identical) in the header of the .imx so I think they should be being applied by the IMX at boot. They could be being overwritten by something in U-boot I guess. I'm wondering if there is a misconfigured DDR freq or something related to the frequency scaling which is different between those versions of u-boot and needs explicit

15:14 <cpringle> configuration/patching.

15:35 <cpringle> U-boot "clocks" command shows the same frequencies for all clocks - so looks like they are good.

16:29 <cambrian_invader> cpringle: are you using DMA for memcpy?

16:29 <cambrian_invader> iirc on imx6 the dma has its own firmware; perhaps it needs updating/downgrading

16:30 <cambrian_invader> if you're not using SPL then the memory controller settings aren't touched by U-Boot

16:36 <cpringle> It should be using sdma for memcpy. As the kernel/OS/cmdline is identical, wouldn't that be the same regardless of what version of u-boot I'm using? Or is there some SDMA setup in U-boot?

16:36 <cpringle> Tried forward porting both clock.c and cpu.c from the original U-boot and made no difference so far.

16:36 <cpringle> Not using SPL

16:47 <cpringle> Had a grep through both codebases and can't see anything obvious to do with SDMA setup on IMX6.

17:05 <cpringle> Anyone got any other thoughts on what could be causing the memcpy slowdown? If put aside SDMA and DDR for a moment, what else is U-boot responsible for that could cause a performance regression for memcpy?

19:20 <Tartarus> cpringle: is it between two vendor versions of U-Boot or ? It sounds like something to git bisect if you have a reproducer

19:45 <cpringle> Technically yes, as its comparing the Freescale/NXP branch with mainline - but I've already been through the differences and there's nothing in there that should cause what I'm seeing. Binary chopping is the next task I think - I deliberately avoided that as a first step as it's not quick to do and will require me to slim down our BSP quite considerably so that I can easily port it between U-boot

19:45 <cpringle> versions (latest BSP uses the driver model, whereas the older one does not).

19:48 <Tartarus> And it's memcpy outside of U-Boot, right?

19:48 <Tartarus> Like one thing that can be it is that hey, asm memcpy/etc in U-Boot is a CONFIG option, but you probably wouldn't have overlooked that

19:56 <cpringle> I think the asm memcpy in U-boot only impacts U-boot. Its enabled in both though.

20:19 <cpringle> And to answer your earlier question - yes it's memcpy in Linux. Not tried it in U-boot - that's maybe a good thing to try as would simplify what I need to get working when binary chopping u-boot versions

20:53 <marex> cpringle: PL310 L2 cache controller is enabled ?

20:54 <marex> cpringle: full line of zeroes optimization bit is toggled ?

21:16 <EdSwarthout> Has anyone used the Sandbox network? Ping to any IP says "is alive" but tftpboot times out

21:17 <cpringle> PL310 L2 cache controller is enabled (part of mx6_common.h, which I also tried forward-porting to reduce changeset between the U-boot versions).

21:17 <cpringle> What's the easiest way to check the full line of zeros optimization bit?

21:27 <marex> cpringle: hold on

21:28 <marex> /* enable BRESP, instruction and data prefetch, full line of zeroes */

21:28 <marex> L310_AUX_CTRL_DATA_PREFETCH_MASK |

21:28 <marex> L310_AUX_CTRL_INST_PREFETCH_MASK |

21:28 <marex> setbits_le32(&pl310->pl310_aux_ctrl,

21:28 <marex> L310_SHARED_ATT_OVERRIDE_ENABLE);

21:29 <marex> cpringle: the imx L2 cache init does more stuff in arch/arm/mach-imx/cache.c v7_outer_cache_enable() , so make sure it is called

21:29 <marex> is it ?

21:41 <marex> cpringle: oh look, Linux even has arch/arm/mm/cache-l2x0.c l2c310_enable() 623 if (cortex_a9 && !l2x0_flz_disable) {

21:41 <marex> flz -- full line of zeroes

21:42 <marex> cpringle: so actually, run dmesg | grep L2C.310 and see if there is anything interesting

21:42 <marex> also check if you have CONFIG_CACHE_L2X0 enabled in Linux

21:42 <cpringle> Thanks - just checking u-boot now. Definitely some differences in that function between the 2 u-boot versions. Will check Linux shortly to

21:43 <cpringle> *too

21:43 <marex> cpringle: the kernel boot log might give you some hint too

21:43 <marex> (and the PL310 regs too, you can read them using => md 0xregister , they are memory mapped )

21:48 <cpringle> v7_outer_cache_enable is definitely being called (trace statements printed). Checked Linux dmesg and a few L2C messages (nothing ominous and identical messages regardless of u-boot version used). Good shout on reading the register addresses - will do that to see if I can see any differences

21:49 <marex> cpringle: did you try and find out what it is that is slow exactly, the memory read or the memory write ?

21:49 <marex> lmbench would tell you

22:05 <cpringle> No I didn't yet. I'll try with lmbench and see what that shows up

22:31 <cpringle> lmbench not behaving and need to figure out why before I can make much more progress with that. Thanks for your help - few things to look at on Monday. If you (or anyone else) has any other ideas of things to check that'd be great. Thanks again and have a great weekend