10:35 <marex> qschulz: the R-Car TFA BL31 board code already passes that DT+DTO combo to the BL31 blob before booting kernel, does it not ?

10:41 <qschulz> marex: I guess? It's still a bit obscure to me this part of the boot. But it would make sense since you want changes made by TF-A to make it to the kernel DT... That might make it a bit trickier since we don't have much storage in DRAM for the DTB in TF-A right now (256KiB; which is kinda tight with DT symbols)

11:02 <marex> qschulz: where does the 256 kiB figure come from ?

11:15 <qschulz> marex: TF-A

11:17 <qschulz> actually 128KiB right now

11:17 <qschulz> https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/34997 for bumping it to 384KiB

11:58 <marex> qschulz: sounds rockchip specific ?

12:04 <qschulz> marex: yes, thought I had mentioned that but forgot to :)

12:13 <marex> qschulz: so ... time to write BL31 hooks for rockchip ? :)

12:19 <qschulz> marex: well, the timing is bad because if we really pass the kernel DT to TF-A with the TF-A in kernel fit, then it'll be a disaster

12:20 <qschulz> because the DT are already above the size limit :/

12:20 <qschulz> so before this patch gets merged, it makes it risky for people to use it

12:20 <qschulz> well at least it shouldn't crash TF-A if it's too big :)

12:21 <qschulz> (as opposed to Rockchip's TF-A, which may or may not since it predates the upstream version which fixed those issues)

13:37 <marex> qschulz: but this is rockchip only problem, right ?

13:38 <marex> qschulz: and ... does the rockchip TFA even support use of external DT right now , or does it ignore it ?

13:38 <qschulz> marex: there's probably a size limit for all architectures in TF-A

13:38 <qschulz> but the limit is architecture-specific

13:39 <marex> qschulz: and ... cant the rockchip TFA simply parse the external DT and configure itself accordingly ? Or does it always hold references to that external DT even during runtime (i.e. during its PSCI providering) ? I don't think it does ...

13:39 <qschulz> Rockchip's TF-A blob accepts external DT but it's so unreliable you better not do it

13:39 <qschulz> upstream TF-A for Rockchip works fine, in the 128KiB limit

13:40 <marex> qschulz: isnt that 128kiB internal DT limit ? Or does that really apply to external DT too ?

13:40 <qschulz> it keeps a copy of the external DT

13:40 <qschulz> don't know what you mean by internal/external DT

13:40 <qschulz> you pass a DT to TF-A and TF-A copies it

13:40 <qschulz> this copy is limited to 128KiB

13:40 <qschulz> right now

13:44 <marex> oh hum

13:44 <qschulz> i haven't checked bloblists, but I assume this could be a better way to handle things

13:46 <marex> qschulz: maybe some bootph-* to reduce the DT passed to TFA might help too ?

13:46 <marex> qschulz: or just fix TFA and up the limit

13:46 <marex> that's likely the easiest way

13:46 <qschulz> we're trying to up the limit but you know things take a long time to land in TF-A

13:46 <qschulz> not sure what bootph-* properties would help with

13:59 <calebccff> marex: i still think re-executing U-Boot at a lower EL (and passing DT via a well-known address without a copy?) is the more generic way to go

13:59 <calebccff> especially if you're doing EFI

14:00 <calebccff> it can even be the same binary you loaded in EL3

14:01 <qschulz> calebccff: it's not that easy, if you don't want to copy the DT, you need to have the address reserved to avoid a lower EL modify it

14:01 <xypron> calebccff: UEFI is expected to run at EL2 or S-mode depending on the architecture. A U-Boot in EL3 would not fit here.

14:01 <calebccff> xypron: that's exactly my point

14:01 <qschulz> calebccff: also, you need enough space to be able to modify (read add stuff to) the DT

14:01 <xypron> But SPL could be running at EL3 or M-mode.

14:02 <calebccff> qschulz: if u-boot is the first thing running at a lower EL then it can presumably just not nuke your FDT?

14:02 <qschulz> calebccff: not sure what you mean here? U-Boot also fixups the DT, possibly the one provided by TF-A

14:04 <xypron> qschulz: Why do you want to take a DTB from a FIT image for TF-A? Why can't it be loaded at a later stage in your use case?

14:05 <calebccff> xypron: qshulz: bake a full-fat U-Boot with an embedded FIT containing the FDT, TF-A, and OP-TEE if you like. Launch it in EL3, it goes "oh im in EL3, loader-mode activate!" and then adjusts the default boot path to launch TF-A with the normal world entrypoint targeting it's own _start address. Then after TF-A/OP-TEE init they re-start u-boot at a

14:05 <calebccff> lower EL, which still finds the same FIT image (containing its DTB), but triggers a different bootflow because it's now at a lower EL

14:06 <calebccff> if you want TF-A to mess with your DT you could feed it a stub, or just patch TF-A to not have such a ridiculous size limit i guess

14:06 <calebccff> but this way you get u-boot in el3 which you can stop at and do anything you like (full fat u-boot woo), otherwise it boots through to normal u-boot at a lower el, it just happens to be the same binary

14:10 <qschulz> sorry, I don't follow. Seems like we have multiple topics in parallel and I just cannot comprehend what we're talking about and what I should answer to

15:48 <calebccff> sorry, i guess marex' fosdem talk and our prior discussions here and on list are kinda necessary context

15:55 <marex> calebccff: btw starting U-Boot again in EL2 is an option if you don't care about boot time optimization

15:56 <marex> calebccff: basically ... SPL -> U-Boot (EL3) -> TFA BL31 -> U-Boot (EL2) -> Linux ... this works, and is easy to do with the board specific hook, but it will add to your boot time

15:57 <marex> calebccff: for security deployments ... SPL -> TFA BL31 -> Linux ... seems the best alternative

15:57 <marex> calebccff: for development purposes, ... SPL -> U-Boot (EL3) -> TFA BL31 -> Linux ...

16:03 <marex> ... and the plumbing is now in place to implement them all ...

16:07 <Tartarus> marex, calebccff, on the EL/etc stuff, I think it was the arm EFI app series from sjg1 that I was hoping for more feedback from someone else on, wrt some of the "we're in EL$X and so ..." parts of.

16:15 <xypron> Tartarus: The EFI App will start on top of an UEFI implementation. So it should already be in EL2 or S-Mode. do_bootefi_exec() calls switch_to_non_secure_mode() which on ARM will get us into EL2 if we aren't yet. A call outside the EFI sub-system does not exist.

16:17 <xypron> Tartarus: EFI spec 2.11 has in chapter 2.3.6 AArch64 Platforms the requirement: Use the highest 64 bit non secure privilege level available; Non-secure EL2 (Hyp) or Non-secure EL1(Kernel).

16:18 <Tartarus> xypron: OK. But I think it was more of the "how is this coded" part because there was in my mind similar issues that marex was needing to have solved

16:19 <marex> xypron: in that case , if U-Boot is in EL3, do_bootefi_exec() may need to inform user to either switch or start the PSCI provider if needed ?

16:19 <xypron> See switch_to_non_secure_mode()

16:19 <marex> and then switch to EL2

16:26 <xypron> I would not assume that a user starts PSCI manually. If TF-A does not offer a PSCI implementation, U-Boot would have to carry the implementation as a runtime driver.

16:46 <marex> Tartarus: besides the fact that 940135eea5df ("Kconfig: Move CONFIG_BOOTCOUNT_ALTBOOTCMD to Kconfig")

16:46 <marex> breaks multiple boards and introduces duplication ... why is the symbol called CONFIG_BOOTCOUNT_* ?

16:46 <marex> altbootcmd has nothing to do with bootcount ?

16:47 <Tartarus> marex: I'm unaware that it breaks multiple boards, and because when the bootcount is exceeded is when altbootcmd is run?

16:47 <Tartarus> I thought I had checked before/after migrating the values, but that is indeed an annoyingly tricky kind to do

16:47 <marex> Tartarus: if I had a chance to review it (as it does touch multiple devices I care about), I could point it out ... currently I am fixing the damage

16:47 <Tartarus> So what'd I break?

16:49 <marex> at least imx8mp edm sbc

16:50 <marex> err ... imx8mm edm sbc

16:50 <marex> I actually wanted to focus on something entirely different, but now I have to deal with this first ... sigh