<sjg1>
clamor: Actually I don't think I can load this onto nyan, or at least not without a lot of effort. I thought I would try an rpi2 since it doesn't have an existing image. It won't boot, of course, but should show the extlinux problem
<Tartarus>
Once I see that, really, yes, that much string change is enough to cause the platform to fail to boot, I'll start comparing the binaries and see what shifts where
<sjg1>
Tartarus: Sometimes these sorts of problems are due to using BSS before relocation
<Tartarus>
Yeah, it's going to be something along those lines
<marex>
Tartarus: gah
<Tartarus>
Indeed. Just hack that board out of CI for the moment
<marex>
Tartarus: I have exfat fixes in the pipeline
<Tartarus>
ok
Jones42 has quit [Ping timeout: 272 seconds]
Jones42 has joined #u-boot
frieder has quit [Remote host closed the connection]
Jones42 has quit [Ping timeout: 248 seconds]
alexxy has quit [Ping timeout: 246 seconds]
<Tartarus>
And, OK, progress, more "would be good to generalize linker scripts"
<Tartarus>
Yup, BSS not aligned
<Tartarus>
Changes in previous content leaving it unaligned, boom
dhruvag2000 has quit [Quit: Connection closed for inactivity]
<calebccff>
sjg1: regarding DT selection, I think you'd need to make the case to the kernel that it makes sense to export a "match table" file of some sort, perhaps per-vendor which would be installed automatically when you do make dtbs_install, this would have the list of base compatibles and the dtb file name
<calebccff>
sjg1: then we'd need to make the case to distros to start using the new file (e.g. in the systemd kernel-install tool), I guess we'd also want this to be added to LTS kernels....
<calebccff>
and teach sd-boot to use it too
<calebccff>
then the FdtCompatible EFI variable (mentioned in the systemd issue i linked) proposal would make a lot more sense
mmu_man has joined #u-boot
warpme has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<Tartarus>
mkorpershoek: Oh, that lz4 thing. I _think_ Simon has some change buried in his tree to deal with this?
<xypron>
calebccff: If a distro needs a table listing fdtfile vs compatible, it should create a script to create such a file. I would not know why the Linux project should do this.
<sjg1>
calebccff: OK, I sent a patch
<sjg1>
clamor: Er, clamor
<clamor>
sjg1: ok
<sjg1>
calebccff: There is already 'make image.fit' which creates such a table in FIT format. I wonder if we can use that somehow?
<sjg1>
calebccff: I suspect we don't need to worry about LTS kernels since we normally don't backport new hardware support to them?
mmu_man has quit [Ping timeout: 260 seconds]
<sjg1>
calebccff: I have to agree with xypron that finding the correct DT is more a job for U-Boot raher than Linux
Jones42 has joined #u-boot
goliath has quit [Quit: SIGSEGV]
apritzel_ has joined #u-boot
<calebccff>
xypron: sjg1: well right now nobody is willing to take responsibilty for DT, distros want the board to hand them a golden DT which works for whatever kernel they're shipping, the kernel expects upstream DT to just work or something despite it not really being complete, and vendors won't embed a DT without assurances it will actually work which nobody is there to give
<calebccff>
so we can argue all day and night over what the "correct" solution is, in the mean time people are buying hardware that they /want/ to run Linux on and they're bouncing off because just booting the installer is such a huge pain
<calebccff>
for e.g. Qualcomm today, it's not reasonable that a dev board could have a fixed DT that would work for newer or older kernels. This is WRONG and BAD, but it's the way it is. Until that's fixed, the only way to have solid distro support is to tie the DT to the kernel version, and that requires that at the point in time where we know which kernel version we're going to boot, we can load the matching
<calebccff>
DT
<sjg1>
calebccff: U-Boot's responsibility is to find the correct DT for the board and it can happily do so, so long as its own compatible string is correct. Perhaps the real problem here is that the platform uses ACPI, and we need to translate from SMBIOS fields to a compatible string? How are you handling that?
<calebccff>
since everybody is mostly sane and shipping EFI nowadays, that point is in the OS loader EFI app
<calebccff>
sjg1: there are no devices with a problem like you have just described
vagrantc has joined #u-boot
<sjg1>
We should certainly allow use of the kernel's devicetree, for the kernel's version. I thought we were just talking about how to select from the available options?
<calebccff>
On the Snapdragon laptops, my ideal scenario is that you boot some small EFI app which asks you what DT your device should use and writes the FdtFile EFI variable
<sjg1>
OK, so getting a compatible string in U-Boot works fine?
<calebccff>
sjg1: we aren't running u-boot code at the point in time where we need to select the DTB
<calebccff>
we're running sd-boot or grub or whatever
<calebccff>
we need to feed in the right information, which you think should be a compatible string
<calebccff>
and then leave it to the OS loader to decide what to do
<sjg1>
But, as I understand it, you don't want to implement FIT matching in other boot loaders, right? It would be too painful?
<calebccff>
i want to take the path of least resistence, and FIT is not that
<calebccff>
i'd rather have a booting device than the "right" solution
<sjg1>
calebccff: Is it possible for now to boot with just U-Boot and leave out the other loaders?
apritzel_ has quit [Ping timeout: 252 seconds]
<sjg1>
calebccff: at least to get things working initially
<calebccff>
sjg1: I want to power on my board, add a boot entry in EFI that points to the minimal debian ARM64 cd image URL, run "bootefi bootmgr" and land in a debian environment
goliath has joined #u-boot
<sjg1>
calebccff: Where do you want to be running 'bootefi bootmgr'
<calebccff>
sjg1: then i want to do it with Fedora, postmarketOS, Chimera Linux, Ubuntu, Fedora KDE remix, Alpine Linux
<calebccff>
at the u-boot prompt
<calebccff>
or, put another way, download the ISO, dd it to a usb drive, plug it into my Snapdragon laptop (running EDK2) or smartphone (running U-Boot) and land in a functional OS
<calebccff>
these are all expecting you to run bootaa64.efi and for them to do their own thing
<sjg1>
calebccff: OK, but EFI has no protocols for dealing with devicetrees, so far as I am aware, It was designed for Itanium and ACPI. Am I correct on that?
<calebccff>
maintained by some nice folks who see our pain, and are willing to extend it so that OS loader implementing that spec can handle DT selection
<sjg1>
(It's certainly the world people are creating, but if we lived in it, it would already work. That spec presuppose EFI, BTW)
<sjg1>
OK so we could design an EFI protocol which can provide the necessary information. For bootloaders which cannot deal even with matching compatible strings, we could handle that too
<calebccff>
so the os loader would call some protocol with a path(?) to search and u-boot would find the dt?
<sjg1>
calebccff: That's what I'm thinking, yes. It would find the correct DT, load it, do fixups and pass it back
<calebccff>
that's just a huge layering violation, completely the wrong scope since now we're enforcing that distros ship all the DTBs as files in a directory, which we already all agree is kinda non-ideal
<calebccff>
but really, os loaders like systemd uki stub already do compatible matching themselves
<xypron>
sjg1: I cannot see why U-Boot should do this.
<xypron>
sjg1: The OS has already all information it needs, which is the compatible string.
<calebccff>
this also doesn't solve the problem for the non U-Boot case, like my snapdragon laptop
<sjg1>
calebccff: Perhaps we should discuss this on a call sometime. I think you making quite a few assumptions here
<xypron>
calebccff: The snapdragon case shows that the problem should not be solved in U-Boot, but in the next boot stage.
<sjg1>
xypron: The thing is, the OS needs a DT, not a compatible string. It can't do anything with a compatible string
<calebccff>
xypron: exactly
<xypron>
The OS gets a DT with a compatible string. If it wants to ignore that device-tree, that is the OS choice.
<sjg1>
calebccff: OK, but I thought you said you don't want to do compatible-string matching in sd-boot or grub?
apritzel has quit [Ping timeout: 268 seconds]
<xypron>
U-Boot cannot make that search any faster than sd-boot or grub.
<sjg1>
xypron: No, we are talking about how to select the correct devicetree, not whether the tree has one or not
<xypron>
All three of them use the same file system driver supplied by the EFI firmware.
<calebccff>
sjg1: i don't see how a call would be productive, any solution you propose which requires changes to the contents of the ESP/XBOOT partition or requires new EFI protocols is by my definition not a solution
<xypron>
sjg1: U-Boot cannot not possibly know what the OS considers a correct device-tree. It does not even know the Linux version chosen interactively by the user.
<calebccff>
sjg1: as was stated in the community meeting, obviously compatible matching is better, but file name is already treated as API, and is drastically easierr to implement and reason about
<calebccff>
in most cases the compatible and file path contain identical information
<sjg1>
You are each talking about different things and different problems. Can I suggest that we first solve the problem Caleb has, then Heinrich you can come in with your problems?
<xypron>
We should make a clear separation of duties. U-Boot starts an EFI binary with the device-tree it has, which with current standards will be the shiniest Linux device-tree available at the time of U-Boot building. The OS can choose use it or not. We should not care in this project.
<calebccff>
yeah
tlwoerner_ has quit [Quit: Leaving]
<calebccff>
sjg1: xypron is just clarifying how things behave today
<sjg1>
That's not going to scale, though. In practice, the devicetree needs to come from Linux, right?
tlwoerner has joined #u-boot
<xypron>
It is a different thing if U-Boot starts a FIT image with dozens of device-tree. Then it is up to U-Boot to choose the configuration.
crb_ has quit [Ping timeout: 252 seconds]
<calebccff>
sjg1: right, in practise we get better compatibility from picking a DTB, and in practise other bootloaders (snapdragon laptops) don't provide a DT, so we need to pick one anyway
<xypron>
sjg1: Yes, and if a Linux based distro cannot work with a device-tree comming from a previous kernel version the distro has to work around it and not U-Boot.
<calebccff>
the problem is entirely outside of U-Boot, the solution I'm proposing has nothing to do with U-Boot
<calebccff>
and the changes I'm suggesting we make in U-Boot only serve to simplify rolling out that proposed solution
<sjg1>
Then why are you discussing it on the U-Boot list and call?
<sjg1>
I thought you were trying to make a change to U-Boot?
<calebccff>
yeah, to set FdtFile EFI var automatically, which is what we discussed and mostly agreed on
<calebccff>
except for you
<calebccff>
so im trying to giving you the big picture so if there is some better solution im missing that we can do that instead
<sjg1>
Are you open to other solutions?
<calebccff>
totally, we just haven't found one that is as good
<calebccff>
FdtCompatible is close but will probably take longer to role out since it would require patches to the kernel
<sjg1>
It seems to me like you have already decided that there is no other solution. That's why I am asking
<sjg1>
If you are open to alternatives, we can discuss how to design an alternative
<clamor>
sjg1: IMHO, u-boot itself is just enough to boot OS. Nothing else is needed
<sjg1>
I firmly believe that U-Boot (or the following bootloader) needs to be able to load a devicetree build by the same kernel version
<xypron>
calebccff: What is wrong about the method that sd-boot uses? Adding all device-trees to the kernel and using the EFI stub to choose the DT?
<sjg1>
The one in firmware is a default, but if the OS has something, we should use it
<sjg1>
But we should design a solution which supports using the U-Boot one, but allow the OS to provide its own
<marex>
about the discussion yesterday ... and DTB filename being an ABI ... do I get it right that DTB filename should roughly match compatible string now , or what is the deal there ?
<marex>
and ... about the display stuff, what about DTOs and top level compatibles, some systems have A LOT of DTOs
<calebccff>
sjg1: to lay out the requirements again: 1) shouldn't require distro-specific changes, 2) should be applied to all EFI implementations (so kinda limited to EFI vars), 3) OS loader changes need to be accepted into the bootloader spec (for sd-boot distros, and Fedora)
<marex>
and we cannot append top level compatible from a stack of DTOs
<calebccff>
sjg1: (3) is the hard one right, because if you start doing stuff like reading hundreds of file they are going to NACK the approach
<sjg1>
Oh good, now we are 4 :-)
<calebccff>
marex: well at least in dts/qcom it usually does, mostly, but no filename isn't officially an API, and yes this does pose issues for DTOs, I would just make the FdtFile variable an array tbh
<sjg1>
We added an EFI fixup protocol, I believe. Is that still used?
<sjg1>
sorry, EFI protocol for DT fixup
<calebccff>
EFI DT fixup is totally different, and is used by somme boards
<xypron>
sjg1: Yes it is used by Ubuntu's GRUB, systemd-boot, and BSD.
<calebccff>
if we want to support overlays then the file names are the only correct way, commpatibles don't work
<calebccff>
unless each overlay adds a compatible and then we somehow can parse that?? but that feels like abuse of the compatible property
<sjg1>
Can we leave overlays aside for now?
<sjg1>
How did the EFI DT fixup protocol get added?
<xypron>
calebccff: Could you, please, explain why systemd's approach of adding all device-trees to the signed kernel and letting the EFI stub choose the device-tree does not work for you?
<xypron>
calebccff: This approach does not require any new interface and not change of any specification.
<calebccff>
xypron: distro images aren't shipping UKIs, and we would have to go around to every single distro and persuade them to pack DTBs into a UKI, either with the kernel or in a format that sd-boot would parse (and then get fedora's GRUB fork to support that too, and i guess forget about debian)
<xypron>
calebccff: No change in GRUB is needed.
<calebccff>
i think this is probably what would make systemd folks happy, but as per https://github.com/systemd/systemd/issues/36835 they seem to understand that this isn't practical right now
<xypron>
calebccff: your just need a tool like ukify.
<calebccff>
xypron: well not if you have a UKI with only DTBs in, then you have the kernel/initrd still loaded by grub
<xypron>
calebccff: 36835 just describes that $fdtfile makes not sense to them. Matching compatible strings is the alternative.
<calebccff>
and since it was confirmed yesterday that ubuntu at least aren't going to be adopting full UKI for a while because they want a separate initrd it would have to be a UKI with only DTBs
<xypron>
calebccff: Yes, we suggested a patch to make separate initrds possible.
<calebccff>
apparently that approach was nack'd by upstream?
<calebccff>
idk, im just not feeling like we'll be able to sell distros on this in a reasonable timeframe
<calebccff>
mean while FdtFile is easy, with the only drawback being that it isn't "technically correct" w.r.t how the powers that be in kernel land want DT to be treated
clamor has quit [Ping timeout: 246 seconds]
<calebccff>
filename is just a better API
clamor has joined #u-boot
<sjg1>
calebccff: If you could the filename idea for a while I think it would help. It is clouding our thinking
<calebccff>
sjg1: if i could .. ?
<sjg1>
sorry, drop
<sjg1>
Just while we're talking about this
<calebccff>
im clocking off anyhow, i don't think i have much to add that i haven't already said so let me know what you come up with i guess
<xypron>
I don't think that loading an unsigned device-tree from a file-system will be compatible with secure boot. This was the reason why the devicetree command was disabled in GRUB in secure-boot scenarios.
<xypron>
So any solution will have to ensure that a signed object provides the device-tree.
<calebccff>
xypron: installer doesn't need to be secure, at the point where you enable secureboot you can build a UKI with the appropriate DTB in
<calebccff>
ctrl+f for "I can imagine a future" on that systemd issue
<xypron>
How would you ever trust a system that was installed with secure boot disabled?
<sjg1>
+1
<xypron>
If the installer is not secure, the system is potentially compromised.
<sjg1>
Also, what if I upgrade my laptop? Does that mean I need to do a fresh install? Or is there some other thing
crb has joined #u-boot
<calebccff>
eh yeah, right, i got mixed up there
<sjg1>
The closest thing to a solution is FIT, IMO, but it isn't quite there and needs some extensions. But FIT has been rejected, so far as I can tell
<calebccff>
then i defer to the other argument which is: distros are shipping DTBs as flat files on the ESP today, so you can't do secureboot anyways
<sjg1>
(aside: at the moment I believe that UKI is FIT re-invented badly, less extensible, less flexible. The only advantage is that it is executable)
<calebccff>
sjg1: ofc when you upgrade you build a new UKI with the same DT lol
<sjg1>
calebccff: Right, but won't they have to stop doing that?
<xypron>
FIT works for U-Boot but not beyond. The UKI approach works on any EFI firmware.
<sjg1>
calebccff: I mean when I upgrade the laptop, not the OS
<xypron>
The problem with UKI as used by systemd boot is that you would have to install it on the ESP which typically is too small and has a non-journaling file system.
<sjg1>
xypron: Yes, I'm assuming that GRUB and sd-boot would never implement it
<sjg1>
xypron: That's another angle and is one reason why I think U-Boot as an EFI app can help extricate us from the FAT limitation
<xypron>
sjg1: But anybody signing kernels can add an EFI header with device-trees like ukify does.
<calebccff>
sjg1: oh right, well you'd have to build a UKI with both DTBs in and match by HWID or manually set that proposed FdtCompatible variable
<sjg1>
calebccff: Sounds right. But it's awful, isn't it
<calebccff>
imo tho it doesn't make sense to discuss secureboot until we have agreed on an implementation for the non-secureboot case
<xypron>
It makes no sense to implement a solution, that will not work for secure boot.
<calebccff>
sjg1: what do you want me to say to "it's awful"? what is technically "awful"?
<xypron>
For laptops secure boot is expected by vendors and customers.
<calebccff>
xypron: yah i didn't say implement
<calebccff>
just propose
<sjg1>
Given what I said about FIT not being of any interest to anyone involved in this effort, perhaps we should just wait for UKI to add more features from the FIT spec, like devicetree selection?
<sjg1>
I believe there was a change to support multiple DTBs in a UKI
<sjg1>
Re awful, I mean that what you actually want is for the firmware/bootloader to just select the correct DT for your new hardware, then boot
ldevulder has quit [Ping timeout: 276 seconds]
<xypron>
sjg1: UKI implemented device-tree selection with e6cb29fa0f49 ("boot: add matching against FW-provided Devicetree blob")
apritzel_ has joined #u-boot
ldevulder has joined #u-boot
clamor has quit [Read error: Connection reset by peer]
enok71 has joined #u-boot
enok has quit [Read error: Connection reset by peer]
ikarso has quit [Quit: Connection closed for inactivity]
rvalue- has joined #u-boot
ldevulder has quit [Ping timeout: 276 seconds]
rvalue has quit [Ping timeout: 260 seconds]
rvalue- is now known as rvalue
goliath has quit [Quit: SIGSEGV]
<sjg1>
xypron: OK good. I wonder what feature will be coming next
prabhakalad has quit [Quit: Konversation terminated!]
goliath has joined #u-boot
ikarso has joined #u-boot
pgreco has quit [Ping timeout: 244 seconds]
pgreco has joined #u-boot
mmu_man has joined #u-boot
enok has quit [Quit: enok]
enok71 has joined #u-boot
<xypron>
K900: The defconfigs for max and plus seem to be nearly identical. How about using an include with all the common settings? Like configs/renesas_rcar2.config is used for multiple defconfigs.
enok71 has quit [Ping timeout: 252 seconds]
<vagrantc>
does u-boot EFI provide a mechanism to save state from the OS so that efibootmgr is able to set the default boot option and such?
<apalos>
With the efivar patch applied you can skip the 'dd' step
<apalos>
distros are slowly starting to pick up this patch, so hopefully we wont need it soon
<apalos>
vagrantc: ^^
<vagrantc>
apalos: thanks ...
* vagrantc
wonders if Debian is carrying that patch
<vagrantc>
it has been a year or two (or more?) since i explored using u-boot's EFI implementation but people are clamoring to switch everything over to it ... so i should probably give it some testing
<vagrantc>
and it has obviously received a lot of improvements in that time...