dustymabe changed the topic of #fedora-coreos to: Fedora CoreOS :: Find out more at https://getfedora.org/coreos/ :: Logs at https://libera.irclog.whitequark.org/fedora-coreos
b100s has quit [Remote host closed the connection]
b100s has joined #fedora-coreos
plarsen has quit [Remote host closed the connection]
gursewak has joined #fedora-coreos
gursewak has quit [Ping timeout: 250 seconds]
bgilbert has quit [Ping timeout: 252 seconds]
b100s has quit [Remote host closed the connection]
b100s has joined #fedora-coreos
b100s has quit [Ping timeout: 250 seconds]
gursewak has joined #fedora-coreos
gursewak has quit [Ping timeout: 250 seconds]
gursewak has joined #fedora-coreos
gursewak has quit [Ping timeout: 260 seconds]
b100s has joined #fedora-coreos
gursewak has joined #fedora-coreos
troglodito has quit [Read error: Connection reset by peer]
gursewak has quit [Ping timeout: 250 seconds]
troglodito has joined #fedora-coreos
gursewak has joined #fedora-coreos
gursewak has quit [Ping timeout: 250 seconds]
jlebon has quit [Quit: leaving]
gursewak has joined #fedora-coreos
Betal has quit [Quit: WeeChat 3.8]
daMaestro has joined #fedora-coreos
gursewak has quit [Ping timeout: 250 seconds]
cyberpear has quit [Quit: Connection closed for inactivity]
jcajka has joined #fedora-coreos
sentenza has quit [Remote host closed the connection]
saschagrunert has joined #fedora-coreos
plundra has quit [Remote host closed the connection]
plundra has joined #fedora-coreos
jpn has joined #fedora-coreos
jpn has quit [Ping timeout: 250 seconds]
daMaestro has quit [Quit: Leaving]
jpn has joined #fedora-coreos
jpn has quit [Ping timeout: 255 seconds]
jpn has joined #fedora-coreos
bgilbert has joined #fedora-coreos
saschagrunert has quit [Remote host closed the connection]
bgilbert has quit [Ping timeout: 260 seconds]
fifofonix has joined #fedora-coreos
fifofonix has quit [Ping timeout: 265 seconds]
fifofonix has joined #fedora-coreos
jbrooks has quit [Read error: Connection reset by peer]
jbrooks has joined #fedora-coreos
vgoyal has joined #fedora-coreos
mheon has joined #fedora-coreos
jlebon has joined #fedora-coreos
<dustymabe> 👋 jlebon
<dustymabe> FYI all: no pods are getting started successfully in the FCOS pipeline
<dustymabe> looks like the jnlp container is having trouble: https://paste.centos.org/view/d3382090
<dustymabe> I can try to restart jenkins to see if that helps
<jlebon> dustymabe: 👋
<jlebon> will take a look too
millerthegorilla has joined #fedora-coreos
nalind has joined #fedora-coreos
<jlebon> restart did help
gursewak has joined #fedora-coreos
gursewak has quit [Ping timeout: 260 seconds]
cyberpear has joined #fedora-coreos
jcajka has quit [Quit: Leaving]
gursewak has joined #fedora-coreos
jpn has quit [Ping timeout: 255 seconds]
jpn has joined #fedora-coreos
plarsen has joined #fedora-coreos
Betal has joined #fedora-coreos
bgilbert has joined #fedora-coreos
<dustymabe> jlebon: any idea why secure boot doesn't seem to work for <= 33 ?
sentenza has joined #fedora-coreos
<bgilbert> dustymabe: what does "doesn't work" mean?
<dustymabe> it won't boot
<bgilbert> dustymabe: what does "won't boot" mean?
<dustymabe> BdsDxe: failed to load Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x2,0x0): Access Denied
<dustymabe> BdsDxe: No bootable option or device was found.
<dustymabe> BdsDxe: Press any key to enter the Boot Manager Menu.
<bgilbert> I vaguely recall some old GRUB vulnerabilities. have the old binaries been denylisted in UEFI?
<dustymabe> but i'm booting images that were created a long time ago (i.e. the bootloader baked in the image shouldn't deny itself)?
<dustymabe> or maybe there is something in newer qemu
<bgilbert> qemu is what I was thinking
<dustymabe> that makes it not boot
<dustymabe> ahh, yeah
<bgilbert> our tests probably shouldn't assume that old images will continue to Secure Boot forever
<dustymabe> yeah, they don't have to (we can add exceptions where needed)
<dustymabe> but when possible it would be nice to point to the reason something doesn't work (other than "we don't know why but this doesn't work")
<dustymabe> bgilbert: I'm guessing that means we have the same problem for secureboot too then
<dustymabe> i.e. anything pre-34 won't continue to boot
<dustymabe> unless someone applies `sudo bootupctl update` ?
<dustymabe> well
<bgilbert> dustymabe: there was a dangling thread from some time ago. maybe it was around "Boot Hole"? whichever vuln prompted the creation of bootupd
<dustymabe> I guess that depends on if they updated their hardware/firmware on their bare metal machines?
<bgilbert> dustymabe: we were told we'd need to update bootloaders in due course because a dbx update was coming
<bgilbert> dustymabe: ...actually, that might be it
<bgilbert> FCOS assumes it's not dual-booting with anything
<bgilbert> so if we don't perform dbx updates ourselves, we can continue to run with a denylisted bootloader and be none the wiser
<bgilbert> and for newer hardware that already has the newer dbx, users start with a newer FCOS anyway
<bgilbert> the July 2020 dbx update includes binaries identified as being from "Fedora Project"
<bgilbert> looking at the "CSV file" (actually .xlsx) at https://uefi.org/revocationlistfile
<dustymabe> bgilbert: so if I take my Compaq server (hypothetical example with unlikely hardware vendor) that has been running f32 with secure boot enabled for years now, unless I do something to the hardware (like run a vendor firmware update), my system will continue to boot?
<bgilbert> dustymabe: right, the OS is responsible for updating the denylist
<bgilbert> it feels like a bit of a time-bomb though
<dustymabe> bgilbert: so what is it in qemu that's making older systems (pre-baked from years ago) not boot?
guesswhat6 has joined #fedora-coreos
<bgilbert> dustymabe: I'm still digging, but presumably it ships with a newer denylist
<bgilbert> as will new hardware
<bgilbert> re the "dangling thread" ^: I don't think we ever followed through
<dustymabe> bgilbert: yeah, i wouldn't be surprised
<bgilbert> okay, looks like dbxtool is the low-level update tool, but it's wrapped by fwupd, which knows to refuse to update dbx if the ESP contains binaries with denied signatures
<bgilbert> (assuming nothing about the FCOS ESP handling confuses it; I haven't tested)
<bgilbert> dustymabe: I just spot-checked `dbxtool -l` inside `cosa run --qemu-firmware uefi-secure` and found hashes from the April 2021 dbx update
<bgilbert> I'd guess that OVMF builds eventually incorporate newer dbx
<bgilbert> hmm. `fwdupdtool get-updates` in that VM says `cannot find default ESP: No ESP or BDP found`
<dustymabe> yeah. I think the thing that confuses me is the relationship between the different pieces
<dustymabe> i.e is the controlling mechanism inside the image that's baked OR is it in the hardware/firmware of the machine
<dustymabe> the sense that I'm getting is that it's a combination of both
<bgilbert> the dbx denylist is baked into the firmware and then updated similar to a firmware update
<bgilbert> OSes are supposed to apply those updates to prevent the machine from later being compromised by a known vuln
<dustymabe> ok it makes sense now
<dustymabe> so.. if a person never updates their own firware, they'll continue to be able to boot
<bgilbert> yes, though somewhat defeating the secure in Secure Boot
<dustymabe> but if they do update their firmware or take the disk and move it to a different machine, then boot could be denied
<bgilbert> right
<dustymabe> well at least I understand it a bit more now :)
<bgilbert> I think we should probably try booting an old FCOS with old UEFI firmware (in qemu), upgrade to current, and then see whether fwupd correctly prevents a dbx update
fifofonix has quit [Read error: Connection reset by peer]
<dustymabe> :) - care to take that one?
<bgilbert> the concern is that it won't find our ESP (not mounted by default) and then decide to update dbx without checking the existing bootloader
<dustymabe> or maybe open an issue for it?
<bgilbert> (or read the code, I suppose)
<dustymabe> yep. that's a worrying concern
<dustymabe> just when you think everything is all stable and nice.. dusty has to go writing tests
<bgilbert> yeah, no more writing tests, all they do is find problems
<bgilbert> I'll open an issue. it might be a good task for someone to get some experience in the space
<bgilbert> actually
<bgilbert> dustymabe: could you open a bug with the test results you saw, and I'll comment on it?
<dustymabe> yep.
<bgilbert> ty
<jlebon> do i understand correctly the concern here is that FCOS isn't "fulfilling its job" of updating the firmware?
<jlebon> presumably as sole owner of the system
<jlebon> and so users' systems are still running with out of date revocation lists?
<bgilbert> that's one concern. it undermines Secure Boot protections
<bgilbert> another concern is that we need to verify that a user deciding to update their firmware with fwupd won't brick their FCOS install
<jlebon> ack gotcha
<jlebon> one the output of this should probably be an entry in the docs for now at least showing how to use fwupd
<bgilbert> yeah
<dustymabe> fill in appropriate context ^^
<dustymabe> because I did a bad job of it
<bgilbert> +1, thanks
<jlebon> nice find dustymabe
gursewak has quit [Ping timeout: 248 seconds]
<bgilbert> dustymabe++
<zodbot> bgilbert: Karma for dustymabe changed to 4 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any
<millerthegorilla> Hi, in my butane/ignition I have a user with a supplementary group membership of the video group.  Boot fails with journalctl message with failed to configure user - useradd: group 'video' does not exist.   But the group 'video' is definitely in /etc/group (cat /etc/group in emergency mode), and if I try and create the group using the ignition
<millerthegorilla> file, boot fails complaining that the 'video' group already exists.   I am using ansible so I can add the user to the group later on, but I am a bit confused as to why this would happen.
<bgilbert> millerthegorilla: known issue: https://github.com/coreos/fedora-coreos-tracker/issues/155
<bgilbert> the problem is that `video` is defined in /usr/lib/group and adduser doesn't see it there
<bgilbert> millerthegorilla: a bit more advice in https://github.com/coreos/butane/issues/411#issuecomment-1407544648
<bgilbert> it's definitely awkward
<jlebon> bgilbert: would you be against ignition learning to promote a group from /usr/lib to /etc ? compile-knob conditionalized probably
<bgilbert> I see this as a shadow-utils bug, but it's so painful that yeah, it probably deserves a workaround
<bgilbert> s/adduser/useradd/
<bgilbert> my Debian roots are showing
<jlebon> i can file an ignition and we can discuss there
<millerthegorilla> bgilbert thanks.  I have been having all sorts of fun recently with the various group files, getent, lib/group, sssd and other forms of weird misdirection.
<bgilbert> jlebon: +1
<bgilbert> millerthegorilla: yeah, it's not intuitive
<uny[m]> Is there a way to have the CoreOS ISO boot to the system it just installed?
<uny[m]> It takes a support request to my ISP to mount/unmount an ISO, so I'd like to try the new system without unmounting the ISO first.
<uny[m]> Was hoping for a boot selector ... or to be able to chainload to /dev/vda from the boot command line ... but I'm not seeing how to do it.
<uny[m]> Also, the installer doesn't seem to have expanded the root partiton.
<uny[m]> Does it do this on first boot? I expected to see around a 52GB partition in there.
gursewak has joined #fedora-coreos
<uny[m]> Right, it won't grow until I can figure out how to boot /dev/vda.
gursewak has quit [Remote host closed the connection]
<bgilbert> uny[m]: the root partition is expanded on first boot, yes. it's after Ignition runs, and doesn't happen if you've used the Ignition config to customize your storage.
<uny[m]> thanks, so far so good.
<bgilbert> uny[m]: the ISO doesn't have a built-in way to launch the installed system directly. kexec exists, but there are no purpose-built tools to use it for that purpose
<bgilbert> oh, maybe you're asking for a "boot to HD" entry in the boot menu
<uny[m]> I didn't see how to kexec to /dev/vda ... is it easy? Some web searching made it look pretty hard.
<bgilbert> haven't done it, but I don't think it's especially easy
<uny[m]> Yes, "Boot to HD" would have been awesome!
<bgilbert> I've seen ISOs with a "boot to HD" entry before. might be reasonable to add
<bgilbert> could you file a feature request? https://github.com/coreos/fedora-coreos-tracker/issues/new/choose
<uny[m]> sure, will do.
jpn has quit [Ping timeout: 260 seconds]
<uny[m]> So I had support unmount the ISO (they responded quick!) and I've booted into my new system ... and I notice I didn't give any users passwords and didn't assign the root user an ssh key in my .ign.
<uny[m]> Just went with the default example.bu in the docs.
<uny[m]> How do I get root permission?
<bgilbert> did you assign the core user an SSH key?
<uny[m]> AH, I didn't realize core was magic.
<bgilbert> it has sudo access by default
<uny[m]> I gave bronson an SSH key, but I don't seem to have sudo access.
<uny[m]> Crud.
<bgilbert> do you have console access?
<uny[m]> Well, I guess I need to reinstall with a core user?
<uny[m]> Yes I do
<jlebon> do you have write access to the console?
<uny[m]> think so
<jlebon> can you catch the grub boot menu?
<jlebon> right :)
<uny[m]> Thanks, giving it a shot.
<millerthegorilla> I included remote files in the config.ign  but when I boot I am getting a tcp lookup error.  Shouldn't networking be finished and working by the time these files are downloaded?  I tried setting timeouts under ignition, of http_total 120, but the http requests didn't timeout, and so I am unable to see the log - is it written to disk at a path I
<millerthegorilla> can locate?
<bgilbert> millerthegorilla: does it drop to an emergency shell prompt, or wait forever?
<millerthegorilla> waits forever.  So, I tried to set the timeouts  and do it again, but it still tries forever (no timelimit)
<bgilbert> hmm. what platform?
<bgilbert> Ignition doesn't wait for networking to come up, which is why it retries forever
<millerthegorilla> its a rpi4b
<bgilbert> "networking up" is hard to define, so Ignition just assumes it can wait long enough
<bgilbert> DHCP?
<bgilbert> wired Ethernet?
<millerthegorilla> So, if I just leave it?  DHCP should be automatic.  If I boot the pi as coreos it is assigned an ip, eventually.  Its not quick.  but it gets there.
<bgilbert> I mean, if it's 30 seconds that seems fine, but if it's multiple minutes, something is wrong
<millerthegorilla> its longer than that, I think, but I will check.  The http request isn't blocking then, and so the networking should come up in the background?  I will try it again.
<bgilbert> correct, NetworkManager runs in parallel
<uny[m]> grub worked, thanks bgilbert and jlebon I can access the core user now.
<bgilbert> uny[m]: 🎉
<jlebon> nice! :)
<uny[m]> And /sysroot is expanded as expected. Time to get to work.
<millerthegorilla> looks like it worked.   Just a missing overwrite and it should boot, thanks!
<bgilbert> millerthegorilla: 🎉
<millerthegorilla> fri night celebrations to all!
millerthegorilla has quit [Quit: millerthegorilla]
<dustymabe> 🎉
<dustymabe> marmijo[m]: around? can you check something in azure for me?
<marmijo[m]> dustymabe: Yeah. What can I do?
<dustymabe> when you log in to azure web interface - you're able to see all the resource groups, right?
<marmijo[m]> normally, yes. Let me log in now
jpn has joined #fedora-coreos
<marmijo[m]> dustymabe: yes I can see them all
<jlebon> bgilbert: ughh, i think it's fallout from https://github.com/coreos/coreos-ci-lib/pull/145. testing a fix
<dustymabe> marmijo[m]: thanks!
<dustymabe> thats all
<marmijo[m]> cool, you're welcome!
brianmcarey[m] has joined #fedora-coreos
<dustymabe> jlebon: this one should be ready now (I hope): https://github.com/coreos/fedora-coreos-pipeline/pull/847
* dustymabe be back in 3 min
<jlebon> dustymabe or bgilbert: mind a review on https://github.com/coreos/coreos-ci-lib/pull/146
<uny[m]> <bgilbert> "I've seen ISOs with a "boot to..." <- https://github.com/coreos/fedora-coreos-tracker/issues/1453
<bgilbert> uny[m]: thanks!
<jlebon> dustymabe: had one comment
<dustymabe> jlebon: good catch, fixed
<jlebon> :lgtm:
<dustymabe> jlebon: do you understand the CI failure?
<dustymabe> from Codacy?
plarsen has quit [Quit: NullPointerException!]
<jlebon> i do not. i've been meaning to ask actually who set this up and what it is
<dustymabe> oh - so it's not something we added? weird
gursewak has joined #fedora-coreos
vgoyal has quit [Quit: Leaving]
Betal has quit [Quit: WeeChat 3.8]
gursewak has quit [Ping timeout: 255 seconds]
jpn has quit [Ping timeout: 255 seconds]
nalind has quit [Quit: bye]
Betal has joined #fedora-coreos
jpn has joined #fedora-coreos
mheon has quit [Ping timeout: 250 seconds]
jlebon has quit [Quit: leaving]