12:32 <fifofonix[m]> dustymabe: question about rawhide fcos builds. is there any material gating/delay from main upstream fedora rawhide or if i see build was N hours ago can I assume it features unadulterated upstream fedora rawhide from ~N hours ago?

12:34 <fifofonix[m]> should have addressed that generally to the group.

13:29 <dustymabe> fifofonix[m]: it's a bit nuanced

13:29 <dustymabe> when it transitions to `stable` in bodhi for f39 that means it will be in the next successful rawhide compose from Fedora releng. Unfortunately sometimes the rawhide composes fail for a few days at a time

13:30 <dustymabe> looks like the one from today completed (FINISHED_INCOMPLETE) status

13:31 <dustymabe> https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20230621.n.0/STATUS

13:31 <fifofonix[m]> so, i'm worried that i'm providing misinformation on the BZ update for the CIFS issue.

13:31 <dustymabe> so you can check under https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20230621.n.0/compose/Everything/x86_64/os/Packages/ to see if the package you are expecting is there

13:51 <dustymabe> fifofonix[m]: https://bugzilla.redhat.com/show_bug.cgi?id=2182643#c35

13:54 <fifofonix[m]> dustymabe: typo on my part. responded.

13:55 <dustymabe> fifofonix[m]: +1

13:56 <dustymabe> fifofonix[m]: I think it's clear from your rpm-ostree status output.. but I'll ask anyway: is this a freshly istalled system?

13:56 <fifofonix[m]> @dustymabe: yes this morning.

13:57 <dustymabe> 👍

14:04 <dustymabe> lorbus: do you think you could open a BZ for the problem described in https://github.com/openshift/os/issues/1274#issuecomment-1597742858 ?

14:21 <baude> howdy folks, i am working on podman machine support of the Apple Silicon hypervisor; which means booting fcos. I am able to make this work with ignition being pulled over the network, etc ... but ran into a problem yesterday where i could use some team insight

14:22 <baude> the applehv is capable of popping up a spice console ... this makes testing and debugging nice

14:22 <baude> but, when using a fcos disk image, i see almost no messages and certainly no kernel messages ... but I see grub, so I know I have early console support

14:22 <baude> so first questions would be around things like ... is something special being done specifically for aarch64 fedora/fedora-core regarding console ... are assumptions being made about aarch64 being headless?

14:22 <fifofonix[m]> dustymabe: so, thanks for all of above. i think i now better understand the flow from bodhi into a stable rawhide which then flows to FCOS rawhide. that has helped me to see that the version i'm testing for this BZ issue is one built on June 15th and so doesn't feature latest latest changes that the maintainer believes address the issue. understanding quickly which commit i'm testing is my struggle i think.

14:24 <baude> fwiw, various attempts to use console=tty0|1 etc do not change results ... and iirc, that only impacts things later in boot?

14:29 <dustymabe> baude: will catch you in 30m

14:30 <baude> dustymabe: cool

14:57 <dustymabe> ravanelli: added a review to https://github.com/coreos/fedora-coreos-pipeline/pull/836

15:00 <dustymabe> fifofonix[m]: if you click through to the individual commits in the history of https://src.fedoraproject.org/rpms/selinux-policy/commits/rawhide it will show you what koji build is related to that commit

15:00 <dustymabe> for example https://src.fedoraproject.org/rpms/selinux-policy/c/38fd9a9006f250daddaf123aef4a7e7d75248d28?branch=rawhide links to https://koji.fedoraproject.org/koji/taskinfo?taskID=102173978

15:06 <dustymabe> baude: it depends on the platform what console settings are getting used. See: https://github.com/coreos/fedora-coreos-config/blob/testing-devel/platforms.yaml

15:06 <dustymabe> by default (if the platform isn't listed in that file) it should have no console= kernel command line entry

15:06 <dustymabe> IIUC

15:19 <fifofonix[m]> dustymabe: thanks for the fishing lessons. that works for me. the commit ids for the rpm definition/spec referencing the github commits (a piece i had not understood)

16:02 <baude> dustymabe: sure, but I dont think console= dictactes kernel messages does it ?

16:03 <baude> dustymabe: example, i'm messing with ignition devel; and i'm almost certain ignition is failing somewhere but I *never* get anything on the screen AND more importantly, I am not dropped to a prompt in the initrd

16:04 <dustymabe> baude: yes, console= dictates kernel messages

16:06 <baude> i have tried console=hvc0|tty0|tty1 and i nichts

16:06 <baude> get nichts

16:06 <dustymabe> this is ARM right?

16:06 <dustymabe> maybe try ttyAMA0

16:06 <baude> if i boot with something like an iso, the tty command returns tty1

16:06 <baude> arm64

16:07 <dustymabe> ok so you are on the VGA console then

16:13 <baude> dustymabe: yes, and i'm testing with a fresh fedora install ... i see all kernel messages

16:14 <baude> and there is NO console in cmdline ... which is why I was wondering about fcos specifically or arm64 fcos more specifically

16:26 <dustymabe> baude: and you are using which FCOS image? one you built locally?

16:27 <ravanelli> dustymabe: Thanks for the review in the brew PR

16:31 <spresti> aaradhak anthr76 apiaseck davdunc dustymabe guidon gursewak jaimelm jbrooks jcajka jdoss jlebon jmarrero lorbus miabbott nasirhm quentin9696[m] ravanelli saqali walters

16:31 <spresti> If you don't want to be pinged remove your name from this file: https://github.com/coreos/fedora-coreos-tracker/blob/main/meeting-people.txt

16:31 <spresti> FCOS community meeting in #fedora-meeting-1

16:32 <ravanelli> .hi

16:32 <zodbot> ravanelli: ravanelli 'Renata Ravanelli' <renata.ravanelli@gmail.com>

16:41 <ravanelli> 🤔 wrong place, sorry

16:56 <jmarrero> travier[m]: rpm-ostree 2023.5 is now in rawhide: https://bodhi.fedoraproject.org/updates/FEDORA-2023-7a8817d80b

17:03 <dustymabe> jmarrero++

17:04 <dustymabe> jmarrero: any chance you want to fast-track that in rawhide and also drop https://github.com/coreos/fedora-coreos-config/blob/f0b99f75441a9da0e3c1d888bcc50e367dec76fe/manifest-lock.overrides.yaml#L12-L36 ?

17:08 <dustymabe> baude: i'll be back in 5m or so.. if you want to do a screenshare then I'll be happy to try to help figure out the console issue.

17:09 <dustymabe> after 1:30 I have to head out for an appointment, though

17:11 <baude> dustymabe: i have used a officially built image as well as custom built ... seemingly no difference in that case.

17:18 <dustymabe> baude: want to do a screen share?

17:18 <jmarrero> dustymabe: sure thing

17:19 <baude> dustymabe: sure, sec

17:19 <baude> dustymabe: i thihnk best way will be to sen dyou a video link in slack

17:21 <baude> done

17:33 <jmarrero> dustymabe: these reference the same bug, should I drop them too? https://github.com/coreos/fedora-coreos-config/blob/f0b99f75441a9da0e3c1d888bcc50e367dec76fe/manifest-lock.overrides.yaml#L77-L86

17:41 <dustymabe> jmarrero: ahh yes. those are all part of the same source RPM

17:50 <jmarrero> dustymabe: https://github.com/coreos/fedora-coreos-config/pull/2474

18:04 <jmarrero> Looks like the CI does not like my rpm-ostree override, I wonder if we even need that in fc39 as it looks like it's been already submitted to rawhide/stable

18:57 <dustymabe> jmarrero: let's see if I fixed it in: https://github.com/coreos/fedora-coreos-config/compare/b4afe6c1824d6ce5c046a5677f54a273770e6a36..73cc7a37c66cd98985ada1ab0d47e5106b9e291d

18:59 <baude> dustymabe: is there an easy way with cosa to build an image (iso or disk) where where i take a specific kernel module and build it into the kernel?

19:00 <dustymabe> like you've compiled it yourself locally?

19:00 <baude> dustymabe: follow up question, it might also just be that the driver is not in the initrd ... iirc that is something I can do too

19:01 <baude> dustymabe: no, like i want to change the default kernel config fcos arm64 uses to build the module in (to eliminate a theory on why video is not working)

19:01 <baude> i would also settle for putting it into the initrd ... which i think i have done before, maybe I should try that first

19:01 <dustymabe> ahh. so currently it's a module (M) and you want to make it built in (Y)

19:03 <baude> right

19:04 <baude> but it may be enuff to simply sneek it into the initrd for now ... im trying to remember where and how i did that

19:06 <dustymabe> baude: you can just add it to something like: https://github.com/coreos/fedora-coreos-config/blob/testing-devel/overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree/module-setup.sh#L20

19:07 <baude> tyvm, thats it

19:08 <dustymabe> I don't know if the module you include will get loaded by default or not, might have to modprobe it

19:09 <baude> well that would be problematic if that module is the video card right?

19:10 <dustymabe> some modules get autoloaded if they exist by the software that's trying to use them. some don't. I guess just include it and see if it works

20:44 <baude> dustymabe: bingo

20:46 <dustymabe> kmod?

20:48 <baude> apparently

20:48 <baude> makes total sense

20:48 <baude> dustymabe: i think this should be fixed upstream

20:48 <baude> do you agree?

20:49 <baude> ( i mean to say, this is not just an apple thing, it is a virtio-gpu thing)

20:49 <dustymabe> as in the ultimate fix lies somewhere other than FCOS? yes, I agree

20:49 <baude> no, i think it should be in the initrd by default ?

20:49 <dustymabe> yes, I agree it should be in the initrd by default

20:50 <dustymabe> but the code that specifies that it should be in the initrd by default should be defined somewhere other than FCOS (in the long term)

20:50 <dustymabe> not unless we are just doing something completely wrong

20:50 <dustymabe> I guess you said you don't see the problem in Fedora Server?

20:51 <baude> right

20:51 <dustymabe> ok hmm

20:51 <baude> i mean, i could file a bug somewhere fcos and we can debate it ?

20:51 <baude> an issue

20:51 <baude> or, it is easy enuff i could also just do a PR

20:51 <dustymabe> can you fpaste me the line you added to get the module to be included?

20:51 <baude> and debate it there

20:53 <baude> https://paste.centos.org/view/3802861b

21:08 <dustymabe> baude: on Fedora Server can you run: `grep -ri instmods /usr/lib/dracut/ | grep virtio-gpu` ?

21:09 <baude> sure

21:09 <dustymabe> trying to figure out the difference

21:11 <baude> i dont see it

21:11 <dustymabe> yeah. weird

21:11 <baude> yeah, but this is explainable

21:12 <dustymabe> :) - explain

21:12 <baude> i wonder if server gets out of the ramdisk quicker?

21:12 <baude> i could boot the iso and peek

21:12 <dustymabe> i think we should compare the initrd from FCOS and server

21:13 <dustymabe> can you run lsinitrd on both and compare the output?

21:14 <dustymabe> on FCOS - something like this should work: `lsinitrd /boot/ostree/fedora-coreos-*/initramfs*img`

21:16 <baude> https://paste.centos.org/view/29d13562 <-- fedora server ... note line 1587

21:18 <dustymabe> i'm wondering it it's because the initrd on Server is not --no-hostonly

21:19 <baude> dunno

21:19 <dustymabe> if you run `dracut --force --no-hostonly` on that server instance and then look at the lsinitrd output, do you see the driver?

21:20 <baude> dustymabe: im looking at the next hurdle, which is the vsock kernel module. It also is not in the initrd. I have added it like virtio_gpu. But nothing probes it automatically like the video card.

21:20 <baude> iirc, i can tell ignition to modprobe a kernel mod right ?

21:21 <dustymabe> if ignition needs the module it can pull it in IIUC

21:22 <baude> yup, installkernel() in modulesetup

21:29 <dustymabe> i need to head out for the day - will check back in here later