dustymabe changed the topic of #fedora-coreos to: Fedora CoreOS :: Find out more at https://getfedora.org/coreos/ :: Logs at https://libera.irclog.whitequark.org/fedora-coreos
cyberpear has joined #fedora-coreos
gursewak has quit [Ping timeout: 260 seconds]
gursewak has joined #fedora-coreos
gursewak has quit [Ping timeout: 260 seconds]
Betal_ has quit [Quit: WeeChat 3.6]
Betal has joined #fedora-coreos
aJJa_ has joined #fedora-coreos
<adamw> Colin Walters: you finally wore me down, i hope you're happy :P
<dustymabe> adamw: nice
<dustymabe> but.. does that mean we get to hear complaining?
<dustymabe> :)
<adamw> gird those loins!
misuto has quit [Remote host closed the connection]
misuto has joined #fedora-coreos
bytehackr has joined #fedora-coreos
plarsen has quit [Quit: NullPointerException!]
paragan has joined #fedora-coreos
jpn has joined #fedora-coreos
jpn has quit [Ping timeout: 260 seconds]
paragan has quit [Ping timeout: 252 seconds]
paragan has joined #fedora-coreos
paragan has quit [Ping timeout: 255 seconds]
Betal has quit [Quit: WeeChat 3.6]
paragan has joined #fedora-coreos
<adamw> dustymabe: so...not complaining, but, say i need to do stuff like in https://bugzilla.redhat.com/show_bug.cgi?id=1936991 to debug a graphics problem
<adamw> how am i supposed to go about that? setting env vars for GNOME and writing udev rules files and so on?
<adamw> hmm, guess i can do those things in /etc, thinking about it
jcajka has joined #fedora-coreos
jpn has joined #fedora-coreos
paragan has quit [Remote host closed the connection]
c4rt0 has joined #fedora-coreos
<stephanepoq> can confirm: "/usr/bin/rpm-ostree install " works again ("openh264" repo)
guesswhat has joined #fedora-coreos
bgilbert has quit [Ping timeout: 268 seconds]
nbsadminaccount- has quit [Quit: You have been kicked for being idle]
guesswhat_ has joined #fedora-coreos
guesswhat has quit [Ping timeout: 260 seconds]
guesswhat__ has joined #fedora-coreos
guesswhat has joined #fedora-coreos
guesswhat_ has quit [Ping timeout: 260 seconds]
guesswhat__ has quit [Ping timeout: 260 seconds]
guesswhat_ has joined #fedora-coreos
guesswhat has quit [Ping timeout: 252 seconds]
paragan has joined #fedora-coreos
vgoyal has joined #fedora-coreos
aJJa__ has joined #fedora-coreos
aJJa_ has quit [Ping timeout: 260 seconds]
aJJa has joined #fedora-coreos
aJJa__ has quit [Ping timeout: 268 seconds]
aJJa_ has joined #fedora-coreos
aJJa has quit [Ping timeout: 268 seconds]
aJJa__ has joined #fedora-coreos
aJJa_ has quit [Ping timeout: 255 seconds]
<dustymabe> adamw: yeah, /etc/udev/rules.d/ should work?
<walters> adamw: cool, always open to feedback! You also will likely want to use https://coreos.github.io/rpm-ostree/administrator-handbook/#using-usroverlay
<walters> for testing stuff transiently I'd say usroverlay is much better than `/etc` unless you really do want it to persist
<walters> I am quite hopeful that the ostree native container stuff is going to finally, completely and totally destroy the mindset of "restriction" and turn to empowerment and flexibility
plarsen has joined #fedora-coreos
mheon1 has joined #fedora-coreos
crobinso has joined #fedora-coreos
<dustymabe> travier[m]: did you ever figure out the issue that you mention here? https://github.com/coreos/coreos-assembler/pull/3036#issuecomment-1228330407
<dustymabe> it's also affecting FCOS CI
<dustymabe> cc jlebon
<dustymabe> I might revert the tag in quay to something that we know was good
<dustymabe> thoughts on reverting to the change that landed Wednesday at 2PM ?
arnulfo_7 has joined #fedora-coreos
arnulfo_7 has quit [Changing host]
arnulfo_7 has joined #fedora-coreos
mheon1 has quit [Quit: WeeChat 3.5]
mheon has joined #fedora-coreos
ravanelli has joined #fedora-coreos
nalind has joined #fedora-coreos
<travier[m]> I'm looking at it
<travier[m]> we can revert it in the meantime
<travier[m]> I think I've found the issue but need to test
* jlebon waves
<jlebon> hmm weird, not sure what's going on there. travier[m] if you have something already I can help test it.
<jlebon> cool too with reverting for now.
<jlebon> hmm, this might have been fixed by https://github.com/coreos/coreos-assembler/pull/3045 actually
<jlebon> specifically https://github.com/coreos/coreos-assembler/pull/3045/commits/24ffe87937b915b2939ca235cf541731c8cf1289, which I forgot to add to the commit message the bit explaining the added / so that find doesn't get thrown off by the symlink
<jlebon> have we seen failures anywhere with latest cosa?
jcajka has quit [Ping timeout: 268 seconds]
<dustymabe> jlebon: I just opened this PR this morning: https://github.com/coreos/fedora-coreos-config/pull/1931
<dustymabe> and CI for it is hitting the problem
<dustymabe> maybe it's something specific about the coreos-ci workflow?
<jlebon> that ran before that PR. let me check the latest cosa image and restart CI on that PR as a test
<dustymabe> jlebon: oh, didn't realize there was something new in the past hour
<jlebon> quay shows a push just happened 10 mins ago, which probably has it
<dustymabe> 👍
<jlebon> but hmm, does quay not have a way to show labels on an image?
<dustymabe> we can use my PR as a guinea pig
<dustymabe> jlebon: click through the hash and you'll see the 4 arches backing the manifest
<dustymabe> then click through the hash there and you can see the labels on each image
<dustymabe> oh I see
<dustymabe> yeah I guess it would if the label was added as a `LABEL foo` Dockerfile step
<jlebon> anyway, restarted CI on that PR. let's see :)
<jlebon> dustymabe: do we even set a label with the commit sha? looking at it from `skopeo inspect` at least, it doesn't seem like it
<dustymabe> Not that I know of
<jlebon> $ host skopeo inspect docker://quay.io/coreos-assembler/coreos-assembler:main | jq .Labels | jq .
<jlebon> { "io.buildah.version": "1.26.1", "license": "MIT", "name": "fedora", "vendor": "Fedora Project", "version": "36"
<jlebon> }
<dustymabe> we just `podman build`
<jlebon> ok, let me file an issue for that since i think it's important
<jlebon> sweet, CI on that PR is passed the issue
<jlebon> fyi travier[m] ^
<dustymabe> jlebon: now you can stamp the PR :)
<travier[m]> Not sure this fixes the issue for RHCOS: https://github.com/coreos/coreos-assembler/pull/3036#discussion_r956076224
<travier[m]> jlebon:
<travier[m]> ^
<dustymabe> travier[m]: that new cosa build only landed in the last 10 minutes
<dustymabe> but yeah, maybe not?
<jlebon> travier[m]: it should I think
<jlebon> dustymabe: wait, how did you determine it was popt? can you add an issue to the tracker?
<jlebon> s/issue/comment/
<jlebon> s/tracker/issue/ :)
<dustymabe> jlebon: by sheer dumb analysis
<dustymabe> let's make sure it passes CI and I'll add a comment
<travier[m]> it's not the same code that is touched by the other PR thus why I think this does not fix it
<travier[m]> I'll retrigger RHCOS CI to check
<travier[m]> Jonathan just did it :)
<jlebon> dustymabe: it seems like smart analysis to me :)
guesswhat_ has quit [Quit: Leaving]
rsalveti has joined #fedora-coreos
<jlebon> travier[m]: that CI run is not using that PR
<jlebon> 04d000798a214c6929a67da9d60593de76fc0081 is before https://github.com/coreos/coreos-assembler/pull/3045
<travier[m]> :(
<travier[m]> ok
<travier[m]> Then we'll have to retry
<dustymabe> I literally don't have any more information than that
<jlebon> dustymabe: ahhh gotcha, I thought you had looked at the logs and had found a smoking gun
<dustymabe> no
<dustymabe> was literally "sheer dumb analysis"
<dustymabe> luckily our rigorous rpm tracking allows us to find these issues that aren't very clear
<dustymabe> we've at least narrowed it down to a package
<jlebon> +1
saroy has joined #fedora-coreos
sandipan_roy has joined #fedora-coreos
bytehackr has quit [Read error: Connection reset by peer]
saroy has quit [Ping timeout: 260 seconds]
<adamw> Colin Walters: thanks! so, uh...can you override replace the kernel?
<walters> adamw: Yes; it's tested in CI upstream for rpm-ostree, we would not ship a release unless that worked
<adamw> nice.
<walters> we dynamically switch to generating the initramfs client side, etc.
<adamw> (this isn't theoretical, btw, i'm trying to figure out why my new laptop won't drive an external monitor at 4K...)
<adamw> shiny.
<dustymabe> adamw: something like this should work `sudo rpm-ostree override replace https://bodhi.fedoraproject.org/updates/FEDORA-2022-bc2a6e0f66`
* dustymabe notes you should update the bodhi URL
<dustymabe> also works with koji URLs
<dustymabe> or remote paths to RPM files
<dustymabe> or local RPM files
paragan has quit [Quit: Leaving]
arnulfo_07 has joined #fedora-coreos
Betal has joined #fedora-coreos
arnulfo_7 has quit [Ping timeout: 260 seconds]
<adamw> yeah, i used koji urls. it worked. didn't solve the bug, but it sure works!
<adamw> looks like it can run a 4K monitor fine directly via its HDMI port, but not via a USB3 dock. hmm.
<dustymabe> jlebon: the *repo files in f-c-c all get synced to other branches from `testing-devel` correct?
<jlebon> dustymabe: correct
jpn has quit [Ping timeout: 260 seconds]
<travier[m]> It's been a few hours and RHCOS CI is still using the old COSA for a reason I don't understand
<dustymabe> travier[m]: I assume RHCOS CI is not using a branched build?
<dustymabe> but :main ?
<dustymabe> or :latest ?
<jlebon> travier[m]: commented in the PRs. i *think* that should do the trick
<dustymabe> jlebon: WDYT of this path/reasoning? https://github.com/coreos/fedora-coreos-config/pull/1932
<travier[m]> Ah, I understand now. We need to force the image rebuild to pick up the latest COSA. THanks!
<travier[m]> (what you wrote in the PR :) )
arnulfo_07 has quit [Remote host closed the connection]
arnulfo_07 has joined #fedora-coreos
<jlebon> i think we need to ask someone from testplatform about the last question in https://github.com/openshift/os/pull/959#issuecomment-1228751147, unless e.g. walters knows
arnulfo_07 has quit [Read error: Connection reset by peer]
<jlebon> we also need to make sure image mirroring from Quay.io is actually working
<jlebon> i asked in #forum-testplatform internally
sandipan_roy has quit [Quit: Leaving]
<travier[m]> thanks!
<dustymabe> want to sync here?
<dustymabe> let's get a branched compose out: https://github.com/coreos/fedora-coreos-config/pull/1934
<adamw> dustymabe: Colin Walters how do you guys install stuff like the RH VPN profiles? overlay is the most sensible choice there?
<dustymabe> yeah
<dustymabe> at least that's what I do since it's already in RPM form
<adamw> i love that the cert package pulls in chkconfig, openldap and java-headless. ugh.
<adamw> sure, i want a java package in my base image so i can get a certificate.
gursewak has joined #fedora-coreos
<dustymabe> yeah
<dustymabe> when I was setting all this up I was trying not to boil the ocean and just get it done
<dustymabe> definitely some of that that could probably be removed by opening PRs to the internal RPMs
<adamw> boiling oceans is my specialty!
<walters> adamw: I used rpm2cpio | (cd / && cpio -div) - it just writes files in `/etc`, yeah I don't get updates this way but it rarely changes
crobinso has quit [Remote host closed the connection]
<travier[m]> I only install the config files as you don't need the rest
<adamw> hmm, maybe i'll go with that.
<adamw> if i do `rpm-ostree uninstall --all` does it revert the overlay perfectly? i'll be as 'clean' as before i did it?
<walters> You can see yourself by comparing the commit hashes
<walters> but yes, it should
<dustymabe> `rpm-ostree db diff` is your friend
<dustymabe> may also need to pair that with `rpm-ostree override reset` to get back to fully clean
<adamw> alright. thanks.
bgilbert has joined #fedora-coreos
flokli has quit [Quit: WeeChat 3.6]
flokli has joined #fedora-coreos
<jlebon> dustymabe: sure
jpn has joined #fedora-coreos
jpn has quit [Ping timeout: 260 seconds]
<dustymabe> jlebon: i'm here
<jlebon> dustymabe: i think we need to reach out to dptp to figure out how to make sure the reverse mirroring is working
<jlebon> i'd like to delete the cosa bc, but it may currently be load-bearing
<jlebon> ok how about
<jlebon> we delete the 4.6 bc, rebuild the 4.6 cosa image, and wait and see if it gets synced
<jlebon> internally at least, we're pointing to quay, and we don't have any e.g. openshift/os 4.6 tests
<jlebon> nor coreos/coreos-assembler tests
<dustymabe> ok
<jlebon> 4.6* tests
<jlebon> ok, i'll delete it. will let you trigger the 4.6
<jlebon> done
<jlebon> +1
<dustymabe> this reminds me that we still need to go back and look at that git webhook plugin again
<dustymabe> so much to do
<jlebon> yeah, indeed :( might be a good spike for someone else
<dustymabe> [2022-08-26T19:10:49.437Z] + ostree container import --repo=/var/srv/upgrade/repo-bare --write-ref fedora/x86_64/coreos/branched ostree-unverified-image:oci-archive:fedora-coreos-37.20220826.92.0-ostree.x86_64.ociarchive:latest
<dustymabe> [2022-08-26T19:10:49.437Z] error: Importing ostree-unverified-image:oci-archive:fedora-coreos-37.20220826.92.0-ostree.x86_64.ociarchive:latest: Expected 1 layer, found 51
killefiz[m] has joined #fedora-coreos
<dustymabe> [2022-08-26T19:10:49.437Z] --- SKIP: fcos.upgrade.basic
<dustymabe> related to your OCI chunked change?
<walters> likely another case of this sidecar container needing an update?
<dustymabe> sidecar container?
<walters> oh sorry no
<walters> hmm upgrade test is using a really old baseline `[2022-08-26T19:09:20.770Z] Downloading image from https://builds.coreos.fedoraproject.org/prod/streams/branched/builds/36.20220324.92.0/x86_64/fedora-coreos-36.20220324.92.0-qemu.x86_64.qcow2.xz
<walters> `
<dustymabe> yeah, it's our branched stream.. which we only turn on for a month or so after fedora branches off rawhide for the next major
<dustymabe> but keep in mind this test has passed this week already
<dustymabe> ^^ that was from earlier today.. the upgrade test passed
<dustymabe> since then I added denylist entries for the tests that failed and respun and here we are.. upgrade test passed before. fails now
<dustymabe> which is why I pinged you :)
<dustymabe> is the problem that f36 has an rpm-ostree that's too old
<dustymabe> if that's the case I can waive the upgrade test on this one run and we won't hit it again in the future
<walters> the build linked above is from march
<dustymabe> right - that's when we disabled branched because we no longer needed it
<walters> it's not about the current rpm-ostree, it's about whatever's in the build chosen as the upgrade test source
<walters> let's skip this upgrade test anyways
<walters> I see we're finding it in `getParentFcosBuildBase` but I am not sure which path we're taking yet
<dustymabe> yeah that's what I was asking when I said "is the problem that f36 has an rpm-ostree that's too old" -> I meand to say is the problem that the rpm-ostree in that old f36 build"
<dustymabe> is too old
<walters> yeah
<dustymabe> ok cool
<dustymabe> now I'm wondering - do we need a barrier for this.. probably not since oscontainers are relatively new
<walters> no, because we haven't cut over to using containers for production on-the-wire updates
<walters> it's just these synthetic update tests that are fetching the target builds this way
<walters> To elaborate slightly for general interest; cutting over the disk images we build to fetch containers by default involves flipping on the code in https://github.com/coreos/coreos-assembler/pull/2523
arnulfo_7 has joined #fedora-coreos
arnulfo_7 has quit [Changing host]
arnulfo_7 has joined #fedora-coreos
<dustymabe> jlebon: the 4.6 cosa build finished
<jlebon> dustymabe: ok cool. so if mirroring works, we should see https://console-openshift-console.apps.ci.l2s4.p1.openshiftapps.com/k8s/ns/coreos/imagestreamtags/coreos-assembler%3A4.6/history get updated within the hour I think
<jlebon> but actually, looking at the last update there, it dates back from June 22, which is definitely before the last time we built 4.6...
<jlebon> hmm, but also before we reversed mirroring. so let's just wait and see.
<jlebon> then at least we can point at it when reaching out to dptp
<jlebon> dustymabe: ughh yeah, we renamed the switch in coreos-installer, but of course that requires a release. let me revert that bit to using the old switch.
jpn has joined #fedora-coreos
<jlebon> dustymabe: it's really great btw that we've detected that so fast
<jlebon> good example right there of why it does provide value to build s390x FCOS
<dustymabe> you mean rather than it filtering all the way down into the s390x rhcos pipeline
<dustymabe> yeah
<jlebon> yeah
<jlebon> ideally we'd be able to selectively test it in PRs too
jpn has quit [Ping timeout: 268 seconds]
<jlebon> but this is the next best thing I guess :)
<dustymabe> jlebon: one more stamp on https://github.com/coreos/fedora-coreos-config/pull/1935
gursewak has quit [Remote host closed the connection]
gursewak has joined #fedora-coreos
vgoyal has quit [Quit: Leaving]
nalind has quit [Quit: until next time]
<adamw> huh. so if i enter a toolbox in gnome-terminal, any new tab or window i open is also inside the toolbox
<adamw> is that...normal?
<adamw> oh, hmm, it seems the new tab/window inherits from whatever tab/window i was in when i opened it. if i open a new tab from a tab that's in the toolbox, i get another toolbox tab. if i open a new tab from a tab that isn't in the toolbox, i get another non-toolbox tab. funky.
<adamw> and to leave the toolbox, i have to run exit from the "original" toolbox tab. exit from a "cloned" toolbox tab just closes the tab. i guess that makes sense in a way? just...hmm.
<jlebon> i think that's by design yeah. see https://github.com/containers/toolbox/issues/218 which wants to undo that :)
<adamw> i guess i was subconsciously expecting it to work like sshing into somewhere, or python virtualenvs
<jlebon> i guess this doesn't really bother tmux users like me :)
<adamw> yeah, that looks hairy. i think i'll just get used to it. :P
<adamw> never managed to get into the habit of using tmux...habitually
<jlebon> if you want to go back to the host context btw, there's `flatpak-spawn --host`, but recently i've started using https://github.com/1player/host-spawn
<adamw> i think i'll just have a bunch of tabs. that's what i usually have anyway. :P
<adamw> tabs solve everything! best ui idea ever!
<jlebon> hehe
<adamw> oof, i can't run 'gedit foobar' from a terminal when gedit's a flatpak? that's going to take some getting used to
<jlebon> `flatpak-spawn --host flatpak run org.gnome.gedit ...` should work
<adamw> `alias gedit=/var/lib/flatpak/exports/bin/org.gnome.gedit`
<adamw> that worked
<adamw> is it doing something awful?
<jlebon> oh that's neat, hadn't realized /var/lib/flatpak was bindmounted in
<jlebon> it looks like it's just shell script wrappers that run `flatpak run`, so i'd say it's by design :)
<adamw> ya
<adamw> y
<adamw> i found out about that directory somewhere while i was setting up autorun links...
<jlebon> and i guess that's why `flatpak run ...` also Just Works
<adamw> ah, that doesn't work from within the toolbox
<adamw> your thing does. so i guess i'll use that
<adamw> ugh. can't find an option that works both in the container and not in it.
c4rt0 has quit [Ping timeout: 260 seconds]
<adamw> okay, bashrc conditionals to the rescue...
<adamw> oof, does the toolbox not respect things in /etc/profile.d ?
<adamw> oh, it just has its own, duh.
<adamw> that can be a bit awkward with a shared /home though. case in point, it just ate most of my bash history.
<adamw> because i set HISTSIZE in /etc/profile.d...
<adamw> how does anyone live without a two-year bash history anyway?
mheon has quit [Ping timeout: 268 seconds]
arnulfo_07 has joined #fedora-coreos
arnulfo_7 has quit [Ping timeout: 248 seconds]
Betal has quit [Quit: WeeChat 3.6]
Betal has joined #fedora-coreos
arnulfo_7 has joined #fedora-coreos
arnulfo_7 has quit [Changing host]
arnulfo_7 has joined #fedora-coreos
arnulfo_07 has quit [Ping timeout: 268 seconds]
agners has quit [Quit: WeeChat 3.6]
jpn has joined #fedora-coreos
arnulfo_07 has joined #fedora-coreos
arnulfo_07 has quit [Read error: Connection reset by peer]
arnulfo_7 has quit [Ping timeout: 260 seconds]
arnulfo_07 has joined #fedora-coreos
jpn has quit [Ping timeout: 260 seconds]
arnulfo_07 has quit [Ping timeout: 252 seconds]
arnulfo_7 has joined #fedora-coreos
arnulfo_7 has quit [Ping timeout: 252 seconds]