#fedora-coreos on 2022-05-18 — irc logs at libera.irclog.whitequark.org

2022-05-11 12:42 dustymabe changed the topic of #fedora-coreos to: Fedora CoreOS :: Find out more at https://getfedora.org/coreos/ :: Logs at https://libera.irclog.whitequark.org/fedora-coreos

00:40 plarsen has quit [Remote host closed the connection]

00:50 cyberpear has joined #fedora-coreos

01:46 bgilbert has joined #fedora-coreos

01:55 bgilbert has quit [Ping timeout: 240 seconds]

02:39 gursewak_ has joined #fedora-coreos

03:11 gursewak_ has quit [Ping timeout: 246 seconds]

04:27 paragan has joined #fedora-coreos

04:39 bgilbert has joined #fedora-coreos

04:55 bgilbert has quit [Ping timeout: 240 seconds]

05:06 Betal has quit [Quit: WeeChat 3.5]

05:58 zodbot_ has joined #fedora-coreos

06:00 djinni`_ has quit [Ping timeout: 246 seconds]

06:01 djinni` has joined #fedora-coreos

06:01 zodbot has quit [Ping timeout: 246 seconds]

06:02 zodbot_ has quit [Excess Flood]

06:03 zodbot has joined #fedora-coreos

06:22 gursewak_ has joined #fedora-coreos

06:35 nemric has quit [Quit: WeeChat 3.5]

07:01 hyperreal has quit [Quit: am leaving for now]

07:02 hyperreal has joined #fedora-coreos

07:05 jcajka has joined #fedora-coreos

08:45 jpn has joined #fedora-coreos

09:35 guesswhat has joined #fedora-coreos

09:58 klaas has quit [Quit: ZNC 1.8.2 - https://znc.in]

09:59 klaas has joined #fedora-coreos

10:40 jcajka has quit [Quit: Leaving]

10:41 bgilbert has joined #fedora-coreos

10:43 bgilbert has quit [Remote host closed the connection]

10:43 bgilbert has joined #fedora-coreos

10:48 paragan has quit [Ping timeout: 260 seconds]

10:49 paragan has joined #fedora-coreos

10:59 crobinso has joined #fedora-coreos

11:30 ravanelli has joined #fedora-coreos

11:40 jcajka has joined #fedora-coreos

11:42 <bgilbert> dustymabe: it turns out we have Ignition 0.29.1-3 in epel7, presumably for ignition-validate. do you think we should update it to current, or retire it?

12:02 <travier[m]> I would vote for retiring

12:10 bgilbert has quit [Ping timeout: 240 seconds]

12:14 jpn has quit [Ping timeout: 260 seconds]

12:22 <dustymabe> bgilbert: I don't have a strong opinion. There is a container for ignition-validate so retire seems reasonable

12:40 jpn has joined #fedora-coreos

12:45 jpn has quit [Ping timeout: 244 seconds]

12:46 nalind has joined #fedora-coreos

13:10 <stephanepoq> curl "https://updates.coreos.fedoraproject.org/v1/graph?stream=stable&platform=metal&basearch=x86_64& <-- is it down?

13:11 mheon has joined #fedora-coreos

13:12 jpn has joined #fedora-coreos

13:13 <lucab> stephanepoq: I hope not, but let me check

13:20 jpn has quit [Ping timeout: 240 seconds]

13:28 plarsen has joined #fedora-coreos

13:32 nemric has joined #fedora-coreos

13:35 jpn has joined #fedora-coreos

13:37 <stephanepoq> @lucab it's up again

13:38 <lucab> stephanepoq: yes, I tried reverting to the version right before https://github.com/coreos/fedora-coreos-cincinnati/pull/65 and it seems to be behave better

13:39 <lucab> the strange thing is that the same service on the staging cluster was working fine, I'm puzzled

13:40 jpn has quit [Ping timeout: 272 seconds]

13:44 <dustymabe> lucab: what problems were you seeing with f36? any logs?

13:45 <lucab> dustymabe: not really, and from the metrics I could see the requests coming in. But the reverse proxy in front of it would never serve the response, eventually timing out.

13:46 <dustymabe> lucab: you think the service hasn't been functioning properly ever since the move to f36? some days?

13:46 <lucab> dustymabe: I tried justrestarting the backend, but it didn't help. Reverting to the previous image though seems to work.

13:48 jpn has joined #fedora-coreos

13:48 <lucab> dustymabe: since yesterday when I deployed the update, yes. I can see lots of timeouts logged by Zincati on my canaries. A few requests were getting through, but most were not

13:49 <dustymabe> super odd. it would seem like the problem should be on the proxy side, but obviously if reverting the image makes the problem go away then it's something in the image

13:50 <lucab> I'll try going through the chain of F<xx> image bumps before F36 and see where it starts going wrong

13:54 <lucab> https://fcos-31-metrics-cdn.lucabruno.net/bridge?selector=zincati -> ~300 timeouts, which started yesterday with the F36 deployment and stopped now with the revert

13:59 <StevenPresti[m]> dustymabe: Hey

14:00 <dustymabe> StevenPresti[m]: 👋

14:00 <dustymabe> StevenPresti[m]: when you build Ignition can you build it for F37 too?

14:00 <dustymabe> https://koji.fedoraproject.org/koji/packageinfo?packageID=27185

14:01 <dustymabe> our rawhide pipeline is now failing because we have tests that need the new Ignition but no new Ignition in rawhide

14:03 <StevenPresti[m]> dustymabe: here is where our progress is on the process, https://github.com/coreos/ignition/issues/1353 are you saying to go back Once the PR merges to rawhide, merge rawhide into the other relevant branches (e.g. f35) then push those, for example:... (full message at https://libera.ems.host/_matrix/media/r0/download/libera.chat/f4bc62db4b23e9b0898aa2362e2287d6f3d28782)

14:04 <StevenPresti[m]> dustymabe: OH, fedpkg build for rawhide..? I thought that branch was auto?

14:05 <dustymabe> yeah, just do a `fedpkg build` on the rawhide branch

14:05 <dustymabe> I guess in your steps you could update it to

14:06 <dustymabe> "On each of those branches, including `rawhide`, run `fedpkg build`"

14:06 <StevenPresti[m]> yeah that would make it more clear :/ I will do that.

14:07 <dustymabe> +1 - thank you

14:08 <dustymabe> StevenPresti[m]: once you do that f37 build can you add an override for it to the f-c-c repo (for the rawhide branch)?

14:09 <StevenPresti[m]> dustymabe: not sure what 'add an override for it to the f-c-c ' is

14:10 <dustymabe> StevenPresti[m]: I can step you through that

14:10 <StevenPresti[m]> dustymabe: f-c-c stand for?

14:10 <StevenPresti[m]> kk

14:10 <dustymabe> f-c-c == https://github.com/coreos/fedora-coreos-config/

14:12 <StevenPresti[m]> Ah thats because rawhide was not included in the fast-tracking huh?

14:15 <StevenPresti[m]> Just for understanding; Could I not just redo these steps

14:15 <StevenPresti[m]> submit them to bodhi and then Submit a fast-track for FCOS testing-devel

14:16 StevenPresti[m] is now known as spresti[m]

14:32 <spresti[m]> dustymabe: Sorry I guess I miss understood, I am now overriding

14:32 <spresti[m]> https://github.com/coreos/fedora-coreos-config/actions/runs/2346197816

14:39 <dustymabe> spresti[m]: for rawhide builds a bodhi update gets created automatically when the build is done

14:40 <dustymabe> https://bodhi.fedoraproject.org/updates/FEDORA-2022-891e0e1d4e

14:40 <dustymabe> you'll need to use that link ^^ for the fast-track

14:53 <spresti[m]> dustymabe: oh, ok thank you, I was sitting at the bohdi page like , where is my build...

14:54 <spresti[m]> dustymabe: the override is still failing on the distro tag. It says its expecting f37 but has f36?

14:57 <dustymabe> spresti[m]: let's investigate that

14:57 <spresti[m]> https://github.com/coreos/fedora-coreos-config/runs/6491597738?check_suite_focus=true

14:57 <dustymabe> rawhide is now based on f37 (not f36 as it was in the past), so we might need to update a variable somewhere

14:58 <dustymabe> when you ran the override did you select `rawhide` twice ?

15:00 <dustymabe> from the logs I see "Switched to a new branch 'testing-devel'"

15:00 <spresti[m]> No, just once. is rawhide special?

15:00 fifofonix has joined #fedora-coreos

15:01 <spresti[m]> In the release-doc it says to using testing-devel

15:01 <dustymabe> when I run an override I usually make sure I select the branch I'm targetting in all dropdowns that have it as an option

15:01 <dustymabe> honestly I'm not really sure there should be more than one option

15:02 jlebon has joined #fedora-coreos

15:02 <dustymabe> `Submit a fast-track for FCOS testing-devel`

15:02 <dustymabe> ^^ I would select `testing-devel` in both dropdowns for that

15:03 <spresti[m]> Yeah I didn't change rawhide there

15:03 <spresti[m]> its all testing-devel

15:04 <spresti[m]> I miss spoke, the only place I used rawhide was for the build it's self

15:05 <dustymabe> spresti[m]: let's do a screenshare if you have time.. see PM

15:08 <spresti[m]> dustymabe: sure one second.

16:01 jcajka has quit [Quit: Leaving]

16:14 paragan has quit [Quit: Leaving]

16:21 jpn has quit [Ping timeout: 260 seconds]

16:24 gursewak_ is now known as gursewak

16:24 jpn has joined #fedora-coreos

16:29 <dustymabe> aaradhak davdunc dustymabe gurssing jaimelm jbrooks jcajka jdoss jlebon jmarrero lorbus miabbott nasirhm ravanelli saqali skunkerk walters

16:29 <dustymabe> FCOS community meeting in #fedora-meeting-1

16:29 <dustymabe> If you don't want to be pinged remove your name from this file: https://github.com/coreos/fedora-coreos-tracker/blob/main/meeting-people.txt

16:32 jpn has quit [Quit: Lost terminal]

16:34 <dustymabe> jaimelm: meeting today?

16:34 mnguyen_ has joined #fedora-coreos

16:35 aaradhak has joined #fedora-coreos

17:29 guesswhat has quit [Quit: Client closed]

18:22 Betal has joined #fedora-coreos

18:24 jlebon has quit [Quit: leaving]

18:44 nemric has quit [Quit: WeeChat 3.5]

18:57 guesswhat has joined #fedora-coreos

19:04 aaradhak has quit [Quit: Connection closed for inactivity]

20:00 nemric has joined #fedora-coreos

20:16 zodbot has quit [Ping timeout: 272 seconds]

20:22 poppajarv4 has joined #fedora-coreos

20:22 fifofonix has quit [Quit: Textual IRC Client: www.textualapp.com]

20:24 poppajarv has quit [Ping timeout: 272 seconds]

20:24 poppajarv4 is now known as poppajarv

20:49 ravanelli has quit [Remote host closed the connection]

20:49 ilius has joined #fedora-coreos

20:54 nemric has quit [Quit: WeeChat 3.5]

21:04 zodbot has joined #fedora-coreos

21:16 zodbot has quit [Remote host closed the connection]

21:17 zodbot has joined #fedora-coreos

21:28 crobinso has quit [Remote host closed the connection]

21:30 zodbot has quit [Remote host closed the connection]

21:30 zodbot has joined #fedora-coreos

21:46 zodbot has quit [Remote host closed the connection]

21:49 zodbot has joined #fedora-coreos

21:51 ravanelli has joined #fedora-coreos

21:58 nalind has quit [Quit: bye]

22:09 <anthr76[m]> Not sure if this room is best suited or if there's another one. On Fedora desktop with ostree I swapped my display manager. I went from gdm to greetd. `rpm-ostree` failed to update my shadow removing the `gdm` user and adding the `greetd` user. Is this expected?

22:29 mnguyen has joined #fedora-coreos

22:29 mheon has quit [Ping timeout: 276 seconds]

22:30 mnguyen_ has quit [Ping timeout: 240 seconds]

22:32 guesswhat has quit [Quit: Client closed]

22:40 nemric has joined #fedora-coreos