<dustymabe>
if you want to apply pressure tell your people to call my people
<dustymabe>
jlebon: did you happen to see the email I sent you a few days back about failures in the ostree pruner? that summary error might affect us when we run the prod builds release job today
<jlebon>
dustymabe: saw it but haven't dug into it yet
<travier[m]>
I had a quick chat with baude today about it
<dustymabe>
travier[m]: oh nice.. any specific outcomes from that chat?
<jlebon>
dustymabe: if you try updating the summary manually again, do you get the same error?
<baude>
dustymabe, equipping me with some answers, i might be able to contribute the ignition support.
<baude>
dustymabe, i'm familiarizing myself with hyperv rn and hoping to be in a better position soon
<dustymabe>
jlebon: if you tell me to try it I'll try it :) - I didn't want to do anything without some expert guidance
<dustymabe>
baude: you're amazing :) - i'd lean on bgilbert and spresti here for expertise on the Ignition side
<baude>
k
<dustymabe>
unfortunately Ignition isn't the only piece in the puzzle to getting full FCOS support, but it is probably the biggest piece of the puzzle
<baude>
yeah, disk format is another one
<baude>
did you have others in mind ?
<dustymabe>
baude: mostly getting the disk image created (i.e. an additional artifact uploaded), and things like afterburn support (if needed), etc.
<dustymabe>
is it possible there was some race condition ther?
<dustymabe>
since obviously I had just deleted a ref in the code and then we tried to update the summary?
<jlebon>
hmm, they're separate invocations though. unless it's some weird NFS behaviour
<dustymabe>
we can have nirik check one of the snapshot backups to see what ref that commit was associated with?
<dustymabe>
if that would be useful
<jlebon>
hmm, could aliases be at play here?
<jlebon>
we `refs --delete` a ref, but i wonder what happens if that ref was aliased
<dustymabe>
maybe - I don't have the logs from that run any longer :( so I can't see what all refs were deleted
<jlebon>
ouch. checking code
<dustymabe>
they might be in the logging platform somewhere - cverna, do you know if logs for old containers are kept anywhere in grafana for the fedora openshift?
<jlebon>
it does delete aliases too. but the inverse could happen, where we delete an alias but the ref remains intact
<dustymabe>
is that necessarily a problem, though?
<jlebon>
so the pruner could delete an alias thinking it removed the reference to that commit, but the ref is still there
<jlebon>
i think the pruner should just ignore aliases. i.e. removed from the configuration list
<jlebon>
ostree will do the right thing
<dustymabe>
jlebon: we can change the configuration to do that, but what real bug is there? will it delete the alias and the commit, but the other ref is still around?
<jlebon>
dustymabe: yeah, that's what i'm thinking. logs would help though :)j
<dustymabe>
yeah they would.. - ok that's interesting. I wonder how the "bad state" that was shown with that ostree summary -u command failure was resolved then?
<dustymabe>
is there some sort of "cleaning up" that ostree will do automatically?
<jlebon>
did the pruner just continue after hitting that issue?
<jlebon>
if so, once it iterated over the real ref, it would've cleaned it up
<dustymabe>
no - it dumped out after the failure
<dustymabe>
IIRC
<cverna>
dustymabe: I don't know, sorry
<jlebon>
dustymabe: in that case, i'm not sure. but any rerun of the pruner would've cleaned it up.
<jlebon>
i do think though we should skip over aliases
<dustymabe>
cverna: we should be unblocked on prod releases now (robosignatory queue is back to normal), you can just kick off new builds (with FORCE=1)
<dustymabe>
jlebon: yeah, ok let's skip over aliases, but maybe rather than change the config we just change the code to say "xyz" is an alias, skipping
<dustymabe>
that way we don't have to know which one is an alias and which one is the real thing