jaeger changed the topic of #crux-arm to: CRUX-ARM 3.6 Released! - http://crux-arm.nu/Documentation/ReleaseNotes3-6 | Logs: https://libera.irclog.whitequark.org/crux-arm/
frinnst has joined #crux-arm
guido_rokepo has joined #crux-arm
<pitillo> I'll start another testing this afternoon when my wife finishes work
<pitillo> clean clone and optimization for arm64 to reproduce coreutils problem (autoconf error at stage1)
<pitillo> interesting find related to http-repgen
guido_rokepo has quit [Quit: guido_rokepo]
guido_rokepo has joined #crux-arm
guido_rokepo has quit [Client Quit]
<pitillo> started make bootstrap here
<pitillo> changed directly in the Makefile the optimizacion (from arm to arm64) instead of using a variable
<jaeger> ok
<pitillo> it's amazing M1 performance.... I can keep looking at the compilations.... reaaaally fast
<jaeger> yeah :)
<pitillo> let's see if sepen has some spare time this night (at 11:30pm here) to make a meeting and share the status. I want to check autoconf problem because it will impact any build probably, but I'm not sure why we haven't noticed until now
<jaeger> I'm very curious about why as well :)
<pitillo> I don't remember if I manually touched ports.stage1 to add coreutils to the first position,.. I doubt, but it's possible. Btw, it's a patch and not a solution
<pitillo> let's see if we can focus all efforts on the generic releases, by polishing crux-arm-release. I hope I can build another generic arm64 RC with good prt-get.conf and pkgmk.conf forgotten files (thanks to beerman for remember me about)
<jaeger> Sounds good
<pitillo> python3 atm... so not many more ahead to finish stage0
<pitillo> started downloading sources before entering st1
<pitillo> =======> Building '/workspace/ports/core/autoconf/autoconf#2.71-1.pkg.tar.xz' succeeded.
<pitillo> package autoconf is installed
<pitillo> prt-get: updating /workspace/ports/core/autoconf
<pitillo> =======> Package '/workspace/ports/core/autoconf/autoconf#2.71-1.pkg.tar.xz' is up to date.
<pitillo> prt-get: reinstalling autoconf 2.71-1
<pitillo> -- Packages updated
<pitillo> autoconf
<pitillo> jaeger: can you please do another test?
<pitillo> but making one change
<pitillo> these are the commands I've used to make this fresh build:
<pitillo> git clone git@github.com:crux-arm/crux-arm-release.git
<pitillo> git checkout 3.7
<pitillo> (sorry, cd crux-arm-release before the checkou)
<pitillo> vim Makefile
<pitillo> changed arm by arm64 directly on the Makefile
<pitillo> make boostrap
<jaeger> Sure, I can try it
<jaeger> To be clear you just want me to change arm to arm64, no other changes?
<jaeger> I was using DEVICE_OPTIMIZATION=raspberrypi4 before, for reference
<pitillo> no no
<pitillo> in your cae, change arm by raspberrypi4
<pitillo> s/cae/case
<pitillo> force the optimization inside the Makefile... this is somwthing sepen tried to avoid, but I've got strange behavior when I use the variable (or the argument/option to make)
<pitillo> so a good test could be change it directly on the Makefile and verify if you get the same behavior with autoconf or if the process ends as expected
<jaeger> ok, testing that now
<jaeger> Just started so will be a bit
<pitillo> not much... around 1 hour and a bit more than half hour
<pitillo> taking in mind that gcc took 56h in the rpi3.... not much xD
<jaeger> :D
<jaeger> OK, no surprise, smae libcrypto.so.3 failure with coreutils
<jaeger> s/smae/same/
<pitillo> :(
<pitillo> I don't understand
<pitillo> is coreutils overlayed somewhere? core-arm or rpi4-arm64?
<pitillo> give me some rime to movw home and I'll check the host ports versions (glibc, gcc ssl and readline) where I am compiling to see if there are differences wirh your host (container in both cases)
<pitillo> it's very strange... I had the feeling that it will go right this time... but I was wrong
<pitillo> I can test some things at home.... as I have a clean sstage arm64 build atm, I'll finish st1 (compiling atm) and I'll use that st0 to build the rpi4 opti
<pitillo> but I believe there is something different in our containers
<pitillo> just finished here
<pitillo> =======> Building '/workspace/ports/core/iproute2/iproute2#6.0.0-1.pkg.tar.xz' succeeded.
<pitillo> [2022-12-07 20:44:28] Bootstrap completed
<pitillo> I'll check at 11:30pm (I must move home and put my wife to bed... and hope the baby sleeps like a groundhog xD)
<jaeger> (build) 796555f6b0af ~/crux-arm-release # find ports -name "coreutils"
<jaeger> ports/core/coreutils
<pitillo> good
<pitillo> jaeger: can you check these ports hosts versions?
<jaeger> bash-5.1# prt-get listinst -v --regex 'glibc|gcc|coreutils|readline|openssl'
<jaeger> gcc 12.2.0-1
<jaeger> coreutils 9.1-1
<jaeger> glibc 2.36-1
<jaeger> openssl 3.0.5-1
<jaeger> readline 8.2.1-1
<jaeger> The host is 3.7
<pitillo> ok, the problem must be there then
<pitillo> $ crux
<pitillo> CRUX-ARM 64b version 3.6
<jaeger> Does your installed coreutils NOT link against openssl?
<pitillo> did you start with 3.7? I thought you started with a 3.6 container
<pitillo> let me check
<jaeger> I started with a 3.6 and upgraded it to 3.7 via toolchain rebuild, etc.
<pitillo> nop
<pitillo> pitillo@e5c69808c01f:~/new_test/crux-arm-release (3.7)
<pitillo> $ ldd /usr/bin/sort linux-vdso.so.1 (0x0000ffff8b8cd000) libpthread.so.0 => /lib/libpthread.so.0 (0x0000ffff8b841000) libc.so.6 => /lib/libc.so.6 (0x0000ffff8b6cd000) /lib/ld-linux-aarch64.so.1 (0x0000ffff8b89c000)
<jaeger> Well, that's definitely a difference
<jaeger> coreutils in 3.7 (non-arm) also links against openssl, it's not a fluke in the arm container
<pitillo> and only upgraded readline (if I'm right)
<jaeger> So while crux-arm-release may work with 3.6 it will break with 3.7 due to that
<pitillo> I think it's sll 3.0.x
<pitillo> sll/ssl
<pitillo> yeah
<jaeger> also, openssl is in core, so in my opinion should be in the build system anyway
<pitillo> we found a side efdect (I don't remember exactly which one)
<jaeger> I would pretty much expect everything in raspberrypi4, core-arm64, and core (overlayed in that order) to be in a stage for the rpi4, for example
<pitillo> yes, that's the objective, but for stage1
<jaeger> Why not for stage1? It seems like you would avoid a lot of headache by NOT trying to short-circuit it like this
<pitillo> do you mean for stage0?
<jaeger> I mean for all bootstrap stages
<pitillo> we currently have 2 stages
<pitillo> stage0: minimal set of core ports, the group of minimal tools for building ports. This collection is trimed down (or cut down, no sure if this expression is right) because rhey are linked with the host
<jaeger> Yeah, I understand that
<pitillo> to keep it minimal, we do the trick of download sources before the chroot on the roofs generated with this minimal collection
<pitillo> because curl links with host's ssl for example
<jaeger> Maybe it would be simpler to just ensure the host has minimal ports installed, building in a container makes this easy
<jaeger> Right, but on what system are you NOT going to have openssl? :)
<pitillo> we are currently building with a minimal ports collection. We are using only core ports
<pitillo> we can start a good discussion at this point
<jaeger> That includes openssl
<pitillo> xD
<jaeger> Yeah, all good. I'm not trying to start a fight :)
<pitillo> and what about crux 3.7 being 4.0? ssl breakage from 1.1.x to 3.0.x
<pitillo> no no, I meant a good discussion not a fight
<pitillo> jajajajaja
<jaeger> As a point of reference, my build container has only core-arm64/core/raspberrypi4 ports installed with only *1* exception which is prt-utils for revdep
<pitillo> it's fine that base, prt-utils won't affect
<jaeger> exactly
<pitillo> I feel the problem is this case when we move to another ABI
<pitillo> it's a bir hard to explain in english what I have at mind
<pitillo> s/at/in
<jaeger> Yeah, ABI breakage is annoying, but it WILL happen
<pitillo> in this case from 3.6 to 3.7...
<pitillo> as you said, where is not openssl installed? it is in both version but with an ABI break
<jaeger> I had to deal with it during the 3.6 to 3.7 rebuild
<jaeger> But only once :)
<pitillo> how to attack this problem? we expect that people will use 3.6 as host to build an optimized 3.7 release or do we expect to use a 3.7 host?
<pitillo> yeah! we have reviewed all your work (yours and the community, but mostly yours)
<jaeger> Well, I guess that's up to you guys... I just assumed 3.7 when I started
<jaeger> Maybe that was a bad assumption :D
<jaeger> I understand that a 3.7 release has to come from somewhere... but my guess was a generic 3.7 would be used to build all optimized 3.7 releases
<pitillo> think that for the next release... 3.8... we'll use the current version to build the newer one
<jaeger> If the intent was to build all of them from 3.6, well, now I know and I can switch to a 3.6 container
<pitillo> yeah... but let's see: "a generic 3.7 would be used to build all optimized 3.7 releases"
<jaeger> I don't have a good (tm) reason for my assumption other than that's how I build new releases for x86
<pitillo> that sentence is right, but we are unserstanding it wrong
<pitillo> you understand that the host will be 3.7 and that's not thw point
<jaeger> In an ideal situation the host should not matter... 3.6 or 3.7, as long as it's crux... but then ABI breakage gets in the way
<pitillo> the point is to get the generic rootfs and use it as stage0 rootfs to chroot into it and build the optimized relesse
<jaeger> I guess a way to summarize is this: I didn't realize that your intent was to use a 3.6 host (or container) to build 3.7
<pitillo> yeah! true, the abi break will be there (it's here btw between 3.6 and 3.7, and we managed in that way)
<pitillo> yeah
<jaeger> I don't have a problem with switching to a 3.6 container to avoid that ABI breakage, if that's how you intend it to work
<pitillo> yeah, but the second part is important too
<pitillo> to use the generic release 3.7 on a 3.6 host to buiñd a 3.7 optimized elease
<pitillo> think in the next release....
<pitillo> we'll be running 3.7... so we start to build 3.8 (using a 3.7 host because rhere is product rele
<pitillo> leased)
<pitillo> fuck, sorry
<pitillo> using a 3.7 host because there isn't a product released)
<pitillo> so we build a generic 3.8 rootfs
<pitillo> now, at that point... someone is running 3.7 and wants to create an opti release for his device
<pitillo> hebhas the possibility to get 3.8 generic
<pitillo> but not deploy to his device....
<pitillo> he can get the generic 3.8 rootfs and put it in crux-arm-release
<pitillo> ln -s crux-arm-3.8.tar.xz stage0-rootfs.tar.xz
<pitillo> and then prepare overlays
<jaeger> ok
<pitillo> and build the opti with nake stage1 for example
<jaeger> I'm going to prepare a 3.6 container and start again to see how it goes
<pitillo> I'm not sure if I'm explaining it right
<jaeger> Yeah, you're explaining it fine, it's just that my assumption was wrong, you guys work differently
<jaeger> Now I know :)
<pitillo> you need to update readline, only that one is needed (I don't remember why, but it broke something)
<jaeger> ok, will keep an eye out for it
<pitillo> yeah, we need to don't waste compilation time (now with the M1 it's another history, but we still need to save resources)
<pitillo> your assumption was wrong may be due my english level too... There is no documentation yet about the process and probably my "guideline" could confuse to take wrong assumptions
<pitillo> but now I can understand some things about crux iso too because you are using a non released 3.7 host to build 3.7 release
<pitillo> prt-get diff|wc -l
<pitillo> 75
<jaeger> Your english is better than many native english speakers :)
<pitillo> this is on the 3.6 container...
<pitillo> jajajaja don't trol at me... I speak with lot of american people and my english level is reeeeeally far
<pitillo> it's a bit near to english people because my pronuntiation sucks
<pitillo> xD
<pitillo> we must make a meeting with voice... we can get some laught... probably you more
<jaeger> No trolling, many native speakers speak it REALLY badly
<jaeger> English is a tough language
<pitillo> $ ls /home/pkgmk/pkg/
<pitillo> bash#5.2.2-1.pkg.tar.gz
<pitillo> ca-certificates#20220719-1.pkg.tar.gz
<pitillo> git#2.37.3-1.pkg.tar.gz
<pitillo> filesystem#3.7-3.pkg.tar.gz
<pitillo> ca-certificates#20221011-1.pkg.tar.gz
<pitillo> openssl#1.1.1q-1.pkg.tar.gz
<pitillo> readline#8.2-1.pkg.tar.gz
<pitillo> strace#5.19-1.pkg.tar.gz
<pitillo> util-linux#2.38.1-1.pkg.tar.gz
<pitillo> wget#1.21.3-1.pkg.tar.gz
<pitillo> these are updated ports on my 3.6 container (ca-certificates, openssl and readline should be the important ones)
<jaeger> ok
<jaeger> Any reason not to build from a fully-updated container?
<pitillo> save time
<jaeger> fair enough
<pitillo> but a fully updated one would be the best scenario
<jaeger> I'm running a sysup in my new 3.6 container, so I was curious :)
<pitillo> this is why we pushed readline on 3.6
<pitillo> good good
<pitillo> I think I should do a snapshot and sysup for testing too
<pitillo> but the ceitical ones must be those ports (readline the most important one)
<jaeger> ok
<pitillo> the sysup mustn't take much time... around the same as the st1
<pitillo> I must go to sleep... the kid won't forgive tomorrow morning
<jaeger> yeah, it shouldn't be too bad
<jaeger> No worries, have a good night :)
<pitillo> I hope you can share results
<jaeger> I'll see how my bootstrap goes from 3.6
<pitillo> I like eating rats before going to sleep... yours came out tonight, so that's nice
<jaeger> heh
<pitillo> have a good day's end there too :)
<jaeger> thanks