cfbolz changed the topic of #pypy to: #pypy PyPy, the flexible snake https://pypy.org | IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end and https://libera.irclog.whitequark.org/pypy | Matti: I made a bit of progress, the tests now only segfault towards the end
LarstiQ has quit [Ping timeout: 246 seconds]
LarstiQ has joined #pypy
<mattip> fijal: find / -name "libexpat*dylib" 2>/dev/null does not give me a usable result
<mattip> it find subdirectories of /Applications/Xcode.app/Contents/Developer/Platforms
<mattip> like AppleTVOS.platform, iPhoneOS.platform, WatchOS.platform
<mattip> digging into the failure, it is trying to use _findLib_gcc_fallback("expat") from rpython/rtyper/lltypesystem/ll2ctypes
<mattip> this calls "gcc -Wl,-t -o /tmp/dummy_name -lexpat" which outputs to stdout a whole bunch of stuff
<mattip> and then parses that to get
<mattip> "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib/libexpat.tbd"
<mattip> but when it runs that command, the command fails, and does not capture stdout, only stderror
<mattip> so the desired line is not caught, the discovery fails, and ll2ctypes cannot load the library
<LarstiQ> does gcc fail or the stdout capturing?
<mattip> fijal: does os.popen("gcc -Wl,-t -o /tmp/dummy_name -lexpat").read() return stdout on your machine?
<mattip> stdout capturing, or maybe gcc somehow sees it is not a terminal and so does not output to stdout?
<mattip> LarstiQ: do you have a macOS machine?
<LarstiQ> mattip: not anymore myself
<mattip> :(
<LarstiQ> I'll ask my colleagues
<mattip> I would be interested if you run "gcc -Wl,-t -o /tmp/dummy_name -lexpat" from the command line
<mattip> and then os.popen("gcc -Wl,-t -o /tmp/dummy_name -lexpat").read()
<mattip> if they print out the same information
<LarstiQ> from any python?
Heston has quit [Remote host closed the connection]
Heston has joined #pypy
<fijal> mattip: hey matti
<fijal> do you have xcode command line tools installed?
<mattip> yup
<mattip> LarstiQ: yes, from any python
<mattip> fijal: do you get that output from os.popen as well?
<fijal> no
<fijal> WAT
<fijal> mattip: how did you find that?
<mattip> that is weird but at least consisten with what I see.
<mattip> somehow your ll2ctypes finds the library to load. So there is something else different between our setups
<mattip> here is my environment, does anything seem different?
<mattip> do you have anything I don't?
<fijal> so expat is found with just -lexpat
<mattip> fwiw, here is what I did after wiping the disk and starting over
<fijal> mattip: have you tried Popen?
<fijal> mattip: what do you need rosetta for?
<mattip> yes, but I think that trail is a false flag, because it works the same for both of us. Something else is different
<mattip> I don't know why/if I need rosetta.
<fijal> I don't think so
<fijal> (I definitely had it not installed)
<mattip> ok. So you don't see anything different between the environment variables?
<fijal> mattip: sorry I just woke up and I'm trying to pack :)
<fijal> how do I get the env variables?
<mattip> set
<fijal> there is a virtualenv mess there
<fijal> mattip: there is extra mess because I migrated my home directory, so it's not a clean install
<fijal> maybe I can try on a different user at some stage?
<fijal> (but not now and maybe not today)
<mattip> thanks
<fijal> mattip: sorry about the mess :/
<mattip> this is helpful:
<fijal> and appreciate your tenacity!
<fijal> but I feel like you are asking me a lot for knowledge I have no clue about!
<mattip> bash not zsh, PATH has other things
<fijal> it is zsh I believe?
<fijal> no, it's bash
<fijal> so PATH has a lot of mess from my old mac, I should really try to get a fresh user to try
<fijal> mattip: can you compile a small program with -lexpat?
<mattip> I will try a bit later
<mattip> I think the problem is ll2ctypes is looking to load the so or dylib or whatever macos uses
<mattip> so it is not a compilation problem rather a ctypes.dlopen one
<fijal> ah
<mattip> this works on linux but returns None on my macmini
<mattip> ctypes.util.find_library('expat')
<mattip> does it return something for you?
<fijal> >>> ctypes.util.find_library('expat')
<fijal> '/usr/lib/libexpat.dylib'
<fijal> yeah
<mattip> right, it does that on python3 on the macmini as well, so something is wrong with the python2.7.18 build via pyenv
<mattip> that narrows it down a lot
<mattip> how did you get a python2 for translation?
<fijal> from homebrew I believe
<fijal> but as soon as I could I started using pypy
<fijal> mattip: this gets me the right directory
<mattip> cool, thanks
<mattip> ctypes changed for macos11 and m1 silicon, the whole search thing changed here
<mattip> we backported that to pypy2.7, but the pyenv 2.7.18 is borked
<LarstiQ> mattip: a colleague reports getting the same failure for both commands
<LarstiQ> fwiw
<mattip> huh. I wonder if popen is broken in my setup
<mattip> LarstiQ: can you ask what hardware, macos version, python they used?
otisolsen70 has joined #pypy
<cfbolz> fijal: also, could you please push the merge to py3.8?
<mattip> I sidestepped the find_library problem for now by using "--translationmodules" to get a minimum pypy
<mattip> but now I get "[translation:ERROR] RuntimeError: ffi_prep_cif_var failed"
<mattip> fijal: did you do something to get a different libffi?
otisolsen70 has quit [Quit: Leaving]
otisolsen70 has joined #pypy
<fijal> mattip: I think you kinda want to compile with system libffi?
<fijal> cfbolz: I was scared :) but I can
<fijal> I don't know how I got system libffi, but how did you get a python 2 in the first place? did you use the apple provided one?
<LarstiQ> mattip: intel macs 11.6.5 and 12.3.1, pythons 3.9.13, 3.10.2 and 2.7.18 "ld: symbol(s) not found for architecture x86_64" Might be an issue of not finding expat at all
<fijal> cfbolz: right, we did not decide on OSX_MIN_VERSION
<mattip> LarstiQ: I think we found the problem with expat: CPython has fixes for ctypes that are not backported to CPython2.7.18
<LarstiQ> mattip: aaah
<mattip> LarstiQ: but are backported to PyPy2.7.18+, so if we can get a pypy2 bootstrapped the problem will be gone
<mattip> fijal: I am stuck. The python2 I have from pyenv has a broken ctypes and cannot translate, something about libffi and c_open and the error I showed above
<fijal> yeah I'm pretty sure I did not use pyenv
<mattip> fijal: can you suggest how I can replicate your work? Where can I get a python2 that can translate?
<fijal> mattip: have you tried the apple-supplied one?
<mattip> apple does not supply a python2
<fijal> I'm pretty sure I used that one
<fijal> yes it does?
<fijal> do you have Moneterey 12.0.1 or something else?
<mattip> monterey 12.4
<mattip> what is the path to your python2?
<fijal> mattip: /usr/bin/python
<fijal> can you try to build yourself one --with-system-libffi or something like that?
<mattip> zsh: no such file or directory: /usr/bin/python
<fijal> --using-system-libffi?
<fijal> but there were a few issues when I tried compiling
<fijal> hm
<fijal> mattip: can I give you my pypy binary simply?
<mattip> yes, zip up the whole thing and put it on buildbot.pypy.org?
<fijal> "the whole thing" being?
<fijal> we have release scripts right?
<mattip> stdlib since ctypes is apparently different?
<mattip> pypy/release/package.py should do it
<mattip> it expects pypy-c, libpypy-c.* to be in pypy/goal
<fijal> ok, I'm trying
<fijal> requires libtcl
<fijal> who does require that?
<fijal> mattip: can you just copy pypy-c and libpypy-c.dylib from ~fijal/tmp on baroque software?
<fijal> the rest should be identical
<fijal> and let's see if you can run it
<mattip> ok
<fijal> I don't quite understand why even without tkinter libtcl tries to be there
<fijal> the whole resuable library story is both completely broken AND utterly bonkers by now
<fijal> "let me save 200k of a library file here with tons of fighting" "but let's ship 2G video file for no reason at all"
<LarstiQ> it's not just about saving space of course, also update management. But yeah
<mattip> can you put the binaries on buildbot.pypy.org? I don't have access to baroquesoftware
<fijal> mattip: it's the same host
<fijal> I think?
<mattip> no
<fijal> mattip: ok, it's there in /tmp
<mattip> nope, translation fails
<mattip> I am willing to admit defeat
<fijal> OK, WHERE do you pull the libffi from?
<fijal> you need to use the homebrew supplied one
<fijal> wait wait wait
<fijal> mattip: I can probablu try to fight on your mac mini?
<fijal> Undefined symbols for architecture arm64:
<fijal> "_main", referenced from:
<fijal> implicit entry/start for main executable
<fijal> ld: symbol(s) not found for architecture arm64
<fijal> but that does not look very good
<fijal> is this python 2 or 3?
<fijal> what you are translating
<mattip> % hg id -> a2887f4ecb60+ tip
<mattip> I think that is because I am printing the trace from the code I was messing with earlier
<mattip> this is after reverting my printing of intermediate cruft
<mattip> % hg id -> a2887f4ecb60 tip ( no "+")
<fijal> eh, did we *not* agree to not use ll2ctypes during translation at all at some stage?
<mattip> tell me what to run as a translation command line
<mattip> fijal: can you translate HEAD with the pypy you sent me?
<fijal> mattip: yes
<fijal> mattip: can I debug it on your mac mini? it seems like trying to do it over IRC is a complete waste of time
<mattip> agreed. I have to reset your access since I reformatted the disk
<fijal> the answer is obviously "you bootstrap with already built pypy-c"
<fijal> but also, I'm a bit mad that ll2ctypes is used during translation, that's a bug
<fijal> that was the deal - it's ok to do ll2ctypes, as long as it's used just for testing
<fijal> mattip: but also, I don't think today
<fijal> who requires tcl-tk?
<mattip> I don't have any idea who these comments are aimed at
<mattip> you should have access at fijal@77.137.28.218
<fijal> mattip: "pypy developers" ;-)
<fijal> more specifically, one way to fix it is to not have ll2ctypes used during translation
<fijal> debug1: No more authentication methods to try.
<fijal> fijal@77.137.28.218: Permission denied (publickey).
<mattip> :(
<mattip> try again?
<fijal> nope
<fijal> if you are using the right key, then the errors are in sshd logs
<mattip> nothing in logs. I think I needed "==" at the end of the key and not whatever I did. Try again?
lritter has joined #pypy
<fijal> nope
<mattip> Please resend me the key by mail
<fijal> mattip: I'll try to clone pypy and run it, but I'll drop off at some point today
reneeontheweb has quit [Quit: Client closed]
Atque has quit [Ping timeout: 240 seconds]
Atque_ has joined #pypy
f4at has joined #pypy
Atque_ has quit [Ping timeout: 240 seconds]
Atque has joined #pypy
Atque has quit [Ping timeout: 240 seconds]
Atque has joined #pypy
greedom has joined #pypy
greedom has quit [Remote host closed the connection]
greedom has joined #pypy
otisolsen70 has quit [Quit: Leaving]
greedom has quit [Remote host closed the connection]
lritter has quit [Ping timeout: 244 seconds]
Atque has quit [Quit: ...]