cfbolz changed the topic of #pypy to: #pypy PyPy, the flexible snake https://pypy.org | IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end and https://libera.irclog.whitequark.org/pypy | hacking on TLS is fun, way more fun than arguing over petty shit, turns out
lritter has quit [Ping timeout: 256 seconds]
lritter has joined #pypy
lritter has quit [Quit: Leaving]
Joannah has joined #pypy
Joannah has quit [Quit: Ping timeout (120 seconds)]
Joannah has joined #pypy
Joannah has quit [Quit: Ping timeout (120 seconds)]
Joannah has joined #pypy
Joannah has quit [Quit: Ping timeout (120 seconds)]
Joannah has joined #pypy
Joannah has quit [Quit: Ping timeout (120 seconds)]
andythenorth has joined #pypy
<andythenorth> morning (EU TZ)
<andythenorth> picking up a chat from mailing list...
<andythenorth> I am using pypy3 on m1 mac; I'm not a programmer but I use pypy to compile game mods
<andythenorth> I've compiled on m1 with x86, and it's roughly same performance as on intel i9 mac, some things faster, some slower
<andythenorth> I can contribute funds directly towards an m1 for a pypy3 dev, not the whole price of a mac mini or macbook air
<andythenorth> but I spend hundreds of hours per year of hobby time on these mods, and saving 4 or 5 seconds on compiling really improves cycle time
<andythenorth> I could easily put in 1/3 of the cost of a mac mini at UK prices
<andythenorth> and it would be nice to support a very valuable open source project
<cfbolz> andythenorth: hey :-)
<andythenorth> hi
<cfbolz> we have a mac mini available and ordered a laptop
<andythenorth> excellent
<andythenorth> need any funds?
<cfbolz> a small donation here is appreciated: https://opencollective.com/pypy
<andythenorth> happy to do that
<cfbolz> but something else you could do, that would actually be extremely valuable: wanna write a blog post?
<cfbolz> just write a bit how you're using pypy
<andythenorth> now you ask the hard questions :D
<cfbolz> hehe
<cfbolz> andythenorth: it's not urgent or anything, just think about it :-)
<andythenorth> I will put a couple of lines of background here, to give me some ideas
<cfbolz> please!
<cfbolz> (we're really interested anyway ;-) )
<andythenorth> I'm a contrib to OpenTTD, which, not talking ourselves up too much, is pretty much the most successful open source game https://www.openttd.org/
<cfbolz> awesome!
<andythenorth> the main mods are compiled to an archaic bytecode format, which we (not me) wrote an intermediate python compiler for
<andythenorth> it historically was dog slow, especially under python 2
<andythenorth> it has got faster and faster under python 3, and has caching between runs, and a small amount of c module I believe
<cfbolz> right
<andythenorth> but on intel mac at least, pypy3 was typically twice as fast in many cases
<andythenorth> cutting cycle times from e.g. 40s to 20s, which is a big deal when drawing tiny pixel changes and testing them in game
<andythenorth> I also do a bunch of automated graphics processing using PIL / Pillow which is also typically faster under pypy3
<cfbolz> oh nice! I had no clue that Pillow benefits from pypy
<andythenorth> although parallelising that via multiprocessing pool made the biggest difference
<cfbolz> makes sense
<andythenorth> I think the pillow improvement is subjective to workload, I am not good at profiling though :)
<andythenorth> I just run 'time make'
<andythenorth> and try different pythons
<cfbolz> that's a good approach ;-)
<andythenorth> I actually code generate the mod language from pure python and abuse of Chameleon (zope/pyramid) templates
<andythenorth> Chameleon is not intended for that, but I know how to use it and it works
<andythenorth> interestingly, last time I tested, Chameleon was notably slower under pypy3
<andythenorth> it pre-compiles templates to pyc, not sure of the details, but maybe that approach is in conflict with JIT (I am out of my depth)
<cfbolz> andythenorth: ah, interesting! did you try with the recent pypy 3.8 beta release?
* andythenorth looks
<andythenorth> meanwhile repos are https://github.com/andythenorth/iron-horse/ and https://github.com/andythenorth/firs if anybody ever wanted to test
<andythenorth> there's no build tool, deps are listed in the docs somewhere though
<cfbolz> we made a change in the recent 3.8 there that has a pretty decent chance to improve chameleon for complicated templates
<andythenorth> oh that's interesting
<andythenorth> is it a known case then?
<cfbolz> andythenorth: not chameleon specifically
<cfbolz> but templating engines that generate code in general
<cfbolz> not really known, we found out this year
<andythenorth> looks like I have `[PyPy 7.3.7 with GCC Apple LLVM 13.0.0 (clang-1300.0.29.3)] on darwin`
<andythenorth> can't remember if I compiled it or installed the binary
<andythenorth> oh you have nightlies ok
<cfbolz> yeah, you could try a 3.8 nightly
<cfbolz> andythenorth: anyway, this is all quite fascinating :-)
* andythenorth just trying to install the beta safely without trashing my env and paths :D
<andythenorth> I will compare times
<andythenorth> hmm something is broken with Pillow when I pip install to the 3.8 beta :)
<andythenorth> I'm not great at resolving that kind of issue, but it's not urgent
<cfbolz> andythenorth: something deep in your setup, I assume?
<andythenorth> probably :)
<andythenorth> I seem to have broken my previous pypy3 venv, but I'll figure it out somehow
<cfbolz> :-)
<cfbolz> anyway, if you want to, this is clearly very cool, we should just dump it into a post. The numbers don't even matter too much
<andythenorth> yup I will hang out in this channel for a bit and do a post at some point :)
<cfbolz> cool
<andythenorth> timings for 3.10 official binary with arm, vs pypy3 x86 running in rosetta2
<andythenorth> ../../pypy3/bin/nmlc -l generated/lang --verbosity=4 generated/iron-horse.nm 11.01s user 1.02s system 98% cpu 12.212 total
<andythenorth> nmlc -l generated/lang --verbosity=4 --nfo=generated/iron-horse.nfo 23.92s user 0.83s system 99% cpu 24.889 total
<andythenorth> even on transpiled x86, that's twice as fast
<LarstiQ> cool
<andythenorth> this is super helpful when just tweaking a few pixels etc
<andythenorth> this is the m1 pro, 10 core thing
<andythenorth> obvs only loads one core
<cfbolz> right
<andythenorth> same python 310 / pypy3, but for the chameleon part of the pipeline
<andythenorth> python src/render_nml.py 1.93s user 0.12s system 94% cpu 2.162 total
<andythenorth> python src/render_nml.py 12.12s user 0.50s system 98% cpu 12.839 total
<andythenorth> this is a case where pypy3 is unusually slower
<andythenorth> I did also try the latest nightly, about the same
<andythenorth> I don't trust the results though, because chameleon builds these bytecode pre-compiled templates
<andythenorth> nah same result after I clear out those
<andythenorth> job is here if anybody likes investigating weird cases :D https://github.com/andythenorth/iron-horse/blob/master/src/render_nml.py
<andythenorth> deps are chameleon and markdown, both installed fine from pip for me
<cfbolz> :-(
<cfbolz> pity
<andythenorth> well I tend to look at the wins :)
<andythenorth> I just call different pythons from different parts of the makefile
<andythenorth> whichever is fastest for the job
<cfbolz> andythenorth: made an issue: https://foss.heptapod.net/pypy/pypy/-/issues/3597
<andythenorth> thanks :)
<andythenorth> I'm doing something chameleon isn't supposed to do :)
<cfbolz> andythenorth: hah, "supposed"
<andythenorth> I haven't asked chameleon devs, but I think generating 600 files from templates in 3s is probably beyond the intended use :)
<andythenorth> in website use, the requests don't usually come that fast, and would have varnish or something in front of them I guess
greedom has joined #pypy
slav0nic has joined #pypy
greedom has quit [Remote host closed the connection]
greedom has joined #pypy
jacob22 has quit [Ping timeout: 265 seconds]
greedom has quit [Remote host closed the connection]
otisolsen70 has quit [Quit: Leaving]
olliemath has joined #pypy
<krono> fascinating read that tells me to pick up openttd again
<andythenorth> ha good to hear :)
olliemath has quit [Quit: Client closed]
<cfbolz> I just made 3.x based pypy-cs smaller by a bit more than 1MB
greedom has joined #pypy
jacob22 has joined #pypy
<mattip> cool
Joannah has joined #pypy
Joannah has quit [Client Quit]
greedom has quit [Remote host closed the connection]
greedom has joined #pypy
greedom has quit []
greedom has joined #pypy
<cfbolz> nice, 5 new 3.9 files pass
greedom has quit []
andythenorth has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
slav0nic has quit [Ping timeout: 240 seconds]