cfbolz changed the topic of #pypy to: #pypy PyPy, the flexible snake | IRC logs: and | hacking on TLS is fun, way more fun than arguing over petty shit, turns out
[Arfrever] has joined #pypy
<Corbin> mattip, cfbolz: I added a JIT to my weekend project. It was interpreting 60x faster than the original CHICKEN Scheme compiled code already, but now the JIT has improved it by another 20x.
<Corbin> That's 1200x faster in total than on Friday, not 80x. Also, the segmented traces seem to work well; I just had this summary:
<Corbin> One note: The segmented traces seem to only work if I override the default trace length limit. I picked 9001 arbitrarily, but anything should do.
<Corbin> ...Huh. Okay, that's strange. The segmented traces seem to only kick in sometimes? It depends on the program I choose and the trace length limit.
[Arfrever] has quit [Quit: leaving]
[Arfrever] has joined #pypy
glyph has quit [Quit: End of line.]
glyph has joined #pypy
jstoker has quit [*.net *.split]
jstoker has joined #pypy
<mattip> Corbin: there are experimental changes to the JIT in the py3.8 branch
<cfbolz> Corbin: sure, they are directly tied to the trace limit
<LarstiQ> Corbin: I for one enjoying reading the rubber-ducking, so at least on my account no need for apologies :)
slav0nic has joined #pypy
<amazigh> +1
wleslie has joined #pypy
dustinm has quit [Quit: Leaving]
dustinm has joined #pypy
Atque has joined #pypy
Olliemath has joined #pypy
<cfbolz> yayyy, seems I can run the stdlib tests now
<cfbolz> on 3.9
<mattip> whohoo
<mattip> a summary of a meeting of Sam Gross and Cpython core devs at the recent sprint
wleslie has quit [Quit: ~~~ Crash in JIT!]
nimaje has quit [Ping timeout: 245 seconds]
<cfbolz> mattip: extension modules again
<cfbolz> If we had rpython 3 we could try to parallelize it ๐Ÿ˜‰
<mattip> well, there is pushback on using non-python2 syntax in rpython ...
fotis has joined #pypy
fotis has quit [Ping timeout: 264 seconds]
<Hodgestar> I'm glad Sam has been invited to join the core development team.
<Hodgestar> I'm a bit sad that there wasn't more "this is awesome, let's make it happen" in the summary (hopefully there was some of that at the meeting itself).
<Olliemath> I'm super excited about it
<Olliemath> But others are quite scared about the amount of bad multithreading code that only works because of the gil - and the more I think about it, the more it seems a legitimate fear
<Hodgestar> Are you specifically worried about C extension code?
Olliemath has quit [Quit: Client closed]
Olliemath has joined #pypy
Gustavo6046 has quit [Read error: Connection reset by peer]
<Olliemath> Definitely some C extensions
<Olliemath> But also any code that does more than one operation on a data structure between IO calls and expects to be the only one accessing that data structure at the time
Atque has quit [Quit: ...]
<Hodgestar> So pure Python code that is already broken but now breaks more easily?
Atque has joined #pypy
Atque has quit [Quit: ...]
<Olliemath> Pure python code that relies on the guarantees the gil makes - broken is a strong term
<Olliemath> For example this code will always work under cpython
<Olliemath> but breaks under nogil
danchr_ is now known as danchr
<Hodgestar> Python could already switch threads between lines 13 and 15.
<Hodgestar> Having dealt with threads a lot for a couple of years, my practical experience is that I would rather such code broke straight away, than having to spend days figuring out what is going wrong when it starts failing once a day when used for real.
<Hodgestar> So I would argue nogil is actually better for such cases in real life.
<Olliemath> It could - I'm not saying it's good code - but practically it doesn't (you can try running it and see)
<Olliemath> IIRC that code always works because each thread only surrenders the gil every 100 bytecode operations
* Hodgestar nods.
<Hodgestar> I agree such cases exist, but they'll exist no matter how one removes the GIL, right?
<Hodgestar> So it's a question of "do you want to remove the GIL or not".
Atque has joined #pypy
<Olliemath> Exactly - or can you have (like they were saying) some sort of "gil decorator" (I think the answer sam gave is no)
<Hodgestar> I also agree that much more code is correct without threads, but I also don't think I am likely to write software for a CPU with a single core using CPython in the foreseeable future.
<fijal> olliemath: "practically it doesn't" is not a thing you can say about threading code, ever
<Hodgestar> Probably there would be a use for many more kinds of locks if the GIL was removed, yes.
<Corbin> I worry that Sam's design effectively allows multiple threads to concurrently access a single global shared mutable heap. This might be fast but it is quite unsafe.
<Olliemath> Haha, it's not something I'd rely on in my codebase
<Olliemath> But, given the current gil implementation, it's code that will be out there in production, maybe even in libraries
<fijal> yes sure
<fijal> I think this is a completely unreasonable demand to uphold though
<fijal> because even if you change some internal detail (like not rely on 100 bytecodes), it would break
<Corbin> "It all depends on how well the community adapts C extensions so they donโ€™t cause downright crashes of the interpreter." This effort might be doomed if no packages adapt; if only numpy/scipy adapts, then it'll work but be crashy for years.
<Hodgestar> olliemath: Tx for the explicit example, btw.
<Corbin> Is it alright to be upset at the core dev note that worries about breaking "dark matter" proprietary code? This seems like yet another unpaid-labor situation; because corporations are making money on Python and they might upgrade their Python installations without telling us, we must not do anything to anger or disturb them~
<Olliemath> fijal yeah, I think I just empathise with their caution, given that they're contemplating subtly breaking lots of their users' code
<fijal> I don't know, seems like alternatives are a bit worse
<fijal> and that said, I think python *is* moving too fast and not enough effort is put into thinking about changes (say how they interact with each other)
<Corbin> It is endlessly funny to me that the alternative mallocs are just FAANG pet projects. One's from Microsoft, one's from Google, and one's from Facebook. Meanwhile, Ravenbrook MPS has from-the-ground-up thread-safety and a BSD 2-clause license; they're not evaluating the possible dependencies well.
fotis has joined #pypy
fotis has quit [Ping timeout: 260 seconds]
<LarstiQ> Corbin: hmm, I see it more as "our users are not obliged to tell us how they're using our code, we don't know what we're breaking". Same holds for all the little scripts anyone might write but sees no point in sharing
<Corbin> LarstiQ: I guess. But in that case, it was quite foolish to include the entire API of C within the Python language; that's a lot of stuff to maintain!
<ctismer> Hi mattip: does the latest 7.3.6 version already have a change for `PyType_Modified`, or when do you expect that?
<ctismer> (mattip I meant 7.3.7)
<mattip> ctismer: no, sorry. Maybe the next release
<ctismer> mattip: ok, good to know, so I will check in what I have now.
<mattip> I pushed the 7.3.7 tags
<ctismer> mattip: Did I miss something? `__del__` is not called, even not on a normal Python class
<mgorny> mattip: have you considered removing/hiding/crossing out pypy3.7 7.3.6 for the time being?
<mgorny> i.e. making somewhat clear that people should not be upgrading
<mattip> mgorny: I am going to publish 7.3.7 in a few hours, if my last checks work
<mattip> ctismer: can you show an example?
<ctismer> mattip: yes...
<ctismer> mattip: Is that a case where some timing issue is relevant, maybe gc?
<mattip> you need to do "import gc;gc.collect()" on pypy to get the gc to call __del__
<ctismer> mattip: OK!!! Soooory, works now with `gc.collect()`
<mattip> cool
fotis has joined #pypy
nimaje has joined #pypy
Julian has joined #pypy
<mgorny> mattip: ok, cool, thanks
<mgorny> are there any changes for pypy2.7 or 3.8?
fotis has quit [Ping timeout: 260 seconds]
<mattip> in this case the complete changelog and the blog post are the same
Gustavo6046 has joined #pypy
<mgorny> ok, cool
Dejan has quit [Quit: Leaving]
nimaje has quit [Ping timeout: 245 seconds]
nimaje has joined #pypy
<mattip> cfbolz: could you tweet about the release? The tarballs are up, the blog post is merged
<mattip> sorry for the mess
<cfbolz> mattip: thanks for your prompt fixes
<cfbolz> mattip: is the blog post up?
<cfbolz> I still see the old release announcement
marky1991 has joined #pypy
marky1991 has quit [Remote host closed the connection]
<mattip> it must be CDN caching issues, even a browser refresh is not showing the newer page
<cfbolz> :-(
<mgorny> in, is this change intentional?
<mgorny> + p.add_argument('-b', '--branch', default='v7.3.6',
<mgorny> - p.add_argument('-b', '--branch', default=host.externals_branch,
<mgorny> ( no clue what it does, it just looks suspicious
<mgorny> )
<mattip> yes. On HEAD we use libffi-8.dll, on the releases libffi-7.dll. In order to differentiate, I made a tag to the externals subrepo
<mattip> I should have made the branch v7.3.7, but it was before we were settled on 7.3.7
<mattip> that is windows only
<mattip> thanks for checking carefully
<mgorny> np
<cfbolz> mattip: I tweeted too
<mattip> thanks
Gustavo6046 has quit [Quit: ZNC 1.8.2 -]
fotis has joined #pypy
fotis has quit [Ping timeout: 260 seconds]
Olliemath has quit [Quit: Client closed]
Atque has quit [Quit: ...]
Julian has quit [Ping timeout: 260 seconds]
Julian has joined #pypy
Julian has quit [Client Quit]
Julian has joined #pypy
fotis has joined #pypy
fotis has quit [Ping timeout: 265 seconds]
Julian has quit [Ping timeout: 264 seconds]
saltrocklamp[m] has quit [Changing host]
saltrocklamp[m] has joined #pypy
<Corbin> Is there a neat way to store multiple entrypoints in a single module? Or should I just make one module per entrypoint and factor everything?
<Corbin> ...Well, okay, when I put it that way, I should probably just factor it. Guess I'm just lazy.
Julian has joined #pypy
Julian has quit [Quit: leaving]
slav0nic has quit [Ping timeout: 264 seconds]