<Corbin>
mattip, cfbolz: I added a JIT to my weekend project. It was interpreting 60x faster than the original CHICKEN Scheme compiled code already, but now the JIT has improved it by another 20x.
<Corbin>
That's 1200x faster in total than on Friday, not 80x. Also, the segmented traces seem to work well; I just had this summary: https://bpa.st/KTNA
<Corbin>
One note: The segmented traces seem to only work if I override the default trace length limit. I picked 9001 arbitrarily, but anything should do.
<Corbin>
...Huh. Okay, that's strange. The segmented traces seem to only kick in sometimes? It depends on the program I choose and the trace length limit.
[Arfrever] has quit [Quit: leaving]
[Arfrever] has joined #pypy
glyph has quit [Quit: End of line.]
glyph has joined #pypy
jstoker has quit [*.net *.split]
jstoker has joined #pypy
<mattip>
Corbin: there are experimental changes to the JIT in the py3.8 branch
<cfbolz>
If we had rpython 3 we could try to parallelize it 😉
<mattip>
well, there is pushback on using non-python2 syntax in rpython ...
fotis has joined #pypy
fotis has quit [Ping timeout: 264 seconds]
<Hodgestar>
I'm glad Sam has been invited to join the core development team.
<Hodgestar>
I'm a bit sad that there wasn't more "this is awesome, let's make it happen" in the summary (hopefully there was some of that at the meeting itself).
<Olliemath>
I'm super excited about it
<Olliemath>
But others are quite scared about the amount of bad multithreading code that only works because of the gil - and the more I think about it, the more it seems a legitimate fear
<Hodgestar>
Are you specifically worried about C extension code?
Olliemath has quit [Quit: Client closed]
Olliemath has joined #pypy
Gustavo6046 has quit [Read error: Connection reset by peer]
<Olliemath>
Definitely some C extensions
<Olliemath>
But also any code that does more than one operation on a data structure between IO calls and expects to be the only one accessing that data structure at the time
Atque has quit [Quit: ...]
<Hodgestar>
So pure Python code that is already broken but now breaks more easily?
Atque has joined #pypy
Atque has quit [Quit: ...]
<Olliemath>
Pure python code that relies on the guarantees the gil makes - broken is a strong term
<Olliemath>
For example this code will always work under cpython
<Hodgestar>
Python could already switch threads between lines 13 and 15.
<Hodgestar>
Having dealt with threads a lot for a couple of years, my practical experience is that I would rather such code broke straight away, than having to spend days figuring out what is going wrong when it starts failing once a day when used for real.
<Hodgestar>
So I would argue nogil is actually better for such cases in real life.
<Olliemath>
It could - I'm not saying it's good code - but practically it doesn't (you can try running it and see)
<Olliemath>
IIRC that code always works because each thread only surrenders the gil every 100 bytecode operations
* Hodgestar
nods.
<Hodgestar>
I agree such cases exist, but they'll exist no matter how one removes the GIL, right?
<Hodgestar>
So it's a question of "do you want to remove the GIL or not".
Atque has joined #pypy
<Olliemath>
Exactly - or can you have (like they were saying) some sort of "gil decorator" (I think the answer sam gave is no)
<Hodgestar>
I also agree that much more code is correct without threads, but I also don't think I am likely to write software for a CPU with a single core using CPython in the foreseeable future.
<fijal>
olliemath: "practically it doesn't" is not a thing you can say about threading code, ever
<Hodgestar>
Probably there would be a use for many more kinds of locks if the GIL was removed, yes.
<Corbin>
I worry that Sam's design effectively allows multiple threads to concurrently access a single global shared mutable heap. This might be fast but it is quite unsafe.
<Olliemath>
Haha, it's not something I'd rely on in my codebase
<Olliemath>
But, given the current gil implementation, it's code that will be out there in production, maybe even in libraries
<fijal>
yes sure
<fijal>
I think this is a completely unreasonable demand to uphold though
<fijal>
because even if you change some internal detail (like not rely on 100 bytecodes), it would break
<Corbin>
"It all depends on how well the community adapts C extensions so they don’t cause downright crashes of the interpreter." This effort might be doomed if no packages adapt; if only numpy/scipy adapts, then it'll work but be crashy for years.
<Hodgestar>
olliemath: Tx for the explicit example, btw.
<Corbin>
Is it alright to be upset at the core dev note that worries about breaking "dark matter" proprietary code? This seems like yet another unpaid-labor situation; because corporations are making money on Python and they might upgrade their Python installations without telling us, we must not do anything to anger or disturb them~
<Olliemath>
fijal yeah, I think I just empathise with their caution, given that they're contemplating subtly breaking lots of their users' code
<fijal>
I don't know, seems like alternatives are a bit worse
<fijal>
and that said, I think python *is* moving too fast and not enough effort is put into thinking about changes (say how they interact with each other)
<Corbin>
It is endlessly funny to me that the alternative mallocs are just FAANG pet projects. One's from Microsoft, one's from Google, and one's from Facebook. Meanwhile, Ravenbrook MPS has from-the-ground-up thread-safety and a BSD 2-clause license; they're not evaluating the possible dependencies well.
fotis has joined #pypy
fotis has quit [Ping timeout: 260 seconds]
<LarstiQ>
Corbin: hmm, I see it more as "our users are not obliged to tell us how they're using our code, we don't know what we're breaking". Same holds for all the little scripts anyone might write but sees no point in sharing
<Corbin>
LarstiQ: I guess. But in that case, it was quite foolish to include the entire API of C within the Python language; that's a lot of stuff to maintain!
<ctismer>
Hi mattip: does the latest 7.3.6 version already have a change for `PyType_Modified`, or when do you expect that?
<ctismer>
(mattip I meant 7.3.7)
<mattip>
ctismer: no, sorry. Maybe the next release
<ctismer>
mattip: ok, good to know, so I will check in what I have now.
<mattip>
I pushed the 7.3.7 tags
<ctismer>
mattip: Did I miss something? `__del__` is not called, even not on a normal Python class
<mgorny>
mattip: have you considered removing/hiding/crossing out pypy3.7 7.3.6 for the time being?
<mgorny>
i.e. making somewhat clear that people should not be upgrading
<mattip>
mgorny: I am going to publish 7.3.7 in a few hours, if my last checks work