06:43 <arigato> ctismer: hi! I'm debugging a test using stackless, so it's good that you're here :-)

06:44 <arigato> somebody is having stacklet objects and calling their .run() method repeatedly. Is that an expected use case? In PyPy it leaks because it keeps adding the same tasklet object in the internal list _squeue, where it ends up several times

06:48 <arigato> never mind, reading the docs now

06:48 <arigato> seems like a pypy bug

06:52 <ctismer> arigato: Hi Armin, nice to meet you. The stackless.run method is usually started from the main taslet and serves as event loop, as I remember. For sure, run is not meant to be called more than once. In CPython stackless, I think it would be even forbidden.

06:53 <ctismer> arigato: But I actually have to look it up, did not care for ten years.

06:55 <ctismer> If PyPy allows more than one call of run, I guess this should be fixed, yes :-)

06:56 <arigato> nice to meet you again, indeed!

06:56 <arigato> no, I'm talking about tasklet.run() to run a specific tasklet

06:56 <arigato> but I've found https://stackless.readthedocs.io/en/3.7-slp/library/stackless/tasklets.html#the-tasklet-class

06:58 <arigato> and if you're interested, https://foss.heptapod.net/pypy/pypy/-/issues/3556 is the issue, where the stackless issue shows up even without cffi embedding

07:07 <ctismer> arigato: ah, sorry. Still in bed. Will read that. I think that stuff comes from the time where I had not implemented channels. The anonymous usage of tasklets via channels is much more elegant. Tasklet.run often used shows that someone should rewrite/think his code. (whow, people still use stachless)

07:08 <arigato> ah, I see. Yes, maybe we can poke the author of that issue in a different direction

07:09 <arigato> I think he wants embedding of Python into his C++ and he also wants the chunks of Python to be runnable concurrently (but cooperatively) to each other

07:10 <arigato> so more microthreads than real stackless-with-channels style

07:12 <arigato> unsure what to suggest to him---maybe greenlets? or something else like using "yield"?

07:15 <ctismer> Ah! And since it‘s embedding in C++, it is most probably not cooperative multitasking. Yes, I guess greenlets do it better. And PyPy switches no stacks at all, right? So I‘d expect very many frames piling up.

07:17 <arigato> there's no soft-switching at all in pypy, if that's the question

07:20 <ctismer> Yes, and if that is called from C++, how does he get back into the caller? He must return to the call site, or use hard switching. Does this work in PyPy?

07:22 <arigato> yes, it should, by always using hard-switching

07:24 <arigato> so the whole call stack should be correct at all points

07:24 <arigato> at least for the currently running tasklet

07:27 <ctismer> And PyPy uses the greenlet switching in principle? Well, imoressive.

07:31 <ctismer> Hmm. Now I lost the reason why that should not work. Has the guy tried his code with Stackless itself before using PyPy? (waking up…)

07:39 <arigato> I don't think so

07:40 <arigato> according to the documentation, though, it should not leak

07:55 <ctismer> As I remember, Stackless has the circular list, and tasklet.run rotates the list only, so the tasklet comes next. If it‘s inserted multiple times in PyPy, that may be wrong.

07:55 <arigato> yes

07:56 <arigato> also it seems that tasklet.run(), tasklet.kill(), and other related methods have a complexity that is linear in the number of tasklets

07:56 <arigato> at least that's the case in pypy

07:57 <arigato> maybe Stackless Python was more clever and used a doubly-linked list which can be updated in constant time?

07:57 <arigato> pypy just uses a deque()

07:58 <arigato> and searches inside it with operator.indexOf()

07:59 <ctismer> Stackless used a double linked circle, methinks.

08:00 <arigato> OK, thanks. I don't think anyone really used stackless.py from pypy for anything serious, otherwise these bogus complexities would show up

08:01 <ctismer> The tasklets themselves have fore and back pointers. There was no list involved at all.

08:01 <arigato> makes sense

08:02 <ctismer> Yes, this is a rare case 🙂

08:08 <ctismer> Actually, these fore and back pointers also built the channels. I carefully tried to make all operations as fast as possible.

08:14 <ctismer> Probably PyPy mostly kept the code the way as Stephan Diehl had written it. As CPython grows all its async stuff, there is probably no point in pushing PyPy.Stackless any further. (although I still think they should not build async on top of generators)

08:24 <arigato> yes, I think so too

08:41 <ctismer> cfbolz: Yes, that was very interesting! I always had the impression that handling code by function is a somewhat arbitrary decision, but there could be other criteria as well to separate code chunks. (admittedly, it would be hard to invent new names for them)

08:51 <ctismer> cfbolz: (talking about your blog post about huge functions)

10:27 <cfbolz> ctismer: you are correct

10:27 <cfbolz> That's a simplification of the blog post

10:28 <cfbolz> It's indeed not a function that I mark as huge

10:28 <cfbolz> But a 'tracing entry point'

10:29 <ctismer> cfbolz: ah, ok

11:21 <ctismer> ronan: Back to the weirdness, there is sth. wrong in the type display.

11:22 <ctismer> This time not running it, since we know it crashes. https://www.irccloud.com/pastebin/nPefgoSV/pypy_crash.py

11:24 <ctismer> https://www.irccloud.com/pastebin/FtXHzOUb/result

11:28 <ctismer> ronan: In short: Both PyPy and Python have a distinction between the `__new__` functions. But PyPy claims that the type is always the same! That must be a lie, and I'm working on that with lldb on lldebug.

12:00 <ctismer> ronan: the code never reaches pypy_g_PyType_FromSpecWithBases

17:06 <ctismer> ronan: mattip: I found it! `PyObject_SetAttr` has in PyPy the side effect to call `PyType_Ready`. It still breaks, but at least the right `__new__` gets called 😜🥂