Atque has quit [Remote host closed the connection]
Atque has joined #pypy
Atque has quit [Ping timeout: 240 seconds]
<arigato>
in link with issue #2889, maybe I should mention that we could still try to embark on the larger project to change the way tuples are rtyped, from GcStruct to a new XxxStruct
<arigato>
which would be exploded in its individual fields in C functions, and with new argument and return value passing conventions, and the largest part is to handle storing it in other data structures (then it becomes a regular non-GC Struct I guess)
<arigato>
the goal being to stop once and for all to have these issues that occur because a function is returning a tuple
<arigato>
or a tuple is passed to a residual call and this makes the GC pressure increase
Atque has joined #pypy
<arigato>
(note that "exploded in C functions" is what I would have said 10 years ago, but maybe nowadays the C compilers are all good enough to have the same result)
Guest96 has joined #pypy
Guest96 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Guest96 has joined #pypy
Atque has quit [Ping timeout: 240 seconds]
Atque has joined #pypy
otisolsen70 has joined #pypy
<cfbolz>
arigato: I wonder whether we really need a new type
<cfbolz>
Or whether it should be done on a best effort basis in the backend opts
<cfbolz>
So we would 'just' need the concept of a return pointer argument
<arigato>
right
<arigato>
maybe expressed as a regular Ptr(Struct), and we change the signature of functions to pass the pointer to the struct that we want to be filled; this would require only a single new llop, "alloca", which can be implemented without actually calling alloca() but same idea
<cfbolz>
We used to have an alloca 😅
<arigato>
yes
<cfbolz>
arigato: does not sound too terrible. I'm unsure about the effects on the GC transformer
<cfbolz>
And don't quite know how it would work in the JIT
<arigato>
why the GC transformer?
<cfbolz>
arigato: does it affect the roots?
<arigato>
yes, the JIT is some work too. maybe that would be a good reason for exploding the fields into local variables, to make the JIT's work easier
<arigato>
and ah, yes, the roots
<arigato>
another reason to explode the fields, then the roots are regularly handled
<cfbolz>
Right
<fijal>
hm
<cfbolz>
arigato: so you would basically only have a type that is conceptually a box that you can store a return value into
<cfbolz>
Which turns into a pointer to a local variable in C calls
<arigato>
the C call itself would be "allocate the box, call the function, read the fields out of the box, and forget the box immediately", so that we're sure there are no GC roots lurking in there?
<cfbolz>
Yes. Using alloca?
<arigato>
in C, just using a local variable of type struct
<cfbolz>
Ah of course
<cfbolz>
But what about the JIT? Where does it make space? The CPU stack explicitly?
<arigato>
yes
<arigato>
meh
<cfbolz>
Needs a bit of care
<arigato>
it would be easier if we had just a lloperation that does it all including the call
<fijal>
llop.call_with_multiple_return_values?
<arigato>
maybe we can keep the existing FuncType declared as Void return and with an extra first argument that is a pointer to the resulting struct,
<arigato>
but indeed have an optional llop.call_with_multiple_return_values() when we call this kind of FuncType
<cfbolz>
It does not fit the flow graph model so well
<arigato>
the jit code writer can check that call_with_multiple_etc is immediately followed by extract_return_from_box, and consolidate it all---I think the jit opcodes can have multiple return values?
<arigato>
maybe not?
<cfbolz>
arigato: no, they can't I think
<cfbolz>
It's a bit of a mess, and the question is a bit where we move that mess
<arigato>
yeah
<cfbolz>
It would add a bit of an implicit structure to how operations around a call must appear
<arigato>
yes, not good
<cfbolz>
We have a few elements of that already in some places already
<cfbolz>
Eg the guard_no_exception after a call in the JIT
<arigato>
in C it's translated as func(&v0, &v1, arg0, arg1, arg2...)
<arigato>
the JIT would produce a residual call func(&v0, &v1, arg0, arg1, arg2...) where v0 and v1 are in the jitframe
<arigato>
so the call_with_multiple_result "spills" v0 and v1 in the sense that after the call, the values are living in the jitframe, and can be reloaded from there if needed later
<cfbolz>
arigato: right. Still a dependency
<cfbolz>
But at least not operations before *and* after the call
<arigato>
only a weak one. It breaks the idea of immutability. the result_of_later_call() are just placeholders that can occur earlier too, as long as you don't use v0 and v1 before the actual call is made
<cfbolz>
Yes
<arigato>
(I think it doesn't need to have any argument, the index 0 or 1 is pointless)
<cfbolz>
And the type of v* is just the lltype
<cfbolz>
Nothing special
<arigato>
yes
<cfbolz>
It can even be chained in theory
<cfbolz>
If the result comes from a recursive call
<arigato>
the gc transformer "should" not worry about gc roots between the result_of_later_call and the actual call_with_multiple_xxx
<arigato>
but result_of_later_call can initialize the variable with null to be on the safe side
<cfbolz>
Yes, was just about to say
<arigato>
chaining: no, I have in the mind that the FuncType should start with types like lltype.Ptr(lltype.Signed)
<arigato>
I think it needs to be lltype.Ptr(lltype.Struct(lltype.Signed)) or something like that
<arigato>
then the return from that function writes each result to its independent location inside a struct-of-one-field
<cfbolz>
It's an array of length 1 I suspect
<arigato>
ah yes
<cfbolz>
Ok I can see how this works with primitive types
<cfbolz>
Does it also work for the GC references?
<arigato>
it should
<cfbolz>
Who keeps them alive?
<arigato>
er, maybe?
<arigato>
tempted to answer "nobody but it doesn't matter"
<cfbolz>
But then we have a dependency again
<cfbolz>
You really have to 'write and immediately return'
<cfbolz>
In the called function
<arigato>
...yes
<cfbolz>
Hmmm, 'clearly': we should not use a pointer to a local variable in C, but a pointer into the shadow stack
<arigato>
oh
<cfbolz>
In the JIT, that's what the plan with the JIT frame already does
<cfbolz>
Really obscure though
<arigato>
the shadowstack never moves around, right?
<cfbolz>
No
<arigato>
then maaaybe?
<cfbolz>
:-)
<cfbolz>
We should put this elaborate plan into an issue and then simply not implement it
<arigato>
fwiw, we already have an operation 'gc_restore_root' which takes an argument that in C is actually an output argument
<arigato>
this llop occurs just after a call, after the graph is gc-transformed
<arigato>
so the right thing should occur if we just genc the operation call_with_multiple(func, gcres0, ..) by passing the address in the shadowstack
<arigato>
because it's re-read from there immediately afterwards
<arigato>
OK, it's a plan that is elaborate and won't get implemented
jacob22 has quit [Ping timeout: 248 seconds]
Guest96_ has joined #pypy
Guest96 has quit [Ping timeout: 244 seconds]
Atque has quit [Quit: ...]
<fijal>
haha
<mattip>
we have lost our macOS buildbot, which was supported by the CPython buildbot worker owner,
<mattip>
he was having problems with python2 last I heard
* fijal
struggles to parse cpython buildbot worker owner
<mattip>
the person behind the macOS buildbot for CPython
Atque has joined #pypy
<mattip>
I guess I can try to run the buildbot worker on my M1 machine in x86_64 mode
otisolsen70_ has joined #pypy
otisolsen70 has quit [Ping timeout: 240 seconds]
otisolsen70__ has joined #pypy
<fijal>
this would take a while....
<fijal>
mattip: did we not have an offer to use some mac somewhere?