<antocuni>
I'm not sure to understand the syntax; handles are not malloc()ed, they are indexes into a RPython list. You can open a new handle and this means putting the w_obj into the list (and closing the handle means to remove it from the list)
<fijal>
"creation of handle"
<antocuni>
ok
<fijal>
ok, so the raw data that can be referenced has to survive between h1 = .... and free(h2)
<fijal>
right?
<antocuni>
not completely. The raw data has to survive for the whole lifetime of the object (obviously, because that's where the object lives!), but the ADDRESS of this data can change inside "stuff"
GianlucaRizzo has quit [Ping timeout: 260 seconds]
<fijal>
I think you would need to have reference counting, because it seems like "gc.pin has to succeed" is a requirement
<fijal>
and you would need to completely redesign our GC for the case of having too many pinned objects
<antocuni>
yes, gc.pin has to succeed and yes, we need to improve our GC to do that
<fijal>
I mean really, "redesign"
<fijal>
it's not a simple improvement
<antocuni>
that's not the impression which I got when talking to Armin, but maybe I misunderstood
<cfbolz>
we should have a call
<cfbolz>
this is not working
<antocuni>
yes
<cfbolz>
as a medium
<antocuni>
but also, there is really no hurry
<fijal>
cfbolz: why is it not working?
<antocuni>
this is for improving the performance of hpy on pypy, but it's not the top priority
<antocuni>
currently, there is no need for pinning because the C data is malloced() anyway
ronny has quit [Quit: Bridge terminating on SIGTERM]
jboi has quit [Quit: Bridge terminating on SIGTERM]
jevinskie[m] has quit [Quit: Bridge terminating on SIGTERM]
<antocuni>
the pinning is needed if we decide to gc-allocate the hpy_data inside W_HPyObject, which is something which we cannot do right now anyway
<cfbolz>
fijal: because you and anto are talking past each other
<cfbolz>
antocuni: is it correct that for a purely python defined instance that you pass to C with HPy, everything is very efficient and you don't need pinning?
<fijal>
no, I disagree
<antocuni>
cfbolz: yes
<fijal>
I maintain I perfectly understood what anto says
<fijal>
the difference is "this is a small improvement to our GC" vs "this is a massive undertaking which requires quite careful redesign"
<fijal>
I would not call it "talking past each other"
<antocuni>
fijal: ok, but you are still talking about a hypotetical optimization which I'm not going to do anytime soon anyway :)
<fijal>
yes yes, I'm only interested in the intellectual pursuit of what's feasible too :)
<cfbolz>
antocuni: something I was wondering: in the current (non-optimized) model for C data, you find and move the object pointers in there in the GC how?
<antocuni>
the C address of hpy_data must be fixed only between points 6 and 9, and only of the code calls HPy_AsStruct, and that's where pinning would be necessary
<cfbolz>
6 and 9?
<cfbolz>
I want to know: a C object points to a Python list. how is the C data updated when the list moves?
<antocuni>
cfbolz: not sure if I understand the question. In the current model, W_HPyObject is GC-managed and can mode, while W_HPyObject.hpy_data is malloced and has a fixed address for the whole lifetime
<antocuni>
cfbolz: in that case I use an HPyField
<antocuni>
which on PyPy they are implemented as gcrefs
<antocuni>
and there is a custom GC tracer which calls the user-provided tp_traverse to know where they are
<antocuni>
(that's what I'm doing these days)
<cfbolz>
so every collect calls all these tp_traverse?
<antocuni>
yes
<cfbolz>
what does the api for the traverse C function look like, if the content of a field moves?
<antocuni>
I don't understand the question
<cfbolz>
so an HPyField in some hpy_data can point to a gcref, right?
<cfbolz>
the gcref can move
<antocuni>
yes
<cfbolz>
who updates the hpy_data
<antocuni>
the GC
<cfbolz>
how does it know the offsets of the hpy_fields?
<antocuni>
because it has the custom tracer (which is implemented by calling tp_traverse)