<fangerer>
antocuni: support for 0.0.2 is merged in our current master; the next GraalVM release (scheduled for July 20th) will include the change; nightly builds are already available
<antocuni>
fangerer: cool! So we should really try to release 0.0.2 before July 20th, which should be doable
<antocuni>
ok, getting somewhere with the pypy bug. It seems that at some point, HandleManager.free_list contains the same index twice
<antocuni>
so that two subsequent calls to new() return the same handle, with all kind of funny thing
<Hodgestar>
fangerer: That is great!
mattip has quit [Ping timeout: 258 seconds]
mattip has joined #hpy
mattip has quit [Ping timeout: 258 seconds]
mattip has joined #hpy
<ronan>
antocuni: I don't know if that's the issue but the free_list logic doesn't seem threadsafe
mattip has quit [Ping timeout: 252 seconds]
mattip has joined #hpy
<ronan>
hmmm, self.free_list.append(index) fails in a way that corrupts the list?!
mattip has quit [Ping timeout: 252 seconds]
<ronan>
and that happens because handles.close() is called with an RPython exception raised
mattip has joined #hpy
<ronan>
aaahh, the problem is that you can't call anything RPython, not even HPyTracker_Close or HPy_Close, with an exception raised
<ronan>
but you need them for cleanup...
<antocuni>
ronan: ouch, indeed
<antocuni>
I think you got it
<antocuni>
I also arrived to the conclusion that the free_list got corrupt, but didn't understand yet what caused it
<antocuni>
so, I think that what happens is this:
<antocuni>
1. to raise an HPy exception we raise an RPython exception
<antocuni>
2. while we are still in C, we call HPy_Close, which causes self.free_list.append() in HandleManager
<antocuni>
3. if the free_list doesn't have enough room, it needs to reallocate. But since the RPython exception is set, the malloc think it fails
<antocuni>
4. and the funny thing happens
<antocuni>
we need to find a fix and/or a workaround for this. I fear that mapping HPy exceptions to RPython exceptions is essential for performance
<antocuni>
one possible workaround is to ensure that self.free_list has always as much capacity as self.handles_w, so that self.free_list.append does not causes any reallocation. But it looks more like a hack than a proper fix :(
<antocuni>
arigato: ^^^, in case you have any idea
<ronan>
no, it's dumber than that. At 3. it calls ll_list_resize() which merely increments the length field, but then it checks for an exception and sees the one from 1. so it exits from append() without setting the item
<antocuni>
I see
<antocuni>
but still, my proposed workaround could work
<ronan>
and the result is that self.free_list looks correct but now ends with some random value that was popped previously
<ronan>
no, append() can never work correctly if there is an exception set
<antocuni>
ah yes, I see it now
<antocuni>
I wonder how it's possible that it works smoothly with -Ojit
<ronan>
I'm wondering too
<ronan>
maybe it's broken too but we don't see it
<antocuni>
another possible fix is to have a special RPython construct (maybe a context manager?) to save the possibly existing RPython exception, clear it, run normal RPython code and re-set the original exception back
<antocuni>
although it would slow down every call to HPy_Close(), because you would need to check for the exception always
<ronan>
that doesn't seem very different from doing things like in cpyext
<antocuni>
well no, you would use this special context manager only in the few carefully selected HPy_* functions which you are allowed to call when an exception is set
<antocuni>
we have already decided that you are allowed to call only few of them in that case (and HPy_Close is one of them, obviously)
<antocuni>
the biggest problem of the cpyext-style solution is what happens when you cross the border multiple times: you end up catching/reraising exception multiple times, one for each level
* antocuni
-> zzz
<ronan>
I'm not sure it's such a big issue
<ronan>
running code with an exception set isn't normally on the fast path