mattip has quit [Ping timeout: 268 seconds]
mattip has joined #hpy
<antocuni> ronan: I'm trying to investigate the ujson+pypy failure that you mentioned, but so far it works for me
<antocuni> I'm using a self-compiled pypy from the py3.7 branch (commit 9c0e0671120f), and a slightly edited version of ujson-hpy modified to work with hpy-0.0.2
<antocuni> I pushed my version of ujson-hpy in the hpy-0.0.2 branch (basically, it's a older version of master, because 0.0.2 still uses HPyContext instead of HPyContext*): https://github.com/hpyproject/ultrajson-hpy/tree/hpy-0.0.2
<antocuni> with that, I could do: pypy3-c setup.py --hpy-abi=universal build_ext -i
<antocuni> and then pypy3-c -m benchmark.main
<antocuni> and the benchmark runs to completion
<antocuni> do you have a way to reproduce the problem you mentioned?
<mattip> antocuni: hijacking the thread to ask if you could show some benchmarks at tomorrow's meeting
<antocuni> benchmarks of what?
<antocuni> mattip: I mean, we don't have any new benchmark around. The only two reasonable benchmarks that we have are still ujson and piconumpy
<ronan> antocuni: python -c "import hpy.universal, os; hpy.universal.load('ujson_hpy', os.path.abspath('ujson_hpy.hpy.so'))"
<antocuni> ronan: and what happens if you run this?
<ronan> it segfaults
<ronan> from PyPyImport_ImportModule()
<antocuni> did you compile ujson_hpy with cpython setup.py build or pypy setup.py build?
<ronan> with pypy
<antocuni> I'm trying to help but it would be easier if you provide a bit more details :)
<antocuni> which pypy revision? Which ujson-hpy revision?
<ronan> The pypy revision is 6aeccdfa7ba4 - a py3.7 nightly from last week
<ronan> and for ujson-hpy, it's hpy-0.0.2
* antocuni tries
<antocuni> ronan: so, if I try to do the same from scratch, I get the following: http://paste.openstack.org/show/807199/
<antocuni> i.e., you can't easily build an hpy module with pypy because it doesn't recognize the setuptools entry points out of the box
<antocuni> which, btw, it's something which I just fixed in the pypy hpy-0.0.2 branch, see commits 1c34dcf8f9b3 and 405dacc22864
<ronan> I think it works if you create a vrtualenv
<antocuni> I think that depending on what you did to allow setup.py to work, you might have ended in using a wrong version of hpy.devel, which would explain the segfault
<antocuni> I can't imagine how it can work in a virtualenv, since until few minutes ago we lacked a hpy.devel.dist-info file in lib_pypy
<ronan> ah, right, I manually put a dist-info or similar in the virtualenv]
<antocuni> ronan: so, the following works for me on bencher7: http://paste.openstack.org/show/807200/
<antocuni> note that I had to manually copy the dist-info from the hpy repo
<antocuni> I suppose that you see the failure on your laptop? Can you try on bencher7?
* antocuni tries on his laptop as well
<ronan> I don't have access to bencher7
<antocuni> arigo: do you have the permission to add ronan to bencher7? ^^^^
<ronan> hmm, the only difference with what I did is that I used a virtual env
<antocuni> could you please check what happens if you use the same commands as me?
<ronan> on it
<antocuni> I confirm that it works also on my laptio
<antocuni> laptop
<ronan> it still segfaults for me with the exact same commands as you
<antocuni> :(
<antocuni> I fear I can't help debugging it if I don't find a way to reproduce :(
<antocuni> the good news is that I managed to run the ujson benchmark on your branch :)
<antocuni> it seems only ~3% slower, which looks like a reasonable slowdown
<antocuni> do you think it is reasonable to keep the current exception logic around for a while (possibly toggled with a config option)? I think it will be interesting to try it again in the future when we have more benchmarks around
<antocuni> by "keep the current logic around" I mean that we should probably use the cpyext-like logic by default, but don't kill the current code yet for a while
* antocuni off
<ronan> note that I can also run the benchmark, with similar results
<ronan> keeping both logics seems tricky