<phlebas>
vstinner: isn't the issue though exactly that: "it doesn't help by itself, just moves it closer". a problem i see (as you write in your post), is that each change has to be carefully checked for performance impact. so doing a lot of little changes always has the danger that you'll run into something where the performance impact will not be acceptable to someone, and then what? did you cause a lot of churn just to fail before reaping any
<phlebas>
benefit?
<phlebas>
i think at the end of the day, if you want to reach your goals, there's no other way besides doing a large scale refactoring of the api to show that it can be done and that it helps and it doesn't cause needless performance or other regressions. then you hope that others agree to apply your huge change in small increments. doing it that way a lot of other projects also failed, but i don't think there is another way
<phlebas>
for example, i'm personally not even convinced that things like tagged pointers would really help so much. e.g. http://export.arxiv.org/pdf/2109.02958 has gained quite good speedup without needing that, simply by quickening bytecode and avoiding repeated boxing/unboxing that way. and that can be done right now in cpython
<phlebas>
and i'm sure that other people may have similar concerns for any one optimization that you write may only be possible in some future. and until that is reached, it looks just like needless churn
<mattip>
I think there are two things at play here. One is a slow refactoring of the C-API by vstinner, another is the approach taken by tools like cffi, cython, pybind11
<mattip>
and hpy to move python users away from using the cpython "internal" API
<mattip>
the two can coexist, and indeed need eachother
<mattip>
some of the refactoring is required so the other approach can be usable
<mattip>
since this is the hpy IRC channel, it is natural we would be more focused on getting hpy working than on vstinner's refactoring
<mattip>
which is fine, I think we should accept that vstinner will work on his things, and we will work on ours, and we should be supportive of eachother where possible
pmp-p has quit [Quit: No Ping reply in 180 seconds.]
pmp-p has joined #hpy
<vstinner>
phlebas: i implemented tagged pointer on top on my C API refactoring work. it wasn't faster :-p but my implementation was limited to None, True, and a very small integer range [-5; 256] :-p
<vstinner>
i'm not trying to pull HPy towards the current C API, but the opposite :)
<vstinner>
phlebas: about changes with negative perf impact, my plan is to write a PEP for these ones
<vstinner>
phlebas: maybe using tagged pointers is a bad way to explain that the C API needs more abstractions
<phlebas>
mattip: you're right, and I'm sorry vstinner if i'm sounding negative
<vstinner>
phlebas: you're not sounding negative
<phlebas>
i really do think the api needs to be cleaned up, and I appreciate your voice to remind python-dev about this over and over
<phlebas>
i just wish it could all be done in a big swoosh, with a big effort to break and then migrate the world
<vstinner>
phlebas: that's not going to happen :)
<phlebas>
yes, i know :/
<vstinner>
my work started by identifying bad patterns. it's now documented at https://devguide.python.org/c-api/ PyPy and HPy devs helped me a lot of identify bad patterns ;-)
<vstinner>
phlebas: my work has different motivations than HPy. for example, the PyThreadState and PyFrameObject changed a lot in Python 3.11. it's going to break many C extensions
<vstinner>
we need more abstractions to avoid that
<vstinner>
(well, the changes were motivated by CPython optimizations)
<Hodgestar>
phlebas: With HPy, we can skip the "break" part and move straight to the "migrate" part. :D
<Hodgestar>
Probably the most important thing to do right now is to get HPy a little bit further along (enough that a trial cython backend can work).
<Hodgestar>
And maybe stable enough that we could, e.g., do a 0.5.x releae branch.
<phlebas>
Hodgestar: well, fangerer has cython far enough to build and run a basic hpy module, but of course there's lots to do
<Hodgestar>
phlebas: Ooh! Nice! Is there a branch somewhere?
<phlebas>
i think not yet
arigo_ is now known as arigato
<vstinner>
phlebas: do you know if it's the main branch of Cython or the 0.29.x branch?
<fangerer>
vstinner: Matti started the work on Cython (see https://github.com/mattip/cython/tree/hpy) and my changes are based on that. So, that's not based on `0.29.x` branch.
<fangerer>
Hodgestar: I hope I can publish my branch soon. I need to get the necessary approvals first.
<fangerer>
as phlebas mentioned, the `hpy_basic` test (basically as added by Matti) works; my goal is to support `piconumpy.pyx`
<Hodgestar>
fangerer: Good luck hopping through the approval hoops!
<fangerer>
🙂 shouldn't be a big problem
<vstinner>
fangerer: so the main branch should exit the beta tunnel!
<fangerer>
I basically agree but I think right now it's fine. My current expectation is that we can merge at least some of the changes to Cython master. Since my changes are not well tested yet, this fits into the beta tunnel.
<Hodgestar>
The great thing about the extended beta is that perhaps HPy support can go into it with less of a wait than into master? So there will be something merged for people to try.
<mattip>
pypy 7.3.6rc2 is ready with hpy0.0.3
<mattip>
I will be releasing the candidate soon, once the download caches update