<Hodgestar>
Ah, our dev call is only *next* Thursday. I need to come up with some ideas before Friday. Suggestions welcome.
<Hodgestar>
Probably it makes sense to ask students to do something *with* HPy and then evaluate it, rather than have them contribute directly to HPy itself.
<mattip>
maybe creating interesting benchmarks?
<steve_s>
one obvious thing that comes to mind is porting Pillow as @mattip suggested last time. I ended up focusing on numpy now instead of having a go at it myself, but I still think it would be a nice self-contained example of a package where performance matters and does not depend on numpy.
<mattip>
but maybe it would be interesting to try to re-write the filters in pure python and see how HPy stacks up
<Hodgestar>
steve_s, mattip: Thanks, that's a good suggestion. Let me look at its codebase a big.
<Hodgestar>
s/big/bit/
<Hodgestar>
mattip, steve_s: Do you have a sense of how pillow performance looks on PyPy and GraalPython at the moment? E.g. are there known use cases where it is much slower?
<steve_s>
No at the top of my head, maybe @phlebas would know
<phlebas>
i don't remember. I know on graalpython the warmup is an issue, which is one thing that would be fixed for us with an Hpy port, too
<Hodgestar>
Perhaps the ImageFilters are a good place to start looking as mattip suggested.
<phlebas>
Pillow doesn't seem to use too much crazy stuff
<phlebas>
no tp_traverse, no base assignments
<phlebas>
a bunch of static types though, those would have to be migrated
<phlebas>
that reminds me, I should push my analysis script for packages to the hpy project github...
<Hodgestar>
It's also nice that Pillow is divided up into reasonably independent extension modules.
<Hodgestar>
Perhaps _imagingmath.c is a better place to look than the filters.
<Hodgestar>
The hard part of this is trying to turn it into an academic project of some sort.
<phlebas>
well, the academic part that would be both interesting and helpful to use is to ask the question what this buys cpython
<phlebas>
at least some of cpython code may want to see more benefits directly for cpython from hpy
<phlebas>
I could imagine we think of a reasonably simple change that is currently not possible with cpython's exposed structs and show that this works just fine with hpy
<phlebas>
or maybe demonstrate the forward ABI compatibility - port something, get a universal build, change hpy, show it still works with the same binary?
<phlebas>
another: i don't know how well pillow would already work with subinterpreters, but if it doesn't and we can show it (can) work fine with hpy, that's another selling point
<Hodgestar>
phlebas: Re forward ABI compatibility: By "change hpy" do you mean "append something to the ctx"? Or just "change what the ctx passed in is"?
<phlebas>
both :)
<phlebas>
i mean, i assume there are going to be concurrent changes to the context, so they could just show "hey, i ported this version, someone else updated hpy, still works"
<phlebas>
and then also demonstrate debug context
<phlebas>
then another thing that could academically interesting is a rigorous performance analysis. not the simple "here's some benchmarks", but also what impact hpy universal vs "hpy cpython ABI mode" (where we still use Incref/Decref macros) has on things like cpu caches, pipeline stalls and such
<Hodgestar>
I wonder if there are other contexts one could write that are like the debug context, but do different things? E.g. a performance profiling context that tracks API calls, or kinds of operations on objects.
<phlebas>
there was the sandboxed pypy thing, and we have a sandboxed graalpython mode, so one could imagine a context that tries to track accesses to certain things at least?
<phlebas>
not sure how useful that would be though
<antocuni>
I don't think that the hpy context is the right abstraction level to make a sandboxed python, because as long as you are in C, if the caller is malicious it's very easy to circumvent the sandbox
vstinner has joined #hpy
<vstinner>
hello. i have a good news for HPy: the Steering Council somehow rejected PEP 670 "convert macros to functions" and PEP 674 "disallow using macros as l-values"
<vstinner>
i understand that the C API must not change ever, and so the only solution to evolve the C API is to write a new C API as HPy does :-)
<vstinner>
(updated PEP 670 *might* be accepted, but this PEP alone is not really interesting)
<Hodgestar>
vstinner: Welcome aboard? :)
<vstinner>
Hodgestar: hehe. i will see. i didn't abandon completely my plan. i'm just... very disappointed
<Hodgestar>
vstinner: Don't keep flogging a dead horse -- <vader>join us</vader>!
<vstinner>
:-)
<Hodgestar>
I am laughing a bit at the "We see no reason to rush" line of the Steering Council response. There needs to be some middle ground between "not rushing" and taking 2.5 years to make a tiny change.
<Hodgestar>
I like your approach about worry less about the time frame, and more about working with the ecosystem of packages that might be broken.
<Hodgestar>
I don't have a huge amount of sympathy with the "there is proprietary code out there that might be broken". It's not that I don't care, but we have no way of knowing what invisible projects are out there, so the argument can be used to block essentially any change, and realistically even invisible proprietary code needs to either freeze all of its dependencies or have a way to track small changes in them. We're not talking about rewriting the world
<Hodgestar>
here.
<vstinner>
Hodgestar: i'm repeating myself for 5 years. i think that everybody who cares about the C API knows my plan. well, i don't buy these arguments about waiting and have a migration plan on 10 years...