12:18 <phlebas> mark has replied to my issue on his repo

12:19 <phlebas> https://github.com/markshannon/New-C-API-for-Python/issues/1#issuecomment-1222172615

12:19 <phlebas> i might not have time to formulate answers today, but i'll try in the evening, unless someone else would like to respond already :)

12:20 <cfbolz> let's brainstorm a good answer maybe

12:25 <Hodgestar> I think this really needs a conversation with Mark more than a reply.

12:26 <Hodgestar> We can go backwards and forwards critiquing unfinished solutions on both our sides or Mark's for ages.

12:26 <Hodgestar> The real question is would Mark like to work with us?

12:28 <Hodgestar> Working together will be a bit painful for everyone at the start, because we have different sets of ideas right now, but it would be really good for the implementation.

13:03 <mattip> who has operator permissions here?

13:03 <mattip> _0az3 has been spamming the IRC logs at https://quodlibet.duckdns.org/irc/hpy/latest.log.html

13:03 <mattip> for quite some time now. Could someone ban them?

14:05 <Hodgestar> mattip: Chanserv says that the channel founder is cfbolz.

14:17 <cfbolz> Who woulda thunk

15:46 <phlebas> cfbolz: well that didn

15:46 <phlebas> +'t work

15:46 <cfbolz> yeah, sorry, only kicked him so far

15:47 <phlebas> Hodgestar: yes, i think we should try to have a call with mark to just talk about how we can work together

15:48 <phlebas> i think Petr Viktorin would also be good to have as part of the conversation

16:04 <phlebas> > Where are the design principles of HPy listed? I couldn't find an explicit list.

16:04 <phlebas> > What is it specifically about subinterpreters that is of concern?

16:04 <phlebas> Indeed we haven't created an exhaustive list, mostly because we keep discovering issues while porting packages like NumPy (which exposes it's own C API for consumption by other packages). There is https://github.com/hpyproject/hpy/wiki/c-api-next-level-manifesto and in that wiki a lot of resources we collected while porting things over time.

16:04 <phlebas> >

16:04 <phlebas> > https://github.com/markshannon/New-C-API-for-Python/blob/main/DesignPrinciples.md#minimum-of-implicit-state

16:04 <phlebas> mentions the conflict between explicitly passing around the interpreter and making it implicit.

16:04 <phlebas> >

16:04 <phlebas> > I might make sense to wait for a decision on https://peps.python.org/pep-0684/ before thinking about this too much.

16:04 <phlebas> Ah, I had overlooked that. This is indeed our thinking in HPy, too, to avoid global state as much as possible. The additional `HPyContext` argument was inspired by APIs such as JNI or SDL2, which have something similar. SDL2 in particular uses this (implicit) global function table to be forward binary compatible. However, for Python there is an issue, since multiple extensions may be compiled against different versions, years apart, and

16:04 <phlebas> still need to run in the same process. Without an explicit context that means any time any API needs to change we need a new global method, whereas with a context, this can be handled by the runtime passing the right context to each module.

16:04 <phlebas> In particular we in HPy take the same view as you do [here](https://github.com/markshannon/New-C-API-for-Python/blob/main/DesignPrinciples.md#api-and-abi-equivalence):

16:04 <phlebas> > All code written to the API will continue to work on future versions of Python without recompilation. Recompilation using newer versions may be more efficient, but code compiled to older versions of the API will continue to work.

16:04 <phlebas> > [...]

16:04 <phlebas> > Once added to the API, a feature will be removed only if there is a very strong reason to do so.

16:04 <phlebas> So we do want to be binary compatible forever, but still under important circumstances need to be able to change the API. With global functions, this would imply a new function, with a context argument, just a different context for the newer binaries.

16:04 <phlebas> > C globals are not supported as they are unsafe.

16:04 <phlebas> We have found dozens of usages of plain C structures stored globally or reachable from other global variables in NumPy, for example. It seemed to us quite a burden to force extensions having to find completely different solutions to this. The thoughts around `HPyGlobal` aren't all fleshed out, but https://github.com/hpyproject/hpy/wiki/dev-call-20220303#hpyglobal has some notes, as this was driven by the NumPy port.

16:04 <phlebas> ahhh

16:05 <phlebas> i was trying to paste this into the pastebin tab and failed miserably

16:06 <phlebas> https://pastebin.com/8YYJqLV8

16:06 <phlebas> ^ how about this as answer to mark?

16:11 <phlebas> Also added this at the end: "I think it would be useful if we (the current HPy devs, you, @encukou) could have a direct conversation about this, maybe you would have time for a call? The people working on HPy would love to see a better future C API and would be glad to help with the experience we gained from HPy."

16:15 <phlebas> what do you think about the answer? just post it? just wondering because mark is thinking about the context argument now, and I would like to give him some links to our wiki before he makes up his mind ;)

20:14 <Hodgestar> The technical arguments look great.

20:15 <Hodgestar> At the end I would avoid offering to help Mark. We would like us and Mark to work together, which is an subtle but important difference.

20:16 <Hodgestar> I'd like to understand why Mark didn't talk to us before starting. Maybe he just wanted to get his own thoughts down first?

20:17 <Hodgestar> phlebas: I'm also fine with you posting it as is. Tip toeing around things too much on our side isn't helpful either. We should just be open and honest and keen to collaborate if we can.