<fijal>
the amount of shit I need to fight with to get the development environment going is annoying :/
marvin_ has quit [Remote host closed the connection]
marvin has joined #pypy
<fijal>
mattip: ok, so the current problem is pkg-config bdw-gc gives us garbage
<fijal>
and I would imagine this would be a problem for everything, since the x86_64 is the default path, can we somehow swap them?
<mattip>
can't you change the PATH to find /opt/homebrew/bin/pkg-config if arch == 'arm64'
<fijal>
that's almost certainly a problem
<fijal>
it seems to me like having by default the non-native architecture would create issues, always
<fijal>
seems like I'm adding hacks for this specific layout, that's probably a bit bad
<fijal>
what do we need x86 homebrew for?
<fijal>
If you’ve been using Homebrew under emulation, you now have two side-by-side installs, so from this point on it’s a matter of working tool by tool, installing an ARM64 versions then removing the x86-64 version
<fijal>
so yes, they suggest removing the x86_64 one once we have one for arm64
<fijal>
mattip: have you found out why I can't use homebrew myself?
<mattip>
"groups matti" and "groups fijal" are equivalent, so no idea
<mattip>
what happens when you try
<fijal>
ah ok, so I need sudo
<fijal>
but I don't have a password
<fijal>
(I never did)
<mattip>
hmm, I wonder how I am different
<fijal>
you don't have to use sudo?
<mattip>
nope
<fijal>
ok, well, the entire /usr/local belongs to matti
<LarstiQ>
has the group been added after fijal logged in? (i.e. relogin or start a new shell)
<fijal>
matti/admin
<LarstiQ>
ah, or that :)
<mattip>
ok, so make it admin/admin
<mattip>
feel free to wipe /usr/local and start over
<fijal>
ok
<fijal>
well, it still belongs to matti
<fijal>
so I can't :)
<fijal>
(I think I'm ok uninstalling packages one by one as they say)
<Guest96>
you can just nuke brew and do the arm install
<Guest96>
although I have both x86 and arm brew side by side for $reasons, which works currently, but will go wrong
<mattip>
it must be some difference with permissions. "chown admin.admin" fails of course since admin is a group
<Guest96>
I don't know of this helps, but here /usr/local belongs to 'system' with rw, and 'wheel' has r and 'everyone' has r, but the brew dirs in /usr/local belong to my actual user
<Guest96>
of / if /s
<Guest96>
I know very little about macOS permissions apart from "try not to change the defaults" and "be cautious with sudo"
<fijal>
mattip: maybe make it fijal:admin then?
<mattip>
ok, I guess we can change it back as needed
<mattip>
jsut one more check: I did "chmod -R g+w", did that change anything?
<fijal>
nope
<fijal>
ls -lh /usr/local/var/homebrew/locks
<mattip>
one more time (this time I did chmod with sudo)
<Guest96>
afaict the situation on m1 doesn't have 'standard' yet, I switched about 4 weeks
<Guest96>
ago
<Guest96>
I think in lots of projects the mac maintainers have been waiting until this m1 Pro was available
<mattip>
Guest96: switched? from what to what?
<Guest96>
intel mac -> arm
Guest96 is now known as andythenorth
<andythenorth>
if the binary is an actual double-click mac app bundle then it's likely to be a Universal build (this is what OpenTTD did)
<andythenorth>
I think we're all hoping the brew ports tree packages just switch to arm builds asap
<mattip>
it seems at least scientific python packagers are hesitant to provide universal2 builds
<mattip>
there are some problems with "sizeof(long long)" and some other strange corner cases
<andythenorth>
super :)
<mattip>
maybe like fijal says, x86_64 will slowly just go away
<andythenorth>
that's my expectation
<andythenorth>
there's precedent with the end of 32 bit support and the prior PowerPC -> intel switch
<andythenorth>
most stuff gets done in about a year, then there are always a few critical things everyone uses it that take longer, but get done eventually
<andythenorth>
my English today is terrible sorry, I am a native English speaker, just have no brain without cofee
<cfbolz>
andythenorth: don't worry, we all aren't
<andythenorth>
at least endian-ness no longer seems to be the issue it was about 20 years ago :P
<mattip>
"use system libffi instead of vendored one on Mac OS 10.15 and up, use correct ABI for variadic functions on arm64, check for libs in the dyld shared cache"
<ctismer>
mattip: might this be related to my problems on macOS?
<mattip>
are you using M1 Apple silicon arm64?
<ctismer>
no
<mattip>
so no :)
<ctismer>
ah :)
<mattip>
well, with that who knows :shrug:
<mattip>
ctismer: btw, I started addng a test for PyCFunction_New, but got stuck
<ctismer>
mattip: Anyway, after very long hacking, I again reached a point where I don't understand the world. I set an attribute on an instance, I can check it with `PyObject_HasAttr`, but it is not found in `obj->ob_dict. With CPython, it is.
<mattip>
what interface do you use to set the attribute?
<ctismer>
mattip: I will now really send you a set of PyPy and Python wheels, to have a look. 🤪
<ctismer>
mattip: This turned out to be the problem: I even can check with `PyObject_HasAttr`, but the according dict is still not touched.
* ctismer
is at least _much_ more eloquent in `lldb` now :)
<fijal>
mattip: ok, let me know if you find out how to do it on python 2.7, I'm making some food
<mattip>
so the obj->ob_dict might be reflected only once when the pyobj is set up, and not updated for the added w_obj.w_dict field?
<mattip>
shouldn't all calls should go through the c-api and not directly access obj->ob_dict?
<ctismer>
mattip: can be, yes. But where does it get the `PyObject_HasAttr` result if no object is set. Is that a cache effect?
<mattip>
there are two objects: the c-level pyobj and the rpython w_obj
<ctismer>
!
<mattip>
dunno what guarentees we give about synchronizing their state
<mattip>
for instance, PyListObject is totally opaque, you cannot get anthing interestng out of the pyobj
<ctismer>
mattip: we are fiddling with ob_dict quite often. I thought we are safe when we are not modifying types?
<mattip>
if you expect the changes in ob_dict to be synced to the w_obj, I think you will be dissapointed
<ctismer>
mattip: yes, that's ok (list object). But SbkObject is our own stuff, that is not clear
* ctismer
building wheels, anyway
<mattip>
once the w_obj is created from the pyobj, or the pyobj is created from the w_obj, the w_obj dict and the obj->tp_dict go their separate ways
<mattip>
so if you manipulate the obj->tp_dict directly from C, you will only see those changes when directly using the obj_tp_dict
<mattip>
all the C-API calls and python code will use w_obj.__dict__
<ctismer>
mattip: aha! (Grübel...) What would be the right way to do what we do...
* ctismer
should have asked this 3 weeks earlier 🙃
vstinner has joined #pypy
<vstinner>
hello. i wrote
<vstinner>
PEP 674 -- Disallow using macros as l-value
<LarstiQ>
ctismer: how often do you do that? Can you guarantee all the changes are done before the other object is created?
<ctismer>
vstinner: that is great!
<vstinner>
hello. i wrote PEP 674 -- Disallow using macros as l-value which says that it will help PyPy cpyext to avoid converting PyPy objects to CPython objects in the long term: https://www.python.org/dev/peps/pep-0674/#pypy-cpyext-module
<vstinner>
do you think that this statement is correct?
<vstinner>
for example, "Py_SIZE(obj) = new_size" must now be replaced with "Py_SET_SIZE(obj, new_size)"
<vstinner>
my assumption is that supporting "Py_SIZE(obj) = new_size" requires to expose the PyVarObject structure and convert a PyPy object to a CPython object
<vstinner>
whereas it might be possible to implement Py_SET_SIZE(obj, new_size)" on the PyPy object directly
<vstinner>
maybe Py_SET_SIZE() is a bad example
<ctismer>
LarstiQ: I don't know exactly. Parts of the code reach back to 2009, I wrote maybe 10 percent or less...
<mattip>
AFAICT in order to call Py_SET_SIZE in any form (macro, function) a PyObject* must exist. So the conversion has to happen no matter how you frame it
<ctismer>
LarstiQ: So to understand this fully: Direct ob_dict access is problematic, also with my self-created objects?
<vstinner>
mattip: ah? there is no way to change a list size in PyPy for example?
<ctismer>
mattip: to make sure: I am talking of `obj->ob_dict``, not `type->tp_dict`
<ctismer>
mattip: to make sure: I am talking of `obj->ob_dict`, not `type->tp_dict`
<mattip>
vstinner: as far as I can tell we don't support Py_SET_SIZE at all
<ctismer>
LarstiQ: So, we have `->ob_dict` 16 times in the code, in 4 files. Oe is a conerator.
<mattip>
at least it is not anywhere in our codebase
<mattip>
ctismer: we do not have any ob_dict in our codebase either, so I am not sure what you are setting :)
<vstinner>
"This PEP alone is not enough to get rid of the CPython objects in the PyPy cpyext module, but it is a step towards this long term goal. PyPy already supports HPy which is a better solution in the long term." :-p
<vstinner>
this sentence is based on the previous section saying that the PEP should help migration C extensions to HPy
<ctismer>
mattip: this ob_dict is _why_ a problem? This is pure C code.
<cfbolz>
ctismer: two questions: is this dict *only* for c code?
<cfbolz>
or should python see these attributes?
<cfbolz>
if you rename the attribute to not ob_dict, you would need suitable __getattribute__ implementations, right?
<ctismer>
cfbolz: this was not clear to me, but right, that's a special name. Have to do a good analysis. But great that I understand now !
<cfbolz>
ctismer: I think what is happening is that you manage to store the attribute into the rpython dict, which is why it's seen by hasattr, but not synced to ob_dict
<ctismer>
cfbolz: so if I want to control these attributes via dict, I should either rename them (private only), or I need to use `obj.__dict__` ?
<fijal>
hm
<fijal>
I wonder if I'm getting stuck on unrelated failures
<ctismer>
cfbolz: that explains a lot 😀
<cfbolz>
ctismer: yes, either you rename the C field and implement a getattribute etc
<ctismer>
cfbolz: mattip: LarstiQ: You made my day (actually 3 weeks)
<cfbolz>
ctismer: I wonder though:
<cfbolz>
maybe it is enough to read ob_dict some other way
<cfbolz>
there is probably some function that reads .__dict__, no?
<cfbolz>
PyObject_GenericGetDict?
<cfbolz>
so it might be worth a try to replace ->ob_dict with a call to that
Julian has joined #pypy
Julian has quit [Ping timeout: 252 seconds]
Julian has joined #pypy
<ctismer>
cfbolz: yes, of course. I will try. Great, that's progress, already.
<ctismer>
cfbolz: BTW. `ob_dict` is not found in the PyPy documentation (keyword search)
<cfbolz>
ctismer: yes, we don't really have a lot of good info about cpyext
Julian has quit [Ping timeout: 252 seconds]
<ctismer>
cfbolz: this is not complaining, I just made sure that I don't forget something knowable
<cfbolz>
ctismer: I think you are in unexplored land here ;-)
Julian has joined #pypy
* ctismer
thinks there is no better way to learn about PyPy than to port a huge library 😅
<ctismer>
cfbolz: we can augment the docs by using my PYSIDE-535 comments...
<cfbolz>
ctismer: yes, we should
<ctismer>
cfbolz: so if I use `PyObject_GenericGetDict` to touch the `ob_dict`, then my `PyDict_SetItem` will land where they should? Probably nothing that is done frequently
<mattip>
ctismer: why not use the PyObject_SetAttr* c-api functions?
vstinner has left #pypy [#pypy]
<ctismer>
mattip: not sure why they did that. Would the PyObject_GenericGetDict not work? It has the advantage of no semantic change of code that I barely understand
<mattip>
dunno really. You might have to call PyObject_GenericSetDict to resync the objects
<mattip>
but maybe not, I am not sure
<ctismer>
trying...
<ctismer>
mattip: In the actual case, `PyObject_SetAttr(source, key, signalInstance)` is called. The initialization is `self->ob_dict = nullptr;`. Would I need to remove that initialization?
Julian has quit [Quit: leaving]
greedom has quit [Remote host closed the connection]
slav0nic has joined #pypy
Julian has joined #pypy
andythenorth has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<cfbolz>
ctismer: I would try generic get dict first, it sounds like a small and mechanical change
<cfbolz>
If it turns out to be a huge perf problem we can think again
<ctismer>
cfbolz: just tried and got from 66 to 34 errors! Not ready, yet :)
<cfbolz>
ctismer: you pessimist, that sounds great 😉
<LarstiQ>
so that's 6 * 3 more weeks were we half the number of errors? ;)
<ctismer>
cfbolz: I'm more than happy 😁
Julian has quit [Quit: leaving]
<ctismer>
LarstiQ: down to 32 errors
Guest96 has joined #pypy
Corbin has quit [Ping timeout: 265 seconds]
Corbin has joined #pypy
Guest96 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]