jinsun has quit [Killed (iridium.libera.chat (Nickname regained by services))]
epony has joined #pypy
lesshaste has joined #pypy
<lesshaste>
I have a performance question. I have some python code that I need to make as fast as possible (it takes 2 weeks to run currently using pypy). Does it make sense to implement the functions that I can using cffi or is the overhead likely to make that irrelevant. There is one function in particular that is run 10^10 times
<cfbolz>
lesshaste: can you share the code? what does that function do?
<lesshaste>
cfbolz, it performs stick breaking on a list. That is chooses a random number between 0 and the length of the list, breaks that much off and then repeats
<lesshaste>
I just need to do it billions of times
<cfbolz>
ok. code? or is that private?
<lesshaste>
it is private sorry
<lesshaste>
but it's simple to reimplement :)
<cfbolz>
if you reimplement it and give me some sample data I can take a look
<lesshaste>
cfbolz, I am really asking a general question. If I have to call a function 10^10 times, is it work writing it in C using cffi or will that cause an overhead that would make it too costly
<cfbolz>
that general question has no answer
<lesshaste>
is it worth
<lesshaste>
cfbolz, how come?
<lesshaste>
what does it depend on?
<cfbolz>
the function
<cfbolz>
the input and output data
<cfbolz>
in your case you would have to copy the list from a python list to a C array, right? that sounds expensive
<lesshaste>
yes that does sound bad
<lesshaste>
even if I can make the array in C I would have to at least copy it back into python
<lesshaste>
which I am guessing is expensive
<lesshaste>
so am I right that in general, if the C function needs very little passed to it and returned then the overhead would be minimal?
<lesshaste>
thanks for your help as well
<cfbolz>
or it needs to run for very long
mattip has quit [Read error: Connection reset by peer]
mattip has joined #pypy
<lesshaste>
cfbolz, got you
<lesshaste>
how expensive is copying an array from C to python?
<lesshaste>
hi mattip
<mattip>
lesshaste: hi
<lesshaste>
mattip, I didn't submit a bug report about matplotlib, sorry. But has there been any progress since we last spoke/chatted?
<mattip>
are you using a virtualenv or some other redirection?
<lesshaste>
mattip, yes I did pypy3.9-v7.3.11-linux64/bin/pypy3.9 -m venv mypypy3.9
<mattip>
ok, so maybe the bug is that tkinter does not work inside a venv
<mattip>
if you use the base environment, does everything work?
<lesshaste>
mattip, bug reported
<lesshaste>
mattip, how would I install matplotlib safely outside the virtual env?
<cfbolz>
mattip: yes, can confirm that tk breaks in a venv
<lesshaste>
cfbolz, thanks for reproducing it!
<cfbolz>
lesshaste: as for "outside a virtual env": every if you unpack a downloaded pypy tarball, every one of the unpacked directories is its own world. so you can safely unpack again and directly install matplotlib in there
<lesshaste>
cfbolz, ok thanks!
<lesshaste>
cfbolz, I can't even find pip in the unpacked tar ball