<mathieu_>
When I run this on a test file (format is a json string per line), the finaly memory usage appears to be about 2x the actual size of the file
lritter has quit [Ping timeout: 258 seconds]
<mathieu_>
i.e., the re.sub() call appears to create a modified copy of the data (ok) but the string input to re.sub() never appears to be freed even though it is not referenced anymore. (not ok)
<mathieu_>
yes, I read that too but, I am not sure I understand your question about gc.collect(): why would I need to call it explicitely ? It appears to be called automatically by the pypy runtime whenever it thinks it is the right time.
<mattip>
objects are not immediatedly collected when they go out of scope, so maybe you have not hit "the right time"
<mathieu_>
i.e., the on_gc_collect is called
<mathieu_>
I should add that I tried adding a call to gc.collect() right before the last sleep with no observable difference
<mathieu_>
so, I concluded that pypy was doing the right thing from that perspective
<mattip>
ahh, I see
<mathieu_>
I wonder what I should investigate next to understand where the arena/rawmalloced memory is actually allocated by pypy
<mathieu_>
I'd like to note that despite this issue with memory usage, I am quite impressed by the resulting cpu performance: the actual pypy code is faster by a factor of 2 compared to the highly-optimized google-produced C++ implementation
<mattip>
Dejan: thanks, fixed. That is for the benchmark runs on speed.pypy.org
<Dejan>
👍
fotis has quit [Ping timeout: 276 seconds]
mathieu_ has joined #pypy
<mathieu_>
is there a means to inspect heap dumps generated with gc.dump_rpy_heap ?
<mathieu_>
I started writing my own parser but I suspect this is a waste of time
fotis has joined #pypy
<mathieu_>
alternatively, I would welcome hints on how to use the 'typeindex' to retrieve a textual description of the type from the typeids.lst and typeids.txt files
fotis has quit [Ping timeout: 268 seconds]
<mathieu_>
ah. The number starts at 1; it is an index in the
<mathieu_>
file typeids.txt
fotis has joined #pypy
mathieu_ has quit [Ping timeout: 268 seconds]
fotis has quit [Ping timeout: 252 seconds]
energizer has quit [Ping timeout: 252 seconds]
stkrdknmibalz has joined #pypy
energizer has joined #pypy
fotis has joined #pypy
<mattip>
mathieu_: (for the logs) I have never used it myself, but digging around in the rpython/tools directory I see gcanalyze.py
fotis has quit [Ping timeout: 265 seconds]
luckydonald has quit [Quit: Leaving.]
ambv has quit [Quit: Bye]
fotis has joined #pypy
Dejan has quit [Quit: Leaving]
Atque has quit [Quit: ...]
fotis has quit [Ping timeout: 276 seconds]
fotis has joined #pypy
fotis has quit [Ping timeout: 265 seconds]
fotis has joined #pypy
fotis has quit [Ping timeout: 258 seconds]
greedom has joined #pypy
jj89 has joined #pypy
fotis has joined #pypy
<jj89>
I reported in this channel on july 25th that autocomplete was not working for me with the pre-built binary. mattip reported it was working for him on linux with pypy3.7.
<jj89>
I tested and 3.7 works, but 2.7 doesn't. "ldd pypy" reports the same libraries for both of them though, including libncursesw and libinfow.
<jj89>
mattip: ^^^
<antocuni>
jj89: AFAIK, autocompletion is not supposed to work out of the box on py27
<antocuni>
this is not a pypy limitation, even on CPython 2.7 autocompletion doesn't work out of the box
<antocuni>
(if autocompletion works for you on CPython 2.7, it's very likely that it's because your vendor and/or distribution configured python to do so, but it's not by default)
<jj89>
wow, you are right!
<antocuni>
if you want autocompletion, I suggest to look at fancycompleter
<jj89>
pypy is just being compatible :-)
<jj89>
looking at fancycompleter, i see a note about rlcompleter. If I "import rlcompleter" into pypy2, then autocomplete works. (I'd like to use pypy without dependencies if possible, so this is perfect for my use case.) Thanks antocuni!
<antocuni>
you're welcome
<antocuni>
I suggest again to try fancycompleter before deciding to avoid it, it's waaaay better than rlcompleter2. But I'm biased because I'm the author :)
<jj89>
I'm hoping to avoid external dependencies since I have no confidence about the future of anything related to Python 2. But it looks like on linux I can ignore pip and easy_install and just copy fancycompleter.py into the site-packages subdirectory. So I just tested fancycompleter, comparying it to rlcompleter. It really does seem better since it
<jj89>
autocompletes even list elements. I'm glad that you helped me, this is great!
fotis has quit [Ping timeout: 268 seconds]
fotis has joined #pypy
fotis has quit [Ping timeout: 265 seconds]
fotis has joined #pypy
fotis has quit [Ping timeout: 252 seconds]
jj89 has left #pypy [#pypy]
fotis has joined #pypy
slav0nic has quit [Ping timeout: 276 seconds]
fotis has quit [Ping timeout: 258 seconds]
fotis has joined #pypy
smarr has quit [Quit: Connection closed for inactivity]
greedom has quit [Remote host closed the connection]