cfbolz changed the topic of #pypy to: #pypy PyPy, the flexible snake | IRC logs: and | the pypy angle is to shrug and copy the implementation of CPython as closely as possible, and staying out of design decisions
pvital has quit [Server closed connection]
pvital has joined #pypy
jryans has quit [Server closed connection]
jryans has joined #pypy
ammar2 has quit [Server closed connection]
ammar2 has joined #pypy
glyph has quit [Quit: End of line.]
glyph has joined #pypy
Lightsword has quit [Quit: ZNC]
Lightsword has joined #pypy
otisolsen70 has joined #pypy
jcea has joined #pypy
jcea has quit [Ping timeout: 246 seconds]
jcea has joined #pypy
wbarnha has joined #pypy
<wbarnha> Hey folks, does anyone know where I can read documentation on how sets in PyPy are different from the CPython implementation? I'm trying to update a project to support PyPy but I'm running into an edge case in
<wbarnha> I understand that sets are supposed to be non-deterministic by definition but I'm running into some cases where that's not necessarily true
<wbarnha> I also noticed that set has the method __sizeof__ defined in CPython but not PyPy
<nimaje> (github doesn't want to show me the logs for that test run without a login)
<nimaje> sets are deterministic, but stuff like iteration order is not specified, so you aren't allowed to assume anything about iteration order
<wbarnha> Sorry to hear that you can't see the GitHub logs, any alternative way you want me to show them to you? Hopefully you'll see why I'm perplexed after seeing them
<nimaje> copy them into a pastebin like
<nimaje> what is unexpected there? set.pop() removes and returns some element from the set, I expect that that MutableSetProxy doesn't add some guarantes about order there
<wbarnha> Thanks for checking, I'll go back and review it
<nimaje> test_MappingProxy.{test_keys,test_values,test_iter} and test_MutableMappingProxy.test_popitem also make assumptions about iteration order
<mattip> about __sizeof__, that is a difference between CPython and PyPy, see about sys.getsizeof on
<mattip> we will not support it since it does not make sense on PyPy
<mattip> I guess we could add a comment that sys.getsizeof() calls __sizeof__, which does not exist on PyPy objects,
<mattip> but really people should be using the official interfaces and not probing dunder attributes
<mattip> (method, not attribute)
<mattip> wbarnha: do you find many people using PyPy with Mode? Are there any benchmarks around it?
<wbarnha> Mode is a dependency of Faust, I'm doing some investigation to see if Faust can support PyPy. In the past, it was claimed that it worked on PyPy, however our unit tests indicate it's *barely* possible but very unstable. Besides people claiming performance boosts in niche applications, we don't have formal benchmarks.
<wbarnha> I've used aiokafka and PyPy together and the performance gains are definitely in terms of CPU usage being halved
<wbarnha> It seems there's some difference in the behavior of sets in CPython vs PyPy, not just Mode's implementation:
<mattip> cool. Unstable in what way?
<wbarnha> There's deadlocks observed while running unit tests, one second, let me open a PR to show you the specifics
<mattip> For sets: they are documented as being unordered.
<nimaje> wbarnha: there is *no* gurantee which element is returned by set.pop, each run of that code could give you another element, the only thing you can assume is that it was an element of the set and was then removed
<mattip> "A set object is an unordered collection of distinct hashable objects"
<cfbolz> yes, in cpython the order is implementation defined and you mustn't rely on it. it can change from version to version
<cfbolz> it depends on the hash values of the elements
<cfbolz> wbarnha: in pypy sets are like dicts, they will keep the *insertion* order of the elements. but again, it's best not to rely on it
<mattip> As for "unstable" - do you mean this PR run?
<wbarnha> The CPython detail makes sense from version to version, I've been trying to wrap my head around why the original author assumed that sets were ordered.
<wbarnha> mattip correct I've had to add a timeout in the meantime while I allocate time for myself and the other maintainer to figure this out
<mattip> OK, cool. There are resource warnings about sockets and uncalled await loops
<mattip> this can happen if the code depends on __del__ being called when an object goes out of scope
<wbarnha> Yeah, those definitely need to be fixed but we never got around to that. Maybe that's worth investigating now
<mattip> so by adding a timeout you allow the PyPy gc to collect those objects, freeing the resources (but still emitting the warning)
<mattip> fwiw, in other places people have changed the test to be something like
<mattip> assert set.pop() in <values>
<mattip> instead of
<mattip> assert set.pop() == value
<wbarnha> That would make sense. I'm going to take a bit to review the existing usage of MutableSet in Faust to make sure there weren't any other erroneous assumptions for ordering
<wbarnha> Thanks for the assistance everyone!
wbarnha has quit [Quit: Client closed]
wbarnha has joined #pypy
wbarnha has quit [Client Quit]
marvin_ has quit [Remote host closed the connection]
marvin has joined #pypy
<Dejan> is it true that Celery/faust author commited suicide?
jcea has quit [Ping timeout: 246 seconds]
<sam_> is that really an appropriate question here...?
<Dejan> sam_, mattip mentioned faust-streaming and i thought maybe someone here knows the answer
<Dejan> sam_, why would you think it is an inappropriate question?
<Dejan> Imagine, God forbid, an author of an opensource project you use for years commits suicide - you would like to know about it, right?
<arigato> Dejan: what you really want to know is if and when the author drops the project. The reason for why may be nobody here's business
otisolsen70 has quit [Quit: Leaving]