14:25 <Diggsey> I'm having some issues with pypy being a lot slower than cpython. One of the issues seems to be `isinstance` - cProfile is showing a lot *more* calls to `isinstance` under pypy than with cpython, and each call is taking much longer

14:26 <Diggsey> is there any reason why either of those things should be the case?

14:29 <Diggsey> under pypy, it's called `8056582` times and spends a cumulative time of 8.412s within that function, compared to 4486368 and 2.888s for cpython

15:16 <arigato> Diggsey: using cProfile is generally not recommended as a way to get a good profile on pypy. About "isinstance", there are some code paths that may internally call isinstance() on pypy, whereas on cpython they would call some C code directly and not show up in the statistics

15:34 <mattip> is your code calling into c-extensions? These are known to be slower on PyPy

16:42 <fijal> also isinstance is one of those functions that will be overrepresented, because on pypy checking the clock twice and putting info in a dictionary is quite expensive compared to just "isinstance" call

16:48 <cfbolz> Diggsey: what does the program do? is it something we can run ourselves?

16:55 <Diggsey> @arigato: I would love to use something else but it seems like no other profiler is supported by pypy on windows

16:55 <Diggsey> fwiw, it seems to take a similar amount of overall time without cprofile

16:56 <Diggsey> @cfbolz: unfortunately no, it's a fairly large legacy python application. I've written a script to exercise one part of it that is known to be slow

16:56 <Diggsey> I was hoping that pypy might give some performance improvements

16:58 <Diggsey> @mattip: I think it does, but I don't think they're called frequently enough to be an issue

16:58 <cfbolz> What kind of stuff does it do?

16:59 <Diggsey> it's exercising some endpoints in a flask CRUD app - it makes uses of sqlalchemy to access a mysql db

16:59 <Diggsey> but a large amount of time is spent recalculating stuff in python that's changed

16:59 <Diggsey> a large amount of time is also spent just in the sqlalchemy code (ie. not actually waiting on the db)

17:00 <fijal> sqlalchemy is *very* slow

17:00 <Diggsey> yeah, I was hoping pypy might help with that...

17:00 <fijal> Diggsey: I have a non-solution solution for you, which is not use sqlalchemy (or any otheR ORM really(

17:00 <fijal> it doesn't

17:00 <Diggsey> haha

17:00 <fijal> I tried optimizing it and I can go for pages why it doesn't

17:01 <fijal> but essentially a JIT-friendly ORM is something that I was wanting to write for a while, but never had a good use-case

17:01 <Diggsey> don't suppose you've written that up anywhere?

17:01 <Diggsey> it would be useful to understand more about what makes something jit friendly and what actually makes sqlalchemy slow

17:06 <fijal> so the main thing that makes sqlalchemy slow is that it's essentially an interpreter

17:18 <Diggsey> right, I guess I just don't really understand why that's so bad for pypy

17:18 <Dejan> I only use SQLAlchemy core

17:22 <Dejan> I wonder if there is something like ActiveJDBC ( https://javalite.io/activejdbc ) in Python world...

17:22 <Dejan> That is the only "ORM" that I may try

17:26 <fijal> Diggsey: well, first of all it's slow to start with

17:27 <fijal> but two, you have very general very megamorphic calls to a bunch of functions and you construct the whole thing each time

17:27 <fijal> so instead of having say select_something_from_something_else(param1, param2) that does the job you have the generic select(*stuff, **even_more_crazy_stuff) that differs from call to call

17:29 <Diggsey> but I mean, my app most only use a few specific combinations right - I would kindof expect a JIT to be able to beat cpython here since it can look at the actual types and specialize

17:29 <Dejan> in most cases it waits for data to come, right?

17:30 <Dejan> I would honestly expect them to be roughly the same

17:30 <Diggsey> pypy is 2x slower overall

17:31 <Dejan> then it probably heavily uses some underlying C extension

17:31 <Diggsey> and that's ignoring startup cost and making the same sequence of requests 20x

17:32 <Diggsey> for a total of 186s in the loop under pypy vs 98s in cpython

17:33 <cfbolz> yeah, that's definitely not great (and my lesson here would be "we should investigate ORMs again")

17:33 <Diggsey> it's using pymysql as the driver, so there should be no C code for that part...

17:34 <Diggsey> is there some way I can see what C calls it's making?

17:35 <fijal> Diggsey: well, pypy can't guess which part of the call is constant and should be specialized on

17:35 <fijal> (it's kind of expected that a very general python code is 2x slower on pypy, your mileage may vary of course)

17:35 <fijal> I would also expect that sqlalchemy got a bit optimized for cpython, which is usually actually bad for pypy