07:51 <mattip> cfbolz: the failure of lib-python.3.test.test_sys after adding "-X utf8" is consistent both yesterday and today

07:51 <mattip> http://buildbot.pypy.org/summary/longrepr?testname=unmodified&builder=pypy-c-jit-linux-x86-64&build=8179&mod=lib-python.3.test.test_sys

07:55 <cfbolz> mattip: that's because 3.7 hasn't been merged in yet I think?

07:55 <cfbolz> On 3.7 it's gone, no?

08:54 <mattip> yup, gone on 3.7 but still there on 3.8, even though 76c3c45c6258 was merged to 3.8

08:55 <cfbolz> :-(

08:56 <cfbolz> mattip: putting it on the todo, need to fix one major remaining problem with utf8 mode

11:16 <cfbolz> test_utf8_mode now passes locally on that branch

12:04 <cfbolz> fijal: I was doing a bit of benchmarking, pypy definitely becomes faster than cpython at urllib3 http requests after a few dozens of them

13:05 <fijal> cfbolz: yep

19:19 <cfbolz> mattip: the 3.8 failures are even deeper into the rabbit hole

19:20 <cfbolz> I might leave them for a while

19:24 <Pylon> Hey, why isn't Pypy 3 accelerating Yt-dlp (Youtube-dl's active fork) faster than I expected?

19:25 <Pylon> It's almost taking 3 times longer for Youtube-dl to run with Pypy 3 compared to Python 3.7

19:26 <cfbolz> Pylon: isn't yt-dlp bandwidth-bound?

19:27 <Pylon> Not really for me as I have a decent internet speed and I am trying to accelerate the execution speed of getting video id + format and downloading thereafter

19:28 <cfbolz> Pylon: it's usually one video per process, right? not a lot of chance for the JIT to warm up

19:29 <cfbolz> pycryptodomex is likely not fast in pypy, it's in C

19:30 <cfbolz> at least parts

19:30 <Pylon> I checked and it at least takes me 3 to 5 seconds to start up yt-dlp with Python 3 and was hoping to accelerate & cut down the initialsation time. However, it's backfiring for me and it's taking 10 to 15 seconds depending on the video

19:31 <Pylon> cfbolz: Yeah that's usually the case unless you are doing in batch from a txt file. Nevertheless, was hoping for JIT to speed up the run time

19:32 <Pylon> Oh are there other libraries/dependencies that are running in C? Wasn't expecting that as it said Pycryptomdomex was compatible with Pypy 3 on the Py pi page

19:34 <cfbolz> Pylon: I don't know, you would have to dig

19:35 <cfbolz> but pycryptodomex seems to have some C code

19:35 <cfbolz> (which is supported in pypy, but it tends to be slower than on CPython)

19:36 <Pylon> I see and would there be any pure python alternatives?

19:36 <Pylon> I was actually hoping for Pypy 3 to cut the overall runtime but I guess it's too much of a lofty ambition for now

19:37 <cfbolz> Pylon: if you have use a python alternative, then the JIT would have to be faster than C to beat CPython ;-)

19:38 <Pylon> At the moment, it takes around 6 seconds just for Yt-dlp to print it's first line with Pypy3 (vs 1 to 2 second for Python 3) and I was hoping for the overall time to take only 1 seconds lol

19:38 <cfbolz> yes, not without extra work, it seems

19:38 <cfbolz> Pylon: if you want to investigate, one thing I would try is to download 10 videos at once, and see whether the later download faster

19:39 <Corbin> Pylon: Where's it slow? IIRC youtube-dl emulates JS a bit, which is slow for irritating computer-science reasons (Python isn't designed for writing fast emulators, basically)

19:39 <Pylon> Yeah I hope that can be a thing that's actually possible and would mean switching out a lot of the libraries to native python versions, just to prove Pypy 3 can run Yt-dlp faster than Python 3.7

19:40 <Pylon> I am not sure exactly where but I can confirm right at the beginning is perhaps one of the worst performing areas

19:41 <Pylon> Initialisation of Yt-dlp alone already takes 6 seconds which proves the start up is slow to begin with here

19:48 <Pylon> Corbin: Just ascertained a 9 vs 19 seconds when downloading 3 videos format and printing them out

19:49 <Pylon> I can post the log if you wish to see

19:50 <Corbin> Pylon: I don't think that the log will help, unless they differ substantially. The typical approach here would be to "profile" the code while it's running; this gives us locations in youtube-dl's code where time is spent.

19:51 <Corbin> (I don't know the right way to profile Python code on PyPy 3 today, but hopefully somebody else does.)

20:10 <Pylon> Corbin: Yeah I would love to help and profile the code while running. It's worth it as you said the the log differs by a lot and doing some debugging might help

20:11 <Pylon> Not sure exactly how Pypy 3 works but can't it just run C code through it's JIT?

20:12 <Pylon> Or perhaps use the JIT for native Python code while using the C compiler (found in Python 3) for C codes when needed