07:39 <mattip> quiz: what error should this produce?

07:40 <mattip> python -c 'b"foo\xacbar".decode("utf-8\udce2\udc80\udc9d")'

07:40 <mattip> pypy3 emits a UnicodeDecodeError,

07:40 <mattip> cpython3 emits a UnicodeEncodeError

07:41 <mattip> cpython wants to encode the encoding into ascii

07:43 <mattip> s/ascii/utf-8/

09:50 <Hodgestar> mattip: I have no idea. :)

09:52 <Hodgestar> mattip: I guess the correct answer is "sanitise the encoding string before trying to use it", but I like UnicodeDecodeError a bit more than UnicodeEncodeError (encoding the encoding seems like an implementation detail that breaks the simple mental model of "decoding should raise decoding errors").

10:17 <mattip> yeah, it confused me too until I looked at the implementation of `unicode_decode`

10:37 <lessshaste> numpy is so confusing. np.sum(A) is twice as slow as A.sum()

10:52 <mattip> what is A.shape, A.dtype ?

10:57 <lessshaste> mattip, just tried with a bigger example

10:57 <lessshaste> In [11]: A.shape

10:57 <lessshaste> Out[11]: (10000,)

10:58 <lessshaste> In [12]: A.dtype

10:58 <lessshaste> Out[12]: dtype('float64')

10:58 <mattip> small examples suffer from the overhead of np.asarray(A) inside np.sum

10:58 <lessshaste> does 10,000 count as small?

10:59 <mattip> basically np.sum(A) is "np.asarray(A).sum() but it has dispatching to allow __array_function__ overrides

10:59 <lessshaste> but what is np.asarray(A) doing given that it is already an numpy array?

10:59 <mattip> this override code has been moved to C to make it faster in the dev version of numpy

11:02 <mattip> https://numpy.org/devdocs/release/1.25.0-notes.html#array-function-machinery-is-now-much-faster

11:19 <lessshaste> mattip, well that's interesting!

11:40 <lessshaste> mattip, it's all very confusing as np.dot(A, B) is faster than np.sum(A)

11:42 <mattip> if you have less than a million elements, I think you are mainly measuring overhead of setting things up

11:42 <mattip> and the difference is microseconds

11:49 <lessshaste> mattip, the problem is that I do this millions of times . What setup is add.reduce(A) or A.sum() doing that np.dot is not?

11:51 <lessshaste> mattip, you are right though. If I set the arrays to length a million than np.dot is very slightly slower

13:11 <mattip> dot is special. It is different from matmul, and broadcasts over the last 2 dimensions for ndarrays with D>2

13:11 <mattip> so it does not allow __array_ufunc__ overloading