12:21 <discocaml> <grianneau> ing

13:58 <discocaml> <barconstruction> Companion_cube, you are singling out the mention of polars because rust is faster for numerical computation than OCaml. First, dataframes are used for more than numerical computation and linear algebra, it is half this and half a database query engine. Second I do not know why you chose to focus on polars and not pandas and numpy which are far bigger, obviously python is much slower than Rust or OCaml but this has not stopped it from b

14:02 <leah2> those are not written in python, at least not the critical parts

14:07 <discocaml> <barconstruction> Yes, this is what I meant when I said "the numerical routines are written in C just as in Numpy". I.e. numerical routines in Numpy are written in C.

14:07 <discocaml> <barconstruction> Still, using Numpy imposes constraints on what is possible to write in a performant way because one can only use what is given through the API. Owl gives substantial more flexibility because one does not have to write every line of code as a library call in order to have good performance.

14:09 <discocaml> <bluddy5> Owl needs to be broken up into easily digestible pieces. Dream has shown how to make these things work.

14:13 <companion_cube> But do you actually get good performance out of owl?

14:14 <companion_cube> I was talking about polars because it also has Arrow support, for example. It's not just numeric crunching. And supporting Arrow in ocaml would also not be the greatest thing (there are bindings, though)

14:14 <discocaml> <regularspatula> For me, if Owl would be comparable to R then it would be good enough. Of course speed critical routines in R are also written in C....but I have tons of work where "slow" R code is fine

14:14 <discocaml> <regularspatula> in terms of speed i mean

14:18 <discocaml> <barconstruction> I had problems with Owl last time I used it but these were primarily problems with an OCaml garbage collection bug. I also had to do some unnecessary duplications to work around an owl bug. I fixed the owl bug but my PR was submitted after they stopped maintaining it. So... Inconclusive 🤷

15:51 <discocaml> <bluddy5> companion_cube: yes. You get performance like numpy. If you also have a lot of OCaml code, that part will obviously be faster than the equivalent python.

15:51 <discocaml> <bluddy5> Owl has one issue whereby it does less sharing of underlying BigArrays than numpy does.

16:01 <palainp> §3

16:01 <palainp> (sorry)

16:04 <discocaml> <mseri.> I agree with this sentiment. I think that having a coherent api between owl and owl_based and proper packaging of the various part could be done properly without jeopardising the initial philosophy since they can still live together in the same repo

16:05 <companion_cube> in my uninformed opinion, OCaml should have some sort of numpy compat layer and try to talk to python for this stuff

16:06 <discocaml> <mseri.> It gets slower if you do lots of small array initialisations, but this is because the way the bigarray is created and transformed and if I recall correctly also due to the fact that blit is a copy and not a view. In principle it can be sped up sensibility by moving it to a c stub though

16:06 <companion_cube> or reuse the C bindings that already speak numpy

16:06 <discocaml> <mseri.> You could send owl arrays to python in the past

16:07 <discocaml> <mseri.> This is harder, some of them really use numpy internal stuff that would be problematic in ocaml

16:07 <companion_cube> then there needs to be a library that does that, and only that

16:07 <discocaml> <mseri.> I don’t know, numpy is a layer over blas as much as owl

16:07 <discocaml> <mseri.> You just need to talk to blas/lapack to be interoperabke in most cases and this is already there

16:08 <discocaml> <mseri.> Also python world has a number of drop-in replacements of numpy, you want jax numba or whatnot very often to speed up code

16:10 <discocaml> <mseri.> Having a thin ocaml shim over numpy imho is slower than directly calling the same libraries and, since they are the same, you can take your arrays and send them to python with pyml or to c with whatever thin stub you prefer

16:10 <discocaml> <mseri.> You could use matplotlib to plot from owl with little effort (when pyml was happy with your python)

16:11 <discocaml> <mseri.> So I think that is already there

16:11 <discocaml> <mseri.> Personally what I like of owl is that I get less dependency failures than with python

16:13 <discocaml> <mseri.> Jupyer nktebooks are currently broken in many systems because of weird incompatibilities between python libraries, even with anaconda

16:14 <discocaml> <mseri.> I had to waste an enormous amount of time to explain our students how to fix it

16:15 <companion_cube> ok where is the best place to rant

16:15 <companion_cube> I need to rant

16:15 <discocaml> <mseri.> 😄

16:16 <discocaml> <mseri.> If you were on discord we could make a rant thread

16:18 <octachron> Such sad days when one can rant no long in #ocaml nor #ocaml-fr.

16:19 <discocaml> <mseri.> ahahahah

16:22 <companion_cube> better to hide it in a thread 😅

17:07 <companion_cube> waiiiiit, has `Stack.t` changes???

17:07 <companion_cube> changed8

17:12 <discocaml> <regularspatula> python compatibility (or whatever your favorite numerical computing kit language) would be great in a more foundational library...pyml says it can do copy-free interop with numpy, but I haven't tested it out speed-wise

17:39 <companion_cube> is this all another lesson in "actually putting stuff in the stdlib is bad"? :/

17:39 <discocaml> <bluddy5> yep

18:04 <discocaml> <deepspacejohn> Well, the example a Stack.t change in the "last few years" was from 9 years ago. And that change was pretty minor.

18:06 <companion_cube> and about strings and arrays I'm not sure what he means

18:06 <companion_cube> I mean besides float arrays, but that's just not visible from within OCaml

18:47 <discocaml> <bluddy5> whaddaya think https://github.com/ocaml/ocaml/pull/12931#issuecomment-1957678759 ?

18:54 <companion_cube> tbh they never really deprecate+remove anything

18:55 <companion_cube> the *one* historical exception was mutation in strings

19:36 <discocaml> <bluddy5> Sure, but just as *theoretically* abstraction allows them to change the representation, creating your function *doesn't really* tie down the current representation at all. It would only do so if they committed to always give you access to the raw internal representation as it is. Not only is this not the case (nothing in the types can guarantee that), but also a comment clarifying this isn't the case would make the situation even clearer.

19:39 <companion_cube> honestly it would cement the current implementation if you guarantee performance

19:45 <zozozo> companion_cube: reading these discussions, I'm almost tented to have code with conditional compilation that just uses unsafe/magic feature to access the internal representation and do what's needed without having to move the stdlib, :p

19:48 <companion_cube> kind of yeah

19:49 <companion_cube> I mean I'd do that if there was a `bytes` function added

19:49 <companion_cube> (cause then I'd be able to guaranteed its continued existence)

19:49 <companion_cube> (I mean I'd provide that on older version probably)

21:38 <discocaml> <sim642> Batteries has a bunch of such magic unsafe things that assume certain Stdlib representation. It's not nice but it has worked surprisingly well since Stdlib internals change so rarely

21:56 <Anarchos> how to run tests only of one specific file with dune ?

23:49 <discocaml> <struktured> https://dune.readthedocs.io/en/stable/tests.html#running-a-single-test