00:42 <discocaml> <elliottcable> a large complicated record from a library

01:08 <discocaml> <darrenldl> the library doesnt give any accessor or whatever to manipulate the data at all?

01:26 <companion_cube> (alcotest always sounds so annoying)

01:28 <discocaml> <elliottcable> it definitely does; but then i'm reduced to a painful deconstruction and manual pretty-printing. bleh.

01:31 <discocaml> <darrenldl> i think im just confused about what the matcher should do by your description

01:32 <discocaml> <elliottcable> ideally i'd love a way to just completely ignore some subset of a construction, but there's just … no way to type that.

01:33 <discocaml> <elliottcable> again, simplest example is "one might want to match `Some blah`, desiring pretty error-messages about the `Some`-ness, without having to _also_ have tons of details and pretty-printing magic for the `blah`-ness"

01:33 <discocaml> <elliottcable> anyway, not that important, can just use a `bool` matcher and `Option.is_some`; just gives less clear messages when it fails. meh nbd.

01:36 <companion_cube> Maybe one day I should make an ezounit2 or something

01:36 <companion_cube> Or fork it to clean it up

02:08 <discocaml> <jumpnbrownweasel> it's hard to come up with a general purpose function signature that would do that. the check is custom, the error message is custom, and the pp is custom

02:08 <discocaml> <jumpnbrownweasel>

02:08 <discocaml> <jumpnbrownweasel> wouldn't it be simpler to make the check directly and `fail` the test with the message you want?

03:20 <discocaml> <elliottcable> eh probably. the bool approach worked for me

12:04 <Guest63> hi! was thinking about this method inspired by some papers I read for type safe matrix operations. Basically the idea (which is well known) is to encode invariants in parametric types ( the 'a of 'a t). So the idea is to define a ('a, 'row, 'col) matrix = { data  : 'a array ; n_rows : 'row; n_cols : 'col) with the constraint that 'row and

12:05 <Guest63> 'col are Peano integers type 'a s = Zero -> () s | Succ : 'a s -> 'a s s

12:06 <Guest63> but it's a bit inefficient in terms of memory to encode numbers like that

12:06 <octachron> That can work, but it is a bit painful, and you will also hit the fact that it gives you huge type.

12:06 <Guest63> wondering if doing so is worth it just to check the dimensions of matrices in common matrix ops

12:07 <octachron> https://github.com/Octachron/tensority uses a binary type-level encoding

12:07 <octachron> with a ppx.

12:07 <Guest63> ah nice!

12:07 <octachron> You can also generate dimension with a generative functor.

12:08 <Guest63> cool

12:10 <Guest63> you can create a custom syntax with a ppx it seems

12:19 <discocaml> <sienhopist> I have a list of filepaths and pipe each of them with `|>` to a function that calls an external command on the filepath and returns the command's stdout. I noticed that when I do this with 100 filepaths in a list, I get 100 external processes created almost simultaneously and the overall speed of the program is considerably slower than I expect it to be. How can I make it execute the external commands one by one to see if that will improve

12:22 <discocaml> <sienhopist> I have a list of filepaths and pipe each of them with `|>` to a function that calls an external command on the filepath and returns the command's stdout. I noticed that when I do this with 100 filepaths in a list, I get about 100 external processes created almost simultaneously and the overall speed of the program is considerably slower than I expect it to be. How can I make it execute the external commands one by one to see if that will im

12:24 <discocaml> <._null._> `|>` is just reverse application, so you have to look into the function you're applying your list to

12:30 <discocaml> <_terence_> @octachron

12:30 <discocaml> <_terence_>

12:30 <discocaml> <_terence_> Does using a `constraint` compulsory for what you want to define here ? https://github.com/Octachron/tensority/blob/master/lib/tensor.mli#L22

12:30 <discocaml> <_terence_>

12:30 <discocaml> <_terence_> Also, while I get what constraint does, do you have some resources about when to use it and its power ?

12:32 <discocaml> <octachron> The constraint is really a constraint here: it mostly here to ensure that the phantom type parameters always have the right shape.

12:33 <discocaml> <_terence_> You could not have use a GADT with a single constructor ?

12:33 <discocaml> <_terence_> You could not have used a GADT with a single constructor ?

12:34 <discocaml> <sienhopist> This is the function in question. I believe `Unix.open_process_args_in` is responsible for this but idk how to process the files sequentially

12:34 <discocaml> <sienhopist> https://cdn.discordapp.com/attachments/436568060288172044/1154032759807938660/duration.ml

12:40 <discocaml> <._null._> There's no list of files here

12:41 <discocaml> <sienhopist> That's just the function

12:42 <discocaml> <sienhopist> I use it like this for example `["file 1.mp4"; ... etc] |> List.filter_map ~f:get_duration`

12:44 <discocaml> <sienhopist> So imagine the list has 100 files and when I execute that ⬆️ I notice almost 100 `ffprobe` commands running the background, most of them zombies, but eventually they do finish and I get their results as expected. However the whole thing is a lot slower than I expected, about 9 times than the equivalent Go version. I'm thinking the problem is that it tries to execute `ffprobe` for almost every list element simultaneously

12:45 <discocaml> <sienhopist> So imagine the list has 100 files and when I execute that ⬆️ I notice almost 100 `ffprobe` commands running the background, most of them zombies, but eventually they do finish and I get their results as expected. However the whole thing is a lot slower than I expected, about 9 times slower than the equivalent Go version. I'm thinking the problem is that it tries to execute `ffprobe` for almost every list element simultaneously

12:46 <discocaml> <._null._> I don't see why your code would be concurrent, you're not using anything to

12:46 <discocaml> <._null._> I don't see why your code would be concurrent, you're not using anything to make it so

12:47 <discocaml> <sienhopist> Maybe it's got something to do with this in the `Unix` documentation page

12:47 <discocaml> <sienhopist>

12:47 <discocaml> <sienhopist> `create_process prog args stdin stdout stderr` creates a new process that executes the program in file prog, with arguments args. The pid of the new process is returned immediately; **the new process executes concurrently with the current process**

12:47 <discocaml> <octachron> @_terence_ , a GADT would have been heavier here: with the constraint, the type-level constraint is only reflected at the type-level and never affect values.

12:47 <discocaml> <sienhopist> It mentions that the new process is concurrent

12:48 <discocaml> <._null._> Sure, but `In_channel.input_line` should be blocking

12:50 <discocaml> <darrenldl> are you sure a single input line exhausts the output?

12:52 <discocaml> <sienhopist> There might be one empty line that I don't read from stdout

12:54 <discocaml> <darrenldl> then the process doesnt exit fully

13:02 <discocaml> <sienhopist> Same problem even with `In_channel.input_all`

13:05 <discocaml> <._null._> `Unix.close_process_in`

13:05 <companion_cube> I hate this API so much

13:09 <companion_cube> (tbh I really dislike most of what's related to the standard channels)

13:13 <discocaml> <sienhopist> Thanks, that's precisely what I was missing. That fixed all the background processes, but the execution time is the same, meaning this wasn't causing the slowness apparently

13:13 <discocaml> <sienhopist> In that case idk why it's so slow

13:29 <Leonidas> companion_cube: the API is also extremely weird, because the process is somehow encoded in the channels you get out of the tuple

13:29 <discocaml> <darrenldl> whats your go version like?

13:30 <Leonidas> What if you construct a second process and pass (ch_proc1, ch_proc2) to `close_process_in`? It probably throws

13:31 <companion_cube> it's terrible in so many respects

13:31 <discocaml> <darrenldl> it is an absolute hassle to use

13:31 <companion_cube> forces channels to have (more) hidden global state; no wait to get the PID while it's running

13:32 <Leonidas> I was also wondering why it didn't close the process, but turns out it fails if you don't read the output with SIGPIPE or something

13:33 <discocaml> <_terence_> gadt affects value because of the constructors ?

13:34 <companion_cube> omg, fuck SIGPIPE

15:16 <discocaml> <functionalprogramming> companion_cube: would you say having Xyz.Make() and passing in PARTIAL_ORD and TOTAL_ORD is a good idiom to follow, as you have done for your containers library? https://github.com/c-cube/ocaml-containers/blob/master/src/core/CCHeap.ml I am writing my own data structure and would like to know the best way to do it (crosspost from #beginners but i don't know if you're in that IRC channel)

15:17 <discocaml> <._null._> (there's only one ocaml IRC channel AFAIK and it's linked here)

15:18 <discocaml> <functionalprogramming> ah gotcha, good to know

15:18 <companion_cube> I'm mostly imitating Map.Make/Set.Make :)

15:18 <companion_cube> so yeah, it's a good idiom for these kinds of data structures with requirements on the tyype

15:18 <companion_cube> type

15:22 <discocaml> <functionalprogramming> i see, thanks!

16:08 <discocaml> <sienhopist> My apologies, I was misremembering that the Go version was multi-threaded. That's why the Go version finished in about 1 second vs Ocaml's 8 seconds in one of my tests. If I make the Go version single-threaded it goes up to 8 seconds too and Ocaml is around 200-300ms slower on average. I guess now the next objective is figuring out how to parallelize the Ocaml version

16:14 <companion_cube> if you use the threads to run subprocesses, the `Thread` module can be enough

16:14 <companion_cube> otherwise there's a handful of thread pools existing these days (esp. on OCaml 5)

17:51 <discocaml> <sienhopist> Can ocaml be compiled statically against musl?

17:54 <discocaml> <darrenldl> yep, i regularly do that

19:02 <discocaml> <sienhopist> Speaking of which what's the easiest way to create a thread pool and have it map a function or two to a list? Or perhaps map every element in a different thread instead of using a pool

19:04 <companion_cube> <disclavimer> I'm the dev </disclaimer> there's https://github.com/c-cube/moonpool/

19:05 <companion_cube> stuff like `List.map (fun x -> Moonpool.Fut.spawn ~on:my_pool (fun () -> compute_with x)) l`

19:08 <companion_cube> more directly: https://c-cube.github.io/moonpool/moonpool/Moonpool/Fork_join/index.html#val-map_list

19:35 <discocaml> <ironside2396> what kinds of ffi exist for OCaml ? only C ?

19:36 <companion_cube> basically

19:36 <companion_cube> and C++ via the C ABI

19:42 <discocaml> <Kali> isn't there also fortran ffi?

19:42 <discocaml> <Kali> https://ocaml.org/docs/calling-fortran-libraries

19:46 <discocaml> <Kali> oh, nevermind; it's still C, just for fortran object files

19:59 <discocaml> <sienhopist> Thanks I'll take a look when I get to ti

19:59 <discocaml> <sienhopist> Thanks I'll take a look when I get to it

21:09 <Ankhers> If I am just getting started with ocaml, should I worry myself with something like Core? Or should I stick with the standard lib that is shipped with ocaml?

21:10 <discocaml> <Kali> standard lib

21:11 <discocaml> <Kali> generally, you only use Core if you are:

21:11 <discocaml> <Kali> - Jane Street

21:11 <discocaml> <Kali> - using a package that depends on it

21:12 <Ankhers> So Core is not a package most ocaml developers will use outside of Jane Street? What is the reason more people do not use it? I was under the impression it was widely used.

21:12 <discocaml> <Kali> you probably got that impression from Real World OCaml, yes?

21:13 <discocaml> <Kali> it was at least partially written by someone from Jane Street

21:13 <discocaml> <Kali> in the second edition

21:14 <discocaml> <Kali> so they used core/base

21:14 <Ankhers> I believe so. It has been a while since I have read the book.

21:15 <Ankhers> Thank you for the information.

21:16 <discocaml> <Kali> if you want an extended Stdlib that does not make the radical changes that Base/Core does sometimes, check out Containers: https://github.com/c-cube/ocaml-containers

21:18 <Ankhers> I think I will just stick with the default stdlib for now and find my way around.

21:18 <discocaml> <Kali> this is reasonable