02:35 <discocaml_> <dinosaure> A solution is to make a new `Format.formatter` and ensure that this one is used by only one domain (the domain which prints out things).

02:36 <discocaml_> <dinosaure> I did a patch for `alcotest` recently about that where the user can use something else than `Format.std_formatter` for `alcotest`'s printers

02:37 <discocaml_> <dinosaure> And let something else be able to use `Format.std_formatter` for another purpose (like logs)

02:37 <discocaml_> <dinosaure> https://github.com/mirage/alcotest/pull/399

02:38 <discocaml_> <dinosaure> So the issue is not strickly related to `progress` but much more about `Format` which is completely not domain-safe

03:29 <companion_cube> it's domain safe if you don't use its global parts, right?

04:10 <discocaml_> <dinosaure> Yes. `std_formatter` can not be used across domains but you can have multiple formatters to `stdout` 🙂

04:13 <companion_cube> race conditions anyway, put a lock around it

08:35 <discocaml_> <kakadu18> Am I understanding correctly, that this will require a change in the progress's code base?

08:37 <discocaml_> <kakadu18> (I have some temptation to add a formatter argument to report function in progress, but don't know where it will lead me)

08:39 <discocaml_> <dinosaure> Depending on your code, a mutex is probably needed if the function which prints out is called by several domains.

08:40 <discocaml_> <dinosaure> Use another formatter is required if `std_formatter` is used by `progress` into a domain and something like `logs` uses it too in another domain.

08:44 <discocaml_> <dinosaure> And actually, you can specify the formatter you want to use with `Config.v`

08:47 <discocaml_> <dinosaure> So I don't think that we need to tweak `progress`. I don't have my computer now but, as far as I can tell, everything is here to avoid a data-race 🙂

08:49 <discocaml_> <kakadu18> Many thanks, I'm going to experiment with this.

09:27 <discocaml_> <kakadu18> I managed to get a desired result with a mutex. Thanks.

09:51 <discocaml_> <polytypic> Note that the Stdlib Mutex does not work well with fibers, because the Stdlib Mutex blocks the entire systhread/domain. I'm hoping to soon rewrite Domainslib internals and make it Picos compatible.

11:34 <companion_cube> Wait what

11:35 <companion_cube> It only blocks the thread

11:35 <companion_cube> Ah yeah, right. The / confused me

11:40 <discocaml_> <polytypic> Yes, I often use thread/domain, because it seems a lot of newcomers don't necessarily know about both of them. In the case of Domainslib, for example, Domainslib spawns domains. Using a Stdlib Mutex in a task running on Domainslib is typically a bad idea, because it prevents Domainslib from running other tasks on that domain.

11:42 <companion_cube> Yeah. I think Mutex still makes sense for short critical sections that won't contain IO or an await though

11:45 <discocaml_> <polytypic> Hmm... Well, sometimes you might not really have significantly better options conveniently available, but generally speaking, in the future, nobody should be using the Stdlib Mutex or Condition except in very specific circumstances when you really want to block the entire systhread. (Like in a scheduler when there is literally nothing else to do.)

11:46 <discocaml_> <polytypic> (Unless the Stdlib Mutex and Condition are fixed to actually allow a scheduler to run other fibers. Like with the Picos Mutex <https://ocaml-multicore.github.io/picos/doc/picos/Picos_sync/Mutex/index.html> and Condition <https://ocaml-multicore.github.io/picos/doc/picos/Picos_sync/Condition/index.html>.)

12:01 <companion_cube> Still don't see why a Mutex would be bad if there's no await inside

12:19 <discocaml_> <polytypic> You might be (and in the future you typically should be) running on a scheduler that could run other fibers while that mutex is owned by someone else. And not just the current owner, because mutexes typically are designed to be fair and so they hold a queue of awaiters, so there could be a queue of other fibers waiting for the mutex.

12:19 <discocaml_> <polytypic>

12:19 <discocaml_> <polytypic> The Stdlib Mutex, as it is, stops the entire systhread so that nothing can run on it (not even signal handlers written in OCaml).

12:19 <discocaml_> <polytypic>

12:19 <discocaml_> <polytypic> Now, perhaps you have a situation where the operations being run with a mutex are extremely short and the mutex is not highly contested. In such a case even the Stdlib Mutex could be ok. But, then, a "user space" (spinlock or a) mutex like the Picos Mutex would be preferable, because it can be faster and need not suffer from the problem of blocking the entire domain while being queued to become the owner of the mutex.

12:19 <discocaml_> <polytypic>

12:19 <discocaml_> <polytypic> Also, if the operations are very short, like adding a thing to a queue, then you probably would be better off using an off-the-shelf non-blocking data structure.

12:19 <discocaml_> <polytypic>

12:19 <discocaml_> <polytypic> So, TBH, I really don't see many good use cases for a pthread mutex (which is what the Stdlib Mutex is) that stops the entire systhread. If the operations are really short, it could be Ok, but then non-blocking data structure would likely be faster and better (better progress properties) and if it is long running (whether with await or not inside) then a user space mutex will be superior.

12:22 <discocaml_> <polytypic> IMHO, the Stdlib Mutex and Condition are kind of broken just like the Stdlib Domain in that they don't work cooperatively. They are fine for low level things to implement a more cooperative layer on top, but they just don't do the right thing in a future with effects based schedulers.

12:24 <discocaml_> <polytypic> (And the same goes for Stdlib Lazy.)

12:31 <discocaml_> <polytypic> You might be (and in the future you typically should be) running on a scheduler that could run other fibers while that mutex is owned by someone else. And not just the current owner, because mutexes typically are designed to be fair and so they hold a queue of awaiters, so there could be a queue of other fibers waiting for the mutex.

12:31 <discocaml_> <polytypic>

12:31 <discocaml_> <polytypic> The Stdlib Mutex, as it is, stops the entire systhread so that nothing can run on it (not even signal handlers written in OCaml).

12:31 <discocaml_> <polytypic>

12:31 <discocaml_> <polytypic> Now, perhaps you have a situation where the operations being run with a mutex are short and the mutex is not highly contested. In such a case even the Stdlib Mutex could be ok. But, then, a "user space" (spinlock or a) mutex like the Picos Mutex could be preferable, because it can be faster and need not suffer from the problem of blocking the entire domain while being queued to become the owner of the mutex.

12:31 <discocaml_> <polytypic>

12:31 <discocaml_> <polytypic> Also, if the operations are very short, like adding a thing to a queue, then you probably would be better off using an off-the-shelf non-blocking data structure.

12:31 <discocaml_> <polytypic>

12:31 <discocaml_> <polytypic> So, TBH, I really don't see many good use cases for a pthread mutex (which is what the Stdlib Mutex is) that stops the entire systhread. If the operations are really short, it could be Ok, but then non-blocking data structure would likely be faster and better (better progress properties) and if it is long running (whether with await or not inside) then a user space mutex will be superior.

12:36 <discocaml_> <polytypic> But to reiterate what I said previously... Currently better options (non-blocking data structures or user space mutexes) aren't necessarily conveniently available. So, don't worry too much about using the Stdlib Mutex, but be aware that you probably want to use something else in the future.

13:32 <companion_cube> Please stop editing details on this channel, it's a lot of text on the irc side 😅

13:33 <companion_cube> Well if I want to update a couple things, a Mutex is still the simplest thing for now

13:33 <companion_cube> I know kcas is what you think should replace it but sometimes simple is good too

14:01 <discocaml_> <polytypic> Sorry, I forgot that the edits are sent as separate messages for the IRC users. Will try to remember to use a separate editor in the future on this channel.

14:03 <companion_cube> no worries

14:04 <companion_cube> (my point is, anyway: if you don't have cooperation within the critical section then I'm not sure why Mutex is bad)

14:05 <discocaml_> <polytypic> > kcas is what you think should replace it

14:05 <discocaml_> <polytypic> Well... For a lot of things, yes, but I wasn't really thinking of that here specifically. Just that the Stdlib Mutex and Condition don't currently do the right thing when it comes to effects based schedulers which I expect to become the norm in OCaml in the future.

14:07 <companion_cube> I 99% agree on conditions! just not on mutex :p

14:07 <companion_cube> (*in some specific cases)

17:33 <discocaml_> <yawaramin> would it be possible to tell the IRC bridge to post edits in the form of diffs or `s/.../.../` rather than reposting the new version of the entire message?

17:36 <companion_cube> not that I know of

19:11 <discocaml_> <sim642> Computing diffs as sets of sed replacements is an interesting challenge

20:36 <discocaml_> <akhilindurti> is there a proposal to add a function composition operator to the stdlib? i.e. `('a -> 'b) -> ('c -> 'a) -> 'c -> 'b`

20:43 <discocaml_> <sim642> Fun.compose exists since OCaml 5.2

20:44 <discocaml_> <sim642> Not an operator though