01:30 <discocaml> <logoooo> Is there a way to read a file encoded in DOS format (i.e with CRLF line terminators) and convert into the Unix format? I thought opening with ``In_channel.open_text`` and input_all would work, but no luck...

01:31 <discocaml> <logoooo> #beginners is probably a better spot, so tell me if I should move there

01:35 <discocaml> <hockletock> text mode should convert unless you are running on a platform with CRLF endings, is that the case?

01:36 <discocaml> <logoooo> I am on Linux

01:48 <discocaml> <logoooo> should I run set_binary_mode?

01:48 <discocaml> <hockletock> It appears I misunderstood, it only considers the \r part of the newline if running on windows and open in text mode.

01:51 <discocaml> <hockletock> I think set_binary_mode has the same effect as opening in text mode

01:53 <discocaml> <hockletock> is preformatting the file with dos2unix an option? The alternative, I think, is to copy the leading substring of every line, after checking that the last character is indeed \r -since the last line may not have \r\n at the end if you don't check every time you could lose a character

01:56 <discocaml> <hockletock> or concatenating all the lines together and using split_on_char '\r' to get a list back

01:58 <discocaml> <logoooo> I would also have to remove the BOM as well

01:58 <discocaml> <hockletock> dos2unix can strip the BOM I think

01:58 <discocaml> <logoooo> Yeah, it does. I would prefer not to have to rely on an external command though.

01:59 <discocaml> <logoooo> I think I might be able to just skip over the header though

02:08 <discocaml> <logoooo> Yeah, that worked 🙂

02:08 <discocaml> <logoooo> Thanks for your help

02:22 <discocaml> <cemerick> companion_cube: just using miou's terminology, where each effect performed is a _quanta_, which ends up being the granularity of work (rather than only being able to talk about higher-level tasks

02:22 <discocaml> <cemerick> That's just my understanding after some reading, I haven't tinkered yet

07:14 <discocaml> <dinosaure> I think you're starting to think about the famine problem that is solved by the round-robin scheduler 🙂 . Basically, it's difficult to schedule correct tasks over a limited number of domains (the task dependency graph is not complete - a task can create new tasks). So you can have a starvation problem where all the domains are busy executing tasks that are waiting for each other. With the notion of "quanta", because we limit tasks in thei

07:17 <discocaml> <dinosaure> On the subject of parallelization, several questions arise (such as memory sharing essentially) but the idea of "splitting" a large task into several small tasks is in fact a fairly basic and simple idea to implement with Miou (or Moonpool): it's the idea of "fork & join" proposed by the "parallel" function: https://docs.osau.re/miou/Miou/index.html#val-parallel

07:25 <discocaml> <dinosaure> But once again, Miou is more interested in availability. We can find an optimal order for the execution of so-called 'pure' tasks (which have no interaction with the system). We can then make a clever calculation by prioritising some tasks over others in order to obtain the result as quickly as possible. The problem with events in the system is that they distort our calculations: a connection can arrive in a second or an hour. If there are t

07:28 <discocaml> <dinosaure> The downside is that Miou can sometimes waste time. It may try to carry out a task when it's only waiting for another task (at the end of our to-do list). Once again, it doesn't prioritise tasks! And I think, companion_cube should confirm, that in this kind of case, Moonpool is more interesting.

08:54 <hannes> I'm still curious about long-running pure tasks -- such as "find a 32k bit prime" -- since that task won't yield (an effect), it'll be scheduled until finished... now can we move such a piece of code into a more fair shared environment? I suspect we'd need preemption for that, and basically interrupt the computation after a certain time.

08:55 <hannes> but: would that be easy with e.g. miou? (i.e. inject an effect into a task after some running time)?

09:03 <discocaml> <dinosaure> this problem exists indeed and, as `lwt` (where you have the same issue), the only way to reschedule is to use `Miou.yield ()` (which performs an effect too)

09:05 <discocaml> <dinosaure> And indeed, the usual choice for the quanta is time but I think it's hard to apply such quanta in OCaml. The `Effect.perform` seems the best tradeoff between the simplicity of the implementation (of Miou) and the availability of applications

09:08 <discocaml> <dinosaure> now for real pure tasks, yes, `Miou.yield ()` is required. Note that the merkle tree tutorial shows that even `Miou.call{,_cc}` performs an effect and reschedule tasks - in my opinion, the _granularity_ to reschedule tasks in Miou is much better then `lwt` (where the `bind` does not suspend)

09:10 <discocaml> <dinosaure> now for real pure tasks, yes, `Miou.yield ()` is required. Note that the merkle tree tutorial shows that even `Miou.call{,_cc}` performs an effect and reschedule tasks - in my opinion, the _granularity_ to reschedule tasks in Miou is much better than `lwt` (where the `bind` does not suspend)

09:36 <hannes> I agree that using effect (instead of time) has advantages -- esp. since we're still in the cooperative task environment (in contrast to the preemptive world)

11:46 <companion_cube> My view is probably more naive or old school, but I'd rather separate the part with effects and IO (lwt, eio, miou?) from the part where I run heavy non yielding computations (domainslib, moonpool...)

11:46 <companion_cube> Also bind not yielding in lwt can help with latency sensitive tasks, I think

11:53 <discocaml> <dinosaure> Yeah, it still depends on what you want to do always. I mean, `lwt` does a good job but not the best 🙂 . You always have some pros and cons about the scheduler policy. Again, the right thing to do is to explain them to the user and say that probably Miou is not your scheduler for certain applications (as eio or moonpool).

12:02 <companion_cube> Yeah I mean, why restrict oneself to a single scheduler? :)