<discocaml>
<dinosaure> Then, you should easily switch and replace `miou` by `miou.with_picos` 🙂
jabuxas has quit [Quit: oops :p]
jabuxas has joined #ocaml
jabuxas has quit [Ping timeout: 255 seconds]
jabuxas has joined #ocaml
<discocaml>
<limp.biskit> you’ve posted before, but what was the issue needing picos to be censored in the first place ?
<discocaml>
<limp.biskit> *vendored
jabuxas has quit [Quit: oops :p]
<discocaml>
<dinosaure> I would like to keep miou dependency free. When it's about building unikernels, we do static link. To be helpful on this purpose, something like a scheduler (which is central for an application) should not depends on anything to be sure that at least and with nothing, we are able to make an unikernel.
<discocaml>
<dinosaure> So I'm not against picos but at this stage, goals (between picos & robur) are not totally aligned (and it's ok) but I would lot like to take the risk to depend on something which will possibly break (for legitimate reasons) the ability to build unikernels 🙂
<discocaml>
<dinosaure> So I'm not against picos but at this stage, goals (between picos & robur) are not totally aligned (and it's ok) but I would not like to take the risk to depend on something which will possibly break (for legitimate reasons) the ability to build unikernels 🙂
<discocaml>
<limp.biskit> i was not under the impression picos would need any system dependencies, while ocaml code is statically linked
<discocaml>
<dinosaure> Yes, and that's super cool but picos is also young 🙂
<discocaml>
<dinosaure> I mean, these questions is more about stability on the long term and, again, I will be happy to not do such trick
<discocaml>
<dinosaure> But I will definitely propose something on this way and provide a fully compatible miou version with picos
<discocaml>
<dinosaure> It just matters of distribution at this stage, I will ping you when it's done (do you have a GitHub login?)
<discocaml>
<dinosaure> Thanks, I will try to do something today 🙂
<discocaml>
<limp.biskit> so essentially, it’s just taking the bits of picos which are immediately useful now, and leaving full interoperability until there is a stable interface to interopt with?
<discocaml>
<limp.biskit> (that hopefully still meets miou and mirage’s needs)
<discocaml>
<dinosaure> Yes, several questions about picos are in suspensions and a full agreement by all is not done yet. My first goal was to integrate what is really needed for miou and validate the picos design. So we reach this step and this is cool but now it's more about an API design according Miou and others schedulers (like moonpool or lwt).
<discocaml>
<dinosaure> And, even if most of the picos design is good for miou, I still disagree on certain points. But the time give to us will improve the situation: we are at the baby step 🙂
<discocaml>
<limp.biskit> i imagine probably more schedulers are needed to sort of battle test picos’ ideas
<discocaml>
<dinosaure> Yes, actually only Miou did this work and I know that companion_cube started something in this way. But that's all at this stage.
<discocaml>
<limp.biskit> fuseau!
<discocaml>
<dinosaure> Ah yes 🙂 but yeah we need more
<discocaml>
<limp.biskit> i don’t know if riot implements any of picos either
<discocaml>
<limp.biskit> the nice thing about picos, if it were to reach adoption which i really hope, is that there will be a space for different ways of doing things
<discocaml>
<limp.biskit> eio annoyed people by being both heavily opinionated and the first and only useful option
<discocaml>
<polytypic> Related to the Picos design, I'm actually currently working to create a new "introduction" to the concepts to hopefully better communicate the ideas. I recently realized I have sort of miscommunicated some key ideas (e.g. what `Computation` is all about) and there are also a lot of details that likely aren't obvious on a quick look (e.g. the design of the `Trigger` mechanism, which is basically 3rd iteration on how to suspend fibers). Hope
<discocaml>
<limp.biskit> i really need to sit down and read the picos docs some time
<discocaml>
<polytypic> If you do, please report anything that you find confusing. It is very difficult (as an author of a library) to see places where some key explanation might be missing.
<discocaml>
<limp.biskit> a goal for me would be a library developed without any specific scheduler in mind
jabuxas has joined #ocaml
<discocaml>
<limp.biskit> that could then run under any picos scheduler with changes only for IO
<companion_cube>
That's totally the idea
<discocaml>
<limp.biskit> yeah, realised while i was typing
<companion_cube>
Part of why polytypic works on picos, I think, is to use it as a basis for his k-cas library
<discocaml>
<limp.biskit> but do you think it’s at the stage where an application level library could/should use it now?
<discocaml>
<polytypic> IO can also be "just a library" with Picos (that you can use with any Picos compatible scheduler). There is a sample `Picos_stdio` library that currently provides a "transparently asynchronous" version of the `Unix` module: <https://ocaml-multicore.github.io/picos/doc/picos/Picos_stdio/index.html>. In the future we hope to provide IO libraries based on io_uring and other asynchronous IO interfaces.
<discocaml>
<polytypic> Picos is not 1.0 yet and likely won't be for some time. I'd love to see people experiment with it, and give feedback, but I cannot recommend people to start writing major applications with it, yet, because it is quite likely that we will make some changes.
<discocaml>
<limp.biskit> think i’ll have a go at translating some of my miou code
<discocaml>
<limp.biskit> that is after i’ve finished making it work on miou 😅
<discocaml>
<limp.biskit> uring would be a dream
<companion_cube>
Eventually we'll need a picos for IOs too
<companion_cube>
If we want to write scheduler agnostic libraries
<discocaml>
<limp.biskit> could that be achieved at a higher level?
<discocaml>
<limp.biskit> gluten is what comes to mind
<companion_cube>
Is that a buffer management library? I can never remember
<discocaml>
<limp.biskit> yeah
<companion_cube>
idk, I'm not sure there's a lot of examples from other ecosystems either
<discocaml>
<limp.biskit> it’s used in httpun
<companion_cube>
We need common interfaces for a lot of things, it might be unrealistic :/
<discocaml>
<limp.biskit> the only thing with io is that at some point you’re gonna abstract away something people actually wanted
<companion_cube>
Fair enough
<companion_cube>
We could still have epoll/kqueue/... Bindings compatible with `Unix` and that rely only on picos for fibers and cancelation
<discocaml>
<limp.biskit> having individual libraries (e.g. a binding for freebsd’s io) that work asynchronously with picos would be fantastic
<discocaml>
<zornthimmberglobber> Getting banned from every programming, discord server makes programming harder on on the other hand. It does make it easier since now I can get to coding without asking questions.
jabuxas has quit [Ping timeout: 246 seconds]
waleee has quit [Ping timeout: 255 seconds]
bartholin has joined #ocaml
xenu has quit [Ping timeout: 268 seconds]
chiselfuse has quit [Ping timeout: 260 seconds]
chiselfuse has joined #ocaml
xenu has joined #ocaml
YuGiOhJCJ has joined #ocaml
Serpent7776 has joined #ocaml
hannes has joined #ocaml
gooby323 has joined #ocaml
toastal has joined #ocaml
<gooby323>
Those who've had industrial experience with OCaml, do you think it could be suicide for a small company to adopt OCaml as their language of choice? I'm thinking how smaller companies cannot afford to maintain their own tools and libs, so maybe they would eventually hit an ecosystem wall
<discocaml>
<bluddy5> It really matters which domain the company is in. The more fleshed out those parts of the ecosystem are, the easier it'll be. Finding programmers is going to be a real challenge.
reynir has joined #ocaml
quernd80 has quit [Read error: Connection reset by peer]
quernd801 has joined #ocaml
remexre has quit [Ping timeout: 264 seconds]
remexre has joined #ocaml
reynir has quit [Ping timeout: 255 seconds]
reynir has joined #ocaml
<gooby323>
Yeah fair enough, it might be the best to keep OCaml to a smart part of the backend where the safety really shines, while reducing amount of dependencies and maintainers you need
reynir has quit [Ping timeout: 260 seconds]
reynir has joined #ocaml
olle has joined #ocaml
<discocaml>
<drupyog> At some point, you're better off juste writing the semantics 🙂
hannes has quit [Quit: leaving]
<discocaml>
<drupyog> (it would actually be a rather good idea eventually)
<discocaml>
<drupyog> (it would actually be a rather good idea eventually, there are several people who would be interested in formally verifying it, if it becomes the "standard async library"
<discocaml>
<drupyog> (it would actually be a rather good idea eventually, there are several people who would be interested in formally verifying it, if it becomes the "standard async library")
<discocaml>
<drupyog> (it would actually be a rather good idea eventually, there are several people who would be interested in formally verifying it, if it becomes the "standard async base library")
<gooby323>
Wym by writing the semantics exactly?
<discocaml>
<polytypic> Yes and no. It is quite possible we'll create a formal semantics for Picos. It has been talked about. However, you can have a formal semantics of any design - good or bad. It tells you nothing about why things are like they are. It just tells you how things work.
<discocaml>
<polytypic> And what I want to primarely communicate is the why.
<discocaml>
<polytypic> And what I want to primarily communicate is the why.
<discocaml>
<polytypic> Why, for example, Trigger is first-order. Why it asks you to allocate a trigger before and then only later perform an effect. And so on...
<discocaml>
<drupyog> I'm not sure the why is really that useful for potential users of picos, as much as the "what is the semantics and how can I project the behavior I want into it"
<discocaml>
<polytypic> Oh... We can and do have both.
<discocaml>
<.armael.> I think the why and the how are two different things, that are both good to document
<discocaml>
<polytypic> Exactly. It is not a dichotomy.
<discocaml>
<drupyog> sure, but if you want to attract/convince other implementors, I don't think "why" is really that important.
<discocaml>
<polytypic> I honestly disagree.
<discocaml>
<polytypic> The documentation has been very explicit about how things work and the code is simple enough that you can figure out exactly how things work.
<discocaml>
<polytypic> People don't really ask about those.
<discocaml>
<polytypic> What they ask about is things like "I don't like how X works, I'd like it to work like this".
<discocaml>
<polytypic> And the answer to such questions is that "it is like that because ..." and that is the why.
<discocaml>
<polytypic> And the feedback is usually of the form "I don't like how X works, I'd like it to work like this".
<discocaml>
<drupyog> isn't that another way of saying "I want to encode feature X and 1) I don't know how to 2) I'm worried that it will not be fast enough" ?
<discocaml>
<polytypic> And the answer to such feedback is that "it is like that because ..." and that is the why.
<discocaml>
<polytypic> I don't think the formal semantics would help with those.
<discocaml>
<drupyog> oh no, it wouldn't
<discocaml>
<drupyog> I didn't say that ^^'
<discocaml>
<polytypic> Usually the answer is that you need to add a level of indirection. That is how you usually solve those issues.
<discocaml>
<polytypic> And, yes, you mention performance, which is one of the things that semantics doesn't really answer and is also one of those things that is a key behind many details.
<discocaml>
<drupyog> What I mean is that if someones come saying "I want to encode X and I don't understand why your API is like that", the why is weak-ish argument. The piece of code encoding it and benchmark showing it goes fast is much stronger for most people.
<discocaml>
<polytypic> That is the motivation behind the sample schedulers and libraries.
<discocaml>
<drupyog> even tho It's still very interesting, of course ! 😉
<discocaml>
<polytypic> Questions on how to implement things are more than welcome. But, yeah, I'd suggest the Picos Discord/GH for such questions (I don't want to spam this forum with Picos).
<discocaml>
<polytypic> That is one of the main motivations behind the sample schedulers and libraries.
chiselfuse has quit [Ping timeout: 260 seconds]
chiselfuse has joined #ocaml
<discocaml>
<lukstafi> Or #concurrency (wouldn't be spamming)
hannes has joined #ocaml
toastal has quit [Ping timeout: 240 seconds]
<discocaml>
<kayceesrk> I've been looking to write down the semantics for Picos. Writing semantics for this is "interesting" since Picos deliberately tries not to describe what the scheduler is. We would like to define the scheduler to be as general as possible. The analogy I have in mind is that you'd like to formally describe the semantics of Mutex and Condition variables without specifying much about threads or the scheduler.
reynir has quit [Ping timeout: 256 seconds]
masterbuilder has quit [Remote host closed the connection]
YuGiOhJCJ has quit [Quit: YuGiOhJCJ]
troydm has quit [Ping timeout: 264 seconds]
gooby323 has quit [Quit: Konversation terminated!]
ocra8 has joined #ocaml
<discocaml>
<dinosaure> Mutex and condition requires an idea of a fiber (to be able to discard the case where a fiber wants to lock and relock). My feeling is more related to the idea of the suspension (whatever is the ordering of suspension). Actually, Miou trusts only on the suspension mechanism and uses picos for that. Depending on what you want to describe but Miou just need: 1) the suspension mechanism 2) a way to the describe uniquely fibers
reynir has joined #ocaml
<discocaml>
<kayceesrk> The observation that I was trying to make is that the semantics of picos primitives should be rich enough that one should be able to describe the semantics of mutex and condition variables using the semantics of picos primitives.
<discocaml>
<polytypic> Yes, the specifics of the trio of the concepts `Trigger`, `Computation`, and `Fiber` is not just to give a way to suspend and resume, but to allow efficient implementation of various things in a manner that is independent of the scheduler.
<discocaml>
<polytypic>
<discocaml>
<polytypic> Mutex and Condition are a good example, because they require many of the features. Stuctured concurrency requires a slightly different set of features.
<discocaml>
<polytypic>
<discocaml>
<polytypic> One thing I mentioned here is that I realized recently that I had miscommunicated the idea of the `Computation` concept of Picos. I previously described it primarly through various analogies (cancelation context, promise, ...). What it really is all about is the omnidirectional ability to be notified of the status change from "running" to "completed". IOW, it is lock-free bag of triggers. It adds the ability to communicate a value or an
<discocaml>
<dinosaure> Yes, you are right but from my point of view, the ability to choose the order policy of tasks is the degree of liberty of picos. Not sure sure how to express that as a proof but it's definitely the game change: we can espress something like moonpool and something like miou and they have different policies 🙂
<discocaml>
<polytypic> The `Fiber` concept has also proven difficult to communicate. From the POV of someone programming against the Picos interface it is just a way to efficiently share certain aspects of the (mutable) state of a fiber with the scheduler (i.e. to be able to tell the scheduler what the fiber wants that state to be at any point) along with the identity of the fiber. The specific way it is implemented, i.e. as a simple abstract data type (internal
<discocaml>
<polytypic> Yes, the specifics of the trio of the concepts `Trigger`, `Computation`, and `Fiber` is not just to give a way to suspend and resume, but to allow efficient implementation of various things in a manner that is independent of the scheduler.
<discocaml>
<polytypic>
<discocaml>
<polytypic> Mutex and Condition are a good example, because they require many of the features. Stuctured concurrency requires a slightly different set of features.
<discocaml>
<polytypic>
<discocaml>
<polytypic> One thing I mentioned here is that I realized recently that I had miscommunicated the idea of the `Computation` concept of Picos. I previously described it primarily through various analogies (cancelation context, promise, ...). What it really is all about is the omnidirectional ability to be notified of the status change from "running" to "completed". IOW, it is lock-free bag of triggers. It adds the ability to communicate a value or an
<discocaml>
<limp.biskit> btw @dinosaure - would a patch to build httpcats on the master branch of httpaf be useful to you?
waleee has joined #ocaml
hannes has quit [Quit: leaving]
<discocaml>
<dinosaure> Yes, we currently are in the pipeline `Cstruct.t -> string` pull-requests, we would like to finish it before anything
<discocaml>
<yawaramin> that's what the company i work at did (before my time though)–their trading stuff is C++ but they wrote a DSL/compiler in OCaml
jabuxas has joined #ocaml
olle has quit [Ping timeout: 252 seconds]
jabuxas has quit [Ping timeout: 260 seconds]
waleee has quit [Ping timeout: 256 seconds]
<discocaml>
<limp.biskit> i have it running in dev. if you still then ping me and ill rebase
bibi_ has joined #ocaml
Tuplanolla has joined #ocaml
<discocaml>
<limp.biskit> btw great work, ported a framework over from eio and the coding style is much much nicer and has lead to stabler code