szkl has quit [Quit: Connection closed for inactivity]
szkl has joined #ocaml
dextaa_ has quit [Remote host closed the connection]
hackinghorn has joined #ocaml
rgrinberg has joined #ocaml
adanwan has quit [Ping timeout: 240 seconds]
adanwan has joined #ocaml
<ns12>
Hello, when I use the "Thread" library in Linux or FreeBSD, are the threads POSIX threads?
<companion_cube>
yes, but there's a runtime lock, just so you know
<companion_cube>
(we are all awaiting impatiently the release of OCaml 5.0, where a new abstraction will give us true multicore)
<ns12>
companion_cube: The docs say "Lightweight threads for Posix 1003.1c and Win32". Are there non-lightweight threads too? Why specifically "lightweight threads"?
<ns12>
companion_cube: "yes, but there's a runtime lock, just so you know" - What is the significance of that?
<companion_cube>
hmmmm there used to be another thing for threads, but now it's just posix threads.
<companion_cube>
the runtime lock means 2 threads won't run at the same time (except when engaged in some particular C calls)
<ns12>
companion_cube: So if I want two things to run at the same time, I should use Unix.fork to create a new process?
<companion_cube>
if you want to use multiple cores, probably, yeah
<companion_cube>
the question then is whether it's worth the extra pain :p
<ns12>
What extra pain?
<companion_cube>
handling multiple processes and coordinating them
<ns12>
If Unix.fork can already make OCaml execute two things in parallel, what is the significance of "multicore OCaml"?
<companion_cube>
that within one process you'll have able to use multiple cores
<companion_cube>
in a more convenient way, with finer grained concurrency (mutexes, thread pools, etc.)
<ns12>
Does this only benefit programs that use threads? If my OCaml program does not use threads, there is no benefit in using "multicore OCaml", correct?
<companion_cube>
it's not even out yet
<companion_cube>
but yeah, it'll need to use some form of threads
<companion_cube>
it won't magically parallelize anything
<ns12>
So why do I hear lots of complaints about OCaml not supporting multicore? The lack of multicore only seems to affect those programs that use threads.
bobo has joined #ocaml
spip has quit [Ping timeout: 272 seconds]
<ns12>
companion_cube: ^
rgrinberg has quit [Ping timeout: 240 seconds]
adanwan has quit [Remote host closed the connection]
azimut_ has quit [Remote host closed the connection]
adanwan has joined #ocaml
azimut has joined #ocaml
<d_bot>
<darrenldl> ns12: well i mean many things dont use threads because we do not have multicore yet, rather than because multicore would not be useful for certain problems
ralu has quit [Ping timeout: 272 seconds]
mbuf has joined #ocaml
<companion_cube>
yeah, right now if you want to use multiple cores you have to do the annoying dance of using multiple processes
<companion_cube>
that's what people complain about
<companion_cube>
even though it's the same in node.js, python, etc.
waleee has quit [Ping timeout: 240 seconds]
zebrag has quit [Quit: Konversation terminated!]
Haudegen has joined #ocaml
<ns12>
Okay. Thanks for the explanations.
<ns12>
I have a question about the "Random" library. Given the same seed, will the random number generator generate the same numbers on different computers?
<d_bot>
<mk-fg> That's usually the idea behind using RNG seeds
<ns12>
Even when the OCaml version is different?
<d_bot>
<mk-fg> I'd expect algorithms behind it to stay the same, but haven't looked myself
<d_bot>
<mk-fg> Crypto stuff doesn't get replaced that often anyway, even when it's something ancient like md5 :)
<ns12>
Since the docs don't guarantee reproducibility of random numbers across computers and across OCaml versions, I think it is best to just implement my own random number generator.
<ns12>
companion_cube: Why is multicore such a big deal for OCaml? Every time OCaml is mentioned, someone inevitably complains about the lack of multicore. But the lack of multicore does not stop them from using Python ...
<d_bot>
<mk-fg> Hm, is it? I've only seen it ever mentioned in this discord and on HN occasionally
gravicappa has joined #ocaml
<d_bot>
<Bluddy> ns12: It's all about the class of programming languages. Within the class of compiled/JIT-based high performance programming languages, OCaml is the only one that doesn't have multicore support. However, within the last 10 years, Javascript has conquered the programming world, relegating all other languages to small niches. At this point, the basis of comparison has changed, since JS is not a high performance language.
ralu has joined #ocaml
shawnw has joined #ocaml
<d_bot>
<darrenldl> ns12: id avoid rolling a bespoke prng if you're using it for security stuff
shawnw has quit [Quit: Leaving]
<d_bot>
<darrenldl> then again Random is not for security to begin with hm (afaik)...
Serpent7776 has joined #ocaml
azimut has quit [Ping timeout: 240 seconds]
azimut has joined #ocaml
hackinghorn has quit [Ping timeout: 272 seconds]
<d_bot>
<Jektrix> V8 is a pretty damn good JIT all in all.
<d_bot>
<Bluddy> sure. It's just not quite at the level of a compiled or semi-compiled language.
<d_bot>
<Bluddy> and it's very memory-hungry
hackinghorn has joined #ocaml
<d_bot>
<Bluddy> but it just doesn't matter that much anymore in most segments. the web and cloud computing have changed the way most programming is done.
<d_bot>
<Bluddy> JS is still pretty bloated and slow on phones though.
<ns12>
darrenldl: I will be using the bespoke PRNG for games and toy simulations. Thanks for the warning.
<d_bot>
<darrenldl> ns12: then Lehmer random number generator is pretty good afail
<d_bot>
<darrenldl> afaik
<d_bot>
<darrenldl> simple and fast
<ns12>
Yes, that's what I have already implemented. Thanks.
szkl has quit [Quit: Connection closed for inactivity]
mro has joined #ocaml
Sofi has quit [Quit: You have been kicked for being idle]
<octachron>
ns12, Random's PRNG will change for OCaml 5.0 (because it is useful to have a splittable PRNG for multicore), but that is the first change of PRNG since 1995
<sadiq>
always pretty sad when your IDE explains the code you're replacing is 25 years old.
<sadiq>
(it's also where learning some French has been handy)
Anarchos has joined #ocaml
jlrnick has joined #ocaml
jlrnick has quit [Read error: Connection reset by peer]
<ns12>
octachron: Thank you for this information.
bartholin has joined #ocaml
<d_bot>
<guymoque-3987> Does anyone know an OCaml library that looks like Python's `urllib` ?
<d_bot>
<guymoque-3987> the `urllib.parse` submodule
<d_bot>
<guymoque-3987> but nevermind, I have make what I want to do 🙂
xenu_ has joined #ocaml
xenu has quit [Read error: Connection reset by peer]
Anarchos has quit [Quit: Client closed]
unyu has quit [Ping timeout: 240 seconds]
<ns12>
Is there a general rule of thumb for choosing between association lists, Maps, and Hashtbls for a small amount of elements (30-40 items)?
<d_bot>
<Et7f3> Association list when you can search by key or value
<d_bot>
<Et7f3> Usually a tuple of map do the same
<d_bot>
<Et7f3> Map vs hashtbl is if you want immutability and ability to come back
<d_bot>
<Et7f3> With map when you add a element your create a new map that share memory
<d_bot>
<Butanium (@me on answer)> what do you mean by come back ?
<d_bot>
<antron> use an earlier state of the map
<d_bot>
<antron> "state"
<d_bot>
<antron> since map is immutable, you (cheaply) create new maps when you add or remove elements. that means the earlier "state" is still around as a complete value, if you retain a reference to it
<octachron>
My rule of thumb is generally to use a Map, which has the advantage to restrict users to the Map.S interface
<d_bot>
<Bluddy> hashtbl has the highest performance, so if you're doing something like game state based on frame rate, go for that
<d_bot>
<antron> ive drifted to using Hashtbl over time, because of the non-functorial default interface. however, Hashtbl does have some strange and surprising semantics for some of its functions, so Map is probably the better choice if one is getting started and not ready for surprises 🙂
<d_bot>
<Bluddy> that's true. plus the fact that it uses generic equality and hashing by default
<octachron>
Indeed, if you know that you need maximal performance at any cost, `Hashtbl` is better. But in that case, you already know what you want, and there is no need for a rule of thumb.
unyu has joined #ocaml
<ns12>
Is Hashtbl efficient even for 10 items? Isn't there some kind of overhead associated with hash tables?
<ns12>
In Common Lisp, for example, I don't use hash tables for 30 items. I would use association lists instead, because the overhead of hash tables makes it less efficient than association lists for small number of items.
<d_bot>
<Bluddy> you choose how much space to initialize the hashtbl with. it's quite efficient.
<d_bot>
<orbitz> As always: measure your program if performance matters
jlrnick has joined #ocaml
<d_bot>
<darrenldl> and very often, performance hit happens at places you were not focusing on up front
dextaa_ has joined #ocaml
dextaa_ has joined #ocaml
dextaa_ has quit [Changing host]
dhil has joined #ocaml
dextaa_ has quit [Remote host closed the connection]
mro_ has joined #ocaml
mro has quit [Ping timeout: 250 seconds]
jlrnick has quit [Ping timeout: 272 seconds]
dextaa_ has joined #ocaml
Haudegen has quit [Quit: Bin weg.]
<d_bot>
<mseri> ns12 also parmap and parany allow you you to parallelize the code
<ns12>
How does Lwt relate to threading and parallelism?
<d_bot>
<Bluddy> Lwt provides green threads ie. not parallelism but concurrency to deal with IO latencies.
<d_bot>
<Bluddy> So the same as Promised/async-await in JS
bobo has quit [Quit: Konversation terminated!]
<ns12>
How does that work? Suppose I use Lwt to execute a blocking system call. Wouldn't that block the entire process, such that there is no concurrency?
<d_bot>
<orbitz> Depends on the system call
<d_bot>
<orbitz> many system calls have a non-blocking mode, and then Lwt behind the scenes handles doing something else until the result is ready
<d_bot>
<Bluddy> that's why, just like in JS land, all calls from Lwt are non-blocking. If you need a blocking call, you can spin it off to another system thread.
spip has joined #ocaml
<d_bot>
<antron> ns12: lwt is using the non-blocking mode of system calls that have it (e.g. socket I/O) or a thread pool for running blocking calls
bartholin has quit [Ping timeout: 272 seconds]
bartholin has joined #ocaml
<ns12>
Given that only one thread can execute at any given time due to the limitations in OCaml, what is the advantage of using Lwt? Even with Lwt, there will still be only one thread executing at a time.
<d_bot>
<orbitz> If your application is doing things that are generally I/O bound, then there is lots of time to do something else while the I/O happens
<haesbaert>
ns12: lwt still gives you concurrency, just doesn't give you parallelism
<haesbaert>
ns12: you can write things that "block on a lwt promise" for example
<d_bot>
<antron> ns12: the thread pool used for blocking I/Os by lwt is implemented entirely in C and does not touch the ocaml runtime or its lock
<ns12>
So instead of wasting time waiting for blocking system calls to complete, I could use Lwt to get the ability to do other things while waiting?
<d_bot>
<antron> yes
<d_bot>
<antron> you can spawn a huge number of outstanding I/Os and do something else on the CPU, like figure out which other I/Os to spawn
<haesbaert>
yeah, you can look at it as a very nice abstraction on select/poll
wonko has joined #ocaml
<haesbaert>
at least that's how my brain understood in the beginning
<d_bot>
<antron> it is, at the center, a wrapper around epoll, kqueue, or whatever each system has
<d_bot>
<Bluddy> ns12: your comparison to using threads is valid. green threads are lighter than system threads. additionally, OCaml can tell which parts of the code are lwt-threaded due to the type system.
<d_bot>
<antron> together with a thread pool
<d_bot>
<antron> Bluddy: lwt-threading is a confusing terminology. because lwt promsies and the internal lwt C thread pool both have the term "thread' in them but they are entirely separate entities from each other
<ns12>
Must Lwt be implemented in C, or could it have been implemented purely in OCaml?
<d_bot>
<antron> if lwt was implemented purely in ocaml, it would incur synchronizations with the runtime lock at least at the start and end of each I/O
<d_bot>
<orbitz> Lwt can be written entirely in Ocaml
<d_bot>
<antron> right now to start an I/O operation from ocaml using lwt, you call some function, it goes to C, and starts a call in the C thread pool
<d_bot>
<antron> if you wrote it entirely in ocaml, you would call the ocaml function, it would go to another ocaml function in another thread, which would force taking the ocaml lock, then it would start the I/O from ocaml, releasing the lock, so you'd still get concurrency, but then you'd need that ocaml worker thread to wake up again after the I/O, take the lock again, only to tell the main thread that the I/O is done, then release the lock, le
<d_bot>
<antron> so it's possible to implement it this way but it would be very inefficient
<haesbaert>
antron: when you say thread pool, you actaully mean a pthread ? (I never read the lwt code, I just imagine how it is)
<d_bot>
<antron> you would still get concurrency, though
<d_bot>
<antron> yeah, lwt uses pthreads on Unix and windows threads on windows
<d_bot>
<orbitz> What @antron says is not true of all calls
<ns12>
"If your application is doing things that are generally I/O bound ..." - For example, making queries to a database server?
<d_bot>
<orbitz> ns12: yes
<haesbaert>
I see, so you can buy a blocking context I guess
<d_bot>
<antron> orbitz: the exception is calls that have non-blocking modes, like pipe and socket I/O
<d_bot>
<orbitz> Yes, I know. I just felt your statement implied all I/O calls, so clarifying.
<d_bot>
<orbitz> You need to interface at some level with whatever language your OS is wirtten in to perform a call, but if you discount that, in many cases, something like Lwt can be written entirely in Ocaml. Many OS's even provides async IO for filesystem operations
<ns12>
It must have been a lot of work to re-implement the Unix library.
<d_bot>
<orbitz> ns12: it isn't really
<d_bot>
<antron> to date most have been terrible, like aio, which is typically implemented using a low-quality lwt-like abstraction in user space
Everything has joined #ocaml
<d_bot>
<antron> low-quality because these things often received little maintenace
<haesbaert>
but aio is virtually unusable in any form
<d_bot>
<antron> while working on lwt i profiled aio and it was much slower than lwt
<d_bot>
<antron> yes it also has a completely broken api and many other problems
<haesbaert>
exactly
<d_bot>
<orbitz> I don't disagree
<d_bot>
<antron> libdispatch is implemented the same way as lwt with the same performance
<d_bot>
<antron> (that's macos gcd)
<haesbaert>
I used a lot of niel's provos libevent (not to be confused with libev)
<d_bot>
<antron> so in practice a library like lwt had to, until maybe io_uring on linux, be written against the "old" system call api, pretty much the way lwt is written
<d_bot>
<antron> libuv is also written the same way
<haesbaert>
I love libevent tbh
wonko has quit [Remote host closed the connection]
<d_bot>
<antron> orbitz: the main limitation of writing lwt in ocaml would be contention on the big runtime lock between the main thread and worker threads, otherwise yes, it could be written in ocaml
wingsorc has quit [Ping timeout: 250 seconds]
wonko has joined #ocaml
<d_bot>
<orbitz> Yep
<ns12>
Thank you all for answering my questions.
wonko has quit [Remote host closed the connection]
<sleepydog>
i have heard good things about windows completion ports, but never used it
wonko has joined #ocaml
wonko has quit [Remote host closed the connection]
wonko has joined #ocaml
Everything has left #ocaml [#ocaml]
<d_bot>
<antron> we never had time to try them in lwt. also, i heard that libuv eventually moved away from them
<d_bot>
<antron> but i did hear good opinions on the api
Haudegen has joined #ocaml
mro_ has quit [Remote host closed the connection]
sleepydog has quit [Ping timeout: 240 seconds]
sleepydog has joined #ocaml
d_bot_ has joined #ocaml
unyu has quit [Ping timeout: 240 seconds]
d_bot has quit [Ping timeout: 240 seconds]
unyu has joined #ocaml
mro has joined #ocaml
mro has quit [Ping timeout: 250 seconds]
dextaa_ has quit [Remote host closed the connection]
<d_bot_>
<darrenldl> any active user of any of the distributed computing libs?
rgrinberg has joined #ocaml
jonasbits has joined #ocaml
mro has quit [Remote host closed the connection]
mro has joined #ocaml
azimut has quit [Ping timeout: 240 seconds]
wonko has quit [Ping timeout: 252 seconds]
mro has quit [Remote host closed the connection]
dalek_caan has joined #ocaml
Haudegen has quit [Quit: Bin weg.]
vicfred has joined #ocaml
vicfred_ has joined #ocaml
vicfred has quit [Ping timeout: 256 seconds]
<d_bot_>
<Continuation Calculus> is there any way to ask merlin to not shadow basic types? (unit, int, string, etc.). or even better, a way to not shadow simple types (like, `type x = single_variable` should not shadow `single_variable`)
vicfred_ has quit [Client Quit]
waleee has quit [Ping timeout: 252 seconds]
salkin has joined #ocaml
mro has joined #ocaml
mro has quit [Remote host closed the connection]
mro has joined #ocaml
<d_bot_>
<octachron> That's probably merlin heuristic for determining the best name for a type kicking in. You could try to use a name that the heuristic doesn't like? (The heuristic is penalizing module names containing `__`, I am not sure about type name containing `__`).
<d_bot_>
<octachron> Defining the alias in a small module could work too.
waleee has joined #ocaml
Haudegen has joined #ocaml
mro has quit [Remote host closed the connection]
rgrinberg has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
xgqt has quit [Read error: Connection reset by peer]
gravicappa has quit [Ping timeout: 272 seconds]
xgqt has joined #ocaml
wyrd has quit [Ping timeout: 240 seconds]
wyrd has joined #ocaml
ejones has quit [Ping timeout: 252 seconds]
rgrinberg has joined #ocaml
dalek_caan has quit [Quit: dalek_caan]
rgrinberg has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
mro has joined #ocaml
mro has quit [Remote host closed the connection]
mro has joined #ocaml
dextaa_ has joined #ocaml
jlrnick has joined #ocaml
salkin has quit [Quit: salkin]
Anarchos has joined #ocaml
Serpent7776 has quit [Quit: leaving]
rgrinberg has joined #ocaml
Anarchos has quit [Quit: Vision[]: i've been blurred!]
bartholin has quit [Quit: Leaving]
mro_ has joined #ocaml
mro has quit [Ping timeout: 256 seconds]
<d_bot_>
<cemerick> Added piaf to my dune dependencies, and now I'm getting screenfuls of messages like: /usr/bin/ld: /home/chas/.opam/default/lib/dream/h2-lwt/h2_lwt.a(h2_lwt.o): in function \`camlH2_lwt__data_end': :(.data+0x90): multiple definition of `camlH2_lwt__data_end'; /home/chas/.opam/default/lib/piaf/h2-lwt/h2_lwt.a(h2_lwt.o)::(.data+0x90): first defined here
<d_bot_>
<cemerick> wiping _build and starting fresh has no impact, it seems
<d_bot_>
<cemerick> any ideas ( @anm'
<d_bot_>
<cemerick> ech; any ideas ( @anmonteiro maybe)?
<d_bot_>
<anmonteiro> is this with dream?
<d_bot_>
<anmonteiro> they both vendor the same deps so there are linking issues
<d_bot_>
<cemerick> ah-ha
<d_bot_>
<cemerick> thanks for the clarification 🙂
mro_ has quit [Quit: Leaving...]
jlrnick has quit [Ping timeout: 252 seconds]
rgrinberg has quit [Read error: Connection reset by peer]
rgrinberg has joined #ocaml
Guest58 has joined #ocaml
Guest58 has quit [Client Quit]
vicfred has joined #ocaml
rgrinberg has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
leah2 has quit [Ping timeout: 240 seconds]
leah2 has joined #ocaml
xand0_ has quit [Read error: Connection reset by peer]
xand0_ has joined #ocaml
caasih has quit [Ping timeout: 260 seconds]
pgiarrusso has quit [Read error: Connection reset by peer]