namkeleser has quit [Quit: Ping timeout (120 seconds)]
gravicappa has joined #ocaml
chrisz has quit [Ping timeout: 256 seconds]
chrisz has joined #ocaml
jlrnick has joined #ocaml
olle has joined #ocaml
xgqt has quit [Remote host closed the connection]
xgqt has joined #ocaml
epony has quit [Read error: Connection reset by peer]
jlrnick has quit [Ping timeout: 240 seconds]
azimut has quit [Remote host closed the connection]
azimut has joined #ocaml
bartholin has joined #ocaml
Haudegen has quit [Quit: Bin weg.]
olle has quit [Remote host closed the connection]
mro has quit [Remote host closed the connection]
jlrnick has joined #ocaml
mro has joined #ocaml
mro has quit [Remote host closed the connection]
mro has joined #ocaml
mro has quit [Remote host closed the connection]
namkeleser has joined #ocaml
terrorjack has quit [Quit: Ping timeout (120 seconds)]
olle has joined #ocaml
Haudegen has joined #ocaml
terrorjack has joined #ocaml
namkeleser has quit [Quit: Client closed]
<ns12>
It's surprising the the Unix module works on Windows. Why not have a separate Windows module instead?
mro has joined #ocaml
<olle>
Windows is Linux under the hood ;D
<olle>
Soon enough...
<octachron>
Unix is a OS-independent Unix-like API
<ns12>
But there are so many functions that don't work on Windows.
<octachron>
Well, Windows is not a very Unix-like OS unsurprisingly.
<Corbin>
ns12: It's desirable, when building portable general-purpose languages, to erase or hide the differences between platforms. Lots of codebases have a couple special pages of code just for Windows, which only activate when on particular platforms.
mro has quit [Remote host closed the connection]
<ns12>
Corbin: "erase or hide the differences between platforms" - But that is clearly not possible with Windows, as evidenced by the large number of functions in the Unix library that are not implemented on Windows.
<Corbin>
ns12: Yes. And this is something of a swing in the wrong direction; being Unix-specific is just as much of a problem as being Windows-specific. Compare with languages that call this sort of thing an "OS" or "system" or "platform" module.
<d_bot_>
<Bluddy> this is a historical accident
<d_bot_>
<Bluddy> all of this stuff should have been abstracted away
waleee has joined #ocaml
<d_bot_>
<Bluddy> but instead, the Unix module was left as the `system` module
<d_bot_>
<Bluddy> and some Windows compatibility was provided
<ns12>
What is a better way? To have an "OS" module that has "Unix" and "Windows" submodules?
adanwan has quit [Remote host closed the connection]
adanwan has joined #ocaml
<olle>
SIG
<olle>
module signature
mro has joined #ocaml
<Corbin>
It depends on policy. Note that it's always possible, in any language, to make a portable program into a non-portable one by adding a platform-specific module. So, as a result, we should probably try to minimize *any* platform-specific code.
<ns12>
But the Unix module is really useful ...
<companion_cube>
Corbin: there's portable IO abstractions in OCaml, they're named `Sys` and all the channel primitives
<companion_cube>
Unix is, well, for unix. It's even a separate library you have to require explicitly
<Corbin>
companion_cube: ns12 might have a bit of a point. Stuff like high-level file access, envp access, Internet addresses and other BSD socket API; this could be portable.
<ns12>
Corbin: I don't think networking is portable.
<companion_cube>
I think the socket part of `Unix` works on windows.
<ns12>
Networking is probably not a compulsory part of a Unix system.
<Corbin>
ns12: BSD's socket API won an API war in the 80s. Nobody really expected such a thing. Since Windows NT, Windows has BSD sockets too.
<companion_cube>
ns12: I think the basics of the socket API are in posix
<companion_cube>
no real system will lack them
<ns12>
It's a bit inconsistent that some signal-related things are in "Sys" while others are in "Unix". I assume that this is due to historical reasons.
<companion_cube>
Sys is a bit higher level, but, yeah
<Corbin>
companion_cube: If Greenspun kept making laws, they'd eventually get to "inside any portable high-level multicore language implementation, there's an event loop"; maybe exposing something Twisted-ish in the language core is not such a bad idea. (See also JS, Deno, etc.!)
<companion_cube>
lol no
<companion_cube>
where's java's event loop?
<companion_cube>
JS is now irrevocably founded on event loops, Deno just follows suit
<companion_cube>
(and neither is multicore)
<companion_cube>
does poly/ML have an event loop?
<Corbin>
It's called "new I/O" or NIO. I'm told the N could have been for "non-blocking".
<companion_cube>
yeah and it's not the default way of doing IO
<companion_cube>
you only use it if you need it… like in OCaml
<Corbin>
Java's classic networking was built on the thread-per-connection paradigm. This is why a vanilla Minecraft server cannot support 1000 clients on a laptop.
<Corbin>
It's also why nginx has displaced Apache.
<companion_cube>
sure
<companion_cube>
servers are where you might want to use NIO
<companion_cube>
all programs are not servers
<d_bot_>
<mbacarella> Greenspun's Eleventh Rule: any sufficiently advanced program contains a slow, buggy ... network server
<Corbin>
GUI clients also need evented I/O, because they must simultaneously check networking and also redraw the screen. Actually, they simultaneously do networking/disk/joysticks/etc., lots of IO.
<companion_cube>
guess you shouldn't write anything more complicated than ocamlopt then :p
<companion_cube>
Corbin: do they now? do you have sources on that?
<companion_cube>
does jetbrains' stuff use NIO?
<d_bot_>
<mbacarella> oh actually I missed the opportunity:
<d_bot_>
<mbacarella>
<d_bot_>
<mbacarella> Zawinski's Law of Software Envelopment states: Every program attempts to expand until it can ~~read mail~~run a network server. Those programs which cannot so expand are replaced by ones which can.
<companion_cube>
(also remember that apache doesn't use threads, it uses worker processes)
<Corbin>
companion_cube: I don't want to throw HCI numbers at you, but it is relatively well-known that we must refresh a screen several dozen times per second in order to create a decent illusion of motion. Similarly, we must empty network buffers several times per second if we want to maintain good throughput.
<Corbin>
So, when would you *not* want an event loop?
<companion_cube>
ah well, you can have a loop alright
<companion_cube>
doesn't mean you have to use async IOs
cedric has quit [Quit: Konversation terminated!]
<d_bot_>
<mbacarella> no shame in using a while loop that polls 😛
<Corbin>
Sure. But that's only because of the legacy APIs providing sync IO options. In a new implementation, the problem of sync IO evaporates, and we aren't left with any compelling reason to readd it.
<companion_cube>
I mean it's not like epoll is new
<companion_cube>
Corbin: also, plot twist, you can use async IOs with OCaml's unix, just set your socket as non blocking
<companion_cube>
it's there if you need it
<companion_cube>
but I think it's wrong to suggest all IOs should be non blocking
mro has quit [Remote host closed the connection]
<Corbin>
companion_cube: Sure, and then just remember to invoke select() so that it will correctly timeout. And set up a priority queue of timers to give a source of timeouts for select(). And also set subprocesses to not block, and put filesystem access in a threadpool, and...
<Corbin>
Anyway, I'm happy to be wrong. Why should we have blocking I/O?
<companion_cube>
because it's simpler? :)
<d_bot_>
<mbacarella> yeah that. you don't have to think of any of that stuff.
<companion_cube>
a lot of OCaml tools do stuff like: read a file (or files); do a lot of computations; write a file (or files)
<companion_cube>
you need precisely 0 non blocking IOs for that
<companion_cube>
that's valid for ocaml itself, Coq, a lot of formal method stuff…
<d_bot_>
<mbacarella> most programming is not sexy interactive application. most programming is, like, processing program logs into compliance reports
<d_bot_>
<mbacarella> please don't make me think of async I/O when i'm bottlenecked on reading things from storage volumes and writing them back to other storage volumes 😭
<companion_cube>
to read files, anyway, async IOs will rarely be worth it
<companion_cube>
it's only recently started being usable on linux
<haesbaert>
15:00 < Corbin> It's also why nginx has displaced Apache.
<haesbaert>
well that's not entirely accurate, one process per socket was something in apache1 design, apache2 is a completely different beast
<companion_cube>
the damage is done anyway I suppose
<haesbaert>
IIRC even late apache1 didn't do it anymore
<companion_cube>
the funny stuff is that back then, people used a lot of php, which is one process per connection _anyway_
<haesbaert>
very true
<companion_cube>
people still use Rails a lot, which is thread-per-connection
<d_bot_>
<mbacarella> i had to edit php code yesterday 😭
<companion_cube>
:D
<haesbaert>
there's nothing inheritenly wrong with process/thread per connection though
<companion_cube>
same goes for Django, which is more popular than anything built on twisted
<olle>
hehe
<companion_cube>
haesbaert: no, and I like it, personnally :p
<haesbaert>
this is not windows 95 where a new process is a world event
<olle>
You can usually use message queues to distribute tasks in a web context
<olle>
No need for async etc
<haesbaert>
companion_cube: "back then" there was this article "how to handle 10k http requests/s"
<haesbaert>
that's when people started to hit "limits" on one fd per process
<companion_cube>
yeah :D
<Corbin>
companion_cube: I want to peel apart your memes; sorry. On your first point: using a programmable-semicolon analogy, there's no reason why non-blocking I/O must *feel* different from blocking I/O. That's a language decision.
<companion_cube>
this is #ocaml, not #haskell
<companion_cube>
thank you
<olle>
It does matter if code executes in the order you read it
<olle>
Leaving that "paradigm" creates... something different. :)
<haesbaert>
oh now I remember Corbin
<olle>
I'm more in to "defer", I think :d
<olle>
"defer" to a later point which does not matter that much
<Corbin>
companion_cube: As usual, I'm not telling you how to change OCaml. I'm telling you two things simultaneously: (1) OCaml is just another programming language; (2) programming languages intended for people should have ergonomic considerations.
<haesbaert>
olle: as long as you don't add concurrency within a single stream I think you're fine
<dh`>
corbin, you're confusing nonblocking I/O with asynchronous I/O
<haesbaert>
but you can achieve a similar thing even without queues, multiple processes calling accept on the same file descriptor which was passed down makes the kernel wake up one process at a time
<companion_cube>
Corbin: effects are going to help us, no worries
<companion_cube>
get rid of monads and all that
<dh`>
furthermore, asynchronous I/O is inherently concurrent and concurrency is hard for most people; consequently, ergonomic considerations mean that the default should be synchronous I/O
<companion_cube>
concurrency is hard for everyone, only sometimes you can't avoid it
waleee has quit [Ping timeout: 252 seconds]
olle has quit [Remote host closed the connection]
<Corbin>
dh`: I'm happy to split the nuance, and I'll admit that I'm talking about async paradigms. The conversation started with a mention of Java NIO, which is sometimes called "non-blockinhg I/O".
<Corbin>
companion_cube: The monads are still there; each effect is carried by a functor, just like a monad, and the effect laws are analogues of the monad laws. Or are you thinking of "monad" purely in terms of e.g. Haskell syntax?
<companion_cube>
each effect is carried by an effect handler, not sure where the functor is
<companion_cube>
it's not even typed, as of OCaml 5.0 :D
<companion_cube>
it's like the exception type
<Corbin>
BTW I'm not sure if there's any non-concurrent stuff running in my userland right now. I'm on a typical Linux; all X11 clients are concurrent with X11, and all services are concurrent with systemd.
<dh`>
I have no idea what Java may mean by it but in Unix, nonblocking I/O means something specific and almost certainly isn't what you're interested in from what else you've been saying
<companion_cube>
Corbin: you don't have a terminal emulator?
<Corbin>
companion_cube: My terminal emulator is concurrent with X11 and has a non-trivial time-dependent interaction between keystroke events from the kernel and window-repainting events sent to X11.
<dh`>
there is no non-concurrent stuff in any general-purpose operating system since MS-DOS, if you look at the whole system
<dh`>
even if you boot single-user with only one process running
<companion_cube>
sure, concurrent with other processes is a thing
<companion_cube>
generally you try to minimize the interactions with the rest
<companion_cube>
then again most code is also incorrect when you account for concurrency
<companion_cube>
like if you write a file, you can't protect against race conditions on the file
<companion_cube>
unless you use good stuff like sqlite
<Corbin>
Yes, most software is incorrect.
<companion_cube>
=> concurrency is hard
<ns12>
companion_cube: "like if you write a file, you can't protect against race conditions on the file" - Isn't file locking the solution?
<haesbaert>
and parallelism even harder
<companion_cube>
ns12: only if everyone agrees to use it
zebrag has joined #ocaml
<companion_cube>
also it seems horribly hard to do correctly.
<dh`>
file locking is straightforward as long as NFS isn't involved
<dh`>
but realistically, for nearly everything, you just don't update the same file twice at the same time; it doesn't make sense to even attempt so it doesn't happen
<companion_cube>
yeah, that's the theory
<sleepydog>
does bash use concurrency?
<companion_cube>
it probably uses select underneath
<companion_cube>
it's clearly single process single thread
<companion_cube>
(source: htop)
<sleepydog>
Corbin: I think almost all of coreutils is single threaded :)
<sleepydog>
I will cede GUIs are more complex, but for most network services you can get very far with just multi-process and some timeout mechanism like alarm(2)
<companion_cube>
GUIs have an event loop, and a lot of callbacks
<reynir>
you can write this in bash just fine { sleep 10; echo 1; } & { sleep 5; echo 2; } & wait
<companion_cube>
that is, if you exclude the React-like stuff in browers
<haesbaert>
sleepydog: I think they were talking about concurrency, not being multi-threaded, I think it would clarify the discussion if we just called I/O multiplexing
<sleepydog>
my understanding is we're discussing if an I/O event loop is a fundamental necessity for a modern programming language runtime
<haesbaert>
but the job control on a shell doesn't imply you need to do IO/multiplexing, historically it was all done via signals (IIRC)
<Corbin>
Yeah. Although right now I'm defending something a little easier: Is there a reason to include sync I/O for user-level code, including the code for the runtime? The runtime could still have sync I/O at its bedrock. (I see this as similar to the page of assembly required to write a kernel; I know it's sometimes required.)
<companion_cube>
it still is the simplest and quickest way to interact with files
<companion_cube>
so, yes
<sleepydog>
i can see arguments for both. it can actually be difficult to reach the full capabilities of physical storage these days using sync I/O
<sleepydog>
especially if what you are doing does not benefit from readahead
<Corbin>
companion_cube: It is quite simple, sure. It's not as fast as a threadpool, which lets the process enqueue multiple disk actions on the kernel; in turn, the kernel's disk scheduler is then able to choose how to interleave. Access to threadpools can be made async-only.
<companion_cube>
you're assuming the calling code is async
<companion_cube>
it's definitely faster than a threadpool if you don't have other threads
<Corbin>
Faster for "file", yes; faster for "files", not necessarily.
<companion_cube>
ah well, that depends on what you do
<companion_cube>
that said I am indeed excited for domainslib
<companion_cube>
and its pool-like abstraction.
<companion_cube>
all in direct style.
<haesbaert>
sleepydog: weellllll technically if the stuff is mounted as async the kernel will reserve a descriptor and return control to you so you can queue a lot still
<haesbaert>
a modern ahci had 1024 descriptor IIRC
<companion_cube>
the fun part is: I'm thinking of removing lwt from my IRC bot(s)
<companion_cube>
because it's just less convenient than threads
<haesbaert>
this is different than a network card where a "flow" will be hashed into one transmit queue, and one transmit queue cannot saturate a 10gbit/link
<sleepydog>
haesbaert: if you are reading a file, taking some action based on the contents of that read, and then reading the next file, or some other section of that file, no amount of "invisible" async will help
<sleepydog>
the explicit async at least forces a programmer to think about what else they could do while waiting for an I/O to complete
<companion_cube>
if there's anything else to do, of course.
<sleepydog>
yes, definitely
<sleepydog>
and it's perfectly valid to say "let the other threads in the program do something productive"
<sleepydog>
it is just as easy to come up with use cases that really *need* async to work well as it is to come up with use cases that are needlessly complicated by requiring async for everything
<companion_cube>
totally
<companion_cube>
but just removing sync IOs is pulling the ladder on simple solutions for simple problems
<companion_cube>
(e.g. reading a config file)
spip has joined #ocaml
bobo has quit [Ping timeout: 256 seconds]
<sleepydog>
I think Go does a really good job in this regard. For the most part, when it comes to I/O, you can write code in a direct style, with minimal callbacks, and the complexity of the event loop is hidden from you. Seemingly you get the best of both worlds.
<companion_cube>
hopefully we get there with effects
<companion_cube>
however, I often hear that Go interop with C is problematic
<companion_cube>
is it related?
<sleepydog>
somewhat. one reason it's problematic is because of Go's calling convention. That is not related. However, another reason is that it is hard, in general, to consistently run a goroutine on the same thread. So C routines and system calls which use thread-local storage are problematic.
<companion_cube>
we'll see how it goes with effects, I'm curious :)
<sleepydog>
yes, i've never used a language with effects, it will be exciting!
bartholin has quit [Ping timeout: 256 seconds]
bartholin has joined #ocaml
motherfsck has joined #ocaml
jlrnick has quit [Ping timeout: 256 seconds]
Haudegen has quit [Quit: Bin weg.]
<dh`>
there's a nice paper on how the haskell foreign function interface deals with that problem (running language threads on consistent OS threads)
<dh`>
anyway, there's several different considerations in this discussion that I think are being run together
<dh`>
one is whether a language or library should have a synchronous I/O interface, and the answer there is almost certainly yes
chrisz has quit [Remote host closed the connection]
<dh`>
one is whether the implementation of that interface should use async I/O under the covers
<dh`>
there is no compelling inherent reason not to, especially if you also have an async I/O interface
<dh`>
it's much more likely that the async interface will work if the sync interface (which will be used much more often and more thoroughly) uses the same infrastructure.
<dh`>
however, there are several extrinsic reasons not to, beginning with the dodgy async I/O implementations in most OSes
<companion_cube>
dh`: what is the point of using async IOs if you're going to block on them immediately, though?
lobo has quit [Quit: lobo]
lobo has joined #ocaml
bartholin has quit [Quit: Leaving]
mbuf has quit [Quit: Leaving]
mro has joined #ocaml
mro has quit [Ping timeout: 240 seconds]
Tuplanolla has joined #ocaml
Haudegen has joined #ocaml
dextaa_ has joined #ocaml
gareppa has joined #ocaml
gareppa has quit [Remote host closed the connection]
<d_bot_>
<mseri> Oh I need to study. What is google tag manager? I block lots of things but that was not on my radar
<d_bot_>
<undu> google analytics, I have it permanently blocked using ublock _and_ umatrix
<d_bot_>
<VPhantom> Yeah GoogleTagManager is a lot more than just a new name for Analytics: it's also a gateway to load all sorts of other JS at the webmaster's (or his marketing consultant's) whim. Facebook tracking, etc.