<olle>
MLKit has the most advanced GC I've ever seen.
tengu1 has joined #ocaml
Haudegen has quit [Quit: Bin weg.]
mro has joined #ocaml
<d_bot>
<Pokegali> So yesterday I was trying to do "opam install disml", and I'm still trying. It went well though all the dependencies, but now it fails while compiling disml : "library decompress not found". However, "opam install decompress" tells me that it's already installed
<d_bot>
<Pokegali> Any idea how to solve this ?
<d_bot>
<Et7f3> Did you use `eval $(opam env)` ?
<d_bot>
<Pokegali> Yes but not some hours ago, maybe I should try it again
tengu1 has quit [Ping timeout: 256 seconds]
<d_bot>
<Pokegali> It has failed again.. With the same error
mro has quit [Remote host closed the connection]
<companion_cube>
olle: probably less advanced than modern JVM GCs…
<olle>
companion_cube: Hmmm
<olle>
Well, MLKit is part of the type system, with region inference. Can't do that in bloody Java.
<olle>
And they made their own "regional calculus", phewww.
<companion_cube>
so it's not just a GC :p
<olle>
I mean, in their case it's not so easy to separate.
mro has joined #ocaml
<Drup>
MLKit is very interesting, yeah
<Drup>
Unfortunately, it doesn't seem to yield to really impressive benchmarks ...
<olle>
Yes, sad.
<olle>
Probably because the usage of regions are very domain or algorithm specific.
<Drup>
yeah
hendursaga has quit [Quit: hendursaga]
hendursaga has joined #ocaml
mro has quit [Remote host closed the connection]
Skyfire has quit [Ping timeout: 240 seconds]
mro has joined #ocaml
<d_bot>
<ggole> Regions seem like such a seductive idea, but they flatly do not work for general purpose MM because it is necessary to free unused storage before the region bounding its lifetime is finished with
<olle>
ggole, I think it can be useful in certain cases, e.g. running the A* algorithm, smaller dynamic programming stuff like that.
<olle>
Apache uses it extensively.
<d_bot>
<ggole> Yes, if you don't have little bits of storage that can become unreachable they work well
<olle>
You need like a contained, well, region. :)
<olle>
I want to add memory-safe regions to my toy language, we'll see how that works out...
<d_bot>
<ggole> That's not the issue, the issue is that if something can be become unreachable early then it needs to be freed early or you get unbounded memory use
<olle>
In the end, it might reduce ergonomics because it gets hard to combine pieces that use different memory allocation strategies.
<olle>
ggole, right, you need a known upper bound.
<d_bot>
<ggole> Consider adding a node to a persistent map, when you only have one ref to a map at a time. he old path
<d_bot>
<ggole> Do that in a loop and you are dead.
<olle>
Some languages add support for only one region at a time, others hierchical regions, other multiple regions. No idea what's best. xD
<companion_cube>
idk about regions, but custom allocators are a common thing in C++ apparently
<olle>
ggole, loop with unknown upper bound, or? Not sure I understand.
<companion_cube>
and… zig
<olle>
In C and C++ you can mix freely, yes.
<olle>
Without the safety.
<olle>
static safety*
<d_bot>
<ggole> Just an ordinary loop
<d_bot>
<ggole> A loop and a map, not exactly advanced constructs
<olle>
I don't see it
<olle>
The map is part of the region?
mro has quit [Remote host closed the connection]
<d_bot>
<ggole> In a region-only system, the nodes of the map are given a region
<d_bot>
<ggole> And they all die at the same time, when the region ends
<olle>
How is that a problem?
nore has quit [Ping timeout: 240 seconds]
<d_bot>
<ggole> Because that's too late to avoid unbounded unreclaimed but unreachable memory
<olle>
Oh, you mean dead memory gets freed too late? Yes, sure.
<olle>
If you care about keeping memory usage down, that is.
<olle>
But that's not such a common constraint.
<d_bot>
<ggole> I'm not convinced you understand the severity of the problem. The size of the unclaimed memory is not a function of the size of live memory; it's unbounded. You can write trivial loops that should run in constant memory but exhaust it in a region-only system.
Haudegen has joined #ocaml
<olle>
ggole, it's linear to the algorithm, tho, no?
<d_bot>
<ggole> Nope
<olle>
Hm, right
<d_bot>
<ggole> Keeping space complexity (in general) fundamentally requires freeing storage earlier than a region-only system can
<olle>
Ah
<d_bot>
<ggole> Which is why regions don't work as a general purpose thing
<olle>
Well, in MLKit, they actually made each region garbage traced.
<d_bot>
<ggole> Yes, that solves the problem by freeing unreachable storage early, but gives up on many of the promised advantages of regions
<olle>
Kind of. Depends on how late you choose to collect...
<olle>
Only on OOM :D
mro has joined #ocaml
mro has quit [Remote host closed the connection]
mro has joined #ocaml
nojb has joined #ocaml
nojb has quit [Remote host closed the connection]
Everything has joined #ocaml
waleee has quit [Ping timeout: 245 seconds]
waleee has joined #ocaml
<d_bot>
<froyo> ggole: doesn't this make it quite close in characteristics to your ordinary generational minor heap? :P
<d_bot>
<ggole> In some ways, yes
<olle>
Often you can trigger a collect programmatically when needed
<olle>
Often = in many languages
<Corbin>
olle: Think of GC as a *transformation* which takes an abstract program that doesn't manage memory at all, and returns a new program which alternates between mutation and cleanup.
<companion_cube>
that's a fun pov
<Corbin>
If we can show that parts of the transformation have predictable behavior, then we can make predictions about relatively large programs.
<olle>
Corbin: I refuse to think like that
<olle>
No
<olle>
I refuse to think
<olle>
Chess mate, liberals
<olle>
check*
<olle>
Corbin: What does it mean, "not manage memory at all"? No malloc? o0
<Corbin>
Accurate reaction to category theory~ I didn't even say "family of functors"~ Anyway, the direction I was thinking was that a GC with regions might transform specific scopes/procedures/threads of the source program into specific kinds of mutation-cleanup cycles.
sleepydog has quit [Read error: Connection reset by peer]
<Corbin>
Oh, uh. In just about any ML, not just OCaml, we could write something like `fun x y -> x + y` to add two numbers, maybe with varying syntax. But an implementation might use bignums, so might have to allocate even for simple addition.
<olle>
Corbin: My current goal is more oriented towards programming ergonomics, as discussed before, and 80/20 rule - optimize ergonomics for the most common scenario.
<Corbin>
Since the implementation can vary, we might talk about *just* the syntax, along with our semantic expectations in general, like that `2 + 3` reduces to `5`. That's what I mean by being abstract and not managing memory.
<olle>
Hm
<olle>
Not sure how to apply that perspective
<companion_cube>
Corbin: OCaml is a bit more specific than that though :)
<companion_cube>
that `int` is not a bigint is very important
<olle>
boxed vs unboxed
<Drup>
Corbin: except that way of thinking is precisely incorrect for most GC, except techniques like Rust.
<Corbin>
Drup: It's used in RPython, so e.g. PyPy uses it in production.
<Drup>
GCs are specifically dynamic because they are not static program transformations. And doing the static transformation that correspond to the work done by the OCaml GC is undecidable (not only in theory :p).
<olle>
for loop = rand(100), alloc node
<Corbin>
companion_cube: Yes. And indeed this entire conversation should have been in #proglangdesign or another language-neutral channel. But that's life.
<companion_cube>
Drup: they're program transformation since you need to insert calls to the GC after allocating :p
<Drup>
Well, RPython is *very* limited, and the resulting transformation is more akin to refcounting (just as in Rust, in many ways)
<companion_cube>
(or before rather)
<Corbin>
Drup: Oh! I mean that the high-level program, e.g. in RPython, is transformed into a low-level program, e.g. to C. That transformation can be thought of as a GC functor!
<Drup>
As I said, RPython is very limited. Your static GC is very weak.
<olle>
"weak"?
<Corbin>
"your"?
<olle>
"is"?
<Tardigreat[m]>
Corbin: can you go in a bit more depth regrading the GC functor? i'm not sure i fully understand what the categories are and how garbage collection is viewed as a functor here
<Armael>
I mean, linking your program against the runtime can be seen as a program transformation
<Armael>
so I would tend to agree with Corbin's view
<Armael>
(just, it's a very general statement that doesn't say much?)
<Drup>
unless I'm mistaken, Corbin is talking about inserting the malloc/free. Not just "calling collect() and a library-GC will do the job"
<Tardigreat[m]>
Corbin: thanks, i'll give it a read
pierrechc has joined #ocaml
<Drup>
ok, so that's not at all what is usually meant in the literature by "static GC" (which is a thing .... annotating the program with malloc/free by static analysis, and is not general purpose :p)
<Corbin>
Please don't put words in my mouth, thanks~
<Drup>
but fair enough, it's a transformation on the space of the semantic of programs :)
<olle>
That transformation? Albert Einstein.
<olle>
Sorry for the reddit idioms...
<Tardigreat[m]>
Corbin: i find that article way too confusing. probably because i'm lacking some information/context. i'll have to read the full paper
<Corbin>
Drup: FWIW I explain at the bottom of that Turing-category discussion a sketch of what I call "Tomb", the ∞-category of all Turing categories. Its maps are compilers, compiler compilers, etc. In this perspective, a GC functor is just a special sort of compiler.
<Corbin>
Ooh, actually, GC functors might witness an obstruction in Tomb. Like, maybe compiling Python to C *requires* adding a GC somehow. Interesting food for thought.
<Tardigreat[m]>
in particular i'm struggling to understand what a concrete instance of a turing category looks like for something like lisp or a simple C program that encodes some instructions into an array and then calls it as a function
<Corbin>
Like, in Scheme, (+ 2 3) reduces to 5 under evaluation, right? So let's talk about `(+ 2 3)` and `5` and reduction, but not evaluation. These are the elements of a Turing object.
<companion_cube>
wdym "reduction but not evaluation"?
<Corbin>
Like, *only* syntax. Abstractions only. No machines. Traditionally, the Turing object was just the natural numbers, and the map was a very specific Gödel-ish map which decoded a number to an AST, reduced the AST, and encoded it back to a number.
<Tardigreat[m]>
Corbin: that i'm okay with, but the thing that makes me a bit confused is something like (eval `(,symbol-append 'x 'y
<companion_cube>
you still can't talk about just reduction of (+ 2 3)
<Tardigreat[m]>
sigh... enter
<companion_cube>
you need a heap :)
<Tardigreat[m]>
(eval `(,symbol-append 'x 'y))
<Tardigreat[m]>
here, it's not so clear as to what's happening, because in general you can't tell what `5` is
<Corbin>
companion_cube, Tardigreat[m]: You don't need for reduction to be fully generalized; in fact, Turing showed it impossible. Instead, we need one partial operation, Kleene application; OCaml `fun f x -> f x`. From this, the entirety of Turing-completeness is implied.
<Corbin>
Or rather, I should say `fun g f x -> g (f x)`, of which the former is a special case; application is a special case of composition.
<companion_cube>
how does that relate to my question? :s
<Tardigreat[m]>
i feel like we may be getting too off-topic here
<Corbin>
We can take this to #proglangdesign at any time~
<Tardigreat[m]>
Corbin: i'll have to read the paper, i remain unconvinced (currently, i can be shown wrong) about turing categories being defined in a sufficient way to meaningfully reason about self-modifying programs or self-generating programs. that being said, it looks pretty cool
<companion_cube>
Tardigreat[m]: this channel is not so verbose that it's a problem imhop
<companion_cube>
-p
<Corbin>
companion_cube: I guess I need a richer example. Take two extremely trivial funs, `fun x -> x + 1` and `fun x -> x * 2`. Their left-to-right Kleene composition is `fun x -> (x + 1) * 2`. We can do this with syntax alone.
<companion_cube>
let's take the trivial function `let r=ref 0 in (fun () -> incr r; !r)`
<companion_cube>
can you evaluate that?
<Corbin>
I'm trying to forget about evaluation, TBH. I picked a shitty first example; I was being lazy and used a literal. Evaluation arises from repeated reductions in context, just like in type theory.
<companion_cube>
agreed, if the context includes a (mutable) heap :p
<Corbin>
Apologies, but: Do not try and allocate on the heap, that's impossible. Instead, only try to realize the truth: there *is no heap*. Then you will see that it is not the heap which allocates, but only yourself.
<Corbin>
The *entirety* of a language, from its heap to its datatypes to its concurrency to its I/O behaviors, is projected from within a Turing object. What we think of as "types" when programming are just fragments of programs which, when composed with other programs, behave as though those "types" really existed.
<olle>
Using an arithmetic example does not really help, rather the ref as in companion_cube or read from a file :)
<olle>
Also, the fact that I can't paste this on r/pcj - SAD
<companion_cube>
Corbin: well, OCaml's semantics requires a heap (or some memory store anyway)
<Corbin>
companion_cube: Yes. Semantics is a functor from the language to an implementation. (By well-known folklore, syntax is technically a functor from the implementation back to the language; the source files and compilers can be stored in computers.)
<olle>
This sounds eerily like auto-generated text...
<companion_cube>
that sounds a bit too detached from reality from me :p
<Corbin>
Okay, let's be done, then. Anything further needs to move to #proglangdesign (which is bridged on Matrix AIUI, for non-IRC folks)
<olle>
proglangdesign is stuck in an infinite loop about utf8
<olle>
The least interesting topic imaginable about formal languages
<companion_cube>
:DDD
<Drup>
Walder's law should be amended for character encoding.
<companion_cube>
utf8 is a simple topic: just use utf8 and ignore other encodings
<companion_cube>
end of story ✔
olle has quit [Ping timeout: 252 seconds]
<Tardigreat[m]>
utf-8 discussions will never stop, they will only be superseded by the next "standard"
<Tardigreat[m]>
companion_cube: i'm with you on that one lol
Haudegen has quit [Quit: Bin weg.]
<companion_cube>
why would there be a next standard? :)
* Tardigreat[m]
links relevant xkcd
<companion_cube>
thing is, people seem to actually converge to utf8
<Tardigreat[m]>
yes but utf-8 can't encode fine-grained spaceships that are necessary to create software in google, so we will now create a new, incompatible standard for....
<companion_cube>
heh, idk
kakadu has quit [Quit: Konversation terminated!]
mro has quit [Remote host closed the connection]
glassofethanol has quit [Quit: leaving]
mro has joined #ocaml
<d_bot>
<froyo> Drup: unless I'm mistaken, Corbin is talking about inserting the malloc/free. Not just "calling collect() and a library-GC will do the job"
<d_bot>
<froyo> Hasn't that been the general design discipline MSR languages like LEAN and Koka have been fitted with?
<companion_cube>
Lean 4 does sth like that, yes
<d_bot>
<froyo> in Koka's case they still have difficulties with cycles
<d_bot>
<froyo> namely if you want one it's your responsibility a la swift
mro has quit [Remote host closed the connection]
mro has joined #ocaml
cedric has joined #ocaml
mro has quit [Remote host closed the connection]
Anarchos has joined #ocaml
Haudegen has joined #ocaml
Tuplanolla has joined #ocaml
bartholin has quit [Quit: Leaving]
mro has joined #ocaml
zebrag has joined #ocaml
<companion_cube>
is memprof in the mainline compiler now?
<companion_cube>
I could install `memtrace` on a normal 4.12 switch nd I find that suspicious
mro has quit [Ping timeout: 252 seconds]
<zozozo>
companion_cube: yup it is
<zozozo>
(i.e. memprof does not need a separate switch, so you only need a recent enough compiler to use it)
<companion_cube>
awesome
<companion_cube>
I have a bad, bad memleak on 4.12 somehow
rgrinberg has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<d_bot>
<Skid> Yes like this paragraph ```
<d_bot>
<Skid> The -output-obj option can also be used to obtain the C source file. More interestingly, the same option can also produce directly a shared library (.so file, .dll under Windows) that contains the OCaml code, the OCaml runtime system and any other static C code given to ocamlc (.o, .a, respectively, .obj, .lib). This use of -output-obj is very similar to a normal linking step, but instead of producing a main program that
<d_bot>
<Skid> ```
<d_bot>
<Skid> I just dont know how to make it do that 😆
mro has joined #ocaml
mro has quit [Remote host closed the connection]
mro has joined #ocaml
<d_bot>
<Skid> it errors when trying to make .so because the libcamlrun runtime is static
<d_bot>
<Skid> if i try`ocamlc -output-obj -o libcaml.so mod.ml modwrap.o`
lisq has quit [Quit: lisq]
<d_bot>
<Skid> and if i compile the mod.ml into mod.o
<d_bot>
<Skid> then it packs the runtime into the .so and only need to include the one .so
rgrinberg has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<cemerick>
Do I need to keep dune at 2.7.0 since vscode-ocaml-platform still needs the .merlin files that it produces? Or is there a way to generate those files from e.g. 2.9.0 etc?
<companion_cube>
I'd also like to know
<d_bot>
<NULL> Answer to first question is no, dune can provide what used to be in .merlin when the extension (in fact the LSP server) asks for it
<d_bot>
<NULL> As for the second question, I can't answer for certain, but the option isn't listed in the `man` page (or I couldn't find it)
<companion_cube>
in practice merlin seems very confused for me :/
<d_bot>
<NULL> Merlin won't work, ocaml-lsp will
<d_bot>
<cemerick> Using the latest release? (1.8.4, I think)
<d_bot>
<cemerick> Not IME
<companion_cube>
wait, what? merlin won't work: ?
<d_bot>
<cemerick> No, I'm using vscode
<d_bot>
<NULL> I may be wrong, but I think merlin still expects .merlin files, while ocaml-lsp can ask dune
<d_bot>
<NULL> ocaml-lsp is a wrapper around merlin, so it doesn't make much difference which one you use, as long as it works
<def>
(though plain merlin can ask dune too)
<d_bot>
<NULL> I'm using latest dune, latest ocaml-lsp-server and latest extension version and it worked last time I opened VSCode
<d_bot>
<NULL> So I am wrong. ocaml-lsp is still what the VSCode extension expects, that I am more certain
<companion_cube>
I'm a bit frustrated because everytime I have a new switch, it takes me hours to get merlin/dune to talk to each other
<d_bot>
<NULL> With my small experience around creating switches, installing ocaml-lsp and dune has never not worked out-of-the-box, so long as VSCode was using the right switch
<d_bot>
<Skid> oh cool when you do it this way you can import the .so directly into python ctypes
<Corbin>
I'm thinking of retrofitting https://github.com/alexozer/flitter to take console input instead of calling Python for global key events. My main complaint is that the global input is both laggy and too needy, but maybe there's fundamental OCaml reasons for doing it this way?
xd1le has joined #ocaml
<d_bot>
<cemerick> Not sure what to say; without a .merlin file, the vscode extension reports "no configuration file found for foo.ml, run `dune build`". Dune 2.7.0 generates a .merlin file (leading to expected vscode behavior) , 2.9.0 doesn't
<cemerick>
companion_cube: are you on some dune later than 2.7.0?
<companion_cube>
Yeah
<d_bot>
<NULL> The build files (`_build/...`) either contain what the LSP server needs or tells it it can ask dune for more info
<d_bot>
<NULL> Are there issues with the build done ?
<companion_cube>
Well merlin doesn't find anything
<companion_cube>
Anyway I'll probably end up downgrading dune, as always
<rgrinberg>
cemerick .merlin files are not generated with dune 2.9 because they are indeed no longer needed. recent version of lsp/merlin will query dune directly
<d_bot>
<cemerick> No, dune build and runtest complete successfully
<companion_cube>
How to debug if it doesn't, rgrinberg?
<d_bot>
<NULL> I meant issues with the LSP server
<rgrinberg>
what version of lsp or merlin are you using?
<d_bot>
<cemerick> 1.4.1 of lsp, 1.8.4 of the vscode platform
Tuplanolla has quit [Quit: Leaving.]
<rgrinberg>
that might be the problem. that version of lsp is too old to query dune
<companion_cube>
For me, dune 2.9 with latest merlin (on 4.12)
* cemerick
grunts
<cemerick>
That _is_ old. How did I get that from a new 4.11 switch......
<rgrinberg>
companion_cube you're using vim right?
<companion_cube>
Yeah
<companion_cube>
I hope this is all supposed to work with dune build -p, btw
<rgrinberg>
cemerick: try 4.12
<rgrinberg>
companion_cube i don't see why it shouldn't, but try it without -p. dune build @check and then open merlin
<companion_cube>
I'll retry tomorrow
<companion_cube>
When I'm back at my work machine
<companion_cube>
But sth to debug what happens would be helpful
<cemerick>
rgrinberg: iirc, I can't due to some restrictive upstream requirements
<rgrinberg>
I don't use the merlin frontend anymore. So I can't be of much help there
<cemerick>
But yeah, sorry for the noise 😕
<companion_cube>
Maybe it's my cue to try ocamllsp, yes
<rgrinberg>
With the lsp frontend, it will say that it cannot find the dune config.
<companion_cube>
Ok, will try
<companion_cube>
Does the lsp support destructuring?
<rgrinberg>
Yeah
<companion_cube>
Hmm nice. Will definitely try it then.
Haudegen has quit [Ping timeout: 245 seconds]
rgrinberg has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]