Leonidas changed the topic of #ocaml to: Discussion about the OCaml programming language | http://www.ocaml.org | OCaml 5.1.1 released: https://ocaml.org/releases/5.1.1 | Try OCaml in your browser: https://try.ocamlpro.com | Public channel logs at https://libera.irclog.whitequark.org/ocaml/
<companion_cube> idk, can you use stuff like `[@unboxed]` with ctypes?
<companion_cube> or `[@@noalloc]`
Inline has quit [Ping timeout: 240 seconds]
rgrinberg has joined #ocaml
Tuplanolla has quit [Quit: Leaving.]
waleee has quit [Ping timeout: 252 seconds]
gwizon has joined #ocaml
omegatron has quit [Quit: Power is a curious thing. It can be contained, hidden, locked away, and yet it always breaks free.]
jmcantrell has quit [Quit: WeeChat 4.1.2]
rgrinberg has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
gwizon has quit [Quit: leaving]
jmcantrell has joined #ocaml
trev has joined #ocaml
rgrinberg has joined #ocaml
azimut has quit [Ping timeout: 240 seconds]
bartholin has joined #ocaml
jmcantrell has quit [Quit: WeeChat 4.1.2]
Serpent7776 has joined #ocaml
bartholin has quit [Quit: Leaving]
gahr has quit [Ping timeout: 268 seconds]
olle has joined #ocaml
gahr has joined #ocaml
<discocaml> <leostera> how do y'all go about scoping down memory leaks? 🤔
<discocaml> <leostera> i'm doing some due-dilligence –– going through every line of code essentially – and i'm finding some low hanging fruit, but I clearly have a leak somewhere and can't seem to find it
<discocaml> <leostera> trying valgrind but it reports _a lot_ of invalid read/writes in the `write_barrier` and `caml_darken`
<discocaml> <leostera> i'm trying attaching finalizers and it at least confirmed my intuition of what data is being leaked
waleee has joined #ocaml
brettgilio has joined #ocaml
jakzale has joined #ocaml
ggb has joined #ocaml
pluviaq has joined #ocaml
_alix has joined #ocaml
whereiseveryone has joined #ocaml
kuruczgy has joined #ocaml
toastal has joined #ocaml
sleepydog has joined #ocaml
immutable has joined #ocaml
soni_ has joined #ocaml
patrick has joined #ocaml
henrytill has joined #ocaml
philipwhite has joined #ocaml
migalmoreno has joined #ocaml
ymherklotz has joined #ocaml
b0o has joined #ocaml
seeg has joined #ocaml
Ankhers has joined #ocaml
arya_elfren has joined #ocaml
richardhuxton has joined #ocaml
waleee has quit [Ping timeout: 264 seconds]
neiluj has joined #ocaml
cedric has joined #ocaml
pi3ce has quit [Read error: Connection reset by peer]
pi3ce has joined #ocaml
cedric has quit [Quit: Konversation terminated!]
pi3ce has quit [Quit: No Ping reply in 180 seconds.]
pi3ce has joined #ocaml
rgrinberg has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
waleee has joined #ocaml
<discocaml> <sim642> Memtrace should allow to find out where things are allocated
mima has joined #ocaml
<discocaml> <Ada> no idea to be honest
Soni has quit [Quit: http://quassel-irc.org - Chat comfortably. Anywhere.]
soni_ is now known as soni
SoniEx2 has joined #ocaml
soni is now known as Guest9335
Guest9335 has quit [Killed (calcium.libera.chat (Nickname regained by services))]
SoniEx2 is now known as Soni
mima has quit [Ping timeout: 252 seconds]
drobban has quit [Remote host closed the connection]
szkl has quit [Quit: Connection closed for inactivity]
soni_ has joined #ocaml
azimut has joined #ocaml
azimut has quit [Remote host closed the connection]
azimut has joined #ocaml
azimut has quit [Remote host closed the connection]
azimut has joined #ocaml
<discocaml> <leostera> i should add that i'm running ocaml 5.1 :/ so afaict memtrace and friends won't work here?
<discocaml> <leostera> i've been also using the runtime events but i couldn't find a way to emit my own to see if i could correlate some of my actions to the gc
<discocaml> <bluddy5> not yet, I think. are you using 5 features?
<discocaml> <leostera> yup, this is for a multicore scheduler i'm working on (github.com/leostera/riot)
<discocaml> <leostera> yup, this is for a multicore scheduler i'm working on (https://github.com/leostera/riot)
<discocaml> <leostera> yup, this is for a multicore scheduler i'm working on (https://github.com/leostera/riot)
<discocaml> <bluddy5> ah ok yeah
<discocaml> <bluddy5> there could potentially be a memory leak in the GC somewhere.
<discocaml> <bluddy5> I don't think many people have put the system through its paces yet
azimut has quit [Remote host closed the connection]
<discocaml> <leostera> interesting
azimut has joined #ocaml
<discocaml> <leostera> i'm gonna assume it is me doing something wrong for now P
<discocaml> <leostera> i'm gonna assume it is me doing something wrong for now 😛
a51 has quit [Quit: WeeChat 4.1.2]
<discocaml> <leostera> but i've tried quiet a few things and i can see _some_ of the riot processes being finalized
<discocaml> <leostera> but not everything, and even if i run gc.force_major periodically
<discocaml> <leostera> i've also tried using only weak ptrs everywhere but one place (a global process table), so that if i remove something from there, it should be gc'able
<discocaml> <bluddy5> what kind of stuff isn't GCed? can you tell?
<companion_cube> (global process table? :/ :/)
<discocaml> <leostera> by the size, i think its the processes themselves – https://github.com/leostera/riot/blob/main/riot/runtime/core/process.ml#L47-L63
<discocaml> <leostera> i see ram usage go up when i spin up say 1M processes, and then as i terminate them, ram usage never goes down
<companion_cube> could also be that you don't give memory back to the OS
<discocaml> <leostera> i had some gc.print_stats somewhere, let me dig 'em out
<discocaml> <bluddy5> releasing to the OS is a different matter
<companion_cube> (I don't recall if jemalloc on 5.1 is enough for that, might be)
<discocaml> <leostera> hmm, so how do i make sure memory is released to the OS periodically/timely?
<companion_cube> try using jemalloc?
<discocaml> <leostera> https://ocaml.org/p/jemalloc/latest ?
<companion_cube> ja
<discocaml> <leostera> this is super useful, thanks @companion_cube!
<discocaml> <leostera> i can clearly see here the growth in allocated/active memory
<discocaml> <leostera> but after i terminate the processes, neither number moves
<companion_cube> ah!! so it's not between malloc and the OS, but really in OCaml
<discocaml> <leostera> ja
<discocaml> <leostera> what i find odd is that i see no change at all even when i "free" all the internal data in a process
<discocaml> <leostera> eg. replace the continuation by an empty one, drop the mailboxes, etc
<companion_cube> even after a full major GC?
<companion_cube> could be fragmentation, too
<discocaml> <leostera> (running the test again with gc.full_major)
<discocaml> <leostera> hmm, no, no visible change
<discocaml> <leostera> wait, yes, its just horrifyingly slow
<discocaml> <leostera> ok, it goes down a little bit but not fully
<discocaml> <leostera> if it was fragmentation, how'd i deal with this?
azimut has quit [Remote host closed the connection]
azimut has joined #ocaml
<companion_cube> :s
<companion_cube> wait for 5.2 I guess
<discocaml> <leostera> 😭
<companion_cube> it's also possible that you do have a leak
<companion_cube> global tables and such do not instill confidence, imho :p
<discocaml> <leostera> i'm gonna bank on my code being leaky
<discocaml> <leostera> at least i have some agency over that haha
<discocaml> <bluddy5> Well, if you figure out it's not your code, make sure to post a bug report.
<discocaml> <leostera> will do
<discocaml> <leostera> is 331100 gc fragments too many fragments
<discocaml> <leostera> yeah, i only see the numebr of live_blocks/workds grow 😦
mima has joined #ocaml
<discocaml> <anmonteiro> how much memory do you have on your machine? space overhead is 120 by default, so I'd say try lowering that quite a bit and see if memory usage goes down
<companion_cube> if you GC by hand it doesn't really matter, does it?
<discocaml> <anmonteiro> oh, I missed he's runing `Gc.full_major`. hmm, probably not
a51 has joined #ocaml
azimut has quit [Remote host closed the connection]
azimut has joined #ocaml
azimut has quit [Remote host closed the connection]
azimut has joined #ocaml
azimut has quit [Remote host closed the connection]
azimut has joined #ocaml
azimut has quit [Remote host closed the connection]
azimut has joined #ocaml
fweht has joined #ocaml
olle has quit [Ping timeout: 264 seconds]
<discocaml> <bluddy5> How much do you guys utilize creating new types for business rule unit separation? For example, I have map-drawing code that works on map pixels, map tile (16 by 16), screen pixels and screen visible tiles. It can get hairy keeping them straight. Would you recommend creating a type for each one instead of using ints?
<discocaml> <bluddy5> business rule => business logic
<companion_cube> private aliases can be good, or phantom types
<discocaml> <Ada> how many people are using ocaml 5 in prod (on codebases that don't use effects)?
<companion_cube> idk if anyone is using ocaml 5 in prod yet :)
<companion_cube> but I'm working on a project that's getting closer to prod and I'm using domains a lot
<discocaml> <bluddy5> generally businesses will only switch once they're sure it's ok
<discocaml> <Ada> companion_cube: that's probably wise
<discocaml> <Ada> i think my main switch is still using 4.13
<discocaml> <bluddy5> I'm on 4.14
<discocaml> <Ada> 4.13 is the version in debian stable and i haven't needed to use In_channel yet, so if its not broke don't fix it
<discocaml> <0aty> I like doing this in rust and haskell, but the ergonomics in ocaml might be a bit different due to the lack of for example overloading arithmetic if you make a bunch of integer types
<discocaml> <0aty> Although maybe doing `T.(...)` is less annoying than trying to not mix up numbers
<discocaml> <Ada> i do wanna port code over to eio if i find an excuse
<companion_cube> 4.14 is fairly robust
Anarchos has joined #ocaml
rgrinberg has joined #ocaml
average has quit [Quit: Connection closed for inactivity]
<discocaml> <Ada> is there a string_of_int equivalant for int64?
<companion_cube> Int64.to_string
<discocaml> <Ada> i have no idea how i missed that and feel very stupid now
<companion_cube> don't :)
rgrinberg has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
neiluj has quit [Quit: neiluj]
deadmarshal_ has quit [Ping timeout: 260 seconds]
Anarchos has quit [Quit: Vision[]: i've been blurred!]
bartholin has joined #ocaml
a51 has quit [Quit: WeeChat 4.1.2]
average has joined #ocaml
Tuplanolla has joined #ocaml
rgrinberg has joined #ocaml
gentauro has joined #ocaml
<discocaml> <leostera> dumb question but do the `v` params of ocalmrun do not work on 5.1?
deadmarshal_ has joined #ocaml
fweht has quit [Quit: Connection closed for inactivity]
trev has quit [Quit: trev]
<discocaml> <leostera> ah yes, dummy me, i should just use the runtime events
<discocaml> <leostera> and when i did, i saw the gc seems to be running just fine
<discocaml> <leostera> so 100% leak on my part – something still holds that reference
a51 has joined #ocaml
pi3ce has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
average has quit [Quit: Connection closed for inactivity]
a51 has quit [Quit: WeeChat 4.1.2]
bartholin has quit [Quit: Leaving]
average has joined #ocaml
<discocaml> <leostera> btw, should the major's look nested like this? 🤔
<discocaml> <leostera> i'm running on one domain here
Serpent7776 has quit [Ping timeout: 260 seconds]
Tuplanolla has quit [Quit: Leaving.]
rgrinberg has quit [Quit: My Mac has gone to sleep. ZZZzzz…]