<hayley>
Does anyone intentionally not use compression when saving SBCL images? I'm trying to prioritise things to work out for my new GC, and compacting before dumping an image is one of those things; image sizes with compression but without compaction are reasonable enough, but no compression leads to very large images.
<beach>
How large is very large?
waleee has joined #commonlisp
<hayley>
From my limited experimentation, about as large as the maximum heap size limit. Which is quite awful.
igemnace has joined #commonlisp
lisp123 has quit [Quit: Leaving...]
<hayley>
...which also doesn't appear to make sense, because I know I allocate in address order, and GC is triggered quite frequently, so the allocator should never manage to allocate into the last pages of the heap.
Posterdati has quit [Remote host closed the connection]
<hayley>
Yep, I managed to mess up counting pages. But my output/sbcl.core file is still 247MB, compared to the 35MB core that a normal version of SBCL uses.
<Shinmera>
hayley: no compression is the default, though anyone using Deploy has been using compression.
<Shinmera>
It used to be the case that some argued for no compression to speed up startup, though that may be less of an issue now with libzstd
<hayley>
Right. And, with the speed of zstd, decompressing could be faster than loading a larger core off a mechanical disk.
<hayley>
It's also possible that the operating system can use copy-on-write for the portion of the heap loaded from an uncompressed core file, but I suspect that benefit is theoretical and that memory gets dirtied quickly enough.
<hayley>
My todo list consists of implementing copying (for part of the heap at a time), re-introducing the generations, adding parallel collection, and some sort of full-heap compaction for saving images. The issue now is trying to work out the best order to do those tasks in; though "investigate why trying to do anything after bootstrapping causes memory faults" is necessarily first still. ):
<hayley>
Well, implementing generations requires that I either implement compaction, or try assigning generations to lines and not pages, which I don't think has been done before. Not doing either would make it very difficult to reclaim memory when tenured objects die. It also appears I should try to reclaim as much memory as possible without moving objects, which makes the latter preferable (though it is still harder).
Oddity has quit [Ping timeout: 276 seconds]
<Shinmera>
hayley: Another thing I personally would want is retention of the ELF-inated core, meaning immovable code and compaction to an ELF-binary.
<Shinmera>
(for the NX port)
<Shinmera>
To be clear, ELFinated core is not the default, but it's an option. Though I have not tried it myself yet.
<hayley>
I think I have an idea for how to do a full compaction, but I have no idea how well it would perform in practise. We perform a full copying GC to defragment, and then copy each object if and only if it would move the object to a lower address. At this point, objects are necessarily closer to the start of the heap, and this hopefully better packed.
cage has quit [Remote host closed the connection]
cage has joined #commonlisp
<hayley>
The second pass is similar to the two-finger compacting algorithm, in a way, but we should have fewer and larger holes in the heap after doing the first copying pass.
Algernon69 has joined #commonlisp
<Shinmera>
hayley: can't you just do something like: find nearest object address larger than barrier, move object to barrier, advance barrier, repeat?
<hayley>
...no, that can't produce good results, as the forwarding pointers still take up space. Though, if I did use actual two-finger compaction, all the forwarding pointers would be on one side of the fingers, and the forwarding pointers would all be dead too.
<Shinmera>
I don't know what forwarding pointers are or what two-finger compaction would be, so I'll just excuse myself from polluting the discussion further :)
waleee has quit [Ping timeout: 244 seconds]
<hayley>
Shinmera: the issue is that, after relocating, I'd need to fix all pointers to the object, which requires producing a mapping of old -> new addresses (and forwarding pointers are one way of implementing the mapping), and that mapping takes up memory too.
<Shinmera>
Wait, why do you need to make a mapping? You just need to know one old and one new address.
<Shinmera>
since it's moving one object at a time.
<hayley>
After moving an object, one needs to update every other object referencing the moved object, so that pointers are still valid.
<Shinmera>
yes.
<Shinmera>
my idea would be quadratic in time since it's essentially a bubble sort kinda deal, but you could still do it in constant memory.
<hayley>
Sure.
<Shinmera>
How is compaction done right now?
<hayley>
Copying GC already does some compaction, so compaction is done by performing two full GCs to move things into place. I can't convince myself it should work, but seemingly it does.
<Shinmera>
heh
<hayley>
I could put a old->new mapping in a separate hash table, come to think of it. It'd use more memory, but no more than copying GC does already.
<hayley>
The only issue with such a compactor is that the new and old locations of an object can overlap, I think SBCL uses memcpy to copy objects, and memcpy is not defined for a source and a destination which overlap. Not too hard to fix, but it's amusingly similar to the bug I found in the Garbage Collection Handbook.
treflip has joined #commonlisp
<hayley>
Nah. I don't need to touch that code, I think. Had better think about it properly today morning; thanks for the talk.
<Shinmera>
In other news, there's only four days left for Kandria's Kickstarter. The next stretch goal would include official modding support and releasing the game's source code, which I would very much like to do. If anyone here knows anyone else that would be interested, please share the project with them: https://www.kickstarter.com/projects/shinmera/kandria
<Josh_2>
hayley: are you writing a new GC for SBCL?
<hayley>
Yes, but it works too well and collects itself, as my C is terrible.
<Shinmera>
to be fair, SBCL's C codebase is... well. Not the nicest, let's say :)
<hayley>
There is a vaguely defined interface between the rest of SBCL and my nasty-in-a-way-that-I-remember code, but it's evidently not much help, as somehow I manage to make Lisp code corrupt itself.
<White_Flame>
while you're in there, could you get the GC code running in lisp instead of c?
<Shinmera>
I'd also like a pony
tyson2 has joined #commonlisp
pranavats has joined #commonlisp
<lagash>
Just one pony?
* contrapunctus
gives lagash an AbstractPonyFactory
Algernon69 has quit [Ping timeout: 276 seconds]
mon_aaraj has quit [Ping timeout: 272 seconds]
rodicm has quit [Ping timeout: 276 seconds]
mon_aaraj has joined #commonlisp
yauhsien has joined #commonlisp
rodicm has joined #commonlisp
yauhsien has quit [Ping timeout: 264 seconds]
rodicm has quit [Ping timeout: 264 seconds]
tyson2 has quit [Remote host closed the connection]
azimut_ has quit [Ping timeout: 268 seconds]
dirtcastle has quit [Ping timeout: 244 seconds]
deadmarshal has quit [Quit: IRCNow and Forever!]
mon_aaraj has quit [Ping timeout: 256 seconds]
deadmarshal has joined #commonlisp
dirtcastle has joined #commonlisp
igemnace has quit [Remote host closed the connection]
Brucio-61 has joined #commonlisp
anticomputer_ has joined #commonlisp
anticomputer has quit [Ping timeout: 268 seconds]
Brucio-61 has quit [Ping timeout: 276 seconds]
subpsunym has quit [Quit: updates]
bege has joined #commonlisp
bege has quit [Changing host]
bege has joined #commonlisp
bege has quit [Quit: leaving]
vassenn has joined #commonlisp
bege has joined #commonlisp
manykwh has quit [Quit: updates]
ebrasca has quit [Ping timeout: 244 seconds]
subpsunym has joined #commonlisp
anticomputer_ has quit [Remote host closed the connection]
anticomputer has joined #commonlisp
subpsunym has quit [Client Quit]
subpsunym has joined #commonlisp
notzmv has quit [Ping timeout: 276 seconds]
<treflip>
contrapunctus: we don't do this in cl!
<jackdaniel>
isn't defclass kind of an abstract pony factor, where pony is a standard class?
ttree has joined #commonlisp
bege has quit [Ping timeout: 244 seconds]
treflip has quit [Quit: i slep]
molson has joined #commonlisp
molson_ has quit [Ping timeout: 240 seconds]
dudek has joined #commonlisp
bege has joined #commonlisp
notzmv has joined #commonlisp
sndr has joined #commonlisp
sander has quit [Ping timeout: 244 seconds]
sndr is now known as sander
<contrapunctus>
xD
cage has quit [Quit: rcirc on GNU Emacs 27.1]
sander has quit [Read error: Connection reset by peer]
sander has joined #commonlisp
vassenn has quit [Quit: Good bye!]
tyson2 has joined #commonlisp
azimut has joined #commonlisp
bege has quit [Ping timeout: 256 seconds]
waleee has joined #commonlisp
<ardon>
Hi, I'm having the following issue with local-time. I'm trying to create a timestamp 2 hours since the start of today via `(local-time:timestamp+ (local-time:today) 2 :hour local-time:+utc-zone+)', but since my `local-time:*default-timezone*' is CEST, I get "@2022-07-09T03:00:00.000000+02:00" instead of what I would expect "@2022-07-09T01:00:00.000000+02:00", even though I'm passing the `local-time:+utc-zone+' optional parameter to
<ardon>
`local-time:timestamp'. Any pointers?
Oddity has joined #commonlisp
orestarod has quit [Quit: Leaving]
<Josh_2>
local-time is dlowe I believe
bege has joined #commonlisp
<contrapunctus>
ardon: use `format-timestring` to check the value
notzmv has quit [Ping timeout: 260 seconds]
bege has quit [Ping timeout: 255 seconds]
<contrapunctus>
ardon: for me (GMT+05:30) - `(adjust-timestamp (today) (offset :hour 2)) => @2022-07-09T07:30:00.000000+05:30`
frgo has quit [Remote host closed the connection]
<contrapunctus>
ardon: hopefully this illustrates the issue and provides an alternative - https://paste.rs/J08.lisp
tyson2 has quit [Remote host closed the connection]
<ardon>
Thanks contrapunctus. Man, this is so odd. Your last example would be exactly what I need, but taking out the call to `format-timestring' yields the timezone-specific value, which is what I don't want
<contrapunctus>
I've been struggling with the same issues with the same library myself 😅️
<contrapunctus>
ardon: what's the use case? i.e. what do you want to do, when you've got the time in UTC?
bege has joined #commonlisp
frgo has joined #commonlisp
<ardon>
contrapunctus: I'm trying to set automatic light/dark themeing, and I need to check if the current time is greater than a user-defined threshold (1 hour in the above example) to toggle the theme.
vassenn has joined #commonlisp
orestarod has joined #commonlisp
<contrapunctus>
ardon: the timestamp in all the examples I gave is the same; you may be getting confused by the printed representation
<contrapunctus>
ardon: behind the scenes, it's all UNIX epoch timestamps, I think
<contrapunctus>
(all UTC)
jjkola has joined #commonlisp
<jjkola>
hi
<contrapunctus>
hello jjkola
<ardon>
contrapunctus: The last one isn't though, and it's I'd want back from an `adjust-timestamp' or `timestamp+' call.
<ardon>
Oh my mistake, it is.
vassenn has quit [Quit: Good bye!]
rodicm has joined #commonlisp
rodicm has quit [Client Quit]
Guest8978 has quit [Quit: WeeChat 3.5]
<contrapunctus>
Hah, I was momentarily confounded :D
cosimone has quit [Remote host closed the connection]
cosimone has joined #commonlisp
<jjkola>
I wanted to help with https://github.com/sharplispers/prepl/pull/4 by adding support for both old and new call, and there was a mention about swank/backend:with-symbol. Is there some general package where I could find similar functionality? Or do I need to re-implement it in prepl?
akoana has joined #commonlisp
frgo has quit [Ping timeout: 256 seconds]
dra has joined #commonlisp
causal has quit [Quit: WeeChat 3.5]
<pjb>
jjkola: AFAIK, it's the only library where it's defined in. So you'll have to duplicate it.
<jjkola>
ok, thanks
<Josh_2>
What about slynk/backend:with-symbol :thinking: :joy:
<pjb>
Josh_2: I don't know slynk, and I said As Far As I Know.
<Josh_2>
I'm sure you do know
<Josh_2>
its Sly, the fork of SLIME
cosimone has quit [Remote host closed the connection]
saura has joined #commonlisp
<jjkola>
I don't think I will want to add slynk as a dependency to prepl :D
X-Scale` has joined #commonlisp
<Josh_2>
Just MOPing around :sunglasses:
X-Scale has quit [Ping timeout: 264 seconds]
X-Scale` is now known as X-Scale
saura has quit [Ping timeout: 244 seconds]
cosimone has joined #commonlisp
frgo has joined #commonlisp
mrcom has quit [Remote host closed the connection]
mrcom has joined #commonlisp
bege has quit [Ping timeout: 244 seconds]
frgo has quit [Ping timeout: 244 seconds]
cosimone has quit [Remote host closed the connection]
cosimone has joined #commonlisp
mrcom has quit [Remote host closed the connection]
mrcom has joined #commonlisp
bege has joined #commonlisp
bege has quit [Ping timeout: 240 seconds]
frgo has joined #commonlisp
frgo has quit [Ping timeout: 240 seconds]
bege has joined #commonlisp
bege has quit [Ping timeout: 276 seconds]
frgo has joined #commonlisp
anticomputer has quit [Remote host closed the connection]
anticomputer has joined #commonlisp
frgo has quit [Ping timeout: 244 seconds]
notzmv has joined #commonlisp
bege has joined #commonlisp
frgo has joined #commonlisp
bege has quit [Ping timeout: 244 seconds]
bege has joined #commonlisp
bege has quit [Ping timeout: 264 seconds]
bege has joined #commonlisp
bege has quit [Ping timeout: 256 seconds]
jmdaemon has joined #commonlisp
<ardon>
dlowe: Just chiming in to say that if you have any pointers on how to operate on timestamps in a timezone-dependent (rather than just timestrings) way I'd be happy to hear them, as I'm currently blocked
saura has joined #commonlisp
X-Scale` has joined #commonlisp
X-Scale has quit [Ping timeout: 240 seconds]
saura has quit [Ping timeout: 276 seconds]
X-Scale` is now known as X-Scale
dra has quit [Ping timeout: 260 seconds]
Lord_of_Life has quit [Ping timeout: 240 seconds]
Lord_of_Life has joined #commonlisp
bege has joined #commonlisp
bege has quit [Ping timeout: 256 seconds]
mitchconnor has joined #commonlisp
azimut_ has joined #commonlisp
azimut has quit [Ping timeout: 268 seconds]
mitchconnor has quit [Client Quit]
dudek has quit [Quit: Leaving]
cosimone has quit [Remote host closed the connection]