xlymian has quit [Read error: Connection reset by peer]
certainty has joined #commonlisp
certainty has quit [Ping timeout: 240 seconds]
Pirx has quit [Ping timeout: 252 seconds]
Lord_of_Life has quit [Ping timeout: 258 seconds]
Lord_of_Life has joined #commonlisp
msavoritias has quit [Ping timeout: 245 seconds]
<gendl__>
Hi, has anyone ever tried using the fork() system call from a Lisp application? My use case is I'd like to run several instances of a server application on a host, each one listening on a different port.
<gendl__>
Currently I am doing this by starting so many of my application executable from a shell script. In htop we can see _some_ sharing in the SHR column. But I think we can do better in terms of sharing by using fork(), based on what I've learned about Linux fork() --- it incurs almost no additional memory cost to fork a child process, because initially it shares all memory with the parent
<gendl__>
only if it stores anything to RAM, does the OS give a new memory page for it (copy-on-write strategy). This would be perfect for my use case, if I could figure out how to spawn a child process then start its webserver on a different port (I would fork before starting the webserver or any other services).
certainty has joined #commonlisp
<gendl__>
Maybe this is more of an OS question than a Lisp question. But i'm just trying to figure out how to get control over the child process after the fork, so i can have it do its needed initialization.
certainty has quit [Ping timeout: 246 seconds]
<moon-child>
gendl__: I would not expect that to work well or reliably
kevingal has quit [Ping timeout: 258 seconds]
<moon-child>
gendl__: I believe you can listen on multiple sockets using select/poll
<moon-child>
if that doesn't work, then I would use multiple threads and mux the connections into a single central queue (I think lparallel has a passable one)
<gendl__>
for (reasons), I want to use a separate OS process for each listener, not threads within a single heavyweight process.
<gendl__>
I'm happy to have the OS manage these processes and e.g. be able to kill any one from the OS level if it misbehaves or becomes unresponsive -- i'm not prepared to do all that from within a single Lisp application using threads. Besides, there is no need for shared data among these listeners, so there's no operational reason to have them all living in the same memory space.
<moon-child>
I didn't suggest to use separate threads; I suggested to use a _single_ thread in a single process
<moon-child>
regardless, if that is what you want, then I suggest using separate processes unless you have a strong reason to believe that this is causing performance problems
<gendl__>
it's just that for economic reasons I'm trying to reduce overall RAM usage on the system, so any improvement in sharing of memory (not to be confused with shared data intentionally used by the program).
<gendl__>
... would be helpful
<gendl__>
launching them as separate processes from scratch means that each one uses like 50MB of RAM out of the starting gate. I think we could do much better than that by using fork() in principle. But moon-child: it sounds like in your experience, or from what you know, that's not a good strategy for (reasons)
<gendl__>
I think the main complication is the newly forked child process will necessarily share a bunch of things with the parent - file handles etc. So we would need to figure out what is shared, what can continue to be shared, what needs to be created anew, how to initialize all that... Seems like an initialization problem which, if that could be solved, the rest would work similarly to what we're already doing now just more
<gendl__>
memory-efficiently.
<moon-child>
unless it explicitly documents otherwise, your implementation may make assumptions that are incompatible with any use of fork. For example, it could use multiple threads to do garbage collection
<moon-child>
the performance advantages of fork are dubious
<moon-child>
having everything in one thread and one process is always going to result in the lowest memory usage
<gendl__>
Sure. But this is a heavy KBE application whose objects are going to gobble up tens of MB of RAM. If i have hundreds of these things running in a single Lisp image the memory of that single image would blow up into the gigabytes. I don't know how efficiently the GC would work at that point. And there's no operational reason to have all the sessions living in the same memory space.
<gendl__>
basically i want my cake and to eat it too. Sharing memory but not appearing (from inside each process) to be sharing memory. Maybe too much to ask.
<moon-child>
I don't understand your objections to having everything in one process
habamax has quit [Ping timeout: 255 seconds]
<moon-child>
should I make every function its own process and have them communicate over pipes because there is no operational reason to have two functions living in the same memory space?
tyson2 has joined #commonlisp
jobhdez has joined #commonlisp
<jobhdez>
hello beach -- if you have time could you give me your perspective as to whether hygienic macros in racket are actually an improvement over common lisp's macros? anybody else can also chime in
<gendl__>
a few things:
<gendl__>
2. All these heavy threads (maybe hundreds of them) all living in the same Lisp image means the garbage collector has to deal with all of their garbage, all together. The garbage from each thread will collectively burden all the others. With running separate OS processes, each one's GC only has to deal with that one's garbage.
<gendl__>
1. If one of the threads gets into trouble e.g. goes into an infinite loop, it's up to my Lisp program to deal with that. With separate processes, the OS can deal with it.
<gendl__>
and follow up to (1) if one of the threads gets into trouble, it won't automatically affect all the others.
<gendl__>
I understand your function analogy. So it's a matter of where to draw the line on these things.
<moon-child>
jobhdez: my position is that scheme's hygienic macros add a great deal of complexity (syntax objects??) for questionable gain. However, I think a more declarative way of writing macros couold be valuable, especially for tooling
<moon-child>
gendl__: first, I suggested to use ONE thread and one process, not one thread per socket
dnhester26 has quit [Remote host closed the connection]
<moon-child>
(if you want to use concurrency to increase throughput, then, unless you expect load to be a priori balanced--which it probably won't be--you probably want to make your own load balancer; i.e., a many-to-many mux from connections to threads, likely with one thread per core)
<jobhdez>
moon-child thanks. common lisps macros are good enough. sbcl is good enough then. thanks. anyways, wouldnt it be cool if the sbcl guys made a discourse for sbcl?
<moon-child>
I'm not sure what kind of trouble you expect your code to get into nor why you expect the operating system to be better equipped to deal with it than you. If you get into an infinite loop, nobody is going to care, because there is no way to tell an infinite loop from a finite one. If you crash--whether the sole thread in a process, or one of many--you will just get dropped on the floor.
<moon-child>
Contrariwise, it seems to me that lisp code is in a much better position to deal with malfeasances in lisp code, with things like restarts. And I think somebody made an implementation of supervisor trees too
Krystof has quit [Ping timeout: 245 seconds]
<moon-child>
I'm not sure what you mean by 'The garbage from each thread will collectively burden all the others.' If the concern is parallelism, the mark-region gc recently written for sbcl can parallelise and scales ok
<gendl__>
Ok thanks. I'll play with some of these ideas.
habamax has joined #commonlisp
Oladon has quit [Quit: Leaving.]
xlymian` has joined #commonlisp
xlymian` has quit [Client Quit]
xlymian has joined #commonlisp
xlymian has quit [Client Quit]
xlymian has joined #commonlisp
xlymian` has joined #commonlisp
xlymian` has quit [Remote host closed the connection]
bh34e5 has joined #commonlisp
jobhdez has quit [Ping timeout: 250 seconds]
bh34e5_ has joined #commonlisp
bh34e5 has quit [Ping timeout: 255 seconds]
certainty has joined #commonlisp
certainty has quit [Ping timeout: 246 seconds]
<Pixel_Outlaw>
Does anyone happen to know the Lisp "family" that describes the Common Lisp likes? I want to say MacLisp InterLisp and (Spice?) Lisp? But I'm not sure if it hails from the original specification on the IBM machine.
<Pixel_Outlaw>
Sometimes, I want to draw distinction between Lisp families and I'm not sure where to put the defun bunch.
xlymian has quit [Ping timeout: 255 seconds]
<aeth>
could just call it mainstream or historic Lisp
<Pixel_Outlaw>
Or Heritage lisp ...
<aeth>
there isn't really a name for it, just names for everything else (Schemes, Clojure and, uh, idk, Clojure-likes?), etc.
<Pixel_Outlaw>
Yeah, kind of 3 main families at the moment.
<aeth>
but it's usually pretty obvious (unless it's designed to e.g. bridge between the two) which Lisp it's going for, a historic/CL/elisp kinda lisp, a Scheme kinda lisp, or a Clojure kinda lisp
<aeth>
though Scheme-like Lisps generally just call themselves Schemes even if they're slightly (Racket) or substantially different
<Pixel_Outlaw>
VisualLISP is CL-ish (if CL got barn damage) maybe the Lisp we just don't talk about.
jobhdez has joined #commonlisp
* Pixel_Outlaw
regresses back into the bushes to open the floor for real CL questions.
msavoritias has joined #commonlisp
certainty has joined #commonlisp
bh34e5__ has joined #commonlisp
bh34e5_ has quit [Ping timeout: 240 seconds]
certainty has quit [Ping timeout: 255 seconds]
xlymian has joined #commonlisp
t400 has quit [Quit: Konversation terminated!]
markb1 has joined #commonlisp
jobhdez has quit [Ping timeout: 250 seconds]
tibfulv_ has joined #commonlisp
tibfulv has quit [Ping timeout: 240 seconds]
edr has quit [Quit: Leaving]
kmroz has quit [Server closed connection]
kmroz has joined #commonlisp
fiddlerwoaroof has quit [Server closed connection]
fiddlerwoaroof has joined #commonlisp
certainty has joined #commonlisp
habamax has quit [Remote host closed the connection]
certainty has quit [Ping timeout: 246 seconds]
habamax has joined #commonlisp
pl has quit [Server closed connection]
pl has joined #commonlisp
tyson2 has quit [Remote host closed the connection]
certainty has joined #commonlisp
certainty has quit [Ping timeout: 252 seconds]
Meritamen has joined #commonlisp
Meritamen has quit [Remote host closed the connection]
waleee has quit [Ping timeout: 264 seconds]
habamax has quit [Ping timeout: 240 seconds]
cedb has quit [Quit: WeeChat 4.0.5]
bh34e5__ has quit [Ping timeout: 246 seconds]
certainty has joined #commonlisp
certainty has quit [Ping timeout: 264 seconds]
habamax has joined #commonlisp
certainty has joined #commonlisp
certainty has quit [Ping timeout: 246 seconds]
notzmv has quit [Ping timeout: 264 seconds]
certainty has joined #commonlisp
certainty has quit [Ping timeout: 246 seconds]
larix1 has joined #commonlisp
shka has joined #commonlisp
Lycurgus has joined #commonlisp
certainty has joined #commonlisp
certainty has quit [Ping timeout: 252 seconds]
rgherdt has joined #commonlisp
igemnace has joined #commonlisp
dcb has quit [Quit: MSN Messenger 4.1.1]
phoe has quit [Server closed connection]
phoe has joined #commonlisp
pve has joined #commonlisp
CO2 has quit [Ping timeout: 245 seconds]
certainty has joined #commonlisp
habamax has quit [Remote host closed the connection]
certainty has quit [Ping timeout: 264 seconds]
certainty has joined #commonlisp
Lycurgus has quit [Quit: leaving]
pranavats has left #commonlisp [Error from remote client]
yosef` has joined #commonlisp
dajole has quit [Quit: Connection closed for inactivity]
habamax has joined #commonlisp
Pixel_Outlaw has quit [Quit: Leaving]
notzmv has joined #commonlisp
thollief has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
mgl_ has joined #commonlisp
zetef has joined #commonlisp
bendersteed has joined #commonlisp
zetef has quit [Ping timeout: 255 seconds]
blunder has joined #commonlisp
zetef has joined #commonlisp
blunder has left #commonlisp [#commonlisp]
danza has joined #commonlisp
pranavats has joined #commonlisp
danza has quit [Read error: Connection reset by peer]
anticrisis has quit [Read error: Connection reset by peer]
cayley5_ has quit [Server closed connection]
cayley42 has joined #commonlisp
danse-nr3 has joined #commonlisp
zetef has quit [Remote host closed the connection]
yosef` has quit [Ping timeout: 250 seconds]
Pirx has joined #commonlisp
dstein64 has quit [Server closed connection]
dstein64 has joined #commonlisp
igemnace has quit [Read error: Connection reset by peer]
zxcvz has joined #commonlisp
<hayley>
I'm brushing off my GC benchmark suite; does anyone have any GC-intensive programs I can add? Such programs should try to be deterministic; for example the seeds for any random number generators should be fixed, and the benchmarked part of the program should not touch the filesystem.
dino_tutter has joined #commonlisp
spec has quit [Server closed connection]
spec has joined #commonlisp
<Shinmera>
my trial physics engine was a lot more gc intensive before I started dx-ing a bunch of stuff :v
<Shinmera>
it still makes a bunch of trash, but idk if the pressure is high enough for your tastes
<larix1>
just tried lem again, its improved a lot and I might actually use it daily now
<Shinmera>
it's also not fully deterministic since frame timing impacts other parts, and that's dependent on GPU/outside factors
<Shinmera>
larix1: Cool! let cxxxr know, I'm sure he'd be happy to hear it! :)
<Shinmera>
hayley: but if you want for fun anyway, (ql:quickload :trial-examples) (trial-examples:launch :physics)
igemnace has joined #commonlisp
<Shinmera>
note that it needs the latest 2.3.10 patches for the DX optimisation to work correctly
<Shinmera>
don't know if you've rebased all your stuff
<Shinmera>
(thanks karlosz)
<hayley>
I've rebased. Tricky part though is that I want to run on a headless VPS, as the benchmarks take annoyingly long and my room is already warm enough now that it is November.
<Shinmera>
ah...
<Shinmera>
there is a way to run GL headless, but the VPS probably wouldn't have a GPU :/
<Shinmera>
could add a headless backend with some elbow grease, but it's a bit more effort than I can just expend on something I don't need right now :u
<hayley>
Don't worry about it.
<Shinmera>
I should probably have that for tests anyway
<Shinmera>
but lol, lmao at test suites for trial
zxcvz has quit [Quit: zxcvz]
Inline has quit [Quit: Leaving]
Inline has joined #commonlisp
jmdaemon has quit [Ping timeout: 246 seconds]
avocadoist has joined #commonlisp
donleo has joined #commonlisp
varjag has joined #commonlisp
gxt has quit [Remote host closed the connection]
gxt has joined #commonlisp
habamax has quit [Quit: ERC 5.5.0.29.1 (IRC client for GNU Emacs 29.1.90)]
amb007 has quit [Ping timeout: 264 seconds]
amb007 has joined #commonlisp
random-nick has joined #commonlisp
gxt has quit [Remote host closed the connection]
gxt has joined #commonlisp
yitzi has joined #commonlisp
Cymew has joined #commonlisp
CO2 has joined #commonlisp
danse-nr3 has quit [Ping timeout: 246 seconds]
tyson2 has joined #commonlisp
rgherdt_ has joined #commonlisp
rgherdt has quit [Read error: Connection reset by peer]
hefner_ has quit [Server closed connection]
hefner has joined #commonlisp
Inline has quit [Ping timeout: 255 seconds]
edr has joined #commonlisp
danse-nr3 has joined #commonlisp
green_ has joined #commonlisp
pfdietz has joined #commonlisp
cage has joined #commonlisp
lucasta has joined #commonlisp
doyougnu- has joined #commonlisp
doyougnu has quit [Ping timeout: 240 seconds]
green_ has quit [Ping timeout: 255 seconds]
rgherdt has joined #commonlisp
rgherdt_ has quit [Ping timeout: 264 seconds]
green_ has joined #commonlisp
rgherdt_ has joined #commonlisp
Inline has joined #commonlisp
rgherdt has quit [Ping timeout: 246 seconds]
bh34e5 has joined #commonlisp
chomwitt has joined #commonlisp
green_ has quit [Remote host closed the connection]
green_ has joined #commonlisp
shka has quit [Ping timeout: 272 seconds]
green_ has quit [Ping timeout: 246 seconds]
shka has joined #commonlisp
shka has quit [Read error: Connection reset by peer]
shka has joined #commonlisp
jonatack has quit [Read error: Connection reset by peer]
jonatack has joined #commonlisp
larix1 has quit [Ping timeout: 240 seconds]
larix1 has joined #commonlisp
green_ has joined #commonlisp
zxcvz has joined #commonlisp
zxcvz has quit [Client Quit]
lucasta has quit [Remote host closed the connection]
shka has quit [Ping timeout: 246 seconds]
larix1 has quit [Ping timeout: 240 seconds]
randomperson132 has joined #commonlisp
dustinm`_ has quit [Server closed connection]
dustinm` has joined #commonlisp
Cymew has quit [Ping timeout: 252 seconds]
donlcn has joined #commonlisp
donleo has quit [Ping timeout: 240 seconds]
lcn__ has joined #commonlisp
tyson2 has quit [Read error: Connection reset by peer]
donlcn has quit [Ping timeout: 240 seconds]
bendersteed has quit [Quit: bendersteed]
Gleefre has joined #commonlisp
dino_tutter has quit [Ping timeout: 264 seconds]
cage has quit [Remote host closed the connection]
Lycurgus has joined #commonlisp
lcn__ has quit [Ping timeout: 246 seconds]
shka has joined #commonlisp
kevingal has joined #commonlisp
donleo has joined #commonlisp
yitzi has quit [Remote host closed the connection]
danse-nr3 has quit [Read error: Connection reset by peer]
chomwitt has quit [Ping timeout: 264 seconds]
larix1 has joined #commonlisp
danse-nr3 has joined #commonlisp
waleee has joined #commonlisp
jrm has quit [Quit: ciao]
jrm has joined #commonlisp
zxcvz has joined #commonlisp
t400 has joined #commonlisp
tyson2 has joined #commonlisp
tyson2` has joined #commonlisp
tyson2` has quit [Remote host closed the connection]
tyson2 has quit [Remote host closed the connection]
tyson2 has joined #commonlisp
igemnace has quit [Remote host closed the connection]
varjag has quit [Quit: ERC (IRC client for Emacs 27.1)]
lagash has quit [Remote host closed the connection]
lagash has joined #commonlisp
chomwitt has joined #commonlisp
dom2 has joined #commonlisp
dom2 has quit [Ping timeout: 272 seconds]
Lycurgus has quit [Quit: leaving]
chomwitt has quit [Ping timeout: 240 seconds]
pfdietz has quit [Quit: Client closed]
pfdietz has joined #commonlisp
oak has joined #commonlisp
zest has joined #commonlisp
waleee has quit [Ping timeout: 252 seconds]
notzmv has quit [Read error: Connection reset by peer]
Guest63 has joined #commonlisp
dino_tutter has joined #commonlisp
Gleefre has quit [Remote host closed the connection]
danse-nr3 has quit [Ping timeout: 252 seconds]
kevingal has quit [Ping timeout: 240 seconds]
randomperson132 has quit [Quit: Client closed]
tyson2 has quit [Remote host closed the connection]
msavoritias has quit [Ping timeout: 246 seconds]
zxcvz has quit [Quit: zxcvz]
kevingal has joined #commonlisp
dcb has joined #commonlisp
theBlackDragon has quit [Server closed connection]
<bike>
"ascii characters" makes it slightly tricky. there is no type that has to include all ascii characters. in practice character does and base-char probably does, but they probably also include non-ascii characters.
<lispmacs[work]>
what's the element type for an 8-bit byte number?
<gilberth>
Either (unsigned-byte 8) or (signed-byte 8). But ASCII is 7 bit. :-)
<edgar-rft>
sounds as if '(integer 0 127) is what you're asking for, but note that ANSI CL doesn not enforce CHAR-CODE to be in ASCII order
larix1 has quit [Ping timeout: 255 seconds]
chrcav has quit [Quit: leaving]
<Shinmera>
bla bla no implementation alive uses anything but unicode
<Shinmera>
though for instance sbcl restricts base-char to ascii, iirc
<moon-child>
I want a variant of m-v-b which is fast and which errors if it does not get exactly the number of values it expects; does such a thing exist? mv-list and mv-call appear to be slow in existing implementations (I was surprised to find sbcl does not inline an mv-called function)
<bike>
no inline? that's a little surprising. maybe it only inlines with &optional and &rest, since it matches m-v-b and then it doesn't have to check the values.
<gilberth>
Unicode? Which? UTF-16 or UCS-4?
<Shinmera>
unicode is unicode
<Shinmera>
the encoding is another matter
<Shinmera>
though everything uses utf-32 because strings are arrays and need constant time access since a lot of functions rely on that sequential access
<bike>
huh, i can't make it inline, indeed. weird.
<gilberth>
Shinmera: Yes, but when your strings are encoded in UTF-16, CHAR-CODE doesn't give you the Unicode code point.
<bike>
it had better give you the unicode code point
<Shinmera>
gilberth: eh?
<bike>
if char-code is giving you half a codepoint because the implementation uses utf-16 you should file a bug report
<Shinmera>
gilberth: you are confusing things.
<gilberth>
Shinmera: Some Lisps expose UTF-16 with the character codes (CHAR-CODE) to you.
<bike>
which? why? what does that even mean?
<moon-child>
actually it seems sbcl will inline if it knows how many values there are
<bike>
like they give you the low 16 bits of a character's code point?
<moon-child>
which is probably good enough for my application
<bike>
that seems rather problematic given that different characters could have the same codes.
<gilberth>
bike: No, they expose you to UTF-16. And use surrogate pairs if needed.
<bike>
what does exposing me to utf-16 mean for char-code?
<moon-child>
gilberth: does that mean I can't say #\x, where x is a unicode codepoint that can't be represented in a single utf16 code unit?
<moon-child>
(but I can say it when x is >7 bits but does fit in a single code unit?)
<gilberth>
moon-child: Correct.
chomwitt has joined #commonlisp
<moon-child>
Thank you. I dislike it.
<gilberth>
That is (code-char #x1F600) won't give you a smiley. You would need two characters for that to form the surrogate pair.
<bike>
so they don't support the unicode character set, they support a subset of it where i can't use the weightlifter emojis or old chinese
<bike>
thus taking me back to "file a bug report"
<gilberth>
moon-child: Unicode once was thought to be just 16-bit. So the very early adopters are affected. That is e.g. Allegro Common Lisp and due to Java ABCL.
certainty has joined #commonlisp
thollief has quit [Remote host closed the connection]
<moon-child>
bike: meh. What is a 'character' is not clearly defined. And I cannot stuff e.g. every grapheme into a character in any cl implementation I know of
<gilberth>
moon-child: char-code-limit is the problem with that. What would it be?
<moon-child>
I dunno if that's the only problem but it's certainly a problem
<bike>
moon-child: character isn't clearly defined, but defining it to be the correspondent to maybe half a unicode thing is pretty dumb.
<moon-child>
cl was not set up to deal with strings as they exist today
<moon-child>
ditto floating point
<moon-child>
kinda bad timing, unfortunately
<moon-child>
bike: not arguing there
<gilberth>
I see no other really. Don't characters behave like integers. That is can (eq #\a #\a) as well be NIL?
<bike>
i mean, yes, though i don't think any actual implementation has it be nil
<gilberth>
So there could be "bigchars".
<bike>
sure, but when you deal with bignums you don't end up with an array where you can index in to get half a bignum.
<gilberth>
But you need to come up with a char-code.
<gilberth>
bike: I don't get you. You can't get at half a character.
<moon-child>
bike: something like abcl or clisp that boxes everything?
<yitzi>
CMUCL is UTF-16 as I recall.
<gilberth>
Since when does CLISP box everything? I don't know about ABCL though.
<gilberth>
CLISP uses tagging like everybody else.
certainty has quit [Ping timeout: 255 seconds]
<moon-child>
perhaps it doesn't adn I am confused. I don't know clisp. Abcl definitely does though because java
<bike>
gilberth: from what you were saying, my understanding was that ín this implementation you either cannot reference post-65535 code point characters, or you get half a surrogate pair which is considered by the implementation to be a character.
<gilberth>
CLISP doesn't. Why would it?
<bike>
i will note that i don't know all the details of how surrogate pairs work because i have thankfully never been compelled to understand the details of utf-16
<gilberth>
bike: I thought you referred to possible "bigchar"s.
<bike>
bigchars i have no problem with.
<bike>
lisp allows the implementation detail of bigchars to be exposed only with eq and arguably with base-char versus character. this is closely analogous to how fixnums work and i don't think it's a problem.
<bike>
but, as i understood it, you were talking about the choice of character encoding somehow influencing other things, like what char-code does.
<gilberth>
I was saying that but CHAR-CODE-LIMIT I see nothing in ANSI-CL that would forbid having bigchars. Then moon-child can have his one grapheme = on CL:CHARACTER. Something which I would like to see as well.
<bike>
you could maybe revive the weird provisions about char-int versus char-code, but i don't know how all the details would work out
<gilberth>
bike: I was pointing out that the assumption that CHAR-CODE these days is an Unicode code point fails with Lisp that expose you to that UTF-16 encoding. Which perhaps is a historical accident, if you like.
<bike>
okay, but what would exposing me to that utf-16 encoding mean, exactly?
<bike>
like can i write #\𐀋? What is (char-code #\𐀋)?
<bike>
You were saying that (code-char #x1f600) won't give a smiley - that you need two characters to form a surrogate pair. ergo, you're talking about defining "character" to mean half a unicode thing
<gilberth>
bike: That won't solve it. This char-int was about including the font and modifier bits. I would need to look it up, but I believe #\a in one font would still compare char= to #\a in another or so.
<bike>
alright. i just threw that out there, like i said i am no expert in that bit of the standard
<gilberth>
bike: When you want to encode a code point beyond #xFFFF with UTF-16 two 16-bit code points are used. That are the surrogates. One is #xDCxx and the other is #xD8xx.
<gilberth>
That is the string "😀" (U+1F600) would be of length 2 with those Lisps.
<bike>
Okay, so that's what I mean, you're accessing half a character (or rather, you've defined "character" to mean half of what other lisps consider a character)
<bike>
this isn't tied to utf-16 - you could represent strings with utf-16 internally but finagle char-code to use code points - it's just a weird implementation decision
<gilberth>
#xD83D and #xDE00 sayz Google.
<gilberth>
bike: ,(code-char #xD83D)
<ixelp>
(code-char #xD83D) => NIL
rgherdt_ has quit [Ping timeout: 264 seconds]
<gilberth>
There is no such char.
<bike>
but there would be, in this implementation. right? that's what you just said. the string is of length 2. there are two characters. or do they have some other random codepoints unrelated to the surrogate pairs for some reason?
<bike>
other random char codes*, rather
<gilberth>
bike: They are the surrogate pairs.
<bike>
...which in this lisp is a character. an object of type cl:character. yes?
<bike>
I do (aref "😀" 1). I get something. (typep (aref "😀" 1) 'character) => T. (char-code (aref "😀" 1)) => probably #xde00.
<yitzi>
Yes, on CMUCL (code-char #xD83D) => #\U+D8#D
randomperson132 has quit [Quit: Client closed]
<gilberth>
bike: With ACL I get (format nil "~{~X~^ ~}" (map 'list 'char-code "😀") => "d83d de00"
<bike>
in my opinion an implementation doing this is standard conforming but kind of broken. it's letting the encoding intrude on the character repertoire. mixing abstraction levels bla bla bla.
<gilberth>
bike: On that line of reasoning an implementation could also have char-code-limit = 256 and expose you to UTF-8.
<bike>
it would just as well to be permissible to use utf-16 for strings internally but expose smiley face characters and have the string be length one, by accepting non constant time access to the array bla bla bla. they are not just choosing to use utf-16, they are choosing to redefine "character" to be closely defined to encoding.
<bike>
gilberth: yes, which would also be pretty silly.
<bike>
closely tied to encoding*
<gilberth>
I regard that as a historical accident.
<yitzi>
bike: It makes parsing a character streams a bit irritating. In shasht (JSON parsing) I have to look for surragate pairs and convert them unless the implementation is UTF-16, which in that case the pairs need be preserved in the string.
<bike>
that it's silly, or that they use utf-16 in this way?
<bike>
yitzi: does babel not have a utf-16 decoder you can use? or am i misunderstanding
<bike>
gotta catch a bus - ta ta for now
<yitzi>
bike: It does, but the JSON allows the pairs no matter what, and babel is kind of slow. Since shasht is for fast network comms, etc.
<gilberth>
bike: 16-bit once suffices for Unicode. This is what I mean by historical accident. They punted and took the easy route and adopted the UTF-16 hack.
<gilberth>
That Java and Windows use UTF-16 as well might also have been a consideration.
<gilberth>
But really, this is confusing external with internal representation. IMHO
yitzi has quit [Remote host closed the connection]
monospod has quit [Quit: Leaving]
<gilberth>
And I'm with moon-child. Even Unicode really is an encoding. I would prefer that a CL:CHARACTER is a grapheme and not a code point.
msavoritias has quit [Ping timeout: 246 seconds]
cmack has quit [Ping timeout: 264 seconds]
xlymian has quit [Ping timeout: 255 seconds]
<Pirx>
variable length character representation is evil
<gilberth>
There is more evil. Like RTL and LTR. Unicode is half a markup language. I waiting for means to colour you characters, or change the size.
rgherdt_ has joined #commonlisp
<Pirx>
I remember a guy (rip) that was porting a language to linux bitterly complaining about the system used for unicode... not sure what it was, utf8 i guess, saying that provision for number of characters was ridiculous, that with 16 bits you could encode every character in existence, including klingon
<Pirx>
I believe that chinese has those many ideograms, but are those really characters?
<gilberth>
What else would it be?
<Pirx>
words
<boigahs>
UTF-8 was designed for networking and I'm pretty sure it has more bits available for characters than UTF-16...
<Pirx>
but are all the characters the same lenght?
<boigahs>
Chinese characters are not words, they are morphemes
<gilberth>
You're mixing up things here. UTF-8 is an external encoding of a stream of Unicode code points.
<Pirx>
whatever, it's one problematic script making life difficult for non-problematic scripts
<gilberth>
UTF-16 is another. As is UCS-4.
<gilberth>
The idea of Unicode was to work for every script. Do you want the times back at which there are many coding systems?
<gilberth>
I don't. It was a mess. Even here.
<Pirx>
I can't do anything about it, except complaining
<boigahs>
Chinese is pretty straightforward compared to other stuff. It's just "there's a lot of it"
<boigahs>
No combining characters or modifiers or connecting separate characters...
<Pirx>
but one thing is true: if we used 16 bits for character no matter what, life would be easier for everybody
<gilberth>
Indeed, Chinese is just a lot of funny characters. Those formatting-like modifiers in Unicode which construct those graphemes are the evil part.
<gilberth>
Pirx: No.
<gilberth>
Chinese would not fit and we would still have different encodings used in different locations. And perhaps many to chose from.
rgherdt__ has joined #commonlisp
<Alfr>
Pirx, unicode already has defined more than 2^16 code points.
<gilberth>
Chinese doesn't make Unicode a iota more complicated. It's silly things like "skin tone modifier" and such. And diacritics.
tyson2 has joined #commonlisp
<Pirx>
Alfr, that's tricky... how many of those are real characters of real languageS?
<gilberth>
There are two ways to encode "å" (and that is not Chinese). One is using the code point for 'å' borrowed from 8859-1 and the other is using "a" and some code point saying "please put a ring above".
<Pirx>
chinese also has pinyin, that's a phonetical representation, so they would be served anyway
<Alfr>
Pirx, don't know, and doesn't matter as those are already assigned.
<Alfr>
Apart from that, storage is sufficiently cheap, that the next encoding shouldn't impose any upper limit on character codes imo.
<ixelp>
“◌̊” U+030A Combining Ring Above Unicode Character
<Pirx>
the question is not limiting number of characters, it's using an internal representation with variable length: it will be solved in the next format
<Pirx>
something like 32 bits characters
<gilberth>
Huh? UTF-8 is variable length as well. And ASCII compatible.
<Alfr>
Pirx, no, that's a 32-bit limit.
<Pirx>
????
<Alfr>
Pirx, if you internal representation admits to only 32-bit characters, then you only can have 2^32 of them.
<Pirx>
so...
<Pirx>
how many characters do you need?
<gilberth>
What are we talking about?
<Alfr>
I don't want to have any limit, so that as to make inventing a new standard unnecessary if a finite code point space runs out.
<Pirx>
my first line: variable length character representation is evil
<bike>
gilberth: i guess we are in agreement about it being a hack then (confusing internal and external representation). sorry if i got a bit snippy there.
<Alfr>
I'd don't think it's evil, transitioning from one to an other is way more evil.
<Pirx>
I'm talking about text
random-nick has quit [Ping timeout: 246 seconds]
<gilberth>
bike: Good! :)
<Alfr>
Pirx, and I don't talk about encoding, but code points; so for a subset of the character set, you may choose a special encoding.
<Alfr>
Real problem would be, that CL would need an overhaul, as char-code-limit needs to be abolished.
<Pirx>
and with code points you mean different characters, don't you?
<bike>
as to the best way to define characters, beyond "please don't incorporate encoding details", i am quite confident that i am not good enough at linguistics to come to a decision and hope to get a directive from the all-knowing unicode consortium any day now
* edgar-rft
wants a Neanderthal coding system with only one character #\Ugh! and hundred million different meanings depending on how you pronounce it.
<Pirx>
my point is that for writing *text* you don't need an infinite number of characters
sebastian_ has joined #commonlisp
<gilberth>
Alfr: Yes, I believe you can pile up as many of those combining character as you see fit. Another hack could be to intern code points and you encounter them. But then my char-code might not agree with yours.
mgl_ has quit [Ping timeout: 255 seconds]
<gilberth>
Pirx: What's the limit then? I might want to have a bar and a dot below and a ring above "a".
pranavats has left #commonlisp [Error from remote client]
<moon-child>
bike: so you want to normalise everything?
<bike>
the other day i learned about ﷽ (one code point, renders hilariously in this monospace terminal) and ᄀᄀᄀ각ᆨᆨ (several hangul thingies, immediately rendered incorrectly in my terminal) and now i'm convinced language is a spook
<moon-child>
what happens when the next version of unicode comes out and changes the normalisation rules?
<Pirx>
gilberth: are you trying to write text in any known language?
<Alfr>
Pirx, yes; but there's also no good reason to introduce arbitrary limits anymore.
* moon-child
still favours raku's strings
<bike>
moon-child: no, i want this to be someone else's problem because i recognize my limitations.
<moon-child>
though I know gilberth dislikes them
<moon-child>
bike: it is your problem as soon as you do anything with strings
<bike>
i know :(
<Alfr>
gilberth, you want to have a 1--1 correspondence between graphemes and code-points; I don't.
<Alfr>
gilberth, if you open that can of worms, how about grapheme clusters?
<gilberth>
Alfr: I didn't say that I demand the mapping being 1:1.
<gilberth>
I said that I would prefer when a string containing one grapheme would contain one CL:CHARACTER.
<gilberth>
And then some guy will come around the corner and demands that you can read any binary garbage as characters and be able to generate the bit-for-bit identical garbage when writing that again. *sigh*
<Shinmera>
just recently someone complained at me that "python is a bad language because you can't write unix' cat in it due to its unicode conversiosn" and I'm just
<Shinmera>
python *is* a bad language, but definitely not for that reason lmao
random-nick has joined #commonlisp
<gilberth>
Python has no binary streams?
<Shinmera>
I think the problem is more that when printing it wants to output unicode
<Shinmera>
or something, I don't know
<Shinmera>
I'm sure you can access the underlying binary stream and use that somehow
<Shinmera>
but really, why would you want to replicate cat
<Shinmera>
or expect another language to be C
mgl_ has joined #commonlisp
<gilberth>
Right. Yes, and why would you want to read binary data as Unicode? Not everything is a stream of octets.
<Shinmera>
anyhow, just an anecdote that seemed related
cmack has joined #commonlisp
certainty has joined #commonlisp
certainty has quit [Ping timeout: 255 seconds]
<sebastian_>
Do you know if, besides Scheme and Emacs Lisp, are there any Lisps from before Common Lisp that are still around today? It seems like there's just Scheme, Common Lisp, and Emacs Lisp.
<sebastian_>
I see a lot referenced on softwarepreservation.org, and reading old Lisp books from my university library suggests there used to be a lot more diversity. Nowadays there's newer stuff, like Clojure, though.
akoana has joined #commonlisp
<gilberth>
There is a reason why it is called Common Lisp.
<sebastian_>
In one book I found, the only control structure used by the auther was "(prog (...) ...)", because, he said, that was the only control structure guaranteed to be on whatever Lisp you had. This struck me as odd. I didn't realize there was so much diversity that not even iteration was guaranteed.
<gilberth>
PROG is the "PROGram feature" you can't write programs without it. :-)
<sebastian_>
Nowadays it seems to be implemented as a macro for (let (...) (tagbody ...)), for Common Lisp. But it is interesting how PROG was available from the very first versions of Lisp.
<gilberth>
PROG is very old. Even LISP 1.5 has it. I was curious and looked at Standard Lisp and it indeed has no DO or DO*.
<gilberth>
sebastian_: Yes, Common Lisp took PROG apart into its three components: Binding lexical variables, having something to RETURN from and having tags you can GO to.
<gilberth>
When you're into history: Standard Lisp allows GO only in tail positions within its body forms. That either a form is just a (GO ...) or it is (COND ((... (GO ...)) ...) or the latter nested, but GO must be the last thing (tail position).
<sebastian_>
Thanks. However, I typed this into the REPL of SBCL, and it yields an infinite loop: (tagbody loop (cond ((= 1 1) (go loop) 5) (t nil)))
<sebastian_>
Even though the (GO LOOP) part isn't in a tail position, since 5 comes right after it
<gilberth>
Yes, CL is fine with that. I talked about Standard Lisp.
<sebastian_>
Oh, my mistake. I misread.
<gilberth>
You can even do (tagbody loop (print 'hi) (mapcar (lambda (x) (go loop)) '(1 2 3)).
<gilberth>
I was just trying to provide some historical context. LISP 1.5 GO in the interpreter took an evaluated argument even. Not so in the compiler though.
<sebastian_>
Oh, wow, that's funny. It seems to me like GO (or goto, or jmp, etc.) messes up the type system of an expression based language: you need a "never" type, like in Rust. I guess it works for CL because it's dynamically typed.
waleee has joined #commonlisp
<sebastian_>
Related question, but do you know if Lisp can explicitly implement jump tables? As in "GO TO ( 10, 20, 30, 40 ), N" from Fortran?
<sebastian_>
That construct uses N as an index into an array of labels (10, 20, 30, 40). It's "computed go to."
<gilberth>
A CASE with just small integers as keys is usually compiled to a jump table.
<gilberth>
Otherwise, speaking of GO, this would work as well (let ((table (vector (lambda () (go here)) (lambda () (go there) (lambda () (go someplace-else))) ... (funcall (aref table n)) ...)
<gilberth>
For practical code I would use CASE though.
<sebastian_>
Yeah, that makes sense. I ask because in C, writing an opcode dispatcher, it's apparently slightly faster to use the GNU computed goto extension rather than looping on a switch statement, because it removes an extra jump between ops.
<sebastian_>
I'm not writing an emulator. I was just curious. I guess you can just write inline assembly, if you're really optimizing.
<gilberth>
Indeed. But that really is an implementation detail of the compiler, isn't it?
<gilberth>
I just wanted to show that you can close over a GO tag like (lambda () (go there)) and stash it wherever you like and then GO there by FUNCALLing it. The closed equivalent of GNU C's extension for computed goto.
<sebastian_>
Thanks for the link.
<sebastian_>
It is an implementation detail of the compiler, yeah.
certainty has joined #commonlisp
<bike>
sebastian_: "never" is in the CL type system as the bottom type, nil. if you try (describe 'error) in sbcl you can see it has a return type of nil (meaning it never returns, instead entering the debugger)
<sebastian_>
Also, thanks for showing me the vector of lambdas. That's an interesting construct, even if the CASE is what you should usually use.
<edgar-rft>
sebastian_: the Audacity audio edtior has an audio programming language named Nyquist which is based on XLISP (older than Common Lisp), also AutoLisp (used in AutoCAD) is based on XLISP.
<aeth>
NIL, btw, is type NULL, not type NIL
<aeth>
since nothing is of type NIL
<sebastian_>
Though, it's not quite the same; it's more like the equivalent of dispatching from an array of function pointers and returning from functions in C.
<sebastian_>
bike: Thanks for telling me! I'll try that.
<sebastian_>
edgar-rft: Thanks for letting me know about those two Lisps.
<Alfr>
aeth, about that nothing ... so, something is of type NIL? ;p
<gilberth>
Hmm, (lambda () (go there)) doesn't return. I just jumps there.
<gilberth>
* It
certainty has quit [Ping timeout: 255 seconds]
<sebastian_>
Oh, yeah, you're right. There's still an extra jump in going to the lambda address, and then to the THERE label. I guess it can't be described in C, though.
<gilberth>
Besides, a funcall at tail position behaves like a goto, no? Modulo possible stack usage, depending on your implementation.
<edgar-rft>
sebastian_: Tom Almy maintains a fork of XLISP that still runs on modern operating systems -> https://almy.us/xlisp.html
<ixelp>
XLISP-PLUS Home Page
<aeth>
Alfr: no because (every #'null (loop :for literally-everything :in *magic-list-that-has-everything-in-it* :collect (typep literally-everything 'nil)) => T
<gilberth>
sebastian_: Perhaps, if the compiler is not smart enough :-) But you can grab a GO and pass it around, although you need a lambda around.
<aeth>
Alfr: but if your implementation doesn't recognize that and optimize it at compile time, it might take a while to run
<aeth>
oh, I left out a )
<aeth>
thankfully gilberth closed it
<gilberth>
You're welcome.
kevingal has quit [Ping timeout: 246 seconds]
<Alfr>
aeth, simply hand me that set-containing-all-sets-not-containing-themselves from your *magic-list-that-has-everything-in-it*, and let me take it for a walk. :)
<sebastian_>
I just realized that you can return a lambda that just goes back into its calling function, like this:
<sebastian_>
Wait, let me put it on Pastebin instead
<bike>
you can also use the CL type system to define the set of types that do not contain themselves: (defun barberp (tspec) (not (typep tspec tspec))), and then the type is just (satisfies barberp). the paradox is cleanly resolved by your system hanging
<mfiano>
:)
<aeth>
Alfr: every object expressable in CL, which isn't that flexible in the first place. Though maybe the type system has issues with that sort of thing, especially once you get into cheating with SATISFIES
<bike>
sebastian_: tagbody tags have dynamic extent, so if you actually run the returned function, the behavior is undefined (hopefully an error).
<mfiano>
Pedantically, Common Lisp doesn't have lambdas to return. But yes, you can do nice things by closing over state.
<mfiano>
Ah
<mfiano>
bike: How about block labels?
<aeth>
ugh, bike beat me to it
<sebastian_>
bike: Yeah, I just realized that, thanks. I bound the return value of (come-back-inside) to a global variable and tried to run it, but the value of the global variable was nil, so it gave me an error.
<aeth>
Alfr: this is IRC, we'll tell you how to implement your joke
<bike>
to get the undefined behavior you would have (tagbody label (print 'hello) (return-from come-back-inside (lambda () (go label)))).
<mfiano>
bike: You are always coming up with these weird examples like nothing. I wonder how much you think about these things.
<bike>
i have spent far far too much time on implementing nonlocal exits
<mfiano>
:)
<bike>
most of my examples caused a segfault in clasp at one point or another
<sebastian_>
Thanks for the tip, haha
<bike>
if you want to learn about a regime where this kind of code is possible, i (sincerely) recommend learning about call/cc
<mfiano>
and that is the start of a derailment if i ever saw one
<bike>
but that's not really supported by common lisp. (you can do some code rewriting stuff to emulate it)
<aeth>
oh, yes, the totalitarian regime of #scheme
<aeth>
(actually kind of the exact opposite, considering how many Schemes there are)
<bike>
even in common lisp, continuations provide the theoretical framework you need to think about tagbody/block in detail without chucking everything and moving to tijuana
<sebastian_>
Thanks, I've actually heard of it, except I was always confused by the concept of continuations.
<mfiano>
You aren't alone
<sebastian_>
Not that I want to start talking about call/cc here
<ixelp>
GitHub - takagi/cl-coroutine: Cl-coroutine is a coroutine library for Common Lisp. It uses cl-cont continuations library [...]
tychoish has quit [Server closed connection]
<sebastian_>
bike: I tried running your code, and it gave me the error "attempt to RETURN-FROM a block or GO to a tag that no longer exists". I guess SBCL does the "right thing" here.
<skin>
It uses call/cc under the hood to do its deal but yeah, probably the better abstraction.
tychoish has joined #commonlisp
<mfiano>
I recall there being problems about cl-cont requiring a code-walker, no?
<bike>
the strength of continuations is that you can use them to implement pretty much any control structure, like tagbody, or go, or coroutines. thus the theoretical framework. the downside of continuations ithat they are indeed really really confusing.
Pixel_Outlaw has joined #commonlisp
<sebastian_>
They're also hard to optimize, apparently
<bike>
sebastian_: mhm. if you do the same thing at (optimize (safety 0)) you might crash
<bike>
in general, yes
<skin>
mfiano: Helpful intel. I was looking at using it for a side project, what's code-walker
<mfiano>
a program that understands the semantics of Common Lisp syntax
rgherdt__ is now known as rgherdt
<mfiano>
Just LET alone is a pain in all its forms. And have fun with the various forms of OLL's
<skin>
As far as I understand it, it uses `cl-cont`, the drawback is cl-cont doesn't work on all implementations. https://cl-cont.common-lisp.dev/
<ixelp>
cl-cont - A Common Lisp Delimited Continuations Library
<mfiano>
defgeneric and defmethod not being supported seems to be expected but a real showstopper for much that is to be gained from Common Lisp.
lucasta has joined #commonlisp
<BrokenCog>
question about drakma usage ... I used quicklisp to load it, typing (drakma:http-request I get the function def shown in the MiniBuffer, however (drakma:http-request "https://lisp.org") or any other URL gives error:
<BrokenCog>
Socket error in "connect": EINTR (A non-blocking socket operation could not be completed immediately.) [Condition of type SB-BSD-SOCKETS:INTERRUPTED-ERROR]
<BrokenCog>
anyone able to clue me in on this??
<mfiano>
That's umm, weird, and seems to be libc or SBCL specific. Try #sbcl maybe
green_ has quit [Ping timeout: 258 seconds]
<mfiano>
(cannot reproduce on latest SBCL release/x86_64)
jmdaemon has joined #commonlisp
rgherdt has quit [Quit: Leaving]
<gilberth>
sebastian_: Interesting. This (GO (COND (... 'L1) (... 'L2))) use was the only use supported in interpreted LISP 1.5. In fact with that PROG you could place GO only at the top-level of the PROG body. The compiler was different. Maybe they put it into MacLisp for backwards compatibility?
<BrokenCog>
ahhh ... likely. I just tried on a different machine with a different sbcl installation and it works.
<BrokenCog>
thanks for the suggestion.
<BrokenCog>
Problem was on a Windows10 machine, so, I guess not a huge surprise socket implementation isn't compatible.