<akater[m]>
Yesterday I had a little conversation about CLOS usage in Emacs. And I formulated something I've been thinking about for a long time. The purpose of Lisp object systems — at least Flavors and CLOS — is to enable modular design via flexible and predictable code reuse. When you don't reuse code by means of inheritance and don't see how and why it would happen, there is no point in using CLOS, at all. Agree?
<White_Flame>
the method dispatch is very useful even if you never use defclass
<semz>
Code reuse need not happen through inheritance specifically, generic functions can do that just fine.
<beach>
akater[m]: Also, people may avoid code reuse and may fail to see how and why it could happen, out of ignorance. Ignorance is a bad excuse for avoiding CLOS.
<White_Flame>
also, the failings of OO in general are not unique failings of CLOS
<White_Flame>
especially since CLOS goes way more powerful than typical simplified object systems
<semz>
I'd actually say that inheritance is the most overrated aspect of OOP. I rarely ever find myself using it. There's also the aspect that classes are redefinable at runtime while structs are not, but this is more of a CL quirk than anything general.
<borodust>
etimmons: hmm, it seems for quicklisp type of a source, clpm expects there to be present *-versions.txt file
<borodust>
etimmons: this file is not required for quicklisp client to work with a ql dist
<mfiano>
borodust: Yes. It needs to know the entire history
<mfiano>
Otherwise how would the lockfile be written?
<hayley>
Inheritance is one way to get mixins, which are underrated (though perhaps over-represented in CLOS).
<borodust>
mfiano: well, it's like with asdf versions, not every dist is versioned :)
<borodust>
i mean, if that's how clpm dictates it must be - i'm fine with that
<hayley>
I admit I made too much spooky action at a distance for myself by using :after methods to achieve some sort of "reactive" programming. But it was only a problem because I need the behaviour of a concurrent program written down in (close to) one place so that I can model it.
<mfiano>
I think the most under-used CLOS feature is progn method combination. I rarely see that used in codes that are not my own. Maybe it is just too domain-specific...idk, but I use it all the time.
<hayley>
If I ever feel like writing it, I'll eventually get to the third iteration of my node software, which hopefully should be space efficient first (we need to check the transitive closure of objects but only temporarily) and then hopefully time efficient.
<hayley>
Been thinking I should test software transactional memory at that layer. If I use a SQL database properly, I would be marking out transactions somewhere anyway.
<mfiano>
Very nice. For me, progn combination is so useful that it is probably my favorite feature of CLOS. It comes in useful far too often for me, and I don't know of a comparable feature in any other object system.
<beach>
Features like that are rarely used, like macros, but when you need them, they are totally essential.
<akater[m]>
White_Flame: Dispatch is useful but it doesn't require CLOS. When only a single method is ever going to be applicable per gf call, you may just as well roll your own interface to recompiling a defun. I think I'd do that without thinking twice.
<White_Flame>
GFs & method dispatch are part of CLOS
<White_Flame>
oh, right
<White_Flame>
but yeah, it's better to use built-in dispatch than roll your own
<beach>
And with generic functions, the methods don't have to be physically in the same place.
<beach>
And yes, rolling your own is not a great idea.
random-nick has joined #commonlisp
<beach>
akater[m]: PRINT-OBJECT is an example of a generic function where typically a single method is applicable for every call, but it is totally essential for application code to be able to add a method to it, without editing a global function.
<akater[m]>
beach: With print-object, it's trivial to imagine a use case for :before or :after method
<akater[m]>
All right, we seem to disagree. CL is missing a simpler inheritance-less dispatch — for legit reasons, I think (there's likely no single good enough design for this) but it is this fact that gives rise to one “Let's make all functions generic!” attempt after another. Meanwhile, people will keep writing such dispatching mechanisms (an reasonable ones) because they want expressive user-space optimization, dependent types and so on.
<beach>
I think you understood that when I mentioned PRINT-OBJECT, I was giving an example of a generic function with many methods, where typically only one is applicable, but where each method belongs to a totally separate "module", so that it is not reasonable to modify a global definition.
<hayley>
I disagree that any "user-space optimization" is any good, even if we limit ourselves to dispatching without inheritance. Merely inlining methods based on type inference with no regard for handling redefinition is definitely a step down.
Inline has joined #commonlisp
<beach>
akater[m]: What kind of optimizations are you thinking of?
<hayley>
Hot take: most of the perceived performance benefits of limiting dispatch techniques are disproven by reading up on the Self compiler. 30 years old isn't old enough to be conventional "wisdom" I guess.
<hayley>
I mean, polymorphic inline caches nail down the set of actually used effective methods. Using dependent types...is another thing which wouldn't fit in the Common Lisp standard unless you are willing to specify a dependent type system too.
<mfiano>
Having undecidable type inference wouldn't be very appealing.
Inline has quit [Quit: Leaving]
cosimone has joined #commonlisp
<akater[m]>
beach: E.g. ones that make Common Lisp programmers migrate to Julia, or motivate them to write libraries like static-dispatch, polymorphic-functions and so on.
<hayley>
I try not to think about people who think that static "dispatch" is related to ad-hoc polymorphism.
<mfiano>
Can you elaborate, hayley?
<hayley>
polymorphic-functions looks great actually, but it requires a few compiler extensions which don't look like "user-space optimization".
kakuhen has quit [Quit: Leaving...]
<mfiano>
After quite a bit of use, I am no longer a fan of these semantic-altering method dispatch extensions.
<mfiano>
Also fail to see how they relate to Julia's dynamic typing parametric polymorphism
<hayley>
PCL mentions it in the chapter on generic functions; that there is no real dispatch done on "overloaded methods" as they exist in Java and C++. They could well be sugar for separate functions (and, in fact, the Java compiler desugars methods internally).
<hayley>
And, as there is no standard type inference algorithm for Common Lisp, which methods are chosen due to type inference are more or less implementation-defined.
<mfiano>
Yes
<akater[m]>
Parametric types is its own example.
<hayley>
(But again, polymorphic-functions propagates type information between functions, and it doesn't mention handling function redefinition, so I assume it would not work.)
cosimone has quit [Remote host closed the connection]
tyson2 has joined #commonlisp
<akater[m]>
Well, my point is, people do write their own dispatch methods, this is not going anywhere. And I still think method combination according to inheritance is the core of CLOS and if it's not actually utilized, and there's no vision as to why it might be, no need to get CLOS involved.
<mfiano>
I don't understand. I rarely use inheritance, but use CLOS quite heavily.
<hayley>
What the Julia language does, to my knowledge, is use a parametric type system, and the compiler generates method code for each instantiation of a function type. The compiler also notably tracks dependencies and recompiles code if types and/or method definitions change.
<mfiano>
hayley: Right. At least in Julia, new code will be jitted as types change at runtime. These CL lbraries alter the semantics of generic functions in this respect.
<hayley>
Yes, exactly my point.
<akater[m]>
This is going to be one single point where I disagree with common CL users' wisdom. But given how underrated define-method-combination is, I don't mind it.
<mfiano>
But yeah, I think we are on the same page. I can no longer think of using such libraries, as the result may differ from compile-time to run-time, or even implementation to implementation. It has resulted in me chasing down some hard to find bugs in the past.
<hayley>
I wouldn't mind more JITing in Common Lisp compilers, but there is the issue that, with some things that a JIT might speculate on, performance "ramps up" or is otherwise inconsistent.
<mfiano>
When I think of performance, the first thing I consider is programmer productivity, not efficiency of computer resources. It is, after all, why I transitioned to CL from Python a couple decades ago.
<hayley>
Though polymorphic-functions-esque "JIT"ing, wherein we just generate specialised code for instances of a polymorphic function, is definitely safe to do, as performance only drops while specialised code is generated again.
pillton has quit [Quit: ERC (IRC client for Emacs 27.2)]
<mfiano>
Yes, I think I favor a consistent execution performance over the nightmares profiling JIT code can incur
<hayley>
(I believe that to be the case as redefinition of functions only occurs with someone poking at it, and in that case, your typing and/or thinking speed is the limiting factor.)
<mfiano>
I will take your word for it. Redefinition and performance conflict each other in so many places in my experience.
<hayley>
On the other hand, things like generic function caches and call-site snippets generated for generic function dispatch (aka polymorphic inline caches, again) still ramp up. But I guess the variation is small enough that people don't care.
<mfiano>
Like, take #'standard-instance-access for example, where we can instruct CLOS on how to locate slot values. We could, and I did before, write a metaclass to generate inlinable accessor functions like structs, which would allow for redefinition, but redefinition of superclasses would pose a problem for live instances.
<mfiano>
For that to work, you would have to severely constrain the user by enforcing the sealing of classes or some such, which is what I did, but it made sense since superclasses were implementation details in the the package.
<mfiano>
in the _library_ package.
Everything has quit [Quit: leaving]
<mfiano>
and I had to present the class definition in a macro to prevent users from defining derivations of this optimized type.
<hayley>
Redefinition could be handled on a path which handles type errors, provided you check types.
<hayley>
The slow path would be used until the function is re-compiled, either by the system or by the programmer.
<mfiano>
I'm not sure how you would handle obsoleted instances in a slow path though.
<hayley>
I'm only really aware of how obsolete instances are handled in SICL, so I can only comment on that. Each, say, "version" of a class has a unique stamp number.
<hayley>
So the fast path would be used if the object is a standard instance, and the stamp of the object is the same as the stamp of the class version we compiled against.
<hayley>
The slow path would be used to detect an obsolete instance, in which case we update the instance and retry, to detect obsolete code, in which case we use a slower lookup procedure (and possibly queue the function for recompilation), or signal a type error.
<mfiano>
I see, though I was considering user code, not implementation code.
<mfiano>
Like how al ibrary might write something that utilizes S-I-A in a redefinition-safe way
<hayley>
I think this procedure would nearly work with user code, except that you would have to be able to detect obsolete code as well as obsolete instances.
<mfiano>
Right, and that's the part I decided not to dive too deep into :)
<hayley>
A normal generic function for the slow path would never have obsolete code, come to think of it. So perhaps use that for the slow path?
<hayley>
The MOP dependency protocol, to my knowledge, does handle obsolete code for generic functions.
<hayley>
mfiano: Though I think programmer productivity can be improved by having a compiler which generates fast code, as you can get away with writing a simple but inefficient algorithm on some workloads. But of course the result isn't an efficient program.
<mfiano>
Sure.
<mfiano>
I have yet to read the call site optimization paper in full.
<hayley>
Another thing: a few days ago I needed to modify a A* implementation in Python to track sets of visited locations. Copying all the sets took significant time according to the profiler. So I asked my colleagues if there was a HAMT implementation in Python.
<hayley>
But there is a problem with that idea, at least when you use CPython: there is no way a pure Python implementation would be fast, and an implementation using C extension would increase the risk that we cannot build the program somehow.
<hayley>
I was vaguely aware of JPS, but not any algorithm with a + on the end of the name.
<hayley>
In my situation, I don't think it would work too well though. The main problems are that substantial time is taken while incrementally loading the world from Minecraft, even with caching, and the cost function I use does not expose many "symmetries" to make use of.
<mfiano>
I see. Tbh I haven't used A* with anything more than an L1 norm heuristic
<hayley>
Experimentally, runs in a straight line tend to only be a few blocks long. But, if I had more time, a proper investigation would interesting. I had to replace all the data structures, as someone only tested the implementation on little 2D mazes and not 3D open areas, and so everything blew up in size.
<hayley>
I nearly did implement something like jump points, but the points were computed while searching, rather than before.
<hayley>
Someone told me about how another group used multiple processes to retrieve world data, and my stupid chunk-at-a-time cache was apparently faster. So I guess there is not much you can do to optimise Python code.
<mfiano>
I haven't used Python in a couple decades and can't remember how slow it is. Not sure how close to acceptable algorithm complexity will buy you over there. I just remember it being "slow", whatever I was comparing it to at the time.
<hayley>
Yeah, that was our conclusion. We wanted to use PyPy instead, but Numpy would refuse to load for whatever reason. (They were pretty surprised when I mentioned that Common Lisp had multi-dimensional arrays out of the box.)
<pjb>
My bad experience with python that it didn't support thread on the platform I needed it (OpenWrt/mips), so in the end I wrote it in C, but it would have been faster to do it in CL and use ecl from the start.
<hayley>
On paper, it is true that performance is a property of implementations and not languages. In practise, it is still true, because you can write a fast compiler for even "slow" languages like Python and JavaScript. In practise (but for real this time), you have to be lucky for programs to not depend on behaviour of the slow implementation.
<hayley>
Common Lisp (hey, some wise people thought it was hard to compile in the 80s, and some less wise people think it is now) and JavaScript got lucky, as there have been multiple implementations of either for a long time.
<mfiano>
I vaguely recall Guido refusing to support TCO many years ago, for some crazy reasons, probably an excuse born out of his early design mistakes making it difficult.
<semz>
I believe the argument was that it made backtraces harder to read.
<hayley>
In #lispcafe a video came up about some "software drag race" where the author said Lisp used bytecode interpretation. More amusingly, the rest of the video dragged on about the bytecodes employed by the Java and C# virtual machines, as if they weren't immediately discarded by the compiler.
<mfiano>
Yeah it's a common misconception about CL :/
<hayley>
I found it more amusing that the same misconception was nearly carried out on Java and C♯.
<mfiano>
I recently recommended to the Julia landugage team to make their disassembler more like cl:disassemble. They were very surprised to learn it showed code size etc (on SBCL) almost as much as they were that an "interpreted language" could have such a feature.
<hayley>
In the end, of course, the benchmark numbers reflected the reality wherein both implementations using optimising compilers.
<mfiano>
But now Julia shows code size etc like SBCL
<hayley>
Funny, I thought they were relatively aware of Common Lisp.
<mfiano>
Yeah me too, especially since the parser is written in Lisp
hendursa1 has quit [Quit: hendursa1]
hendursaga has joined #commonlisp
Fare has joined #commonlisp
cosimone has joined #commonlisp
<jcowan>
In Femtolisp, which is not quite CL and not quite Scheme -- and is interpreted. So the idea of Lisp *compilers* probably wasn't in view.
<jcowan>
Oh. JVM bytecode is by no means discarded: all cold code is interpreted until the JIT kicks in. The CLR bytecodes are never interpreted, though.
<jcowan>
mfiano, hayley: ^^
<hayley>
Right, yes. I should elaborate: the subject of testing is a method run in a tight loop, which the JVM would run the optimising compiler on. So performance is not a function of bytecode design.
<mfiano>
jcowan: Speaking of, that seems to be you in the first comment of the above article :)
<jcowan>
It is.
Fare has quit [Ping timeout: 245 seconds]
<jcowan>
If you see a John Cowan or a johnwcowan, it's probably me.
Fare has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
Alfr has quit [Quit: Leaving]
srhm has joined #commonlisp
IPmonger has joined #commonlisp
srhm has quit [Read error: Connection reset by peer]
IPmonger has quit [Remote host closed the connection]
srhm has joined #commonlisp
Alfr has joined #commonlisp
<jmercouris>
well, that's the thing, Julia has nothing to do with Lisp
<jmercouris>
they keep claiming it does
<jmercouris>
but it really doesn't
<jmercouris>
you could make the argument that it has something to do with /A/ Lisp
<jmercouris>
but that's not really the same thing
<mfiano>
Who is "they"? How can _you_ claim that, with its borrowing of many Common Lisp specific features?
Inline has joined #commonlisp
Alfr has quit [Remote host closed the connection]
Alfr has joined #commonlisp
<markasoftware_>
Do java and c# compile their bytecodes to native at runtime?
<markasoftware_>
And if so is that a relatively recent development?
<mfiano>
Yes, and no
amk has quit [Remote host closed the connection]
amk has joined #commonlisp
aug0stus has joined #commonlisp
amb007 has quit [Ping timeout: 252 seconds]
Guest93 has joined #commonlisp
amb007 has joined #commonlisp
Guest93 has quit [Client Quit]
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
<jcowan>
markasoftware_: csc compiles all code it enters in a just-in-time fashion, whereas javac does that only with hot code, using a bytecode interpreter for cold code.
<mfiano>
jmercouris: It's obviously not _a_ Lisp, but it has more in common with (Common) Lisp than most modern languages. Even its dynamic type system is a weird hybridization of structural/nominal typing in much of the same way CL is.
amb007 has joined #commonlisp
<mfiano>
and layers on top of that have many similarities borrowed from CL. Therefor your comment to me just sounded like a SLW rant except with no basis
<random-nick>
I don't know if it still does that, but .NET used to have a windows service which compiles registered CLR .dll files to native images
<random-nick>
so it's also partially ahead out time
Guest949 has joined #commonlisp
Guest949 has quit [Client Quit]
waleee has joined #commonlisp
<jcowan>
"slw rant"? Dr. Google (and UD) not helpful today.
<beach>
The value of youth is greatly overestimated.
VincentVega has quit [Ping timeout: 265 seconds]
<beach>
mfiano: That's a very disturbing URL.
<mfiano>
But, for the record, I am not a Julia user. It is a decent language if I ever needed a non-Lisp for data science. It surely is a better choice than Python, R, or MATLAB in this regard, but I am not a data scientist. I use pragmatic in that I use the language that gets the task done. It's just coincidental that that has always been CL :)
<mfiano>
s/use/am/
<mfiano>
I suppose it is. I didn't see that.
<mfiano>
Apologies
asarch has joined #commonlisp
john-a-carroll has joined #commonlisp
john-a-carroll has quit [Quit: Ping timeout (120 seconds)]
asarch has quit [Quit: Leaving]
<jmercouris>
Perhaps I am a SLW :-)
karlosz has joined #commonlisp
karlosz has quit [Quit: karlosz]
aleamb has quit [Quit: bye]
karlosz has joined #commonlisp
cosimone has quit [Remote host closed the connection]
cosimone has joined #commonlisp
<jmercouris>
I must say that page explaining SLWs could be very offensive
<jcowan>
Anything of the from "smug $TOPIC weenie" is inherently offensive, unless reclaimed by the people it describes.
<jmercouris>
I meant the images and descriptions, and the divisive culture it promotes
<jmercouris>
But yes, the premise of it is also offensive
taichi is now known as mariari
waleee has quit [Quit: WeeChat 3.2]
<jcowan>
I mean, I say things like "Oops, I'm being a smug Lisp weenie here, aren't I."
waleee has joined #commonlisp
Oladon has joined #commonlisp
<jcowan>
PyCall and PyJulia form a pretty good bidirectional bridge, it would seem. Perhaps "Reach for Julia when CPython is too slow" is better advice than "Reach for C when CPython is too slow". Also, Julia's interface to C/C++/Fortran doesn't involve writing fugly glue code in C.
<jcowan>
I especially like that you can transparently pass a Julia functions as an argument to a Python function and vice versa, something that is usually treated as Just Too Hard.
karlosz has quit [Quit: karlosz]
Inline has joined #commonlisp
shka has quit [Ping timeout: 265 seconds]
nature_ has joined #commonlisp
asarch has joined #commonlisp
vats has joined #commonlisp
yitzi has joined #commonlisp
mariari has quit [Ping timeout: 252 seconds]
<moon-child>
jcowan: 'just too hard' it's certainly not _hard_ to implement, but you need to somehow manage the lifetime of the closure
mariari has joined #commonlisp
Fare has quit [Ping timeout: 245 seconds]
tyson2 has quit [Quit: ERC (IRC client for Emacs 27.2)]
Fare has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
cage has quit [Remote host closed the connection]
amb007 has quit [Ping timeout: 252 seconds]
amb007 has joined #commonlisp
shka has joined #commonlisp
shka has quit [Client Quit]
shka has joined #commonlisp
yitzi_ has joined #commonlisp
yitzi has quit [Ping timeout: 260 seconds]
yitzi_ has quit [Client Quit]
shka has quit [Ping timeout: 260 seconds]
Inline has quit [Ping timeout: 260 seconds]
scymtym_ has quit [Ping timeout: 252 seconds]
tyson2 has joined #commonlisp
Wairu_Goodman has joined #commonlisp
kakuhen has joined #commonlisp
asarch has quit [Quit: Leaving]
icer has joined #commonlisp
Inline has joined #commonlisp
akoana has joined #commonlisp
Lord_of_Life_ has joined #commonlisp
Lord_of_Life has quit [Ping timeout: 252 seconds]
Lord_of_Life_ is now known as Lord_of_Life
cosimone has quit [Remote host closed the connection]
cosimone has joined #commonlisp
vats has quit [Ping timeout: 245 seconds]
flip214 has quit [Ping timeout: 252 seconds]
flip214 has joined #commonlisp
gaqwas has quit [Ping timeout: 252 seconds]
slyrus has joined #commonlisp
cosimone has quit [Ping timeout: 252 seconds]
pve has quit [Quit: leaving]
<jcowan>
I came up with an idea for a non-portable extension of CL. It may be doable with the MOP, but I never learned the MOP. It looks like this: (define-type-class name type-specifier). The instances of such a class are all the objects which belong to the specified type. Otherwise they work like built-in types (or perhaps I should say they are built-in types: no slots and no constructors. However, parent types are possible, in
<jcowan>
which case the
<jcowan>
parent type specifiers are rewritten to include the subtypes with OR>
<jcowan>
What do people think of this?
<hayley>
(beach: For what it is worth, I would believe that the URL was an honest mistake on mfiano's behalf. Usually you don't see server URLs on Matrix, so it is hard to spot bad servers. And there are some of _that_ kind of person present in Lisp rooms; I know cause I had to get rid of some of them.)
<Gnuxie>
( there's some of those people lurking in this channel for sure for what it is worth )
cosimone has joined #commonlisp
<hayley>
What does rewriting the parent types achieve that inheritance doesn't achieve? Is it that classes defined with define-type-class don't inherit any slots or anything like that?
<slyrus>
any vellum and/or teddy users around? Other data-frame like libraries I should consider?
<jcowan>
Hayley: they have no slots
<hayley>
Right, I think it would be quite possible with the MOP. You could just rig COMPUTE-SLOTS to return the empty list.
<jcowan>
The trick is to get class-of to use the type specifier.
<hayley>
I seem to have missed that part. Not sure if you can do that.
<hayley>
If it was possible, I was going to use it for network proxying once, as I can handle everything else lazily.
cosimone has quit [Remote host closed the connection]
cosimone has joined #commonlisp
<jcowan>
Anyway, I think the better approach is to say that where the type specifier of the class being defined is t_c and the type specifier of each superclass is t_s, then for each superclass, subtypep(t_s, t_c) returns T T.
<hayley>
With t_s <: t_c, isn't it the case that t_c is the superclass?
<hayley>
That arrangement reminds me of stealth mixins, though you would reuse one mixin for multiple classes.
cosimone has quit [Remote host closed the connection]
cosimone has joined #commonlisp
moon-child has quit [Ping timeout: 240 seconds]
childlikempress has joined #commonlisp
childlikempress is now known as moon-child
cjb has joined #commonlisp
Wairu_Goodman has quit [Ping timeout: 260 seconds]
cosimone has quit [Ping timeout: 260 seconds]
<stylewarning>
Are there any JSON libraries that let you customize the structure that gets built?
<stylewarning>
Everything I’m seeing let’s you do small customizations (like make an alist or plist, or use lisp booleans or symbols or …), but I’m not seeing anything that allows me to instruct the parser what exactly I want it to build