<ixelp>
Make ANSI Common Lisp available on Haiku (again) - #177 by alpopa - Ports - Haiku Community
<Pixel_Outlaw>
Very cool to see Haiku has some Lisp on there.
Eoco has quit [Ping timeout: 252 seconds]
Eoco has joined #commonlisp
rkazak has quit [Ping timeout: 276 seconds]
dtman34 has joined #commonlisp
cammy has left #commonlisp [ERC 5.5.0.29.1 (IRC client for GNU Emacs 29.4)]
<beach>
I don't know much about Haiku, but I read in the Wikipedia article that it has ASLR and DEP which would make it harder to create a Common Lisp for it.
<beach>
ASLR in particular seems to me to be one of those things where, instead of using a safe programming language, they introduce more complications at the OS level to compensate for that lack of safety.
rkazak has joined #commonlisp
<beach>
ASLR must have been invented by C++ people, because it basically forces all programming languages to create application objects at startup time, as opposed to at compile time. This must be the reason why some applications take so long to start without any apparent reason.
<beach>
I was told (but I have not confirmed this) that SBCL deals with ASLR by essentially making a complete garbage collection at startup time, which works I guess, but again makes startup times longer.
rkazak has quit [Ping timeout: 246 seconds]
<beach>
What I meant to say is that the traditional way of creating application objects in C++ seems to be at startup time.
<Pixel_Outlaw>
Well, it looks like the author of that blog atleast has SOMETHING up and running. Maybe mysterious bugs still abound.
<beach>
How so C applications deal with ASLR? Do they use the linker to fix up the address of all objects created at compile time? That would also work, but then startup times would again be longer, and longer the more objects were created at compile time.
rkazak has joined #commonlisp
v88m has joined #commonlisp
pve has joined #commonlisp
<beach>
Pixel_Outlaw: Oh, sure. I didn't mean to imply that it is impossible to create a Common Lisp system for an operating system with ASLR; just that it makes things harder and slower.
<Pixel_Outlaw>
Right, I understand. :) I wonder how limited the language choice is there. Might be cool if a real Lisp was available among very few.
<beach>
Why would the choice be limited?
<beach>
And the "language choice" for what purpose?
amb007 has quit [Ping timeout: 244 seconds]
amb007 has joined #commonlisp
<Pixel_Outlaw>
Well, it's a newer OS. Granted it's an extension of BeOS so I'm sure it has the usual suspects like C. It would be nice if Common Lisp were more visible to newcomers to Haiku as one of the more prominent languages offered. This might help hook a few new folks to try it.
<beach>
I see.
<Pixel_Outlaw>
It might be a cool project to hack up some little utility or tool for the OS while it's still a younger project. You can note it was proudly made in Common Lisp.
<Pixel_Outlaw>
Hopefully the GUI system ism
<Pixel_Outlaw>
*isn't hell.
v88m has quit [Remote host closed the connection]
v88m has joined #commonlisp
<beach>
And I wonder how a Lisp system would deal with executable-space protection (DEP). I guess it must be possible to disable that feature, or turn it off for some pages.
alternateved has joined #commonlisp
<jackdaniel>
beos had (has?) unusual choice to make each top level window run in a separate thread, even in a single app
<jackdaniel>
something like mcclim
<beach>
Interesting.
shka has joined #commonlisp
<beach>
Executable-space protection seems to be another invention by people who use programming languages where no code is created at execution time.
rkazak has quit [Ping timeout: 260 seconds]
<beach>
Anyway, I guess all this is off topic, so I'll be quiet now.
ndanilov has joined #commonlisp
v88m has quit [Remote host closed the connection]
v88m has joined #commonlisp
Pixel_Outlaw has quit [Quit: Leaving]
mishoo has joined #commonlisp
ldb has joined #commonlisp
v88m has quit [Ping timeout: 260 seconds]
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
<ldb>
C facilitates ASLR by generate Position Independent Code, which now days is the standard practice
<beach>
ldb: Sure, but what about objects created at compile time? That's my issue.
v88m has joined #commonlisp
rkazak has joined #commonlisp
<ldb>
I cannot say for x86, but for ARM64, loading address of static objects on a different segment of executable is done by specify a page number, the OS does the address translation, then add the relative address offset,
ndanilov_ has joined #commonlisp
<beach>
Right, that I knew, like the use of the GOT or whatever it is called.
<beach>
That already slows down access to global objects.
rkazak has quit [Ping timeout: 248 seconds]
ndanilov has quit [Ping timeout: 260 seconds]
<beach>
But what about references to an object in the same file? In traditional C, the address was resolved by the linker when an executable was created.
<beach>
But if the address can change in some unpredictable way at startup time, then either this practice must be excluded, or it must be left to the dynamic linker to resolve at startup time.
<ldb>
the assembler computes the relative offset, and all the addressing is done purely by computing the actualy address using the program counter. the linker does not to fix up for that, it is only responseble for linking address from different objects.
<beach>
So all pointers contain PC-relative addresses? Interesting!
<ldb>
the program loader would not scramble the content executable, it only decides a random address offset of dynamic library and the executable itself
<ldb>
Yes, by the power of RISC
<ldb>
"the content of the executable"
<beach>
Thanks for the explanation. I should conduct some experiments to see what is going on.
mgl has joined #commonlisp
jjnkn has joined #commonlisp
rkazak has joined #commonlisp
v88m has quit [Read error: Connection reset by peer]
varjag has joined #commonlisp
v88m has joined #commonlisp
rkazak has quit [Ping timeout: 248 seconds]
v88m has quit [Read error: Connection reset by peer]
Inline has quit [Read error: Connection reset by peer]
Inline has joined #commonlisp
rkazak has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
v88m has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
v88m has quit [Read error: Connection reset by peer]
khinsen has joined #commonlisp
khinsen has left #commonlisp [#commonlisp]
rkazak has quit [Ping timeout: 265 seconds]
wobbol has quit [Ping timeout: 260 seconds]
v88m has joined #commonlisp
v88m has quit [Read error: Connection reset by peer]
bendersteed has joined #commonlisp
v88m has joined #commonlisp
rkazak has joined #commonlisp
v88m has quit [Read error: Connection reset by peer]
ingeniot has joined #commonlisp
rkazak has quit [Ping timeout: 252 seconds]
ndanilov_ has quit [Remote host closed the connection]
v88m has joined #commonlisp
rainthree has joined #commonlisp
v88m has quit [Read error: Connection reset by peer]
edgar-rft` has joined #commonlisp
edgar-rft_ has quit [Ping timeout: 260 seconds]
rkazak has joined #commonlisp
ndanilov has joined #commonlisp
Th30n has joined #commonlisp
v88m has joined #commonlisp
v88m has quit [Read error: Connection reset by peer]
<beach>
Sorry for the off-topic, but it is related to how a Common Lisp system is implemented in the presence of ASLR.
<beach>
ldb: So if you do {x *p = &q;} to which PC value is p relative?
ndanilov has quit [Ping timeout: 260 seconds]
<beach>
ldb: And if that is followed by x *r = &q; does that means that p and r do not contain the same bit pattern?
rkazak has quit [Ping timeout: 272 seconds]
v88m has joined #commonlisp
v88m has quit [Read error: Connection reset by peer]
v88m has joined #commonlisp
attila_lendvai_ has joined #commonlisp
alendvai__ has joined #commonlisp
<Alfr>
beach, likely &q gives an absolute address.
attila_lendvai_ has quit [Ping timeout: 265 seconds]
<beach>
Alfr: Thanks, but now I am confused, then. So not *all* pointers are PC relative.
<Alfr>
beach, I guess, shouldn't. Memory you dynamically allocate doesn't change with PC.
<beach>
Makes sense. But this was not a dynamic allocation. I assumed q was a global variable.
<beach>
Anyway, we should probably quit this guesswork, and I should try to find out how things work in real life.
rkazak has joined #commonlisp
v88m has quit [Ping timeout: 248 seconds]
v88m has joined #commonlisp
livoreno has joined #commonlisp
<ldb>
beach: lets say p, r are allocated as full width register in assembly, q is a C file local global variable and has offset q@OFFSET relative to the start of __data segment.
<ldb>
{x *p = &q;} would compile to something like, load p, q@PAGE; addu p, q@OFFSET.
<ldb>
which q@PAGE is a compile time constant determinded by PC address of load p, q@PAGE and start of the __datqa segment, and addu is the unsigned addition instruction
rkazak has quit [Ping timeout: 260 seconds]
<ldb>
and x *r = &q would be compile to load r, q@PAGE; addu
<ldb>
r, q@OFFSET, which q@PAGE and q@OFFSET represents the same constant
mwnaylor has joined #commonlisp
<ldb>
the rule is very specific to ARM64, but q@PAGE is the difference between the address of current instruction to the start of data segment divided by 0x1000, so if load r, q@PAGE and load p, q@PAGE are close enough, q@PAGE can be the same literal
<ldb>
but q@OFFSET would always be the same value
<ldb>
and the effect of load r, q@PAGE would be r = PC + (q@PAGE * 0x1000), addu r, q@OFFSET is r = r + q@OFFSET, which would leave the actual address of q in register r
<ldb>
even if p and r are allocated on stack, their memory address are computed by the stack pointer + compile time offset, and the computed address would be stored from the register to memroy
v88m has quit [Read error: Connection reset by peer]
<beach>
The value of bq (and bp is equal to it) is different in each execution. So that value must be assigned at run time. So if I create a huge object graph for Common Lisp using global structs and the & operator, lots of stuff must be filled in at run time.
<beach>
ldb: In fact, x *p = &q loads as a value relative to PC.
<beach>
... which is reasonable in that the global struct address can be determined relative to PC at compile (or link) time.
Perflosopher05 has joined #commonlisp
<beach>
I am talking x86 here, by the way.
v88m has joined #commonlisp
<ldb>
beach: on my ARM64, they are always the same :)
<beach>
p and r? Sure! :)
Perflosopher0 has quit [Ping timeout: 252 seconds]
Perflosopher05 is now known as Perflosopher0
<ldb>
bp and bq
<beach>
Yes, those too. Luckily! :)
<beach>
I just wanted to demonstrate that bq cannot be computed at compile time or link time.
<beach>
... because of ASLR.
<ldb>
Sure, which is a bad thing if objects are moving around
<beach>
Even when the object graph is static, you can't compute it at compile or link time.
<ldb>
yes, as they will be computed using PC
<ldb>
and program loader can give random address to programs
<beach>
Possibly. But this is bad, because without ASLR, you could create the graph at link time, and startup time would be instant. Now, startup time depends on the complexity of the graph.
v88m has quit [Read error: Connection reset by peer]
<beach>
And this was my point from the beginning.
<beach>
So if you create a Common Lisp system by writing the existing object graph as static objects in the executable, you still have to create code to compute (or fix up) every pointer at startup time.
<beach>
... which tell me that ASLR was created by people who did not have this use case very often, or at all.
rkazak has joined #commonlisp
v88m has joined #commonlisp
<ldb>
I see, but for a modern RISC machine, load a constant to a register is almost as fast as compute the address using PC, and both takes one or two instructions
<beach>
But the pointers in the graph would have to all be computed before the execution starts.
<beach>
So it might be fast, but it is now done at startup time instead of at compile/link time.
v88m has quit [Read error: Connection reset by peer]
<ldb>
If the graph is save as data, instread of code, yes a kind of startup fix would be needed
<beach>
That's exactly my point, yes. Which would be the case if you create a Common Lisp system by computing the object graph in (say) another Common Lisp system and then save it in the executable file.
<beach>
The same problem would happen if you save it as code. It would have to be executed at startup time.
<beach>
And I am guessing that's why some applications are really slow to start.
<beach>
In other words, ASLR is a good way to force Common Lisp programs to be as slow to start as C++ programs must be.
v88m has joined #commonlisp
<ldb>
or else it would require more complicated offset computation when emit such a graph
v88m has quit [Read error: Connection reset by peer]
rainthree has quit [Ping timeout: 260 seconds]
<beach>
You can't handle it when the graph is saved to the executable, given that every pointer must be computed at startup time. You can of course abandon the idea of using normal pointers and use offsets instead. Yes, that would work.
<beach>
So you would then save the address of the start of your graph in a register, and then "pointers" would be relative to the value of that register. So to dereference a pointer, you would need to add two registers and perhaps an offset.
<beach>
I guess most architectures can handle that.
<beach>
ldb: Well, it looks to me like it loads the object graph at startup time.
notzmv has joined #commonlisp
kdlv has quit [Quit: Ping timeout (120 seconds)]
jonatack has quit [Ping timeout: 252 seconds]
kdlv has joined #commonlisp
jonatack has joined #commonlisp
<ldb>
> So to dereference a pointer, you would need to add two registers and perhaps an offset. < Exactly, which is also a bit challenging for efficient code generation
<beach>
And I am looking at RISC-V and I think the LOAD instruction takes a single register and an offset.
random-nick has joined #commonlisp
<ldb>
no, I did not mean the actual instruction name, on ARM64 LOAD is "adrp"
rkazak has quit [Ping timeout: 244 seconds]
<beach>
So ASLR forces us to either be as slow to start up as a C++ application, or to be slower that we should be for every object access.
<beach>
*than we should
v88m has joined #commonlisp
<ldb>
^ memory access: usually, RISC memory store/load operation can take two registers and use their sum as the address
<ldb>
which costs the same as using one register
<beach>
Apparently not RISC-V.
amb007 has quit [Ping timeout: 265 seconds]
amb007 has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
<ldb>
I've seen this on MMIX, ARM64, PPC64
mwnaylor has quit [Ping timeout: 276 seconds]
amb007 has joined #commonlisp
<beach>
I am sure there are architectures with such instructions, yes.
<ldb>
RV is too minialistic sometimes :D
v88m has quit [Ping timeout: 265 seconds]
<gilberth>
It is done at start up. I mean you still want to able to say char *foo = "hello"; and pass &foo around and have another routine then deference that pointer to get at the "hello" text. In C when you say *p you get a plain vanilla load instruction with no address calculation.
v88m has joined #commonlisp
v88m has quit [Remote host closed the connection]
<beach>
Yes, we figured it out.
v88m has joined #commonlisp
<gilberth>
For dumping heap images one could as well dump a relocatable object, be it a binary or a dynamic lib, and let the OS have fun with it.
<ldb>
beach: do you think it would give penality for a CL implementation if a memory page can not be readwrite and executable at sametime?
<beach>
That was one of the options I suggested.
<beach>
ldb: Maybe not much. You would make it writable when you create a function and then make it executable again afterwards.
<beach>
... and creating a function is usually a costly operation anyway, involving compiler optimizations and such.
<gilberth>
I'd rather not do that which each #'(lambda ...). An implementation implementing closures by consing little trampolines would lose.
<beach>
gilberth: Can you elaborate on that? I don't know what implementing a closure by consing trampolines means.
<ldb>
I guess that means a ((var . value) ...) local environment which is subject to GC
<gilberth>
beach: You have to set up some [hidden] argument that points to the lexical environment essentially. Some vector where pointers to closed over variables can be found. Either you make all functions be a pair and implement function calls by (f->fun)(f->env, ...) or you say that each function pointer is just that, an address you can branch to.
rkazak has joined #commonlisp
<beach>
Sure, I understand.
<gilberth>
In the latter when making a closure you can make a small piece of code that says MOV ENV-REG, #whatever ; B <real-body> and that's your closure.
pranav has quit [Read error: Connection reset by peer]
<beach>
I see.
<beach>
And how would that idea be penalized by having to switch between read/write and execute?
<gilberth>
beach: GCC/clang does this with local functions. On ARM-64 with Darwin they amortize the huge costs of flipping page permissions my having a page full for little trampolines, loading the env pointer and the function address from some data vector. It's then managed as a resource with explicit allocation and release.
<gilberth>
beach: I'd rather not do this everytime I say #'(lambda ...) or pass a local function around. What used to be cheap (consing a handful or words) now costs tens of thousands of instructions or even more and usually you need to flush caches too.
<beach>
Right, but that's true with both closure implementations.
<gilberth>
Not quite.
<gilberth>
It makes a huge difference whether I cons a small struct of two or three words, or whether I do system calls and flush caches. I might also need to stop the world while twiddling pages from executable to writeable.
<beach>
Are you saying that there is a situation where the trampoline would need to be created, but not the code for the function body?
<gilberth>
Yes.
<beach>
What would such a situation be?
<gilberth>
Consider (loop for i below 100 do (let ((i i)) (foo (lambda () i))))
rainthree has joined #commonlisp
<beach>
Ah, yes, I see. The body is the same in every case.
<gilberth>
Yes, and the trampoline setting up the env pointer argument is not. And that's code too in the trampoline-consing implementation.
<beach>
... just like it is in case 1, but in case 1 you don't allocate a trampoline, just a static environment.
<beach>
I get it. Thanks for the explanation.
<gilberth>
Yes, with the added costs that a function is not just a (*fun)() but some (fun->fun)(fun->env, ...);
<beach>
Indeed.
<beach>
Though, of course, my call-site optimization trick will fix that problem.
<gilberth>
My plan is: Do the "full" function call (fun->fun)(fun->env, ...) on FUNCALL and APPLY. But use just a CALL at known function calls, like in (FOO 1 2). When you say (setf (fdefinition 'foo) ...) make a trampoline if needed. I think that would be a good compromise as most functions ending up in as the function definition of a symbol are not closures, and (setf fdefinition) should be rare.
<beach>
But, yes, I see. This necessity to flip between read/write and execute is another way to make Common Lisp applications just as slow as C++ applications.
<gilberth>
I'm sure we still can beat C++. Easily.
<beach>
But they make things unnecessarily hard.
<beach>
And, sure, as I often say, it is impossible to create a large C++ application that is both fast and maintainable.
<beach>
s/maintainable/modular/ is usually what I say.
<gilberth>
beach: You don't like puzzles? See at as a puzzle.
<beach>
Whether I like puzzles depends on the context. But I get your point.
<gilberth>
Besides writing code may indeed upset CPU caches regardless of whether the OS wants to sell itself as more secure enforcing such silly restrictions.
<beach>
Sure.
pranav has joined #commonlisp
ingeniot has quit [Ping timeout: 240 seconds]
ldb has quit [Quit: zzz]
<beach>
I should document how my call-site optimization makes it unnecessary to use the trampoline trick.
gnoo has quit [Ping timeout: 272 seconds]
rkazak has quit [Ping timeout: 245 seconds]
gnoo has joined #commonlisp
<beach>
Anyway, time for a well deserved lunch break.
<jackdaniel>
how would you call in a single word lexical variables that are not part of locals, that is closed-over variables?
<jackdaniel>
"closures" clearly would be misleading
<jackdaniel>
lexical on the other hand covers both closed-over and local variables (and other entities)
<scymtym>
would a but not b in (let ((a 1)) (lambda () (let ((b 2)) ...))) be an example of that concept with respect to the anonymous function (because b is "local')?
<jackdaniel>
yes
<scymtym>
maybe "lexical variable from outer scope" or "closed-over variable from outer scope" then?
<jackdaniel>
I think that "closed-over variable" in the latter would be enough (because you don't close over a local variable), in any case quite wordy
JuanDaugherty has joined #commonlisp
Josh_2 has joined #commonlisp
<jackdaniel>
each groups (closed-over and local lexical variables) have different characteristics, so it is weird that both are simly referred as lexicals
<jackdaniel>
simply*
yaneko has quit [Quit: parting]
yaneko has joined #commonlisp
<jackdaniel>
group*
rkazak has joined #commonlisp
v88m has quit [Ping timeout: 245 seconds]
v88m has joined #commonlisp
herjazz has joined #commonlisp
rkazak has quit [Ping timeout: 276 seconds]
decweb has joined #commonlisp
kpg has quit [Read error: Connection reset by peer]
JuanDaugherty has quit [Quit: JuanDaugherty]
rkazak has joined #commonlisp
<Josh_2>
ayoo
nkatte has joined #commonlisp
rkazak has quit [Ping timeout: 260 seconds]
<phoe>
jackdaniel: I don't think there's anything better than "closed-over variables"
<beach>
One can always invent a new word, depending on the context.
xaltsc has quit [Quit: WeeChat 4.5.1]
<phoe>
I can't think of one :D
jjnkn has quit [Ping timeout: 244 seconds]
<phoe>
the neologisms "lexivar" and "dynavar" are kind-of unambiguous, but how would you do the same to a closed-over var
<beach>
Stuff like "remote".
<beach>
Not particularly that one, perhaps, but something that suggests distance.
rkazak has joined #commonlisp
yaneko has quit [Quit: parting]
<random-nick>
gilberth: isn't the trampoline (even without having to do a system call) a significant performance penalty for closures, while the extra function argument is relatively insignificant?
<beach>
random-nick: I think the gain has to do with not having to access the static environment in the function object. That requires a memory access.
yaneko has joined #commonlisp
yaneko has quit [Quit: parting]
yaneko has joined #commonlisp
<random-nick>
beach: yes, but if the env pointer is next to the code pointer, then those 2 memory accesses are not that much more expensive than just 1
<beach>
Good point.
yaneko has quit [Client Quit]
bitspook has joined #commonlisp
<beach>
Either way, I much prefer the way my suggested call-site optimizer deals with it.
<beach>
Too many mechanisms in current Common Lisp implementations seem to assume separate compilation as used in static programming languages.
<random-nick>
though I haven't considered the fact that when using trampolines, you don't need a function object at all and so the non-closures do not need that memory access
<beach>
Ah, yes, another interesting point.
herjazz has quit [Quit: leaving]
rkazak has quit [Ping timeout: 252 seconds]
yaneko has joined #commonlisp
<random-nick>
actually, for the case of statically calling a symbol, you could inline the function object into the function cell so you could save that one memory access? you'd still have the extra probably-cached access for the environment pointer, though
<random-nick>
but of course that would be trickier to implement
<beach>
I think the call-site optimization technique will handle those situations as well.
<beach>
But it is basically a superset of the trampoline in that it needs to allocate code. And it does so only for calls to named functions.
<beach>
And I should document those advantages as well.
<ixelp>
Mapping Multiple Memory Views in User Space
v88m has quit [Read error: Connection reset by peer]
<random-nick>
maybe you could look into javacript JITs to see how they handle W^X, but not sure if their solutions would be suitable for CL
v88m has joined #commonlisp
Josh_2 has joined #commonlisp
JoshYoshi has joined #commonlisp
v88m has quit [Read error: Connection reset by peer]
kpg has joined #commonlisp
Josh_2 has quit [Client Quit]
JoshYoshi has quit [Remote host closed the connection]
Josh_2 has joined #commonlisp
rkazak has joined #commonlisp
rkazak has quit [Ping timeout: 276 seconds]
v88m has joined #commonlisp
v88m has quit [Read error: Connection reset by peer]
kpg has quit [Read error: Connection reset by peer]
rkazak has joined #commonlisp
zxcvz has joined #commonlisp
zxcvz has quit [Client Quit]
v88m has joined #commonlisp
v88m has quit [Read error: Connection reset by peer]
<yottabyte>
I usually export my functions, but I know you can do my-package::my-function even if they're not exported... what's the point of that?
<beach>
yottabyte: You would not normally do that. It would indicate some problem in your protocols.
<beach>
yottabyte: The point of exporting symbols is to indicate to client code what they are allowed to use in normal cases.
<yottabyte>
could it be for unit testing purposes?
<yottabyte>
like, I don't want to export a function, but I still want to test it
<beach>
Yes, that's a good use for it.
JuanDaugherty has joined #commonlisp
<yottabyte>
do you write a lot of unit tests?
<beach>
Me? I try to, yes.
<yottabyte>
nice
<random-nick>
that syntax isn't in the standard afaik, it's a very common extension
<beach>
random-nick: What? my-package::my-function is not standard?
<JuanDaugherty>
me? never
<JuanDaugherty>
not enduing ones anyway
<JuanDaugherty>
*enduring
rkazak has quit [Ping timeout: 245 seconds]
josrr has joined #commonlisp
<random-nick>
beach: looks like I misremembered something, because the standard does say that you should print internal symbols with ::
<random-nick>
but I can't find where it specifies reading the package prefix
<beach>
Probably in chapter 2.
<random-nick>
found it, it's 2.3.5
<beach>
Yes, item 3.
<random-nick>
I don't know where I heard that it's an extension
saulosilva has joined #commonlisp
<phoe>
foo:(bar baz quux) and foo::(bar baz quux) is an extension
<phoe>
but foo::bar is standard
livoreno has quit [Quit: Connection closed for inactivity]
rkazak has joined #commonlisp
saulosilva has quit [Quit: Client closed]
<gilberth>
random-nick: It might or it might not work. However, beyond actually making new code (via LOAD or COMPILE) I plan to do without. If it would work, I bet that OS vendors would consider this as a loophole, and expect that to be plugged.
<gilberth>
There always will be some support for "JIT"ing. And dlopen(3) has to work somehow as well. With macos there is a certain you are expected to use to deposit new code. You're supposed to call some pthread_jit_write_protect(3) function or so to flip this back and forth. Fine for LOAD, not so for #'(LAMBDA ...).
<gilberth>
On that note: I'm tired of Lisp implementations just not accepting the restrictions set up by C essentially and always play "tricks" that are prone to break with every new OS release for more feeble attempts to make C secure.
rkazak has quit [Ping timeout: 265 seconds]
v88m has joined #commonlisp
bitspook has quit [Ping timeout: 272 seconds]
mgl has quit [Ping timeout: 244 seconds]
v88m has quit [Read error: Connection reset by peer]
<random-nick>
gilberth: what are "restrictions set up by C"?
<gilberth>
The very things discussed recently. Like not liking pages being executable and writable at the same time. ASLR. Some hostility wrt to page faults. More hostility wrt to interrupts.
<gilberth>
Then there is poor support for fast TLS at times.
<random-nick>
but some kinds of C programs were also hit by those changes
<gilberth>
Then any dependency of a Lisp implementation that the heap is mapped to some specific address or in "low memory" <2GB.
JuanDaugherty has quit [Quit: JuanDaugherty]
<gilberth>
random-nick: Yes, and did this make the vendors backpedal? We're the smaller and less important party. Life is easier when we just accept that fact.
<random-nick>
but there are proper reasons to disregard those limitations, usually performance
saulosilva has joined #commonlisp
<jackdaniel>
I want my tagged memory back (not that I have really ever seen one :)
<gilberth>
Performance is not particular good if my program doesn't work at all.
<random-nick>
but I guess the best way to do it would be to implement both a conservative codepath and an "optimised" codepath, and use the appropriate one for each platform
<jackdaniel>
that's not true -- the fastest program is the one that does not run
<gilberth>
jackdaniel: How so? It never finishes. /me scratches head.
<jackdaniel>
I'm a firm believer that the cave empty of dragons is the same thing as the cave empty of knights
<gilberth>
random-nick: Take Lisp heap dumping. It's somewhat lazy and not so much performance that Lisp implementation just write out the heap and expect it to loaded again at a certain address. As discussed a relocatable object could be dumped instead and let the OS have fun with it.
<gilberth>
random-nick: A pity also is that turning on alignment checks is somehow fragile. It's ignored altogether on x86_64 macos and libc is prone to have issues on Linux. Otherwise I could have a completely safe CAR and CDR that is just one load instruction.
puke has quit [Ping timeout: 244 seconds]
<gilberth>
Or if I were able to map negative addresses, I could have overhead-free NaN boxing. That also would be nice.
vardhan__ has quit [Ping timeout: 248 seconds]
kurfen has quit [Ping timeout: 252 seconds]
rkazak has joined #commonlisp
kurfen has joined #commonlisp
<beach>
gilberth: It sounds like you are working on a Common Lisp implementation. Is that the case?
<gilberth>
I'm conducting experiments on having a fast runtime while compiling to C and playing by the rules. I'm not there yet to actually attempt to port a CL.
<beach>
I see.
* bjorkintosh
assumes gilberth IS in fact an ideal walking Common lisp implementation and standard.
<bjorkintosh>
what do I know?
<gilberth>
GC is an issue and I want to do w/o scanning the CPU stack without consent. Rather I have a separate Lisp stack.
saulosilva has quit [Quit: Client closed]
rainthree has quit [Ping timeout: 264 seconds]
<gilberth>
Results so far are promising. I can run some Gabriel Benchmarks faster than SBCL. Most notably DERIV and BOYER because that's the kind of code I'm interested in. Pointer-chasing, dispatchy, cons-happy code.
bendersteed has quit [Quit: bendersteed]
rainthree has joined #commonlisp
akoana has joined #commonlisp
saulosilva has joined #commonlisp
akoana has quit [Client Quit]
cage has joined #commonlisp
cage has quit [Excess Flood]
akoana has joined #commonlisp
cage has joined #commonlisp
mgl has joined #commonlisp
rkazak has quit [Ping timeout: 246 seconds]
rkazak has joined #commonlisp
bigbookofbug has quit [Quit: Lost terminal]
bigbookofbug has joined #commonlisp
rkazak has quit [Ping timeout: 252 seconds]
Oladon has quit [Ping timeout: 252 seconds]
bitspook has joined #commonlisp
rainthree has quit [Ping timeout: 265 seconds]
v88m has joined #commonlisp
varjag has quit [Quit: ERC 5.5.0.29.1 (IRC client for GNU Emacs 29.3)]
rkazak has joined #commonlisp
wacki has joined #commonlisp
rainthree has joined #commonlisp
v88m has quit [Remote host closed the connection]
v88m has joined #commonlisp
ndanilov has joined #commonlisp
v88m has quit [Read error: Connection reset by peer]
chomwitt has quit [Quit: WeeChat 3.8]
ndanilov has quit [Remote host closed the connection]
v88m has joined #commonlisp
v88m has quit [Read error: Connection reset by peer]
bitspook has quit [Ping timeout: 252 seconds]
rkazak has quit [Ping timeout: 246 seconds]
v88m has joined #commonlisp
v88m has quit [Read error: Connection reset by peer]
wacki has quit [Ping timeout: 245 seconds]
ndanilov has joined #commonlisp
wacki has joined #commonlisp
ndanilov has quit [Ping timeout: 260 seconds]
akoana has left #commonlisp [Leaving]
akoana has joined #commonlisp
rkazak has joined #commonlisp
shmorg83 has quit [Ping timeout: 248 seconds]
Lord_of_Life_ has joined #commonlisp
shmorg83 has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
Lord_of_Life has quit [Ping timeout: 276 seconds]
Lord_of_Life_ is now known as Lord_of_Life
amb007 has joined #commonlisp
cercopith has quit [Remote host closed the connection]
rainthree has quit [Ping timeout: 244 seconds]
rkazak has quit [Ping timeout: 260 seconds]
rkazak has joined #commonlisp
amb007 has quit [Ping timeout: 265 seconds]
amb007 has joined #commonlisp
ndanilov has joined #commonlisp
saulosilva has quit [Quit: Client closed]
triffid has quit [Remote host closed the connection]
triffid has joined #commonlisp
jonatack has quit [Quit: WeeChat 4.5.0]
khrbtxyz has quit [Ping timeout: 260 seconds]
zxcvz has joined #commonlisp
rkazak has quit [Ping timeout: 244 seconds]
khrbtxyz has joined #commonlisp
soweli_iki has quit [Remote host closed the connection]
jonatack has joined #commonlisp
amb007 has quit [Ping timeout: 244 seconds]
amb007 has joined #commonlisp
<yottabyte>
do you generally do asdf:defsystem as :serial t? why or why not?
<phoe>
I do, because I'm lazy and because in 90% of all situations the files I write are meant to be loaded serially
<phoe>
for dozens of files, specifying dependencies manually is a major chore and a place where subtle bugs may live
<yottabyte>
I feel like serial t is sort of manual, no?
<yottabyte>
because you have to make sure they're in the right order
<phoe>
without :serial t you need to specify dependencies for each and every file
<yottabyte>
ohhh
<phoe>
if you have (:file "package") (:file "foo") (:file "bar") (:file "baz"), then you'll probably need to depend on (:file "package") for the three latter ones
amb007 has quit [Read error: Connection reset by peer]
<phoe>
and then ensure that any logical dependencies between the files are also expressed in the dependency structure
amb007 has joined #commonlisp
<phoe>
you get an arbitrary DAG rather than a list, and good luck maintaining that for large systems
<phoe>
with :serial t, you don't need to care about this DAG - only about the order in which your files appear in the asd file
wobbol has joined #commonlisp
mishoo has quit [Ping timeout: 260 seconds]
cage has quit [Remote host closed the connection]
cage has joined #commonlisp
cage has quit [Excess Flood]
cage has joined #commonlisp
rkazak has joined #commonlisp
rogersm has joined #commonlisp
gorignak has joined #commonlisp
JuanDaugherty has joined #commonlisp
zxcvz has quit [Quit: zxcvz]
rkazak has quit [Ping timeout: 264 seconds]
rkazak has joined #commonlisp
akoana has quit [Quit: leaving]
varjag has joined #commonlisp
ndanilov has quit [Read error: Connection reset by peer]
ndanilov has joined #commonlisp
rogersm has quit [Quit: Leaving...]
zxcvz has joined #commonlisp
jonatack has quit [Quit: WeeChat 4.5.1]
zxcvz has quit [Client Quit]
robin_ is now known as robin
mwnaylor has joined #commonlisp
chomwitt has joined #commonlisp
jonatack has joined #commonlisp
mgl has quit [Ping timeout: 244 seconds]
jonatack has quit [Quit: WeeChat 4.5.1]
xaltsc has joined #commonlisp
jonatack has joined #commonlisp
kpg has joined #commonlisp
Shinmera has quit [Quit: WeeChat 3.8]
Shinmera has joined #commonlisp
Shinmera has quit [Client Quit]
Shinmera has joined #commonlisp
Shinmera has quit [Client Quit]
cage has quit [Quit: rcirc on GNU Emacs 29.4]
Shinmera has joined #commonlisp
pve has quit [Quit: leaving]
kevingal has joined #commonlisp
josrr has quit [Remote host closed the connection]
JuanDaugherty has quit [Quit: JuanDaugherty]
Shinmera has quit [Quit: WeeChat 4.5.1]
Shinmera has joined #commonlisp
<aeth>
For large systems, the alternative is package-inferred-system, which basically means (at least) one package per file, with that package tied to a system, and dependencies automatic (with a bit of boilerplate, still, because now you have to add a lot of DEFPACKAGEs and you need to create a foo.lisp for every subdirectory foo, etc.).
<aeth>
But I think most people prefer to just keep things small and break things up into many things manually if needed.
<aeth>
Also, if you actually need other packages than just one-per-file, you have to tell the ASDF file where to find them. So it can build the dependency graph.
<reb>
yottabyte: I always specify ASDF file dependencies explicitly to minimize recompilation when I make a change.
shka has quit [Quit: Konversation terminated!]
<tux0r>
reserved tantrum: it is at least a little annoying that in common lisp there are many ways to solve a problem, but only exactly one really good library for this problem.
<tux0r>
and if you create a feature request for this library, there is a risk that the maintainer will reply ‘fuck you, i'm not doing lisp anymore’.
<tux0r>
(tl;dr: sorry that all packages of eudoxia0 are now officially ‘unmaintained’. i swear that i will never again suggest a new feature to a lisp developer).
<tux0r>
open source was a mistake.
<tux0r>
</tantrum>
<aeth>
reb: well, package-inferred-system also minimizes recompilation, and that tends to be the real reason to be stuck with it on a huge project with glacial > 1 second compilation times on some files
Thermoriax has joined #commonlisp
<reb>
tux0r: I think the canonical approach is for you to fork the library, make the change and maintain the library. Thre's also sharplispers, who have taken over maintaining several Common Lisp libraries.
<aeth>
Yes, I was about to mention sharplispers
<reb>
aeth: So far, I've been able to maintain dependencies manually, but my projects are small.
<aeth>
they're the canonical approach
saulosilva has joined #commonlisp
<tux0r>
reb: my own approach was "fork the library, remodel it so it fits exactly one use case, namely: mine, and never release it on github because ugh"
<tux0r>
sharplispers would be plan b, indeed
<aeth>
tux0r: Yes, that, too. I already don't release on GitHub and never would start with that because there's too little friction to get involved so issues tend to be... low quality. Often just comment flamewars etc. It's social media.
<aeth>
But given the past however many years, I probably wouldn't even release the source at all until it's done. Things take a long time, and people have unreasonable timeline expectations if it's out there.
<tux0r>
github's biggest "pro" is its biggest "con" as well: a low entry barrier. hooray, it's easy to publish stuff there. ugh, weirdos can file bullshit tickets there.
<tux0r>
i'm a big fan of open source, in a way, but i strongly believe in the cathedral system
<tux0r>
the bazaar has proven its uselessness too many times.
<tux0r>
"here's my code. take it. i don't care. no, i don't want your patches, go away."
<tux0r>
instant sanity.
<aeth>
Right.
<aeth>
I have zero expectation to receive a meaningful contribution from anyone. So I've definitely been forced into a cathedral instead. In which case, I probably wouldn't develop it in the open (committing every day to public internet repos) because there's always a risk of it going viral years too soon by some random hyperlink on some random site pointing to something that doesn't even work yet.
<aeth>
Which also doesn't mean closed source, it just means, social media open source coding provides no benefits to me.
<aeth>
Maybe a bazaar made more sense when it was more forum (or mailing list) culture rather than today's social media culture.
<tux0r>
my open development, minus the very project i was talking about a few minutes ago, happens on one of my own servers on my own fossil installation, for exactly one reason: instant backups.
<tux0r>
(the fossil repos are synch'ed every night. to an offshore server.)
<aeth>
I'm not even sure backups are important anymore. I've rewritten things before, every time far superior than before, so if I was forced to rewrite something, I'd probably come up with a better program, although I'd lose time.
<tux0r>
most of my repos don't have open registration. "look, don't touch."
<tux0r>
:)
Pixel_Outlaw has joined #commonlisp
olnw has joined #commonlisp
<tux0r>
a bazaar makes much sense if you know that you suck at what you do. example: if i ever wanted to write a web forum (and you cannot even imagine how often i consider that...), i *KNOW* that i *WILL* fail at auth security. that's where a bazaar is useful: "please, everyone, contribute as many login/auth/registration features as you like!". ok, very simple example, but nevertheless...
<tux0r>
a bazaar is a horrible idea for roughly 9 out of 10 projects.
<tux0r>
(which is why git, modeled around the bazaar concept, is a bad idea for 9 out of 10 projects.)
<tux0r>
(but i digress.)
Lord_Nightmare has joined #commonlisp
<aeth>
Well, for your example, libraries/frameworks can exist. But also, I think the internet was a lot more naïve in the '00s and a lot more willing to give away things for free because nobody knew how profitable software would get (the dot-com bubble burst and software was uncool)
<aeth>
So it was probably easier to get free, high-quality, focused contributions back then.
<random-nick>
I don't get your viewpoint... why do you hate low effort issues so much?
<tux0r>
today, nobody STILL knows how profitable software would get, so they sell remote access to free software ("cloud") instead.
<tux0r>
solving the wrong problem.
<random-nick>
and is it not hypocritical on tux0r's side to hold this viewpoint and yet complain about a feature request being rejected and the project going unmaintained?
<Pixel_Outlaw>
Delivery of low quality and buggy software behind a subscription. :)
<Pixel_Outlaw>
Lock em in, provide garbage.
<tux0r>
random-nick: it is rather easy (and even popular) to be unhappy with things you do.
<tux0r>
my primary complaint was about the single point of failure.
<tux0r>
lisp needs more traction. (welcome to the 70s. i know.)
<aeth>
random-nick: Well, my viewpoint is different from tux0r's. From my perspective, I regret developing in public because there's no point in the public seeing anything that I'm doing. Something that is not ready is not ready, and software takes longer than it looks. In a sense, all issues filed by an outsider are inherently useless while something is incomplete.
<aeth>
and if I had been on GitHub, I would've gotten probably 20x more.
ndanilov has quit [Remote host closed the connection]
Inline has quit [Ping timeout: 244 seconds]
<Pixel_Outlaw>
The average programmer doesn't realize the work required for a library. They'd latch onto something half finished, blog about it then, realizing it's half finished, complain about it. Plus a lot of Lispers are more interested in pet projects and programming the language than making web libraries.
ndanilov has joined #commonlisp
wacki has quit [Quit: Leaving.]
<bjorkintosh>
Pixel_Outlaw: it's because it's not their 9-5 language. they have time to complain.
ndanilov has quit [Remote host closed the connection]
chomwitt has quit [Ping timeout: 272 seconds]
Josh_2 has quit [Ping timeout: 248 seconds]
alternateved has quit [Remote host closed the connection]