<geri>
when you inc a bignum, you still reuse all original memory right?
<geri>
if it gets bigger you just cons for example one more cell and then attach that to the rest of linked list
<abu[7]>
Only a few operations, like add, sub, neg, abs
<abu[7]>
eg. adding a cnt to a big
<abu[7]>
Not "reuse", but "share"
<geri>
wait so if you got a bignum 123 and you need to add digit 4 at the end, itll reuse same beginning part?
<geri>
of cons cells
<geri>
or do you have to cons all the digits again
<abu[7]>
123 is just a cnt. If you have a big with two cells, adding 4 will allocate just one new cell
<geri>
interesting
<abu[7]>
mul or div makes all cells new
<abu[7]>
or shift etc.
<geri>
wait, but inc/dec actually modify the number in-place
<geri>
so the original is gone, so it's more reuse than sharing, no?
<abu[7]>
Not the number
<abu[7]>
The 'var'
<geri>
okay yeah
<abu[7]>
(inc 'N)
<abu[7]>
The original may still be referred somewhere
<geri>
oh
<geri>
(let X N (inc 'N) X)
<abu[7]>
yep
<geri>
should return original number, but N has been modified
<abu[7]>
So Lisp-level functions are never destructive
<abu[7]>
Internal functions are
<abu[7]>
for efficiency
<geri>
yeah, makes sense
<geri>
pretty nice 2 similar numbers reuse most of the cells
<abu[7]>
No need to be similar
<abu[7]>
Just no overflow
<abu[7]>
carry
bjorkintosh has quit [Ping timeout: 260 seconds]
<beneroth>
I believe that's again a point where linked cells vs. arrays come into play
<beneroth>
cells allow much easier re-use, just change/keep pointers. but it gives fragmentation. arrays might be more "transfer-/cache-efficient"(?) in theory (related memory is close together, likely to be paged together), but that this results in an actual practical benefit is questionable, and costs overhead when modifying the memory.
<beneroth>
abu[7], geri, kinda right or am I on the wrong track?
<abu[7]>
No, completely correct
<beneroth>
yay
<abu[7]>
👍
<geri>
i got a recommendation that AST being somewhat inefficient doesn't matter much because it's a very tiny part of the program
<geri>
so the idea of everything is arrays, but AST is built of arrays of 2 items is alright-ish
<geri>
arrays of 2 items == slightly less memory-efficient cells
<geri>
gc is totally gonna be a mess though, lets be real
<geri>
beneroth: yeah, it really depends if and how you modify/access your data
<beneroth>
right. The thing is, it depends on the specific program and specific use case. If you know the kind of inputs/processing and memory behavior your program does, then you can optimize it (maybe even in ASM), but in general that is not worth the effort.
<abu[7]>
geri worries too much about efficienc. Just go ahead and try :)
<geri>
do i now? :D
<beneroth>
in case of Picolisp, where the whole VM and whole software logic is modeled using cells, late-binding and arguably a kind of in-memory de-duplication/copy-on-write this is probably the best general solution combined with a global stopping mark-and-sweep gc, and it's simple.
<abu[7]>
T
<geri>
im more worried not about efficiency but about flexibility
<abu[7]>
ok
<beneroth>
the aim of simplicity is taken more important than pure efficiently, because simplicity makes easier/faster/better understanding of the whole program by the developer, and that is more likely the bottleneck for efficiency (and effectivity) than some low-level optimization which in most cases doesn't matter with todays hardware (and if you need/want to do that, you can still do that in picolisp libraries as e.g. in the ht-library)
<abu[7]>
But then arrays are inflexible
<beneroth>
geri, good point. I love picolisp (and pilDB) for its flexible
<beneroth>
abu[7], T
<geri>
true about picolisp libraries
<geri>
abu[7]: nope
<geri>
cons cells are fixed size
<geri>
arrays can be any size, but resizing is a pain
<abu[7]>
Lists are not
<beneroth>
geri, how not? arrays bring in the additional constraint of having continues memory blocks
<abu[7]>
Lists are not fixed size
<beneroth>
con cells are the smalles usable size
<geri>
use a bunch of arrays of size 2 and you get linked lists
<geri>
but you can build literally any data structure
<beneroth>
where is the difference to con cells?
<beneroth>
except the array syntax, but you still in effect have 2 pointers and nothing else
<geri>
hash maps, vectors, tries and any other advanced kind of tree
<beneroth>
you can do all that in picolisp too, without more overhead
<geri>
beneroth: its just cons cells made of arrays, yes
<beneroth>
mandatory reading for people new to picolisp :)
<abu[7]>
T :)
<geri>
i have read it, yes
<abu[7]>
Arrays are *only* better if you need *fast* numeric indexing
<geri>
let me reread it again though
<beneroth>
the only good use case where arrays are without good alternative is e.g. bitmaps/video buffer etc. But picolisp is not designed to be used for that primarily, does not support that natively, but it can easily be achieved using the (native) mechanism.
<geri>
how do you build tries in picolisp abu[7]
<geri>
ik you dont like hash maps but you cant do them in-language either
<beneroth>
you can do hash maps using 'idx
<abu[7]>
I did not, but you can build *any* data structure from cells
<beneroth>
tries you can build yourself using symbols
<geri>
aren't typical trie levels arrays of size N
<beneroth>
any nested OOP structure (e.g. a parsed json OOP dataset) is arguably a trie
<beneroth>
ah well, not exactly right
<beneroth>
but same principle
<abu[7]>
B-Trees also have big nodes
<abu[7]>
20 or 40 or more branches
<abu[7]>
With arrays you give up simplicity for marginal efficiency gains
<abu[7]>
for certain cases
<abu[7]>
like beneroth said, images
<abu[7]>
or also editors
<abu[7]>
but Vip works perfectly well
<beneroth>
cases where you really want the memory en-bloc
<abu[7]>
It is only efficiency you want
<geri>
actually maybe
<beneroth>
geri, efficency/optimization is by its very nature counter flexibility/generalism
<geri>
but if you write code and it runs way too slow to be useful that locks the language from being used in the domain
<geri>
picolisp has a workaround of having cffi at least
<abu[7]>
PicoLisp is very efficient because it is so simple
<beneroth>
geri, do you have a specific case at hand where your picolisp code is too slow? or is it premature optimization?
<abu[7]>
ffi is not for efficiency
<geri>
beneroth: but again, arrays are just general cells
<beneroth>
ffi is for not having to rewrite existing software but re-using it instead
<abu[7]>
yes
<geri>
ok
<beneroth>
geri, no, they're not. arrays are always packed memory blocks. never fragmented.
<geri>
beneroth: vip on large files lol
<beneroth>
that is what makes the re-sizing painful
<beneroth>
geri, point, I guess vip was not made for that. I don't use vip at the moment :)
<beneroth>
vip not supporting large size comes only from the fact that it loads the whole file, like most editors, instead of just part of the file
<beneroth>
nothing to do with arrays.
<geri>
in-memory, cell is just 2 64 bit words stuck together, arrays are as many 64 bit words stuck together as you need
<abu[7]>
Vip is not the intended use
<abu[7]>
on large files
<beneroth>
geri, yes, but how many you need changes all the time, no?
<geri>
depends on use case
<geri>
for ast you just use arrays of size 2 to build the thing, thus getting cells
<beneroth>
I argue: nearly all use cases it changes all the time
<beneroth>
there is no logical nor binary difference between 2 neighboring 64bit words and an int 64 array of size 2 - or I'm overlooking something?
<geri>
for binary you need to store array size somewhere so its gonna occupy one more cell
<geri>
but functionally its the same as the cell
<beneroth>
which language/stack are you talking about?
<beneroth>
C/C++ does not store array sizes
<beneroth>
(basically the source of its security issues)
<geri>
if you build a lisp interpreter i mean
<geri>
you need to store array size
<beneroth>
why? you can always count
<geri>
lmao
<beneroth>
that is how C/C++ arrays work?
<geri>
in that case theres literally 0 difference between array of size 2 and a cons cell
<beneroth>
exactly?
<geri>
that's what i had been telling you since morning though ._.
<beneroth>
well why use size 2 arrays instead of cons cells?
<abu[7]>
But you want to support also longer arrays (?)
<beneroth>
talking about arrays this way only makes any sense if you talk about larger arrays
<geri>
because you get full array support alongside all the power of cells beneroth
<beneroth>
in picolisp it was a consciousness language design decisions to not natively support larger arrays in the language
<abu[7]>
The power of cells is uniformity!
<geri>
abu[7]: if you use first word of an array to store size in words, you can have up to 2^64(?) words stored in an array
<geri>
but true, there's a limit and it can be a problem
<beneroth>
what's the benefit?
<geri>
and true, abu[7] cells make for a way simpler GC
<beneroth>
in a VM model where you resize and re-allocate cells all the time
<geri>
you dont resize them
<geri>
keep arrays if size 2 as your ast blocks normally
<abu[7]>
You implement all the powerful list processing functions on arrays?
<beneroth>
with cells we don't have to, but with arrays (instead of linked lists) we would need to re-size arrays all the time
<abu[7]>
T
<abu[7]>
append, cons, conc
<geri>
abu[7]: you implement linked lists based on cells based on arrays and do stuff on that if you need
<beneroth>
geri, are you arguing keeping all the existing picolisp the same way it is and additionally support long arrays?
<geri>
no
<beneroth>
then I don't get your point, the goal
<geri>
im arguing that you can do more with arrays as base type instead of cons cells
<abu[7]>
wrong
<beneroth>
yeah, but haven't we just established that size 2 array is the same thing as cons cells?
<abu[7]>
(cdr Array) ?
<geri>
nuh uh
<geri>
okay
<geri>
on implementation level its all arrays
<abu[7]>
(cons 1 Array)
<geri>
on language level you get cells (just arrays of size 2)
<abu[7]>
*very* inefficient
<geri>
and those give you normal lists
<beneroth>
no, on implementation level its all size 2 arrays, that size 2 matters a lot
<beneroth>
abu[7], good point
<geri>
man
<beneroth>
geri, how to implement (cons 1 Array), aka front-pushing a value into a list, with arrays
<abu[7]>
In Pil the non-language level also has arrays
<beneroth>
with arrays, you need to move all the content of the arrays one element to the right, no?
<abu[7]>
copy
<geri>
you keep on misunderstanding what i meant i think
<beneroth>
abu[7], the clojure way
<beneroth>
geri, yeah :D
<abu[7]>
moving right would be destructive
<geri>
cons cell is just a special case of array
<geri>
it has 2 elements, thats it
<beneroth>
the question is, are you fully understanding what you mean :)
<geri>
yes
<geri>
is my point about cons cells clear so far?
<beneroth>
sure? is not array a special case of cons cells?
<geri>
opposite
<beneroth>
why?
<geri>
because you can have arrays of any size, but cons cells only of size 2
<beneroth>
exactly. more limits = simpler = more basic
<geri>
cons cell is a specialized array
<geri>
array is a generalized cons cell
<beneroth>
copying cons cell around: you know you have to copy 2, have memory for that, etc.
<abu[7]>
This is ok, the problem is using both
<beneroth>
copying arrays: you need to look up size first
<geri>
beneroth: thats why im saying, use cons cells for AST, but those cons cells are just arrays of size 2
<geri>
then (cons 'thing array) is just a normal cons
<beneroth>
ok. why support longer arrays?
<geri>
creates an array of size 2, first element is 'thing, second is array
<beneroth>
if you do the (cons 'thing array) you end up with the picolisp we already have
<geri>
beneroth: wdym
<geri>
about longer arrays
<abu[7]>
I give up
<beneroth>
why support arrays with sizes > 2, which brings in a lot of additional overhead needed whenever handling them
<abu[7]>
Build it!
<abu[7]>
Then let's sec
<abu[7]>
see
<geri>
im planning to!
<abu[7]>
bye
abu[7] has left #picolisp [#picolisp]
<geri>
damn
<geri>
beneroth: exactly, you get picolisp but you have full arrays as well if you need them
<geri>
and you do need them for some of the data structures you may wanna use
<geri>
or just use cons cells (arrays of size 2), its fine too
<beneroth>
you need to see every 3 years he has the same damn discussion and he thought he fully explained his point in the essay https://picolisp.com/wiki/?ArrayAbstinence
<beneroth>
geri, "you have full arrays as well" THATS WRONG. if you want full arrays as well, all code handling con pairs must also be able to handle arrays of arbitrary size, no?
<beneroth>
else you end up with different set of functions for simple linked lists (lisp style) and arrays, no?
<geri>
you have different set of functions, yes
abu[7] has joined #picolisp
<abu[7]>
This discussion makes me sick, I thougt this array issue was clear
<geri>
just how you have functions for idx, enum, etc.
<beneroth>
abu[7], yeah I just wrote. chill :)
<abu[7]>
bbl
<beneroth>
geri, yeah so double all functions or make all functions also able to handles arrays
<beneroth>
that's the core point
<beneroth>
not doing that
<geri>
you dont need to double all functions - car & cdr dont make much sense to do on arrays for example
<geri>
but yes, added complexity
<geri>
yes, more complex gc
<beneroth>
not for the very few use cases where arrays actually would be better. if you have specific use cases where arrays are better than flexible linked lists, and the performance matter, than you probably should anyway write specific code and memory handling for that data handling
<beneroth>
geri, exactly. the whole point of picolisp is a good trade-off/balance between as least complexity (also in the number of concepts) and still have something useful for fast-paced development speeds
<beneroth>
a much simpler runtime and gc also allows much easier porting of the runtime
<geri>
yes!
<geri>
and it's great!
<beneroth>
picolisp throws away compilers in favour of minimal optimized interpreter
<geri>
never said that elusive lisp would need to be compiled
<beneroth>
as you just said yourself, you can not have such a simple system and also array support
<beneroth>
that's the point
<geri>
yeah, it's a tradeoff
<beneroth>
everything always is
<geri>
question is, is it worth it?
<geri>
for picolisp no, as it wants simplicity
<geri>
but for some other project?
<beneroth>
abu[7], and picolispers, say: yes worth it
<beneroth>
use another language for other project?
<beneroth>
picolisp doesn't claim, as most languages falsely do, being the optimal tool for everything
<geri>
by other project i meant other language
<beneroth>
so you say the whole discussion is not about picolisp but theorized unspecific language design?
<geri>
yes
<geri>
also again, with cons cells you can build a ton of things of different structures, and you'll have some functions that only apply to a particular structure and not another
<beneroth>
why discuss this here? didn't make abu[7] is point already clear with picolisp, that he believes this kind of trade-off is worth it, didn't he make clear that this was a purposeful decision (see the essay) ?
<beneroth>
the thing with "some functions only apply to a particular structure" goes counter the uniformity and simplicity goal of picolisp
<geri>
well try to do assoc on a plist
<beneroth>
only 3 datatypes (and all other duck-type modeled on top of them) is very radical
<geri>
and tell me hwo that goes
<beneroth>
faster than trees up to a few thousand elements. then you use 'idx tree instead.
<beneroth>
stop theorizing. start building. if you don't believe try it out.
<geri>
why discuss here - cause i hoped you guys got an open enough mind to just discuss a maybe cool idea, never had i said this is how picolisp gotta be
<beneroth>
abu[7] has earned his living and whole career by building practical daily-used software with picolisp for decades
<beneroth>
I earn my living using picolisp for real-world applications for more than 10 years now
<beneroth>
it works in practice
<beneroth>
believe it or not
<geri>
never said it doesnt
<beneroth>
well cool idea.. clearly we thought about it and have thrown it out. what's so hard to get about that :D
<geri>
from what i get from array abstinance you thought about an extra type for arrays, not only arrays and no cons cells, but i might be wrong
<beneroth>
the point of the essay is that picolisp as on purpose no array type
<beneroth>
because that brings a lot of benefits. and the downside is minimal, because there are hardly any real-world use cases where arrays are better than alternatives
<beneroth>
Tree's and Trie's are not based on arrays but fragmented linked graph structures. that's very far away from arrays.
<geri>
also i hadnt seen any points about using arrays to build cons cells in array abstinance
ello has quit [Read error: Connection reset by peer]
ello_ has joined #picolisp
<beneroth>
because that point is nonsense when you mean size-2-array, and if you mean different sizes, then you are again talking about arrays in general
<geri>
the phrase of that whole page is just: The advantages in efficiency do not outweigh the disadvantages of increased complexity.
<geri>
and that's fine
<beneroth>
exactly
<beneroth>
and as you said yourself, adding arrays to that would necessitate additional complexity, right?
<beneroth>
complexity is bad
<geri>
well, at least i got inspired to write a usable poc lol
<beneroth>
”Perfection is attained not when there is nothing left to add but when there is nothing left to take away” (Antoine de Saint-Exup´ery)
<beneroth>
geri, do that
<beneroth>
then we can discuss and compare specifics :)
<geri>
a bit later
<beneroth>
discussing here about picolisp not having arrays is like discussing with the python author about the lack of brackets...
<geri>
its not about not having arrays, its that you could do everything cons cells do and more with arrays and no cons cells
<geri>
but anyway
<geri>
just say "complexity isnt worth it" and theres that
<beneroth>
yeah exactly. it's a settled and reflected decision.
<beneroth>
if you doubt the arguments in favour of that decision, than present concrete counter evidence.
<geri>
i just didnt like that you turned it into "everything must be a resizable array", when i meant you can have cons cells as special case of arrays
<beneroth>
"why did you do X and not Y?" - "I thought about that and then decided against Y because of the benefits I saw in my use case and have proven in countless experience since then"
<beneroth>
geri, well that was because we understand you as saying "use arrays instead of cons cells"
<beneroth>
base everything on arrays, not on cells
<geri>
yeah, but it didnt mean get rid of linked lists, just that cons cells could be modeled on arrays
<beneroth>
this way around your question is "why didn't you extended the language with support for a native array type?" and the answer is "because it was never needed"
<beneroth>
"cons cells modeled on arrays", IF you mean size-2 arrays (and ONLY size 2 arrays) is already how it is...
<beneroth>
if you disagree on that than we don't define the word "array" the same, I guess
<geri>
it is like this minus ability to have size of an array be any bigger than 2 in general
<beneroth>
of course you can still question the validity of that design decision, but with all the arguments, thoughts and experience behind it your questioning turns into questioning the authenticity and truthfulness of abu[7]
<geri>
find a place where i said thats how picolisp shouldve done it
<geri>
ill wait
<beneroth>
that's why its understood as a personal attack. because it is not the same as asking for the arguments for that decision in the first place, you got answers to those questions and didn't like them.
<geri>
i dont like that you dont try to understand my point
<geri>
and because of that you bring up nonsense like consing symbol and array being broken somehow
<beneroth>
the feeling is mutual. we think we understand your point.
<geri>
yeah gg
<beneroth>
you naming it nonsense just shows you don't understand the argument
<geri>
if youre answering the wrong question how do you expect me to accept the answer? xd
<geri>
this convo either has to be restarted or scratched, maybe forever
<beneroth>
either you want to base everything on (not-only-size-2-) arrays, instead of cons cells - then consing values at the front of an array becomes a very complex operation - or you just ask why is there not additionally also support for larger-than-size-2-arrays, for which the answer is "never needed in our experience". so why persist in asking again?
<beneroth>
we feel we've answered your questions pretty early on
<geri>
consing doesnt become more complex if you treat arrays of size 2 differently from arrays of size any
<beneroth>
how not? if the result of the consing must be an array?
<geri>
yes, an array of size 2
<geri>
aka cons cell
<geri>
if you wanna prepend to an array thats indeed a complex operation
<geri>
but if you wanna prepend to a list based on arrays its as usual
<beneroth>
lets discuss specifics. we have a (on logic level) a list of 2 elements, in your system represented as an array [a b], right?
<beneroth>
ah
<beneroth>
well then you are not working with arrays
<beneroth>
your arguing the picolisp way of working then again
<geri>
a lost of 2 elements is [a, [b, NIL]], just like normal
<geri>
[a, b] is (a . b)
<geri>
nothing about picolisp here
<beneroth>
well that is how picolisp works right now
<geri>
yes
<beneroth>
so where is your point?
<geri>
if you need an actual array for some reason (maybe you dont, idk)
<geri>
you allocate [a, b, c, d, e, f, g] or whatever
<beneroth>
then malloc one using (native) and that's it
<geri>
also true
<geri>
honestly best argument so far lol
<beneroth>
but than the system is not "based on arrays"
<geri>
yes it isnt
<geri>
you still get simple gc and no overload of functions but can get arrays (not conviniently, but it works) on langauge level
<geri>
thats honestly a good option
<beneroth>
well you have to do the memory management yourself for such an array. but you likely also want to that anyway if your use case really makes sense to use an array
<beneroth>
in all other cases you are better of with normal list and tree structures
<geri>
with everything-built-on-arrays approach you can do that purely in language but its a good question if its needed considering system complexity overhead
<geri>
compared to implementing (native)
<geri>
could be worth it if you want your language to be more cross-platform, but again its a non-goal for picolisp
<beneroth>
we tell you: no. if you question that, go and make a good case and come then again. Until then, second-guessing our answers is second-guessing ourselves. No?
<beneroth>
how does array support help with cross-platform? it makes the VM bigger, needs more implementation effort, not less
<geri>
you dont need to rely on C for some stuff
<beneroth>
we don't rely on C now?
<beneroth>
honestly you just haven't thought-through your arguments
<geri>
you are not your answers, get some self-confidence lol
<geri>
beneroth: if you need an array you rely on native, which is basically just cffi
<beneroth>
true, but you keep asking the same point senselessly
<geri>
okay, "arrays bad" man
<geri>
i think im done too
<beneroth>
arrays bad, that's the point of the "array abstinence" essay, whats so hard to get. just accept it.
<beneroth>
come again with a real-world use case where you need an array. and then lets discuss how picolisp is still otherwise the optimal language for that use case and how you don't have a libc implementation on the same platform
<beneroth>
that argument is just exotic nonsense
<beneroth>
what kind of software are you even working on?
<beneroth>
usually when such a discussion comes up here it's just theorizing of language crafting by people who did no language design and somehow believe there is a global optimal without any trade-offs
<geri>
i talked about like 3-4 different points and all you hear is "array", and answer with "array bad"
<geri>
arrays good for beign cached better and constant time access, thats not even questionable
<geri>
i talk about downsides of that approach myself, i know there are some
<geri>
but you just trash all upsides for no reason as well
<geri>
literally all software benefits from cache hits on modern hardware
<beneroth>
well we looked into it and came to this conclusions
<beneroth>
bring a specific case to benchmark then
<beneroth>
if you don't believe us
<geri>
where
<geri>
the fuck
<geri>
did i say i dont believe you
<beneroth>
by asking the same point constantly?
<geri>
which point
<geri>
copy the liens
<geri>
lines
<beneroth>
<geri> so the idea of everything is arrays, but AST is built of arrays of 2 items is alright-ish
<beneroth>
<geri> im arguing that you can do more with arrays as base type instead of cons cells
<beneroth>
<geri> could be worth it if you want your language to be more cross-platform
<geri>
first sentence is about it being viable overall, second about efficiency enabling certain use cases, third one about possibly being more cross platform than using (native) because no reliance on libc/shared objects/whatever
<geri>
how is that the same point ~w~
<geri>
like i really have nothing against you guys
<geri>
honestly yes, im thinking about minmaxing efficiency
<geri>
while keeping simplicity in focus as much as possible
<beneroth>
well if you think you can do better than picolisp within similar trade-offs, then go ahead and proof it ;-)
<geri>
i think it could be better at some things and worse at others
<geri>
the usual stuff
<geri>
better if you need constant access stuff, worse at gcing and maybe needing to overload some builtins, like map
<geri>
better at allowing you to build anything reasonably from within the language without dropping to (native) too
<beneroth>
well then implement it so I can write software with it. unless it can be used for practical stuff it's worthless.
<geri>
also true
<abu[7]>
Thanks beneroth for the good explanations!
<geri>
well, there were good points and i reacted to them as such, others not so much
<geri>
lets just unite that picolisp doesnt need arrays and if you want picolisp with them go write it
<geri>
lol
<abu[7]>
T
<abu[7]>
It is not PicoLisp then
<abu[7]>
not the philosophy
<geri>
ofc
<geri>
i called my previous picolisp-inspired interpreter centilisp cause its a little bigger than picolisp lol
<geri>
would be nanolisp if the name wasnt taken, or whatever next metric prefix was
<abu[7]>
I see
bjorkintosh has joined #picolisp
bjorkintosh has joined #picolisp
bjorkintosh has quit [Ping timeout: 260 seconds]
bjorkintosh has joined #picolisp
bjorkintosh has quit [Changing host]
bjorkintosh has joined #picolisp
NobleNibble has joined #picolisp
beneroth has quit [Remote host closed the connection]
beneroth has joined #picolisp
oldf8l has quit [Ping timeout: 252 seconds]
NobleNibble has quit [Remote host closed the connection]