<freemint>
Hey abu[7] long time no see. I really like the picolisp db, sadly i do not do much work in picolisp nowadays and am occasionally day dreaming about reimplementing PilDB in another language. Would you be fine if i named said library BurgerDB? Do you have a different suggestion?
<abu[7]>
Hi freemint! :)
<abu[7]>
I think my name should not be used, but thanks!
<abu[7]>
Something with object? Or symbols? Or Pil?
<freemint>
Mhhh PillDB
<abu[7]>
2 'l's sounds like a medicine ;)
<freemint>
Well it is PIcoLisp Like and it has a visual image of a pill people can remember. Similar to Burger where can have a Burger as image.
<abu[7]>
True
<abu[7]>
But a burger has something "in between"
<abu[7]>
like the onion in Tor?
<freemint>
Well that would be the users structs and data :D
<abu[7]>
I have no nifty idea atm
<abu[7]>
Perhaps beneroth? He has a fresh and practical view on the Pil DB
<beneroth>
hi abu[7], hi freemint
<bjorkintosh>
freemint: since it's PIcoLIsp, why not call it piDB?
<freemint>
I would basically thinking reimplementing external objects mechanism and then slowly build on top. Since the language (i am vague since last you chewed me out for bringing up a different language and i still feel awkward about that) has arrays I might allow different type of disk files more optimized for fixed length entries. But other than that is
<freemint>
should be pretty traditional
<bjorkintosh>
there's a handy symbol in greek somewhere you can use with it.
<abu[7]>
Hi beneroth!
<freemint>
mhh pidb already has a few non programming term associated with.
* bjorkintosh
is thinking of Go, Rust, Lisp, among others.
<beneroth>
I'm skeptical about how well it is possible to separate pilDB from picolisp. A lot in pilDB rests on the 3 (strong static) datatypes of picolisp. I guess it will become something very different quite quickly when you add a lot of additional datatypes not conforming to the 3 fundamental types.
<abu[7]>
Right
<bjorkintosh>
well as long as the implementation language is homoiconic, it shouldn't be too difficult.
<freemint>
i can just tag the data stored with a tag that confirm the type when i write it. I also plan to store the make up these serialized types in a disk file.
<freemint>
The language is homoiconic enough.
<beneroth>
the whole schema system of pilDB (relations, index management, the "daemon objects" as they are named in pilDB) is based upon the picolisp OOP system, which among its other specifics has most importantly ordered inheritance (which solves the diamond problem and keeps multiple inheritance, which is widely used in pilDB relation management)
<beneroth>
of course you can leave the schema system of pilDB away - then you have a simple key/value store. It's just the PLIO data format and very few additions. There is/was a PLIO implementation in C and even JS around.
<beneroth>
if you want the pilDB schema/relation system, you also require the picolisp OOP system.
<freemint>
I am not sure if i implement that OOP schema the same way, i'd rather build something that matches the to way inheritance works in that language.
<beneroth>
yeah, the "ordered inheritance" solution to multiple inheritance has its own merit. Very elegant. Never saw that in any OOP system.
<beneroth>
abu[7], I guess that just came to you naturally? or did you see ordered inheritance somewhere else?
* beneroth
suspects that is a true A. Burger invention
<abu[7]>
Yes, I just did it
<beneroth>
:D
<abu[7]>
was under influence of Smalltalk
<freemint>
I also came up with ordered inheritance up naturally before i knew PicoLisp. It was when i finally groked OO by building my own object oriented system in Lua.
<abu[7]>
(though never used it)
<beneroth>
I find it truly elegant. it solves the diamond problem, which the weak attempts of "interfaces" in other most newer languages (java and newer) doesn't solve.
<freemint>
I think it emerges naturally from the implementation in an interpreter.
<beneroth>
abu[7], well nobody doing OOP can claim not being under influence of Smalltalk :)
<abu[7]>
freemint: T
<beneroth>
freemint, T. Hard to do in non-interpreter actually
<abu[7]>
yeah
<beneroth>
instead of one vtable (C++ style) you would need a series of vtables
<freemint>
I think this could be flattened into one vtable again by the compiler under mild assumptions.
<beneroth>
freemint, what language are you using nowadays, and what would you use for your pilDB clone? julia?
<freemint>
mhhh, maybe not the compiler but the linke.
<beneroth>
compiler
<freemint>
Yeah i still like Julia since i work a lot with floating point numbers.
<beneroth>
no love for precision yet? :P
<beneroth>
scnr
<freemint>
In fact the opposite, quite a lot of love for that. I do concern myself with that.
<beneroth>
to answer the main question: additional language interfaces into pilDB (especially for data I/O into a pre-defined pilDB) is certainly a worthwhile quest. That can be done by implementing some interface adapters in pil. Re-implementing whole pilDB without ending up with a picolisp-clone I deem unlikely, because most of it is actually implemented in picolisp and not in the VM (some essentials are in the VM, like the lazy-loading).
<abu[7]>
and, another question was a name ;)
<freemint>
*nods* I do not value binary compatibility with picolisp but i think i could implement something which can also be extended to read pildbs.
<beneroth>
pilDB-inspired different databases which just take inspiration of pilDB (e.g. PLIO data format or similar, multiple files with data-specific block-sizes, lazy-loading (!!!), maybe even the IPC system) might be worth it. But it would very quickly evolve in something very different.
<beneroth>
I actually would probably not copy the IPC but maybe implement a WAL for better read-isolation.
<freemint>
What do you mean with very different? When would you consider it something different?
<beneroth>
incompatible and different main concepts in it?
<freemint>
Ahh, yeah.
<freemint>
I like the idea of objects in the DB inheriting from a common object, i might keep that.
<beneroth>
if it's quite different, than the name should also not be directly related to pilDB or picolisp
<beneroth>
pilDB and abu can still be honorably mentioned
<freemint>
Well then that removes constraints on the name finding problem, which makes it harder to solve :-D
<abu[7]>
:)
<beneroth>
freemint, that's not specific to pilDB (think Java, everything inheriting from Object class), it's just the vehicle for the pilDB schema system implementation in pilDB
<beneroth>
which also means you could easily implement different DBMS in pilDB using only the low-level key/value store :)
<beneroth>
just with a different parent class instead of +Entity and overloading (rel ...) and thelike :D
<beneroth>
freemint, T
<beneroth>
that is a wise insight. more constraint makes it easier.
<freemint>
I very much also like the idea of building the indexing datastractures in the homo-iconic language.
<beneroth>
yeah, its very valuable. But then you get into the territory where you also need to clone picolisp. Then you move quickly into territory where it would just be easier to use picolisp and pilDB and add an interface for whatever other language you are using.
<beneroth>
in a current project I need to support floats and decimals. I basically just store them in pilDB as a class with 3 +Number relations: integer value, fractional value (after the decimal point) and scale value (aka *Scl)
<beneroth>
you could of course also model the float/double value with just 2 +Number relations
<abu[7]>
or use a single sufficiently big scale througout the whole app
<abu[7]>
though sometimes I use 2
<beneroth>
freemint, you should think hard about what your requirements and goals for your database system are. concurrency? if so - concurrent writes or concurrent reads? space-efficient storage?
<abu[7]>
2 scales
<freemint>
The hard requirement is approaching the expressivity pil db gives.
<freemint>
Thinking about SQL just makes me sad.
<beneroth>
yeah, you probably will not get that without doing a clone of picolisp
<freemint>
What would i need to clone picolisp for?
<beneroth>
to gain the similar expressivity and flexibility that pilDB gives
<abu[7]>
you need symbols
<beneroth>
T
<freemint>
The language is homoiconic and has symbols.
<abu[7]>
with properties
<abu[7]>
ok
<beneroth>
Lisp Symbols
<abu[7]>
yes. Not just names
<freemint>
It has symbols a property (Value). If one needs value and property list ( i am bit rusty on my picolisp i just noticed) then one can just look up the symbols in an additional scope which maps to property lists.
<abu[7]>
PLists are not important
<abu[7]>
P*lists*
<beneroth>
pilDB is simply: picolisp, "save-able" (= persistent) picolisp symbols, a bit file IO (locking, writing, reading), the global standardized tuple adress format (the naming of external symbols), both combined for lazy-loading. That's pilDB key/values.
<beneroth>
everything else is pure picolisp and picolisp OOP
<freemint>
*nods*
<beneroth>
the expressivity and flexibility in pilDB come from picolisp
<beneroth>
nearly all of it, is simply "emergent" behavior stemming from the 3 fundamental datatypes and picolisp evaluation rules.
<beneroth>
the power comes from the restriction on this 3 fundamental datatypes, making it easy to support them in all functionalities, thus resulting in those functionalities being highly re-usable
<freemint>
I would say it comes from the interpretedness and thereby dynamicness and the homo-iconicity and the ability to handle orthogonal concerns via ordered inheritance and being gced.
<abu[7]>
For me it is perhaps the direct-linked-ness of external symbols
<abu[7]>
gc depends on it
<abu[7]>
dbgc
<freemint>
Can you elaborate?
<beneroth>
abu[7], T
<abu[7]>
The propertiet may be other external syms
<beneroth>
which is a result of the uniform address format implemented in the external symbol naming format
<abu[7]>
and values too
<freemint>
Yeah i definetly want that.
<abu[7]>
👍
<freemint>
And i think that is possible too.
<abu[7]>
as a result you get lazy loading
<beneroth>
for comparison, records/tuples in relational DBMS don't have a unique identifier. That's why Joins are so expensive, because they need lookups in multiple tables/trees.
<freemint>
I need an example to comprehend that.
<freemint>
the records/tuples calim
<beneroth>
abu[7], T. The lazy loading is a very important property of pilDB. Other DBMS and ORMs are trying to get there, but are very restricted by the separation of DBSM and application.
<abu[7]>
freemint, needing join tables
<beneroth>
foreign keys
<abu[7]>
with indexes
<freemint>
*nods* ahh
<abu[7]>
In pil (put '{7} 'a {123})
<beneroth>
in SQL: UPDATE users SET orgID = 5 WHERE id = 7;
<beneroth>
and orgID is an integer value, not the address of another record
<beneroth>
so to get the org record, you need to look it up in the org.id PK index first
<abu[7]>
for the records: I forgot a quote above ;)
<beneroth>
aye ;)
<freemint>
Mhh i am currently thinking about Julia not being list focused, instead the fundamental growable data structure is a multi-dimensional array (which can be sprecialized to be 1D, 2D) ....
<beneroth>
technically, relational databases handle/store everything as arrays (C/C++ arrays, continues memory blocked, accessing elements only by index)
<abu[7]>
A direct benefit of Lisp is that the code of queries can be build easily from data
<abu[7]>
as 'search' does
<beneroth>
freemint, do you have a length property in Julia arrays which is always updated, or does determining length require going through the elements?
<beneroth>
abu[7], T
<freemint>
They have a length property, if not it would be trivial to implement an array type which does have that.
<beneroth>
do they keep data in a continues block or might they be implemented whatever methodology under the hood (e.g. a tree) ?
<beneroth>
growing/shrinking with continues blocks necessitates copying
<beneroth>
I wouldn't consider that really an array. I would call that an 'vector' as it is named in C++ STL
<beneroth>
but yeah
<beneroth>
abu[7], that's another source of the array abstinence debate... people usually don't mean array when they say array
<abu[7]>
T
<beneroth>
abu[7], important to keep in mind. People got lazy and newer languages got improper in their use of terminology
<abu[7]>
Julia arrays are un-simple
<freemint>
*nods*
<beneroth>
I would say: then they aren't arrays
<beneroth>
why not call it vectors like C++ STL
<freemint>
Vectors conotate being 1D to mathematicians but they can be multi-D.
<abu[7]>
seems even more
<beneroth>
it's hard to compare and debate anything if the terminology is not shared
<abu[7]>
a storage manager
<freemint>
The mathematical correct term would be Tensor.
<beneroth>
abu[7], T
<beneroth>
it's just an abstract key/value container with integer used for keys
<beneroth>
freemint, why not simply write an interface that represents pilDB as an array into Julia? :P
<beneroth>
then you have half of a relational DBMS xD
* beneroth
is snarky
<freemint>
Key value container where the keys are from a cartesian product of ranges.
<beneroth>
abu[7], next time someone shows up talking about arrays, lets just say "yes picolisp has arrays, we call then 'enum'" :P
<freemint>
*giggles*
<abu[7]>
:)
<abu[7]>
mapping ints to data
<freemint>
Mapping a fixed size tupple of ints to data.
<beneroth>
abu[7], that seems to be the common meaning of array nowadays
<abu[7]>
freemint, fixed size?
<freemint>
Fixed dimension, flexible size.
<abu[7]>
dim?
<abu[7]>
ok, but 'enum' supports more dimensions
<abu[7]>
args
<beneroth>
freemint, the flexibility and expressiveness of picolisp and pilDB comes from purposely removing/leaving away as many restrictions as possible. On the other end you can gain implementation optimization by adding restrictions (like "only integer values are allowed as keys").
<beneroth>
abu[7], our arrays are more powerful than julias!!! :P
<abu[7]>
hmm ;)
<beneroth>
freemint, but you cannot have both - either flexibility (less optimization, more generalization) - or more verbosity, more optimizations but also more restrictions
<abu[7]>
As Paul Graham says, succinctness is power
<freemint>
mhh, fair enough if you throw arbitrary keys which do not belong to a range (something iterable with a start and stop) you would not call that an array in julia.
<bjorkintosh>
PG.
<bjorkintosh>
met him once, a long time ago.
<abu[7]>
oh
<beneroth>
abu[7], his right on that. but he also tells a lot of bullshit
<abu[7]>
nice
<bjorkintosh>
I should have gotten an autograph.
<abu[7]>
beneroth, T, arc
<bjorkintosh>
he paid for dinner. What a gentleman.
<abu[7]>
cool
<beneroth>
and even there.. the elegance of lisp language is that it could be reduced two a handful functions taking up like half A4 paper of code. Still, nearly all lisps implement and hardcode much more of their functionality for implementation/performance reasons.
<beneroth>
bjorkintosh, cool. How did that happen? were you in YC?
<abu[7]>
business purpose?
<beneroth>
I tried once to give stickets to RMS for free. Something happened, the result was that RMS didn't not accept my free stickers and I bought his expensive stickers. Good salesman, RMS.
<bjorkintosh>
beneroth: not at all. I was in Boston visiting and I knew he was giving a talk to launch ... YC!
<bjorkintosh>
I should have paid attention and snuck in a start up.
<bjorkintosh>
but I didn't.
<beneroth>
hindsight is easy :)
* bjorkintosh
doesn't want to talk about how he missed bit coin completely.
<beneroth>
did you miss it?
<beneroth>
I just understood in the beginning that it can never be tied to some useful function besides payment.
<beneroth>
and I extremely underestimated people getting into it just for gambling without a meaningful purpose
<bjorkintosh>
beneroth: yeah. the stupidity of gamblers was something I failed to comprehend.
<beneroth>
abu[7], you got some BTC from early times? I remember when you asked for donations for pil, a few BTC would be alright :D
<beneroth>
bjorkintosh, yeah same
<bjorkintosh>
and I know I'll be missing the next round of stupid gambling again, just as I didn't understand NFCs.
* beneroth
still overestimates common intelligence
<abu[7]>
beneroth, yes, but now I think Monero is better
<abu[7]>
the original plilosophy
<beneroth>
how? just because its more anonymous?
<freemint>
Julia's underlying homoiconic code is made from Expr (a struct with property head which is a symbol and property tail which is a Vector{Any} (growable continous integer indexed collection of any type)).
<abu[7]>
No, you can mine on a PC
<beneroth>
bjorkintosh, NFCs was just a ponzi. Dutch Tulip craze. But many crypto scams are not ponzi!
<bjorkintosh>
beneroth: so's crypto.
<beneroth>
freemint, sounds like javascript
<bjorkintosh>
people still love Ponzi
<beneroth>
T
<freemint>
I think you mean NFTs. NFC is near field comminucation
<beneroth>
freemint, right! T
<bjorkintosh>
freemint: sorry, yes.
<beneroth>
NFC is also a lie tough
<bjorkintosh>
see? I missed it again.
<beneroth>
'near' can be stretched quite a lot
<beneroth>
bjorkintosh, didn't you buy trump coins when he told you? :D
<abu[7]>
haha
<bjorkintosh>
beneroth: well, I stopped listening to NPR 20 years ago, and that was my last news hold out.
<beneroth>
freemint, everything in javascript is objects (structs) except for some native scalar values
<bjorkintosh>
so I only see memes.
<beneroth>
bjorkintosh, haha I understand you. memes are the fastest news source.
<bjorkintosh>
yes!
<beneroth>
lot of propaganda in it though. a lot worse than 10y ago. dangerous.
<bjorkintosh>
but my memes are around programming themes, so I still miss the ponzi ones.
<beneroth>
where do you read memes? reddit?
<bjorkintosh>
beneroth: naturally.
<beneroth>
I've yet to find a good meme source with primarily programming memes. And TBH most programming memes are shit.
<beneroth>
bjorkintosh, kk
<bjorkintosh>
well yes.
<beneroth>
freemint, abu[7] where picolisp uses the 'list' datatype to represent functions, javascript is using objects
<bjorkintosh>
yes. all my other news comes from PGs site: ycombinator.
* beneroth
goes crying some more
<freemint>
I originally did not bring up arrays to start an array abstinance discussion. In my equivalent to +Entities in Julia i also have the option that instead of Vector{Any} (closes match to list) I could Tensors of any dimension. Instead of lists i could make my DB system aware that there are Matrices, 3-D tensors stored. Do you see a
<freemint>
problem/oppotunities with that?
<beneroth>
bjorkintosh, there I read a lot too. but its very annoying. because my habit is to open everything that looks interesting to me in tabs, and then taking years to read them (and only after reading I add it to bookmarks or ditch it)
* beneroth
has probably multiple botnets living solely in JS on his open tabs
<abu[7]>
freemint, what do you think is the advantage?
<beneroth>
freemint, yeah no worries. I brought up the array abstinence discussion (primarily to tell abu that most don't mean array when they say array).
<beneroth>
freemint, I see no problems
<abu[7]>
All in Tensors is overhead for thhh common cases
<beneroth>
freemint, you know about the +UB index prefix class
<abu[7]>
the*
<abu[7]>
beneroth, right
<abu[7]>
multi dim index
<freemint>
It is not about it being an advantage, it is about exposing the underlying language as natively as possible. A consequence of that decision instead of using a nested lists for 2d info is that the types tell you need to two indixes to access a member. This is different from an +UB index as all the entries in the tensor would be eagerly loaded when
<freemint>
the tensor is loaded.
<abu[7]>
exposing the underlying language is good, but I cannot judge about Julia
<abu[7]>
The above sounds like over-generalization to me
<bjorkintosh>
Julia famously has a lisp kernel, unless I misheard?
<abu[7]>
I think I saw once a lang which had only *one* single flow statement, a LOOP with conditional exit. No 'if' etc.
<beneroth>
bjorkintosh, WTF is a lisp kernel and when is it not a lisp kernel? :D
<freemint>
It's parser in the implemenation is written in FemtoLisp and noone has rewritten it in Julia, so the Julia executable caries around a femto lisp
<bjorkintosh>
beneroth: well they used femtolisp as the basis for creating the rest of julia.
<freemint>
you can access a femtolisp repl using julia.exe --lisp
<beneroth>
freemint, and in the array implementation link above they talk about how arrays are implemented in C and nobody was able yet to do it in Julia.
<bjorkintosh>
abu[7]: minimalist RISC: SISC
<beneroth>
freemint, can femtolisp also be accessed from within a running julia program? like calling femto (eval ...) ?
<bjorkintosh>
single instruction set computers. typically SBNEZ subtract and branch if negative or something.
<abu[7]>
bjorkintosh, T, a single instruction
<bjorkintosh>
sorta like lambda, if you squint just right.
<bjorkintosh>
which I'm so ashamed to say, took me a long time to realize that juxtaposition was one of its functions!
<freemint>
*nods* The StarwarsArrays was meant to show that the accessing arrays behaviour can be redefined to behave arbirtrarily.
<bjorkintosh>
the perils of trying to teach yourself things.
<beneroth>
freemint, yeah, then its not an array anymore in my book
<beneroth>
bjorkintosh, aye, I feel you
<beneroth>
abu[7], bjorkintosh so should be easy to build a lisp machine on top of it then?
<freemint>
I am not aware to access FemtoLisp directly, it is quite likely, you can with the loading it direcly.
<bjorkintosh>
beneroth: lisp can be built on anything, since its origin is paper and pencil!
<abu[7]>
There is also "no instruction computer", no opcode needed
<abu[7]>
just 2 addresses
<bjorkintosh>
oh yes. the move machine.
<abu[7]>
T
<beneroth>
freemint, bjorkintosh ok. sounds to me that just the first versions of Julia VM were implemented in FemtoLisp (this Scheme dialect) and now most of it is implemented in Julia and C ?
<bjorkintosh>
without memory, you have no computing.
<beneroth>
abu[7], sounds very binary!
<bjorkintosh>
beneroth: it's still there. it's what converts everything to an AST
<beneroth>
bjorkintosh, so they're not even yet self-hosting but claim to be a universal language? :P
<bjorkintosh>
marketing is a powerful tool.
<freemint>
Julia is a written in C and Julia and talks to LLVM a lot, the parser uses femtolisp.
<beneroth>
picolisp is writtein in LLVM :D
<beneroth>
probably Julia was too big to go that route
<beneroth>
s/was/is
<abu[7]>
Pil is not really written in LLVM, it compiles its sources to LLVM-IR
<bjorkintosh>
so Pil is retargetable then!
<abu[7]>
retarget to where?
<freemint>
beneroth Julia is currently not self hosting. That is correct, it's implementation relies on C and while one could statically compile julia to produce equivalent code to what the C code does, that isn't done, as that would just add an additional build step or requiring Julia already installed to build Julia.
<bjorkintosh>
abu[7]: well anything. Wasm for instance.
<freemint>
Julia gets rid of that Tensor overhead (for example to Vector{Type} being an alias for Array{1,Type}). Julia allows specialized implementation depending on finer, parametrized types and compilation for those types so vtable stuff can be completly eliminated if it is clear the call site will only use combinations of types which lead to one entry in
<freemint>
the vtable. (borrowing vtable from C++, not a direct concept in Julia)
<abu[7]>
bjorkintosh, theoreti6ally yes
<abu[7]>
at least if it is SSA
<freemint>
Anyhow ...
<beneroth>
bjorkintosh, abu[7] yeah WASM might be an idea (if WASM wouldn't be such a bad idea in itself...)
<freemint>
beneroth, how do you deal in picolisp, if your +Entities in the database are changed?
<freemint>
E.g. the structure and way it is indexed is changed.
<beneroth>
I make a new +Entity subclass and use (set> . +Entity) to change the object class?
<freemint>
What is the new entitey is incompatible how the data is currently stored?
<beneroth>
freemint, the Julia stuff sounds a lot like C++ templating. Though I doubt that it is as powerful
<abu[7]>
I have that often
<abu[7]>
run a fix over the DB
<abu[7]>
on Sunday :)
<freemint>
It is more powerful then C++ templating. It is extrodinarily poweful.
<beneroth>
freemint, (set> . +Entity) goes through all properties, if they are existing (property/relation with same name) in the new class definition, they get read from old definition and stored in new definition (possibly updating/creating index entries))
<beneroth>
freemint, how can it be more powerful than C++ templating? C++ templates are turing complete xD
<beneroth>
ok it can be more powerful in away, as Julia is interpreted not compiled?
<freemint>
It is more concise and the templating can occour at run time, vs at compile time in C++.
<beneroth>
ah it does JIT
<freemint>
Julia behaves like an interpreter but every function call is compiled like in an traditional compiled language.
<beneroth>
so it's just like a C++ programm that calls a compiler sometimes :P
<beneroth>
that's not more powerful
<abu[7]>
or FORTH
<abu[7]>
or the Lisp reader
<beneroth>
FORTH is an interpreter without JIT, without any compilation, like lisp, right?
<abu[7]>
It "compiles" immediately
<freemint>
it depends on the forth.
<abu[7]>
to a threaded structure
<abu[7]>
freemint, yes
<beneroth>
I see, thanks for the explanation freemint and abu[7]
<abu[7]>
direct or indirect threaded, or compiled to native
<freemint>
Well it is more succinct too. Things which are not expressable without writing a bug ridden, underspecified implementation of half of Julia in C++ are easy in Julia.
<beneroth>
why not get rid of the JIT? does it produce multi-threaded execution?
<freemint>
It produces multi-threaded execution and comes with green threads too. The "just ahead of time" (as Julia call it) allows Julia to allow of code reuse and specialization, where otherwise compiling all variants ahead of time would lead to a huge combinatorical explosion of specialized methods. Julia lazy decides what to compile "ahead of
<freemint>
time"-style.
<freemint>
*lazily.
<beneroth>
yeah JIT is just lazy-compilation
<freemint>
Mhh modern JITs also do profile guided reoptimization and so on. Julia doesn't.
<beneroth>
how lazy
<bjorkintosh>
one would argue that (to use the current fad), JITs and AOTs are artificially intelligent!
<freemint>
This is a detail not exposed to users though, but it makes a difference in a bunch of places.
rob_w has quit [Remote host closed the connection]
<beneroth>
bjorkintosh, no, it's all by natural intelligence design, no?
<beneroth>
freemint, yeah don't give any control to users
<beneroth>
is Julia multi-threaded at all? implicitly or explicitly? because green threads is not multi-threading. it's just concurrency aka event handling.
<freemint>
beneroth When a function call is to be executed, Julia looks whether it has compiled this method with these types before and if so, calls that methods and if not more or less recursively inlines and compiles all code with as specialized typing as possible.
<beneroth>
ok
<beneroth>
so runtime performance can vary a lot?
<beneroth>
in unexpected ways
<freemint>
First call is slow, all later calls are fast.
<beneroth>
per function
<beneroth>
so depends on what is getting called
<freemint>
it is possible to precompile certain function + arguement typings while installing a package.
<freemint>
Julia is multi-threaded, you can start a new task(green thread) on a seperate thread. Task(green threads) can be stolen between threads (run on different theads then were created). Julia also supports some implicit things, like arrays which multi thread (or even execute on GPU) array operations.
<freemint>
I would say that it is implicit multi-threading.
<freemint>
And explicit green threading.
<freemint>
With some libraries which do explicit multi-threading in a native feeling way.
<freemint>
Assuming this definition. "If parallelism is managed internally by ROOT, we talk about implicit multi-threading. On the other hand, if the application itself is in charge of creating threads and distributing work among them, the multi-threading is explicit."
<freemint>
Anyhow ... beneroth can you elaborate in more detail what (set> . +Entity) does and how one could update if the way content is encoded differs?
<abu[7]>
set> simply sets a new class to an object
<beneroth>
freemint, in pilDB you usually implement records by having one (or multiple) classes which inherit by +Entity, right?
<freemint>
*nods*
<abu[7]>
So it depends on the old and new class
<beneroth>
so e.g. (class +A +Entity) with some (rel ...) definitions and (class +B +Entity)
<beneroth>
if +A and +B have differing relation declarations, the properties and indexing of Obj gets updated accordingly
<freemint>
*nods*
<beneroth>
if +A declares relations not existing in the +B declaration, they get removed from Obj
<beneroth>
(but generally you switch from types to types which have additional relations)
<beneroth>
that's all :)
<freemint>
Okay, mhh.
<abu[7]>
If you just *add* a new relation, nothing has to be done
<beneroth>
+A and +B might declare the same relations and similar indexes, but stored in separate index BTrees
<beneroth>
abu[7], T
<beneroth>
same if +B is a subclass of +A, then it might change nothing but bring additional relations. or even just changed methods.
<freemint>
If am kinda thinking how to handle if the user code reloads the database with a new version of code where the struct changed created by previous version of code.
freemint has quit [Quit: Client closed]
<beneroth>
(set>) is changing the type. Type in picolisp is a list of classes. Not necessarily just a single class. So could also be from '(+A) to '(+PrefixBar +PrefixFoo +A)
freemint has joined #picolisp
<freemint>
(back)
<beneroth>
(set>) is changing the type. Type in picolisp is a list of classes. Not necessarily just a single class. So could also be from '(+A) to '(+PrefixBar +PrefixFoo +A)
<freemint>
*nodsÜ
<abu[7]>
PilBox has this problem
<abu[7]>
at start it checks the version
<beneroth>
not really a problem
<beneroth>
abu[7], so you do a full restart?
<abu[7]>
and runs updates of DB if needed
<abu[7]>
Not usually
<beneroth>
ah I see. yeah, it executes pending migrations
<abu[7]>
jus fix what is needed
<beneroth>
I think freeming means schema definition changes, not the data changes
<abu[7]>
yss
<abu[7]>
DB model
<abu[7]>
say an index changed
<beneroth>
freemint, standard way: restart process with new definition / new source code of (class ... +Entity) (rel ...) definitions
<abu[7]>
from +Ref to +IdxFold
<beneroth>
dynamic way: modify and update the so-called "daemon object"
<abu[7]>
then rel> all objects
<beneroth>
abu[7], I actually implemented that for my current project where I allow JSON API clients to add custom fields to database entities.
<freemint>
Or data changes which change how data is stored. E.g. You changed what way you encode country codes in strings.
<abu[7]>
yeah
<abu[7]>
*any* change
<beneroth>
freemint, well that needs migration. so either run a one-off job to clean up the whole database, or have the new encoding format stored in a new property, and upon reading, create if from the old representation if needed. or whatever.
<freemint>
If the way you serialized the Entity using the old version and upon deserialisation will lead to a an object which behaves differently then intended.
<abu[7]>
The point is to do it at *start*
<abu[7]>
you have new code
<freemint>
*nods*
<abu[7]>
check the version and change the DB as needed
<abu[7]>
one or more updates
<abu[7]>
then start user mode
<beneroth>
it needs migration. how you do that migration is up to you, there are many many ways.
<abu[7]>
T, depends
<beneroth>
one-time, or per object slowly over time, or upon reading time, or upon writing time...
<freemint>
The issue is that in Julia you can re-use a lot of code from other packages. I do want to minimize the maintainance burden that imposes.
<freemint>
(code and datastructures)
<beneroth>
yeah that you can only do if you have some reliably constraints on all the packages complying to your system
<freemint>
*nods*
<freemint>
That should be possible, i wonder if there is a way i can make migrations easier though.
<freemint>
Anyhow, thanks so far.
<abu[7]>
np
<beneroth>
I've experimented with an additional DSL layer on top of the normal pilDB schema declaration which in turns then produces versioned Entity classes and versioned daemon objects. Works well, but of course you then also need implementation of (tree), (iter), (db), (collect) and pilog to be able to handle the versioned entities
<freemint>
*nods*
<freemint>
If you the choice between versioning being supported by default, or not, what would you prefer?
<beneroth>
depends on the use case :)
<abu[7]>
You have better control manually
<beneroth>
T
<beneroth>
absolutely
<freemint>
Should the implementation support/leave room for versioning or not?
<abu[7]>
A simple versioning is needed
<beneroth>
I've never came around to make GUI editors for the versioned system, so manual updates where in most cases more work than without the additional versioning system. The versioning system was useful for cases where I wanted to keep old data in old versions and only make newer data/records comply to the new schema version.
<abu[7]>
unless it is a single server
<beneroth>
abu[7], yeah but source code versioning is good enough for that
<beneroth>
(git or pil snapshot or whatever)
<abu[7]>
I meant DB model versions
<freemint>
*nods* So i should leave room for a version tag in my db scheme?
<beneroth>
just use a naming convention for it
<beneroth>
that's simpler
<freemint>
*nods*
<abu[7]>
I use a file
<abu[7]>
"version"
<abu[7]>
in the DB dir
<beneroth>
yes that also makes sense
<abu[7]>
"V
<beneroth>
it is configuration, it is not app data
<abu[7]>
PilBox has "Version", and "version" in each app dir
<freemint>
Sorry for the dumb question, but can you store "non-entity" objects in a picolisp db?
<abu[7]>
yes
<abu[7]>
+Blob
<freemint>
*nods*
<abu[7]>
or just external syms with no class
<abu[7]>
e.g. b-tree nodes are so
<freemint>
Mhh, so when i add a new +Entity or a different blob encoding of a non entity i used far i need to add that to the versioning db.
<freemint>
*file
<abu[7]>
Depends
<abu[7]>
if you just add something no need to worry
<beneroth>
adding new stuff doesn't make necessary to migrate existing data
<abu[7]>
I make a new version with each release, even if the DB did not change
<beneroth>
only when existing data needs to be re-written or re-indexed a migration is necessary
<abu[7]>
T
<freemint>
Mhhh, if i add new field and code now assumes that field is initialized, i still need to rewrite things do i not?
<abu[7]>
Initialization can be dons
corecheckno has joined #picolisp
<abu[7]>
done
<abu[7]>
But a new field is usually NIL
<abu[7]>
so no change
<freemint>
That assumes the object definition tells what inital value to expect.
<beneroth>
if the initialization value is always the same, then its much more efficient just to use (default) when reading (so you don't need to store the default value)
<abu[7]>
in Pil NIL properties simply don't exist
<freemint>
*nods*
<abu[7]>
I think I never had a case where a new property needed an initialization
<freemint>
I guess i will not get away with that in Julia. Different expectations from the user and language.
<abu[7]>
You need to init with "", null or 0?
<freemint>
Well in Julia i have more types to worry about.
<bjorkintosh>
freemint: what do you use Julia for? do you do a lot of heavy duty computations?
<abu[7]>
ok
<freemint>
i think i can figure that out though.
<freemint>
doing nummerical stuff.
<abu[7]>
In Pil everything can be NIL, also numeric fields
<bjorkintosh>
is that a euphemism for linear algebra AI?
<freemint>
No?
<abu[7]>
NIhiUism
<abu[7]>
NIhiLism
<abu[7]>
L and U are close in Steno ;)
<freemint>
So in summary, idea seems feasible but it will need to do some work to adapt things how Julia works, this will require imposing some structuring not present in the picolisp db.
<freemint>
a name which has pil in it is fine.
<abu[7]>
Let's think
<abu[7]>
Sorry, must go
<abu[7]>
bbk
<abu[7]>
bbl
<beneroth>
freemint, yeah as I said, you likely want to add more datatypes. You either model them on top of existing picolisp types, or you lose the power and flexibility of picolisp. that's how it is.
<freemint>
I lose flexibility on those more specialized data types.
<freemint>
I might also go for pilidb where pili is the plural of some cellular structure which is involved in bacteria sticking to surfaces and horizontal gene transfer (== data code)
<beneroth>
why not mintDB, with your name in it? :)
<freemint>
seems vain, also, not having fun meanings.
<beneroth>
pilimintDB
<bjorkintosh>
freemint: or as I suggested, piDB. there's a handy greek symbol for the first two letters.
<beneroth>
piliDB is too easily mixed up with pilDB
<freemint>
Also mintDB is already taken in Rust.
<freemint>
Mhh DBPili then.
<beneroth>
bjorkintosh, we have unicode! no reason to write it differently. well pronunciation, but who cares about that. πDB
<bjorkintosh>
voila.
<bjorkintosh>
pronounced "pee dub"
<bjorkintosh>
sorry. "pea dub"
<beneroth>
xD
<beneroth>
or "pee-D-Bee"
<bjorkintosh>
yes. "peed bee"
<bjorkintosh>
work in progress.
<beneroth>
with a bee with an π on the body as mascot
<bjorkintosh>
voila.
<beneroth>
much smaller than the postgres elephant
<beneroth>
more dynamic
<bjorkintosh>
that Slonik can dance.
<bjorkintosh>
name a style, it can do it.
<freemint>
beneroth (rel nm (+Sn +IdxFold +String)) can be thought of as a soundex of a index fold of a string. Can you think of a case where you needed multiple inheritance and the additional classes did not build on each other like a tree (with one root).
<freemint>
(sorry if that question is strangely phrased.
<freemint>
(My motivation is: In Julia you can have parametric types (Stack{Matrix{Int}}) but not inherent from multiple sources. I am trying to see if that that is a concern.)
<abu[7]>
ret
<abu[7]>
freemint: "additional classes did not build on each other" isn't that often the case with simple prefix classes?
<freemint>
*nods*
<abu[7]>
eg. in the pil gui are classes like +Set, +Val, +Fmt, +Chg
<freemint>
Yeah, let's try that phrasing, case where multiple inheritance wasn't just from a singe prefix class and all the things it or it's prefixed classes prefixed?
<abu[7]>
Where do you need that phrase?
<abu[7]>
ako doc?
<freemint>
i need that phrase in asking my question and seeing if the examples which are involving prefix classes can't be mapped into Julias typing system
<abu[7]>
ah
<freemint>
Basically, Julia has single inheritance but functions are multiple dispatch.
<abu[7]>
You could roll your own inheritance
<freemint>
prefix classes are easily implementable in Julias typing. but i am not sure everything multi inheritance offers can be well expressed that way. Since this a search space noone thought much about before, i am asking for examples where multiple inheritance was useful.
<abu[7]>
The cases in the Pil E/R or GUI are not good examples for your case?
<beneroth>
freemint, I also made classes inherit from two +Entity subclasses
<abu[7]>
Or, simpler, the chess program? (+White +Queen), (+Black +Pawn)
<beneroth>
afk
<abu[7]>
cu :)
<abu[7]>
I must head for my Stammtisch now
<abu[7]>
Still here, but sparsely
z4k4ri4 has quit [Ping timeout: 252 seconds]
z4k4ri4 has joined #picolisp
<beneroth>
back
corecheckno has quit [Remote host closed the connection]
corecheckno has joined #picolisp
freemint has quit [Quit: Client closed]
corecheckno has quit [Remote host closed the connection]