beneroth changed the topic of #picolisp to: PicoLisp language | The scalpel of software development | Channel Log: https://libera.irclog.whitequark.org/picolisp | Check www.picolisp.com for more information
<abu[7]> Moin
<abu[7]> Now looking at the _foo issue
<abu[7]> I cannot rename _foo to PIL_foo or so. The leading "_" has a special meaning in the compiler in @src/lib/llvm.l
<abu[7]> I think I'll go with _Foo
<abu[7]> upper case should be safe
<abu[7]> Done!
<abu[7]> Builds
<abu[7]> @lib/test.l passes
<abu[7]> Doing some more tests
<abu[7]> Looks good
<abu[7]> I made a simi-official release. Can be downloaded from https://picolisp.com/pub/pil21.tgz
<abu[7]> tankf33der: If you have time, can you run your big test suite?
<abu[7]> For example, I have a test case
<abu[7]> (%@ "_prog1" T '(T prog1 7 (println 1 2 3)))
<abu[7]> This breaks now
<abu[7]> must be (%@ "_Prog1" T ...
<abu[7]> Searching for more such cases
<tankf33der> testing on linux
<abu[7]> 👍
<tankf33der> [native.l:1] !? (%@ "_cons" T '(T cons (+ 3 4) (* 3 4)))
<tankf33der> Bad ffi
<tankf33der> ?
<abu[7]> yeah
<abu[7]> see above
<abu[7]> Must be "_Cons" now
<tankf33der> all passed.
<tankf33der> i can test freebsd in several hours.
<abu[7]> Cool!
Iacob has quit [*.net *.split]
<tankf33der> will you wait ?
<abu[7]> No hurry
<abu[7]> I will release officially when we believe all is good
<tankf33der> what tests would help?
<tankf33der> macos, openbsd?
<tankf33der> i believe all linuxes will be fine
<abu[7]> Yes, and Mac etc. too
<abu[7]> I'm more worried that some existing programs may break
<abu[7]> E.g. if the rewriting of the sources has errors
<abu[7]> or like cases above "_cons" vs. "_Cons"
<abu[7]> I think the "_cons" test is rather special
<abu[7]> Nobody did such things I believe ;)
<tankf33der> T
<tankf33der> On non linuxes tests does not working
<tankf33der> only compilation
<abu[7]> I see
<abu[7]> System-dependent tests fail?
<gahr> morning abu[7]
<gahr> let me test your semi-official tarball
<abu[7]> Good morning gahr
<abu[7]> good
<abu[7]> tankf33der, does the @test/ stuff also fail on non-Linuxes?
<gahr> it starts fine
<gahr> is there any test suite I should be running?
<abu[7]> yes, @test/
<abu[7]> $ ./pil lib/test.l -bye +
<abu[7]> Takes a few seconds, then prints OK
<abu[7]> or stops at an error
<gahr> it did neither :)
<abu[7]> just hanging?
<gahr> no, it finishes and exits with 0
<gahr> but it doesn't print OK
<abu[7]> Never saw that
<gahr> I needed to tweak pil not to point to /usr/...
<abu[7]> How do you mean that?
<abu[7]> symbolic link?
Iacob has joined #picolisp
<tankf33der> macos ok
<gahr> #!./bin/picolisp ./lib.l etc
<gahr> instead of /usr/...
<gahr> I haven't created symlinks in the system dir yet
<abu[7]> This is not necessary
<gahr> just trying it out in the untarballed directory
<abu[7]> yes, just do everything local
<gahr> well ./bin/pil doesn't work with everything local
<abu[7]> Pil uses no absolute pathes
<gahr> #!/usr/bin/picolisp /usr/lib/picolisp/lib.l
<abu[7]> no
<gahr> this is the shebang of bin/pil
<abu[7]> not bin/pil
<abu[7]> bin/bil is for a global installation
<abu[7]> Just do
<abu[7]> $ ./pil +
<abu[7]> this is the official standard call
<abu[7]> or, if you are somewhere else:
<abu[7]> $ ../../foo/bar/pil +
<abu[7]> 'pil' does everything relative to the invocation path
<gahr> oh
<gahr> I mistook ./pil for ./bin/pil :)
<abu[7]> $ ./pil lib/test.l -bye +
<gahr> > ./pil lib/test.l -bye +
<gahr> OK
<gahr> \o/
<abu[7]> :)
<gahr> hopefully _[A-Z] won't clash with any implementation symbol
<abu[7]> This is the core principle of pil invocation
<gahr> still, it's a bit waiting for a problem to happen, no?
<abu[7]> you can do ../foo/pil @lib/test.l -bye +
<gahr> yep saw that
<abu[7]> I think standard libs do not use upper case names
<abu[7]> It is a lot less dangerous than the old _[a-z]
<abu[7]> What I still don't understand:
<abu[7]> libc on FreeBSD has both?
<gahr> yes I agree it's a lot less dangerous
<abu[7]> _open AND open ?
<gahr> ok let me double check why we have a _open exposed
<abu[7]> ok
<tankf33der> Linux has __libc-open
<abu[7]> ok
<tankf33der> freebsd ok
<abu[7]> Great!
<abu[7]> Also your full tests?
<tankf33der> i did a special test suite without native, io and net
<abu[7]> ah, ok
<gahr> I think _open is our syscall
<gahr> for open
<abu[7]> I understand. The official entry point is 'open', but the direct system call is public too
<abu[7]> I'm very glad we found that. Much better now.
<abu[7]> The main question is now if anything existing may break
<abu[7]> quit improbable I think
<gahr> I'm not completely sure what's going on..
<abu[7]> "normal" programs don't use the above direct native calls
<gahr> https://cgit.freebsd.org/src/tree/lib/libc/sys/Symbol.map#n811 this is _open in the private section
<gahr> but I don't understand the meaning of private
<abu[7]> Seeems not really "private"
<gahr> yeah
<gahr> I'm not familiar with how our symbol maps work
<tankf33der> Release and make announce in ml
<abu[7]> Yeah, let's everybody test it
<beneroth> thanks guys
<abu[7]> Hi beneroth
<gahr> I was looking at the source code, and was wondering why some functions use "X" for argument names instead of X
<gahr> was that an older convention?
<beneroth> no, "X" is a transient symbol
<abu[7]> No, this is still actual
<beneroth> meaning it's scoped to the file
<abu[7]> It is not really a naming convention, but means "transient" symbols
<abu[7]> Usually you can use (private) symbols now for those
<abu[7]> Transients are automatically private in the current source file
<gahr> because symbols are dynamically scoped?
<abu[7]> Not scoped
<beneroth> binded
<abu[7]> yes they are "bound"
<beneroth> transients for parameters are mainly used in F-Expressions which don't receive their arguments evaluated but evaluate them themselves (or not, whatever they like to do). Using transient "X" instead of normal global X helps in case the code the fexpr receives contains X which could give name collision
<abu[7]> The scope of a symbol is something different
<beneroth> maybe we can say "scope of the name" ?
<gahr> right
<abu[7]> yes, the name
<abu[7]> "scope" means "visibility"
<beneroth> the name in source code points to a symbol, evaluated during reading.
<gahr> (de bar () (prinl X) ) (de foo (X) (bar X) ) (foo "hello")
<gahr> I guess I am a bit puzzled that bar prints hello :)
<gahr> I mean, I'm unused to this
<beneroth> well think through what happens, step by step
<beneroth> (de ...) defines the functions (binds the value of the symbol to the list which makes the function definition)
<beneroth> now during call (foo "hello")
<beneroth> first the (X) from (de foo (X)) is getting bound to your "hello" symbol (also a transient symbol)
<abu[7]> 'X' is called a "free" variable in 'bar'
<gahr> I can think of symbols as parameters in scheme.. or dynamic variables in lisp
<beneroth> "hello" is evaluated, but as it has no value, transients symbols have the special mechanic of evaluating to themselves.
<beneroth> then we call 'bar with X bound to "hello"
<gahr> I mean, all symbols are dynamic at global scope (or file scope for transient)
<gahr> yep the evaluation is clear :)
<beneroth> you must differ between name scoping during reading a file/repl code, and bindings during evaluaitng
<abu[7]> dynamic refers only to binding
<beneroth> calling (bar) is like (let (X ...) ..) environment
<beneroth> the X gets locally bound, points to a new value, independent of the value it has in the upper environment
<beneroth> all symbols which don't get a local binding, keep their bindings from the upper environment - that's the dynamic binding
<gahr> yes
<abu[7]> "all symbols are dynamic" this is correct if you mean "all symbols are dynamically bound"
<gahr> yes
<abu[7]> there is no lexical binding in pil
<beneroth> and datatype is an attribute of the value, not the variable, in picolisp
<beneroth> (variables in pil are actually always symbols)
<gahr> so how do you pick a variable name for you variable locals?
<beneroth> so in the (bar X) call, with X being already bound to "hello" from (foo "hello"), the X gets again bound to "hello". Then (prinl X) gets called, receiving "hello", evaluating "hello" which gives "hello", and writes that to stdout
<gahr> do you need to know every free variable used in everuy function you call?
<abu[7]> Free variables are usually avoided
<beneroth> easy rule: uppercase symbols, and when it's a FEXPR make it a transient or namespaced
<abu[7]> The convention is that functions having free variables start with "_"
<abu[7]> (de _foo () (bar X))
<gahr> ok, so if free variables are avoided, what's the benefit of using a transient symbol for function parameters?
<abu[7]> 'lint' will otherwise warn
<beneroth> if you use variables in a function which were not handed over as parameters, they should be globales *Uppercase or the function prefixed with _ to indicate that behaviour to the programmer
<abu[7]> The transients are for other purpose
<abu[7]> To avoid shadowing
<gahr> sure but what would you shadow, if not a free variable?
<beneroth> transient = during reading (in repl step), only search for definitions for that name in the current file scope, not global scope
<beneroth> ah good thinking
<beneroth> T in a fexpr you get usualyl a lot of free variables, because you are evaluating code from an upper environment usually
<beneroth> *usually
<abu[7]> There is a FAQ entry: Are there no problems caused by dynamic binding?
<beneroth> you got started with this by the (de tag "X" ...) definition in @lib/xhtm.l ?
<gahr> still you don't know if the expr you pass unevaluated contains any transient symbol
<beneroth> it could contian transient symbols, but those will be scoped to the reading environment when they where loaded
<gahr> not specifically, I was just seeing that some functions used transient symbols as params, and some did not
<beneroth> so if they came from a different file, it's a different scope
<beneroth> if its the same file, the names will be bound to the same symbols
<abu[7]> gahr, it is always for the two reasons
<abu[7]> A transient or private won't conflict
<abu[7]> (de myPush (Var Val) (set Var (cons Val (val Var]
<abu[7]> (push 'L 7)
<abu[7]> BUT: (push 'Var 7) won't work
<abu[7]> At "(set Var" Var *is* Var
<abu[7]> So the value is unbound when 'myPush' returns
<abu[7]> thus (de myPush ("Var" Val) ...
<gahr> but you can't (myPush "Var" 7)
<abu[7]> This is safe, as "Var" never can be passed
<abu[7]> You can
<abu[7]> "Var" is another symbol
<gahr> even if it's in the same file?
<abu[7]> No
<abu[7]> If you use "Var" in the same file in a call to myPush
<abu[7]> you must do (====) to cut off the scope
<abu[7]> or now better use (private)
<abu[7]> (private) Var
<gahr> right I see
<gahr> so it's probably best to use a transient symbol for an argument for which you expect a symbol
<abu[7]> (private) is more flexible and more readable
<gahr> and you must be careful how you call it within the same file
<abu[7]> Just expecting a symbol is not a problem
<abu[7]> Only if the value of that symbol is modified
<abu[7]> e.g. 'set' like above
<gahr> yep that makes sense
<beneroth> gahr> so it's probably best to use a transient symbol for an argument for which you expect a symbol
<beneroth> yup
<gahr> I guess this needs a bit of getting used to :)
<gahr> it's a mindset shift
<beneroth> everything does
<beneroth> but yeah
<beneroth> pil looks like other languages, but quite different things happen
<beneroth> VM with cells linking to cells linking to cells
<beneroth> instead of translation into words and arrays being copied around
<gahr> yep, the concepts are very simple, which is what attracts me, but the consequences might be deep
<gahr> needs some thinking..
<abu[7]> In praxis it is no problem, as long as one keeps the two cases in mind
<tankf33der> abu[7]: will you take a look why native does not work on macos or *bsd ?
<tankf33der> or will wait until first request?
<abu[7]> This was always not working, right? Not just with today's change?
theruran has quit [Quit: Connection closed for inactivity]
<tankf33der> Always, from beggining
<abu[7]> Would be nice to find the reason
<tankf33der> lets debug then on freebsd
<tankf33der> did you released todays modifications?
<gahr> can I help with some freebsd-foo?
<abu[7]> tankf33der, I did not release yet
<abu[7]> only the above pub/
<abu[7]> gahr, yes, nice
<abu[7]> gahr, to the bindings issue, a simple example:
<abu[7]> : (de foo (Fun) (mapcar Fun (1 2 3)))
<abu[7]> then
<abu[7]> (let X 2 (foo '((N) (+ X N))))
<abu[7]> -> (3 4 5)
<abu[7]> All right
<abu[7]> But:
<abu[7]> (let Fun 7 (foo '((N) (+ Fun N))))
<abu[7]> it gives "((N) (+ Fun N)) -- Number expected"
<abu[7]> in 'foo', Fun should be private or transient
<gahr> yep
<abu[7]> So sources lik @lib/xhtml, @lib/svg.l make heavy use of privates
<gahr> that's point 1 in the Are there problems.. faq right?
<abu[7]> Yes
<gahr> so whenever your argument is not an atom, i.e., it can be expanded (e.g., by evaluation for its value or by list execution), it's better not to use an internal symbol
<gahr> probably atom is not a pil terminology, but you get the point
<abu[7]> only if it is passed throug a binding
<abu[7]> 'foo' binds 'Fun'
<gahr> yes
<abu[7]> Atom or list does not matter
<abu[7]> The issue is that 'Fun' may overshadow another binding of 'Fun'
<abu[7]> This is only if an executable list is passed
<abu[7]> a function, an exe or a prg
<abu[7]> exe and prg are used in the references
<abu[7]> prg is a list of exe's
<abu[7]> and a function is a parameter list and a prg
<abu[7]> (params . prg) is a function
<abu[7]> ((X Y Z) (foo X) (bar Y Z))
<abu[7]> (foo X) is an exe
<abu[7]> ((foo X) (bar Y Z)) is a prg
<abu[7]> The concepts are technically simple, but it is hard initially to get one's head around it ;)
<abu[7]> I also have to rethink it sometimes
<abu[7]> But it is sufficient to just keep the two rules in mind
<abu[7]> Ok, let't tackle 'native' calls!
<abu[7]> What error happens?
<abu[7]> Does it crash on BSD's?
<gahr> what should I test?
<gahr> ? (native "libcrypto.so" "SHA1" '(B . 20) "abcd" 4 0)
<gahr> -> (129 254 139 254 135 87 108 62 203 34 66 111 142 87 132 115 130 145 122 207)
<gahr> took this from the example
<tankf33der> powering freebsd, wait
<gahr> I mean, the libcrypto example works fine on my system, and I get the same result as the example in the docs
<abu[7]> good
<gahr> (right, atom or list doesn't matter, as long as you don't execute it)
<abu[7]> MacOS was always a bit special
<abu[7]> right
<abu[7]> and only a list may contain references to shadowed bindings
<abu[7]> So a built-in (an atom (a number)) is never a problem
<abu[7]> The typical cases in the sources are "Prg" bodies (in FEXPRs as beneroth mentioned)
<abu[7]> @lib/form.l re-privatizes 'Prg' several times
<gahr> that makes sense, yep
<gahr> anything else I should try for (native) ?
calle has joined #picolisp
<abu[7]> Let's see what cases tankf33der has
<abu[7]> It may well be that FreeBSD is all right, and only OpenBSD (and thus MacOS) have problems
<gahr> does (native) do memoization?
<tankf33der> # pil native.l
<tankf33der> ?
<tankf33der> [native.l:1] !? (%@ "_Cons" T '(T cons (+ 3 4) (* 3 4)))
<tankf33der> Bad ffi
<gahr> (forget about my question)
<abu[7]> gahr, yes, pointers are kept in the values of lib and fon after the first call
<gahr> ok that makes sense
<gahr> (%@ "_Cons" T '(T cons (+ 3 4) (* 3 4)))
<gahr> -> (7 . 12)
<tankf33der> gahr: are you on freebsd ?
<abu[7]> "Bad ffi" is after 'ffiPrep'
<gahr> tankf33der: yes
<tankf33der> why it works on your side then? :/
<gahr> 13.2-RELEASE amd64
<tankf33der> # uname -a
<tankf33der> FreeBSD freebsd 13.2-RELEASE-p2 FreeBSD 13.2-RELEASE-p2 GENERIC amd64
<abu[7]> 'ffiPrep' is in @src/lib.c
<gahr> with today's pub/ tarball
<tankf33der> me to
<tankf33der> me too
<abu[7]> strange
<tankf33der> gahr: how you compile pil21? it is not compile out of the box
<gahr> I have llvm15 installed, so I needed to append 15 to the llvm tools in Makefile
<gahr> and prepend -L/usr/local/lib to MAIN
<gahr> and use gmake instead of make
<tankf33der> http://ix.io/4CQH
<tankf33der> my custom makefile for freebsd
<tankf33der> should be fine
<gahr> let me try yours
calle has quit [Ping timeout: 260 seconds]
<tankf33der> http://ix.io/4CQI
<tankf33der> show me this output
<gahr> where's llc on your system?
<tankf33der> llc?
<gahr> yeah
<gahr> oh, you have llvm intsalled
<gahr> which depends on llvm15, but you get all the tools without the 15 suffix
<tankf33der> alpine:~ $ which llc
<tankf33der> alpine:~ $ llvm-config --version
<tankf33der> 16.0.6
<tankf33der> alpine:~ $ which llc
<tankf33der> usr/bin/llc
<tankf33der> everything in the PATH
<gahr> yep
<gahr> ok, I get bad ffi now
<tankf33der> hm
<gahr> but I also get warnings at compile time
<tankf33der> ignore
<gahr> can you try to build with the usual Makefile?
<gahr> you just need to -L/usr/local/lib in MAIN
<tankf33der> compiled
<tankf33der> [root@freebsd ~/pil21]# ./pil @lib/test.l +
<tankf33der> OK
<tankf33der> :
<tankf33der> wow
<tankf33der> how it could be?!
<gahr> -relocation-model=pic is missing from your makefile
<tankf33der> i do not remember why
<gahr> when it's making picolisp.s
<gahr> and -rdynamic is missing when it's linking picolisp
<gahr> uhm
<gahr> no idea
<tankf33der> abu[7]: freebsd issue closed
<gahr> if I remove -relocation-model=pic from picolisp.s line I get bad FFI
<tankf33der> my bad
<gahr> cool :)
<tankf33der> installing openbsd
clacke has quit [Ping timeout: 240 seconds]
<tankf33der> openbsd hang on tests
<tankf33der> openbsd# ../pil @lib/test.l +
<tankf33der> ((let P (port 0 "ListenPort") (unless (fork) (close P) (until (connect "localhost" "ListenPort") (wait 80)) (out @ (pr '(a b c))) (bye)) (prog1 (in (listen P) (rd)) (close P))))
<tankf33der> ?
<tankf33der> [../test/src/net.l:4] (a b c) -- 'test' failed
<tankf33der> http://ix.io/4CQY
<tankf33der> and hang here
<tankf33der> everything rest is passed
gahr has quit [*.net *.split]
Abhishek_ has quit [*.net *.split]
<abu[7]> So network stuff fails
Abhishek_ has joined #picolisp
gahr has joined #picolisp
<abu[7]> I think I can release pil21 now, and send a mail to the list
<tankf33der> +1
<abu[7]> Done
<gahr> abu[7]: is there a place where you keep release tarball that isn't overwritten by new releases?
<gahr> I thinking of making a port of picolisp for FreeBSD, and I need an upstream that stays stable
<abu[7]> The version at every three months is kept
<abu[7]> Next will be 23.9.tgz
<abu[7]> or you take tankf33der's repo
<abu[7]> or https://github.com/picolisp/pil21 (Mansur Mamkin) or https://github.com/picolisp (Alex Williams)
<gahr> thanks!
<tankf33der> gahr: use tgz from software-lab
<gahr> I will
aw- has quit [Quit: Leaving.]
aw- has joined #picolisp
aw- has quit [Quit: Leaving.]
aw- has joined #picolisp
theruran has joined #picolisp
hrberg has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
hrberg has joined #picolisp
<tankf33der> fixed macos compilation
<abu[7]> Thanks! I talked with Heiko Henrich (new member of the mailing list) today, he also succeeded on his Mac finally
<abu[7]> Do we have many variants of Makefile now? Schould we cosolidate them somehow?
<tankf33der> i do not know, i have separate repo for macos and INSTALL-.md in pil21-tests repo
<abu[7]> OK, better keep it separate
<abu[7]> I hate over-complicated Makefiles ;)
<tankf33der> now ext:Snx "mike" is working on macos
<tankf33der> $ pil ./native.l
<tankf33der> [./native.l:95] !? (native "libcrypto.so" "SHA1" '(B . 20) "abcd" 4 0)
<tankf33der> ?
<tankf33der> "libcrypto.so" -- [DLL] dlopen(libcrypto.so, 9): image not found
<tankf33der> googling
<tankf33der> - (native "libcrypto.so" "SHA1" '(B . 20) "abcd" 4 0) )
<tankf33der> + (native "libcrypto.3.dylib" "SHA1" '(B . 20) "abcd" 4 0) )
<tankf33der> native is fully working on macos
<tankf33der> afk.
<beneroth> great, thanks tankf33der and abu[7] :)
calle has joined #picolisp
calle has quit [Ping timeout: 260 seconds]
<beneroth> [security] it seems Intel got scary to lose the lead on fundamental CPU security falws after zenbleed on AMD ... but Intel keeps failing: https://downfall.page/
calle has joined #picolisp
<beneroth> s/scary/scared
<beneroth> s/falws/flaws
* beneroth has english broken
calle has quit [Ping timeout: 246 seconds]
aw- has quit [Ping timeout: 260 seconds]
aw- has joined #picolisp
stultulo has joined #picolisp
f8l has quit [Ping timeout: 260 seconds]
stultulo has quit [Remote host closed the connection]
stultulo has joined #picolisp
oldf8l has joined #picolisp
stultulo has quit [Ping timeout: 244 seconds]
oldf8l has quit [Read error: Connection reset by peer]
oldf8l has joined #picolisp