nkatte has quit [Read error: Connection reset by peer]
Lord_of_Life has quit [Ping timeout: 255 seconds]
Lord_of_Life has joined #commonlisp
nkatte has joined #commonlisp
phantomics has quit [Ping timeout: 276 seconds]
random-nick has quit [Ping timeout: 276 seconds]
NicknameJohn has quit [Read error: Connection reset by peer]
waleee has quit [Ping timeout: 260 seconds]
skin has joined #commonlisp
kevingal has joined #commonlisp
kevingal_ has joined #commonlisp
nkatte has quit [Read error: Connection reset by peer]
dcb has quit [Quit: MSN Messenger v2.16.1]
dcb has joined #commonlisp
nkatte has joined #commonlisp
nkatte has quit [Read error: Connection reset by peer]
dcb has quit [Remote host closed the connection]
nkatte has joined #commonlisp
nkatte has quit [Read error: Connection reset by peer]
nkatte has joined #commonlisp
<jeosol>
good morning all!
nkatte has quit [Read error: Connection reset by peer]
<jeosol>
Following my recent posts regarding memory leak and restarting of server machines, I finally wrote a small code to capture (room) output after few iterations. For a small test code, dynamic space usage grew to 129MB linearly from around 85MB.
<jeosol>
Note this is a small test code. After I call (gc) dynamic space usage dropped to ~58MB. This is on SBCL. I have a question: I suspect for my longer running computation (2-3weeks) this memory probably increases in a similar fashion. I don't call (gc) at all. Do guys explictly call (gc) or that is code smell.
<edwlan[m]>
You shouldn’t need to
<edwlan[m]>
But you might need to tune some of the variables that affect GC for your workload
<edwlan[m]>
Also, sb-sprof lets you profile code to find what allocates the most
<edwlan[m]>
Maybe that would give you a hint about where to focus optimizations.
karlosz has quit [Quit: karlosz]
<jeosol>
edwlan[m]: thanks for that. I think I need to rethink a few design, but that is going to take a lot of code redesign. I stated other day, I am running some calculations for K iterations, and I create all necessary array for the K iterations. Ideally, I only need k-1 and k, at any time, can write results for k-1 to file before updating the counter.
<jeosol>
I was hoping this is not a major issue if I can at least assign all needed memory (initialized to zeros) and just will with correct values as the computation progresses
<edwlan[m]>
The other day I discovered a dumb coding mistake in one of my libraries with allocation profiling
<edwlan[m]>
I was calling local-time:reread-timezone-database in a tight loop
<jeosol>
edwlan[m]: yes, good point. I did also capture the breakdown of the cons objects, string-objects, vector-objects, for the dynamic space. I will add this code to the long running process to see which is the culprit. My guess is that I am generating a lot of arrays
<jeosol>
good point. It's possible, I have something that is running some errant code. It's a costly optimization for sure, but I have been able to run it on another machine in the past, but I have since update and modified the code.
<jeosol>
refactor is a pain, I am hoping I don't have to do that:-(
<edwlan[m]>
You might be surprised that the solution is really simple
<edwlan[m]>
As a rule, programmers are a bad judge of what needs to be optimized
nkatte has joined #commonlisp
<jeosol>
edwlan[m]: haha, I hope so, but the codebase is massive
<ixelp>
Gource visualization of a Common Lisp Application repository (last commit 5/27/2022) - Updated - YouTube
lucasta has quit [Quit: Leaving]
nkatte has quit [Read error: Connection reset by peer]
<jeosol>
edwlan[m]: but I do agree, I have to go at it like an onion, little by little. I think I have to go the direction of only save the previous and current states, more post-processing work later and less memory requirements
<jeosol>
thing is i generated some results with the code, but generating the rest has been a pain so far.
<jeosol>
edwlan[m]: i appreciate your input. I will add the 'room' utility and see what the trend looks like
nij- has joined #commonlisp
lucasta has joined #commonlisp
<nij->
(uiop:run-program "nc -zv localhost 1-9999" :output t :output-error t :ignore-error-status t)
<nij->
The above form doesn't print out anything.. But when I try the same command in my terminal, it shows open ports.
<nij->
Why so?
<edwlan[m]>
Does it print anything into the inferior-lisp buffer?
<jeosol>
good question, I limit what is printed to the "console" to just a status line
<jeosol>
not sure what you buffer here, but above I mean (format t "~..." ...)
neon has joined #commonlisp
<jeosol>
edwlan[m]: are in the US or EU?
<edwlan[m]>
US
<jeosol>
I will plumb this small utility and run it overnight so by tomorrow AM US time, I should have some results
<jeosol>
edwlan[m]: ok, cool, I am CST
nkatte has joined #commonlisp
nkatte has quit [Read error: Connection reset by peer]
<nij->
edwlan[m] Nope, nothing is printed into inf-lisp buffer.
<nij->
Is it a bug of uiop?!
nkatte has joined #commonlisp
<nij->
It did hang for a second. That means nc was run. It's just that nothing gets printed..
<edwlan[m]>
I forget if launch-program or run-program is the right one for this use
<edwlan[m]>
One of them is async and will not print to stdout
nkatte has quit [Read error: Connection reset by peer]
nij- has quit [Ping timeout: 276 seconds]
nkatte has joined #commonlisp
nkatte has quit [Read error: Connection reset by peer]
NotThatRPG has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
nkatte has quit [Read error: Connection reset by peer]
<jeosol>
_death::-) Thanks, I appreciate the pointers.
<jeosol>
I was thinking of something, hence me writing something capture (room) output, so if I can an API perhaps I can query it to track the metric.
<jeosol>
_death: bummer, haha, that prometheus.cl does essentially does what I am trying to do. I just wrote the small utility to capture (room) parameters. It is also does capture the breakdown of the different dynamic memory categories
<jeosol>
_death: you are the man, thanks a lot
tyson2 has quit [Remote host closed the connection]
<jeosol>
_death: I think I need to add it to code, and run it off a different port perhaps it can work. My previous approach was going to be "blind" as in, I run first, them dump memory-state to a file, and analyze file after a run
<_death>
well, with prometheus you can also set alerts on certain conditions
<_death>
so that and keeping an open swank server in case you need it may be a good idea
<jeosol>
that could be the next step. I did try to test on the cloud on gcp, that could be relevant when I test again. I had to get a big machine which could be costly. Now I am testing on local, to shake things out very well.
<jeosol>
I am trying to incorporate the prometheus.cl code, hope it's not bit rot, gettinng some CLOS class precedence errors, but still looking into it.
nkatte has joined #commonlisp
dcb has joined #commonlisp
dcb has quit [Client Quit]
dcb has joined #commonlisp
dcb has quit [Client Quit]
dcb has joined #commonlisp
nij- has joined #commonlisp
mariari has quit [Ping timeout: 256 seconds]
mariari has joined #commonlisp
nkatte has quit [Ping timeout: 276 seconds]
nkatte has joined #commonlisp
neon has quit [Quit: Sleep]
nkatte has quit [Read error: Connection reset by peer]
<nij->
edwlan[m] run-program is sync, and launch-program is async. So run-program should have printed..
<nij->
I'm very confused.. I must rely on something that will print whatever I will see in a terminal.
jeosol has quit [Ping timeout: 260 seconds]
jeosol has joined #commonlisp
<jeosol>
_death: still here?
nkatte has joined #commonlisp
<jeosol>
_death: I am not able to use prometheus.cl due a bug but will try a again later.
nkatte has quit [Read error: Connection reset by peer]
<jeosol>
The see-saw pattern is gc being called I supposed.
nij- has quit [Ping timeout: 264 seconds]
rtypo has quit [Ping timeout: 240 seconds]
Josh_2 has joined #commonlisp
lucasta has quit [Remote host closed the connection]
<Josh_2>
Hey :trumpet:
<Josh_2>
Can I redirect the output that I have to use my system tools like journalctl to read back into my sly repl?
semz has quit [Quit: ZNC 1.8.2+deb2build5 - https://znc.in]
semz has joined #commonlisp
semz has quit [Remote host closed the connection]
semz has joined #commonlisp
ebrasca has joined #commonlisp
nkatte has joined #commonlisp
nkatte has quit [Read error: Connection reset by peer]
ryanbw has quit [Quit: I'll be back.]
nij- has joined #commonlisp
xristos has quit [Remote host closed the connection]
dcb has quit [Quit: MSN Messenger 3.8]
<nij->
On the other hand, (sb-ext:run-program "/opt/homebrew/bin/nc" (list "-zv" "localhost" "1-99") :output *standard-output*) does print something. In the terminal, I see 5 lines got printed (port 22, 80, 5000, 7000, 8080). sb-ext:run-program only prints port 22 and 80. uiop:run-program prints nothing..
czy has joined #commonlisp
<nij->
Ah... (sb-ext:run-program "/opt/homebrew/bin/nc" (list "-zv" "localhost" "1-9999") :output *standard-output*) does print 5 lines. So the only problem now is on uiop:run-program.
czy has quit [Remote host closed the connection]
chipxxx has quit [Ping timeout: 248 seconds]
rgherdt has joined #commonlisp
nij- has quit [Ping timeout: 276 seconds]
<Alfr>
minion, memo for nij-: On you uiop:run-program problem: The joys of &allow-other-keys, try :error-output instead of :output-error in the call.
<minion>
Remembered. I'll tell nij- when he/she/it next speaks.
kevingal_ has quit [Ping timeout: 240 seconds]
kevingal has quit [Ping timeout: 240 seconds]
shka has joined #commonlisp
zxcvz has joined #commonlisp
zxcvz has quit [Client Quit]
pve has joined #commonlisp
jrx has joined #commonlisp
treflip has joined #commonlisp
jeosol has quit [Quit: Ping timeout (120 seconds)]
czy has joined #commonlisp
nkatte has joined #commonlisp
azimut_ has quit [Ping timeout: 255 seconds]
Josh_2 has quit [Ping timeout: 255 seconds]
nkatte has quit [Read error: Connection reset by peer]
nkatte has joined #commonlisp
nkatte has quit [Read error: Connection reset by peer]
jeosol has joined #commonlisp
nkatte has joined #commonlisp
nkatte has quit [Read error: Connection reset by peer]
thonkpod has quit [Ping timeout: 252 seconds]
thonkpod has joined #commonlisp
haoms has joined #commonlisp
cage has joined #commonlisp
mariari has quit [Ping timeout: 240 seconds]
karlosz has joined #commonlisp
mariari has joined #commonlisp
pjb has quit [Read error: Connection reset by peer]
random-nick has joined #commonlisp
jmdaemon has quit [Ping timeout: 264 seconds]
haoms has left #commonlisp [#commonlisp]
nkatte has joined #commonlisp
mariari has quit [Ping timeout: 252 seconds]
nkatte has quit [Read error: Connection reset by peer]
mariari has joined #commonlisp
nij- has joined #commonlisp
tyson2 has joined #commonlisp
nij- has quit [Ping timeout: 240 seconds]
nij- has joined #commonlisp
<nij->
Thanks! I knew what I did wrong.
<minion>
nij-, memo from Alfr: On you uiop:run-program problem: The joys of &allow-other-keys, try :error-output instead of :output-error in the call.
nkatte has joined #commonlisp
nkatte has quit [Read error: Connection reset by peer]
leo_song has quit [Quit: ZNC 1.7.2+deb3 - https://znc.in]
leo_song has joined #commonlisp
nkatte has joined #commonlisp
jrx has left #commonlisp [ERC 5.4 (IRC client for GNU Emacs 28.2)]
nkatte has quit [Read error: Connection reset by peer]
ebrasca has quit [Remote host closed the connection]
<JeromeLon>
3.1.2.1.2 Says that the car of a compound form can only be a symbol or a lambda. Why is the spec so restrictive (as opposed to just "a function descriptor" that would accept much more, without more work for the implementation as lambda already requires the whole thing to work)? Is there something deeper to it?
<Bike>
what's a function descriptor?
<Bike>
do you mean a function designator?
<JeromeLon>
yes, sorry
<Bike>
a function designator is a kind of value, but 3.1.2.1.2 is talking about form syntax. the car of a function form is not evaluated
<Bike>
that's part of being a lisp-2. if the car was evaluated, when you did (car foo) it would have to look up the variable CAR rather than the function CAR
<JeromeLon>
except that it is when it's a lambda
<Bike>
no, it isn't.
nkatte has joined #commonlisp
<Bike>
lambda expressions are specially recognized. it doesn't go through the normal evaluation rules.
nkatte has quit [Read error: Connection reset by peer]
<JeromeLon>
ok, thanks!
<Bike>
you could set up some semantics where symbols in the head are not evaluated normally but cons forms are, but that would be pretty patchworky in a weird way
<JeromeLon>
yes, that's what I had in mind. detect function instead of detecting lambda
<Bike>
what i mean by function designators being a type of value is that to "detect" them you would need to do a full evaluation
<JeromeLon>
oh right, where are lambda is guaranteed to return a function
<JeromeLon>
*whereas
<Bike>
the evaluator sees (head . rest), and has a test something like (cond ((symbolp head) (apply (fdefinition head) (evlist rest env))) ((and (consp head) (eq (car head) 'lambda)) ...process lambda expr...) (t (error ...)))
<Bike>
(simplified obviously, it would have to test for macro and special operators as well)
<splittist>
Shinmera: cool!
nkatte has joined #commonlisp
* splittist
is tantalizingly close to... something. https://snipboard.io/0Cw4BX.jpg But time for a break from million-monkeys 'debugging'
<ixelp>
CLHS: Standard Generic Function MAKE-LOAD-FORM
puchacz has quit [Quit: Client closed]
NicknameJohn has joined #commonlisp
euandreh has quit [Ping timeout: 260 seconds]
ryanbw has joined #commonlisp
prokhor has joined #commonlisp
dcb has joined #commonlisp
<jmercouris>
What is the canonical way in usocket to wait for usocket:socket-connect?
<jmercouris>
I thought about looping until no USOCKET:CONNECTION-REFUSED-ERROR is raised
<jmercouris>
but that doesn't seem right at all
<jmercouris>
well, I did the loop for now :-D
<random-nick>
what do you mean by that? wait for the other side to start accepting connections?
<random-nick>
if that's the case, I don't think tcp/udp support any way to know that? how would that even work on the network level?
<_death>
a loop makes sense, but you probably want to wait for some time (backoff) before retrying.. each failure could increase that amount of time (say, exponentially) and perhaps with addition of a bit of noise (jitter) to avoid multiple clients connecting at the same time (a thundering herd)
<pjb>
JeromeLon: these restrictions allow the implementations to generate efficient code.
<pjb>
JeromeLon: this doesn't remove much power, since you can always use funcall if you need to compute something else as function.
<pjb>
(funcall (compute-some-function a) b c)
<pjb>
JeromeLon: note that if the symbol names a function in the same compilation unit, the compiler is allowed to 1- directly call the code vector, without going thru (symbol-function name), and 2- possibly inline the function (as long as it's not declared notinline, even if it is NOT declared inline!), and 3- if it's a CL function, it can even do "more" than inline it, what's so called "open-code" it. Ie. basically generate code as if it
<pjb>
was a special operator.
tyson2 has quit [Remote host closed the connection]
euandreh has joined #commonlisp
euandreh has quit [Client Quit]
euandreh has joined #commonlisp
McParen has left #commonlisp [#commonlisp]
chrcav has quit [Ping timeout: 246 seconds]
chrcav has joined #commonlisp
kevingal has joined #commonlisp
kevingal_ has joined #commonlisp
amoroso has joined #commonlisp
tyson2 has joined #commonlisp
Devon has joined #commonlisp
kevingal_ has quit [Ping timeout: 240 seconds]
kevingal has quit [Ping timeout: 240 seconds]
tyson2 has quit [Remote host closed the connection]
amoroso has quit [Quit: Client closed]
cage has quit [Quit: rcirc on GNU Emacs 28.2]
JeromeLon has quit [Quit: WeeChat 3.5]
Devon has quit [Ping timeout: 246 seconds]
tyson2 has joined #commonlisp
<nij->
Some CL implementation supports native threads (e.g. sbcl). How is it done? Particularly an easy question is: How do they evaluate a variable? Do all threads share the same lookup table?
<nij->
Isn't bordeaux threads just a wrapper around all threads implementations?
<gilberth>
It is, based on CLIM-SYS which is based on the Lisp Machine.
shka has quit [Ping timeout: 240 seconds]
<gilberth>
Anyhow, special variables share their value among threads unless bound. When you bind a special variable the value is thread-local. Was that your question?
jmdaemon has joined #commonlisp
<Bike>
nij-: yes, but when they put it together they hammered out a bit of the semantics involved.
<Bike>
such as the special binding behavior jut mentioned
<nij->
I was wondering how thread B can get the value of a symbol that's stored in thread A.
<nij->
If B does it simply by querying and waiting, I'd imagine that creates lots of slow downs..
<Alfr>
nij-, why or what kind of query are you thinking of? Threads share address space, so they can simply access a symbol directly.
<Bike>
nij-: usually threads do not access each others' dynamic bindings. there is no way to do so outside of some specialized functions like sb-thread:symbol-value-in-thread
<Bike>
nij-: you may need to elaborate on what you mean by "value of a symbol". are you talking about special bindings or lexical bindings or something else?
<nij->
Alfr oh I see..
<nij->
I have to think more about it.
nij- has quit [Remote host closed the connection]
<Alfr>
Uhm ... did he just leave?
<Bike>
i wouldn't worry about it
rgherdt has quit [Remote host closed the connection]