beneroth changed the topic of #picolisp to: PicoLisp language | The scalpel of software development | Channel Log: https://libera.irclog.whitequark.org/picolisp | Check www.picolisp.com for more information
ello has quit [Quit: ZNC 1.9.1 - https://znc.in]
ello has joined #picolisp
fuxoft has joined #picolisp
<fuxoft> Hello, I am solving a problem which probably has very simple solution but I am overcomplicating things:
<fuxoft> From pil, I launch an external command that produces (potentially infinite number of) lines of text.
<fuxoft> Using (task), I am reading the output of this external command.
<fuxoft> Whenever the external command terminates, I want to immediately restart it and continue reading its output.
<fuxoft> I have a minimal example THAT DOES NOT WORK here:
<fuxoft> When the external script ("ls" in this case) terminates and there are no more lines available, the (task) keeps executing and I am unable to stop it, not to mention restart it with different pipe
<abu[7]> Hi fuxoft, 'task' usually checks for EOF (ie. NIL)
<abu[7]> (in @ (while (line T) reads it all
<abu[7]> Why not just a single line?
<abu[7]> (task .. (in @ (ifn (line) (task Pipe) (handleLine @) ..
<fuxoft> My example is simplified. In reality I am reading char-by-char and the input parsing routine is more complex. There are potentially no newlines in the stream.
<abu[7]> "the (task) keeps executing" so there is no EOF?
<fuxoft> Ideally there is no EOF. But I want to write it in such a way that when there the external script stops for whatever reason (e.g. crashes), it's immediately restarted.
<abu[7]> But then the file descriptor is closed and invalid
<abu[7]> Closing the task with (task Pipe) is good, but then you need a new task
<fuxoft> Yep, so I want to replace the original (task) with a new (task) without FHandlerChar function even noticing that something happened
<fuxoft> That's why I am trying to call (commandReader) again at the end of the task (when the pipe closes)
<abu[7]> (commandReader) does not create a new task
<abu[7]> wait
<fuxoft> It should
<abu[7]> yeah
<fuxoft> Oops, the "(when *PPid..." line should not be there, I forgot to remove it when creating the minimal example
<abu[7]> what confuses me is the (while (line) .. Is (run FDisconnect) ever reached?
<abu[7]> That's only a log
<fuxoft> Yep. When ls finishes, (line T) returns NIL, doesn't it?
<abu[7]> Yep
<abu[7]> btw, i ran it and it seems to work fine
<fuxoft> Wait, it runs fine on your machine, e.g. produces the output of ls repeatedly endlessly?
<abu[7]> it closes and restarts '(ls ".")
<abu[7]> Yes
<abu[7]> Many ("Got:"...")
<fuxoft> On my machine, it starts failing with "Pipe error: Too many open files" after a while
<abu[7]> the exactly same script?
<fuxoft> Wait a moment, I will check again.
<abu[7]> Only (commandReader ..) starts the pipe
<fuxoft> Yes.
<abu[7]> and it should not be called before the first pipe closed
<abu[7]> looks all right
<fuxoft> At first it runs OK for a while, then it starts signalling "Pipe was closed" without reading any lines, and after some more iterations it starts failing with "Pipe error: Too many open files"
<abu[7]> I try a longer run
<abu[7]> Still running, but seems to get slower
<fuxoft> No, sorry, the "Too many open files" error starts appearing first but otherwise it still produces the correct output.
<fuxoft> My pil version is (24 6 27)
<abu[7]> ok
<abu[7]> Seems I don't get that error
<abu[7]> (on Termux)
<abu[7]> Is it from (pipe)?
<fuxoft> BTW it behaves the same way both on my desktop PC (Ubuntu) and in Linux containter on Chromebook
<fuxoft> ((ls ".") "finished")
<fuxoft> ("Pipe was closed")
<fuxoft> [./commandread.l:23] !? (pipe (apply exec Command))
<fuxoft> Pipe error: Too many open files
<fuxoft> Another run:
<fuxoft> ("Pipe was closed")
<fuxoft> ? !? (pipe (apply exec Command))
<fuxoft> Pipe error: Too many open files
<fuxoft> Stack overflow
<fuxoft> ? Segmentation fault (core dumped)
<fuxoft> I have around 10 files in my current directory so every loop produces around 10 lines of "Got:". Maybe that's different on your machine?
<fuxoft> Should I update pil to latest version?
<abu[7]> (task Pipe) may be the problem
<abu[7]> 'Pipe' is not bound in the task body
<abu[7]> random
<fuxoft> Ah!
<abu[7]> (task (pipe (apply exec Command)) (let Pipe @
<fuxoft> Like this? http://pb1n.de/?67ad96
<abu[7]> Not (let Pipe @)
<fuxoft> Oh, sorry
<fuxoft> Like this http://pb1n.de/?ee47bc
<abu[7]> Yes, looks good
<fuxoft> That still fails with "Pipe error: Too many open files"
<abu[7]> oh
<abu[7]> Runs fine here
<abu[7]> (again :)
<fuxoft> On both my machines
<fuxoft> The Pipe number gets to 1018.
<fuxoft> OK, I will update to lates pil version
<abu[7]> I think it is not a version problem
<abu[7]> pil did not change here
<abu[7]> ah!
<abu[7]> (task (close Pipe))
<abu[7]> the fd is never closed otherwits
<abu[7]> otherwise
<abu[7]> Perhaps I did not let it run long enough here
<abu[7]> but this might explain why it got slower and slower
<fuxoft> Yes! That seems to work
<fuxoft> I thought it closes automatically.
<abu[7]> only (pipe exe . prg)
<abu[7]> (pipe exe) does not
<fuxoft> Now it works and the Pipe number stays constant. Great
<abu[7]> 👍
<fuxoft> Have a nice day
<abu[7]> Thanks, you too!
fuxoft has quit [Quit: Client closed]
<beneroth> :)
neuroevolutus has joined #picolisp
neuroevolutus has quit [Ping timeout: 256 seconds]
beneroth_ has joined #picolisp
beneroth has quit [Ping timeout: 276 seconds]
beneroth_ has quit [Ping timeout: 276 seconds]