kuler has quit [Remote host closed the connection]
lisp123 has joined #commonlisp
kuler has joined #commonlisp
lisp123 has quit [Ping timeout: 265 seconds]
Fare has quit [Ping timeout: 252 seconds]
Fare has joined #commonlisp
azimut_ has joined #commonlisp
yitzi has quit [Quit: Leaving]
azimut has quit [Ping timeout: 276 seconds]
kakuhen has joined #commonlisp
waleee has quit [Ping timeout: 252 seconds]
lad has joined #commonlisp
<beach>
Good morning everyone!
griffinmb has joined #commonlisp
Fare has quit [Ping timeout: 252 seconds]
semz has quit [Ping timeout: 245 seconds]
Bike has quit [Quit: Lost terminal]
robin has quit [Remote host closed the connection]
robin has joined #commonlisp
hexology has joined #commonlisp
hexology has quit [Quit: hex on you ...]
semz has joined #commonlisp
lisp123 has joined #commonlisp
amb007 has quit [Ping timeout: 265 seconds]
amb007 has joined #commonlisp
robin has quit [Remote host closed the connection]
Everything has joined #commonlisp
robin has joined #commonlisp
Inline has quit [Quit: Leaving]
robin has quit [Remote host closed the connection]
robin has joined #commonlisp
cyberbanjo has quit [Ping timeout: 240 seconds]
vats has joined #commonlisp
lisp123 has quit [Ping timeout: 252 seconds]
amb007 has quit [Ping timeout: 260 seconds]
amb007 has joined #commonlisp
peterhil has joined #commonlisp
<ecraven>
anddam: I just moved it to .quicklisp and then changed the path in .sbclrc
lisp123 has joined #commonlisp
robin has quit [Remote host closed the connection]
robin has joined #commonlisp
gaqwas has joined #commonlisp
zagura__ is now known as zagura
Cymew has joined #commonlisp
Alfr has quit [Ping timeout: 252 seconds]
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
shka has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
pve has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
sander has quit [Ping timeout: 252 seconds]
dualinverter[m] has joined #commonlisp
sander has joined #commonlisp
john__ has joined #commonlisp
gaqwas has quit [Read error: Connection reset by peer]
robin has quit [Remote host closed the connection]
robin has joined #commonlisp
frgo has joined #commonlisp
frgo has quit [Ping timeout: 252 seconds]
frgo has joined #commonlisp
vats has quit [Ping timeout: 252 seconds]
frgo has quit [Ping timeout: 252 seconds]
cjb has joined #commonlisp
frgo has joined #commonlisp
Alfr has joined #commonlisp
frgo has quit [Ping timeout: 252 seconds]
elderK has quit [Quit: Connection closed for inactivity]
Lycurgus has joined #commonlisp
john__ has quit [Ping timeout: 265 seconds]
robin has quit [Remote host closed the connection]
khrbt has quit [Ping timeout: 240 seconds]
khrbt has joined #commonlisp
khrbt has quit [Client Quit]
khrbt has joined #commonlisp
robin has joined #commonlisp
hendursa1 has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
hendursaga has quit [Ping timeout: 276 seconds]
makomo has joined #commonlisp
cjb has quit [Quit: cjb]
attila_lendvai has joined #commonlisp
cjb has joined #commonlisp
tfb has quit [Quit: died]
frgo has joined #commonlisp
frgo has quit [Remote host closed the connection]
frgo has joined #commonlisp
makomo has quit [Quit: WeeChat 3.2]
makomo has joined #commonlisp
makomo has quit [Client Quit]
makomo has joined #commonlisp
kakuhen has quit [Quit: Leaving...]
seere has quit [Read error: Connection reset by peer]
azimut_ has quit [Remote host closed the connection]
azimut has joined #commonlisp
cjb has quit [Quit: cjb]
lotuseater has joined #commonlisp
VincentVega has joined #commonlisp
amk has quit [Remote host closed the connection]
amk has joined #commonlisp
amk has quit [Remote host closed the connection]
amk has joined #commonlisp
amk has quit [Remote host closed the connection]
amk has joined #commonlisp
amk has quit [Remote host closed the connection]
amk has joined #commonlisp
cosimone has joined #commonlisp
treflip has joined #commonlisp
loke has joined #commonlisp
frgo has quit [Remote host closed the connection]
frgo has joined #commonlisp
frgo has quit [Remote host closed the connection]
frgo has joined #commonlisp
frgo has quit [Remote host closed the connection]
frgo has joined #commonlisp
pranavats has left #commonlisp [Disconnected: Replaced by new connection]
pranavats has joined #commonlisp
attila_lendvai_ has quit [Ping timeout: 252 seconds]
scymtym has quit [Ping timeout: 252 seconds]
azimut_ has joined #commonlisp
amb007 has quit [Ping timeout: 252 seconds]
azimut has quit [Ping timeout: 276 seconds]
amb007 has joined #commonlisp
khoa has quit [Quit: leaving]
random-nick has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
frgo has quit [Remote host closed the connection]
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
treflip has quit [Remote host closed the connection]
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
yitzi has joined #commonlisp
cosimone has quit [Ping timeout: 252 seconds]
lad has quit [Ping timeout: 265 seconds]
Fare has joined #commonlisp
waleee has joined #commonlisp
X-Scale has joined #commonlisp
cosimone has joined #commonlisp
vats has joined #commonlisp
speskk has joined #commonlisp
cage has joined #commonlisp
VincentV` has joined #commonlisp
VincentVega has quit [Ping timeout: 252 seconds]
lisp123_ has joined #commonlisp
ec_ has joined #commonlisp
lisp123 has quit [Ping timeout: 252 seconds]
Bike has joined #commonlisp
attila_lendvai has quit [Ping timeout: 260 seconds]
<Demosthenex>
anyone have any recommended reading regarding lexers, parsers, and handling text records as opposed to languages? i'm tinkering with cl-lexer and trying to figure out if cl-yacc is needed or if i should just use trivia
karlosz has joined #commonlisp
<pve>
Demosthenex: what does a text record look like?
<pve>
is it like csv or something?
<jackdaniel>
Demosthenex: if you look into grammars you should look into the library esrap
karlosz has quit [Client Quit]
karlosz has joined #commonlisp
lisp123 has joined #commonlisp
hendursa1 has quit [Quit: hendursa1]
hendursaga has joined #commonlisp
lisp123_ has quit [Ping timeout: 265 seconds]
speskk has quit [Ping timeout: 265 seconds]
karlosz has quit [Quit: karlosz]
Jach has quit [Remote host closed the connection]
pranavats has left #commonlisp [#commonlisp]
Inline has joined #commonlisp
ec_ has quit [Remote host closed the connection]
speskk has joined #commonlisp
speskk has left #commonlisp [#commonlisp]
speskk has joined #commonlisp
<Demosthenex>
pve: it's output from a mainframe style report, multiline records. i'm using TXR currently to extract fields, but i'm having issues scaling and so was considering going back to CL. rather than do the whole parsing thing by hand, i was hoping to do field validation and extraction with a little more sophistication
<Demosthenex>
plus, using slime to test and develop would be better than launching my programs repeatedly from the cli
<jcowan>
Does anyone know if it's common to use keywords rather than symbols in S-expression data?
<phoe>
I think so, yes
<bhyde[m]>
Definitely
<adeht>
keywords are symbols
<phoe>
I understood that as "non-keyword symbols"
cosimone has quit [Remote host closed the connection]
cosimone has joined #commonlisp
adeht is now known as _death
<lisp123>
I am looking at some nested list data, for example (A B (C D (E F) G) H (I) J) --> I am traversing the list and doing many things to each element. Now, my question is, is it better for me to store properties of each item against a string representation of the list hierarchy. By which I mean, I can use "`1.2.3" to refer to the third element of the second element of the first element of the main list. By doing so, i can create a hash & also so
<lisp123>
rt the data well
<lisp123>
So I am thinking, to get the properties associed with C for example, I just do a hash look up against "2.1" (C is the first element of the second element of the main list)
<Demosthenex>
pve: https://termbin.com/ihko that's a fictional example with the key issues. identifying a header, parsing out fields, and taking some multiline free form
<lisp123>
vs. storting all the properties in a nested list that has the exact same form as the original list, and then having to traverse to each element. Thoughts?
<phoe>
lisp123: why not use a mirror list representation?
<lisp123>
What is that?
<phoe>
in order to look up the properties of C, you take the CADDR of your property list
<lisp123>
Isn't that more expensive vs. a hash lookup
<jcowan>
Yes, that's what I meant. It seems safe to put keywords into a data file, but other package indicators are messy, as the recipient may not have that package. Plain symbols can be dealt with by creating a :data package and rebinding current-package to (find-package :data) around the read(s). Does that sound sound? :-)
<phoe>
the question is what do you hash
<lisp123>
if it is reasonably deep
<Demosthenex>
jackdaniel: esrap looks interesting, instead of a lexer then a parser, it's both in one?
<phoe>
maybe you don't need to stringify anything, just use a list of numbers as a key
<lisp123>
because I can store the positions as strings, e.g. "1.1.3.4.5.6"
<phoe>
something like (2 1)
<phoe>
no need to make strings out of it this way, you can feed the numbers directly into NTH or something
<lisp123>
strings are easier to sort no?
<lisp123>
and also easier to use as a key for hash table?
<phoe>
why do you want to sort strings though
<lisp123>
If I use (2 1), it would have to check for equality of the list, which I'm not sure if its more expensive?
<phoe>
lisp123: you need to use EQUAL for strings anyway
<jcowan>
lisp123: You'd need a specialized sorting routine to sort "11.1.2" after "1.1.2". Using lists of integers doesn't have that problem.
<Demosthenex>
anyway, all the reading material i can find on lexer/parser is about making new languages, not applying those techniques to text record parsing/import for a db. that's why i asked for general reading material ;]
<phoe>
and if you sort strings in a naive way like via STRING<, then "1.11" will arrive before "1.2"
<phoe>
oh, what jcowan said, yes
<jcowan>
If you're worried about performance, use a vector of fixnums instead a list.
<lisp123>
Hmm ok. So a list of numbers then, each representing the position in each node (as one goes down the tree)
<lisp123>
And that would be equivalent vs. using strings for performance?
<lisp123>
I think that works
<phoe>
benchmark if you really have to
<lisp123>
I was thinking sorting (1 2 2) , (1 1 2), (0 2 3) would be slower than sorting 1.2.2, 1.1.2, 0.2.3 (as strings)
cosimone` has joined #commonlisp
<phoe>
and note what jcowan said
<phoe>
in order to sort, you'll need a custom routine or you'll need to unparse the string
<phoe>
otherwise "1.11" versus "1.2"
cosimone has quit [Ping timeout: 252 seconds]
<lisp123>
I could do 1.02 though
<lisp123>
Then the problem is okay as long as less than 99 nodes
<lisp123>
(I got 99 problems, but lisp ain't one :D)
<phoe>
that's working around the problem and not solving it
<lisp123>
I guess my initial question is (a) do you agree that either using strings or a number lookup is much better than storing the whole set of properties in a big list and having to traverse the list to get the element I want (e.g. via nth or something)
<phoe>
use a vector of fixnums instead of a string, that'll be the fastest
<lisp123>
Ok let me look that up
<lisp123>
And that can be used as hashkeys yes?
<phoe>
yes, with an EQUALP hashtable
<lisp123>
Thanks!
<lisp123>
Wish me luck, hopefully this algorithm I am doing works out
scymtym has joined #commonlisp
makomo has quit [Ping timeout: 260 seconds]
cyberbanjo has joined #commonlisp
pranavats has joined #commonlisp
<jcowan>
Of course, you'll need a vector of integers instead of fixnums if you have more than 2^60 (or so) nodes. But I wouldn't worry. :-)
robin has quit [Ping timeout: 245 seconds]
robin has joined #commonlisp
karlosz has joined #commonlisp
yitzi has quit [Quit: Leaving]
<beach>
lisp123: Your problem is not clear, because you haven't told us how your program will receive those numbers in the hierarchy. Like, if those numbers come as variables, it would be a disaster to create a string first, and perhaps even a list of those numbers. It would then be better to keep your data as a nested dictionary, perhaps with each node as an alist or a vector or a hash table depending on the density of your numbers.
<beach>
lisp123: Whenever you have a problem with a data structure like that, you must explain the protocol for accessing it. If you don't, there is no way to figure out what the best representation is.
<beach>
lisp123: In other words, you first determine an "abstract data type" (or ADT) consisting of a set of protocol functions, and then you can determine one or more reasonable implementations of that ADT.
robin has quit [Remote host closed the connection]
robin has joined #commonlisp
<jcowan>
beach: In principle, yes. However, Lisp is notoriously the "big ball of mud" pattern's favorite language. You often simply don't know what the API is at first, so the most malleable implementation strategy may be better than the most performant
<beach>
Sure, but then the question of what the best implementation is, is not relevant.
<beach>
So the answer is "do what you want since we don't know how it is going to be used.
<jcowan>
It is, but the definition of "best" has changed.
<jcowan>
And we know pretty well how useful performance hacks tend to be, especially in the absence of measurements.
makomo has joined #commonlisp
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
robin_ has joined #commonlisp
robin has quit [Read error: Connection reset by peer]
cyberbanjo has quit [Read error: Connection reset by peer]
amb007 has quit [Ping timeout: 265 seconds]
amb007 has joined #commonlisp
cyberbanjo has joined #commonlisp
lad has joined #commonlisp
molson has joined #commonlisp
robin_ has quit [Remote host closed the connection]
robin has joined #commonlisp
hendursaga has quit [Remote host closed the connection]
hendursaga has joined #commonlisp
karlosz has quit [Quit: karlosz]
karlosz has joined #commonlisp
ec_ has joined #commonlisp
karlosz has quit [Quit: karlosz]
Alfr has quit [Quit: Leaving]
amb007 has quit [Read error: Connection reset by peer]
amb007 has joined #commonlisp
fengshaun has joined #commonlisp
robin has quit [Remote host closed the connection]
Fare has quit [Ping timeout: 245 seconds]
fengshaun has quit [Quit: bibi!]
robin has joined #commonlisp
_whitelogger has joined #commonlisp
Alfr has joined #commonlisp
molson has quit [Quit: Leaving]
Devon has joined #commonlisp
Fare has joined #commonlisp
silasfox has joined #commonlisp
amk has quit [Ping timeout: 252 seconds]
kevingal has joined #commonlisp
molson has joined #commonlisp
tyson2 has joined #commonlisp
silasfox has quit [Ping timeout: 245 seconds]
Michal has joined #commonlisp
<Michal>
Hi
<Michal>
I had a quick ParEdit Question
<Michal>
Is there a way to take all the expressions out of a LET form (excluding the bindings) outside (group splice)?
<jcowan>
Is there any particular advantage to synthesizing a readtable from scratch rather than just having a big cond or case in a loop?
robin has quit [Remote host closed the connection]
abraham has joined #commonlisp
<lisp123>
jcowan: I think it comes down to personal choice IMO
<jcowan>
Okay, so not particularly more or less performant?
<lisp123>
beach: p.s. Just saw your message, I agree with you but too much complexity for me for now, so something for another day to consider
cyberbanjo has quit [Ping timeout: 245 seconds]
<lisp123>
jcowan: Sorry, I'm not knowledgable enough to answer on that. I personally avoid readtables since it adds complexity
<jcowan>
How so?
<lisp123>
A cond / case is relatively easily to digest
<jcowan>
Set up your readtable, and then you can call read and get a token of whatever language you are parsing.
<jcowan>
True.
<lisp123>
Readtable requires somebody to know what readtables are and how they work - probably okay for the lisp wizards on this channel, but I can imagine a lot of people would struggle with it
Josh_2 has joined #commonlisp
<Bike>
I don't understand the question. where would the big cond or case be?
robin has joined #commonlisp
<lisp123>
Bike - I think jcowan is parsing some text. So I assume he will loop through each token, and there would be a big cond table to determine how to parse that token and then move onto the next
<jcowan>
Looping through characters, not tokens. Presumably readtables have better than O(n) search time.
makomo has quit [Quit: WeeChat 3.2]
<Bike>
so the question is why use a dynamic dispatch on characters rather than having a fixed dispatch?
amb007 has quit [Read error: Connection reset by peer]
shka has quit [Ping timeout: 252 seconds]
amb007 has joined #commonlisp
robin has quit [Remote host closed the connection]
Lycurgus has quit [Quit: Exeunt]
ec_ has joined #commonlisp
<jcowan>
I would guess that a readtable contains either a hash table or a vector or some combination of them, which means that you get better than linear behavior when dispatching with them. Of course you can use the same data structure(s) yourself, but with the same cost to human readability due to "spooky actions at a distance" as with readtables.
<Bike>
is that a yes
<jcowan>
Perhaps some compilers can optimize case if the key form and the candidate keys are all known to be of type character (or fixnum).
<jcowan>
?? You asked a why question which I tried to answer.
<Bike>
i was asking whether your question was "why use a dynamic dispatch on characters rather than having a fixed dispatch?"
<jcowan>
Both of these methods are fixed dispatch, at least if the hashtable is not changed inside the loop, and that is fine for this application
<jcowan>
(I think we may be at cross-purposes)
robin has joined #commonlisp
<Bike>
i'm just trying to understand your original question
cosimone` has quit [Ping timeout: 252 seconds]
<jcowan>
Rephrasing: "Is it better (in whatever sense) to use a special-purpose readtable to dispatch when parsing a stream of characters, or is it better to dispatch with cond?
<lisp123>
jcowan: Minor point, but I assume you mean 'case' here to avoid having to evaluate the terms in the cond
<lisp123>
Unless the effect is the same
<jcowan>
Case or cond, I don't think it matters much, though perhaps compilers can optimize case on characters better.
<Bike>
i don't think i understand what you mean by "special-purpose readtable" and how that would be different from cond. if you mean having READ include (cond ((char= char-i-just-read #\() ...) ...) kind of code, the advantage of the readtable over that would just be that the readtable can be altered, and that different readtables can be passed to read to control its behavior.
<Bike>
some implementations can do efficient multiway branching with case. pretty sure sbcl does. it definitely does on numbers and even symbols.
<jcowan>
I mean a readtable built from scratch and used to parse something other than S-expreassions.
karlosz has quit [Ping timeout: 260 seconds]
VincentV` has quit [Ping timeout: 245 seconds]
<pve>
Demosthenex: I dunno, I'd probably just hand-roll a parser for that kind of input, but esrap should work too (just saw your reply)
karlosz has joined #commonlisp
amk has joined #commonlisp
<Demosthenex>
pve: no prob. today i'm using TXR, which does a nice job of field extraction, the issue is debugging, no live interactive debug (ie: slime), and that i'm maintaining 20 tiny little parser programs in txr, shell, and sql. with cl i could likely just have a main one with some libraries.
<Demosthenex>
pve: and i've handrolled them before, perl, python, cl, etc. i'm familiar there, just trying to find the "elegant" maintainable solution. that is why i asked about lexing and parsing. ;]
kevingal has quit [Remote host closed the connection]
<pve>
Demosthenex: is it always 2 or more spaces separating the foo:bar pairs? Is there ambiguity there?
karlosz has quit [Ping timeout: 265 seconds]
<pve>
oh I get it, the fields are fixed
<Bike>
oh. i don't think it's a good idea to use READ for much of anything but lisp code since you can't control how tokens are read short of putting a macro on every single component character. you're like writing your own square peg of an entirely different parser and trying to fit it into the round hole of READ
<Demosthenex>
pve: it varies. sometimes fields overfill, often whitespace, etc. i don't control the output
<Demosthenex>
Bike: just the idea of lexing in the first place is rather cool, it'd help remove much ambiguity as opposed to just regexp parsing
<Demosthenex>
its not computer language, just text records
<Demosthenex>
using a lexer into trivia sounds pretty cool, just having to read more to wrap my brain around it
<Demosthenex>
i didn't have a code example, i was just asking for reading recommendations to self educate ;]
Cymew has quit [Ping timeout: 265 seconds]
<pve>
Demosthenex: ok, I just felt like parsing "This thing: 123 That thing: 456" could become brittle, depending on the values
<Bike>
racket's readtables don't seem to let you change token parsing either, huh
<Demosthenex>
yeah i was thinking lexing to regexp the field names, and using a good regexp/length filter to validate the data, and throw the lexed list into trivia to match the parts
<Demosthenex>
i think tomorrow i'll code up a test ;]
<Demosthenex>
because i'm just doing text records (with some variability), maybe a grammar isn't really the thing.
<pve>
Demosthenex: I'm not sure I understand what "lexing to regexp the field names" means
<Demosthenex>
pve: just taking in things like "this thing" into (:field "this thing") so i can match it in trivia
<Demosthenex>
pve: really, this is where multi-line regexp falls apart...
<Demosthenex>
i've done grotesque things in perl to try and do multiple line extraction
<Demosthenex>
shameful things that read like line noise
cage has quit [Quit: rcirc on GNU Emacs 27.1]
<Demosthenex>
multiline regexp is really fragile too, one thing can knock it off and make debugging impossible. lexed values and a pattern matcher sound better
abraham has quit [Quit: My MacBook Air has gone to sleep. ZZZzzz…]
<pve>
Demosthenex: Ok, I get what you mean (although I think you might mean parsing, instead of lexing)
abraham has joined #commonlisp
abraham has quit [Client Quit]
<Demosthenex>
pve: so as i understood, the lexer recognizes text by regexp and turns it into strongly typed tokens. then the parser consumes the token chain to create an internal representation.
<Demosthenex>
i was thinking just lex the data, and then use trivia (cl pattern matching lib) to extract matching items
<Demosthenex>
i'm just generating sql to make a spreadsheet
abraham has joined #commonlisp
robin has quit [Remote host closed the connection]