00:24 <farkuhar> beerman: http://sprunge.us/4lxyhJ is one way to make prt-get smarter about parsing install/update commands with multiple ports specified. It will proceed after issuing a warning, if there's at least one port where the install/update command makes sense. That seemed to me the most charitable way of interpreting a malformed command.

00:29 <farkuhar> Completely unrelated, http://sprunge.us/rra0b9 will trim the output of prt-get listorphans, removing from the list all the core ports (which are never explicitly mentioned in a "Depends on:" line anyway)

05:09 <beerman> core ports are mentioned, when they explicitly link against a core port.

05:10 <beerman> but thanks for the patch, like I said, I would prefer others to judge on it

05:10 <beerman> ping jue Romster

05:33 <beerman> and jaeger too of course ;)

11:18 <farkuhar> my intention with http://sprunge.us/rra0b9 was to bring prt-get more in line with other tools, in case anyone was tempted to run `prt-get listorphans | xargs prt-get remove` as an (unsafe!) alternative to pkgfoster or pkg-clean. By trimming the list to non-core ports, the output is easier on the eyes and less dangerous when piped to xargs prt-get remove.

12:02 <jue> farkuhar: sorry, but I don't like your changes to listorphans. Why should a core port treated differently if I want a list of ports no other port depends on it? That's just hidden miracle to me

12:14 <braewoods> there's naturally going to be ports that nothing else depends on. they're usually applications, etc.

13:16 <farkuhar> jue: different treatment for core ports is a consequence of the practice that we omit them from the "Depends on:" line if they're not dynamically linked in. There might be a build-time dependency, and I didn't want prt-get to give new users the impression that any core ports were somehow unneeded.

13:32 <farkuhar> before applying the patch, prt-get listorphans would output all the applications that braewoods referred to, intermingled with core ports that we recognize as essential for proper system operation. You could run this list through a filter that strips out the core ports, but a new user might not think of that step.

15:22 <jaeger> Found a bug in prtsweep. When it tries to determine the compression mode from /etc/pkgmk.conf it does this:

15:22 <jaeger> $ grep COMPRESSION_MODE /etc/pkgmk.conf | sed "s/[\t w]*#.*//; s/.*=//"

15:22 <jaeger> "xz"

15:22 <jaeger> Note that it includes the quotes

15:22 <jaeger> If it didn't match it would fall back using this:

15:23 <jaeger> [ -n "$COMPRESSION_MODE" ] || COMPRESSION_MODE="gz"

15:23 <jaeger> which would not include quotes

15:23 <jaeger> I use xz for the ISO builds and prtsweep in my automation and just noticed recently some failures due to this

15:24 <jaeger> A naive solution would be this:

15:24 <jaeger> $ grep COMPRESSION_MODE /etc/pkgmk.conf | sed "s/[\t w]*#.*//; s/.*=//; s/\"//g"

15:24 <jaeger> xz

15:24 <jaeger> Just adding s/\"//g to the sed call

15:28 <braewoods> not surprising. it's a hack job in any case to try to extract shell variables this way.

15:29 <jaeger> Lots of other ways to do it, maybe something like:

15:29 <jaeger> eval `grep COMPRESSION_MODE /etc/pkgmk.conf | sed 's/PKGMK_//'`

15:29 <jaeger> Or just source pkgmk.conf, heh

15:29 <braewoods> it won't account for changes to defaults though

15:29 <braewoods> since those are set in the actual pkgmk script

15:31 <braewoods> another option could be to just match all *.pkg.tar.* files

15:31 <jaeger> And in the case of the ISO automation it uses a local pkgmk.conf anyway

15:31 <braewoods> and ignore the compression method

15:31 <jaeger> yeah, that's probably better

15:31 <jaeger> Otherwise it could just leave cruft

15:32 <braewoods> chances are there's not going to be more than one tarball for a given package/version/release set

15:32 <braewoods> if there is it means the person changed it and rebuilt it

15:32 <braewoods> a very unlikely scenario since it usually is left alone

15:33 <jaeger> SRCGLOB and PKGGLOB checks in prtsweep have the same problem

15:33 <jaeger> That scenario happens frequently in the ISO automation

15:34 <jaeger> And part of my automation was to use prtwash (and later prtsweep) to fix this

15:35 <jaeger> Neither prtwash nor prtsweep work properly at the moment and I hadn't had time to check into why until now

15:35 <braewoods> the only way i know of to get a good data extraction here is to use a bash builtin to dump it. it's the only way i know of to access the internal data structures used in bash.

15:36 <braewoods> it avoids all the issues that can emerge from the fact that variables can be assigned expressions

15:37 <braewoods> more work but guaranteed to work correctly in more situations

15:58 <farkuhar> jaeger: sorry for introducing those error-prone hacks to extract shell variables. I was trying to avoid sourcing any files, out of respect for the security issue raised in FS#1851.

16:00 <braewoods> related to Pkgfile or similar having malicious code?

16:00 <farkuhar> that's the ticket.

16:01 <braewoods> ah.

16:01 <braewoods> well this is a genuine issue... but malicious code is a risk with any packaging. can't do much about the trust angle. but this is a different thing.

16:02 <braewoods> it could render the whole check pointless

16:02 <jaeger> farkuhar: no worries, just a surprise :) nothing's perfect, heh

16:03 <braewoods> the executable code of the port should not be used before attempting to verify the signature

16:03 <jaeger> For the short term I've just added a small shell script to my ISO tree but long term we could come up with a better solution

16:03 <farkuhar> installtransaction.cpp lines 642:671 (in prt-get source tree) has a clever way to extract shell variables without sourcing the entire file. I might want to see if it can be adapted to prtwash or prtsweep.

16:04 <braewoods> still flawed. i'd probably use wordexp to help out there though.

16:04 <braewoods> can disable command substitution

16:04 <braewoods> at least

16:10 <farkuhar> braewoods: by "flawed" do you mean a security risk, since those lines of installtransaction.cpp still allow command substitution when launching the "eval " process?

16:26 <braewoods> farkuhar: it's less than ideal because it's like trying to extract the final value of a variable from the source code of a programming language.

16:26 <braewoods> if we could start over it would probably make more sense to avoid using shell so much.

16:28 <braewoods> i mean, in this context, if Pkgfile is sourced at all before verification, then the fact signify can be overwritten it moot. they can also run other code. it's a shell script after all.

16:32 <farkuhar> i like jaeger's solution: eval `grep COMPRESSION_MODE /etc/pkgmk.conf | sed 's/PKGMK_//'`

18:33 <farkuhar> jaeger: thanks for the hint to use eval. Try the following patch and let me know how it works. http://ix.io/3Ysm

19:36 <stenur> ..except that it should support (non-even number of reverse solidus last) line continuation as the shell does

19:40 <stenur> and also hm we had that in the past somewhen didn't we; not farkuhar however iirc; i think i pointed to wordexp(3) by then

19:40 <stenur> actually it was braewoods who did

19:41 <stenur> Dec 30 2020

19:41 <stenur> Today he did, too. So, then.

19:46 <braewoods> wordexp basically just gives partial shell expansion to C programs

20:10 <farkuhar> wordexp on GNU/Linux is provided by glibc, but on NetBSD the manpage has this surprising statement: "The wordexp() function is implemented as a wrapper around the

20:11 <farkuhar> undocumented wordexp shell built-in command."

20:12 <farkuhar> which prompts the question, is it true of all NetBSD shells, or just for the default shell?

20:33 <groovy3shoes> just /bin/sh

20:35 <groovy3shoes> (both in that NetBSD's libc wrapper uses that shell explicitly, and that it's the only shell in the base system with a wordexp builtin (the other shells being ksh and csh))

22:03 <stenur> Yes. Yes, it is a buggy thing with crashes on MacOS Snow Leopard (that used perl to implement it!!) and Linux on wordexp(3) errors etc, and then some CVE in 2014, whatever.

22:04 <stenur> I replaced it with a self-written thing in 2016.

22:05 <jaeger> farkuhar: seems ok at a quick glance, I'll give it some more testing with the automated ISO builder

22:07 <stenur> I struggle with Donenfeld's usage of READ_ONCE() several times in his random implementation. In crng_has_old_seed() for example, and crng_make_state(), and get_random_u64(). I mean .. it has to be read.

22:09 <stenur> GNU thing had another CVE last year, also. But like frinnst said "all software is shit, just update it and you're good". Or something around that.