<whitequark[m]>
i actually control the bit of code where the issue occurs
<whitequark[m]>
yep, i can just do it
<josHua[m]>
I have a dumb question.
<whitequark[m]>
nya?
<josHua[m]>
if importing the modules only happens to get the help strings, rather than adding a load-bearing dependency that requires understanding cpython internals and ensuring that argparse never eagerly evaluates something in the future, would it be potentially preferable to look at os.args for --help and only load the modules if someone asks for it?
<whitequark[m]>
good question
<whitequark[m]>
first off i want to say that it doesn't actually require understanding cpython or argparse internals (any more than the existing `cli` module does)
<josHua[m]>
this seems extremely clever but also likely to be quite brittle
<whitequark[m]>
like it's a single `str()` in the existing `TextHelpFormatter` override
<whitequark[m]>
the reason i'm looking at this instead of the `if '--help' in args` solution is that i don't like that it ignores our complex CLI semantics entirely and speculatively does something that may be completely unwanted
<josHua[m]>
if it takes the original developer reading argparse internals to figur eout how to do it one way in particular it may take someone else reading the same later to avoid falling into the same trap later, anyway
<josHua[m]>
indeed. `'--help' in args` can overkill
<whitequark[m]>
i mean we're already in that boat. the existing `glasgow.cli` module overrides argparse internals already
<josHua[m]>
(also, -h, and also, other usage faults, may underkill)
<whitequark[m]>
yes
<whitequark[m]>
i'd rather have a fragile solution that works as designed while some condition is satisfied and breaks loudly, than a fragile solution that doesn't really work when it's added
<josHua[m]>
anyway, I guess my only point is that this seems rather extreme and trips my intuitive alarm bells for 'maybe there is enough thrust that can be put behind a different brick that might be less extreme'
<josHua[m]>
agh train time. afk
<whitequark[m]>
if you find out how to do it in a way that isn't functionally worse than what i'm proposing we can do that instead
<whitequark[m]>
so... I found a conceptual problem with this approach
<whitequark[m]>
namely, it would be no longer possible to restrict glasgow test and (more importantly) glasgow tool subcommands to display only those commands which actually have tests and tools
<whitequark[m]>
for tests this isn't a big deal
<whitequark[m]>
for tools, this is actually kind of confusing
<whitequark[m]>
what might work here, and also solve the problem josHua (@_discord_256468461818085377:catircservices.org) is highlighting (i.e. fragility), is to grab the rest of the arguments after `run` (instead of using a subparser` and then instantiate another argparse instance manually
<whitequark[m]>
s/`//
<whitequark[m]>
s/`/)/
<whitequark[m]>
it's a *lot* of work though: you have to override `-h`/`--help` to load every applet, special-case getting the applet as an argument, while also receiving other unrelated options, etc
redstarcomrade has joined #glasgow
redstarcomrade has quit [Changing host]
redstarcomrade has joined #glasgow
FFY00 has quit [Remote host closed the connection]
FFY00_ has joined #glasgow
redstarcomrade has quit [Ping timeout: 260 seconds]
redstarcomrade has joined #glasgow
redstarcomrade has quit [Read error: Connection reset by peer]
redstarcomrade has quit [Read error: Connection reset by peer]
altracer[m] has joined #glasgow
<altracer[m]>
sounds like it's a hard problem, similar to git-fsmonitor? i.e. to display glasgow cli help, you'd have to walk all its module files, to either extract help every time or to check modification timestamps and rebuild help cache...
<whitequark[m]>
altracer (@_discord_102082282268934144:catircservices.org) oh... you gave me an idea
<whitequark[m]>
I can actually do that
<whitequark[m]>
I can do a diff of sys.modules before and after loading all plugins
<whitequark[m]>
and whenever I load them properly, I can check the cache against the actual values; if my cache says they should be X but they're actually Y i can display a warning
<whitequark[m]>
so over time we'll at least become aware of cache bugs as opposed of them being just a part of life
<altracer[m]>
I can also give a worse idea of displaying built-in central help first, then lagging for 0.5s crawling the help from dynamic plugins, that way you have both responsiveness and no daemon/caches
<altracer[m]>
Or because of argparse or python or windows stdio you can't do that easily?
<altracer[m]>
The new and open source CLI utils I've had the pleasure of working with are BMDA (C), bmpflash (C++), and bmputil (Rust), but they don't have plugins, they have the benefit (for this point) of being written in a compiled language, and so they print their help immediately
<altracer[m]>
pyocd -h, for comparison, does take a sweet 0.6s on my 10yo machine
<altracer[m]>
`glasgow -h` is 865+75ms, user+sys (repeated), that's... tolerable because I don't use it at all? If I used daily a tool like this, then I might have a different opinion
<altracer[m]>
mounted a tmpfs at `~/.local/pipx/venvs` and reinstalled, time did not change, so SATA SSD cached (by Linux fs buffers) install behaves as fast as DDR3-based one, on my machine, and `perf stat` agrees it's in user time of python3.10 of Ubuntu jammy
<whitequark[m]>
argparse requires me to give it everything before i can parse any arguments
<whitequark[m]>
mind you, if this was written in rust i'd be in an even worse position because lolrust doesn't have plugins
ewenmcneill[m] has joined #glasgow
<ewenmcneill[m]>
On a almost 6 year old laptop, `glasgow -h` takes about 1.2s (wallclock), and `glasgow list` takes about 1.1s, with an older version of glasgow (`0.1.dev2068+g93400d6`; and no hardware connected). Which TBH feels "fine" for something not really "on the fast path" of any usage.
<altracer[m]>
~~Tsoding does hot reloading in C of all things~~
<ewenmcneill[m]>
Would it be "smoother" if it was a bit faster? Probably. But it's at most 4x longer than "imperceptible" (which is say 250ms), so I'm with Attie from up thread that I wouldn't consider improving it any further *that* high a priority.
<altracer[m]>
Would caching sys.modules or other user-modified plugin list allow shrinking this down? Like, what would you cache, stdout contents?
<whitequark[m]>
it's easier to do it in C than in Rust
<whitequark[m]>
~~mostly because C programmers don't expect your stuff to not segfault randomly~~
<Attie[m]>
*sad* ... thanks for looking into it at least!
<Attie[m]>
I had a very quick look into it yesterday, and there are some small gains to be had (-15% wasn't too challenging)
<_whitenotifier-3>
[glasgow] github-merge-queue[bot] created branch gh-readonly-queue/main/pr-648-7e2604169a6fce95ba63c9d67df1055178c1cf37 - https://github.com/GlasgowEmbedded/glasgow
<Attie[m]>
but it's not exactly a big hitter (and there may well be other issues with what I did that I'm unaware of)
<whitequark[cis]>
I don't feel that splitting the argument parsing code from argument using code is an improvement
<whitequark[cis]>
it's already hard to maintain, but at least you have ctrl+f
<Attie[m]>
it's almost like argparse should be extended to support callbacks when it wants to resolve certain branches of the tree - I imagine what you were trying to achieve with `lazy`(?)