<joshua_>
hm, can NBD implement a flash-like device usefully? one problem is that flash semantics are not really compatible with block semantics (the least of which being that there is a third verb, but also, sector sizes are a weird concept and there are usually metadata you want to expose). Linux tried having a block device-like layer for this and decided it was a bad idea http://linux-mtd.infradead.org/doc/general.html#L_mtdblock
<whitequark[cis]>
if you have your own FTL you can expose the stuff on top of it via NBD
<joshua_>
having an out-of-process abstraction layer for a flash access mechanism would probably be useful, and NBD could even form part of that, but I dunno how NBD in its normal form (with standard `cmd`s) would implement this, and feeding one to any existing NBD client (including the kernel NBD client) would surely go wrong for writes (either burning a hole in the flash, or filling a sector with 0s), or for reads (providing no access to spares)?
<whitequark[cis]>
I mean an FTL provides you with something you can mount as a block device, right? like /dev/nvme0n1 is a block device
<joshua_>
that would work, though I would assume that part of the goal of using glasgow to talk to nand flash would be to experiment with a device's FTL (perhaps in ways that may not make sense in glasgow upstream), rather than to use it as an arbitrary storage device for a Glasgow-specific FTL
<whitequark[cis]>
so if you implement an entire FTL in Python (probably by reverse-engineering someone else's) you could do that
<whitequark[cis]>
we could perhaps have a simple JFFS2 or UBI implementation
<joshua_>
yes indeed, if you want to enforce that any FTL be in-process with glasgow
<whitequark[cis]>
i think there is a way to emulate MTD devices
<joshua_>
those are particularly difficult examples because jffs2 very much does not haev a block device FTL format :)
<whitequark[cis]>
oh, you are right
<joshua_>
I do not know enough about UBI to comment
<whitequark[cis]>
UBI is weird
<whitequark[cis]>
I think you have a solid point actually.
<whitequark[cis]>
this would not work for most MTD Linux filesystems
<whitequark[cis]>
I guess what you could do is to use the spare area to do ECC and then expose the rest via NBD
<whitequark[cis]>
I'm not sure what the utility is but it would work as a read-only abstraction at least
<whitequark[cis]>
since now you get nice PO2 sized blocks
<joshua_>
but (as someone who has dumped a NAND flash to reverse engineer it in the past; https://joshuawise.com/projects/ndfrecovery ), the ops that I want while reverse engineering are often things like "repeatedly read the idcode block to make sure that my BER is reasonably low"
<joshua_>
other problems include that sometimes row and column space is not even linear, for instance on MLC or TLC parts
<whitequark[cis]>
no yeah you are right
<joshua_>
now that said: this mtd thing is not useless
<whitequark[cis]>
I paged out most of the NAND flash info from my mind
<joshua_>
er, this NBD thing
<whitequark[cis]>
I want the NBD thing for floppies
<joshua_>
this is, like, useful work that should go in pypi somewhere, I just don't know that it should go in glasgow. a glasgow-of-the-almighty that was big enough to drive a PATA would want this
<whitequark[cis]>
hm
<whitequark[cis]>
I think glasgow should be able to drive CF, just about
<joshua_>
certainly SD
<whitequark[cis]>
yes, SD/MMC
<whitequark[cis]>
I actually started working on an MMC target at one point
<whitequark[cis]>
but it's a pretty big and complex spec
<whitequark[cis]>
also at the time i only had a revA
<whitequark[cis]>
8-bit CF is 19 pins
<joshua_>
though these also have problems that are hard to solve with NBD (eMMC has plenty of weird ioctls / vendor requests)
<whitequark[cis]>
i mean you do want to be able to just read random eMMCs too
<whitequark[cis]>
oh, there is gluebi
<joshua_>
just reading is a good start, though for many such applications you wire it up and slam it into your laptop's SD card slot. reading the boot partitions and/or accessing the RPMB seems useful. on the other hand, you could certainly define vendor specific extensions and write userspace code to talk to that NBD stuff
<whitequark[cis]>
joshua_: random laptop SD card readers are often *awful* about crumbling SD cards
<whitequark[cis]>
mine is completely useless for example
<joshua_>
I would believe this
<joshua_>
though 'crumbling SD cards' is also the sort of situation where you probably want finer grained control than a NBD can give you ('read this sector ten times, resetting the SD card Vdd between each one')
<whitequark[cis]>
in this particular case i would settle for reading each sector exactly once
<joshua_>
('read this sector seven times, with Vdd between 3.0V and 3.6V')
<whitequark[cis]>
well
<whitequark[cis]>
the card eventually just went into the bin.
<joshua_>
rip
<joshua_>
anyway, again, not, like, to come in here and shit on someone else's work without having so much as made ac ommit of my own. but it does seem like someone might be preparing to put in a lot of time on something that they might suddenly discover to be a lot less useful than they think
<whitequark[cis]>
I'm 100% on board having the NBD driver in Glasgow
<whitequark[cis]>
we will need it
<joshua_>
(the only other question again being whether this is an abstraction that also is useful outside of Glasgow, in such a way that it might want to be a pypi of its own)
<whitequark[cis]>
that is not the right question to ask imo
<whitequark[cis]>
the question to ask is: who will be burdened by maintaining it
<joshua_>
indeed
<joshua_>
hmm st506/st412 also would benefit from a NBD driver (though, again, one question is whether the demodulator wants to be in-process or out-of-process w.r.t. glasgow itself, but I suppose even were it out of process it coul dimport glasgow.protocol.nbd)
<whitequark[cis]>
demodulator should be on the FPGA
<whitequark[cis]>
just like for the floppy
<whitequark[cis]>
you're not thinking ambitious enough :p
<joshua_>
for normal read/write, sure. given that all these devices are aging, I am sort of thinking about data recovery applications
<whitequark[cis]>
well, for floppies, you would have a "read raw" command that is just a 48M or something logic analyzer
<joshua_>
sure, fair enough. hard to express that ioctl through NBD! but I guess you could have a different applet for that.
<joshua_>
afk half an hour or so
<whitequark[cis]>
a different subcommand, really
<whitequark[cis]>
or a different applet, whichever makes the most sense
<Darius>
just use iSCSI, then you can issue your own custom command ;)
<joshua_>
seems preferable, but only insofar as I have not read the iSCSI spec
<Darius>
I imagine iSCSI is terribly complex
redstarcomrade has quit [Read error: Connection reset by peer]
<whitequark[cis]>
that was a nice modernization of one of the oldest applets ever
<whitequark[cis]>
joshua_: thought: you could expose a raw floppy as an MTD
<whitequark[cis]>
like... you could consider it as track-eraseable, page-writable
<whitequark[cis]>
and then you could put a flash-oriented filesystem there, with wear levelling disabled, to take advantage of the much greater density that having little to no gaps allows
<_whitenotifier-3>
[glasgow] github-merge-queue[bot] created branch gh-readonly-queue/main/pr-629-eaff9e47ebaf0438d2f137e4efe2f040b49fc556 - https://github.com/GlasgowEmbedded/glasgow
<whitequark[cis]>
I have had an idea
<whitequark[cis]>
we have a bunch of --invert-something options in Glasgow... I think UART lets you invert RX and TX, and the current SPI applet lets you invert the clock
<whitequark[cis]>
but Amaranth actually parameterizes every single I/O core over inversion
<whitequark[cis]>
so we can ditch all of those options in favor of a single unified "give me this pin in its natural state" or "give me this pin inverted"
<whitequark[cis]>
this is very handy in case you have e.g. someone use a hex inverter to bridge voltage domains or something
<tpw_rules>
i like being able to clean up the implementation, but how would that be presented to the user?
<whitequark[cis]>
--pin-tx 0#
<tpw_rules>
ah, that would be very slick
<whitequark[cis]>
it seems straightforward, I'll look into it and then clean up the implementation
<whitequark[cis]>
this is all in preparation for moving spi-controller and qspi-controller to use amaranth-stdio blocks
<joshua_>
what defines the name of an applet? context: I am preparing to bring up a board that has a 24x memory later and want to preprogram it with a glasgow. tried already: searched Glasgow docs for `applet`, looked at use/basic.html, tried reading applet/__init__.py and saw the concept of a metadata but not where it came from, and ran out of steam and figured I would ask
<whitequark[cis]>
this is done so that the glasgow CLI can be extended
<joshua_>
yeah, this makes sense, I wasn't sure if it was doing something funky like somehow managing to come up with a default class name by scanning for .py files or something
<whitequark[cis]>
it used to do something like that
<whitequark[cis]>
it's no longer doing it because that's awful
<whitequark[cis]>
(there was a file called all.py which imported every applet)
<joshua_>
sounds fast
<whitequark[cis]>
we're still importing every applet, to extract the help text from them
<whitequark[cis]>
i'm not totally sure how to avoid it
<joshua_>
would a PR be accepted to docs/manual/src/use/basic.rst to describe the applet-first nature of Glasgow (and how applets are found)? I am basically on the couch covidated but I might be able to summon enough brains to write some english after a nap
<whitequark[cis]>
is that even the right place for it? this is basically documenting internals
<whitequark[cis]>
not basic use
<whitequark[cis]>
"basic use" in our context means, i would say, "i don't know how to code"
<whitequark[cis]>
I'm still not ready to offer a public API so adding an applet is kind of inherently messing with internals
<joshua_>
I think this is more of a question of describing the 'applet-major' nature. i.e., in order to successfully use glasgow, you do need to understand that applets are the major interface for defining what you can do with a glasgow, and even if you don't plan to write one, you need to know that they provide the primary abstraction (rather than a magical concept of 'uart' or 'i2c-initiator')
<joshua_>
I will write some text later and propose it. if you do not like it, I can iterate on it or it can die on a branch :)
<whitequark[cis]>
oh yes, sure, we can pair on that
<whitequark[cis]>
I'm fairly picky about copy-editing but if we can work out an understanding of what makes docs good I'm very happy to let others write them
<joshua_>
that's fine, I usually need a copyeditor to put the concision back in my text
<joshua_>
I am definitely not personally offended by someone ripping through text I wrote and removing words
<joshua_>
I will dream on this as I allow antipyretics to do their work
<whitequark[cis]>
so the thing is that if I have to rewrite it all anyway I might as well write it in first place
<whitequark[cis]>
and I'm trying to reduce my workload not increase it
<joshua_>
I will try to make it not suck to start with
<whitequark[cis]>
thank you
<whitequark[cis]>
my main constraint is just "how much time do I have to spare and how miserable will the process make me to get a high quality result"
<whitequark[cis]>
I eventually worked out how to write really good docs for Amaranth but it took some effort
<joshua_>
writing good docs is not at all easy
<whitequark[cis]>
it's harder than writing good code
<whitequark[cis]>
and it's way harder to find people who can write good docs than who write good code
<whitequark[cis]>
to the point where I found it easier to become one
<joshua_>
like writing programs, it is easy to write spaghetti documentation that does not set up concepts appropriately and that does not tell an appropriate story aimed at a particular user, but it is hard to write useful documentation that meaningfully improves a user's experience over just asking them to `git grep`
<whitequark[cis]>
yes.
<tpw_rules>
do you have exampled of bad (perhaps amaranth, perhaps not) docs?
<whitequark[cis]>
this is why I refused for years to have any documentation at all for much of Amaranth
<whitequark[cis]>
I would rather have no docs than bad docs
<joshua_>
I think I am vaguely fine with https://nvdla.org/primer.html , but I think I have learned to write better documentation since then (and, in particular, that is not very skimmable)
<tpw_rules>
do you work for nvidia?
<tpw_rules>
ah, yuo did
<whitequark[cis]>
something I'm thinking about changing, which is going to be fairly breaking, is the weird level of indirection where pins are numbered 0..15 or sometimes 0..7 and this corresponds to --port
<whitequark[cis]>
basically I think I want eg glasgow run uart -V 3.3 --pin-rx A0 --pin-tx A1