<_whitenotifier>
[amaranth-lang/amaranth-lang.github.io] github-merge-queue[bot] 501ba76 - Deploying to main from @ amaranth-lang/amaranth@012f12339daeac1a405712e8f679d34c8249b8f5 🚀
<galibert[m]>
The line is understandable only for people who know python rather well. Amaranth does not necessarily target those people
<galibert[m]>
Especially the self.dict.update part. Also it’s potentially fragile against evolution of either interfaces or python. A dwim interface to me feels better
<whitequark[cis]>
I don't think "DWIM" is a thing that exists
<whitequark[cis]>
assuming you're referring to Perl's "Do What I Mean"
<galibert[m]>
True
<whitequark[cis]>
I think self.__dict__.update is not fundamentally more confusing than setattr()
<galibert[m]>
But at least do what I want and is documented as part of the language?
<galibert[m]>
Sure, but setattr is confusing too :-)
<whitequark[cis]>
if setattr is confusing then how would wiring.create_signature_members not be confusing since the latter is expands to the former?
<galibert[m]>
"wiring.create_signature_members(object): For elaboratables with a dynamic signature (e.g. created at runtime and not as annotations on a Component), this function indicates that the signature is complete and creates the signal members associated to it. Components, which have static signatures, call it automatically."
<galibert[m]>
Feel free to bikeshed to death
<galibert[m]>
There is a what, no how, people caring about the how can see the implementation (or guess when they see the members.create() documentation)
<whitequark[cis]>
I think if you are creating elaboratables with a dynamic signature it's OK to expect you to understand fairly core parts of the Python language
<galibert[m]>
As you wish. I won't insist on that :-)
omnitechnomancer has quit [Quit: Idle timeout reached: 172800s]
<_whitenotifier>
[amaranth-lang/amaranth] whitequark 20e8bbd - Bring `__version__` retrieval up to date. NFCI
<crzwdjk>
What I'm doing is not that dynamic, I just want signals that are big enough for the range of values that need to be represented, which range is determined by a command-line parameter (it's a display mode), so the only thing that changes is the shape of the signals, not their presence or absence or number
<galibert[m]>
A lot of dynamic is going to be just that
<galibert[m]>
Like wishbone but width
<galibert[m]>
s/but/bus/
nimh has quit [Ping timeout: 245 seconds]
nimh has joined #amaranth-lang
<crzwdjk>
But also one line to copy and paste everywhere is not too terrible
<_whitenotifier>
[amaranth-lang/amaranth-lang.github.io] github-merge-queue[bot] fb6a319 - Deploying to main from @ amaranth-lang/amaranth@20e8bbdfbd6443590d9727ea53374102c60bc0e2 🚀
<whitequark[cis]>
<crzwdjk> "But also one line to copy and..." <- yes
<whitequark[cis]>
<galibert[m]> "That way lies cargo-culting ;-)" <- I think the obsession with "DRY" and other principles like this is in and of itself an example of 'cargo-culting'
<whitequark[cis]>
I am perfectly happy for people to copy entire modules around if keeping them in the standard library isn't worth it
Chips4MakersakaS has quit [Quit: Idle timeout reached: 172800s]
<whitequark[cis]>
so i have good news and bad news
<whitequark[cis]>
the good news is that I think I have a pretty decent idea of how streams should be modelled
<whitequark[cis]>
the bad news is that the model is probably not actually implementable
<galibert[m]>
Cool
<galibert[m]>
Less cool
<whitequark[cis]>
this follows a conversation with Paul Roukema, who gave me an important insight: you might want to have packets-in-lanes, lanes-in-packets, or both
<vegard_e[m]>
when will you have packets in lanes? something like QSGMII?
<whitequark[cis]>
i.e.: you might want to have a packet that is transferred in chunks of up to 4 bytes per transfer (in which case the last indicator covers all of those bytes), or you might want to transfer up to 4 packets at a time (each of which can be up to a few bytes in length for example)
<whitequark[cis]>
vegard_e: yes, 200GE and 400GE were an example
<galibert[m]>
isn't packet-in-lanes a fancy array of streams?
<zyp[m]>
yes, with common flow control
<whitequark[cis]>
I don't actually expect many people use packets-in-lanes with Amaranth but it gave me the following insight
<whitequark[cis]>
the core idea of a "stream" is iterating over sequences of things (in hardware), but our basic implementation of a stream (single payload, single ready/valid) only covers one case: one thing per cycle
<whitequark[cis]>
lanes are "more than one thing per cycle", packets are "less than one thing per cycle", these can be combined in essentially unlimited number of ways
<whitequark[cis]>
in an ideal world, we would not make users of the stream library deal with ready/valid signalling at all, but instead they would have an algebra of streams where operations could be composed together
<whitequark[cis]>
this is probably still a worthy goal, but we also cannot exactly spend a year perfecting this algebra
<whitequark[cis]>
and moreover, Amaranth is a low-level language where details that are unimportant from a mathematical perspective do matter quite a bit
<whitequark[cis]>
for example, USB 3 is designed for exactly 32 bit datapath width, and any reasonable gateware will actively take advantage of that fact
<whitequark[cis]>
trying to squeeze USB3 operations into an algebra of streams with parameterizable width, only to then parameterize it to exactly 32 bits at all times, is ... not a good idea
<whitequark[cis]>
anyway, I wasn't sure about transactional FIFO before, but now I see that FIFO : stream :: transactional FIFO : packetized stream; i.e. where the basic operation on streams is iterating over words, the basic operation on packet streams is iterating over packets
<whitequark[cis]>
all of the complexity that appears here stems from the fact that Amaranth is a low-level language that requires you to care about the exact latency of every operation
<whitequark[cis]>
for something less abstract, consider for example a simple DSP primitive: abs()
<whitequark[cis]>
a stream.AbsoluteValueTransformer should not care whether it is given a stream with or without lanes
<whitequark[cis]>
something like a 8b10b encoder would fall into the same class
<whitequark[cis]>
conversely, something like a 64b66b encoder would have to maintain state
<whitequark[cis]>
there are all sorts of interactions with other modules, e.g. the CRC module that adamgreig wrote cannot cope with there being more than one packet per cycle even though that's not that weird
<whitequark[cis]>
one conclusion I'm coming to is that stream infrastructure and packet infrastructure should probably be entirely separate from one another
<whitequark[cis]>
i.e. there should be no "packet stream that is a subtype of stream", but rather there should be a packet serializer (inputting a series of packets, outputting a stream; potentially multilane stream) and a packet deserializer (inputting a stream, outputting a series of packets)
<whitequark[cis]>
then, whether to use first, last, or some other type of signaling actually becomes an implementation detail of a particular serialization
<adamgreig[m]>
<whitequark[cis]> "something like a 8b10b encoder..." <- Not really relevant but wouldn't this also need to track state?
<whitequark[cis]>
oh, you're completely right
<whitequark[cis]>
I forgot about running disparity
<adamgreig[m]>
That does give me an idea for another RFC though :p
<whitequark[cis]>
a stdlib 8b10b encoder/decoder?
<whitequark[cis]>
yes please
<adamgreig[m]>
I wonder just how much of my codebase will end up replaced with imports from amaranth stdlib, lol
<adamgreig[m]>
Lfsr first anyway
<galibert[m]>
hopefully most of it? ;-)
<whitequark[cis]>
yeah!
<whitequark[cis]>
well a big standard library has major downsides, just ask, uh, Python
<adamgreig[m]>
Perhaps both its best and worst feature
<whitequark[cis]>
I think it inches close to "worst" as bitrot gets deeper and deeper
<adamgreig[m]>
I like rust's comprehensive but restrained stdlib
<whitequark[cis]>
my aim is to be closer to Rust, but not quite as close
<whitequark[cis]>
e.g. Rust wouldn't have standard library LFSRs
<adamgreig[m]>
Like, ints have a million really useful little methods , but there's no SMTP parser
<whitequark[cis]>
well, "hardware equivalent of Rust"
<adamgreig[m]>
Whereas python presumably still ships with a gopher server or something
<whitequark[cis]>
lmfao
<crzwdjk>
Rust of course also has cargo, which includes some packages from the rust team
<galibert[m]>
Catherine: you have an habit of unit tests, combined with the natural division in independant components that should reduce the amount of bittor substantially
<whitequark[cis]>
well it also helps that an LFSR or a CRC once shipped never really changes
<galibert[m]>
true
<galibert[m]>
well, amaranth itself may change (see rfc#2) and library components should follow
<whitequark[cis]>
adamgreig: I'm finding myself questioning if even up/down-converters should be in the stdlib
<whitequark[cis]>
can we ship a unified multilane representation?
<adamgreig[m]>
And is it useful, or do you just want good ecosystem support for like wishbone and axi4s and pipe or whatever
<whitequark[cis]>
yes, exactly
<adamgreig[m]>
You can definitely ship some sort of upconverter but I'm not sure who uses it
<whitequark[cis]>
and consider PIPE for example
<whitequark[cis]>
your typical FPGA PIPE implementation will convert from 2 lanes to 4 lanes
<whitequark[cis]>
can our generic upconverter even do that?
<whitequark[cis]>
I guess we could make it able to go from n lanes to f×n lanes, where f is an integer factor
<adamgreig[m]>
Dunno though, I guess my stuff is all streaming frames byte by byte through a bunch of processing blocks and fifos and such, and maybe some of those would like to run multiple bytes in parallel if I could plug something in? I haven't actually needed to though
<adamgreig[m]>
Yea, but perhaps a lot of work to make generic vs the PIPE library writing its own specific and optimised one?
<whitequark[cis]>
it's not very hard to make a well working upconverter; LUNA has one and it's very simple
<galibert[m]>
Stupid questions maybe, but what (std)lib or whatever is going to provide w.r.t streams if converters are out?
<galibert[m]>
s/questions/question/
<whitequark[cis]>
the signature and the semantics, at the bare minimum
<zyp[m]>
FIFOs are the other obvious thing
<whitequark[cis]>
FIFOs, yes, there is nothing wrong with FIFOs
<adamgreig[m]>
Even just a convenient and shared interface so that stdlib and ecosystem components become easier to integrate into your own stuff seems a nice win
<adamgreig[m]>
Like these crc and lfsr and 8b10b encoders or whatever could all happily take streams in/out and presumably that makes them simpler to plug together and use
<whitequark[cis]>
basically, we have a very clear and strong need for "exactly one thing per cycle, or nothing" transfers
<galibert[m]>
That makes sense
<whitequark[cis]>
and there aren't many implementation questions
<whitequark[cis]>
but it is less clear how to implement "up to n things per cycle", or "up to n cycles per thing" transfers in a generic way, and the need isn't as strong
<whitequark[cis]>
also, "exactly one thing per cycle, or nothing" transfers are used in AXI4
<galibert[m]>
The axi4 streams thing?
<zyp[m]>
no plain AXI, it's effectively a collection of streams
<galibert[m]>
fun
<whitequark[cis]>
AXI is pretty cool
<zyp[m]>
the read side of an AXI interface is effectively one stream with the addr you want to read from and another stream with the read data
<whitequark[cis]>
the write side is addr, data, feedback
<zyp[m]>
and each being unidirectional streams, it's fairly easy to CDC, you just stick them in async FIFOs
<whitequark[cis]>
oh, I haven't even thought of that part
<whitequark[cis]>
that's really cool
<zyp[m]>
I once wrote a memory read/write control request handler for LUNA that's compatible with ValentyUSB's wishbone bridge so it works with the same host side tools, to make it easy to CDC I made it hook up to AXI-Lite, added CDC and just plugged that into the LiteX memory interconnect which handles converting that to wishbone
<whitequark[cis]>
that's cursed
<zyp[m]>
probably, but it works :)
<whitequark[cis]>
most cursed things work!! that does not make them not cursed!!!
<zyp[m]>
I could have made custom request/response streams and have CDCed those instead, but that'd be more work and pretty much the same thing
<zyp[m]>
does amaranth have a wishbone CDC yet? litex didn't at the time (and I don't know whether it's gained one since)
<whitequark[cis]>
anyway, I'm starting to actually write the streams RFC soon; besides FIFO, what do you want to see in the stdlib?
<whitequark[cis]>
zyp[m]: nope
<zyp[m]>
pipeline
<whitequark[cis]>
@zyp: elaborate? the skid buffer thing that was discussed recently, or?
<zyp[m]>
litex' stream.Pipeline is a module, but I figure it'd work just as well as a free function like connect()
<whitequark[cis]>
oh, like connecting source..sink, source..sink, etc?
<whitequark[cis]>
the main thing I'm avoiding is introducing more magic names into the standard library / language; we have added one, signature, which is okay I guess
<whitequark[cis]>
e.g. this way we do not have to bikeshed src/snk vs source/sink :)
<whitequark[cis]>
(though you could argue that by introducing pipeline we could ensure the consistency of this choice...)
<zyp[m]>
you wouldn't use it to connect two modules, you'd use it to connect a list of modules to be chained
<zyp[m]>
a list that you might build programmatically
<whitequark[cis]>
oh I see.
<zyp[m]>
e.g. if CDC is required, you stick an asyncfifo in the list
<adamgreig[m]>
Would enough things have zero or one inputs/outputs to make that useful, vs calling connect in a loop?
<zyp[m]>
you'd need to make a loop that matches up pairs
<whitequark[cis]>
zyp: I'm not sure I think having *asyncfifo* as an example is good?
<whitequark[cis]>
let's consider what the other choice is...
<whitequark[cis]>
source = x.source
<whitequark[cis]>
if y: connect(m, source, y.sink); source = y.source
<whitequark[cis]>
if z: connect(m, source, z.sink); source = z.source
<whitequark[cis]>
it does everything pipeline does, in little more code (if you consider the code required to construct the list, and without any of the magic
<zyp[m]>
I like the conciseness of making a list of modules to chain, also when not modifying it programatically
<whitequark[cis]>
is it worth the cost though?
<crzwdjk>
I feel like connect makes connecting things easy enough now
<whitequark[cis]>
I'm actually interested in some sort of flow graph automation, that's one of the oldest open issues in Amaranth
<whitequark[cis]>
but that would definitely not limit modules to a single source and a single sink
<whitequark[cis]>
incidentally, I think I also came up with a better serialization for packets than first/last
<whitequark[cis]>
it consists of two parts: "data[n]" for n lanes, and "length"
<whitequark[cis]>
this removes the ambiguity where with the lane mask, you would be able to have a mask of something like 0b1010 for a 4-lane stream, which is invalid
<whitequark[cis]>
oh, no, you still need last
<whitequark[cis]>
if you want to be able to send a stream of lane-count-multiple sized packets back to back
<whitequark[cis]>
I guess you could have length with a range of 0..n (inclusive), where length of n means the packet is continued
<whitequark[cis]>
this is functionally the same as length and a !last flag
<whitequark[cis]>
* !last flag for a power-of-2 lane count
<galibert[m]>
Assuming you know the length at the start of a packet is big, isn’t it?
<zyp[m]>
not the length of the packet, just the length of the transfer this cycle
<whitequark[cis]>
^
<whitequark[cis]>
length is better than a lane mask but it is not quite good enough to replace last signaling
GenTooMan has quit [Ping timeout: 256 seconds]
GenTooMan has joined #amaranth-lang
cr1901_ is now known as cr1901
<cr1901>
Why is 0b1010 for a 4-lane stream invalid?
<whitequark[cis]>
what does it mean?
<zyp[m]>
it doesn't have to be, but it makes sense to define that you can only use contiguous lanes, starting with 0
<cr1901>
"take the bytes from lanes 1 and 3, ignore 0 and 2". Kinda like AXI null bytes
<zyp[m]>
because that makes the handling logic simpler
<whitequark[cis]>
cr1901: remember how I said there are also packets?
<cr1901>
zyp[m]: I mean, that's fair. Brain is still in AXI stream land.
<whitequark[cis]>
which byte is the last byte of the packet?
<cr1901>
The top-most used lane for that particular transaction with mask 1010, so lane 3
<cr1901>
I'm tired, let me reread
<zyp[m]>
the way I see it, there's two ways to handle last for a multilane stream
<zyp[m]>
one way is to have a last bit on the transfer, and some way of telling you how many lanes have data for this last transfer
<zyp[m]>
another way is to have a last bit per lane, which means you can get the first token for the next stream in the next lane over
<zyp[m]>
you might still want to have enable bits or similar though, so you don't have to actually start a new packet
<whitequark[cis]>
if you look at most protocols that operate at reasonable speeds, like PCIe up to Gen3 at least, the packet starts in lane 0
<zyp[m]>
and by two ways, I don't count alternative ways of signalling last that's functionally similar
<whitequark[cis]>
I think a last bit per lane is actually unrealistic
<whitequark[cis]>
when you can transfer multiple packets per clock cycle, i am certain there will be limitations on where the packets can start
<whitequark[cis]>
so it would be something like last bit per every 8th lane
<whitequark[cis]>
(of course you could say this is 1/8 as many lanes)
<zyp[m]>
a last bit per lane is what you'd effectively get if you put last in the payload and upconverted it
<cr1901>
>one way is to have a last bit on the transfer, and some way of telling you how many lanes have data for this last transfer Yes, 0b1010 w/ a last bit is probably "undesirable", I just maybe took "0b1010 is illegal" too literally
<whitequark[cis]>
I mean, if it has no specific defined meaning, it is (or should be) illegal
<whitequark[cis]>
and if you do give it the meaning of "pluck these random bytes" you realize that two thirds of your FPGA is spent doing that
<cr1901>
>oh, no, you still need last <-- Putting aside your "length can be 0 to n inclusive" proposal 1/2
<cr1901>
length of 0-(n-2) means "this is the last transaction of a packet", length of n-1 means "packet continues, except when last is asserted"?
<whitequark[cis]>
undefined
<cr1901>
Yea, I don't get where you're going with this. I will bow out until an RFC is avail
<cr1901>
I'll have plenty of time to catch up then
<whitequark[cis]>
this won't be in the RFC
<whitequark[cis]>
what I'm saying is that if you have both length and last, there are some combinations that make no sense
<whitequark[cis]>
for example, length==0 and last==1
<zyp[m]>
I disagree, I've wanted that before
<whitequark[cis]>
you're right; and actually USB does exactly that
<zyp[m]>
«I want to terminate a packet, but has no more data to send»
<zyp[m]>
and yeah, also to signal a ZLP
<whitequark[cis]>
okay, so I guess all combinations of last + length are valid
<whitequark[cis]>
(I'm tired)
<cr1901>
Okay, lemme just ask the question I actually want to ask with the risk that I'm a dumbass and missed it
<cr1901>
>oh, no, you still need last
<cr1901>
if you want to be able to send a stream of lane-count-multiple sized packets back to back
<cr1901>
Wouldn't you need last if _any_ packet ends on a lane-count-multiple?
<zyp[m]>
the problem with defining stream semantics is that there's so many things that make no sense for some applications, but are very useful for others
<zyp[m]>
even first vs last
<cr1901>
Maybe a table tallying up length, last, and semantics for a n lane serialized packet stream would make it clearer for me
<zyp[m]>
I think the suggestion you're responding to was the suggestion that for e.g. a 4-lane wide stream, a length of 4 would signal a full transfer that continues, while a length of 0 would signal a full transfer that ends the packet
<zyp[m]>
which takes 3 bits to communicate, and is effectively the same as a two-bit length (0-3) and a last bit, just remapped
<cr1901>
Couldn't you do a length of 3 as well, with one less xfer of a length of 4?
<zyp[m]>
it'd have the advantage that it saves a bit on a non-power of two width, but otherwise doesn't seem very worthwhile
<zyp[m]>
the issue is that you need to distinguish «here's the last 4 bytes of the packet» and «here's another 4 bytes of the packet», in addition to «here's the last {1, 2, 3} bytes of the packet»
<whitequark[cis]>
thanks zyp
<cr1901>
zyp[m]: Thinking, it makes sense. Figured out why I got confused at least. Essentially length is a Rust enum with variants {Last4=0, Last1=1, Last2=2, Last3=3, Continue=4}
sauce has quit []
<cr1901>
But still, any packet Y _followed by_ "a packet X whose length is a multiple of the number of lanes" requires Last4 to signify the end of X, correct?
<zyp[m]>
Catherine: by the way, what would the cost be of letting the stream payload be either a shape castable or a signature?
<zyp[m]>
I'd like to e.g. make zip and unzip blocks for the payload that takes two or more stream signatures and derive a combined stream signature, and I don't like the risk of them coming up with different orders of signals in a StructLayout
<zyp[m]>
although when I think about it, that's solvable by simply sorting it by name
<cr1901>
Disregard my last question, I misread wq's msg that I linked