<whitequark[cis]>
getting very close to the theoretical maximum of about 98.7% utilization
<whitequark[cis]>
what i'm unsure about is how the sum of two counters seems to approach 99.5% here, which would be physically impossible if the other MAC uses the proper preamble+IPG length
<whitequark[cis]>
i think this might be happening because performance counter readout isn't atomic
<_whitenotifier-6>
[glasgow] github-merge-queue[bot] created branch gh-readonly-queue/main/pr-561-8c9b0b7793b810efbc29f840dc85eb94d1ab3d8f - https://github.com/GlasgowEmbedded/glasgow
<whitequark[cis]>
you know, so much of glasgow is written to an incredibly high standard because i have mild OCD (this is not a good thing) and avoid pushing things that aren't as close to perfection as i'm willing to do without getting completely overwhelmed
<whitequark[cis]>
the high standard this produces is a big part of the appeal, everything works out of the box and in a way that makes people happy
<whitequark[cis]>
but it doesn't scale, and i cannot in good faith ask anyone else to do this, because it's just an insane way to function
<whitequark[cis]>
i don't know what to do about this. sure, i will happily recruit, mentor, and give ownership to anyone willing to do the same, but it's such a small subset of population
<whitequark[cis]>
i cannot blame anyone for not holding themselves to the standard of "world leading tech for this one-off applet they probably just want to solve their one problem asap"
<whitequark[cis]>
i know how to compromise, but compromising a lot eventually drains my desire to participate in the project, and given my core role in it that's not really a solution either
<whitequark[cis]>
the amount of effort required to go from "good for 95% of practical purposes" to "nearly perfect" rises exponentially as you close in on the latter, and people just... don't have time. they have day jobs, kids, significant others, and all of those things are more important than my desire for perfection
<whitequark[cis]>
this doesn't even take into account that most people don't have the technical expertise to do this in first place. I believe that anyone can acquire it if they so desire, but also it's completely reasonable to just. Not desire that.
<whitequark[cis]>
<whitequark[cis]> "you know, so much of glasgow..." <- it's difficult to even say that because many core parts (FX2 arbiter, the awful I2C register interface, non-zero-copy-capable USB interface...) are things i feel are sub-par. I know that they're so good that most people don't even notice there are flaws until they push it to the absolute limits, but knowing that there *are* flaws bothers me
<whitequark[cis]>
they are clearly "good enough" in some sense because building an Ethernet MAC (including some unrelated infrastructure) from scratch in 4 days is basically unheard of. you will not find an HDL developer willing to commit to that, and I expect most to find the idea fundamentally unserious
<whitequark[cis]>
* they are clearly "good enough" in some sense because building a great (high-performance, low-latency, no-bufferbloat) Ethernet MAC (including some unrelated infrastructure) from scratch in 4 days is basically unheard of. you will not find an HDL developer willing to commit to that, and I expect most to find the idea fundamentally unserious
<whitequark[cis]>
* they are clearly "good enough" in some sense because building a great (high-performance, low-latency, no-bufferbloat) Ethernet MAC and its driver (including some unrelated infrastructure) from scratch in 4 days is basically unheard of. you will not find an HDL developer willing to commit to that, and I expect most to find the idea fundamentally unserious
<josHua[m]>
so I was actually going to DM you like the other day to see if you had any ideas about, like, how one ought handle this in projects in general. I am working on a project that is definitely a hobby project for me, but I hold a high standard for code that lands in it. I have contributors who can be more prolific than I in terms of the time they have, but then I am faced with the question of, like, whether I am prepared to take changes
<josHua[m]>
that do not meet my code quality standards (some, architecturally; some, bits and bytes in terms of whether, like, indentation is consistent), or whether I instead am going to stand in the way of getting things done by enforcing a standard that I do not have time at this moment to write code that meets
<whitequark[cis]>
some of the things that help are explicitly written guidelines and standards that guide people to build things that are good right away and don't need rewriting; templates, essentially
<whitequark[cis]>
josHua: this is an absurdly hard problem that i do not have a solution for
<whitequark[cis]>
i am better at this than i used to be, because i actually communicate my requirements/preferences
<whitequark[cis]>
which is a big step up from simply sitting on a PR i don't like and not explaining why i don't like it
<josHua[m]>
yeah
<whitequark[cis]>
it turns out that people who really like a project are willing to go quite far to fulfill even seemingly unreasonable requirements
<whitequark[cis]>
but there are limits for that
<josHua[m]>
I sometimes feel like I am nitpicking even still, which I could imagine to be incredibly frustrating to be on the other end of
<josHua[m]>
(but, like, sheesh, shell scripts in production.)
<whitequark[cis]>
I think nitpicking can be fairly well received if you kinda just give people orders. like if they don't put their ego (I mean in a neutral sense) in their code they spend a minute changing it to whatever you want instead
<whitequark[cis]>
nitpicking doesn't work if you just go "I don't like this". wtf is a contributor going to do here? get upset about the intractable situation
<josHua[m]>
yeah
<whitequark[cis]>
it's still friction, but hitting "commit suggestion" isn't a lot of friction
<whitequark[cis]>
I used to feel guilty about it but these days I just go for it. you hit commit suggestion, I hit merge, everyone is happy
<tpw_rules>
(as the receiver of such i get annoyed when it means an extra week/month for the PR to be accepted. that does not seem to be a big issue with these projects)
<whitequark[cis]>
what I can't really deal with is people who can't pick up what I want from what I'm asking to change, because it results in having to repeat the same fundamental things over and over
<josHua[m]>
with this particular project that is somewhat more difficult for truly awful reasons (like, 'the thing that is versioned in git is actually a diff')
<josHua[m]>
yes, that is I think the problem I am facing in this particular case
<sapphire_arches[>
would out of tree applets help? since then there's a path for 60-95% hacks to live somewhere
<sapphire_arches[>
though it's maybe counter to the goal of building an instrument and the priesthood required to care for it, as there's less incentive to ever do the last 5%
<josHua[m]>
every single time, I expect every commit to be tested, and indentation to be correct each time
<sapphire_arches[>
and designing a stable API is real difficult
<whitequark[cis]>
it seems like the level of annoyance is basically "stalled PR > unclear instructions > lots of nitpicking > yolo merging"
<whitequark[cis]>
so I try to avoid stalled PRs as much as possible even if it means doing other things that seem like I'm being unfair
<josHua[m]>
nod
<whitequark[cis]>
but like... I think of it this way, the purpose of someone submitting a PR is to get it merged
<josHua[m]>
right.
<whitequark[cis]>
they might feel slightly bad about nitpicking but they get what they want in the end and that makes a happy contributor
<whitequark[cis]>
it took so long to learn this
<whitequark[cis]>
sapphire_arches[: I just can't commit to that right now
<whitequark[cis]>
nightmarishly difficult even in something like Rust (so many projects ignore the problem), worse in Python, extremely difficult if USB is involved
<whitequark[cis]>
the demultiplexer can't do zero-copy into user provided buffers and this prevents the ethernet applet from doing 100/100 instead of like... 80/80
<tpw_rules>
stable API?
<whitequark[cis]>
80/80 is still incredibly good for something you can make from e-waste and fourteen wires
<whitequark[cis]>
I don't think anyone actually expects 80/80 with sub-20ms latency from the glasgow applet
<whitequark[cis]>
the standard is set by tools like buspirate, which will architecturally never manage this. not even within an OOM, probably
<whitequark[cis]>
my TODO list is like "write DPDP in less than a week but also good and readable and easy to get started with"
<whitequark[cis]>
s/DPDP/DPDK/
<josHua[m]>
hm, ok, there is actually one piece of advice that I have that I am learning slowly, and I'm not sure if I like it, but it hit me over the head pretty hard recently with some work I am doign for one of my clients (on 1b2 discord, this saga has been in #eda-and-pcb ), and I think this almost certainly applies for Glasgow
<whitequark[cis]>
to anyone familiar with the domain that problem statement that sounds outright absurd
<whitequark[cis]>
like, I am already succeeding at things that as far as I know have never been done before
<whitequark[cis]>
but like, wouldn't it be nice if we had DPDK that's not a pain to use, right?
<josHua[m]>
I have been doing a bunch of reworks to bring a fatally flawed board up as much as possible, and I would consider a lot of these reworks to be in the 'heroic' category. I think the client does not understand the degree of 'heroic rework' that is involved here (this is like 'two hours under a microscope' stuff for me). as a result, they have sort of come to the expectation that the heroic is ordinary. 'oh, these four differential pairs
<josHua[m]>
are swapped to the wrong receivers, we can just swap them all back', that sort of thing.
<whitequark[cis]>
oh, yeah, this is really bad with contract work (and full time too)
<whitequark[cis]>
(but especially contract(
<whitequark[cis]>
s/(/)/
<gsuberland>
gotta admit, I'm still kinda reeling from the whole "you wrote a functioning NIC in Python in a few days" of it all, let alone it managing 80/80 data rates consistently.
<whitequark[cis]>
it's different with OSS/OSHW though
<josHua[m]>
I was doing a relatively mundane rework by the scale of these kinds of things, 'just' tacking some 34awg wire down to a pin, and I found myself frustrated that I could not convince my hands to behave, like, *really* frustrated
<josHua[m]>
like I had not had too much caffeine, and I had not even had the ADHD meds that morning, so my hands should have been stable but I was just having quite some difficulty pulling it off
<whitequark[cis]>
gsuberland: my headmate's coworker once asked her how she manages to work this fast and she replied, dead serious, "I didn't want to work slowly"
<gsuberland>
lol fair
<whitequark[cis]>
I don't think she realized exactly what she is saying
<whitequark[cis]>
but it's. kind of. same idea I think
<josHua[m]>
and I realized that not only did the client expect me to do heroics as the normal, which is fine (sort of), but that I expected myself to be able to pull off heroics as the normal, and that is a fundamentally untenable concept
<gsuberland>
I can certainly relate to the "completing in one evening what a whole team usually take weeks to do" side of things, because ADHD.
<whitequark[cis]>
yes. good on you for realizing it
<josHua[m]>
so I think this is the bit that applies to OSS/OSHW. like pulling off this RGMII MAC absolutely is 'heroics'
<whitequark[cis]>
the way I addressed that is twofold: first of all, working on glasgow should bring me joy instead of fear/guilt/etc. second, I will build Glasgow in a way such that working on it consistently brings me joy
<whitequark[cis]>
building the RGMII MAC was relaxing
<gsuberland>
ah, yes, the Marie Kondo approach.
<whitequark[cis]>
my life is better than if I did not do it
<gsuberland>
except backwards I guess
<josHua[m]>
yeah, I understand also the way in which flowy stuff like that is relaxing
<gsuberland>
"this will spark joy"
<whitequark[cis]>
gsuberland: what if marie kondo was an angel girl
<whitequark[cis]>
"this will spark joy or it will stop existing. this is a threat"
<gsuberland>
I'll take "describe whitequark in one sentence" for $100, Bob.
* whitequark[cis]
giggles
<gsuberland>
:D
<josHua[m]>
when it works, anyway, and wheny ou can get yourself to do it. the problem is that if you use enjoyable flowy things like that (or, well, I am projecting -- if I am using something like that) as an excuse to say that other heroics -- especially writing everything to a high standard -- are what you must expect of yourself at all times
<whitequark[cis]>
actually, i built most of the existing applets in a truly horrible living situation, no housing security, no job security, so much anxiety i couldn't even go outside for a literal year, and it was very cathartic to be able to not just do things, but do things that have value
<josHua[m]>
indeed.
<whitequark[cis]>
josHua: I think there's a triangle of sorts. "built to a deadline", "built to a high standard", "does not have a high chance to contribute to burnout"
<gsuberland>
josHua[m]: I definitely feel you with that one. the constant overanalysis of prose and rewriting stuff over and over until it meets a high enough standard to feel like you are expressing what you want to express in a clear and cogent manner.
<whitequark[cis]>
you can get two out of three
<gsuberland>
oh jebus yes, 100% there.
<whitequark[cis]>
gsuberland: oh, you have *no idea* (maybe a good idea) how much eraser i have taken to the amaranth docs
<josHua[m]>
yeah, I think that's the case
<whitequark[cis]>
I write stuff then I delete every word that doesn't seem to convey information
<whitequark[cis]>
unfortunately this made the documentation very good and as a result contributions almost never meet the standard that's now implicitly set
<whitequark[cis]>
this sucks! I want other people to write docs too
<whitequark[cis]>
I've been doing literally nothing but writing Amaranth docs for literal weeks
<josHua[m]>
right.
<whitequark[cis]>
unfortunately programmers don't generally know how to write docs much less really good docs
<gsuberland>
back when I was pentesting I burned out *HARD* trying to do high quality work and write good reports to unreasonably short deadlines, made worse by the fact that half of the people I was doing this for not only did not appreciate it but were actively upset by me doing it.
<whitequark[cis]>
I didn't know it and I avoided it like hell until just biting the bullet and learning how to do it by jumping in and figuring it out
<whitequark[cis]>
gsuberland: yeah that is so relatable
* tpw_rules
has been unfairly accused of being good at writing...
<sapphire_arches[>
is there a reason Glasgow needs to proceed at any particular pace?
<josHua[m]>
oh yeah.
<whitequark[cis]>
the way I see it: I work to achieve my own goals. the client or the company contracts me to achieve theirs. in general, there is nothing that aligns their goals with mine, so either I create that alignment or I just work-to-rule
<sapphire_arches[>
the hardware itself is like 2 years behind "schedule" but like, that's fine, whatever, i intend to be here long enough for that to be insignificant and found other things to do in the mean time
<whitequark[cis]>
they can't really complain about work-to-rule and if they expect highest quality work without creating an environment conducive to creating high quality work it's time to quit
<sapphire_arches[>
OSS/OSHW is a gift, if you want it done faster or differently pay someone to do it
<whitequark[cis]>
sapphire_arches: the only real constraint is: contributors get upset/alienated if their PR stagnates for 2-3 years
<whitequark[cis]>
which is understandable and I empathize
<whitequark[cis]>
having 2000 glasgow devices shipped naturally creates more potential contributors
<gsuberland>
I guess also useful to strike while the iron is hot, so to speak, both in terms of personal motivation and while others are most actively interested
<whitequark[cis]>
yeah
<gsuberland>
although speaking of docs, I should try to figure out what's going on with conf.py and the link validation stuff
<whitequark[cis]>
despite some people jumping to conclusions out of frustration, I recognize all this
<gsuberland>
I rebased with the reordered commits but no luck, actions still fails on that validation step
<whitequark[cis]>
I just don't know what to do with it, given like... severe enough disability I'll probably need a cane to walk for the short term future
<whitequark[cis]>
for a lot of my build system / packaging shenanigans i was able to basically completely automate it
<whitequark[cis]>
yowasp essentially runs itself, which i think is a remarkable achievement in its own right
<whitequark[cis]>
especially given how it creates version specific branches
<gsuberland>
I have concluded that my brain is incompatible with CI configuration. I just cannot.
<whitequark[cis]>
it's not maintenance-free but it's not like... debian package level maintenance
<whitequark[cis]>
gsuberland: in this situation i usually apply more force until the brain becomes compatible
<gsuberland>
I think it's probably the high friction and repeated delays in testing config changes that kills it for me. if I could somehow do it instantly on the command line locally without some massive onerous setup first I could probably manage it ok.
<gsuberland>
right looks like the PR is fully passing tests now :D
<whitequark[cis]>
gsuberland: this is why most of yowasp's ci consists of `./build.sh`
redstarcomrade1 has joined #glasgow
<whitequark[cis]>
the rest i try to build in a way that doesn't require attention
<whitequark[cis]>
it's hard yeah
<gsuberland>
my frustration intolerance certainly doesn't help either lol
<whitequark[cis]>
i seem to have a tolerance for frustration that;s higher than basically anyone i know
<gsuberland>
half the reason I can't manage owning 3D printers either
<whitequark[cis]>
considering that i willingly combined cross-compilation, FPGA tooling, and Python packaging together, and ship it
<gsuberland>
yeah I think I'd end up throwing things out of a window if I tried that lol
<whitequark[cis]>
i've been cross-compiling things since before i could drink
redstarcomrade has quit [Ping timeout: 256 seconds]
<whitequark[cis]>
that statement makes those two things sound related. hm
<gsuberland>
haha
<whitequark[cis]>
by the way, the packet queue used in the glasgow NIC is quite interesting
<whitequark[cis]>
it's only 2K in size, which sounds weird, because it's not a multiple of 1500
<whitequark[cis]>
however, the way it works is that you can keep pushing in a packet while there's still data space in the queue, assuming no runt frames
<gsuberland>
oh so basically it ends up being enough space so you can push a packet while another is in the buffer, and the buffer size is a nice round number for easy management?
<gsuberland>
iirc you were also planning something with 16K MTUs too right, for the SEM SCSI stuff?
<whitequark[cis]>
e.g. you can push in a 1500 octet packet, then start pushing another one and until it gets absolutely out of space you get to keep doing it. so if at the same time you're transferring it to USB, and you manage to finish that before you get to the first third of the second packet, both of them go in
<gsuberland>
nice
<whitequark[cis]>
it basically speculatively assumes that something from the buffer will be read out before it runs out of space
<gsuberland>
:D
<whitequark[cis]>
and it's a proper packet queue where a packet is atomically committed or discarded
<Wanda[cis]>
the queue also has a funny commit/rollback thing where it resets write pointer (discarding theinflight packet) if it runs out of space or CRC turns out to be bad
<Wanda[cis]>
(which also means you don't get to start reading until the data is committed)
<gsuberland>
neat idea
<Wanda[cis]>
actually we share the exact same queue gateware for both directions, because it just happens to work well for both of their requirement sets
<whitequark[cis]>
as a result this NIC has excellent performance despite objectively tiny FPGA-side buffers (2048 for packet queue with 32 (2048/64) metadata entries, 512 for the FX2 interface)
<Wanda[cis]>
(for transmit you really want the whole frame to be in the buffer before you start TX, so that you don't get surprised pikachu face when you run out of data in the middle of packet, so we also do the commit thing for TX)
<whitequark[cis]>
like, timonsku posted a datasheet for a NIC that can do 30 Mbps on a good day and that has 16K of SRAM
<Wanda[cis]>
(and in RX direction the commit thing accomplishes FCS checking)
<whitequark[cis]>
vs 5K that we have
<gsuberland>
hah
<whitequark[cis]>
7K if you also count the FX2 buffers, if I recall correctly
<gsuberland>
dang now I'm wondering what'd happen if we set y'all loose on a 200GbE platform
<whitequark[cis]>
realistically, there is no need in the 512-byte buffers for the FX2 buffers, we could skip them at all with a slightly better interface
<whitequark[cis]>
so 6K total on-device buffers is feasible and should be possible to get 100M both sides concurrently
<whitequark[cis]>
200GbE gets kind of annoying because you want really wide datapath in the fabric
<whitequark[cis]>
like you could realistically have a 256 byte (2048 bit) wide datapath, which lowers your operating frequency to fairly tolerable 100 MHz
<whitequark[cis]>
this requires you to be able to handle four packets per word
<whitequark[cis]>
or you'll have 75% drop at a stream of back-to-back min-sized packets
<whitequark[cis]>
well, slightly less due to preamble and IPG
<gsuberland>
heh I actually hadn't thought about this that much but yeah I guess there's a reason the switching stuff is almost all pure ASIC and the frontends are essentially TDMA'd
<whitequark[cis]>
you could do 1024 bit wide, but that still requires you to handle two packets per word, but now you have half the delay to do it
<gsuberland>
not like you can clock the logic at 200GHz
<whitequark[cis]>
nothing in 200 GbE runs at 200 GHz
<gsuberland>
oh right yes derp PAM4
<whitequark[cis]>
as far as I know it's multiple lanes even after the PHY
<whitequark[cis]>
and yes PAM4
<whitequark[cis]>
it looks like the minimum for 200GbE is two-lane operation
<gsuberland>
I think it's 56GHz fundamental iirc
<whitequark[cis]>
uyeah
<whitequark[cis]>
* yeah
<whitequark[cis]>
that's still a fuckton of GHz but it's not as high as that
<gsuberland>
which is still utterly bonkers to clock logic at but yeah
<whitequark[cis]>
and the datapath out of the SerDes is very wide, even in an ASIC
<gsuberland>
yeah makes sense
<whitequark[cis]>
i'm not sure if you clock the logic as high as that
<whitequark[cis]>
I'd expect something like equivalent time samping
<whitequark[cis]>
s/samping/sampling/
<gsuberland>
yeah I took a look at some of the Intel presentations when they were ramping up in that space and it's basically a shitload of multiplexing
<whitequark[cis]>
like you have (idk) 16 PAM4 (de)modulators and they're connected via delay lines of varying width
<gsuberland>
yep
<whitequark[cis]>
I've never seen them I'm just theorizing based on what I know about the space
<whitequark[cis]>
1.6TBASE, holy crap
<gsuberland>
that's pretty much it from what I've seen. one of the companies they acquired had patents on the lithography stuff for producing reliable microwave delay lines with suitable cross-line isolation.
<gsuberland>
lol yeah that came out a while back, there are even optics you can buy for it
<gsuberland>
there's also some stuff you can do with Faraday rotators for similar trickery
<whitequark[cis]>
this is nuts
<gsuberland>
:D
<omnitechnomancer>
High numbers of gigabit ethernet are where the PCS and PMA interfaces start getting more complex
<omnitechnomancer>
s/interfaces/layers/
<gsuberland>
re: that Nature article, iirc there's already a commercial VCSEL-based offering for building it into CPO ASICs
<gsuberland>
I recall it being mentioned in a TbE working group presentation somewhere but I can't recall which one
<omnitechnomancer>
Above 100GbE is also where you get mandatory Reed Solomon FEC
<gsuberland>
yeah it's RS-FEC, 256/257b line code, NRZ on 100GbE
<gsuberland>
then PAM4 on 200GbE
<omnitechnomancer>
There is a 2 lane 100G form that uses the 56Gbps SerDes with PAM4
<gsuberland>
yeah they ahve different generation numbers for the different ones, but I always forget what they are
<gsuberland>
*have
<gsuberland>
I remember 1st gen is 10x10G but that's ancient and nobody uses it
<gsuberland>
and then I think 2nd gen is 4x25G? so I guess 2x50G is 3rd and 1x100G is 4th? but could be some other weirdness in between
<gsuberland>
it gets a bit strange because you've also got the oddball data rates from the 40G IB stuff
gundy9558[m] has quit [Quit: Idle timeout reached: 172800s]
GNUmoon2 has quit [Remote host closed the connection]
GNUmoon2 has joined #glasgow
<theorbtwo[m]>
So related to the conversation earlier about how you deal with stalled PRs and maintain the quality of the project, IMHO, the thing is to have two pathways for applets, with two different target quality levels.
<theorbtwo[m]>
One for in-tree applets, which should be the quality level of "as if whitequark wrote them to full ocd", and one for out-of-tree but mentioned in an in-tree document, which has a much lower quality requirement.
<whitequark[cis]>
this is basically the plan i decided on in 2020
<whitequark[cis]>
unfortunately it requires a stable API, which isn't feasible for a good while
<Wanda[cis]>
(the thing with 6 ones is calle bitstuffing btw)
<Wanda[cis]>
s/calle/called/
<Wanda[cis]>
lmao
<gruetzkopf>
USB 3 SHOULD not exhibit that behavior but..
<whitequark[cis]>
usb 3 uses 8b10b or 128b130b, depending
<omnitechnomancer>
CAN does bit stuffing too
<jn>
hmm, the anatomical metaphor (ventral/dorsal) seems to imply that the natural orientation of a glasgow is with its probe sockets and LEDs facing down
<whitequark[cis]>
to be honest i only submitted the PR because it sounded fun and someone suggested that as terminology for chip orientation in a rust embedded channel
<whitequark[cis]>
it's probably not actually a good fit for glasgow, nor would it even be valuable
<whitequark[cis]>
plus i don't really have the goal of confusing everyone who isn't a biologist or a doctor or something
<jn>
ah, i see :)
<jn>
well if it *was* the goal, it would be well achieved, as a non-biologist i was confused
<Attie[m]>
😂
<_whitenotifier-5>
[glasgow] github-merge-queue[bot] created branch gh-readonly-queue/main/pr-557-ff7f6f26ecc12a1d3804adba9326d268b0f7aa39 - https://github.com/GlasgowEmbedded/glasgow
<sorear>
"many core parts (FX2 arbiter, the awful I2C register interface, non-zero-copy-capable USB interface...) are things i feel are sub-par." is there a comprehensive list of these somewhere?
jstein has joined #glasgow
redstarcomrade has quit [Read error: Connection reset by peer]
siriusfox has quit [Quit: ZNC 1.8.2+deb2build5 - https://znc.in]
siriusfox has joined #glasgow
<carlomara[m]>
Just fine. I have the feeling that the issue is I'm abusing the control registers write
<carlomara[m]>
But it's an inkling, nothing a hard certainty
<esden[cis]>
<Attie[m]> "carlomara: sounds odd... how..." <- This is what carlomara was referring to. (Discord replies are not propagated back to matrix :( )
GNUmoon2 has quit [Remote host closed the connection]
GNUmoon2 has joined #glasgow
gsuberland has quit [Remote host closed the connection]