<sonny>
Do you have to make a custom program on both sides when using serial? Inside the guest there will be a driver, and outside you can talk from the host using system interfaces?
<clever>
sonny: ive done that for xmodem transfer of files during boot, but uart turned out to be pretty darn slow
<sonny>
is uart slow or is xmodem slow?
<kazinsal>
little of column a, little of column b
<sonny>
I tried xmodem once and I thought it wasn't working ... ended up deleting the cisco OS
<clever>
sonny: the uart is slow, relative to usb
<klange>
General serial speeds top out in the kilobytes-per-second range.
<sonny>
oof
<kazinsal>
xmodem is also a very barebones protocol
<kazinsal>
small block size, minimal checksumming
<clever>
xmodem also hit many bugs in my context switching routines
<clever>
due to constant irq's while also scheduling
<clever>
prior to that, the irq load was very low
<sonny>
guess I'm doing polling then
<kazinsal>
you send 132 bytes (3 bytes of header, 128 bytes of data, 1 byte of footer) and then you wait for an ack or a nak
<kazinsal>
so you have a very low transfer efficiency
<kazinsal>
zmodem is a bit better because it can just fire off packets and only retransmit when it gets a nak
<kazinsal>
and it has some better header metadata allowing rewinds etc
<clever>
yeah, having a window and seq/ack would speed it up
<clever>
kazinsal: does zmodem retransmit starting from the nak, or only the missed packet?
<kazinsal>
you send it a NAK <position in file> and it rewinds and retransmits sequentially from there
<clever>
ah
<clever>
so it will also repeat data you already had
<kazinsal>
yeah, not the most efficient
<clever>
but less state for the receiver to deal with
<bslsk05>
github.com: lk/uart.c at pl011 · librerpi/lk · GitHub
<clever>
kazinsal: if the cbuf is full, this will disable the rx irq, then the uart hw fifo will fill up, and hw flow control can do its job (if wired)
<clever>
without that, receiving data too quickly would cause bytes to just be lost silently
<sonny>
clever how did you figure that out?
<clever>
sonny: which part?
<sonny>
the sleep
<sonny>
my bad, read something else
srjek has quit [Ping timeout: 240 seconds]
elastic_dog has quit [Ping timeout: 256 seconds]
eddof13 has joined #osdev
srjek has joined #osdev
elastic_dog has joined #osdev
sonny has quit [Ping timeout: 256 seconds]
sonny has joined #osdev
k8yun has quit [Quit: Leaving]
eddof13 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
sonny has quit [Quit: Client closed]
Matt|home has joined #osdev
Ali_A has joined #osdev
nyah has quit [Ping timeout: 252 seconds]
Arthuria has joined #osdev
Ali_A has quit [Quit: Connection closed]
seds has quit []
seds has joined #osdev
srjek has quit [Ping timeout: 240 seconds]
wand has quit [Remote host closed the connection]
wand has joined #osdev
sonny has joined #osdev
Jari-- has joined #osdev
<Jari-->
morning all
<Jari-->
still thinking about putting up software company
gog has quit [Ping timeout: 250 seconds]
<geist>
sounds like a lot of work
sonny has left #osdev [#osdev]
MiningMarsh has quit [Ping timeout: 240 seconds]
MiningMarsh has joined #osdev
sonny has joined #osdev
sonny has left #osdev [#osdev]
eroux has quit [Ping timeout: 272 seconds]
matrice64 has joined #osdev
matrice64 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
matrice64 has joined #osdev
matrice64 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
_xor has joined #osdev
Arthuria has quit [Ping timeout: 240 seconds]
jjuran has quit [Ping timeout: 250 seconds]
jjuran has joined #osdev
xenos1984 has quit [Read error: Connection reset by peer]
xenos1984 has joined #osdev
jjuran has quit [Ping timeout: 272 seconds]
<kazinsal>
once in a while I think about building up a network hardware company but then I realize I am not a rich man
Jari-- has quit [Ping timeout: 272 seconds]
alpha2023 has quit [Read error: Connection reset by peer]
jjuran has joined #osdev
alpha2023 has joined #osdev
GreaseMonkey has quit [Read error: Connection reset by peer]
<kazinsal>
oh neat, Dave Plummer (retired Microsoft systems engineer, wrote some of the MS-DOS drive cache and interface stuff and a bunch of stuff in NT 3.x and 4.0) has a youtube channel where he talks about systems stuff and rebuilds vintage computers
<papaya>
What editor/ide do ya'll like to use for osdev? I've been using vscode but have thought about moving over to clion
<klange>
Mine.
<kazinsal>
vscode
<papaya>
klange: does yours run on your own OS, or did you custom build one to run on say, linux?
<kazinsal>
iirc it'll run on anything with vt100 control code support
<papaya>
klange
<papaya>
klange: is that the bim editor on your gitlab?
<klange>
I don't know how up to date the gitlab mirror is, I moved back to github a while ago and only occasionally remember to sync things
<papaya>
ah, why the change?
<klange>
but yes, that is my editor; it runs on my OS, it runs on Linux, it runs on macOS, older versions ran on pretty much anything with a capable terminal, newer versions have another dependency that make things trickier (my language)
<kazinsal>
but kuroko can be ported to just about anything with a bit of elbow grease, so
<kazinsal>
(often minimal elbow grease)
<klange>
depends on if you want thread support, though the editor doesn't need it, so yeah
<kazinsal>
(eventually I will port it. eventually I will also fix the rest of my OS's scheduling and user system)
<klange>
papaya: i moved to gitlab when microsoft bought out github, and I moved back when I was content that microsoft hadn't completely screwed it up
<papaya>
klange: fair lol, I just use gitlab b/c its what I use at work also
<kazinsal>
surprised you haven't pointed kuroko-lang.org there yet
<kazinsal>
(disclaimer: I do not know what's at that link)
<klange>
nothing, thankfully
<klange>
but now that you've said it here expect a bot to pick it up soon enough
<kazinsal>
yeah looks like it's not yet taken
<klange>
papaya: My editor was originally pure C. I wanted something more dynamic to write configuration scripts and syntax highlighters in, so I read through Crafting Interpreters and somewhat accidentally ended up building a near-complete Python interpreter from scratch.
<clever>
i also wanted more dynamic config files, so i just ported lua, lol
<clever>
depends on if your goal is the goal, or the journey
<klange>
The hidden part of that goal was that I needed to write something myselef, with zero external dependencies, as per the rules of the OS project.
<klange>
And you might think that saying I accidentally ended up with a near-complete Python... is colorful hyperbole, but really, seriously, going in I just wanted to do CLox with significant whitespace, and things snowballed from there.
<clever>
and i assume you wrote those rules like that, so you could have fun writing everything from scratch?
<CompanionCube>
did you ever port bim to uefi or was it just kuroko
<klange>
Pre-Kuroko I had attempted it, but otherwise no, just Kuroko got an EFI port.
papaya has quit [Remote host closed the connection]
<mrvn>
It's not like the round trip time on serial is measurable compared to transmitting 132 bytes. Even if you think long distance modem calls.
<mrvn>
kazinsal, sonny: ^^ regading xmodem/zmodem
the_lanetly_052 has joined #osdev
<clever>
mrvn: but there is also the cpu overhead of processing the packet and sending an ack, and scheduler latency on both ends
<clever>
might miss one or 2 bytes per packet
<clever>
as-in, 2 bytes of dead-time when your not transmiting
<mrvn>
clever: That would be <2% slowdown then.
<mrvn>
2 bytes deadtime seems awfully high for modern systems though.
<clever>
checking the math, 115k baud, is 11,520 bytes/sec assuming perfect spacing
<clever>
132 byte packets with 128 bytes of payload, then brings you down to 11,170 bytes/sec
<mrvn>
That's 0.1 ms.
<clever>
now, lets assume 2 bytes of dead time, even if thats horrible
<clever>
that brings you down to 11,004 bytes/sec
<clever>
and yeah, thats nothin
<mrvn>
compared to even one file upload with flipped bits that loss is still a gain.
<clever>
in a perfect scenario, only 96% of the bytes are payload, due to xmodem headers
<clever>
with 2 bytes of deadtime, thats down to 95%
<mrvn>
And you mentioned zmodem doesn't stop sending waiting for an ACK. So that 2 bytes is gone as well.
<clever>
but if you only wait for a NAK, you could waste time sending data when the receiver isnt even there
<clever>
tcp solves that with a scaling window
<clever>
where the ack can be up to $window bytes behind what you last sent, and it will still be happy
<clever>
allowing for $window bytes of un-ack'd data to be in-flight
<mrvn>
One thing you didn't figure in though is the time it takes to transmot the ACK. How big is that?
<bslsk05>
github.com: lk-overlay/stage1.c at master · librerpi/lk-overlay · GitHub
<clever>
looks like every control message is a single byte
<mrvn>
proper serial has hardware flow control. Can't send when nobody is prepared to listen.
<clever>
the aux-uart on the rpi lacks true hw flow control, but it could be emulated with gpio
<clever>
the pl011 is what i use though, and it does have the option for true hw flow control
<mrvn>
Nowadays hardware is so fast that having the serial FIFO run full is unheard of. So no need for flow control.
<clever>
the qemu pl011 driver in lk, will also copy bytes from the hw fifo, to a software fifo
<clever>
and if the software fifo fills up, it turns off the rx irq, and stops reading the hw fifo
<mrvn>
like on the RPI default uart with just send/recv/gnd.
<clever>
so threads not reading the fifo, can provide back-pressure over the flow control
papaya has joined #osdev
<mrvn>
clever: With IRQs set at 50% FIFO you have 0.5ms to read out data from serial. That's over 500k cpu cycles at 1GHz.
<papaya>
klang's os runs doom, gets stamp of approval
<clever>
mrvn: but the VPU is only running at 500mhz, so 250k cycles then
<clever>
and yeah, thats huge
<clever>
even the largest vector opcode isnt that slow
<mrvn>
If you are stuck in IRQ handlers for 250k cycles then you have other problems. :)
<clever>
yeah
<clever>
i could see the problem being more from handling gzip poorly
<clever>
once you get a full gzip block, you stop receiving to decompress, and then the fifo overflows
<mrvn>
And as lk does your IRQ handler should just copy the hardware FIFO into a software FIFO and that would be 4k or something like that. AGES to fill up.
<clever>
#define RXBUF_SIZE 16
<kazinsal>
iirc zmodem uses variable length frames as a sort of "window scaling"
<clever>
this define sets the size of that buffer
<kazinsal>
which is handy for reducing overhead on larger links
<mrvn>
clever: isn't that the size for the hardware buffer?
<clever>
nope
<clever>
// create circular buffer to hold received data
<clever>
the rx irq, then moves bytes from the hw fifo to the cbuf
<mrvn>
Still, enough to clear the hardware FIFO once.
<clever>
prior to me adding xmodem, that was only ever used for the shell to the user
<clever>
so the worst you have to deal with is a command being pasted
<clever>
and yeah, the irq handler will also read multiple bytes per call
<clever>
so it can clean the entire fifo in one shot
<mrvn>
clever: gives you 2.2ms to react to serial data
<clever>
the rp2040 port of LK, ran the uart at 1,000,000 baud
<clever>
and i had no issues interfacing with that from an rpi
<clever>
i need to try that on usb still
<mrvn>
If you have a user space app reading the serial without prioritized preemption that will run full.
<clever>
thats what the xmodem sender was
<mrvn>
e.g. a fixed 10ms time slice round-robin scheduler.
<clever>
regular old linux, with steam, chromium, and a little xmodem sender
<clever>
oh, another thought, usb overheads
<clever>
one sec
<mrvn>
The sender / linux has a much larger buffer.
<clever>
but xmodem can only queue up a 132 byte packet at once'
<clever>
and wont take advantage of those buffers waiting for an ACK
<mrvn>
yeah, but that will happen in one syscall.
<kazinsal>
boo, cisco removed xmodem/ymodem transfers from the rommon firmware on the ISR 4Ks... guess I gotta break out an old box and see if they ever supported zmodem or not
<clever>
aha, the ftdi adapter i use, has 64 byte max-packet in its endpoints
<mrvn>
And the ACK fits in the hardware FIFO, so you can't drop that. The danger is on the reciever dropping bytes.
<clever>
ignoring any ftdi overheads, that means a 132 byte xmodem packet, requires 3 usb packets
<clever>
mrvn: i think the biggest bottleneck, is the sender not reacting to an ack for 5ms, because linux is too busy with a dozen other things
<mrvn>
clever: that is bad but wouldn't corrupt data.
<clever>
yeah
<mrvn>
How much time do you loose if one byte in the 132 byte packet is dropped?
<clever>
11ms to send a 132 byte packet
<mrvn>
131 byte are recieved, no ACK is send, What happens then?
<mrvn>
How long do you wait before the reciever sends a NAK?
<mrvn>
papaya: hyperthreading totaly ruins any predictability
<mjg>
so, what should one deafult to? for example clang defaults to bumping regs with lea, not inc
<GeDaMo>
inc doesn't set all the flags which can cause false dependencies
<mrvn>
mjg: write what you mean and let the compiler care about that
<GeDaMo>
Although I don't know if that's still a problem nowadays
<GeDaMo>
Maybe add eax, 1 would be better
<papaya>
I suppose lea is safer since there's a non-zero chance increment could overflow and set it to 0 unless you know its already 0 to start with
<mrvn>
papaya: lea overflows too
<GeDaMo>
lea doesn't set any flags, maybe that's why clang uses it
<papaya>
so the only surefire way would be to explicitly set it with a mov?
<papaya>
or to xor it first
<papaya>
sounds like xor->lea is the goto way then
Ali_A has quit [Quit: Connection closed]
<GeDaMo>
Maybe xor eax,eax; mov al,1
Oli has joined #osdev
<papaya>
if the goal is for it to be non-zero, wouldnt the xor be redundent in that case
<GeDaMo>
Yeah, possibly
<papaya>
since it shouldnt matter what the upper bits are
<mjg>
it gets xored at the beginning of the routine for other reasons
<mjg>
i would not roll with xor only to have it not-zero
dude12312414 has joined #osdev
dude12312414 has quit [Client Quit]
<papaya>
what irc client does everyone use? tbh joining #osdev is my first time on IRC in... decades, am currently using Irrsi
<papaya>
Irssi*
<clever>
irssi as well
<mrvn>
why do you ask us instead of the irc clients?
<papaya>
wut?
<mrvn>
16:38 CTCP VERSION reply from papaya: irssi v1.2.3 - running on Linux x86_64
<clever>
/ctcp USER version
<papaya>
I see.
<papaya>
VERSION
<clever>
that wasnt a user!
<papaya>
but it worked?
<clever>
the request was sent
<clever>
check the status window in your client
<mjg>
i hear weechat is the thing to try if you don;t have a preference
<mjg>
personally i'm on irssi for 16-ish years now and can't be arsed to try anything else
<sham1>
Weechat is quite basic
<sham1>
Well, basic in terms of its UI. It's in some ways more powerful and feature-packed than, say, Irssi
<sham1>
Weechat looks like your typical GUI IRC client, except that most users use it in a terminal emulator of some sort
<papaya>
I see a lot of ZNC but that appears to be a server hosted web-based client?
<sham1>
ZNC is a bouncer, so it's basically an IRC client that is connected to by other IRC clients that it then multiplexes and keeps you online and records history and whatnot
<papaya>
and Quassel looks like a wechat gui
<kingoffrance>
i think you are supposed to write a client and join the sortie network
<kingoffrance>
and then success is when everyone in this channel has ascended to planet sortie
<kingoffrance>
so, 243 ppl to go :)
<mjg>
welp, this is the first time i even looked at the count
<mjg>
more people than i expected
<mjg>
still, probably only 1/10 even got a hello world kernel and 1/100 booted multiuser
* mjg
did not quite get to multi user with his custom kernel
the_lanetly_052 has quit [Ping timeout: 260 seconds]
mahmutov has quit [Ping timeout: 272 seconds]
mahmutov has joined #osdev
srjek has quit [Ping timeout: 240 seconds]
xenos1984 has quit [Remote host closed the connection]
Oli_ has quit [Ping timeout: 256 seconds]
xenos1984 has joined #osdev
diamondbond has joined #osdev
Matt|home has quit [Ping timeout: 260 seconds]
knusbaum has quit [Ping timeout: 256 seconds]
knusbaum has joined #osdev
<catern>
what's a good word or phrase to refer to "a single machine" (which may have many cores and be quite complicated) as opposed to "a distributed system"?
<catern>
as unambigiuous as possible
<GeDaMo>
Computer :P
<catern>
dang, that's actually pretty good, but maybe there's better
<graphitemaster>
I just call them a box
<catern>
maybe I should just say "a PC" but I want the word to cover more than just PC-architecture machines
<jjuran>
"Computer" covers PCs and SBCs
GreaseMonkey has joined #osdev
<jjuran>
"Box" implies the possibility of customizing the internals, which doesn't apply in all cases
<zid>
computer.
<zid>
or 'machine'
diamondbond has quit [Ping timeout: 240 seconds]
Matt|home has joined #osdev
<catern>
all these are kind of ambiguous though
<catern>
such that I'd need to say "a single computer" or "a single machine"
<catern>
e.g. "There is only one fault domain in a single computer" vs (ambiguous) "There is only one fault domain in a computer/machine"
GeDaMo has quit [Remote host closed the connection]
<catern>
it would be nice if there was a word which had a built-in meaning of "single"
srjek has joined #osdev
<moon-child>
jjuran: does it? To me, there is a (perhaps very faint) implicit 'black' in front of the 'box'
<jjuran>
moon-child: Well, to me "box" suggests that you might put (or replace) something in it like an adapter card
<catern>
(for me, i don't get any sense of customization from "box", but also no sense of being unable to customize. it has no difference from "machine" for me)
* kingoffrance
.oO( boxes lead to boxen, boxen lead to imagining beowulf clusters, clusters lead to car analogies/suffering )
<jjuran>
kingoffrance: I sense much fear in you.
<zid>
So you're upset that the word 'a' doesn't exist?
* zid
finds his tippex and his dictionary
<catern>
zid: is that addressed to me? the first sentence in my example is clearly more clear (lol) than the second one, so "single" is definitely useful
<catern>
i could say "node" but that implies abstracting over/ignoring the complex internal details
<moon-child>
how is 'there is only one fault domain in a computer' ambiguous?
<kazinsal>
computers are a pile of fault domains that occasionally produce numbers
<moon-child>
perhaps go for 'given computer' if you care to. But it seems clear enough to me
<bslsk05>
twitter: <PassMarkInc> New class of RAM test coming to #MemTest86 .  We found some RAM errors only occur during DMA transfers (Direct Memory Access) and not via interaction with the CPU. So this will be introduced as Test #14 in next MemTest86 release. https://pbs.twimg.com/media/FOfssaiVIAAJKFI.jpg [MemTest86 DMA test, new test method, Test #14]
<kazinsal>
ooh, neat
<zid>
makes sense
<mjg>
re fault domains
<gog>
a group of computers is called a "catastrophe"
<moon-child>
haha
<zid>
every emulator test suite tests if dma works, why not memtest
lainon has quit [Killed (zirconium.libera.chat (Nickname regained by services))]
lainon_ is now known as lainon
<kazinsal>
I'm sitting here staring at a work problem I really don't want to solve by hand and the first thing I can think of is "I should write a CSV library for Kuroko and use that"